From 9d9472c63028d216071c32895a76b001f30084e4 Mon Sep 17 00:00:00 2001 From: rustfuture <121799572+rustfuture@users.noreply.github.com> Date: Mon, 28 Sep 2026 22:29:01 +0300 Subject: [PATCH] State that quoted lines are verbatim but not guaranteed relevant Say plainly what the answer contract guarantees. - Quotes are verbatim source lines; the README's own measured results (12 false accepts in 42 questions) show relevance is not guaranteed, so the opening and the second bullet now say so. - Mark `evaluation/evaluate_modes.py` as legacy: it overwrites `evaluation/results_modes.json` in place, unlike the documented `evaluation/v3/evaluate.py`. Docstring only, no behaviour change. --- README.md | 4 ++-- evaluation/evaluate_modes.py | 6 +++++- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 80ff7a7..f584f4d 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Repository Intelligence -Repository Intelligence searches local code and returns exact source lines to answer questions instead of generating prose. +Repository Intelligence searches local code and returns verbatim source lines for a question instead of generating prose; relevance is not guaranteed. [![CI](https://github.com/rustfuture/repository-intelligence/actions/workflows/ci.yml/badge.svg)](https://github.com/rustfuture/repository-intelligence/actions/workflows/ci.yml) [![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) @@ -8,7 +8,7 @@ Repository Intelligence searches local code and returns exact source lines to an **Status:** Experimental research prototype (v0.1.0) for local code inspection. - Searches code using text matching, offline hash embeddings, or local neural models. -- Answers questions by extracting and quoting exact source lines instead of generating text. +- Responds to questions by quoting exact source lines instead of generating text; quotes match the source, but relevance is not guaranteed. - Stores portable repository index snapshots without external database dependencies. - Synchronizes file additions, changes, and deletions incrementally through Git diffs. - Exposes local search and index reload endpoints over a lightweight HTTP server. diff --git a/evaluation/evaluate_modes.py b/evaluation/evaluate_modes.py index 60a9f71..3001137 100644 --- a/evaluation/evaluate_modes.py +++ b/evaluation/evaluate_modes.py @@ -1,5 +1,9 @@ #!/usr/bin/env python3 -"""Compare lexical, semantic, and RRF hybrid retrieval modes across splits.""" +"""Compare lexical, semantic, and RRF hybrid retrieval modes across splits. + +Legacy script: it overwrites evaluation/results_modes.json in place. The documented +reproduction path is evaluation/v3/evaluate.py, which writes to a new directory. +""" import json import os import re