itisar-345 · Oct 20, 2026 · Oct 17, 2026 · Oct 20, 2026
diff --git a/.gitignore b/.gitignore
@@ -24,6 +24,7 @@ venv/
 backend/data/*.db
 backend/data/*.json
 deploy/backups/
+backend/evals/results/
 
 # Editor
 .idea/

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -18,7 +18,26 @@ Notable changes to Knowledge Hubs. The format follows [Keep a Changelog](https:/
 - `API_DOCS`: `false` hides `/docs`, `/redoc` and `/openapi.json`; the production stack sets it.
 - [`deploy/backup.sh`](deploy/backup.sh): a consistent ArcadeDB backup copied to `deploy/backups/`, with old copies pruned. The deployment guide describes restoring.
 
+- `GET /knowledge/items` (filtered, paged items) and `GET /knowledge/items/{id}/context` (what an item's page shows).
+- GraphRAG answers whose model names its sources without writing their ids get those sources inferred from the wording, marked "matched".
+
+- [`backend/evals/`](backend/evals/): an end-to-end evaluation of extraction and GraphRAG against a real model, with a fixed corpus, answer key and questions.
+
 ### Changed
+- **GraphRAG answers are grounded, cited by number, and about 6× faster** (measured with `backend/evals/` on llama3.1:8b, CPU: 15 → 19 of 19 answers correct, 4 → 0 false "not covered" replies, median answer time 78 s → 13 s):
+  - The model now sees each item's details (who, why, due, steps), its source's title, and the passage it came from. It used to see titles only, and answered "the context doesn't say who" for owners on record.
+  - Context entries are numbered and cited as `[1]`, `[2]`, mapped back to items; small models copied long ids poorly.
+  - One fixed reply when nothing in the knowledge base is about the question ("The knowledge base doesn't cover this."), with no sources. A refusal no longer gets sources inferred from the topics it names.
+  - The LLM question rewrite (sub-queries + HyDE), LLM rerank and LLM routing are off by default (`GRAPHRAG_QUERY_REWRITE`, `GRAPHRAG_RERANK`, `GRAPHRAG_LLM_ROUTE`): each cost about as long as the answer itself, and retrieval found the right source every time without them.
+  - Shorter answers (at most about 120 words, no preamble or closing summary).
+- Extraction replies are constrained to a JSON schema (Ollama structured outputs), and a decision is kept in one item with its reason.
+- Ollama keeps the model loaded for 30 minutes between calls (`OLLAMA_KEEP_ALIVE`) instead of unloading it after 5.
+- **Lists are bounded.** `GET /knowledge` returns a page (default 100) with `totals`, filters items on the server, and leaves out document text unless `include_content=true`; the review queue, decision register, graph and search have limits too. On 3,000 items the Hub's response was 4 MB and, under load, ran ArcadeDB out of memory. The pages show **Load more**.
+- **Ingestion no longer waits for the summary.** The LLM summary of a new source is built after the response (about 30 s on a CPU).
+- **App instances run several server processes** (`APP_WORKERS`, default 2), and bcrypt runs on a worker thread.
+- The Ollama status behind the public model catalog and sign-in is cached for 5 seconds (`OLLAMA_STATUS_CACHE_SECONDS`).
+- **Passwords:** at least 8 characters (was 6), not a common password, not the username. Existing passwords keep working.
+- Accessibility: one `<h1>` per page, labelled filters, a group role for the graph, underlined links in text, and text colours with at least 4.5:1 contrast in both themes.
 - **License: GPL-3.0 → AGPL-3.0.**
 - Angular 17 → 21. Building the frontend needs Node 20.19+ or 22.12+; the Docker images use Node 22.
 - Backend dependencies upgraded to current releases, including FastAPI 0.142 (Starlette 1.7), Pydantic 2.13, sentence-transformers 6, NumPy 2 and Celery 5.6.
@@ -27,6 +46,15 @@ Notable changes to Knowledge Hubs. The format follows [Keep a Changelog](https:/
 - Docker base images: Caddy 2.10, Node 22.
 
 ### Fixed
+- A write that collided with another on the same ArcadeDB storage page failed outright ("Concurrent modification… Please retry"): a 3,000-item import was lost when a summary was saved meanwhile. Ingest, embedding and summary writes now run again on such a conflict (only the writes; extraction isn't repeated).
+- Finding the relationships for a page scanned every edge in the space: about a second for the Hub on 6,000 items. They're now found by walking out from the page's nodes (Hub 2.4 s → 0.2 s, graph 1.3 s → 0.16 s).
+- Signing in with an unknown username answered faster than with a known one, which told them apart.
+- Source results in search couldn't be opened.
+- First run: the onboarding page had no setup-token field, so with `SETUP_TOKEN` set the first account couldn't be created there. It now asks for the token, as the sign-in page does.
+- Switching spaces while on the Hub kept showing the previous space's knowledge until a reload (the switch navigated through `/`, which redirects to the Hub itself).
+- A group, playbook or other record created a moment ago could be missing from the next list in the clustered stack: reads went to a follower that hadn't applied the write yet. The app now sends its work to the cluster's leader.
+- With Redis down, sign-in took up to 15 seconds and sometimes answered 503 "Ollama is not running". The rate limiter and the health probe now give Redis 2 seconds and then leave it alone for 30.
+- Members opening `/admin/users` saw the account form with an error; they're now sent to the Hub.
 - URL fetching could be redirected to a private address by DNS rebinding. The address is now checked again when connecting, and the connection goes to exactly that address.
 - The production frontend build no longer loads its stylesheet through an inline `onload` handler (`inlineCritical` off), which the Content Security Policy would block.
 - Reloading a page whose path is also an API path (such as `/knowledge`) made the app show "Failed to load knowledge": the browser reused the cached page for the app's data request. The web server now sends `Vary: Accept`.

diff --git a/README.md b/README.md
@@ -13,7 +13,9 @@ A **local-first knowledge base** for teams. It turns meeting notes, retros, deci
 - **Personal and group spaces.** Everyone has a private space and can create or join groups that share one. A switcher in the nav picks where you're working.
 - **Decisions have a history.** A decision can supersede, reverse or amend an older one, and risks can be mitigated or actually happen. The app tracks each item's status and shows how it evolved.
 - **Connects only where you point it.** Knowledge can come from folders, git repositories (ADRs), GitHub, Jira and Linear, but only from addresses a user enters, and only when they click **Sync now**.
+- **Answers you can check.** GraphRAG answers from the items it retrieves, the details recorded on them and the passages of the sources they came from, cites them by number, and says plainly when the knowledge base doesn't cover a question. A [built-in evaluation](backend/evals/) measures answers, citations and extraction against a real model.
 - **One command to try it, a resilient stack for production.** `docker compose up` runs everything on one machine. [`deploy/`](deploy/) runs it behind a TLS load balancer, with two app instances, a Celery worker and a three-node ArcadeDB cluster.
+- **Ready to face the internet.** In production the first (admin) account needs a setup token, sign-ins are throttled, sessions can be revoked, and the web server sends a strict Content Security Policy. See [Security notes](docs/security.md).
 
 | Page | What you do there |
 |---|---|
@@ -23,7 +25,7 @@ A **local-first knowledge base** for teams. It turns meeting notes, retros, deci
 | [**Decisions**](docs/images/decisions.png) | The decision log and risk register, with each item's status and what replaced it |
 | [**Playbooks**](docs/images/playbooks.png) | Repeatable procedures, written as steps or built from how-tos and checklists in the space |
 | [**Sources**](docs/images/sources.png) | Connectors that pull in transcripts, notes, ADRs, pull requests and issues |
-| [**GraphRAG**](docs/images/graphrag.png) | Ask questions; answers cite the items they used and flag superseded decisions |
+| [**GraphRAG**](docs/images/graphrag.png) | Ask questions; answers cite their sources, flag superseded decisions, and say so when nothing covers the question |
 | [**Groups**](docs/images/groups.png) | Create groups, invite people and share a space with them |
 | [**Models**](docs/images/models.png) | Download, switch or remove the local LLMs; re-embed the knowledge base |
 | [**Users**](docs/images/users.png) *(server admins)* | Create accounts, change roles, see inactive accounts due for deletion |
@@ -42,6 +44,8 @@ docker compose up --build
 
 Open **http://localhost:8080**, create an account (the first one is the server admin) and pick a model to download. No configuration file is needed: secrets are generated on first start.
 
+The local stack listens on this machine only. Before letting other machines in (`KH_BIND=0.0.0.0`), set `SETUP_TOKEN` so that nobody else can create the admin account; for anything on the internet, use the [production stack](docs/deployment.md).
+
 To use a GPU, an Ollama you already run, or to work on the code with live reload, see [Getting started](docs/getting-started.md).
 
 ## Documentation
@@ -62,7 +66,9 @@ To use a GPU, an Ollama you already run, or to work on the code with live reload
 | [API](docs/api.md) | Every endpoint |
 | [Security notes](docs/security.md) | How accounts, data, credentials and inputs are protected |
 | [Testing](docs/testing.md) | Running the backend and frontend test suites |
+| [LLM evaluation](backend/evals/README.md) | Measuring answer quality, citations, extraction and speed against a real model |
 | [Known limitations](docs/known-limitations.md) | What it doesn't do (yet) |
+| [Changelog](CHANGELOG.md) | What changed, including steps to take when upgrading |
 
 ## Contributing
 

diff --git a/backend/.env.example b/backend/.env.example
@@ -66,9 +66,16 @@ OLLAMA_NUM_CTX=8192
 # Max size of an uploaded file or fetched URL, in MB (also caps request bodies)
 MAX_UPLOAD_MB=20
 
-# GraphRAG: LLM rerank of retrieved candidates (on|off). It only runs when there
-# are more than 2x top_k candidates; "off" always keeps the fusion (RRF) order.
-GRAPHRAG_RERANK=on
+# GraphRAG: extra LLM calls per question, each about as slow as the answer itself on a
+# CPU. Off by default: on evals/ they didn't make answers better (see backend/evals/).
+#   GRAPHRAG_RERANK: rerank retrieved candidates (only with more than 2x top_k of them)
+#   GRAPHRAG_QUERY_REWRITE: rewrite the question into sub-queries + a hypothetical answer
+#   GRAPHRAG_LLM_ROUTE: classify questions the embedding classifier is unsure about
+GRAPHRAG_RERANK=off
+GRAPHRAG_QUERY_REWRITE=off
+GRAPHRAG_LLM_ROUTE=off
+# How long Ollama keeps the model loaded between calls (its default unloads it after 5 min)
+OLLAMA_KEEP_ALIVE=30m
 
 # ── Offline enforcement ───────────────────────────────────────────────────────
 TRANSFORMERS_OFFLINE=1

diff --git a/backend/app/arcadedb.py b/backend/app/arcadedb.py
@@ -9,7 +9,10 @@
 
 import asyncio
 import logging
-from typing import Any, Dict, List, Optional, Tuple
+import random
+import time
+from typing import Any, Awaitable, Callable, Dict, List, Optional, Tuple, TypeVar
+from urllib.parse import urlsplit
 
 import httpx
 
@@ -49,12 +52,20 @@ class ArcadeClient:
     election). Timeouts and dropped connections are NOT retried elsewhere:
     the statement may have been applied, and running it twice could
     duplicate a write. A transaction stays on the node where it began.
+
+    With several nodes the client prefers the current leader (looked up at
+    most every LEADER_REFRESH seconds). A follower can answer a read before it
+    has applied a write just made through it, so a group created a moment ago
+    could be missing from the next list; the leader has applied everything it
+    acknowledged. ArcadeDB's linearizable reads would fix that too, but cost
+    about three seconds per read that follows a write.
     """
 
     # A dead node must cost seconds, not the whole request timeout: connecting is
     # quick or it isn't happening. Queries themselves may legitimately run longer.
     CONNECT_TIMEOUT = 3.0
     PROBE_TIMEOUT = 2.0
+    LEADER_REFRESH = 10.0
 
     def __init__(self, url: str, database: str, user: str, password: str, timeout: float = 60) -> None:
         self.database = database
@@ -65,6 +76,7 @@ def __init__(self, url: str, database: str, user: str, password: str, timeout: f
                                       timeout=httpx.Timeout(timeout, connect=self.CONNECT_TIMEOUT))
         self._preferred = 0
         self._session_nodes: Dict[str, str] = {}   # transaction id -> node it lives on
+        self._leader_checked = float("-inf")
 
     @staticmethod
     def _error(response: httpx.Response) -> ArcadeDBError:
@@ -87,9 +99,33 @@ def _order(self) -> List[int]:
         n = len(self.nodes)
         return [(self._preferred + i) % n for i in range(n)]
 
+    async def _leader_address(self) -> Optional[str]:
+        """The leader's HTTP "host:port", as the cluster reports it; None if unknown."""
+        for node in (self.nodes[i] for i in self._order()):
+            try:
+                response = await self.http.get(f"{node}/server", params={"mode": "cluster"},
+                                               timeout=self.PROBE_TIMEOUT)
+            except httpx.HTTPError:
+                continue
+            if response.status_code < 400:
+                return (response.json().get("ha") or {}).get("leaderAddress")
+        return None
+
+    async def _follow_leader(self) -> None:
+        if len(self.nodes) < 2 or time.monotonic() - self._leader_checked < self.LEADER_REFRESH:
+            return
+        self._leader_checked = time.monotonic()
+        address = (await self._leader_address() or "").lower()
+        for index, node in enumerate(self.nodes):
+            if address and urlsplit(node).netloc.lower() == address:
+                self._preferred = index
+                return
+
     async def _post(self, path: str, body: Optional[Record] = None,
                     session_id: Optional[str] = None) -> Tuple[httpx.Response, str]:
         headers = {"arcadedb-session-id": session_id} if session_id else None
+        if not session_id:
+            await self._follow_leader()
         if session_id:
             node = self._session_nodes.get(session_id, self.nodes[self._preferred])
             try:
@@ -229,9 +265,19 @@ async def aclose(self) -> None:
         await self.http.aclose()
 
 
+T = TypeVar("T")
+
+
+def is_write_conflict(exc: Exception) -> bool:
+    """ArcadeDB's "someone else changed this page first; retry the operation"."""
+    return isinstance(exc, ArcadeDBError) and "ConcurrentModification" in f"{exc.exception} {exc.detail}"
+
+
 class ArcadeSession:
     """One per request. Use query() for reads and execute() for writes."""
 
+    WRITE_ATTEMPTS = 4
+
     def __init__(self, client: ArcadeClient) -> None:
         self.client = client
         self._session_id: Optional[str] = None
@@ -263,6 +309,29 @@ async def commit(self) -> None:
             session_id, self._session_id = self._session_id, None
             await self.client.commit(session_id)
 
+    async def transaction(self, work: Callable[[], Awaitable[T]]) -> T:
+        """
+        Run `work` (writes, no commit of its own) and commit it, running it again
+        after a write conflict. ArcadeDB checks concurrency per storage page, so two
+        transactions writing different records on the same page conflict and the one
+        that commits second is told to retry: a long import failed that way when a
+        summary was saved meanwhile. Retried only when no transaction was already
+        open here, since rolling back would also undo the writes made before it.
+        """
+        retryable = self._session_id is None
+        for attempt in range(1, self.WRITE_ATTEMPTS + 1):
+            try:
+                result = await work()
+                await self.commit()
+                return result
+            except ArcadeDBError as exc:
+                await self.rollback()
+                if not (retryable and is_write_conflict(exc)) or attempt == self.WRITE_ATTEMPTS:
+                    raise
+                logger.warning("Write conflict, retrying (%d of %d)", attempt + 1, self.WRITE_ATTEMPTS)
+                await asyncio.sleep(0.05 * 2 ** attempt + random.uniform(0, 0.1))
+        raise AssertionError("unreachable")
+
     async def rollback(self) -> None:
         if self._session_id is not None:
             session_id, self._session_id = self._session_id, None

diff --git a/backend/app/auth.py b/backend/app/auth.py
@@ -53,6 +53,44 @@ def hash_password(password: str) -> str:
     return bcrypt.hashpw(_bcrypt_input(password), bcrypt.gensalt()).decode("ascii")
 
 
+# New passwords only; existing ones keep working. Short of a full breached-password
+# check, refuse the handful that guessing tools try first.
+PASSWORD_MIN_LENGTH = 8
+PASSWORD_RULES = f"at least {PASSWORD_MIN_LENGTH} characters, not a common password and not your username"
+_COMMON_PASSWORDS = frozenset("""
+password password1 password12 password123 passw0rd p@ssw0rd p@ssword 12345678 123456789 1234567890
+12341234 11111111 00000000 87654321 qwertyui qwerty123 qwertyuiop 1q2w3e4r 1qaz2wsx asdfghjk
+iloveyou letmein1 welcome1 welcome123 admin123 administrator changeme changeme1 football baseball
+sunshine princess superman starwars trustno1 whatever monkey123 dragon123 abcd1234 abc12345
+""".split())
+
+
+def check_new_password(password: str, username: str = "") -> str:
+    """Validate a password for a new account. Raises ValueError (a 422 from Pydantic validators)."""
+    if len(password) < PASSWORD_MIN_LENGTH:
+        raise ValueError(f"Password must be {PASSWORD_RULES}")
+    lowered = password.lower()
+    if lowered in _COMMON_PASSWORDS or len(set(password)) == 1:
+        raise ValueError("That password is too common. Choose a less predictable one.")
+    if username and lowered == username.strip().lower():
+        raise ValueError("The password can't be the same as the username.")
+    return password
+
+
+# bcrypt of a random value, checked when the username doesn't exist, so that answering
+# "Invalid credentials" takes as long either way: the time no longer says whether a
+# name is registered (it was 15 ms for an unknown name against 250 ms for a real one).
+_DUMMY_HASH: Optional[str] = None
+
+
+def burn_password_check(plain: str) -> None:
+    global _DUMMY_HASH
+    if _DUMMY_HASH is None:
+        import secrets
+        _DUMMY_HASH = hash_password(secrets.token_urlsafe(16))
+    verify_password(plain, _DUMMY_HASH)
+
+
 def verify_password(plain: str, hashed: str) -> bool:
     try:
         return bcrypt.checkpw(_bcrypt_input(plain), hashed.encode("ascii"))