diff --git a/AGENTS.md b/AGENTS.md index 76a5ff9..0b48dbb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,11 +72,16 @@ Every page uses `templates/page.md`. Non-negotiable rules: 1. **One case per page.** A page answers one situation. If you are writing "and also…", split the page and cross-link under `related`. 2. **≤ 120 lines of body.** Precision beats coverage. Link, don't inline. -3. **Positive guidance only.** Every directive is "In situation X, do Y". - Anti-patterns may only appear in the `Instead of` table, where each row MUST pair - the anti-pattern with its replacement action. A "don't" without an "instead" is a - lint failure — a prohibition with no replacement invites the reader to improvise, - which is how hallucinations happen. +3. **Positive guidance; a prohibition must pair.** Every directive is "In situation X, + do Y". A prohibition (`don't` / `do not` / `never` / `avoid` / `must not`) may + appear anywhere in a page, as long as the same directive item also carries its + replacement action or the mechanism that makes the prohibition true. The + `Instead of` table remains the place for anti-pattern/replacement pairs; each row + there MUST still pair the anti-pattern with its replacement action. The unit of + pairing is the directive item — a table cell or a bullet — not the page as a + whole. A bare prohibition, alone in its item, is a lint failure: a prohibition + with no replacement invites the reader to improvise, which is how hallucinations + happen. Enforced by `node scripts/wiki-lint-prohibitions.js`. 4. **No vague qualifiers.** Words like "usually", "consider", "might want to", "generally", "as appropriate" are banned in directive sentences. State the condition that decides it: "When X, do A. When Y, do B." If you cannot state the @@ -126,8 +131,8 @@ Run these via the skill files, which contain the full step-by-step workflows: merge into existing pages before creating new ones, cite sources, update indexes and `log.md`. - **Query** (`skills/wiki-query/SKILL.md`) — answer a question from the wiki with citations; if the answer required synthesis across pages and is re-askable, file it as a new page. -- **Lint** (`skills/wiki-lint/SKILL.md`) — health check: unsourced claims, "don't"s without - "instead"s, banned vague qualifiers, orphan pages, broken links, stale `last_verified`. +- **Lint** (`skills/wiki-lint/SKILL.md`) — health check: unsourced claims, unpaired + prohibitions, banned vague qualifiers, orphan pages, broken links, stale `last_verified`. Two further skills use the wiki to run development work (rather than maintain the wiki): diff --git a/INDEX.md b/INDEX.md index cb6cd09..9a24186 100644 --- a/INDEX.md +++ b/INDEX.md @@ -10,7 +10,7 @@ follow the cross-pointers in their index or take the next matching seeded domain | Domain | Status | Route here when | |--------|--------|-----------------| | [databases](wiki/databases/index.md) | **seeded** | Designing schemas/tables/keys, choosing or evaluating indexes, writing or optimizing queries, choosing transaction/isolation behavior, surveying live data to derive a rule, verifying additive migrations | -| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, call-site enumeration before a contract change, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) | +| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, call-site enumeration before a contract change, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, consuming LLM APIs (completion validation, context budgeting), consuming external-API responses, externally-owned defaults, object-storage references) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) | | [frontend](wiki/frontend/index.md) | **seeded** | Web UI code: state placement, rendering performance, in-UI data fetching (races, infinite scroll), auth token handling, forms, XSS-safe output, accessibility | | [infrastructure](wiki/infrastructure/index.md) | **seeded** | CI/CD pipelines, secrets in build/deploy, container image builds, rollout/rollback strategy, observability (logs/metrics/alerting), per-environment/path-valued config, multi-agent orchestration (worker liveness signals, shared run state, tmux pane delivery, completion gates, worktree-isolated workers) | | [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests (release-process quality → qa) | diff --git a/README.ko.md b/README.ko.md index b7dfd20..5b4a6a3 100644 --- a/README.ko.md +++ b/README.ko.md @@ -197,8 +197,12 @@ deny-net을 분류기는 볼 수 없기 때문입니다. 코디네이터 세션 발화합니다. 아무것도 하지 않아도 PR이 나타납니다. 가드: 킬 스위치 `DEV_LOOP_AUTOFLUSH=0`, `DEV_LOOP_AUTOFLUSH_INTERVAL`(기본 3600초)당 1회, 대기 항목 `DEV_LOOP_AUTOFLUSH_MIN`(기본 3)개 이상일 때만, - single-flight 잠금, 재귀 안전. `claude` + `gh`가 PATH에 있고 gh 인증이 - 필요합니다. 하나라도 없으면 조용히 no-op — 수동으로 fallback. + 아래 수동 플러시와 공유하는 owner-token single-flight 잠금 + (`DEV_LOOP_FLUSH_LOCK_TTL`, 기본 900초 — 이 시간이 지나면 죽은 홀더의 + 잠금을 재점유 가능), 그리고 두 진입점이 같은 후보를 동시에 인제스트하지 + 않도록 하는 런당 큐 클레임(`DEV_LOOP_CLAIM_TTL`, 기본 3600초), 재귀 + 안전. `claude` + `gh`가 PATH에 있고 gh 인증이 필요합니다. 하나라도 + 없으면 조용히 no-op — 수동으로 fallback. - **수동** — 언제든 `/dev-loop:knowledge-flush`로 큐를 지금 비웁니다. ### 이 순서는 훅으로 강제됩니다 diff --git a/README.md b/README.md index 1730760..ed0f321 100644 --- a/README.md +++ b/README.md @@ -200,9 +200,13 @@ The wiki is meant to grow from what you actually learn. Three moving parts: rate-limit window has elapsed, so PRs appear without you doing anything. Guarded: kill switch `DEV_LOOP_AUTOFLUSH=0`, once per `DEV_LOOP_AUTOFLUSH_INTERVAL` (default 3600s), only at - `DEV_LOOP_AUTOFLUSH_MIN` (default 3) pending items, single-flight lock, and - recursion-safe. Needs `claude` + `gh` on PATH and gh authenticated; if either - is missing it silently no-ops and you fall back to manual. + `DEV_LOOP_AUTOFLUSH_MIN` (default 3) pending items, an owner-token + single-flight lock shared with the manual flush below + (`DEV_LOOP_FLUSH_LOCK_TTL`, default 900s, before a crashed holder's lock + is reclaimable) plus a per-run queue claim (`DEV_LOOP_CLAIM_TTL`, default + 3600s) so the two entry points never ingest the same candidate, and + recursion-safe. Needs `claude` + `gh` on PATH and gh authenticated; if + either is missing it silently no-ops and you fall back to manual. - **Manual** — invoke `/dev-loop:knowledge-flush` any time to drain the queue now. ### This ordering is enforced by a hook diff --git a/hooks/auto-flush.sh b/hooks/auto-flush.sh index d1d24c7..39c4506 100755 --- a/hooks/auto-flush.sh +++ b/hooks/auto-flush.sh @@ -15,7 +15,14 @@ # and inside the flush checkout (~/.dev-loop/repo) # - rate limit: at most once per DEV_LOOP_AUTOFLUSH_INTERVAL sec (default 3600) # - threshold: only when >= DEV_LOOP_AUTOFLUSH_MIN pending items (default 3) -# - single-flight: a TTL lock dir +# - single-flight: scripts/flush-lock.sh, an owner-token mkdir lock shared +# with skills/knowledge-flush/SKILL.md step 0 (issue #77). +# DEV_LOOP_FLUSH_LOCK_TTL sec before a crashed holder's lock +# is reclaimable (default 900). DEV_LOOP_CLAIM_TTL sec before +# a claimed-but-unfinished queue row (hooks/queue-claim.js) +# is reclaimable by another run (default 3600). +# DEV_LOOP_FLUSH_LOCK (lock path) and DEV_LOOP_FLUSH_RUN_ID +# (this run's id) are test/override seams, not user knobs. # - fail-safe: if `claude`/`gh` are missing it silently no-ops; the manual # /dev-loop:knowledge-flush skill still works. set +e @@ -42,6 +49,9 @@ QUEUE="$DIR/queue" [ -d "$QUEUE" ] || exit 0 # --- threshold: count PENDING rows (exclude the retired .processed.jsonl) -- +# The literal '"status":"pending"' match already excludes "claimed" rows +# (hooks/queue-claim.js), so a live sibling run's in-flight claims cannot +# re-trip this threshold. PENDING=0 for f in "$QUEUE"/*.jsonl; do case "$f" in *"/.processed.jsonl") continue ;; esac @@ -59,13 +69,14 @@ if [ -f "$STAMP" ] && [ -n "$(find "$STAMP" -mmin "-$INTERVAL_MIN" 2>/dev/null)" exit 0 fi -# --- single-flight lock (TTL ~15 min via mkdir atomicity) ----------------- -LOCK="$DIR/.autoflush.lock" -if ! mkdir "$LOCK" 2>/dev/null; then - # stale lock older than 15 min → reclaim - [ -n "$(find "$LOCK" -mmin -15 2>/dev/null)" ] && exit 0 - rmdir "$LOCK" 2>/dev/null; mkdir "$LOCK" 2>/dev/null || exit 0 -fi +# --- single-flight lock (shared with skills/knowledge-flush/SKILL.md step 0 +# via scripts/flush-lock.sh — see issue #77) --------------------------- +# One run id for both the acquire below and the release in the backgrounded +# subshell, so release's owner-token check matches this run (DEV_LOOP_FLUSH_RUN_ID +# is exported here and inherited by the "(...) &" subshell further down). +RUNID="${DEV_LOOP_FLUSH_RUN_ID:-$(date +%Y%m%d-%H%M%S)-$$}" +export DEV_LOOP_FLUSH_RUN_ID="$RUNID" +sh "$(dirname "$0")/../scripts/flush-lock.sh" acquire >/dev/null 2>&1 || exit 0 touch "$STAMP" 2>/dev/null # --- spawn the detached headless flush ------------------------------------ @@ -76,7 +87,7 @@ PROMPT='Run the dev-loop:knowledge-flush skill now. Drain ~/.dev-loop/queue: for DEV_LOOP_FLUSHING=1 nohup "$CLAUDE_BIN" -p "$PROMPT" \ --permission-mode bypassPermissions \ > "$DIR/autoflush.log" 2>&1 - rmdir "$LOCK" 2>/dev/null + sh "$(dirname "$0")/../scripts/flush-lock.sh" release >/dev/null 2>&1 ) & disown 2>/dev/null diff --git a/hooks/queue-claim.js b/hooks/queue-claim.js new file mode 100755 index 0000000..3dac53b --- /dev/null +++ b/hooks/queue-claim.js @@ -0,0 +1,195 @@ +#!/usr/bin/env node +/* + * queue-claim.js — claim/release insight-queue rows by run id (issue #77). + * + * scripts/flush-lock.sh makes the two knowledge-flush entry points mutually + * exclusive, but a lock alone is not proof against a stale actor (a paused or + * TTL-expired holder can resume mid-work). This tool makes the QUEUE + * self-defend: a run claims the rows it is about to ingest, so a second run — + * racing past the lock, or reclaiming after a dead holder — sees those rows + * as unavailable instead of re-ingesting them. + * + * Mutates each *.jsonl row IN PLACE (status "pending" -> "claimed", plus + * claimedBy/claimedAt) rather than moving it to another file — this keeps the + * row in hooks/harvest.js's dedupe seed (which reads the session queue file + * AND .processed.jsonl), so a still-pending duplicate is never re-harvested. + * .processed.jsonl (the retired store) is never touched by any subcommand. + * + * Row identity is hooks/harvest.js's own dedupe key, the "hash" field — + * reused here rather than inventing a second id for the same row. + * + * usage: + * node queue-claim.js list print claimable row ids, one per line + * node queue-claim.js claim [--max N] claim up to N claimable rows, print their ids + * node queue-claim.js release ... set specific claimed rows back to pending + * + * env overrides: + * DEV_LOOP_QUEUE_DIR queue directory (default: ~/.dev-loop/queue) + * DEV_LOOP_FLUSH_RUN_ID this run's id (default: -) + * DEV_LOOP_CLAIM_TTL seconds before a claimed row is reclaimable (default: 3600) + */ +'use strict'; + +const fs = require('fs'); +const path = require('path'); +const os = require('os'); + +function queueDir() { + return process.env.DEV_LOOP_QUEUE_DIR || path.join(os.homedir(), '.dev-loop', 'queue'); +} + +function defaultRunId() { + const d = new Date(); + const pad = (n) => String(n).padStart(2, '0'); + const ts = + `${d.getFullYear()}${pad(d.getMonth() + 1)}${pad(d.getDate())}-` + + `${pad(d.getHours())}${pad(d.getMinutes())}${pad(d.getSeconds())}`; + return `${ts}-${process.pid}`; +} + +function runId() { + return process.env.DEV_LOOP_FLUSH_RUN_ID || defaultRunId(); +} + +function claimTtlSec() { + const v = parseInt(process.env.DEV_LOOP_CLAIM_TTL, 10); + return Number.isFinite(v) ? v : 3600; +} + +// Every *.jsonl in the queue dir except the retired store, sorted for a +// deterministic claim order. +function queueFiles() { + const dir = queueDir(); + if (!fs.existsSync(dir)) return []; + return fs + .readdirSync(dir) + .filter((f) => f.endsWith('.jsonl') && f !== '.processed.jsonl') + .sort() + .map((f) => path.join(dir, f)); +} + +// Read one queue file into {raw, parsed} rows. `raw` is the exact original +// line — kept byte-for-byte for any row this run does not rewrite, so an +// unparseable line always survives untouched. `parsed` is null for a +// malformed line and for blank lines (which are dropped, matching +// harvest.js's own "non-blank line" convention). +function readRows(file) { + const text = fs.readFileSync(file, 'utf8'); + const rows = []; + for (const raw of text.split('\n')) { + if (!raw.trim()) continue; + let parsed = null; + try { + parsed = JSON.parse(raw); + } catch { + parsed = null; + } + rows.push({ raw, parsed }); + } + return rows; +} + +function writeRowsAtomic(file, rows) { + const body = rows.map((r) => r.raw).join('\n') + '\n'; + const tmp = `${file}.tmp`; + fs.writeFileSync(tmp, body); + fs.renameSync(tmp, file); +} + +function isClaimable(row, ttlSec, nowMs) { + if (!row || !row.hash) return false; + if (row.status === 'pending') return true; + if (row.status === 'claimed') { + const claimedMs = Date.parse(row.claimedAt); + return nowMs - claimedMs > ttlSec * 1000; + } + return false; +} + +function cmdList() { + const nowMs = Date.now(); + const ttlSec = claimTtlSec(); + const ids = []; + for (const file of queueFiles()) { + for (const { parsed } of readRows(file)) { + if (isClaimable(parsed, ttlSec, nowMs)) ids.push(parsed.hash); + } + } + if (ids.length) process.stdout.write(ids.join('\n') + '\n'); +} + +function cmdClaim(argv) { + let max = Infinity; + for (let i = 0; i < argv.length; i++) { + if (argv[i] === '--max') { + const n = parseInt(argv[i + 1], 10); + if (Number.isFinite(n)) max = n; + i++; + } + } + + const nowMs = Date.now(); + const ttlSec = claimTtlSec(); + const claimedAt = new Date(nowMs).toISOString(); + const who = runId(); + const claimedIds = []; + let remaining = max; + + for (const file of queueFiles()) { + const rows = readRows(file); + let changed = false; + for (const row of rows) { + if (remaining <= 0) break; + if (!isClaimable(row.parsed, ttlSec, nowMs)) continue; + row.parsed.status = 'claimed'; + row.parsed.claimedBy = who; + row.parsed.claimedAt = claimedAt; + row.raw = JSON.stringify(row.parsed); + changed = true; + remaining -= 1; + claimedIds.push(row.parsed.hash); + } + if (changed) writeRowsAtomic(file, rows); + } + + if (claimedIds.length) process.stdout.write(claimedIds.join('\n') + '\n'); +} + +function cmdRelease(argv) { + const ids = new Set(argv); + if (!ids.size) return; + + for (const file of queueFiles()) { + const rows = readRows(file); + let changed = false; + for (const row of rows) { + if (!row.parsed || !row.parsed.hash || !ids.has(row.parsed.hash)) continue; + row.parsed.status = 'pending'; + delete row.parsed.claimedBy; + delete row.parsed.claimedAt; + row.raw = JSON.stringify(row.parsed); + changed = true; + } + if (changed) writeRowsAtomic(file, rows); + } +} + +function main() { + const [cmd, ...rest] = process.argv.slice(2); + switch (cmd) { + case 'list': + cmdList(); + break; + case 'claim': + cmdClaim(rest); + break; + case 'release': + cmdRelease(rest); + break; + default: + process.stderr.write('usage: queue-claim.js list|claim [--max N]|release ...\n'); + process.exit(2); + } +} + +main(); diff --git a/log.md b/log.md index c930fc2..bd066cf 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] ". Release deletes the lock only when the caller's +# run id matches the recorded owner, so a stale caller can never delete the +# next holder's lock (see wiki/backend/common/concurrency/distributed-locks.md). +# +# usage: +# flush-lock.sh acquire # mkdir the lock; on success writes the owner file +# # and exits 0 ("acquired " / "reclaimed "). +# # exits 3 ("held ...") when another run holds it. +# flush-lock.sh release # removes the lock iff this run id is the owner. +# # exits 4 ("refused: ...") when it is not. +# # exits 0 when no lock is present (already released). +# flush-lock.sh holder # prints the owner line; exit 0 held / exit 1 free. +# +# env overrides (mainly for tests / non-standard layouts): +# DEV_LOOP_FLUSH_LOCK lock dir path (default: $HOME/.dev-loop/.flush.lock) +# DEV_LOOP_FLUSH_RUN_ID this run's id (default: -) +# DEV_LOOP_FLUSH_LOCK_TTL seconds before a lock is stale-reclaimable (default: 900) +set -eu + +LOCK="${DEV_LOOP_FLUSH_LOCK:-$HOME/.dev-loop/.flush.lock}" +RUNID="${DEV_LOOP_FLUSH_RUN_ID:-$(date +%Y%m%d-%H%M%S)-$$}" +TTL="${DEV_LOOP_FLUSH_LOCK_TTL:-900}" + +usage() { + echo "usage: flush-lock.sh acquire|release|holder" >&2 + exit 2 +} + +# Reads "$LOCK/owner" into H_RUNID/H_PID/H_EPOCH. Sets OWNER_OK=1 only for a +# well-formed line (numeric epoch) — a missing, empty, or malformed owner file +# leaves OWNER_OK=0 so the caller treats the lock as infinitely stale rather +# than wedging on a file it cannot parse. +_owner_fields() { + H_RUNID="" + H_PID="" + H_EPOCH="" + OWNER_OK=0 + if [ -f "$LOCK/owner" ]; then + read -r H_RUNID H_PID H_EPOCH < "$LOCK/owner" 2>/dev/null || true + case "$H_EPOCH" in + '' | *[!0-9]*) OWNER_OK=0 ;; + *) OWNER_OK=1 ;; + esac + fi +} + +# $1 = pid. True only for a well-formed, live pid; a malformed or empty pid +# field never crashes this check — it is simply "not alive". +_pid_alive() { + case "$1" in + '' | *[!0-9]*) return 1 ;; + esac + kill -0 "$1" 2>/dev/null +} + +_write_owner() { + printf '%s %s %s\n' "$RUNID" "$$" "$(date +%s)" > "$LOCK/owner" +} + +cmd_acquire() { + if mkdir "$LOCK" 2>/dev/null; then + _write_owner + echo "acquired $RUNID" + exit 0 + fi + + _owner_fields + + # Re-entrant: this caller already holds the lock (same run id). A run + # commonly spans more than one process under one run id — e.g. + # hooks/auto-flush.sh acquires, then exports DEV_LOOP_FLUSH_RUN_ID into the + # headless session it spawns, and that session's own step-0 acquire must + # not see itself as a foreign holder. Refresh the epoch and continue rather + # than falling into the held/reclaim logic below. A different run id still + # goes through that logic unchanged. + if [ -n "$H_RUNID" ] && [ "$H_RUNID" = "$RUNID" ]; then + _write_owner + echo "already-owned $RUNID" + exit 0 + fi + + if [ "$OWNER_OK" = 1 ]; then + now=$(date +%s) + age=$((now - H_EPOCH)) + else + age=999999999 + fi + + if [ "$age" -le "$TTL" ] || _pid_alive "$H_PID"; then + echo "held ${H_RUNID:-unknown} ${age}s" >&2 + exit 3 + fi + + # Past TTL and the recorded pid is not alive: reclaim. + rm -rf "$LOCK" + if mkdir "$LOCK" 2>/dev/null; then + _write_owner + echo "reclaimed $RUNID" + exit 0 + fi + echo "held (reclaim raced)" >&2 + exit 3 +} + +cmd_release() { + [ -d "$LOCK" ] || exit 0 + _owner_fields + if [ "$H_RUNID" = "$RUNID" ]; then + rm -rf "$LOCK" + exit 0 + fi + echo "refused: lock owned by ${H_RUNID:-unknown}" >&2 + exit 4 +} + +cmd_holder() { + if [ -f "$LOCK/owner" ]; then + cat "$LOCK/owner" + exit 0 + fi + exit 1 +} + +case "${1:-}" in + acquire) cmd_acquire ;; + release) cmd_release ;; + holder) cmd_holder ;; + *) usage ;; +esac diff --git a/scripts/wiki-lint-prohibitions.js b/scripts/wiki-lint-prohibitions.js new file mode 100644 index 0000000..21c6740 --- /dev/null +++ b/scripts/wiki-lint-prohibitions.js @@ -0,0 +1,171 @@ +#!/usr/bin/env node +// wiki-lint check 2 — mechanically enforceable prohibition-with-replacement rule. +// +// SCOPE : page body only (no frontmatter), and NOT the `## Sources` section +// (citation text quotes prohibitions and is not addressed to the reader). +// UNIT : one directive item — a table cell, or a bullet/numbered item +// (multi-sentence allowed; wrapped lines rejoined). +// DIRECTIVE : a clause in that unit that BEGINS with a prohibition token AND is +// >= 3 words long (a 2-word cell like "Never read" is a state value +// in a data column, not an instruction to the reader). +// `never-fails` (hyphenated compound) is not a token. +// COMPLIANT : the unit is an `## Instead of` row, OR the unit carries at least one +// other clause (>= 1 word) — the replacement action, or the mechanism +// that makes the prohibition true. +// INFO : a unit that is nothing but a bare 2-word prohibition clause (no other +// content) — the declared blind spot (D5). Ambiguous by shape between a +// state value (`Never read`) and a real directive (`Never retry`), so it +// is reported for a human to look at, never as a checker error. +// +// Ported from the validated rule at .orchestration/evidence/i36-rule-probe.js — do +// not re-derive the parse; it was measured against the live corpus (61 directive +// units, 0 violations) before this script existed. +'use strict'; + +const fs = require('fs'); +const path = require('path'); + +const TOKEN = "(?:don't|do not|never|avoid|must not)"; +const STARTS = new RegExp(`^${TOKEN}(?![\\w-])`, 'i'); +const ANY = new RegExp(`(? s.replace(/[*`_]/g, '').trim(); +const words = (s) => s.split(/\s+/).filter(Boolean).length; + +function walk(dir, out = []) { + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const p = path.join(dir, entry.name); + if (entry.isDirectory()) walk(p, out); + else if (entry.name.endsWith('.md')) out.push(p); + } + return out; +} + +// Reassemble a page body into directive items: one per table cell, or per +// bullet/numbered item with its wrapped continuation lines rejoined. +function collectItems(body) { + let section = ''; + let buf = []; + const items = []; + const flush = () => { + if (buf.length) { + items.push({ text: buf.join(' ').replace(/\s+/g, ' ').trim(), section }); + buf = []; + } + }; + + for (const raw of body.split('\n')) { + if (/^#{1,3}\s+/.test(raw)) { + flush(); + section = raw.replace(/^#{1,3}\s+/, '').trim(); + continue; + } + if (raw.trim().startsWith('|')) { + flush(); + if (/^\|[\s|:-]+\|$/.test(raw.trim())) continue; // table separator row + raw + .split('|') + .slice(1, -1) + .forEach((cell) => cell.trim() && items.push({ text: cell.trim(), section })); + continue; + } + if (!raw.trim()) { + flush(); + continue; + } + if (/^\s*(?:\d+\.|[-*])\s+/.test(raw)) { + flush(); + buf.push(raw.replace(/^\s*(?:\d+\.|[-*])\s+/, '')); + } else { + buf.push(raw.trim()); + } + } + flush(); + return items; +} + +function lint(files) { + let directiveUnits = 0; + let okInstead = 0; + let okPair = 0; + const violations = []; + const infos = []; + + for (const file of files) { + const raw = fs.readFileSync(file, 'utf8'); + const body = raw.replace(/^---\n[\s\S]*?\n---\n/, ''); + const items = collectItems(body); + + for (const { text, section } of items) { + if (/^Sources$/i.test(section)) continue; + const unit = strip(text); + if (!ANY.test(unit)) continue; + + const clauses = unit + .split(CLAUSE_SPLIT) + .map((c) => c.trim()) + .filter(Boolean); + const directives = clauses.filter((c) => STARTS.test(c) && words(c) >= 3); + + if (!directives.length) { + // Known blind spot (D5): a unit that is nothing but a bare 2-word + // prohibition clause is undecidable between a state value and a real + // directive. Report it at info; never treat it as a violation. + if (clauses.length === 1 && STARTS.test(clauses[0]) && words(clauses[0]) === 2) { + infos.push(`${file}:${section}: ${unit}`); + } + continue; + } + + directiveUnits++; + if (/^Instead of$/i.test(section)) { + okInstead++; + continue; + } + if (clauses.some((c) => !directives.includes(c) && words(c) >= 1)) { + okPair++; + continue; + } + violations.push(`${file}:${section}: ${unit}`); + } + } + + return { directiveUnits, compliant: okInstead + okPair, violations, infos }; +} + +function main() { + const target = process.argv[2] || 'wiki'; + + let stat; + try { + stat = fs.statSync(target); + } catch { + stat = null; + } + if (!stat) { + console.error(`wiki-lint-prohibitions: no such directory: ${target}`); + process.exit(2); + } + const files = stat.isDirectory() ? walk(target) : [target]; + + const { directiveUnits, compliant, violations, infos } = lint(files); + + if (violations.length) { + console.log('--- violations ---'); + violations.forEach((v) => console.log(v)); + } + if (infos.length) { + console.log('--- info: bare 2-word prohibition cells (state value or directive — undecidable by shape) ---'); + infos.forEach((i) => console.log(i)); + } + + console.log('--- summary ---'); + console.log(`directives: ${directiveUnits}`); + console.log(`compliant: ${compliant}`); + console.log(`violations: ${violations.length}`); + console.log(`info: ${infos.length}`); + + process.exit(violations.length ? 1 : 0); +} + +main(); diff --git a/skills/knowledge-flush/SKILL.md b/skills/knowledge-flush/SKILL.md index 58fe393..85393b5 100644 --- a/skills/knowledge-flush/SKILL.md +++ b/skills/knowledge-flush/SKILL.md @@ -19,6 +19,38 @@ row is a candidate: `trigger, directive, why, evidence, domain, tags, content`. `hooks/pre-flush-pr-gate.sh` blocks `gh pr create` on a knowledge branch unless an `INGEST_REPORT.md` with three filled sections exists. So do the work first: +0. **Acquire the shared flush lock — before anything else, including the + checkout reset in step 1.** This skill and the `hooks/auto-flush.sh` Stop + hook are the two entry points that drain the same queue; both serialize + through the same lock (`scripts/flush-lock.sh`, issue #77). + + Generate this run's id **once, here**, and record it — **every** later + `flush-lock.sh` call in this flush (the abort-path release below, and + step 5's release) must be prefixed with it. This is mandatory, not a + style choice: each step in this skill runs in its own fresh shell, so + nothing carries between them on its own — an unprefixed call computes a + brand-new run id that no longer matches the one that acquired the lock, + so `release` is refused as a foreign caller and the lock leaks for the + full TTL. + ```sh + RUNID="flush-$(date +%Y%m%d-%H%M%S)-$$" + echo "$RUNID" # record this — every later flush-lock.sh call needs it + DEV_LOOP_FLUSH_RUN_ID="$RUNID" sh "${CLAUDE_PLUGIN_ROOT}/scripts/flush-lock.sh" acquire + ``` + - Exit 0 → proceed to step 1. + - Non-zero exit → **do not touch `~/.dev-loop/repo`.** Read the failure + line (`held s` on stderr) and tell the user: "a + flush is already running (holder ``, started `s` + ago) — stopping." Then stop. Do not retry, poll, or wait for the lock. + The lock is released once, at the very end of step 5 on a successful + flush. If you abort or hit an unrecoverable error at any point after + acquiring it, release it before stopping — **prefixed with the same + `$RUNID`** recorded above — so the next run does not wait out the TTL: + ```sh + DEV_LOOP_FLUSH_RUN_ID="" \ + sh "${CLAUDE_PLUGIN_ROOT}/scripts/flush-lock.sh" release + ``` + 1. **Prepare a writable checkout** of the dev-loop repo (never edit the installed plugin dir — it is read-only and untracked): ```sh @@ -48,8 +80,18 @@ an `INGEST_REPORT.md` with three filled sections exists. So do the work first: `AGENTS.md`), NOT `${CLAUDE_PLUGIN_ROOT}`. The push + PR use the ambient `gh` auth, so the PR is opened by whichever account this user is logged in as. -2. **For each queued candidate, run the pre-PR pipeline** (this is the whole point - — a raw harvested block is a *candidate*, not vetted knowledge): +2. **Claim your candidates, then for each one run the pre-PR pipeline** (this is + the whole point — a raw harvested block is a *candidate*, not vetted + knowledge): + + **Claim first, before reading or ingesting anything:** + ```sh + node "${CLAUDE_PLUGIN_ROOT}/hooks/queue-claim.js" claim + ``` + Work only the ids this command prints — those are the rows this run now + owns. A row already claimed by another (live or not-yet-TTL-expired) run, + or claimed by a sibling run that raced past the lock, is not printed; + skip it — do not read or ingest a row this command did not print. a. **Research & verify the best-practice.** Do a real search — official docs, primary sources, reputable references (use WebSearch / context7 / the @@ -140,17 +182,37 @@ an `INGEST_REPORT.md` with three filled sections exists. So do the work first: Do NOT `gh pr merge`. The owner reviews open `dev-loop:knowledge` PRs and merges or rejects each one. -5. **Retire processed candidates — every one you handled, not only the ingested.** - Move each handled row out of the active queue (append it to +5. **Retire processed candidates — every claimed row you handled, not only the + ingested.** Move each handled row out of the active queue (append it to `~/.dev-loop/queue/.processed.jsonl` and rewrite the session file without it): rows you ingested, rows you merged into existing pages, AND rows you dropped - as unverifiable or duplicate. A dropped row left `pending` re-crosses the - auto-flush threshold forever — the headless flush would re-run hourly on + as unverifiable or duplicate. Retire only rows step 2 claimed for **this** + run — never a row you did not claim. A dropped row left `pending` re-crosses + the auto-flush threshold forever — the headless flush would re-run hourly on candidates that can never be promoted. When the rewrite leaves a session file empty, delete the file — empty leftovers otherwise accumulate in the queue directory (the harvester also removes its own empty file on later Stops). + If step 2 claimed a row you did not end up handling (the run is aborting, + or the candidate is being left for a retry), release it instead of retiring + it, so a later run does not wait out the claim TTL to see it again: + ```sh + node "${CLAUDE_PLUGIN_ROOT}/hooks/queue-claim.js" release ... + ``` + + Once every claimed row is retired or released, release the flush lock — + this is the normal end of a successful flush. Prefix with the **same** + `$RUNID` recorded in step 0: this step runs in its own fresh shell, so an + unprefixed call computes a new run id and `release` is refused as a + foreign caller, leaking the lock for the full TTL: + ```sh + DEV_LOOP_FLUSH_RUN_ID="" \ + sh "${CLAUDE_PLUGIN_ROOT}/scripts/flush-lock.sh" release + ``` + ## Guardrails +- Never run two flushes at once — step 0's lock is the only serialization + point, and it is shared with `hooks/auto-flush.sh`. - PR-only. Never auto-merge, never push to `main`, never force-push `main`. - Commit under the **user's own ambient git/gh identity** — never hardcode an account, never commit as an assistant, never add a `Co-Authored-By` trailer. diff --git a/skills/wiki-lint/SKILL.md b/skills/wiki-lint/SKILL.md index 1b54968..67ca333 100644 --- a/skills/wiki-lint/SKILL.md +++ b/skills/wiki-lint/SKILL.md @@ -21,7 +21,7 @@ Run all of these; report findings grouped by severity. | # | Check | Severity | |---|-------|----------| | 1 | Page with `confidence: verified` but empty/unverifiable `sources:` | error | -| 2 | "Don't/never/avoid" directive outside an `Instead of` table, or an `Instead of` row missing its replacement | error | +| 2 | A prohibition (`don't`/`do not`/`never`/`avoid`/`must not`) alone in its directive item (table cell or bullet), carrying no replacement action or mechanism — checked via `node scripts/wiki-lint-prohibitions.js`; `Instead of` rows must still pair the anti-pattern with its replacement | error | | 3 | Broken `related:` id or inline link | error | | 4 | Page not listed in its domain `index.md`, or index entry whose "load when" line no longer matches the page trigger | error | | 5 | Vague qualifiers in directive sentences (usually, consider, might, generally, as appropriate) | warn | @@ -30,13 +30,16 @@ Run all of these; report findings grouped by severity. | 8 | `last_verified` older than 12 months on `verified` pages (docs move, defaults change) | warn | | 9 | `contradiction` entries in `log.md` still unresolved | warn | | 10 | `gap` entries in `log.md` with no page created after 30 days | info | +| 11 | Bare 2-word prohibition cell (e.g. `Never read`) — undecidable by shape between a state value and a real directive, so it is surfaced rather than judged; reported by `node scripts/wiki-lint-prohibitions.js` | info | ## Fix protocol - Fix mechanical findings (3, 4, 6 splits, index lines) directly. -- For 1 and 2: fix when the correct source/replacement is known with certainty; - otherwise downgrade to `unverified` / move the prohibition into `Instead of` with a - `TODO replacement` marker and report it — do not invent sources or replacements. +- For 1: fix when the correct source is known with certainty; otherwise downgrade + to `unverified` and report it — do not invent sources. +- For 2: add the replacement action or the mechanism in place, in the same + directive item; moving the row into `Instead of` is one option, not the required + one. Do not invent a replacement — report it if none is known with certainty. - For 5: rewrite the sentence as a conditional ("When X, do A") only when the condition is stated elsewhere in the page; otherwise report it. - Append `## [YYYY-MM-DD] lint | errors fixed, reported` to `log.md`. diff --git a/tests/fixtures/prohibitions/bad.md b/tests/fixtures/prohibitions/bad.md new file mode 100644 index 0000000..ef53476 --- /dev/null +++ b/tests/fixtures/prohibitions/bad.md @@ -0,0 +1,11 @@ +--- +title: bad fixture (negative control) +--- + +# Bad fixture + +## Do this +- Never log secrets. + +## Sources +- Never log secrets, per the vendor incident report (accessed 2026-08-12). diff --git a/tests/fixtures/prohibitions/good.md b/tests/fixtures/prohibitions/good.md new file mode 100644 index 0000000..9d5d8fa --- /dev/null +++ b/tests/fixtures/prohibitions/good.md @@ -0,0 +1,22 @@ +--- +title: good fixture +--- + +# Good fixture + +## Do this +- Never retry a 500 without a backoff — the server may be transiently overloaded. +- Do not cache the response. The upstream marks it no-store. + +## Instead of +| Anti-pattern | Replacement | +|---|---| +| Retrying blindly | Never retry blindly — always confirm the response is safe to retry | + +## Notes +A function that never-fails still needs input validation. + +## State values +| Key | Behavior | +|---|---| +| Unknown key | Never read | diff --git a/tests/flush-lock.bats b/tests/flush-lock.bats new file mode 100644 index 0000000..ccb1025 --- /dev/null +++ b/tests/flush-lock.bats @@ -0,0 +1,143 @@ +#!/usr/bin/env bats +# Tests for scripts/flush-lock.sh — the owner-token mkdir-atomicity lock shared +# by hooks/auto-flush.sh and skills/knowledge-flush/SKILL.md (issue #77). +# +# Every test points DEV_LOOP_FLUSH_LOCK at a path under BATS_TEST_TMPDIR so no +# test ever touches the real ~/.dev-loop. + +setup() { + SCRIPT="${BATS_TEST_DIRNAME}/../scripts/flush-lock.sh" + export DEV_LOOP_FLUSH_LOCK="${BATS_TEST_TMPDIR}/flush.lock" +} + +_now() { date +%s; } + +# Write a lock dir + owner file directly (bypassing the script) so tests can +# construct held/stale/malformed states precisely. +_seed_lock() { # runid pid epoch + mkdir -p "$DEV_LOOP_FLUSH_LOCK" + printf '%s %s %s\n' "$1" "$2" "$3" > "$DEV_LOOP_FLUSH_LOCK/owner" +} + +# A pid guaranteed dead: spawn, wait for exit, reap it. +_dead_pid() { + ( exit 0 ) & + local pid=$! + wait "$pid" 2>/dev/null || true + echo "$pid" +} + +@test "acquire on a clean dir: exit 0, lock dir + owner file exist" { + run env DEV_LOOP_FLUSH_RUN_ID=run-a sh "$SCRIPT" acquire + [ "$status" -eq 0 ] + [[ "$output" == "acquired run-a" ]] + [ -d "$DEV_LOOP_FLUSH_LOCK" ] + [ -f "$DEV_LOOP_FLUSH_LOCK/owner" ] + read -r runid pid epoch < "$DEV_LOOP_FLUSH_LOCK/owner" + [ "$runid" = "run-a" ] +} + +@test "second acquire while held (young, holder alive): exit 3, lock untouched" { + _seed_lock run-holder "$$" "$(_now)" + run env DEV_LOOP_FLUSH_RUN_ID=run-b sh "$SCRIPT" acquire + [ "$status" -eq 3 ] + [[ "$output" == *"held run-holder"* ]] + read -r runid _ _ < "$DEV_LOOP_FLUSH_LOCK/owner" + [ "$runid" = "run-holder" ] +} + +@test "release with a different DEV_LOOP_FLUSH_RUN_ID: exit 4, lock still exists" { + _seed_lock run-holder 1 "$(_now)" + run env DEV_LOOP_FLUSH_RUN_ID=run-someone-else sh "$SCRIPT" release + [ "$status" -eq 4 ] + [[ "$output" == *"refused"* ]] + [ -d "$DEV_LOOP_FLUSH_LOCK" ] + [ -f "$DEV_LOOP_FLUSH_LOCK/owner" ] +} + +@test "own release: match removes the lock, exit 0" { + _seed_lock run-mine "$$" "$(_now)" + run env DEV_LOOP_FLUSH_RUN_ID=run-mine sh "$SCRIPT" release + [ "$status" -eq 0 ] + [ ! -d "$DEV_LOOP_FLUSH_LOCK" ] +} + +@test "stale lock (past TTL) + dead pid: acquire reclaims, exit 0" { + dead="$(_dead_pid)" + _seed_lock run-crashed "$dead" "$(( $(_now) - 1000 ))" + run env DEV_LOOP_FLUSH_RUN_ID=run-new sh "$SCRIPT" acquire + [ "$status" -eq 0 ] + [[ "$output" == "reclaimed run-new" ]] + read -r runid _ _ < "$DEV_LOOP_FLUSH_LOCK/owner" + [ "$runid" = "run-new" ] +} + +@test "young lock + dead pid: still held, not reclaimed (exit 3)" { + dead="$(_dead_pid)" + _seed_lock run-crashed "$dead" "$(_now)" + run env DEV_LOOP_FLUSH_RUN_ID=run-new sh "$SCRIPT" acquire + [ "$status" -eq 3 ] + read -r runid _ _ < "$DEV_LOOP_FLUSH_LOCK/owner" + [ "$runid" = "run-crashed" ] +} + +@test "stale (past TTL) but pid alive: still held, not reclaimed (exit 3)" { + _seed_lock run-holder "$$" "$(( $(_now) - 1000 ))" + run env DEV_LOOP_FLUSH_RUN_ID=run-new sh "$SCRIPT" acquire + [ "$status" -eq 3 ] + read -r runid _ _ < "$DEV_LOOP_FLUSH_LOCK/owner" + [ "$runid" = "run-holder" ] +} + +@test "release with no lock present: exit 0 (empty/absent state)" { + run env DEV_LOOP_FLUSH_RUN_ID=run-a sh "$SCRIPT" release + [ "$status" -eq 0 ] + [ ! -d "$DEV_LOOP_FLUSH_LOCK" ] +} + +@test "malformed/empty owner file: acquire reclaims rather than wedging" { + mkdir -p "$DEV_LOOP_FLUSH_LOCK" + : > "$DEV_LOOP_FLUSH_LOCK/owner" + run env DEV_LOOP_FLUSH_RUN_ID=run-new sh "$SCRIPT" acquire + [ "$status" -eq 0 ] + [[ "$output" == "reclaimed run-new" ]] +} + +@test "holder: prints the owner line and exits 0 when held" { + _seed_lock run-holder 123 "$(_now)" + run sh "$SCRIPT" holder + [ "$status" -eq 0 ] + [[ "$output" == "run-holder 123"* ]] +} + +@test "holder: exits 1 when no lock is held" { + run sh "$SCRIPT" holder + [ "$status" -eq 1 ] +} + +@test "unknown or missing verb: exit 2" { + run sh "$SCRIPT" bogus + [ "$status" -eq 2 ] + + run sh "$SCRIPT" + [ "$status" -eq 2 ] +} + +@test "re-entrant acquire: same run id as current holder refreshes epoch, exit 0, already-owned" { + old_epoch=$(( $(_now) - 500 )) + _seed_lock run-a "$$" "$old_epoch" + run env DEV_LOOP_FLUSH_RUN_ID=run-a sh "$SCRIPT" acquire + [ "$status" -eq 0 ] + [[ "$output" == "already-owned run-a" ]] + read -r runid pid epoch < "$DEV_LOOP_FLUSH_LOCK/owner" + [ "$runid" = "run-a" ] + [ "$epoch" -gt "$old_epoch" ] +} + +@test "re-entrant acquire does not extend to a foreign run id: still exit 3, owner unchanged" { + _seed_lock run-a "$$" "$(_now)" + run env DEV_LOOP_FLUSH_RUN_ID=run-b sh "$SCRIPT" acquire + [ "$status" -eq 3 ] + read -r runid _ _ < "$DEV_LOOP_FLUSH_LOCK/owner" + [ "$runid" = "run-a" ] +} diff --git a/tests/queue-claim.bats b/tests/queue-claim.bats new file mode 100644 index 0000000..c0b89f1 --- /dev/null +++ b/tests/queue-claim.bats @@ -0,0 +1,178 @@ +#!/usr/bin/env bats +# Tests for hooks/queue-claim.js — claim/release insight-queue rows by run id +# (issue #77). Makes the queue self-defend against a lock that has already +# been bypassed or reclaimed from a dead holder. +# +# Every test points DEV_LOOP_QUEUE_DIR at a path under BATS_TEST_TMPDIR so no +# test ever touches the real ~/.dev-loop/queue. + +setup() { + SCRIPT="${BATS_TEST_DIRNAME}/../hooks/queue-claim.js" + command -v node >/dev/null || { + echo "node is required to run the queue-claim tests" + return 1 + } + export DEV_LOOP_QUEUE_DIR="${BATS_TEST_TMPDIR}/queue" + mkdir -p "$DEV_LOOP_QUEUE_DIR" +} + +_pending_row() { # hash + printf '{"hash":"%s","status":"pending","trigger":"t","directive":"d"}' "$1" +} + +_claimed_row() { # hash claimedBy claimedAtIso + printf '{"hash":"%s","status":"claimed","claimedBy":"%s","claimedAt":"%s"}' "$1" "$2" "$3" +} + +# An ISO8601 timestamp $1 seconds in the past — via node, not `date -d` +# (GNU-only) or `date -v` (BSD-only), so this works on macOS and Linux alike. +_iso_seconds_ago() { + node -e "console.log(new Date(Date.now() - $1 * 1000).toISOString())" +} + +@test "claim over a 3-row pending file: all 3 become claimed with this run id" { + { _pending_row a; echo; _pending_row b; echo; _pending_row c; } > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + run env DEV_LOOP_FLUSH_RUN_ID=run-a node "$SCRIPT" claim + [ "$status" -eq 0 ] + [[ "$output" == $'a\nb\nc' ]] + + run node -e " + const rows = require('fs').readFileSync('$DEV_LOOP_QUEUE_DIR/s1.jsonl','utf8') + .split('\n').filter(Boolean).map(JSON.parse); + if (rows.length !== 3) throw new Error('expected 3 rows, got ' + rows.length); + for (const r of rows) { + if (r.status !== 'claimed') throw new Error(r.hash + ' not claimed'); + if (r.claimedBy !== 'run-a') throw new Error(r.hash + ' wrong claimedBy'); + if (!r.claimedAt) throw new Error(r.hash + ' missing claimedAt'); + } + " + [ "$status" -eq 0 ] +} + +@test "a second run's list on an already-claimed file: prints nothing" { + { _pending_row a; echo; _pending_row b; } > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + env DEV_LOOP_FLUSH_RUN_ID=run-a node "$SCRIPT" claim >/dev/null + + run env DEV_LOOP_FLUSH_RUN_ID=run-b node "$SCRIPT" list + [ "$status" -eq 0 ] + [ -z "$output" ] +} + +@test "--max 2 on 3 pending rows: exactly 2 claimed, 1 still pending" { + { _pending_row a; echo; _pending_row b; echo; _pending_row c; } > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + run env DEV_LOOP_FLUSH_RUN_ID=run-a node "$SCRIPT" claim --max 2 + [ "$status" -eq 0 ] + [[ "$output" == $'a\nb' ]] + + run env DEV_LOOP_FLUSH_RUN_ID=run-b node "$SCRIPT" list + [ "$status" -eq 0 ] + [[ "$output" == "c" ]] +} + +@test "a claimed row past the claim TTL reappears in list" { + past="$(_iso_seconds_ago 4000)" + _claimed_row a run-old "$past" > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + run node "$SCRIPT" list + [ "$status" -eq 0 ] + [[ "$output" == "a" ]] +} + +@test "a claimed row within the claim TTL does not reappear in list" { + recent="$(_iso_seconds_ago 5)" + _claimed_row a run-recent "$recent" > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + run node "$SCRIPT" list + [ "$status" -eq 0 ] + [ -z "$output" ] +} + +@test "empty queue dir (no files at all): list prints nothing, exit 0" { + run node "$SCRIPT" list + [ "$status" -eq 0 ] + [ -z "$output" ] +} + +@test "queue dir does not exist: list/claim/release do not crash, exit 0" { + rm -rf "$DEV_LOOP_QUEUE_DIR" + run node "$SCRIPT" list + [ "$status" -eq 0 ] + + run node "$SCRIPT" claim + [ "$status" -eq 0 ] + + run node "$SCRIPT" release whatever + [ "$status" -eq 0 ] +} + +@test "a queue file with zero rows: list prints nothing, exit 0, no crash" { + : > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + run node "$SCRIPT" list + [ "$status" -eq 0 ] + [ -z "$output" ] +} + +@test "one malformed line plus one valid pending row: malformed survives verbatim, valid row claimed" { + printf '{not valid json\n' > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + _pending_row b >> "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + echo >> "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + + run env DEV_LOOP_FLUSH_RUN_ID=run-a node "$SCRIPT" claim + [ "$status" -eq 0 ] + [[ "$output" == "b" ]] + + run grep -Fx '{not valid json' "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + [ "$status" -eq 0 ] +} + +@test ".processed.jsonl present with pending-looking rows: never claimed" { + _pending_row z > "$DEV_LOOP_QUEUE_DIR/.processed.jsonl" + run node "$SCRIPT" list + [ "$status" -eq 0 ] + [ -z "$output" ] + + run node "$SCRIPT" claim + [ "$status" -eq 0 ] + [ -z "$output" ] + + run node -e " + const r = JSON.parse(require('fs').readFileSync('$DEV_LOOP_QUEUE_DIR/.processed.jsonl','utf8')); + if (r.status !== 'pending') throw new Error('processed row was mutated: ' + r.status); + " + [ "$status" -eq 0 ] +} + +@test "release: sets a claimed row back to pending and clears claimedBy/claimedAt" { + { _pending_row a; echo; _pending_row b; } > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + env DEV_LOOP_FLUSH_RUN_ID=run-a node "$SCRIPT" claim >/dev/null + + run node "$SCRIPT" release a + [ "$status" -eq 0 ] + + run node -e " + const rows = require('fs').readFileSync('$DEV_LOOP_QUEUE_DIR/s1.jsonl','utf8') + .split('\n').filter(Boolean).map(JSON.parse); + const a = rows.find(r => r.hash === 'a'); + const b = rows.find(r => r.hash === 'b'); + if (a.status !== 'pending') throw new Error('a not pending: ' + a.status); + if ('claimedBy' in a || 'claimedAt' in a) throw new Error('a still carries claim fields'); + if (b.status !== 'claimed') throw new Error('release touched an id it was not given'); + " + [ "$status" -eq 0 ] +} + +@test "release: unknown id is a no-op, exit 0, no crash" { + _pending_row a > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + run node "$SCRIPT" release does-not-exist + [ "$status" -eq 0 ] + + run node -e " + const r = JSON.parse(require('fs').readFileSync('$DEV_LOOP_QUEUE_DIR/s1.jsonl','utf8')); + if (r.status !== 'pending') throw new Error('untouched row was mutated'); + " + [ "$status" -eq 0 ] +} + +@test "release with no ids given: no-op, exit 0" { + _pending_row a > "$DEV_LOOP_QUEUE_DIR/s1.jsonl" + run node "$SCRIPT" release + [ "$status" -eq 0 ] +} diff --git a/tests/wiki-lint-prohibitions.bats b/tests/wiki-lint-prohibitions.bats new file mode 100644 index 0000000..e5b5b0d --- /dev/null +++ b/tests/wiki-lint-prohibitions.bats @@ -0,0 +1,119 @@ +#!/usr/bin/env bats +# Tests for scripts/wiki-lint-prohibitions.js (wiki-lint check 2). +# +# The checker's own report is not evidence it works until it has been shown to +# fail on something — a checker observed only ever returning 0 violations proves +# nothing (wiki/testing/quality/checks-that-cannot-pass.md). tests/fixtures/ +# prohibitions/bad.md is that negative control: a bare prohibition the checker +# must catch. good.md is the paired-case control: every shape the rule allows +# (em-dash pairing, sentence pairing, an `Instead of` row, a hyphenated +# `never-fails` compound, and the D5 bare-2-word-cell blind spot) must NOT be +# reported as a violation. + +setup() { + CHECKER="${BATS_TEST_DIRNAME}/../scripts/wiki-lint-prohibitions.js" + REPO_ROOT="${BATS_TEST_DIRNAME}/.." + FIXTURES="${BATS_TEST_DIRNAME}/fixtures/prohibitions" +} + +# --- normal: the real corpus is already compliant --------------------------- + +@test "real wiki: exits 0 with 0 violations and 61 directive units" { + cd "$REPO_ROOT" || return 1 + run node "$CHECKER" wiki + [ "$status" -eq 0 ] + [[ "$output" == *"directives: 61"* ]] + [[ "$output" == *"violations: 0"* ]] +} + +# --- error: the negative control (D6) --------------------------------------- + +@test "fixture dir: exits 1 and names bad.md as the sole violation" { + run node "$CHECKER" "$FIXTURES" + [ "$status" -eq 1 ] + [[ "$output" == *"bad.md"* ]] + # exactly one violation — the Sources quote (same file) must not inflate this. + [[ "$output" == *"violations: 1"* ]] +} + +# --- normal: every paired/permitted shape passes ----------------------------- + +@test "good.md alone: exits 0, no violations" { + run node "$CHECKER" "$FIXTURES/good.md" + [ "$status" -eq 0 ] + [[ "$output" == *"violations: 0"* ]] +} + +# --- boundary: `## Sources` is excluded (D4) --------------------------------- + +@test "a Sources line quoting a bare prohibition is not flagged" { + run node "$CHECKER" "$FIXTURES" + [ "$status" -eq 1 ] + # the Sources bullet repeats "Never log secrets" verbatim; if the Sources + # exclusion broke, violations would be 2, not 1. + [[ "$output" == *"violations: 1"* ]] +} + +# --- boundary: the 2-word blind spot is reported at info, never error (D5) -- + +@test "a bare 2-word prohibition cell is reported under info, exit still 0" { + run node "$CHECKER" "$FIXTURES/good.md" + [ "$status" -eq 0 ] + [[ "$output" == *"info: 1"* ]] + [[ "$output" == *"--- info:"* ]] + [[ "$output" == *"Never read"* ]] +} + +# --- explicit DoD case: the single most common corpus shape (D3) ------------ +# `wiki/backend/common/reliability/timeouts-and-retries.md` pairs a bare +# "Never retry" (2 words) with its reason via em-dash/semicolon. Because the +# prohibition clause itself is under the 3-word directive threshold, the unit +# falls out of `directives` entirely — it is never classified as a violation, +# compliant pair, or info row. rule 3 must still be read as PERMITTING this +# shape (it is never an error); this test is the checked-explicitly record for +# that DoD line, not a claim that the checker recognizes it as a directive. + +@test "the DoD's cited row (bare 2-word prohibition + reason via em-dash/semicolon) is not a violation" { + dir="${BATS_TEST_TMPDIR}/dod-example" + mkdir -p "$dir" + printf -- '---\ntitle: dod example\n---\n\n## Status codes\n| Codes | Behavior |\n|---|---|\n| 400/401/403/404/422 | Never retry — the request itself is wrong; the same bytes fail again |\n' > "$dir/row.md" + run node "$CHECKER" "$dir" + [ "$status" -eq 0 ] + [[ "$output" == *"violations: 0"* ]] +} + +# --- boundary: empty input --------------------------------------------------- + +@test "an empty directory: exits 0, no crash" { + empty_dir="${BATS_TEST_TMPDIR}/empty-wiki" + mkdir -p "$empty_dir" + run node "$CHECKER" "$empty_dir" + [ "$status" -eq 0 ] + [[ "$output" == *"directives: 0"* ]] + [[ "$output" == *"violations: 0"* ]] +} + +@test "a page with an empty body: exits 0, no crash" { + empty_body_dir="${BATS_TEST_TMPDIR}/empty-body-wiki" + mkdir -p "$empty_body_dir" + printf -- '---\ntitle: empty\n---\n' > "$empty_body_dir/empty.md" + run node "$CHECKER" "$empty_body_dir" + [ "$status" -eq 0 ] + [[ "$output" == *"directives: 0"* ]] + [[ "$output" == *"violations: 0"* ]] +} + +# --- error: bad directory argument ------------------------------------------- + +@test "a nonexistent directory: exits 2 with a stderr message" { + run node "$CHECKER" "${BATS_TEST_TMPDIR}/does-not-exist" + [ "$status" -eq 2 ] + [[ "$output" == *"no such directory"* ]] +} + +# --- syntax ------------------------------------------------------------------- + +@test "the script is syntactically valid" { + run node --check "$CHECKER" + [ "$status" -eq 0 ] +} diff --git a/wiki/backend/index.md b/wiki/backend/index.md index cee0276..b84910d 100644 --- a/wiki/backend/index.md +++ b/wiki/backend/index.md @@ -5,7 +5,7 @@ three stack subtrees — route by concern first, stack second: | Subtree | Route there when | |---------|------------------| -| [common](#common-language-agnostic) (below) | The concern is language-agnostic: API contracts, enumerating call sites before a contract change, idempotency, JWT issuance, outbound calls, caching, jobs, transactions in app code, shared state/pools, exception structure, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references | +| [common](#common-language-agnostic) (below) | The concern is language-agnostic: API contracts, enumerating call sites before a contract change, idempotency, JWT issuance, outbound calls, caching, jobs, transactions in app code, shared state/pools, exception structure, consuming LLM APIs (completion validation, context budgeting), consuming external-API responses, externally-owned defaults, object-storage references | | [java](java/index.md) | You are writing/reviewing JVM backend code (Java/Kotlin, Spring, JPA/Hibernate) and the concern is stack-specific: entity mapping, persistence context, proxy pitfalls, JVM threads/memory | | [node](node/index.md) | You are writing/reviewing Node.js/TypeScript backend code: event-loop blocking, promise error handling, runtime validation at boundaries, graceful shutdown | | [python](python/index.md) | You are writing/reviewing Python backend code: GIL/concurrency model, pydantic validation, WSGI/ASGI workers, language traps | diff --git a/wiki/databases/index.md b/wiki/databases/index.md index 66083a7..c789ae8 100644 --- a/wiki/databases/index.md +++ b/wiki/databases/index.md @@ -1,7 +1,8 @@ # databases — Domain Index Route here for: schema/table/key design, index decisions, query writing and -optimization, transaction/concurrency behavior. +optimization, transaction/concurrency behavior. Datastore backup/restore and +data-loss planning → wiki/infrastructure/data/backup-and-restore.md. Match your situation to a "load when" line; load only matching pages. diff --git a/wiki/debugging/concurrency/intermittent-failures.md b/wiki/debugging/concurrency/intermittent-failures.md index 0f417f1..afe49ee 100644 --- a/wiki/debugging/concurrency/intermittent-failures.md +++ b/wiki/debugging/concurrency/intermittent-failures.md @@ -8,16 +8,18 @@ sources: - https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html - https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come-from.html last_verified: 2026-07-10 -related: [debugging-methodology-isolate-by-bisection, debugging-methodology-reproduce-first] +related: [debugging-methodology-isolate-by-bisection, debugging-methodology-reproduce-first, testing-flaky-diagnosing-flaky-tests] --- # Making an Intermittent Failure Reproducible ## When this applies -A failure happens only sometimes: the test passes on retry, the bug appears only -under load, only in CI, only occasionally in prod. You cannot debug it because -you cannot trigger it on demand. +The intermittent failure is in the system itself: an occasional prod error, a +bug that appears only under load, or behaviour you can trigger only +sometimes. You cannot debug it because you cannot trigger it on demand — the +job is to amplify it into a reproduction. If the unreliable thing is a test in +your suite, go to wiki/testing/flaky/diagnosing-flaky-tests.md. ## Do this diff --git a/wiki/debugging/index.md b/wiki/debugging/index.md index a6c383b..8452e6f 100644 --- a/wiki/debugging/index.md +++ b/wiki/debugging/index.md @@ -32,4 +32,4 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| -| [intermittent-failures](concurrency/intermittent-failures.md) | A failure happens only sometimes: passes on retry, fails under load, fails only in CI, flaky test, occasional prod-only error — amplifying it into an on-demand reproduction | +| [intermittent-failures](concurrency/intermittent-failures.md) | The intermittent failure is in the system itself — an occasional prod error, a bug that appears only under load, behaviour you can trigger only sometimes — and you need to amplify it into an on-demand reproduction by finding the hidden variable (the unreliable thing is a test in your suite → wiki/testing/flaky/diagnosing-flaky-tests.md) | diff --git a/wiki/qa/document-verification/spec-document-gates.md b/wiki/qa/document-verification/spec-document-gates.md index 9c2fea6..369a1cd 100644 --- a/wiki/qa/document-verification/spec-document-gates.md +++ b/wiki/qa/document-verification/spec-document-gates.md @@ -20,6 +20,8 @@ related: testing-quality-tests-that-cannot-fail, platforms-environment-unicode-text-matching, qa-deliverables-generated-artifacts-as-deliverable-source, + testing-quality-spec-artifact-checks, + testing-quality-checks-that-cannot-pass, ] --- @@ -27,10 +29,13 @@ related: ## When this applies -You are writing or reviewing automated checks (grep/script) that decide whether a -written deliverable — RFC, API spec, schema doc, design doc — satisfies its stated -requirements; a document passed its checklist but a reviewer still found the -requirement unmet; you are fixing gate patterns for a document that is not written yet. +A document passed its checklist but a reviewer still found the requirement +unmet: you are deciding whether a doc gate's assertions are sufficient to +accept the deliverable — table structure, MUST-vs-SHOULD demotion, closed-set +completeness, cross-section consistency — and what release policy the gate's +verdict feeds. Authoring or validating the check code itself belongs to +wiki/testing/quality/spec-artifact-checks.md and +wiki/testing/quality/checks-that-cannot-pass.md. ## Do this @@ -59,6 +64,11 @@ requirement unmet; you are fixing gate patterns for a document that is not writt 5. **Write patterns against the literal text in the document**, not a remembered stem — for non-ASCII text apply [platforms-environment-unicode-text-matching]. +Items 1, 2, 4, and 5 above validate the check's own mechanism; the general +technique lives in wiki/testing/quality/spec-artifact-checks.md and +wiki/testing/quality/checks-that-cannot-pass.md. Item 3 (the four axes) states +what this gate must assert about the document and stays here. + ## Edge cases | Case | Then | diff --git a/wiki/qa/index.md b/wiki/qa/index.md index 2278dfd..be96aca 100644 --- a/wiki/qa/index.md +++ b/wiki/qa/index.md @@ -29,7 +29,7 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| -| [spec-document-gates](document-verification/spec-document-gates.md) | Writing or reviewing automated checks (grep/script) that decide whether a spec/RFC/schema document meets its requirements; a document passed its checklist but the requirement is still unmet; choosing what a doc gate must assert beyond keyword presence (table structure, MUST-vs-SHOULD demotion, closed-set completeness, cross-section consistency); validating a gate pattern for a document that does not exist yet | +| [spec-document-gates](document-verification/spec-document-gates.md) | Deciding whether passing a doc gate is enough to accept the deliverable: a document passed its checklist but the requirement is still unmet; choosing what the gate must assert beyond keyword presence (table structure, MUST-vs-SHOULD demotion, closed-set completeness, cross-section consistency); setting the release policy a gate verdict feeds (authoring or validating the check code itself → wiki/testing/quality/spec-artifact-checks.md, wiki/testing/quality/checks-that-cannot-pass.md) | | [editing-a-gated-document](document-verification/editing-a-gated-document.md) | Editing or rewording a document that grep/regex gates or a lint config check; a gate fails on wording whose meaning did not change; describing what an upstream spec says without tripping a "do not redefine it" gate; a check matches the pattern your own document quotes; recording an audit verdict inside the document that was audited; deciding which checks to re-run after editing a gated document | ## environments diff --git a/wiki/testing/flaky/diagnosing-flaky-tests.md b/wiki/testing/flaky/diagnosing-flaky-tests.md index 4f49231..bd2b5c6 100644 --- a/wiki/testing/flaky/diagnosing-flaky-tests.md +++ b/wiki/testing/flaky/diagnosing-flaky-tests.md @@ -9,17 +9,20 @@ sources: - https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html - https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come-from.html last_verified: 2026-07-10 -related: [testing-data-test-data-and-isolation, testing-mocking-what-to-mock] +related: [testing-data-test-data-and-isolation, testing-mocking-what-to-mock, debugging-concurrency-intermittent-failures] --- # Fixing a Test That Fails Intermittently ## When this applies -A test sometimes fails and sometimes passes with no code change: it fails in CI -but passes locally, fails only when run with other tests, or passes on retry. -This page covers test-specific causes and fixes; for general root-cause -methodology on the intermittent failure itself, see wiki/debugging/. +The unreliable thing is a test in your suite: it fails in CI but passes +locally, fails only when run with other tests, or passes on retry. You are +hunting its test-level cause (order dependence, shared fixture state, +time/timezone, async waits) or setting quarantine-vs-retry policy for it. If +the same intermittency also appears outside the suite, or the test-level +causes are exhausted and it still will not reproduce on demand, see +wiki/debugging/concurrency/intermittent-failures.md. ## Do this diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 5f3ed78..6b26016 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -30,8 +30,8 @@ Match your situation to a "load when" line; load only matching pages. | [minimum-case-set](quality/minimum-case-set.md) | Writing tests for a function/endpoint/change and choosing which cases to cover; reviewing whether coverage suffices; picking boundary values by input type; adding a regression test for a bug fix | | [behavior-not-implementation](quality/behavior-not-implementation.md) | Deciding what a test should assert; a behavior-preserving refactor broke tests; tempted to expose privates for testing; deciding whether a snapshot test is appropriate | | [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect | -| [checks-that-cannot-pass](quality/checks-that-cannot-pass.md) | Authoring a check whose target does not exist yet (grep/regex gate on an unwritten file or doc section, lint/scan rule, schema assertion on an unbuilt endpoint, a plan's verification command) and it has only ever been observed failing; reviewing a plan's gates before adopting them; separating "target missing" from "content missing" in a gate's exit status | -| [spec-artifact-checks](quality/spec-artifact-checks.md) | Writing or reviewing an automated check that a mapping table covers every rule/field/enum case, or that ids resolve across documents; deciding whether a green check earned "verified" or only "present"; designing one negative control per check in a multi-check harness; parsing Markdown table rows programmatically in a doc-as-spec repo | +| [checks-that-cannot-pass](quality/checks-that-cannot-pass.md) | Authoring a check whose target does not exist yet (grep/regex gate on an unwritten file or doc section, lint/scan rule, schema assertion on an unbuilt endpoint, a plan's verification command) and it has only ever been observed failing; reviewing a plan's gates before adopting them; separating "target missing" from "content missing" in a gate's exit status. This owns unwritten-target gate patterns for spec/RFC documents too — qa/document-verification does not | +| [spec-artifact-checks](quality/spec-artifact-checks.md) | Authoring or reviewing the check itself: that a mapping table covers every rule/field/enum case, that ids resolve across documents; deciding whether a green check earned "verified" or only "present"; designing one negative control per check in a multi-check harness; parsing Markdown table rows programmatically in a doc-as-spec repo (deciding whether a passing gate is enough to *accept the deliverable* → wiki/qa/document-verification/spec-document-gates.md) | | [schema-additions-under-a-golden-gate](quality/schema-additions-under-a-golden-gate.md) | Adding a node kind, variant, discriminator value, or field to a document format (IR, JSON Schema, spec artifact) whose only automated gate builds its negatives by mutating one committed golden example; the gate or the whole suite comes back green right after a schema change; deciding which negative each new schema keyword needs, and whether a green suite that never loads the schema is evidence at all | | [harness-reverse-controls](quality/harness-reverse-controls.md) | You built a harness that scores how well something is verified (mutation run, doc/spec gate suite, CI check matrix) and are about to cite its score in a commit, PR, README, or report; its verdicts come out uniform (every case caught, or every case green); deciding what control run proves the harness discriminates, how to score errored/never-ran cases, and what the harness's isolated working tree must contain | @@ -53,7 +53,7 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| -| [diagnosing-flaky-tests](flaky/diagnosing-flaky-tests.md) | A test fails intermittently with no code change: on retry, in CI only, or only when run with other tests; deciding policy for a newly identified flaky test (quarantine vs retry) | +| [diagnosing-flaky-tests](flaky/diagnosing-flaky-tests.md) | The unreliable thing is a test in your suite — it fails in CI but passes locally, passes on retry, or fails only when run with other tests — and you are hunting its test-level cause (order dependence, shared fixture state, time/timezone, async waits) or setting quarantine-vs-retry policy for it (the same intermittency also appears outside the suite, or the test-level causes are exhausted and you still cannot trigger it on demand → wiki/debugging/concurrency/intermittent-failures.md) | ## async diff --git a/wiki/testing/quality/checks-that-cannot-pass.md b/wiki/testing/quality/checks-that-cannot-pass.md index 2043351..bff6252 100644 --- a/wiki/testing/quality/checks-that-cannot-pass.md +++ b/wiki/testing/quality/checks-that-cannot-pass.md @@ -18,10 +18,9 @@ related: [testing-quality-tests-that-cannot-fail, testing-quality-minimum-case-s ## When this applies You are authoring a check that will decide pass/fail for work not yet done — -a grep/regex gate on an unwritten file or doc section, a lint/scan rule, a -schema assertion against an unbuilt endpoint, a plan's verification command — -and the only run you have observed is a failing one. Also applies when -reviewing a plan whose gates have never been seen to pass. +a grep/regex gate on an unwritten file or doc section (including an unwritten spec, RFC, or schema document), a lint/scan rule, a schema assertion against +an unbuilt endpoint, a plan's verification command — and the only run you have observed is a failing one. Also applies when reviewing a plan whose gates +have never been seen to pass. This owns unwritten-target gate patterns for spec, RFC, and schema documents too; wiki/qa/document-verification/spec-document-gates.md does not. ## Do this diff --git a/wiki/testing/quality/spec-artifact-checks.md b/wiki/testing/quality/spec-artifact-checks.md index fcb66c8..49b5140 100644 --- a/wiki/testing/quality/spec-artifact-checks.md +++ b/wiki/testing/quality/spec-artifact-checks.md @@ -17,10 +17,12 @@ related: [testing-quality-tests-that-cannot-fail, testing-quality-schema-additio ## When this applies -You are writing or reviewing an automated check that an artifact conforms to a -source of truth: a mapping table that must hold a row per rule, a key per field, -or a case per enum; ids that must resolve across documents; required sections in -an RFC. Includes doc-as-spec repos where the artifact is Markdown. +You are authoring or reviewing **the check itself** that an artifact conforms +to a source of truth: a mapping table that must hold a row per rule, a key per +field, or a case per enum; ids that must resolve across documents; required +sections in an RFC. Includes doc-as-spec repos where the artifact is Markdown. +Whether a passing check is enough to accept the deliverable is a release +decision owned by wiki/qa/document-verification/spec-document-gates.md. ## Do this