From ae333dad58b7f72879fc90dd7a93eaa7f14e67d6 Mon Sep 17 00:00:00 2001 From: dch0202 Date: Thu, 6 Aug 2026 13:04:21 +0900 Subject: [PATCH 01/61] knowledge: ingest 6 insights into existing pages, retire 5 duplicates --- .dev-loop/INGEST_REPORT.md | 149 +++++++++++++----- log.md | 1 + .../control-signals-vs-primary-artifacts.md | 5 +- .../worktree-isolated-workers.md | 4 +- wiki/infrastructure/index.md | 4 +- .../filesystems/permissions-and-exec-bits.md | 2 +- wiki/testing/index.md | 4 +- .../quality/guard-shape-vs-consequence.md | 8 +- .../testing/quality/tests-that-cannot-fail.md | 6 +- 9 files changed, 130 insertions(+), 53 deletions(-) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..331397b 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,116 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 11 insight(s) -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Drained 9 queue files (11 candidate rows) from `~/.dev-loop/queue/`. 5 candidates +were exact duplicates of knowledge already ingested by the #42–#46 consolidation +and were retired without edits; 6 were merged into existing pages (merge-before-create +— no new pages, no new categories). Open-PR dedup check: `gh pr list --label +dev-loop:knowledge --state open` returned zero open PRs, so nothing here overlaps +an in-flight review. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +**1. Bats assertions as `[[ ]]` mid-test are decoration on bash 3.2 → `verified`.** +Claim: under bats on macOS system bash (3.2.57), a false `[[ ]]` that is not the +test's last command does not fail the test; `[ ]` and `grep -qF` pipelines fail at +any position. Verified this session by fresh local reproduction (Bats 1.14.0, GNU +bash 3.2.57, arm64): 4-test probe file gave `ok` for mid-test false `[[ ]]`, +`not ok` for mid-test `[ ]`, mid-test grep, and last-line `[[ ]]`. Mechanism +isolated outside bats: `bash -ec '[[ … ]]; echo survived'` exits 0 while the `[ ]` +form aborts — a pre-4.0 errexit semantic, not a bats defect. Sources: +https://tiswww.case.edu/php/chet/bash/COMPAT (bash-4.0 changed `set -e` to exit on +compound-command failure), https://bats-core.readthedocs.io/en/stable/gotchas.html +and https://www.shellcheck.net/wiki/SC2314 (the documented same-shape gotcha for +negated `!` commands: they "can never fail when used in the middle of a test"). + +**2. Widened scan surface turning old tests red → triage as first true positive → `field-tested`.** +Claim: when a leak/masking detector gains a previously unscanned output channel and +an existing test reddens, diff what the new surface saw against what the fixture +declares, and fix the fixture's smuggled data rather than the detector. Evidence is +the session reproduction (linkly #43: `result.bindings` widening reddened +`test_when_guard_removed_diverges`; fixture carried an undeclared `password` key; +narrowing to declared fields → 1218 green). No independent external source claims +this exact triage order, so it stays field-tested; it is consistent with the page's +existing sourced principle (change-detector tests / "the guard reddens on a genuine +S-and-C artifact means the guard is working"). + +**3. Worker usage-limit pause looks alive → check pane tail for the limit marker → `field-tested`.** +Claim: simultaneous quiet workers with green liveness are a usage-limit pause; +find the `You've hit your session limit · resets HH:MM` marker, then after reset +send a resume prompt ordering state-recheck → remaining DoD → completion signal. +Evidence: 2026-08-06 run, three workers paused on one reset with identical markers; +the structured resume prompt recovered all three at their exact interruption point. +Vendor docs do not document the marker string, so no external citation is possible — +kept field-tested with the context described on the page. + +**4. Dispatch after `worker_done` needs a substrate-idle wait; a failed dispatch consumes the task → `field-tested`.** +Evidence: reproduced twice in the 2026-08-06 run (immediate dispatch → +`runtime_unavailable` + task consumed; dispatch after `orca terminal wait --for +tui-idle` succeeded first try). Orca is an internal tool; no external source exists. + +**5. Guardrail `worktree_escape` can escalate on read-only cross-worktree access → `field-tested`.** +Evidence: two read-only commands (`awk`/`grep` over an upstream FINDINGS file, +`git status`) each raised `ask` and stopped the watch with exit 5; both approved +after review. This **conflicts** with the existing page's 1.0.0 reproduction where +reads passed — handled as a condition-dependent (rule-version) edge case, not an +overwrite (see below). + +**6. `sh "$SCRIPT"` stub seam under EDR — second reproduction → enriches existing `field-tested` row.** +Directive already on the page from the #42–#43 reconciliation; this flush adds the +independent second reproduction (8 stall-handler stubs injected without any +`chmod`, bats suite 331/331 green under SentinelOne). + +**Dropped as exact duplicates (no edit, retired from queue):** gate quoting-form +parsing (already `command-text-inspected-before-execution` step 8 + Instead-of row ++ field context citing the same bats tests 12–13), stderr-warnings-with-exit-0 +capture incl. redirection order (already the whole of +`tool-diagnostics-without-a-failing-exit-code`), Homebrew clang `-isysroot +$(xcrun --show-sdk-path)` (already the whole of `compiler-sysroot-on-macos`, incl. +the same 69-failure repro), temp-artifact prefix counting + AST-rule enforcement + +RED-first guard (already `artifact-leakage-from-a-suite` steps 1/4/5), and +vacuously-green pre-implementation usage-error test proven by guard mutation +(already a `checks-that-cannot-pass` edge row describing the identical +unknown-subcommand/exit-1 case). ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Read before deciding: root `INDEX.md`; domain indexes for testing, platforms, +infrastructure; and the nine candidate-overlapping pages +(`command-text-inspected-before-execution`, +`tool-diagnostics-without-a-failing-exit-code`, `compiler-sysroot-on-macos`, +`artifact-leakage-from-a-suite`, `worktree-isolated-workers`, +`tests-that-cannot-fail`, `checks-that-cannot-pass`, `guard-shape-vs-consequence`, +`control-signals-vs-primary-artifacts`, `permissions-and-exec-bits`, +`destructive-operations-on-shared-daemons`). + +- **Merged, not created:** all 6 surviving insights landed as edge-case/table rows + and evidence on existing pages. No new page, no new category. +- **Conflict flagged and resolved as condition-dependent:** + `worktree-isolated-workers` states reads pass the guardrail (1.0.0 repro); the + new observation shows a rule version escalating on reads. Added as an edge row + ("guardrail rules differ by version — probe one read before fanning out") and + noted in the log entry; the Do-this table was not overwritten. +- **Duplicates:** the 5 dropped candidates matched existing pages + trigger-for-trigger and directive-for-directive (the #42–#46 reconciliation had + already ingested earlier harvests of the same sessions' insights). +- **Related-links:** no new cross-links needed — every edited page already links + the pages the new rows reference (e.g. `tests-that-cannot-fail` ↔ + `checks-that-cannot-pass`, `control-signals` ↔ `worktree-isolated-workers` via + the agent-orchestration index). ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| Insight | Target (existing page) | Why this page | +|---------|------------------------|---------------| +| bats/bash-3.2 mid-test `[[ ]]` | `testing/quality/tests-that-cannot-fail` — new never-fails-pattern row + sources | The page owns "assertions that cannot detect a defect"; this is a shell-level instance of that exact class | +| widened-scan-surface red | `testing/quality/guard-shape-vs-consequence` — When-this-applies clause + edge row + field evidence | The page owns guard-red triage; existing edge row already covered "guard reddens on genuine S-and-C"; this adds the widened-surface trigger and fixture-diff triage | +| usage-limit worker stall | `infrastructure/agent-orchestration/control-signals-vs-primary-artifacts` — edge row + field evidence | The page owns done/alive/stalled/dead verdicts; this is a new stalled-state cause with recovery protocol | +| dispatch-after-done timing | same page — edge row | "Done signal ≠ substrate release" is precisely the page's signal-vs-artifact distinction | +| read-only guardrail escalation | `infrastructure/agent-orchestration/worktree-isolated-workers` — edge row (condition-dependent conflict) | The page owns the guardrail's read/write asymmetry; the conflicting observation must sit next to the claim it qualifies | +| EDR stub second repro | `platforms/filesystems/permissions-and-exec-bits` — evidence sentence | Directive already lives there; only evidence strengthened | + +Domain hints from the queue were respected except where a page already owned the +case: the "testing"-hinted EDR-stub insight routes to platforms (the page that owns +exec-bit/EDR invocation style), and the "platforms"-hinted orchestration insights +route to infrastructure/agent-orchestration (dedicated category), consistent with +prior flushes. Index "load when" lines updated for the four pages whose routing +surface grew. diff --git a/log.md b/log.md index c930fc2..6b31cb1 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] -backup/…`) | The guardrail does not fire — the match requires a path separator after the main root — but the write is still outside the worktree; keep it out of the brief | +| A read-only command (`ls`, `grep`, `awk`, `git status`) naming the main checkout's or another worktree's absolute path raises an `ask` escalation anyway, halting the watch | Guardrail rules differ by version: a conservative rule treats any cross-worktree path reference in command text as a potential write, reads included. Extend the step-6 dry run with one read probe to learn which behavior you have; when reads escalate, budget the round-trip into the phase (read the escalation record → approve or deny → clear the escalation state → restart the watch) and state in the worker's first briefing which reads are pre-approved and that writes and system-temp use stay forbidden — this cuts repeat escalations for the same access | ## Instead of @@ -69,4 +70,5 @@ wait loop keeps escalating with no error from the task itself. - https://git-scm.com/docs/git-worktree — linked worktrees are separate checkouts sharing one repository; each has its own working directory - Field reproduction 2026-08-05 (groundwork guardrails 1.0.0 `hooks/bash-guard.sh`, `worktree_escape` rule, macOS): from a linked worktree, `cp ./a /b` and `echo z > /f` were both stopped; `cat /f`, `ls /.orchestration`, and `grep -n x /f` all passed. The rule matches an absolute main-root mention together with a write verb (`rm|mv|cp|tee|mkdir|touch|install|dd`) or a redirect to an absolute path +- Field evidence 2026-08-06 (dev-loop orchestrate, Wave 2 worker consuming an upstream worktree's FINDINGS file): a read-only `awk`/`grep` verification and a `git status` check each raised `worktree_escape` as `ask` and stopped the coordinator's watch with exit 5; both were confirmed read-only and approved. This rule version fired on reads, unlike the 1.0.0 reproduction above where bare `cat`/`ls`/`grep` passed — the read/write asymmetry in the Do-this table is version-dependent, so probe before fanning out - Field context: a parallel run stalled at the same phase for two workers whose brief's `` named a main-checkout absolute path; the coordinator's wait loop returned its escalation status repeatedly. Rewriting the contract to worktree-relative paths let the remaining workers record their plans locally diff --git a/wiki/infrastructure/index.md b/wiki/infrastructure/index.md index 36a0771..7483121 100644 --- a/wiki/infrastructure/index.md +++ b/wiki/infrastructure/index.md @@ -12,11 +12,11 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| -| [control-signals-vs-primary-artifacts](agent-orchestration/control-signals-vs-primary-artifacts.md) | An orchestrator is about to restart, discard, merge, or keep waiting on a worker based on a status file, a watcher's exit code, or a heartbeat; a monitor reports a worker dead while it is committing; a worker's status write produced no output and you must decide whether it landed; distinguishing alive-and-progressing from stalled from dead | +| [control-signals-vs-primary-artifacts](agent-orchestration/control-signals-vs-primary-artifacts.md) | An orchestrator is about to restart, discard, merge, or keep waiting on a worker based on a status file, a watcher's exit code, or a heartbeat; a monitor reports a worker dead while it is committing; a worker's status write produced no output and you must decide whether it landed; distinguishing alive-and-progressing from stalled from dead; several workers went quiet at once while every liveness check passes (usage-limit pause); a dispatch issued right after a worker's done signal fails runtime-unavailable | | [shared-run-state](agent-orchestration/shared-run-state.md) | Several agent/worker sessions coordinate through files in one repository (status directory, briefs, escalations, claim files); choosing the path layout for that state; starting an orchestration in a repo that may already have one running; a watcher woke on a task id it did not create; the default branch moved during a run | | [pane-delivery-confirmation](agent-orchestration/pane-delivery-confirmation.md) | An orchestrator drives another program through a terminal multiplexer (`tmux send-keys` + `capture-pane`) and must decide whether the input was consumed, retry, or escalate; a pane diff is being used as delivery evidence; the target echoes but never runs the input | | [session-completion-gates](agent-orchestration/session-completion-gates.md) | Writing a Stop/completion hook that blocks a worker session from ending while its phase is non-terminal; the gate fires on a worker that followed its own prompt; deciding the terminal phase set, the unknown-phase default, and how the gate bounds its own repetition | -| [worktree-isolated-workers](agent-orchestration/worktree-isolated-workers.md) | Authoring the brief/output contract for parallel workers each confined to its own git worktree; workers stall at the same phase with no task-level error; deciding where shared or produced artifacts live and which direction (read vs write) a worktree guardrail stops | +| [worktree-isolated-workers](agent-orchestration/worktree-isolated-workers.md) | Authoring the brief/output contract for parallel workers each confined to its own git worktree; workers stall at the same phase with no task-level error; deciding where shared or produced artifacts live and which direction (read vs write) a worktree guardrail stops; a guardrail escalates on read-only access to another worktree | ## ci-cd diff --git a/wiki/platforms/filesystems/permissions-and-exec-bits.md b/wiki/platforms/filesystems/permissions-and-exec-bits.md index 5774423..879e833 100644 --- a/wiki/platforms/filesystems/permissions-and-exec-bits.md +++ b/wiki/platforms/filesystems/permissions-and-exec-bits.md @@ -71,4 +71,4 @@ channels that preserve modes. - https://docs.docker.com/engine/containers/run/ — container default user is root (uid 0); `--user`/`-u` overrides with `uid:gid` - https://docs.docker.com/engine/storage/bind-mounts/ — bind-mount mechanics (host uid/gid visibility rows are field practice, not stated on this page) - https://nodejs.org/api/fs.html — `fs.writeFileSync(file, data, { mode })` sets the mode when the file is created -- Field measurement 2026-08-04 (Node v25.8.1, macOS, umask 022): `writeFileSync` with `mode: 0o755` produced mode `755`; a second write to the same path with `mode: 0o644` left it at `755`, confirming the creation-only semantics. The EDR row is operational practice, not vendor-documented — a 25-fixture suite was moved off `tmpdir()` + `chmod +x` to a gitignored build-output directory and ran green (59/59) with no alert +- Field measurement 2026-08-04 (Node v25.8.1, macOS, umask 022): `writeFileSync` with `mode: 0o755` produced mode `755`; a second write to the same path with `mode: 0o644` left it at `755`, confirming the creation-only semantics. The EDR row is operational practice, not vendor-documented — a 25-fixture suite was moved off `tmpdir()` + `chmod +x` to a gitignored build-output directory and ran green (59/59) with no alert. Second reproduction 2026-08-06 (bats, macOS + SentinelOne): eight stall-handler stub scripts injected through an env-var seam as plain non-executable files into a caller invoking `sh "$SCRIPT"` — full suite green (331/331) with no `chmod` anywhere in the tests diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 5f3ed78..7c4c010 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -24,12 +24,12 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [completion-predicates](quality/completion-predicates.md) | Writing the "everything is done" condition a monitor, wait loop, or polling script uses to decide background work has finished; a monitor declared completion far sooner than the work could have finished; matching a status marker that contains regex metacharacters, or passing that pattern through wrapper/ssh/send-keys quoting layers; deciding completion by counting rather than by absence | | [differential-run-agreement](quality/differential-run-agreement.md) | Two implementations of one spec were run on the same input and the harness reported agreement (EQUIVALENT / no diff / N-of-N checks pass) and you are about to cite it; the two sides model different amounts of state (one stubs out a repository, cache, clock, or session); choosing the input that forces an asymmetric dimension to decide the outcome | -| [guard-shape-vs-consequence](quality/guard-shape-vs-consequence.md) | A repo-wide guard asserting that no shipped artifact (example, config, migration, fixture) has a structural shape has gone red on a legitimate new artifact; authoring such a scanning guard; deciding between exempting an artifact, deleting the guard, and sharpening it; an existing guard has accumulated an exemption/allow list | +| [guard-shape-vs-consequence](quality/guard-shape-vs-consequence.md) | A repo-wide guard asserting that no shipped artifact (example, config, migration, fixture) has a structural shape has gone red on a legitimate new artifact; authoring such a scanning guard; deciding between exempting an artifact, deleting the guard, and sharpening it; an existing guard has accumulated an exemption/allow list; a widened scan surface (a detector now scans a new output channel) turned an existing test red | | [injected-clock-duration-assertions](quality/injected-clock-duration-assertions.md) | Asserting an elapsed duration between two readings of an injected/fake float clock (rate-limit interval, backoff, debounce, TTL); choosing that fake clock's start value; a single duration test fails on correct code by a margin in the far decimal places; choosing a comparison tolerance, or deciding between float seconds and integer nanoseconds | | [write-path-assertions](quality/write-path-assertions.md) | Writing an HTTP-level test for an endpoint that persists something (form submit, create/update, onboarding step) and choosing what to assert beyond the status code; such a test is green while the records are empty or defaulted; sending repeated form fields from a client (httpx/TestClient) and deciding the `data=` shape | | [minimum-case-set](quality/minimum-case-set.md) | Writing tests for a function/endpoint/change and choosing which cases to cover; reviewing whether coverage suffices; picking boundary values by input type; adding a regression test for a bug fix | | [behavior-not-implementation](quality/behavior-not-implementation.md) | Deciding what a test should assert; a behavior-preserving refactor broke tests; tempted to expose privates for testing; deciding whether a snapshot test is appropriate | -| [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect | +| [tests-that-cannot-fail](quality/tests-that-cannot-fail.md) | Reviewing tests that always pass; a bug shipped through an area the suite reported as covered; auditing a suspiciously green suite; judging whether an assertion, error-path test, or mock-based test can actually detect a defect; a shell-test (bats) assertion passes mid-test on code it should fail | | [checks-that-cannot-pass](quality/checks-that-cannot-pass.md) | Authoring a check whose target does not exist yet (grep/regex gate on an unwritten file or doc section, lint/scan rule, schema assertion on an unbuilt endpoint, a plan's verification command) and it has only ever been observed failing; reviewing a plan's gates before adopting them; separating "target missing" from "content missing" in a gate's exit status | | [spec-artifact-checks](quality/spec-artifact-checks.md) | Writing or reviewing an automated check that a mapping table covers every rule/field/enum case, or that ids resolve across documents; deciding whether a green check earned "verified" or only "present"; designing one negative control per check in a multi-check harness; parsing Markdown table rows programmatically in a doc-as-spec repo | | [schema-additions-under-a-golden-gate](quality/schema-additions-under-a-golden-gate.md) | Adding a node kind, variant, discriminator value, or field to a document format (IR, JSON Schema, spec artifact) whose only automated gate builds its negatives by mutating one committed golden example; the gate or the whole suite comes back green right after a schema change; deciding which negative each new schema keyword needs, and whether a green suite that never loads the schema is evidence at all | diff --git a/wiki/testing/quality/guard-shape-vs-consequence.md b/wiki/testing/quality/guard-shape-vs-consequence.md index e3875e8..8a32c44 100644 --- a/wiki/testing/quality/guard-shape-vs-consequence.md +++ b/wiki/testing/quality/guard-shape-vs-consequence.md @@ -7,7 +7,7 @@ confidence: field-tested sources: - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html - https://pitest.org/ -last_verified: 2026-08-04 +last_verified: 2026-08-06 related: [testing-quality-tests-that-cannot-fail, testing-quality-behavior-not-implementation, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, qa-process-regression-scope] --- @@ -19,7 +19,9 @@ A guard test scans every shipped artifact of a kind — example files, configs, migrations, fixtures, schema docs — and asserts that none of them has a structural shape S. A newly added, legitimate artifact now has S, so the guard is red and you are deciding what to do about it. Also applies when authoring -such a guard, before the first legitimate collision happens. +such a guard, before the first legitimate collision happens — and when you have +just widened a guard's scan surface (a leak/masking detector now scans a +previously unscanned output channel) and a previously green test reddens. Reviewing a guard that has never been red → [testing-quality-tests-that-cannot-fail]. @@ -66,6 +68,7 @@ Reviewing a guard that has never been red → [testing-quality-tests-that-cannot | Reusing the production derivation means a bug in that derivation silently greens the guard | Accept the coupling — it is what keeps the guard's meaning in sync — and keep the step-4 fixture as the independent control that would catch the greening | | Computing C over every artifact is expensive | Keep the shape check as a cheap prefilter and compute C only for the artifacts S matched; the assertion stays "S and C" | | The guard reddens on an artifact that has S and genuinely has C | This is the guard working — fix the artifact, not the guard | +| The guard's scan surface was just widened and an existing test reddens | Triage it as the widened guard's first true positive before touching guard or production code: diff what the new surface saw against what the fixture declares. Fixtures written before the surface existed often smuggle realistic data through paths the type system never classified (undeclared keys, copied payloads) — narrow the fixture to its declared fields, preserving the test's original intent. Re-greening old tests by loosening the detector reinstates the blind spot the widening closed | ## Instead of @@ -80,4 +83,5 @@ Reviewing a guard that has never been red → [testing-quality-tests-that-cannot - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — Alex Eagle, "Testing on the Toilet: Change-Detector Tests Considered Harmful" (2015-01-27): "Change-detector tests do not add clarity, and you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again." A shape-only guard that must be exempted for each new legitimate artifact is this failure mode at repo scope - https://pitest.org/ — "Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived" — the basis for step 4's required-red fixture +- Field evidence (linkly #43, 2026-08-06): adding `result.bindings` to a differential masking surface immediately reddened `test_when_guard_removed_diverges`; the repro showed the fixture's seeded row carrying an undeclared `password` key raw through the new channel while the entity declared only `id`/`email`/`token`. Narrowing the payload to declared fields preserved the test's guard-divergence intent and returned 1218 tests green with the widened detector intact - Field evidence (linkly #35, 2026-08-04): `test_no_shipped_example_has_a_guarded_repository_call` asserted that no shipped `.lnpl` example contained a repository call under a guard. `examples/checkout.lnpl` legitimately added a `create` under `when stock > 0` — the issue's own reproduction shape — turning the guard permanently red. Re-expressing it as "a guarded call that could actually fail", with the conflict/miss decision taken from the production `_lnpl_ops` derivation via `seeded_entities`/`repository_calls`, returned the suite to `Ran 518 tests / OK` while a fixture holding a guarded-and-can-fail create still drove the guard red diff --git a/wiki/testing/quality/tests-that-cannot-fail.md b/wiki/testing/quality/tests-that-cannot-fail.md index b0534af..c95fc99 100644 --- a/wiki/testing/quality/tests-that-cannot-fail.md +++ b/wiki/testing/quality/tests-that-cannot-fail.md @@ -15,7 +15,7 @@ sources: - https://www.gnu.org/software/sed/manual/html_node/Exit-status.html - https://git-scm.com/docs/git-checkout - https://git-scm.com/docs/git-restore -last_verified: 2026-08-05 +last_verified: 2026-08-06 related: [testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] --- @@ -73,6 +73,7 @@ suite reported as covered, or you are auditing a suspiciously green suite. | Testing the mock instead of the code (mock returns X, test asserts X came back) | Assert the unit's transformation of its inputs, not the pass-through; when no transformation exists at this layer, test the layer that has one ([testing-mocking-what-to-mock]) | | Copied test body with the name changed but identical inputs and expectation | Give each case distinct inputs and its own expectation; delete exact duplicates — a renamed copy re-proves the same fact and guards nothing new | | Assertion inherited from a shared base class, mixin, or parameterised harness, whose name announces the new subject's whole shape while its body pins the original narrow scope | Read the inherited body and list what it compares; add a subject-specific assertion for each part of the shape the name claims, then prove each one with its own mutation | +| Bats assertion written as `[[ … ]]` anywhere but the test's last command, when bats resolves to bash 3.2 (macOS system bash) — a false `[[ ]]` mid-test does not fail the test | Write bats assertions as simple commands — `[ … ]` or `printf '%s\n' "$output" \| grep -qF "expected"` — which fail at any position; before bash 4.0, `set -e` ignores a failing compound command, so a mid-test `[[ ]]` is decoration on that shell (same shape as the documented bats `!`-negation gotcha) | 6. **Coverage note:** a covered line is only an executed line. Use coverage to find untested code; it cannot certify tested behavior. The proof a test @@ -119,3 +120,6 @@ suite reported as covered, or you are auditing a suspiciously green suite. - https://man7.org/linux/man-pages/man2/execve.2.html — the shebang is honoured only on direct execution, not when a file is passed to an interpreter - https://www.gnu.org/software/sed/manual/html_node/Exit-status.html — a `sed` expression that matches nothing still exits 0 - https://git-scm.com/docs/git-checkout, https://git-scm.com/docs/git-restore — `checkout -- ` restores the index copy, discarding unstaged changes; measured 2026-08-05: with the fix unstaged the checkout removed fix and mutation together, and the lost import surfaced as `Ran 1042 … errors=1` where the intact tree ran 1098 +- https://tiswww.case.edu/php/chet/bash/COMPAT — bash-4.0 changed `set -e` handling so the shell exits when a compound command fails; bash-3.2 and earlier do not, which is what lets a false mid-test `[[ ]]` pass silently under bats on macOS system bash +- https://bats-core.readthedocs.io/en/stable/gotchas.html, https://www.shellcheck.net/wiki/SC2314 — the documented same-shape gotcha: bats commands whose failure is excluded from errexit (negated `!` commands) "can never fail when used in the middle of a test" +- Local reproduction 2026-08-06 (Bats 1.14.0, GNU bash 3.2.57, macOS arm64): a false `[[ "a" == *"zzz"* ]]` mid-test → `ok`; the same false comparison as `[ "a" = "zzz" ]` or piped `grep -qF` mid-test → `not ok`; the `[[ ]]` as the test's last line → `not ok`. Outside bats, `bash -ec '[[ … ]]; echo survived'` printed and exited 0 while the `[ ]` form aborted — bash-3.2 errexit semantics, not a bats defect From 3c85dc53fc333ebb2bd0957793d398b0ad5fbbb1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Thu, 6 Aug 2026 14:33:11 +0900 Subject: [PATCH 02/61] knowledge: ingest 3 verified insight(s) --- .dev-loop/INGEST_REPORT.md | 235 ++++++++++++++---- log.md | 1 + wiki/testing/index.md | 5 +- .../quality/behavior-not-implementation.md | 5 +- .../quality/differential-run-agreement.md | 3 +- .../quality/harness-reverse-controls.md | 2 +- wiki/testing/quality/minimum-case-set.md | 3 +- .../quality/stale-artifact-baselines.md | 97 ++++++++ .../testing/quality/tests-that-cannot-fail.md | 7 +- .../quality/unasserted-return-fields.md | 103 ++++++++ .../value-preserving-refactor-assertions.md | 96 +++++++ 11 files changed, 506 insertions(+), 51 deletions(-) create mode 100644 wiki/testing/quality/stale-artifact-baselines.md create mode 100644 wiki/testing/quality/unasserted-return-fields.md create mode 100644 wiki/testing/quality/value-preserving-refactor-assertions.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..ec68e03 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,202 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 3 insight(s) -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Drained the 3 pending rows in `~/.dev-loop/queue/` (session files `5d6b4056`, +`cea5f63a`, `ef805210`; the other 10 queue files were already empty). All three +carried `domain: testing`. None was dropped and none is left `unverified`. + +Every URL cited below and on the pages was opened in this session — no citation +is carried over on trust. Three new pages, four existing pages merged into, no +new category. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +**1. A value-preserving literal→SSOT refactor cannot be guarded by an output assertion → `verified`.** +Claim: when the config value renders byte-identical to the literal you removed, +every assertion on that output passes on the reverted-literal version too, so the +regression test must substitute the constant with a sentinel instead. +Verification — the mechanism is the documented *equivalent-mutant* condition: +[Stryker](https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/) +states there is "no definitive way for Stryker to find and ignore them" and that +"the only solution is by finding these by hand", and +[PIT](https://pitest.org/quickstart/basic_concepts/) defines the class as a mutant +whose result "behaves in exactly the same way as the original". A change that +provably cannot be distinguished by observing output is exactly what a +value-preserving refactor is, which is why the natural assertion is vacuous. +Research changed the candidate's directive in two places rather than transcribing +it: (a) the raw insight said `try/finally` to restore the constant — +[pytest](https://docs.pytest.org/en/stable/how-to/monkeypatch.html) documents that +monkeypatch modifications "will be undone after the requesting test function or +fixture has finished" and that `monkeypatch.context()` applies patches "only in a +specific scope", so the page directs the runner's scoped patcher (it covers exit +paths a `finally` block only covers when every path runs through it); (b) added the +patch-site precondition from +[unittest.mock](https://docs.python.org/3/library/unittest.mock.html) — "you must +ensure that you patch the name used by the system under test" — because a module +that copied the constant into a local at import time never reads the substitution +and the test would report green while measuring nothing. +Session evidence recorded on the page: `"DB ×%s" % E.DB_MULT` under the shipped +config renders `DB ×1.3`, i.e. the exact literal being removed (computed before the +test was written); only the sentinel form reddened on the restored-literal version. + +**2. A composite return's unread fields are unguarded, and cross-field invariants need their own assertion → `verified`.** +Claim: diff the returned field list against the fields assertions mention, confirm +each absence with a per-field mutation, then assert the relations binding the +fields (`lo ≤ point ≤ hi`, parts == total) across the input grid. +Verification — the "assert relations between outputs rather than each expected +output" step is metamorphic testing: +[arXiv:2211.12003](https://arxiv.org/abs/2211.12003) (Alzahrani, Spichkova, +Harland, *Application of property-based testing tools for metamorphic testing*) +states "The core concept in MT is metamorphic relations (MRs) which provide formal +specification of the system under test". The grid/generated-input half is +property-based testing as its maintainers define it +([hypothesis.works](https://hypothesis.works/articles/what-is-property-based-testing/)): +tests "such that, when these tests are fuzzed, failures in the test reveal problems +with the system under test that could not have been revealed by direct fuzzing". +Reading a survived per-field mutation as an unguarded field is PIT's own +attribution model (a kill belongs to the covering test; **No coverage** is a state +distinct from **Survived**) plus Stryker's +[RIP model](https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/). +Honest limit: I tried to source a definitional "property/invariant" quote from +Hypothesis's own `readthedocs` quickstart and the page does not contain one, so the +page cites the maintainers' article instead of a fabricated docs quote. +Session evidence on the page: 58 passing assertions over a `lo`/`sp`/`hi` return in +which `lo` and `hi` appeared in none of them; four formula mutations left all 58 +green while the no-op control survived (so the harness discriminated); the invariant +over the full discrete grid found 13 combinations with `sp > hi`. + +**3. A previously published artifact needs its generation dated and a row-level diff before it is a baseline → `verified`.** +Claim: matching aggregates do not establish matching rows — summation is not +injective, and a row the aggregate already excludes (rollup parent, cancelled +record) can differ freely while the total is unchanged. Date the artifact's +generation from schema fields, and rebuild the before side by reverting only the +change under measurement when the generation differs. +Verification — the generation/stamping directive is +[SLSA v1.0 provenance](https://slsa.dev/spec/v1.0/provenance): provenance is "the +verifiable information about software artifacts describing where, when and how +something was produced", recorded so consumers "can verify that the artifact was +built according to expectations", and a build records "the specific git commit that +the URI resolved to as a dependency" — the field a data artifact needs for the same +reason. The approved-snapshot edge case is +[Jest](https://jestjs.io/docs/snapshot-testing) verbatim: "we would need to fix the +bug before re-generating snapshots to avoid recording snapshots of the buggy +behavior", with snapshots committed and reviewed "as you would any other type of +test or code in your project". The golden-master naming caution comes from a +practitioner blog +([octopusinvitro](http://octopusinvitro.gitlab.io/blog/code-and-tech/approval-testing): +"'Golden Master' is not a great name for the snapshot, as it implies that it will +never change, and this is not always true") — flagged here as a blog, not a primary +spec, and used on the page only for that framing. The arithmetic core (equal sums +do not imply equal multisets) is definitional. +Session evidence on the page: `estimated.json` counted-SP total 211.48 matched HEAD +exactly, while the row diff found NEWRTB-2182 differing (분석조사/0.19 vs +인프라/0.56); the file predated a classifier lookbehind change, its `dead` field +being `None` (current code always populates it) dated the generation, and the issue +was a rollup parent excluded from the counted total — which is why the totals agreed +while a row did not. ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Pages read: testing-quality-tests-that-cannot-fail, testing-quality-minimum-case-set, testing-quality-harness-reverse-controls, testing-quality-differential-run-agreement, testing-quality-behavior-not-implementation, testing-quality-guard-shape-vs-consequence + +Read in full: the first five. `guard-shape-vs-consequence` was read as its PR #47 +hunks plus its index line (partial — stated rather than implied). Screened by their +`load when` lines in `wiki/testing/index.md` without opening the body, having found +no trigger overlap: completion-predicates, injected-clock-duration-assertions, +write-path-assertions, checks-that-cannot-pass, spec-artifact-checks, +schema-additions-under-a-golden-gate. I also read `wiki/qa/index.md` end to end to +test whether insight 3 belonged in `qa` rather than `testing` (see Routing). + +**Overlaps found, and merge-vs-create.** `tests-that-cannot-fail` owns the general +rule all three insights descend from ("a test proves something only if it can +fail"), but its trigger is retrospective — reviewing an always-green suite. All +three candidates trigger at authoring time on a specific shape. The house already +resolves this shape as *dedicated page + pointer row* (`write-path-assertions`, +`injected-clock-duration-assertions`, `schema-additions-under-a-golden-gate` are all +narrower cases of the same page, each linked from its Edge cases), so I followed +that precedent rather than growing a page already at 105 body lines. Merged instead +of duplicated: `tests-that-cannot-fail` +3 Edge rows, `minimum-case-set` +1 Edge row +(its "assert an observable outcome" is underspecified when the outcome is a +composite — the fields *and* their invariants), `differential-run-agreement` +1 Edge +row (its subject is two live runs; a stale artifact as one side is the adjacent +case). + +**Conflict found and resolved in place, not overwritten.** +`behavior-not-implementation` step 2 asserts the invariant "a refactor that +preserves behavior keeps every test green", which pulls directly against insight 1's +test — one that goes red when someone re-inlines a literal, a change with identical +output. Rather than let two pages disagree, I added a condition-dependent Edge row +to *both*: configurability is itself behavior, so the assertion runs through the +config seam an operator controls (not a private field), and re-inlining removes an +observable capability — therefore it is not behavior-preserving and step 2's +invariant is intact. That page's `load when` line and `last_verified` were updated +for the new use case; its step 2 text was left exactly as it was. + +**Related links added both ways** between the three new pages and +`tests-that-cannot-fail`, `harness-reverse-controls`, `minimum-case-set`, +`differential-run-agreement`, `behavior-not-implementation`, plus one-way references +to `checks-that-cannot-pass`, `write-path-assertions` and +`backend-common-change-impact-call-site-enumeration`. Verified programmatically: +every `related:` id and inline `[page-id]` reference **repo-wide** resolves to an +existing page, all three new pages are listed in `wiki/testing/index.md`, every +touched page's body is under the 120-line limit (max 105), the template's required +sections and frontmatter keys are present, and no banned vague qualifier survives in +the new pages (one "usually" was caught in a directive sentence and rewritten as the +condition that decides it). The checker was itself controlled: injecting a bogus +`related:` id and a banned qualifier made it report both, and it returned to PASS +after restore — so its green is discriminating rather than vacuous. + +## Open-PR check + +`gh pr list --repo choiyounggi/dev-loop --state open` → **#47** +(`knowledge/dch0202-20260806-130040`, label `dev-loop:knowledge`) and #48 +(`feat/tmux-coordinator-gaps`, code not wiki). I fetched #47's head and diffed +`origin/main...origin/knowledge/dch0202-20260806-130040 -- wiki/` in full. + +| Candidate | Verdict | Basis | +|---|---|---| +| 1. value-preserving refactor assertions | **new** | #47's only `tests-that-cannot-fail` change is a bats/bash-3.2 `[[ ]]` row plus three sources; no overlap of trigger or directive | +| 2. unasserted return fields | **new** | #47 touches no page about return-value coverage or invariants | +| 3. stale artifact baselines | **new** | #47 touches no page about baselines, snapshots, or impact measurement | + +Textual adjacency handled rather than ignored: #47 also edits +`wiki/testing/quality/tests-that-cannot-fail.md` and `wiki/testing/index.md`, the two +files this flush edits too. My edits were placed away from its hunks (Edge-cases +table and the `related:` line; three *appended* index rows below the last row it +touches), and where we both bump `last_verified` on `tests-that-cannot-fail` I set +the identical value it sets (`2026-08-06`, today) so the two branches converge +instead of conflicting. Whichever merges second should still be re-read at merge +time — the claim here is that no hunk overlaps, not that git is guaranteed silent. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +`INDEX.md` routes all three to **testing** ("writing or structuring automated +tests: level choice, cases/assertions…"), and within it to the existing +**quality** category. No new category: quality already holds the +"is this assertion capable of failing / may I cite this verdict" family +(`tests-that-cannot-fail`, `harness-reverse-controls`, `differential-run-agreement`, +`spec-artifact-checks`), which is precisely what all three are. + +| Insight | Target | Page | +|---|---|---| +| 1 | `testing/quality` (new page) | `value-preserving-refactor-assertions.md` — `testing-quality-value-preserving-refactor-assertions` | +| 2 | `testing/quality` (new page) | `unasserted-return-fields.md` — `testing-quality-unasserted-return-fields` | +| 3 | `testing/quality` (new page) | `stale-artifact-baselines.md` — `testing-quality-stale-artifact-baselines` | + +Rejected alternatives, with the reason each was rejected: +- **Insight 2 → merge into `minimum-case-set`**: that page selects cases over the + *input* space (normal/error/boundary per behavior); insight 2 is coverage of the + *output* shape and the relations inside it. Distinct axis, so it became a page and + the two are cross-linked — the same split the house already made for + `write-path-assertions`. +- **Insight 3 → `qa` domain**: I read `wiki/qa/index.md` in full. `qa` owns + release-process quality (gates, regression scope, deliverable documents) and + explicitly sends automated-test-code concerns to `testing/`. Insight 3 is about + whether a comparison is valid evidence, which is what `testing/quality` already + hosts for non-test-code artifacts (`differential-run-agreement`, + `harness-reverse-controls` govern reports and PR bodies, not only suites). Routed + to `testing/quality` for that precedent, with a `related:` link to + `qa-deliverables-generated-artifacts-as-deliverable-source`. +- **All three → Edge rows on `tests-that-cannot-fail`**: rejected on the page's own + constraint. It is at 105 body lines against a 120 limit and #47 adds to it; three + full cases would push it over and bury three distinct triggers inside a page whose + routing line is retrospective auditing. diff --git a/log.md b/log.md index c930fc2..ae1d80e 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] hi` diff --git a/wiki/testing/quality/value-preserving-refactor-assertions.md b/wiki/testing/quality/value-preserving-refactor-assertions.md new file mode 100644 index 0000000..614d5f0 --- /dev/null +++ b/wiki/testing/quality/value-preserving-refactor-assertions.md @@ -0,0 +1,96 @@ +--- +id: testing-quality-value-preserving-refactor-assertions +domain: testing +category: quality +applies_to: [general] +confidence: verified +sources: + - https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/ + - https://pitest.org/quickstart/basic_concepts/ + - https://docs.pytest.org/en/stable/how-to/monkeypatch.html + - https://docs.python.org/3/library/unittest.mock.html +last_verified: 2026-08-06 +related: [testing-quality-tests-that-cannot-fail, testing-quality-harness-reverse-controls, testing-quality-behavior-not-implementation, testing-quality-minimum-case-set, testing-quality-unasserted-return-fields, testing-quality-checks-that-cannot-pass, backend-common-change-impact-call-site-enumeration] +--- + +# Regression Tests for a Value-Preserving Refactor + +## When this applies + +You replaced a hardcoded literal with a read from config, a constants module, or +another single source of truth (`"DB ×1.3"` → `"DB ×%s" % E.DB_MULT`), the +current config holds the same value the literal did, and you are adding the test +that stops the literal from coming back. Also when reviewing such a test. + +## Do this + +1. **Render the output under the current config and compare it byte-for-byte + with the literal you removed, before writing any assertion.** When the two + strings are equal, every assertion on that output passes on both the + refactored code and the reverted-literal code: the refactor is a + semantics-preserving change, and no observation of the output can separate + the two versions. This is the equivalent-mutant condition — a change whose + result "behaves in exactly the same way as the original", which Stryker + documents as having "no definitive way … to find and ignore them". + +2. **Choose the assertion target from that comparison:** + +| Byte comparison of output vs. removed literal | Assert | +|---|---| +| Differs (the refactor changed the rendering) | The new output directly; the natural assertion already discriminates | +| Identical (the common case) | The *dependency*: substitute the source constant with a sentinel value no literal would produce, then require the output to carry the sentinel | +| Identical, and the constant is not reachable from the test process (remote config, build-time inlining) | Assert the read at the seam you control — a stub config object the code resolves at call time — and record the substitution as the test's subject | + +3. **Pick the sentinel so that the reverted code fails loudly.** Use a value + outside the plausible range and distinct in rendering (`DB_MULT = 9.9` when + production is `1.3`), then assert the rendered sentinel (`"DB ×9.9"`) is + present and the production literal (`"DB ×1.3"`) is absent. The absence half + is what fails on the reverted version; the presence half proves the + substitution reached the renderer. + +4. **Restore the constant with the framework's scoped patcher rather than a + hand-rolled `try/finally`.** pytest's `monkeypatch` states that "All + modifications will be undone after the requesting test function or fixture + has finished", and `monkeypatch.context()` applies a patch "only in a + specific scope" — both survive an assertion failure mid-test, which a + `finally` block only matches when every exit path runs through it. + +5. **Patch the name the renderer looks up, not the name where the value is + defined.** `patch()` "works by (temporarily) changing the object that a + *name* points to", so "you must ensure that you patch the name used by the + system under test". A module that copied the constant into a local at import + time does not read your patch — assert the substitution took effect (the + sentinel appears) before treating the test as proof. + +6. **Prove the test reddens on the pre-refactor code once.** Restore the literal + in a scratch copy, run the test, require red, then discard the copy + ([testing-quality-tests-that-cannot-fail] for the restore mechanics). + +## Edge cases + +| Case | Then | +|------|------| +| The constant is one of several the output renders | Substitute one constant per test so a red run names which read regressed; a single test substituting all of them cannot localize the reversion | +| The value is a float and the rendering rounds it | Pick a sentinel that survives the rounding (differs in a kept digit), and assert the rendered form rather than the raw value | +| The refactor moved the literal into the same module's own constant, not a shared SSOT | The substitution test still applies and is the only thing that detects re-inlining; note in the test name that the source is module-local | +| The config value legitimately equals a magic number the output already contains for another reason | Assert the sentinel-substituted output only, and drop the "production literal absent" half — it would match the unrelated occurrence and redden on correct code | +| Several call sites were meant to switch to the SSOT and you tested one | Enumerate the call sites and give each its own substitution assertion — a passing test on one site says nothing about the others ([backend-common-change-impact-call-site-enumeration]) | +| Asserting the substitution looks like testing an internal, against "a behavior-preserving refactor keeps every test green" | State the assertion as the behavior it is — the rendered output is a function of the configured value — and drive it through the seam an operator controls (the config object), not a private field. Re-inlining the literal removes that configurability, so it is not a behavior-preserving change and the invariant in [testing-quality-behavior-not-implementation] holds | +| The test cannot substitute anything because the value is inlined at build time | Record the reversion risk as uncovered in the test file, and move the guard to a static check that greps for the literal outside the SSOT ([testing-quality-checks-that-cannot-pass] for authoring that gate) | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Assert the rendered text (`assert "DB ×1.3" in out`) as the refactor's regression test | Substitute the constant with a sentinel and assert the output follows | The rendered text is identical before and after, so the assertion passes on the reverted literal too — it guards nothing it was written to guard | +| Read a green run plus coverage of the changed line as proof the SSOT read is enforced | Require a red run against a copy that has the literal restored | A value-preserving change leaves output-level observations unchanged by construction; only the substitution or a red control discriminates | +| Wrap the substitution in `try/finally` to restore the constant | Use the runner's scoped patch fixture (`monkeypatch`, `patch` as a context manager) | The fixture undoes the change on every exit path including collection errors, and needs no restore code to review | +| Skip the byte comparison and write whichever assertion looks natural | Compute the comparison first and let its result choose the assertion target | The comparison is what tells you whether the natural assertion can fail at all | + +## Sources + +- https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/ — an equivalent mutant leaves the output identical, "There is no definitive way for Stryker to find and ignore them", and "the only solution is by finding these by hand" — the reason a value-preserving refactor cannot be detected by observing output +- https://pitest.org/quickstart/basic_concepts/ — equivalent mutations: "The resulting mutant behaves in exactly the same way as the original", so no correct test can distinguish it +- https://docs.pytest.org/en/stable/how-to/monkeypatch.html — "All modifications will be undone after the requesting test function or fixture has finished"; `monkeypatch.context()` applies patches "only in a specific scope" +- https://docs.python.org/3/library/unittest.mock.html — "Where to patch": `patch()` "works by (temporarily) changing the object that a *name* points to with another one … you must ensure that you patch the name used by the system under test" +- Field reproduction 2026-08-05 (manday report renderer): the removed literal `DB ×1.3` and `"DB ×%s" % E.DB_MULT` under the shipped config rendered byte-identical, computed before writing the test — so `assert "DB ×1.3" in out` passed on both the SSOT version and the restored-literal version. Substituting `E.DB_MULT` with a sentinel and asserting the rendered sentinel was the only form that reddened on the literal version From 346dd95cdbba2f54da3fba3cbc94459a77a133ae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Thu, 6 Aug 2026 14:49:30 +0900 Subject: [PATCH 03/61] =?UTF-8?q?knowledge:=20fix=20cross-check=20findings?= =?UTF-8?q?=20=E2=80=94=20drop=20miscited=20RIP=20attribution,=20downgrade?= =?UTF-8?q?=202=20confidences=20to=20field-tested,=20correct=204=20mechani?= =?UTF-8?q?sm=20claims?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .dev-loop/INGEST_REPORT.md | 257 ++++++++++-------- log.md | 3 +- wiki/testing/index.md | 2 +- .../quality/behavior-not-implementation.md | 4 +- .../quality/harness-reverse-controls.md | 2 +- wiki/testing/quality/minimum-case-set.md | 2 +- .../quality/stale-artifact-baselines.md | 42 ++- .../quality/unasserted-return-fields.md | 12 +- .../value-preserving-refactor-assertions.md | 83 +++--- 9 files changed, 240 insertions(+), 167 deletions(-) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index ec68e03..ce27936 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -2,97 +2,120 @@ Drained the 3 pending rows in `~/.dev-loop/queue/` (session files `5d6b4056`, `cea5f63a`, `ef805210`; the other 10 queue files were already empty). All three -carried `domain: testing`. None was dropped and none is left `unverified`. +carried `domain: testing`. None was dropped. -Every URL cited below and on the pages was opened in this session — no citation -is carried over on trust. Three new pages, four existing pages merged into, no -new category. +Cross-Check: independent adversarial review (claude CLI headless, `--permission-mode plan`) returned BLOCK on the first commit with 17 findings — 2 critical, 9 major, 6 minor; all were accepted and fixed in the second commit, the most serious being a source miscitation inherited from an existing page and two `verified` confidences downgraded to `field-tested`. + +**Confidence after review:** `unasserted-return-fields` = `verified`; +`value-preserving-refactor-assertions` and `stale-artifact-baselines` = +`field-tested`. The first commit marked all three `verified`; the review showed +that for two of them the cited docs support only the background mechanism, not the +central directive, so they were downgraded rather than defended. ## Verified best-practice -**1. A value-preserving literal→SSOT refactor cannot be guarded by an output assertion → `verified`.** +**1. A value-preserving literal→SSOT refactor cannot be guarded by an assertion that holds the config fixed → `field-tested`.** Claim: when the config value renders byte-identical to the literal you removed, -every assertion on that output passes on the reverted-literal version too, so the -regression test must substitute the constant with a sentinel instead. -Verification — the mechanism is the documented *equivalent-mutant* condition: -[Stryker](https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/) -states there is "no definitive way for Stryker to find and ignore them" and that -"the only solution is by finding these by hand", and -[PIT](https://pitest.org/quickstart/basic_concepts/) defines the class as a mutant -whose result "behaves in exactly the same way as the original". A change that -provably cannot be distinguished by observing output is exactly what a -value-preserving refactor is, which is why the natural assertion is vacuous. -Research changed the candidate's directive in two places rather than transcribing -it: (a) the raw insight said `try/finally` to restore the constant — -[pytest](https://docs.pytest.org/en/stable/how-to/monkeypatch.html) documents that -monkeypatch modifications "will be undone after the requesting test function or -fixture has finished" and that `monkeypatch.context()` applies patches "only in a -specific scope", so the page directs the runner's scoped patcher (it covers exit -paths a `finally` block only covers when every path runs through it); (b) added the -patch-site precondition from -[unittest.mock](https://docs.python.org/3/library/unittest.mock.html) — "you must -ensure that you patch the name used by the system under test" — because a module -that copied the constant into a local at import time never reads the substitution -and the test would report green while measuring nothing. -Session evidence recorded on the page: `"DB ×%s" % E.DB_MULT` under the shipped -config renders `DB ×1.3`, i.e. the exact literal being removed (computed before the -test was written); only the sentinel form reddened on the restored-literal version. +every assertion that holds the config fixed passes on the reverted-literal version +too; the separating input is the config value itself, so the test must vary it +through a seam a caller or operator reaches. +Sources opened: [pytest monkeypatch](https://docs.pytest.org/en/stable/how-to/monkeypatch.html) +("All modifications will be undone after the requesting test function or fixture +has finished"; `monkeypatch.context()` applies patches "only in a specific scope"), +[unittest.mock](https://docs.python.org/3/library/unittest.mock.html) ("you must +ensure that you patch the name used by the system under test"), +[Stryker equivalent-mutants](https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/), +[PIT basic concepts](https://pitest.org/quickstart/basic_concepts/). +What the review changed here, and why it matters: the first commit called this "the +equivalent-mutant condition" and cited Stryker/PIT as its mechanism. That was +self-refuting — an equivalent mutant is one *no input* separates, while this page +then tells you to separate the versions by varying the config. The two are now +cited as a near analogy with the difference stated explicitly. Three further +corrections: the `try/finally` rationale was factually wrong (a `finally` block does +run on assertion failure, so the honest reasons are "no restore code to review" and +"covers a patch applied in a fixture whose test body never runs"); the +wrong-patch-site failure mode is a *misleading red*, not a silent green (the +sentinel assertion fails on correct code); and the presence/absence halves were +described backwards — presence is the discriminator, absence is a supplement. +Confidence is `field-tested` because no cited document supports the central +directive; the only evidence for it is one in-house reproduction (manday renderer, +2026-08-05: `"DB ×%s" % E.DB_MULT` rendered exactly the removed literal `DB ×1.3`, +computed before the test was written; only the sentinel form reddened on the +reverted version). **2. A composite return's unread fields are unguarded, and cross-field invariants need their own assertion → `verified`.** Claim: diff the returned field list against the fields assertions mention, confirm -each absence with a per-field mutation, then assert the relations binding the -fields (`lo ≤ point ≤ hi`, parts == total) across the input grid. -Verification — the "assert relations between outputs rather than each expected -output" step is metamorphic testing: -[arXiv:2211.12003](https://arxiv.org/abs/2211.12003) (Alzahrani, Spichkova, -Harland, *Application of property-based testing tools for metamorphic testing*) -states "The core concept in MT is metamorphic relations (MRs) which provide formal -specification of the system under test". The grid/generated-input half is -property-based testing as its maintainers define it -([hypothesis.works](https://hypothesis.works/articles/what-is-property-based-testing/)): -tests "such that, when these tests are fuzzed, failures in the test reveal problems -with the system under test that could not have been revealed by direct fuzzing". -Reading a survived per-field mutation as an unguarded field is PIT's own -attribution model (a kill belongs to the covering test; **No coverage** is a state -distinct from **Survived**) plus Stryker's -[RIP model](https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/). -Honest limit: I tried to source a definitional "property/invariant" quote from -Hypothesis's own `readthedocs` quickstart and the page does not contain one, so the -page cites the maintainers' article instead of a fabricated docs quote. -Session evidence on the page: 58 passing assertions over a `lo`/`sp`/`hi` return in -which `lo` and `hi` appeared in none of them; four formula mutations left all 58 -green while the no-op control survived (so the harness discriminated); the invariant -over the full discrete grid found 13 combinations with `sp > hi`. - -**3. A previously published artifact needs its generation dated and a row-level diff before it is a baseline → `verified`.** -Claim: matching aggregates do not establish matching rows — summation is not -injective, and a row the aggregate already excludes (rollup parent, cancelled -record) can differ freely while the total is unchanged. Date the artifact's -generation from schema fields, and rebuild the before side by reverting only the +each absence with a per-field mutation plus the harness no-op control, then assert +the relations binding the fields across the input grid. +Sources opened: [arXiv:2211.12003](https://arxiv.org/abs/2211.12003) (Alzahrani, +Spichkova, Harland, *Application of property-based testing tools for metamorphic +testing*) — "The core concept in MT is metamorphic relations (MRs) which provide +formal specification of the system under test"; +[hypothesis.works](https://hypothesis.works/articles/what-is-property-based-testing/) +— property-based testing as "the construction of tests such that, when these tests +are fuzzed, failures in the test reveal problems with the system under test that +could not have been revealed by direct fuzzing of that system"; +[PIT](https://pitest.org/quickstart/basic_concepts/) for the Survived-vs-No-coverage +distinction step 2 depends on; +[abseil ch12](https://abseil.io/resources/swe-book/html/ch12.html). +This page keeps `verified`: the two sources above back its central directive, and +its field reproduction carries measured numbers (58 passing assertions over a +`lo`/`sp`/`hi` return in which `lo` and `hi` appeared in none of them; four formula +mutations left all 58 green while the no-op control survived, so the harness +discriminated; the invariant over the full discrete grid found 13 combinations with +`sp > hi`). +Honest limit: Hypothesis's own `readthedocs` quickstart does not contain a +definitional "property/invariant" statement — I fetched it, found none, and cited +the maintainers' article rather than inventing a docs quote. + +**3. A previously published artifact needs its generation dated and a row-level diff before it is a baseline → `field-tested`.** +Claim: matching aggregates do not establish matching rows; date the artifact's +generation from its schema fields, and rebuild the before side by reverting only the change under measurement when the generation differs. -Verification — the generation/stamping directive is -[SLSA v1.0 provenance](https://slsa.dev/spec/v1.0/provenance): provenance is "the -verifiable information about software artifacts describing where, when and how -something was produced", recorded so consumers "can verify that the artifact was -built according to expectations", and a build records "the specific git commit that -the URI resolved to as a dependency" — the field a data artifact needs for the same -reason. The approved-snapshot edge case is -[Jest](https://jestjs.io/docs/snapshot-testing) verbatim: "we would need to fix the -bug before re-generating snapshots to avoid recording snapshots of the buggy -behavior", with snapshots committed and reviewed "as you would any other type of -test or code in your project". The golden-master naming caution comes from a -practitioner blog -([octopusinvitro](http://octopusinvitro.gitlab.io/blog/code-and-tech/approval-testing): -"'Golden Master' is not a great name for the snapshot, as it implies that it will -never change, and this is not always true") — flagged here as a blog, not a primary -spec, and used on the page only for that framing. The arithmetic core (equal sums -do not imply equal multisets) is definitional. -Session evidence on the page: `estimated.json` counted-SP total 211.48 matched HEAD -exactly, while the row diff found NEWRTB-2182 differing (분석조사/0.19 vs -인프라/0.56); the file predated a classifier lookbehind change, its `dead` field -being `None` (current code always populates it) dated the generation, and the issue -was a rollup parent excluded from the counted total — which is why the totals agreed -while a row did not. +Sources opened: [SLSA v1.0 provenance](https://slsa.dev/spec/v1.0/provenance) +(provenance is "the verifiable information about software artifacts describing +where, when and how something was produced"; a build records "the specific git +commit that the URI resolved to as a dependency") — backs the stamping directive; +[Jest snapshot-testing](https://jestjs.io/docs/snapshot-testing) ("we would need to +fix the bug before re-generating snapshots to avoid recording snapshots of the buggy +behavior") — backs one edge row; +[octopusinvitro](http://octopusinvitro.gitlab.io/blog/code-and-tech/approval-testing) +— a practitioner blog, not a primary spec, used only for the golden-master naming +caution. +What the review changed here: the first commit explained the failure with +non-injectivity of summation, which describes *cancelling* summands — but the field +evidence is a rollup parent **excluded from the total**, which was never a summand, +so no cancellation occurred. An engineer following the stated reason would hunt for +offsetting deltas, find none, and wrongly trust the total. The page now enumerates +three mechanisms and leads with exclusion (the one requiring no coincidence). The +generation-dating heuristic was also split: an *absent* field dates the file before +the field existed, a *present-but-empty* field dates it before the writer populated +it — two different conclusions the first commit conflated. +Confidence is `field-tested`: the three central directives (dating by schema field, +row-level full match, in-memory single-revert rebuild) cite nothing and rest on one +in-house reproduction (manday engine, 2026-08-05: counted-SP total 211.48 matched +HEAD exactly while the row diff found NEWRTB-2182 differing — 분석조사/0.19 vs +인프라/0.56 — with `dead: None` dating the file before the classifier's lookbehind +change, and the issue being a rollup parent excluded from the counted total). + +**Retraction.** The first commit's report asserted "Every URL cited below and on the +pages was opened in this session — no citation is carried over on trust." That was +false. Two of my new pages attributed the reachability–infection–propagation (RIP) +fault-detection model to +`https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/`. +I had not opened that URL; I copied the attribution from an existing wiki page. The +cross-check flagged it and I then fetched the page: it contains the mutant-state set +and the metric formulas and says nothing about reachability, infection, propagation, +or RIP. Both citations are removed from my pages (the Stryker URL is retained on +`unasserted-return-fields` for the mutant states it does document, and dropped +entirely from `stale-artifact-baselines`). + +**Pre-existing defect flagged, not silently edited.** The same miscitation exists in +`wiki/testing/quality/differential-run-agreement.md:92` ("reachability, infection, +and propagation (RIP) model for fault detection") and its directives at `:60` and +`:83` rest on it. That page is not mine to rewrite in this flush — the RIP model is +real but belongs to Ammann & Offutt, *Introduction to Software Testing*, not to +Stryker's docs. Flagged here and in `log.md` for the owner. ## Existing-layer check @@ -120,30 +143,43 @@ composite — the fields *and* their invariants), `differential-run-agreement` + row (its subject is two live runs; a stale artifact as one side is the adjacent case). -**Conflict found and resolved in place, not overwritten.** -`behavior-not-implementation` step 2 asserts the invariant "a refactor that -preserves behavior keeps every test green", which pulls directly against insight 1's -test — one that goes red when someone re-inlines a literal, a change with identical -output. Rather than let two pages disagree, I added a condition-dependent Edge row -to *both*: configurability is itself behavior, so the assertion runs through the -config seam an operator controls (not a private field), and re-inlining removes an -observable capability — therefore it is not behavior-preserving and step 2's -invariant is intact. That page's `load when` line and `last_verified` were updated -for the new use case; its step 2 text was left exactly as it was. - -**Related links added both ways** between the three new pages and -`tests-that-cannot-fail`, `harness-reverse-controls`, `minimum-case-set`, -`differential-run-agreement`, `behavior-not-implementation`, plus one-way references -to `checks-that-cannot-pass`, `write-path-assertions` and -`backend-common-change-impact-call-site-enumeration`. Verified programmatically: -every `related:` id and inline `[page-id]` reference **repo-wide** resolves to an -existing page, all three new pages are listed in `wiki/testing/index.md`, every -touched page's body is under the 120-line limit (max 105), the template's required -sections and frontmatter keys are present, and no banned vague qualifier survives in -the new pages (one "usually" was caught in a directive sentence and rewritten as the -condition that decides it). The checker was itself controlled: injecting a bogus -`related:` id and a banned qualifier made it report both, and it returned to PASS -after restore — so its green is discriminating rather than vacuous. +**Conflict found, and the resolution tightened after review.** +`behavior-not-implementation` step 2 asserts "a refactor that preserves behavior +keeps every test green", which pulls against insight 1's test — one that goes red +when someone re-inlines a literal, a change with identical output. I resolved it +with a condition-dependent Edge row on *both* pages rather than overwriting either. +The cross-check then attacked my first wording as a rationalization, correctly: it +justified the test by "configurability is behavior … through the seam an operator +controls", while another edge row extended the same test to a *module-local* +constant no operator can set — where re-inlining removes no observable capability +and the assertion becomes exactly what `behavior-not-implementation:41` orders +deleted. Generalized, that would let any implementation detail be relabelled a +capability and would quietly remove step 2's diagnostic force. Both rows now carry +the boundary: the value must be settable through an interface a caller or operator +reaches **without editing source** (config file, env var, DI parameter, CLI flag); +when it is not, step 2 stands unchanged and the guard moves to a static check. The +module-local edge row was rewritten to route there instead of asserting the +substitution test "still applies". That page's step 2 text is untouched, and its +`last_verified` was returned to `2026-07-10` — I added an edge row without +re-opening its three sources, so bumping the date would have claimed a verification +I did not do. + +**Related links** are now genuinely bidirectional for all five adjacent pages (the +first commit left two one-way: `minimum-case-set` → value-preserving and +`harness-reverse-controls` → stale-artifact-baselines; both back-links added). +One-way references remain, by design, to `checks-that-cannot-pass`, +`write-path-assertions` and `backend-common-change-impact-call-site-enumeration`. + +Verified programmatically: every `related:` id and inline `[page-id]` reference +**repo-wide** resolves; all three new pages are listed in `wiki/testing/index.md`; +every touched page's body is under the 120-line limit (max 105); template sections +and frontmatter keys present; no banned vague qualifier in a directive sentence (one +"usually" was caught and rewritten as the condition that decides it). The checker +was itself controlled — injecting a bogus `related:` id and a banned qualifier made +it report both, and it returned to PASS after restore. Note the limit that mattered: +that checker validates link *resolution*, never whether a source says what a page +claims, which is why the RIP miscitation above needed the independent review to +surface. ## Open-PR check @@ -200,3 +236,12 @@ Rejected alternatives, with the reason each was rejected: constraint. It is at 105 body lines against a 120 limit and #47 adds to it; three full cases would push it over and bury three distinct triggers inside a page whose routing line is retrospective auditing. + +## Reviewing this PR + +The two `field-tested` pages are the ones to read hardest: their central directives +rest on single in-house reproductions, described on each page, and are the parts no +external source backs. If you would rather not carry a page at that confidence, +`stale-artifact-baselines` is the most self-contained one to drop — the other two are +cross-linked from four existing pages. The `differential-run-agreement:92` RIP +miscitation is pre-existing and left for you to decide on. diff --git a/log.md b/log.md index ae1d80e..2864ab2 100644 --- a/log.md +++ b/log.md @@ -43,4 +43,5 @@ Append-only. Format: `## [YYYY-MM-DD] Date: Thu, 6 Aug 2026 17:32:25 +0900 Subject: [PATCH 04/61] knowledge: ingest 1 verified insight (per-return-site policy coverage); 1 folded to #47, 1 dropped --- .dev-loop/INGEST_REPORT.md | 229 ++++++++++++++---- log.md | 1 + .../change-impact/call-site-enumeration.md | 2 +- wiki/testing/index.md | 1 + .../quality/policy-at-several-return-sites.md | 84 +++++++ 5 files changed, 273 insertions(+), 44 deletions(-) create mode 100644 wiki/testing/quality/policy-at-several-return-sites.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..94b22c8 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,196 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 3 insight(s) -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: 3 pending candidates across 2 session files. **1 ingested here**, +1 folded onto open PR #47, 1 dropped as a pending duplicate. This PR therefore +carries a single new page. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### C1 — one test per success-return site of a handler that applies one policy (ingested) + +**Claim.** When a function applies the same policy at several of its own success +returns (a CLI handler computing an exit code and returning it from three +branches), enumerate those return sites first, write one test per site, and prove +each by reverting that specific call and requiring exactly its owning test to +redden. Testing only the most obvious branch leaves the others gated but +unverified — deleting the policy from them keeps the suite fully green. + +**Sources checked this run (fetched, quotes taken verbatim):** + +- `https://raw.githubusercontent.com/nedbat/coveragepy/master/doc/branch.rst` — + coverage.py's own statement-vs-branch example: "Statement coverage would show + all lines of the function as executed. But the `if` was never evaluated as + false, so line 2 never jumps to line 4"; branch coverage "will flag this code + as not fully covered because of the missing jump from line 2 to line 4. This is + known as a partial branch." This grounds the *why*: a green line-coverage + report is compatible with a return path never being taken. +- `https://pitest.org/quickstart/basic_concepts/` — "'Survived' means the + mutation was not detected by the covering test"; "'No coverage' is the same as + **Survived** except there were no tests that exercised the line of code where + the mutation was created." These are the two distinct verdicts a per-site + reversion separates, and they map to two different fixes (weak assertion vs. + missing case). +- `https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/` + — the Block Statement mutator "removes the content of every block statement", + i.e. deleting the statement at one site is a standard mutation operator, not an + ad-hoc edit invented for this procedure. +- Not cited: `https://pitest.org/quickstart/mutators/` was fetched to check + whether PIT documents per-return-statement mutant placement; it documents the + return-value mutator family (`EMPTY_RETURNS`, `FALSE_RETURNS`, `TRUE_RETURNS`, + `NULL_RETURNS`, `PRIMITIVE_RETURNS`) but does **not** state that placement is + per return statement, so no such claim was made. NIST SP 500-235 (basis-path + testing) was fetched for the "one test per independent path" grounding and + could not be read — the PDF did not extract on this machine (no + poppler/pdftotext/pypdf) — so it is **not** cited rather than cited unread. + +**Field verification (reproducible check).** linkly `impl/lnpl/cli.py`, `cmd_spec` +returns 0 from three branches (`-o` without `--run`, stdout dump, `--run` with all +cases passing). Reverting the two no-run `_strict_rc(...)` calls to bare +`return 0` left 10 tests passing and reddened exactly the 2 new path-specific +tests; before those tests existed the same reversion reddened nothing. The file +was restored from a pre-mutation copy with an identical `sha256`, and the suite +total rose 1272 → 1275 rather than dropping. + +**Confidence: verified** — documented mechanism (coverage.py, PIT, Stryker) plus a +reproducible per-site mutation with a stated negative result. + +### C2 — `worktree_escape` raising `ask` on read-only cross-worktree access (dropped) + +Claim: the guardrail fires `ask` even for reads naming another worktree's absolute +path, stopping the coordinator's watch with exit 5. This was **not re-verified or +re-ingested**: open PR #47 already carries it, in the same page, with the same +evidence. See Open-PR check. + +### C3 — dispatch-binding failure stages (folded onto PR #47) + +Claim: before binding a next-phase task to an existing terminal, confirm the +preview shows the idle prompt, and branch on the reported failure stage — +`runtime_unavailable` (busy substrate → wait for idle, create a fresh task, the +consumed one cannot be retried) vs `agent_unconfigured` (the agent process died +though the terminal survives → close the terminal and create a new worker-mode +agent) — and always pass `--terminal` together with `--worktree`. + +Confidence: **field-tested** — three measurements in one dev-loop/Orca run (i43 +busy-terminal dispatch consumed its task; i45 `agent_unconfigured` recovered by +closing the terminal and recreating the agent, that worker then finishing with +1284 tests passing; `terminal_worktree_mismatch` resolved by pairing +`--worktree`). No external documentation exists for these vendor-specific status +strings, so it is not claimed as `verified`. PR #47 already covers the +`runtime_unavailable` third of it; the remaining two stages were posted to that PR +rather than ingested here (see Open-PR check). ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed C1 via `INDEX.md` → testing ("writing or structuring automated tests: +level choice, cases/assertions…") → `wiki/testing/index.md`, quality category. +Routed C2/C3 via `INDEX.md` → infrastructure ("multi-agent orchestration (worker +liveness signals, shared run state, tmux pane delivery, completion gates, +worktree-isolated workers)") → `wiki/infrastructure/index.md`, agent-orchestration. + +Pages read: testing-quality-tests-that-cannot-fail, testing-quality-minimum-case-set, testing-quality-harness-reverse-controls, infrastructure-agent-orchestration-worktree-isolated-workers, backend-common-change-impact-call-site-enumeration + +Also read (index/plumbing, not pages): `INDEX.md`, `wiki/testing/index.md`, +`wiki/infrastructure/index.md`, `templates/page.md`, `log.md`. Plus the three +unmerged pages proposed by open PR #49 (`unasserted-return-fields`, +`value-preserving-refactor-assertions`, `stale-artifact-baselines`), fetched from +that PR's head — they are not in this checkout and so are deliberately absent from +the `Pages read:` line above. + +**Overlaps found, and why C1 is a new page rather than an append:** + +- `tests-that-cannot-fail` is the closest neighbour and already covers the *proof + technique* — "Seed one mutation per assertion, not one per file, and require + exactly the test that owns that assertion to redden", plus the restore-by-copy + and confirm-by-suite-total steps that C1's evidence exercised. C1's contribution + is a different axis: an enumeration of the *production code's* return sites + performed **before** any test exists. That page's granularity is per existing + assertion; a handler with three returns and one test has nothing at two of the + sites to be granular about. Appending would have buried a case-selection rule + inside a page whose load-when is "reviewing tests that always pass". +- `minimum-case-set` owns case selection but organises it by behaviour and input + boundary. Two return sites can share every input class and differ only in which + branch ran, so the exit-path axis is not derivable from its tables. The new page + states this relationship explicitly rather than restating the normal/error/ + boundary rule. +- `harness-reverse-controls` governs citing a harness's aggregate score; C1's step + 5 (cite sites, not percentages) points at it in spirit, but the trigger differs — + no harness is involved here, only hand-seeded reversions. +- `backend-common-change-impact-call-site-enumeration` is the same *method* + (enumerate the sites in the code before acting) applied to callers rather than + returns. Genuinely adjacent → linked both ways. + +**Conflicts flagged:** none. No existing page gives a directive C1 contradicts. + +**Related-links added:** forward links from the new page to +`testing-quality-tests-that-cannot-fail`, `testing-quality-minimum-case-set`, +`testing-quality-harness-reverse-controls`, +`backend-common-change-impact-call-site-enumeration`; back-link added on +`call-site-enumeration`. + +**Back-links deliberately deferred** on `tests-that-cannot-fail`, +`minimum-case-set` and `harness-reverse-controls`: both open PRs (#47 and #49) +already modify those files' `related:`/`last_verified` lines, and a third +same-line edit would produce a merge conflict for no routing benefit — the forward +links already work, and `wiki/testing/index.md` (which this PR does edit, one row) +is the routing surface. Add the three back-links in a follow-up once #47/#49 land. +`wiki/testing/index.md` may still conflict trivially with #49's quality-table +additions; the resolution is to keep all added rows. + +## Open-PR check + +Open `knowledge/*` heads listed with +`gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"`: + +| PR | Head | Touches | +|----|------|---------| +| #47 | `knowledge/dch0202-20260806-130040` | `infrastructure/agent-orchestration/{control-signals-vs-primary-artifacts,worktree-isolated-workers}`, `platforms/filesystems/permissions-and-exec-bits`, `testing/quality/{guard-shape-vs-consequence,tests-that-cannot-fail}`, both indexes | +| #49 | `knowledge/dch0202-rsquare-20260806-142309` (fork `dch0202-rsquare`) | `testing/quality/` — 3 new pages (`unasserted-return-fields`, `value-preserving-refactor-assertions`, `stale-artifact-baselines`) + 5 modified, `testing/index.md`, `log.md` | + +Diffed both against `origin/main` before ingesting (#47 via `git diff`; #49's head +lives on a fork, so its files were read through +`gh api …/contents/…?ref=refs/pull/49/head`). + +**Per-candidate verdicts:** + +- **C1 → new.** No overlap with either PR. #49's three new testing/quality pages + are about unread return *fields* of a composite value, value-preserving refactor + assertions, and stale "before" baselines — none concerns which return *site* + executed. #47's `tests-that-cannot-fail` edit is unrelated (its usage-limit and + dispatch-timing rows land in the infrastructure pages). +- **C2 → drop.** #47 already adds to `worktree-isolated-workers.md` the edge-case + row "A read-only command (`ls`, `grep`, `awk`, `git status`) naming the main + checkout's or another worktree's absolute path raises an `ask` escalation + anyway, halting the watch", with the version-dependence caveat, the + read-probe-before-fan-out instruction, the escalation round-trip budget, and the + "state pre-approved reads in the first briefing" advice — plus a source line + carrying the identical exit-5 field evidence. The candidate adds nothing. Worth + recording: that row **contradicts the 2026-08-05 reproduction already in the + same page's Do-this table** (bare `cat`/`ls`/`grep` passed under guardrails + 1.0.0). #47 resolves it as version-dependence rather than overwriting, which is + the right call, so no new conflict is raised here. +- **C3 → fold.** #47's new `control-signals-vs-primary-artifacts` row covers the + `runtime_unavailable` stage (done-signal ≠ substrate release, wait for + `tui-idle`, consumed task must be re-created). The `agent_unconfigured` stage, + the `--terminal`/`--worktree` pairing, and the idle-prompt precheck are not in + it. Those unique additions were posted to PR #47 as a suggested edge-case row + with their field evidence + (`https://github.com/choiyounggi/dev-loop/pull/47#issuecomment-5202279174`) + rather than pushed to that branch — #47 is already under review, and both halves + belong in one table row its author should merge deliberately. Not re-ingested + here; doing so would have meant a second edit to the same table and a certain + conflict. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| Insight | Target | Action | +|---------|--------|--------| +| C1 | `testing` / `quality` / `policy-at-several-return-sites` (id `testing-quality-policy-at-several-return-sites`) | **New page.** Registered in `wiki/testing/index.md` under quality; `log.md` appended; back-link added on `backend-common-change-impact-call-site-enumeration` | +| C2 | `infrastructure` / `agent-orchestration` / `worktree-isolated-workers` | **No change** — already carried by open PR #47 | +| C3 | `infrastructure` / `agent-orchestration` / `control-signals-vs-primary-artifacts` | **No change here** — unique half folded onto open PR #47 as a review comment | + +No new category was needed: `testing/quality` already holds case-selection and +test-provability pages, and C1 is both. No new domain, no `INDEX.md` change. + +**Queue disposition.** All 3 rows retired to `~/.dev-loop/queue/.processed.jsonl` +— the ingested one, the folded one, and the dropped duplicate — so none re-crosses +the auto-flush threshold. diff --git a/log.md b/log.md index c930fc2..81267ee 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] Date: Thu, 6 Aug 2026 18:36:32 +0900 Subject: [PATCH 05/61] =?UTF-8?q?knowledge:=20ingest=205=20verified=20insi?= =?UTF-8?q?ght(s)=20=E2=80=94=202=20new=20pages,=203=20merges?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Drains 8 queued candidates: 2 new pages (qa/deliverables/quantitative-claims-in-a-published-document, security/data/commit-identity-in-public-repos), 3 merges into existing pages (debugging hypothesis-testing, agent-orchestration worktree-isolated-workers and pane-delivery-confirmation), 3 drops (2 already merged on main verbatim, 1 out of wiki scope). One candidate's stated mechanism was corrected by reproduction against guardrails 1.2.0 before ingest rather than overwriting the conflicting page. --- .dev-loop/INGEST_REPORT.md | 259 +++++++++++++++--- INDEX.md | 4 +- log.md | 1 + .../methodology/hypothesis-testing.md | 6 +- .../pane-delivery-confirmation.md | 8 +- .../worktree-isolated-workers.md | 7 +- ...titative-claims-in-a-published-document.md | 76 +++++ wiki/qa/index.md | 1 + .../data/commit-identity-in-public-repos.md | 83 ++++++ wiki/security/index.md | 3 +- 10 files changed, 399 insertions(+), 49 deletions(-) create mode 100644 wiki/qa/deliverables/quantitative-claims-in-a-published-document.md create mode 100644 wiki/security/data/commit-identity-in-public-repos.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..363fce9 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,226 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 8 insight(s) -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: 8 pending rows across 4 session files. Outcome: **2 new pages, +3 merges into existing pages, 3 drops** (2 already-merged duplicates, 1 out of scope). ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### C1 — an explicit error refutes a silent-suppression hypothesis → `verified` +**Claim.** When one suspect is a mechanism designed to be invisible to the affected +party (silent moderation, shadowban, silent drop), receiving an explicit refusal +falsifies it; and a query returning zero results cannot separate "removed" from +"never accepted". +**Checked.** https://news.ycombinator.com/newsfaq.html — a killed post is `[dead]` and +"aren't displayed by default"; the FAQ documents **no** notification to the author, +which is the property the directive rests on. The generalized form (evidence must be +diagnostic between competing hypotheses; run a positive control to prove the query +path works) is already the frame of the page it was merged into, whose own sources are +debuggingbook and the Google SRE troubleshooting chapter. +**Confidence: verified** — sourced for the mechanism, plus a field application +(control query returning 8,462 hits proved the endpoint healthy, so the zero-hit +result carried no information). + +### C2 — outline instead of draft when a community forbids generated text → dropped +Substantiated (the platform's guideline is explicit, and the field evidence is a +comment marked `dead=true` while a sibling item from the same account survived, i.e. +per-item filtering rather than an account ban). Dropped for **scope**, not for truth — +see Routing decision. + +### C3 — recompute every quantitative claim before publishing a document → `verified` +**Claim.** Enumerate all counts in a hand-maintained document about to be published and +recompute each from source; treat a reported-wrong number as a sample, not the defect. +**Checked.** https://google.github.io/styleguide/docguide/best_practices.html — "Dead +docs are bad. They misinform, they slow down…"; "Change your documentation in the same +CL as the code change." https://www.writethedocs.org/guide/writing/docs-principles/ — +sources of truth must be "clearly defined and disjoint". +**Reproduced.** In the originating repo, 5 of 9 claims were stale (tests 386→1209, +mutations 53→77, node kinds 20→21, EBNF productions 51→58, "Twelve Accepted"→13); only +one had been noticed. Re-checked today: `grep -on '[0-9]\+ tests\|[0-9]\+ node +kinds\|[0-9]\+ productions' README.md` → `1209 / 21 / 58`, the corrected values. +A second observation (back-to-back suite runs on one commit reporting 1195 then 1209) +became the non-determinism edge case. +**Confidence: verified.** + +### C4 — enumerate call sites by callee, not parameter name → dropped (duplicate) +Already on `main` in `backend-common-change-impact-call-site-enumeration`, **including +the same field incident** the candidate cites: "a `rows_for()` test helper kept +reproducing a removed rule for five call sites while appearing as a single hit". No +unique content. Retired. + +### C5 — `${VAR:-default}` discards an empty off-switch → dropped (duplicate) +Already on `main` in `platforms-shells-env-var-off-switches` (and cross-linked from +`platforms-shells-unset-versus-empty-parameters`), including the same +`WATCH_TMUX=/nonexistent-tmux-disable` sentinel resolution. No unique content. Retired. + +### C6 — the worktree guardrail on read-only access → `verified`, and the candidate's stated mechanism was **corrected** before ingest +**Candidate claimed.** `worktree_escape` fires `ask` on read-only access. +**Existing page said** (verified against guardrails 1.0.0) reads pass. Rather than +overwrite a conflicting directive, I read the shipped rule and reproduced it. +**Reproduced** (groundwork guardrails 1.2.0 `hooks/bash-guard.sh`, macOS, run from a +linked worktree against a sibling worktree path): + +| Command | Verdict | +|---|---| +| `grep -n foo
//FINDINGS.md` | pass | +| `awk 'NR<5'
//FINDINGS.md` | pass | +| `cat
/README.md` | pass | +| `git -C
/ status --short` | pass | +| `mkdir -p .claude/tmp && grep -n foo
//FINDINGS.md` | **ask** | +| `cp
/README.md ./x` | **ask** | +| `grep -n foo
//FINDINGS.md > /tmp/out` | **ask** | + +**Mechanism** (read from the rule source): it fires when a main-root mention survives +the strip **and** `(rm|mv|cp|tee|mkdir|touch|install|dd)` or a redirect to an absolute +path matches *anywhere* in the command — the two tests are independent, not correlated. +And the strip removes only the worker's **own** worktree path, so a sibling worktree's +path survives. So the read never fires on its own; it fires when it shares a command +line with any write verb. The existing page's directive stands; this sharpens it. +**Confidence: verified** (reproduction + rule source). The candidate's own wording is +**not** what was ingested. + +### C7 — bind a dispatch only to an idle pane; branch recovery on the failure stage → `field-tested` +**Claim.** A worker's "done" is a report emitted from inside its turn, not its end, so +completion hooks keep the pane busy; binding into a busy pane spends the unit. An +occupied-runtime failure wants wait-and-rebind; a dead-agent failure wants a new agent. +**Checked.** Partially corroborated in the shipped source: +`skills/orchestrate/scripts/orca-worker-start.sh` carries "rejects the pair with +`terminal_worktree_mismatch` (verified live)", and `skills/orchestrate/SKILL.md` +documents that a failed unit is replaced with a fresh one rather than retried in place +(naming `runtime_unavailable`). No external/official source exists for a tool-internal +lifecycle. +**Confidence: field-tested** — three same-session observations plus the two +shipped-source corroborations. Written into the page without confidence inflation; the +page's own `confidence: verified` is carried by its termios/tmux sources and the +2026-08-05 reproduction, and the new rows are attributed to a dated field-observation +source line. + +### C8 — the author identity a commit publishes → `verified` +**Claim.** Compare `git config user.email` against `git log -1 --format=%ae` before +committing to public history; override per commit with `git -c`, never globally. +**Checked.** https://git-scm.com/docs/git-commit — "the information is taken from the +configuration items `user.name` and `user.email`, or, if not present, the environment +variable `EMAIL`, or, if that is not set, system user name and the hostname used for +outgoing mail", with `GIT_AUTHOR_*`/`GIT_COMMITTER_*` taking precedence. +https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address +— "GitHub uses the email address set in your local Git configuration to associate +commits pushed from the command line with your account"; a per-repository address "will +override your global Git configuration settings in this one repository, but will not +affect any other repositories." +**Note on one sub-claim.** The candidate asserted a pushed email is "effectively +unrecallable". GitHub's page does not state that, so the page does not claim it as +sourced — it is expressed as an edge-case action (treat as disclosed; rewrite and +force-push before the first fork or archive) rather than as a documented fact. +**Reproduced today, in this very flush:** the checkout's `git config user.email` was an +employer address while `git log -1 --format=%ae` showed a GitHub no-reply address — the +mismatch is silent, exactly as the page says. This PR is committed under the per-repo +identity, not the ambient one. +**Confidence: verified.** ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed via `INDEX.md` → domain `index.md` → every page whose "load when" overlapped. + +Pages read: backend-common-change-impact-call-site-enumeration, platforms-shells-env-var-off-switches, platforms-shells-unset-versus-empty-parameters, infrastructure-agent-orchestration-worktree-isolated-workers, infrastructure-agent-orchestration-pane-delivery-confirmation, debugging-methodology-hypothesis-testing, qa-deliverables-generated-artifacts-as-deliverable-source, security-secrets-secrets-in-code + +Also read at index level (not opened as pages): the `agent-orchestration`, +`document-verification`, `process`, `data` and `secrets` sections of the +infrastructure / qa / security indexes, plus `AGENTS.md` and `templates/page.md`. + +**Overlaps found and what was done:** + +| Candidate | Overlap | Action | +|---|---|---| +| C4 | `backend-common-change-impact-call-site-enumeration` covers it, same incident text | **Drop** — nothing unique | +| C5 | `platforms-shells-env-var-off-switches` covers it, same sentinel | **Drop** — nothing unique | +| C1 | `debugging-methodology-hypothesis-testing` owns "several suspects compete, pick what to test next" | **Merge** — 2 edge-case rows + 2 source lines | +| C6 | `infrastructure-agent-orchestration-worktree-isolated-workers` owns guardrail direction | **Merge** — 4 edge cases + a 1.2.0 reproduction source | +| C7 | `infrastructure-agent-orchestration-pane-delivery-confirmation` owns pane consumption | **Merge** — 4 edge cases + 1 Instead-of + 1 source | +| C3 | `qa-deliverables-generated-artifacts-as-deliverable-source` is adjacent (choose-your-source) | **New page** — different decision (verify-before-publish); cross-linked via `related:` | +| C8 | `security-secrets-secrets-in-code` (leak response) and `security-data-pii-handling` (PII) are adjacent | **New page** — an author email is neither a rotatable secret nor feature-held PII; `related:` to both | + +**Conflict flagged, not overwritten.** C6 as harvested contradicts the merged page +("reads fire" vs "reads pass"). I reproduced the rule instead of trusting either: the +page was right, the candidate's mechanism was wrong, and the real finding (independent +AND of write-verb and main-root mention; sibling worktrees not stripped) was written as +a refinement. The 1.0.0 source line is kept alongside the new 1.2.0 one. + +**Related-links added:** the two new pages link to +`qa-deliverables-generated-artifacts-as-deliverable-source`, +`qa-document-verification-spec-document-gates`, `testing-quality-spec-artifact-checks`, +`security-secrets-secrets-in-code`, `security-data-pii-handling`. All five ids were +resolved against `wiki/` before commit. + +## Open-PR check + +**The `gh` API was unavailable for this flush** — the stored token is expired +(`gh auth status`: "The token in default is invalid", `gh api user` → 401), so +`gh pr list --search "head:knowledge/"` could not be run. SSH to the remote works, so +the check was done over remote refs instead: `git ls-remote --heads origin +'refs/heads/knowledge/*'` → **24 heads**, of which **23 still carry wiki diffs against +`origin/main`** (`git diff --name-only origin/main... -- wiki/`) and are therefore +in flight. This substitutes for open/closed state; a closed-but-undeleted branch would +be counted as in-flight here, which errs toward more dedup, not less. + +**Per-candidate verdicts:** + +| Candidate | Overlapping in-flight heads | Verdict | +|---|---|---| +| C4 call-site enumeration | 11 heads carry a call-site page at 8 different paths (`backend/common/change-impact/`, `backend/common/refactoring/`, `qa/process/`, `testing/quality/`, `testing/migration/`) — `dch0202-20260804-141726`, `-151729`, `-174423`, `-191843`, `20260805-095530`, `-105839`, `-144711`, `-155452`, `-164544`, `-175537`, `20260806-172420` | **drop** — pending duplicate; also already merged on `main` | +| C5 env-var off switch | 4 heads (`20260805-095530`, `-144711`, `-164544`, `-175537`) at 4 path variants | **drop** — pending duplicate; also already merged on `main` | +| C6 worktree guardrail | 2 heads touch `agent-orchestration/worktree-isolated-workers.md` (`20260805-164544`, `20260806-130040`) | **new** — diffed both; neither carries the read-vs-write-verb AND mechanism or sibling-worktree stripping. Ingested here | +| C7 pane binding | 1 head touches `pane-delivery-confirmation.md` (`20260805-164544`); `20260805-130054` and `-155452` carry adjacent tmux pages | **new** — none carries the pre-bind idle gate or the failure-stage branch | +| C3 published-document counts | none — no in-flight head touches `qa/deliverables/` beyond the already-merged page | **new** | +| C8 commit identity | none — closest is `20260806-115351` (`infrastructure/ci-cd/secrets-handling.md`, credential channels), a different concern | **new** | +| C1 hypothesis diagnosticity | none — no in-flight head touches `debugging/methodology/` | **new** | +| C2 generated-text policy | none | **drop** (scope, see below) | + +**Backlog note for the owner:** 23 unmerged `knowledge/*` heads is the same pile-up +pattern recorded in #39. The call-site-enumeration insight alone exists at 8 paths +across 11 branches while the canonical page is already on `main`; the env-var off-switch +at 4 paths across 4 branches, likewise already on `main`. Those branches are largely +re-litigating merged content. This PR deliberately adds nothing to either cluster. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| Insight | Target | New category? | +|---|---|---| +| C1 | `debugging` / `methodology` / `hypothesis-testing` (merge: 2 edge cases + 2 sources) | no | +| C2 | **none — dropped as out of scope** | — | +| C3 | `qa` / `deliverables` / **new page** `quantitative-claims-in-a-published-document` | no — `deliverables` fits | +| C4 | none — dropped (merged duplicate) | — | +| C5 | none — dropped (merged duplicate) | — | +| C6 | `infrastructure` / `agent-orchestration` / `worktree-isolated-workers` (merge) | no | +| C7 | `infrastructure` / `agent-orchestration` / `pane-delivery-confirmation` (merge) | no | +| C8 | `security` / `data` / **new page** `commit-identity-in-public-repos` | no | + +**Why C3 is a new page rather than a merge.** `generated-artifacts-as-deliverable-source` +answers *where a deliverable's body should come from*; C3 answers *what to verify in a +hand-maintained document before it goes out*, and its trigger (about to publish; one +number reported wrong) does not match that page's "When this applies". One case per +page (`AGENTS.md` rule 1). Cross-linked rather than folded. + +**Why C8 is a new page rather than a merge.** `secrets-in-code` is scoped to values that +can be rotated after a leak; `pii-handling` is scoped to personal data a *feature* +stores. A commit author address is neither — it is published by the VCS itself, cannot +be rotated, and the decision is which identity to bind at commit time. Placed under the +existing `data` category (personal data being published) with `related:` to both. + +**Why C2 was dropped.** Its directive — when a community forbids generated text, give +the user the argument structure and let them write the prose — is true and evidenced, +but this wiki's ten domains cover software-engineering practice, and none covers +content-provenance policy for community posting. Inventing an eleventh domain for one +insight over-fits the routing layer, and filing it under `platforms` would contradict +that domain's stated scope ("OS-level differences that break code and scripts"). It +belongs in the operator's own working-practice notes (`HABITS.md`), not the shared +wiki. Retired from the queue rather than left pending, so the auto-flush threshold does +not re-raise a candidate that can never be promoted. + +**No new categories were created.** + +## Indexes and log + +`wiki/qa/index.md` and `wiki/security/index.md` each gained one "load when" row; both +domains' header lines and the matching `INDEX.md` rows were widened to name the new +concerns. `log.md` has the appended `## [2026-08-06] ingest` entry. All five touched +pages are ≤ 120 body lines (76 / 83 / 77 / 82 / 73). Banned vague qualifiers: none in +the new pages. diff --git a/INDEX.md b/INDEX.md index cb6cd09..f725937 100644 --- a/INDEX.md +++ b/INDEX.md @@ -14,9 +14,9 @@ follow the cross-pointers in their index or take the next matching seeded domain | [frontend](wiki/frontend/index.md) | **seeded** | Web UI code: state placement, rendering performance, in-UI data fetching (races, infinite scroll), auth token handling, forms, XSS-safe output, accessibility | | [infrastructure](wiki/infrastructure/index.md) | **seeded** | CI/CD pipelines, secrets in build/deploy, container image builds, rollout/rollback strategy, observability (logs/metrics/alerting), per-environment/path-valued config, multi-agent orchestration (worker liveness signals, shared run state, tmux pane delivery, completion gates, worktree-isolated workers) | | [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests (release-process quality → qa) | -| [qa](wiki/qa/index.md) | **seeded** | Release-quality process: release gates, regression scoping, bug reports, severity/priority triage, exploratory testing (guarded-path coverage, override matrices), scope-purity gates, sourcing deliverable documents from generated artifacts, automated verification of document deliverables (spec/RFC gates) (writing automated test code → testing) | +| [qa](wiki/qa/index.md) | **seeded** | Release-quality process: release gates, regression scoping, bug reports, severity/priority triage, exploratory testing (guarded-path coverage, override matrices), scope-purity gates, sourcing deliverable documents from generated artifacts, verifying the quantitative claims in a document before publishing it, automated verification of document deliverables (spec/RFC gates) (writing automated test code → testing) | | [debugging](wiki/debugging/index.md) | **seeded** | Diagnosing a failure — finding what is wrong and why: reproducing, bisection, hypothesis testing, traces/logs, intermittent failures (fixing the diagnosed fault → its owning domain) | -| [security](wiki/security/index.md) | **seeded** | Trust-boundary decisions: input validation, session-vs-token auth choice, per-resource authorization (IDOR), secrets hygiene, dependency trust, PII handling (XSS rendering → frontend; CI secrets → infrastructure; JWT implementation → backend/frontend auth) | +| [security](wiki/security/index.md) | **seeded** | Trust-boundary decisions: input validation, session-vs-token auth choice, per-resource authorization (IDOR), secrets hygiene, dependency trust, PII handling, the author identity a commit publishes to a public repository (XSS rendering → frontend; CI secrets → infrastructure; JWT implementation → backend/frontend auth) | | [platforms](wiki/platforms/index.md) | **seeded** | OS-level differences breaking code across macOS/Linux/Windows: shell portability, BSD-vs-GNU CLI, filesystem case/line endings, Unicode normalization in text/file-name matching, commands inspected before execution, background services/cron, invoking prompt-capable CLIs non-interactively, toolchain version pinning | | [mobile](wiki/mobile/index.md) | **seeded** | App-side iOS/Android/cross-platform: process death/state survival, offline-first sync, mobile-network calls, store rollout/hotfix strategy, startup time | diff --git a/log.md b/log.md index c930fc2..271ad1c 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] -backup/…`) | The guardrail does not fire — the match requires a path separator after the main root — but the write is still outside the worktree; keep it out of the brief | +| A worker reads another **worker's** worktree (`//FINDINGS.md`) — a later wave consuming an earlier wave's output | The rule strips only the worker's *own* worktree path before looking for a main-root mention, so a sibling worktree's path stays in the string and the read is one write verb away from firing. Give the consumer a copy in its own worktree, or keep the cross-worktree read on a command line of its own | +| A read of a main-root path shares a command line with any write verb (`mkdir -p .claude/tmp && grep … /x`) | It fires `ask`, even though the write targets a worktree-relative path — the two conditions are matched independently over the whole command string, not correlated with each other. Split the write and the read into separate commands | +| A read of a main-root path is redirected to an absolute path (`grep … /x > /tmp/out`) | It fires — the redirect-to-absolute branch matches regardless of what is being read. Redirect to a worktree-relative path | +| The escalation is read-only in substance and the coordinator must clear it | Budget the round trip (read the recorded escalation → approve → clear `escalations/` → restart the watcher) and state in the worker's first brief that reads are approved and only writes outside the worktree are refused, so the same class does not re-escalate | ## Instead of @@ -69,4 +73,5 @@ wait loop keeps escalating with no error from the task itself. - https://git-scm.com/docs/git-worktree — linked worktrees are separate checkouts sharing one repository; each has its own working directory - Field reproduction 2026-08-05 (groundwork guardrails 1.0.0 `hooks/bash-guard.sh`, `worktree_escape` rule, macOS): from a linked worktree, `cp ./a /b` and `echo z > /f` were both stopped; `cat /f`, `ls /.orchestration`, and `grep -n x /f` all passed. The rule matches an absolute main-root mention together with a write verb (`rm|mv|cp|tee|mkdir|touch|install|dd`) or a redirect to an absolute path +- Local reproduction 2026-08-06 (groundwork guardrails 1.2.0 `hooks/bash-guard.sh`, `worktree_escape`, macOS), run from a linked worktree against a sibling worktree's path: `grep -n foo //FINDINGS.md`, `awk 'NR<5' //FINDINGS.md`, `cat /README.md` and `git -C / status --short` all passed; `mkdir -p .claude/tmp && grep -n foo //FINDINGS.md`, `cp /README.md ./x` and `grep -n foo //FINDINGS.md > /tmp/out` each returned `ask`. Reading the rule confirms why: it fires when a main-root mention survives the strip **and** `(rm|mv|cp|tee|mkdir|touch|install|dd)` or a redirect to an absolute path matches anywhere in the command — the two tests are independent - Field context: a parallel run stalled at the same phase for two workers whose brief's `` named a main-checkout absolute path; the coordinator's wait loop returned its escalation status repeatedly. Rewriting the contract to worktree-relative paths let the remaining workers record their plans locally diff --git a/wiki/qa/deliverables/quantitative-claims-in-a-published-document.md b/wiki/qa/deliverables/quantitative-claims-in-a-published-document.md new file mode 100644 index 0000000..0973ed1 --- /dev/null +++ b/wiki/qa/deliverables/quantitative-claims-in-a-published-document.md @@ -0,0 +1,76 @@ +--- +id: qa-deliverables-quantitative-claims-in-a-published-document +domain: qa +category: deliverables +applies_to: [general] +confidence: verified +sources: + - https://google.github.io/styleguide/docguide/best_practices.html + - https://www.writethedocs.org/guide/writing/docs-principles/ +last_verified: 2026-08-06 +related: + [qa-deliverables-generated-artifacts-as-deliverable-source, qa-document-verification-spec-document-gates, testing-quality-spec-artifact-checks] +--- + +# Numbers in a Document About to Be Published Outside the Team + +## When this applies + +You are about to publish or hand out a hand-maintained document that describes +the repository — README, landing page, launch post, architecture overview — and +it states counts: tests, rules, supported types, grammar productions, endpoints, +documents in a given state. Also when someone reports that one number in such a +document is wrong. + +## Do this + +1. **Enumerate every quantitative claim in the document first, then verify.** + Extract them as a list before checking any of them + (`grep -on '[0-9][0-9,]*' README.md` as the starting sweep), so the work is + bounded by the document and not by which number was noticed. +2. **Recompute each one from the source by a command, and record the command + next to the number.** A claim is verified when a reader can re-run the check: + +| Claim shape | Recompute with | +|-------------|----------------| +| Test count | The suite's own summary line from a full run, not a per-file sum | +| Rules / cases / mutations in a harness | A count over the harness's own table or registry, not the prose that describes it | +| Kinds, variants, enum members | The schema or type definition — count the branches (`anyOf`, enum members), because the implementation and the spec can disagree | +| Grammar productions | A count over the grammar file's production separators | +| Documents in a state ("twelve Accepted RFCs") | The linter or index that already computes it | + +3. **Treat a reported wrong number as a sample, not the defect.** When one + claim is stale, verify the rest in the same pass — the mechanism that let one + drift (no test asserts it) applies to all of them equally. +4. **Fix the source of drift where the number is load-bearing**: assert it in + the suite, or generate the line from the artifact + ([qa-deliverables-generated-artifacts-as-deliverable-source]). +5. **Record the date and commit the numbers were computed at** in the document, + so the next reader can tell freshness from a glance instead of re-deriving. + +## Edge cases + +| Case | Then | +|------|------| +| The count is non-deterministic across runs (parallel collection, generated cases) | State it as the run's reported value with the command, or state a floor ("1200+"); a bare exact number that moves between runs cannot be verified by anyone | +| The spec and the implementation give different counts for the same concept | Publish the one matching the artifact the sentence is about, and name which — a README sentence about the schema counts schema branches, not implementation branches | +| The number appears in more than one document (README, its translation, a landing page) | Fix every copy in the same change; a translated README drifts independently and is the copy most often missed | +| Verifying a claim requires a build that does not run on this machine | Mark the claim with the environment it was measured in rather than dropping it, and say what is untested here | +| The document is generated | Re-run the generator instead of editing the number ([qa-deliverables-generated-artifacts-as-deliverable-source]) | +| The number is decorative ("dozens of tests") | Leave it; the rule covers claims a reader could check and find false | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Correct the one stale number that was reported and publish | Enumerate and recompute every number in the document in the same pass | Counts drift silently because no test asserts them; the reported one is the one someone happened to check, not the only one wrong | +| Take a number from an adjacent document or an earlier session's summary | Recompute from the source now | The adjacent document has the same drift mechanism and no check, so two documents agreeing is not corroboration | +| Sum per-file test counts to get a suite total | Read the suite runner's own summary line | Per-file sums miss collection errors, skips, and dynamically generated cases | +| Publish "N tests" as a durable fact | Publish it with the command and date that produced it | An unattributed count cannot be re-verified and becomes stale the next commit | + +## Sources + +- https://google.github.io/styleguide/docguide/best_practices.html — "Dead docs are bad. They misinform, they slow down, they incite despair in engineers and laziness in team leads"; "Change your documentation in the same CL as the code change" +- https://www.writethedocs.org/guide/writing/docs-principles/ — sources of truth must be "clearly defined and disjoint" so the same fact is not maintained in parallel +- Field incident 2026-08-06 (`linkly`, pre-launch README audit): of nine quantitative claims, five were stale — tests 386→1209, harness mutations 53→77, IR node kinds 20→21 (counted as JSON Schema `anyOf` branches), EBNF productions 51→58, "Twelve Accepted" RFCs→13. One had been reported; the other four were found only because the whole document was swept. Re-checked on 2026-08-06: `grep -on '[0-9]\+ tests\|[0-9]\+ node kinds\|[0-9]\+ productions' README.md` returns the corrected 1209 / 21 / 58 +- Field observation 2026-08-06 (same repo): back-to-back full-suite runs on one commit reported 1195 then 1209 tests — an exact published test count is only reproducible when the collection is deterministic diff --git a/wiki/qa/index.md b/wiki/qa/index.md index 2278dfd..b5e9166 100644 --- a/wiki/qa/index.md +++ b/wiki/qa/index.md @@ -23,6 +23,7 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| +| [quantitative-claims-in-a-published-document](deliverables/quantitative-claims-in-a-published-document.md) | About to publish or hand out a hand-maintained document that states counts about the repository (README, landing page, launch post, architecture overview) — tests, rules, supported types, grammar productions, endpoints, documents in a given state; one number in such a document was reported wrong and you are deciding the scope of the fix; deciding how to publish a count that moves between runs, or which of two disagreeing sources (spec vs implementation) a sentence's number comes from | | [generated-artifacts-as-deliverable-source](deliverables/generated-artifacts-as-deliverable-source.md) | Asked to produce a document (ERD, schema reference, API surface list, dependency inventory) for a hand-off, review, or external partner when the repo already generates that content from code; deciding whether to re-run a stale generator or hand-write the deliverable; a hand-written reference document disagrees with the live system (checks that gate a document → document-verification) | ## document-verification diff --git a/wiki/security/data/commit-identity-in-public-repos.md b/wiki/security/data/commit-identity-in-public-repos.md new file mode 100644 index 0000000..1c4f5f9 --- /dev/null +++ b/wiki/security/data/commit-identity-in-public-repos.md @@ -0,0 +1,83 @@ +--- +id: security-data-commit-identity-in-public-repos +domain: security +category: data +applies_to: [git, github, general] +confidence: verified +sources: + - https://git-scm.com/docs/git-commit + - https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address +last_verified: 2026-08-06 +related: [security-secrets-secrets-in-code, security-data-pii-handling] +--- + +# The Author Identity a Commit Carries into a Public Repository + +## When this applies + +You are about to commit to a repository whose history is public (or will be +published) from a machine whose git identity was configured for something else — +a work laptop set to an employer address, a shared build box, a container image. +Also when a repository's existing history was authored under a different address +than the one your git config now holds. + +## Do this + +1. **Compare the identity in force against the identity the history uses, + before the commit:** + + ```sh + git config user.email # what your next commit will carry + git log -1 --format=%ae # what this repository's history carries + ``` + + When they differ, decide which one this repository should have rather than + letting the ambient one win. + +2. **Override per commit, not globally**, when the machine's identity is correct + for its other repositories: + + ```sh + git -c user.name="…" -c user.email="…" commit -m "…" + ``` + + Git takes the author from `GIT_AUTHOR_*`, then `user.name`/`user.email`, then + `EMAIL`, then a hostname-derived guess — so a `-c` override binds the identity + for exactly this invocation and changes nothing else. + +3. **Set it per repository when you will commit here again:** + `git config user.email "…"` inside the clone. GitHub: "This will override + your global Git configuration settings in this one repository, but will not + affect any other repositories." + +4. **Use the forge's no-reply address for public work** so the published history + carries a routable identity that is not a mailbox you must defend. + +5. **Verify after the first commit, before pushing** — `git log -1 --format=%ae` + — because that is the last point at which the fix is a local amend. + +## Edge cases + +| Case | Then | +|------|------| +| The wrong address is already pushed to a public repository | Treat it as disclosed: the address is served by the forge API, present in every clone, and in downloadable archives. Fix the identity going forward and, when the address must not be associated with the project, rewrite history and force-push before it is mirrored — the window is short and closes on the first fork or archive | +| The repository is public and the commit is old | Rewriting shared history breaks every existing clone; correct forward and accept the historical entry unless the exposure is material | +| Only the committer differs from the author (rebase, cherry-pick, a merge you performed) | Both identities are published; set both with `GIT_COMMITTER_EMAIL`/`-c user.email`, which feeds both unless the `GIT_*` variables are set | +| The commit is produced by CI or a bot | Give it its own dedicated identity in the workflow environment, not a person's — a human address on machine commits misattributes authorship | +| Your git config has no identity at all | Git falls back to the system user name plus a hostname-derived domain, which publishes the machine's hostname; set the identity explicitly rather than relying on the fallback | +| The project requires a real address (DCO sign-off, CLA) | The no-reply form is not acceptable for sign-off; use a personal address you are willing to publish | + +## Instead of + +| If you are about to | Do this instead | Why | +|---------------------|-----------------|-----| +| Commit and rely on the machine's configured identity being right | Compare `git config user.email` against `git log -1 --format=%ae` first | Git does not warn on a mismatch — it commits silently with whichever identity resolves, so the first signal is the published commit | +| Change the global config so this repository gets the right address | Override per invocation (`git -c`) or per clone (`git config` inside it) | A global change silently re-identifies every other repository on the machine, including the ones the original address was correct for | +| Fix a leaked address by deleting the branch | Rewrite the commits and force-push, and treat the address as already disclosed | Deleting a branch does not remove reachable commit objects from the forge's API or from clones already taken | +| Publish a personal mailbox to keep commits linked to your account | Use the forge's no-reply address | It links commits to the account without publishing a mailbox | + +## Sources + +- https://git-scm.com/docs/git-commit — "the information is taken from the configuration items `user.name` and `user.email`, or, if not present, the environment variable `EMAIL`, or, if that is not set, system user name and the hostname used for outgoing mail"; `GIT_AUTHOR_*`/`GIT_COMMITTER_*` take precedence over config +- https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/setting-your-commit-email-address — "GitHub uses the email address set in your local Git configuration to associate commits pushed from the command line with your account"; a per-repository address "will override your global Git configuration settings in this one repository, but will not affect any other repositories" +- Field incident 2026-08-06 (`groundwork`, public repository, macOS): `git config user.email` resolved to an employer address while `git log -1 --format=%ae` showed the repository's history authored under a GitHub no-reply address — the mismatch was silent and would have been published by the next commit diff --git a/wiki/security/index.md b/wiki/security/index.md index 9c6ec2c..3e6da9d 100644 --- a/wiki/security/index.md +++ b/wiki/security/index.md @@ -2,7 +2,7 @@ Route here for: trust-boundary decisions — input validation, authn approach choice, per-resource authorization, secrets hygiene, dependency trust, PII -handling. Mechanics owned elsewhere are linked: XSS rendering → frontend, +handling, the author identity a commit publishes. Mechanics owned elsewhere are linked: XSS rendering → frontend, CI secrets → infrastructure, JWT implementation → backend/frontend auth. Match your situation to a "load when" line; load only matching pages. @@ -49,4 +49,5 @@ Match your situation to a "load when" line; load only matching pages. | Page | Load when | |------|-----------| | [pii-handling](data/pii-handling.md) | A feature stores/processes personal data (emails, names, phones, addresses, government ids); reviewing a log/analytics/export/URL path that can carry PII; designing retention/erasure or handling an erasure request; choosing staging/test data for tables holding PII | +| [commit-identity-in-public-repos](data/commit-identity-in-public-repos.md) | About to commit to a repository whose history is public (or will be published) from a machine whose git identity was configured for something else — a work laptop, a shared build box, a container image; a repository's history uses a different author address than your current git config; deciding between a global, per-repository, and per-commit identity override; responding to an employer or personal address already pushed to a public repository; giving CI/bot commits their own identity | | [masking-verification](data/masking-verification.md) | About to claim masking/redaction of a sensitive field works (logger filter, type-driven masking, serializer); a masking check passed on one output channel; reviewing an automated "masking PASS" verdict | From 569121f35a974cd69b3f2a75f169dda46c24b50f Mon Sep 17 00:00:00 2001 From: choiyounggi <74581798+choiyounggi@users.noreply.github.com> Date: Thu, 6 Aug 2026 20:53:56 +0900 Subject: [PATCH 06/61] knowledge: ingest 1 verified insight (closed value table widening); fold 2 duplicates --- .dev-loop/INGEST_REPORT.md | 141 +++++++++++++++++- log.md | 1 + .../api-design/unenforced-declarations.md | 2 +- .../change-impact/call-site-enumeration.md | 2 +- .../widening-a-closed-value-table.md | 79 ++++++++++ wiki/backend/index.md | 1 + 6 files changed, 220 insertions(+), 6 deletions(-) create mode 100644 wiki/backend/common/change-impact/widening-a-closed-value-table.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 363fce9..d55d536 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,7 +1,21 @@ -# Knowledge flush — 8 insight(s) +# Knowledge flush — 9 insight(s), two passes on one branch -Queue drained: 8 pending rows across 4 session files. Outcome: **2 new pages, -3 merges into existing pages, 3 drops** (2 already-merged duplicates, 1 out of scope). +Queue drained: 8 pending rows across 4 session files (pass 1) plus 3 further rows +(pass 2). Outcome: **3 new pages, 3 merges into existing pages, 3 drops** +(2 already-merged duplicates, 1 out of scope). + +**Why one branch carries two passes.** Pass 1 (2026-08-06 18:30) completed its +ingest and pushed this branch, but `gh` could not open the PR — the stored token +was invalid at the time (`gh api user` → 401), so the branch was left on the +remote with no PR. Pass 2 (2026-08-06 20:45) found the token working again and +found that 2 of its 3 queued candidates were **already carried by this branch** +(as C6 and C7 below). Per the flush skill's open-PR rule, an overlapping +in-flight head is folded into rather than duplicated — so pass 2 added its one +unique insight (C9) to this branch instead of opening a second PR against the +same pages. One PR, one review pass, no duplicate ingestion. + +Pass 2 independently re-verified C6 and C7 before folding rather than trusting +the earlier pass — see their sections. ## Verified best-practice @@ -116,11 +130,56 @@ mismatch is silent, exactly as the page says. This PR is committed under the per identity, not the ambient one. **Confidence: verified.** +### C9 — enumerate a closed value table by its values, not its name → `verified` (pass 2) +**Claim.** When widening a closed table that maps names to magnitudes or codes and +that lives as a named constant, grep the table's *values* and member strings — not +only the constant's name — and reconcile every inlined copy to read the single table +before adding the new entry. +**Checked.** https://refactoring.com/catalog/replaceMagicLiteral.html — *Replace Magic +Literal*, alias "Replace Magic Number with Symbolic Constant": the refactoring exists +because the inlined literal is the default state of such a value, which is precisely +what makes the value (not the name) the reliable search handle. +https://pragprog.com/tips/ — Tip 15, DRY: "Every piece of knowledge must have a +single, unambiguous, authoritative representation within a system." A copied table is +a second representation, and widening one representation is what produces the split. +**Reproduced today** (`linkly`, Python, macOS): `grep -rn "DURATION_UNITS" +impl/lnpl/*.py` → **1** hit (the definition alone); `grep -rn "60000" impl/lnpl/*.py` +→ **7** across four files — a second named table (`DURATION_UNIT_MS`, `lexer.py:23`), +three independently inlined `(("ms",1),("s",1000),("m",60000))` tuples +(`condition.py:353`, `interp.py:1020`, `backend.py:446`) and two bare-literal +arithmetic sites (`condition.py:269-270`). The failure the directive predicts was +already live: the canonical map carries `h` and `d` while all three inlined copies +stop at `m`, so those paths cannot convert a unit the lexer already accepts. +**Confidence: verified** — the mechanism is sourced to two primary references and the +enumeration gap is a reproducible check (1 name hit vs 7 value hits, commands above). +The harvested candidate said "2 name hits / 5 value hits"; the re-run on the current +worktree gives 1 and 7, and the page states the re-measured numbers. + +### C6 and C7 re-verification (pass 2) +Both were re-checked from source before folding, not accepted on the earlier pass's word. + +- **C6 (worktree guardrail).** Read the rule in + `guardrails/1.2.0/hooks/bash-guard.sh` and re-ran a live probe from a linked + worktree against the main root: `grep`, `awk`, `cat` and `git -C
status + --short` each returned **no** decision; the same `grep` redirected to an absolute + path under the main root returned `ask`; redirected to `./probe.log` it returned + none; `cp ./a /b` returned `ask`. This reproduces the merged page + exactly — reads pass, and it is the write verb or the absolute redirect that fires. + The pass-2 queue row asserted the opposite ("read-only access fires `ask`"); it is + refuted as stated, and the true mechanism it was reaching for (a read *captured* + into an absolute path is a write to the rule) is already the merged edge case. +- **C7 (dispatch binding).** `runtime_unavailable` and `terminal_worktree_mismatch` + are present in the shipped repo (`skills/orchestrate/SKILL.md:186,224`; + `skills/orchestrate/scripts/orca-worker-start.sh:88,252`, the latter carrying the + "verified live" comment the page cites). `agent_unconfigured` is **not** a dev-loop + constant — it comes from the harness — which is why the page names it descriptively + ("a stage naming the agent as unconfigured") rather than as a repo symbol. + ## Existing-layer check Routed via `INDEX.md` → domain `index.md` → every page whose "load when" overlapped. -Pages read: backend-common-change-impact-call-site-enumeration, platforms-shells-env-var-off-switches, platforms-shells-unset-versus-empty-parameters, infrastructure-agent-orchestration-worktree-isolated-workers, infrastructure-agent-orchestration-pane-delivery-confirmation, debugging-methodology-hypothesis-testing, qa-deliverables-generated-artifacts-as-deliverable-source, security-secrets-secrets-in-code +Pages read: backend-common-change-impact-call-site-enumeration, backend-common-api-design-unenforced-declarations, platforms-shells-env-var-off-switches, platforms-shells-unset-versus-empty-parameters, infrastructure-agent-orchestration-worktree-isolated-workers, infrastructure-agent-orchestration-pane-delivery-confirmation, debugging-methodology-hypothesis-testing, qa-deliverables-generated-artifacts-as-deliverable-source, security-secrets-secrets-in-code Also read at index level (not opened as pages): the `agent-orchestration`, `document-verification`, `process`, `data` and `secrets` sections of the @@ -137,6 +196,19 @@ infrastructure / qa / security indexes, plus `AGENTS.md` and `templates/page.md` | C7 | `infrastructure-agent-orchestration-pane-delivery-confirmation` owns pane consumption | **Merge** — 4 edge cases + 1 Instead-of + 1 source | | C3 | `qa-deliverables-generated-artifacts-as-deliverable-source` is adjacent (choose-your-source) | **New page** — different decision (verify-before-publish); cross-linked via `related:` | | C8 | `security-secrets-secrets-in-code` (leak response) and `security-data-pii-handling` (PII) are adjacent | **New page** — an author email is neither a rotatable secret nor feature-held PII; `related:` to both | +| C9 | `backend-common-change-impact-call-site-enumeration` (same category, adjacent) and `backend-common-api-design-unenforced-declarations` (closed vocabulary, runtime side) | **New page** — different trigger and a different enumeration handle; `related:` added in both directions | + +**Pass 2 (C9) — why not a merge.** `call-site-enumeration` triggers on *changing a +callee's contract* and enumerates by the callee's name; C9 triggers on *widening a +value table* and enumerates by the table's literals, because the sites that break are +copies that never mention the constant. Its closest existing row ("the change reshapes +a data structure … enumerate the structure's producers by its field names") is about +producers of a shape, not inlined copies of a value set. `unenforced-declarations` +owns the *runtime* side of a closed vocabulary (what to do with a name outside the +table); C9 owns the *change-impact* side (widening the table across an implementation). +Distinct triggers → new page under the existing `change-impact` category +(`wiki-ingest` step 4), cross-linked both ways rather than folded. The receiving page +was also already at 94 body lines, leaving no room for the case at full fidelity. **Conflict flagged, not overwritten.** C6 as harvested contradicts the merged page ("reads fire" vs "reads pass"). I reproduced the rule instead of trusting either: the @@ -174,6 +246,52 @@ be counted as in-flight here, which errs toward more dedup, not less. | C1 hypothesis diagnosticity | none — no in-flight head touches `debugging/methodology/` | **new** | | C2 generated-text policy | none | **drop** (scope, see below) | +### Pass 2 (2026-08-06 20:45) — `gh` working, re-run properly + +`gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"` → +**3 open PRs**: #50 (`knowledge/dch0202-20260806-172420`), #49 +(`knowledge/dch0202-rsquare-20260806-142309`), #47 +(`knowledge/dch0202-20260806-130040`). The pass-1 backlog has since been merged down +from 23 in-flight heads to these 3. `git branch -r --list 'origin/knowledge/*'` also +shows **this branch (`…-183029`) with no PR at all** — the token failure above — which +is what pass 2 diffed its candidates against first. + +All three open heads were fetched and diffed (`git diff --name-only origin/main + -- wiki/`), plus this branch: + +- **#50** → `backend/common/change-impact/call-site-enumeration.md`, `testing/index.md`, + `testing/quality/policy-at-several-return-sites.md` +- **#49** (fork `dch0202-rsquare`) → 7 pages, all under `testing/quality/` +- **#47** → `infrastructure/agent-orchestration/{control-signals-vs-primary-artifacts, + worktree-isolated-workers}.md`, `platforms/filesystems/permissions-and-exec-bits.md`, + `testing/quality/{guard-shape-vs-consequence,tests-that-cannot-fail}.md`, 2 indexes + +| Pass-2 candidate | Overlapping head | Verdict | +|---|---|---| +| worktree guardrail read-vs-write (queue hash `28fd6dfe`) | **this branch** already carries it as C6 in corrected form; **#47 also carries it**, on the premise this pass refuted | **fold** into C6 — re-verified by live probe today, nothing unique to add; row retired. Conflict with #47 flagged below | +| dispatch/terminal binding (queue hash `ba3b56ad`) | **this branch** already carries it as C7 (4 edge cases + 1 Instead-of + a field-observations source) | **fold** — stage names re-checked against the shipped scripts; row retired | +| closed value table widening (queue hash `64bb3517`) | none on content. #50 touches the same *directory* (`change-impact/`) but only edits `call-site-enumeration.md`'s `related:` frontmatter line, adding `testing-quality-policy-at-several-return-sites` | **new** — ingested here as C9 | + +**Conflict flagged for the owner — #47 vs this branch, same page.** Both edit +`worktree-isolated-workers.md`. #47 adds an edge case asserting that "a conservative +rule treats any cross-worktree path reference in command text as a potential write, +reads included", hedged as version-dependent. This pass probed the rule live on the +installed **guardrails 1.2.0**: a bare `grep` / `awk` / `cat` / `git -C
status` +naming the main root produced **no** decision, while the same `grep` redirected to an +absolute path returned `ask` and to `./probe.log` returned none. Reading the rule +source shows why — it requires a surviving main-root mention **and** a write verb or an +absolute redirect. So the read-only premise does not hold for 1.0.0 (already sourced on +`main`) or 1.2.0 (probed today), and #47's prescribed escalation round-trip rests on it. +Not overwritten here: this branch's C6 states the mechanism and the redirect case +positively. **Suggested merge order: this branch first, then re-review #47's row +against it** — the two rows will otherwise sit on one page giving opposite answers. + +**Textual conflict to expect (not a disagreement).** #50 and this branch each append a +different id to the same `related:` line of `call-site-enumeration.md` (#50 adds +`testing-quality-policy-at-several-return-sites`, this branch adds +`backend-common-change-impact-widening-a-closed-value-table`). Whichever merges second +needs both ids kept on that line. + **Backlog note for the owner:** 23 unmerged `knowledge/*` heads is the same pile-up pattern recorded in #39. The call-site-enumeration insight alone exists at 8 paths across 11 branches while the canonical page is already on `main`; the env-var off-switch @@ -192,6 +310,7 @@ re-litigating merged content. This PR deliberately adds nothing to either cluste | C6 | `infrastructure` / `agent-orchestration` / `worktree-isolated-workers` (merge) | no | | C7 | `infrastructure` / `agent-orchestration` / `pane-delivery-confirmation` (merge) | no | | C8 | `security` / `data` / **new page** `commit-identity-in-public-repos` | no | +| C9 | `backend` / `common/change-impact` / **new page** `widening-a-closed-value-table` | no — `change-impact` fits | **Why C3 is a new page rather than a merge.** `generated-artifacts-as-deliverable-source` answers *where a deliverable's body should come from*; C3 answers *what to verify in a @@ -224,3 +343,17 @@ domains' header lines and the matching `INDEX.md` rows were widened to name the concerns. `log.md` has the appended `## [2026-08-06] ingest` entry. All five touched pages are ≤ 120 body lines (76 / 83 / 77 / 82 / 73). Banned vague qualifiers: none in the new pages. + +**Pass 2.** `wiki/backend/index.md` gained one "load when" row under `change-impact` +for `widening-a-closed-value-table` (79 body lines). `related:` links added in both +directions — the new page ↔ `call-site-enumeration`, and the new page ↔ +`unenforced-declarations`. `log.md` has a second appended `## [2026-08-06] ingest` +entry covering this pass. `INDEX.md` needed no change: the backend row already routes +"call-site enumeration before a contract change" to this domain. + +## Queue disposition + +All 11 handled rows were moved to `~/.dev-loop/queue/.processed.jsonl` — the 8 from +pass 1 (including the 3 drops) and the 3 from pass 2 (1 ingested, 2 folded). Nothing +handled was left `pending`; a dropped or folded row left pending would re-cross the +auto-flush threshold indefinitely. diff --git a/log.md b/log.md index 271ad1c..db85e6e 100644 --- a/log.md +++ b/log.md @@ -44,3 +44,4 @@ Append-only. Format: `## [YYYY-MM-DD] Date: Fri, 7 Aug 2026 10:09:05 +0900 Subject: [PATCH 07/61] knowledge: ingest 4 verified insights (mutant triage, source-text wiring assertions, query state vs fetch state, python text-io encoding) --- .dev-loop/INGEST_REPORT.md | 260 +++++++++++++++--- log.md | 1 + wiki/backend/python/index.md | 1 + .../language/bytecode-cache-staleness.md | 2 +- .../language/default-encoding-in-text-io.md | 98 +++++++ .../frontend/data-fetching/async-ui-states.md | 3 +- .../query-state-vs-fetch-state.md | 98 +++++++ wiki/frontend/index.md | 1 + .../environment/timezone-and-locale.md | 2 +- wiki/testing/index.md | 2 + .../quality/behavior-not-implementation.md | 2 +- .../quality/guard-shape-vs-consequence.md | 2 +- .../quality/harness-reverse-controls.md | 2 +- .../quality/source-text-wiring-assertions.md | 103 +++++++ .../surviving-mutant-equivalence-triage.md | 97 +++++++ .../testing/quality/tests-that-cannot-fail.md | 4 +- 16 files changed, 627 insertions(+), 51 deletions(-) create mode 100644 wiki/backend/python/language/default-encoding-in-text-io.md create mode 100644 wiki/frontend/data-fetching/query-state-vs-fetch-state.md create mode 100644 wiki/testing/quality/source-text-wiring-assertions.md create mode 100644 wiki/testing/quality/surviving-mutant-equivalence-triage.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..21f6875 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,227 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 4 insights -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: 4 pending candidates across 3 session files +(`9dab7c31…` ×2, `a26ea793…` ×1, `df9561a2…` ×1). All 4 ingested; 0 dropped. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### 1. Classify a surviving mutant before writing a test for it (`testing`) + +**Claim.** When a mutant survives, decide whether it is a missing test, an +_equivalent_ mutant, or an uncovered line before changing anything. When it is +equivalent, the branch it mutates is redundant — delete it and correct the +comment that justified it, rather than adding a test. + +**Sources checked (all opened this session).** + +- + — "There is no definitive way for Stryker to find and ignore them"; the + documented remedy is "by finding these by hand, which is time consuming and + try to rewrite the code so it won't occur, or accept that you won't make + 100%". This is the primary support: the docs prescribe _rewriting the code_, + not adding a test. +- — "Not all mutations will + behave differently than the unmutated class. These mutants are referred to as + **equivalent mutations**"; "The resulting mutant behaves in exactly the same + way as the original"; and the two distinct verdicts "Survived: The mutation + was not detected by the covering test" vs "No coverage: The same as Survived + except there were no tests that exercised the line of code where the mutation + was created" — which is the three-way split the page's step-1 table encodes. +- + — the mutant state set and `detected / valid` scoring. +- — inserting + faults and requiring failure is the measurement. + +**How verified.** The mutation-testing docs substantiate the classification and +the "rewrite the code" remedy directly. The comment-correction step is the +session's field observation, recorded as a dated field-measurement line in the +page's Sources rather than attributed to a doc. + +**Confidence: verified** (classification + remedy doc-backed; the +comment-correction step carries its field measurement inline). + +### 2. Anchor source-text wiring assertions per site instead of counting (`testing`) + +**Claim.** A guard that asserts by regex that a call is present, using +`toHaveLength(n)` or `>= n` over match count, stays green when one of the N call +sites is deleted. Bind each occurrence to its own context — a bounded lazy order +anchor `A[\s\S]{0,N}?B`, or a function-body slice — and prove each by deleting +only its own site. + +**Sources checked (all opened this session).** + +- + — the Block Statement mutator "removes the content of every block statement", + so deleting one call site is a standard mutation operator, not an ad-hoc edit. +- — "'Survived' means the + mutation was not detected by the covering test": the per-site deletion that + leaves the suite green is exactly this verdict. +- + — `{min,max}` "repeats an atom a minimum of `min` times and a maximum of `max` + times"; adding `?` makes it non-greedy so "the quantifier will try to match as + few times as possible". This is what makes the bounded lazy form limit an + anchor's reach to one site. +- — `toHaveLength` compares a `.length` value; on + a match array it is a total and carries no per-site information. +- + — the refactor cost a source-text guard accepts, which is why the page scopes + it to wiring and routes behavior coverage elsewhere. + +**How verified.** The "a lower bound survives deleting one of N" property is +arithmetic and is stated as such. The mutation-operator and regex-semantics +claims are doc-quoted. The concrete red/green pair (count assertion green vs +anchored assertion red on the same mutant, comment-only control green) is the +session's field measurement, dated in the page's Sources. + +**Confidence: verified.** + +### 3. `data === undefined` is not "loading" in TanStack Query (`frontend`) + +**Claim.** A component contract of `data | undefined` collapses two orthogonal +axes. A disabled (`enabled: false`) or offline-paused query is `status: 'pending'` +with `isLoading === false` and `isError === false` and `data === undefined`, so +"undefined means loading" renders a spinner no fetch will resolve. + +**Sources checked (all opened this session).** + +- — + "The `status` gives information about the `data`: Do we have any or not? The + `fetchStatus` gives information about the `queryFn`: Is it running or not?"; + "Background refetches and stale-while-revalidate logic make all combinations + for `status` and `fetchStatus` possible"; the value definitions including + `paused`: "The query wanted to fetch, but it is paused". +- — + `isLoading` "Is `true` whenever the first fetch for a query is in-flight. Is + the same as `isFetching && isPending`"; `data` "Defaults to `undefined`". +- + — a disabled query with no cached data is "status === 'pending' and + fetchStatus === 'idle'"; "Lazy queries will be in `status: 'pending'` right + from the start because `pending` means that there is no data yet … you likely + cannot use this flag to show a loading spinner"; the `skipToken`/`refetch` + incompatibility quoted in the page's edge table. +- — + "Queries can be in `state: 'pending'`, but `fetchStatus: 'paused'` if they are + mounting for the first time, and you have no network connection"; "it might + not be enough to check for `pending` state to show a loading spinner". + +**How verified.** Docs above, plus a local source check of the shipped build: +`@tanstack/query-core@5.100.14`, `build/modern/queryObserver.js` line 308 +`const isPending = status === "pending"`, line 310 +`const isLoading = isPending && isFetching`, line 332 +`isPaused: newState.fetchStatus === "paused"`. The derivation in the shipped +code matches the reference, so `pending` + non-`fetching` yields +`isLoading === false` with `data === undefined`. + +**Confidence: verified.** + +### 4. Prove a Python `encoding=` fix with `EncodingWarning`, not byte round-trip (`backend/python`) + +**Claim.** On a UTF-8 locale, removing `encoding="utf-8"` from `open()` produces +byte-identical output, so a round-trip regression test is green on the defect. +Run the real entry point under `-X warn_default_encoding -W always::EncodingWarning` +and assert zero warning lines naming the file under test. + +**Sources checked (all opened this session).** + +- — `EncodingWarning` "is emitted when the + `encoding` argument to `open()` is omitted and the default locale-specific + encoding is used"; "The `-X warn_default_encoding` option and the + `PYTHONWARNDEFAULTENCODING` environment variable are added. They are used to + enable `EncodingWarning`"; "Developers using macOS or Linux may forget that + the default encoding is not always UTF-8". +- — UTF-8 mode by default targets Python + 3.15; "many Python developers using Unix forget that the default encoding is + platform dependent … Inconsistent default encoding causes many bugs"; "this + change mostly affects Windows users". This is the mechanism for "invisible on + your machine". +- — `open()`: "The default + encoding is platform dependent (whatever `locale.getencoding()` returns)"; + "For reading and writing raw bytes use binary mode and leave _encoding_ + unspecified". + +**How verified.** Reproduced locally this session (CPython 3.14.6, macOS, +`locale.getpreferredencoding(False) == 'UTF-8'`): a script containing one +`open(p, "w")` and one `open(p, "w", encoding="utf-8")` produced byte-identical +output — the round-trip assertion cannot distinguish them. Running +`python3 -X warn_default_encoding -W always::EncodingWarning script.py out.txt` +emitted exactly one line, naming the unencoded call by file and line number; the +same run without the flag emitted nothing (which is why the page requires proving +the harness reddens on a deliberately unencoded `open()`). + +**Confidence: verified.** ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed each candidate via `INDEX.md` → domain `index.md`, then read every page +whose "load when" line overlapped. + +Pages read: testing-quality-tests-that-cannot-fail, testing-quality-harness-reverse-controls, testing-quality-behavior-not-implementation, testing-quality-guard-shape-vs-consequence, frontend-data-fetching-async-ui-states, frontend-state-client-vs-server-state, frontend-data-fetching-race-conditions, platforms-environment-timezone-and-locale, backend-python-language-mutable-state-traps, backend-python-language-bytecode-cache-staleness + +**Overlaps found and what was done.** + +| Existing page | Overlap | Action | +| --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `testing-quality-tests-that-cannot-fail` | Its whole-suite edge row read _every_ surviving mutant in changed code as "missing or defective tests" — the exact naive reading insight 1 corrects. Page is already ~100 body lines, so appending a triage procedure would break the ≤120 rule | **Refined, not overwritten.** The row now routes surviving mutants through classification first. New page created for the procedure; `related:` both ways | +| `testing-quality-harness-reverse-controls` | Already covers equivalent mutants — but as a _deliberately constructed_ no-op control whose correct verdict is "survived". Different trigger (building/citing a harness vs triaging one live mutant) | Kept separate; `related:` both ways, and the new page routes uniform-verdict cases to it | +| `testing-quality-behavior-not-implementation` | Source-text assertions are implementation-coupled, which this page governs | Kept as the upstream decision ("should you assert on source at all"); insight 2's page opens by routing there, and links back | +| `testing-quality-guard-shape-vs-consequence` | Also about scanning-guard design, but its trigger is a repo-wide guard over _shipped artifacts_ going red on a legitimate one — the opposite failure (false positive) from insight 2 (false negative) | Kept separate; `related:` both ways | +| `frontend-data-fetching-async-ui-states` | Owns the loading/error/empty/data design and mentions `isLoading` vs `isFetching` in one line. It has no coverage of the status × fetchStatus product, and its four-state model has no cell for disabled/paused | **Merged where it fit** (+1 edge row routing the disabled/paused case) + new page for the mechanism; `related:` both ways | +| `frontend-state-client-vs-server-state`, `frontend-data-fetching-race-conditions` | Grepped for `isPending`/`fetchStatus`/`isLoading`/`isFetching`: zero hits. No overlap | No change | +| `platforms-environment-timezone-and-locale` | Owns locale as a hidden input generally, and pins `TZ` for tests. Says nothing about text-encoding defaults or `EncodingWarning` (repo-wide grep for `EncodingWarning`/`warn_default_encoding`/`getpreferredencoding`/`cp949`: 0 hits before this PR) | New page in `backend/python/language`; `related:` both ways, new page routes upward for the general case | +| `backend-python-language-mutable-state-traps` | Same category, unrelated trigger (state leaking across calls) | No change | +| `backend-python-language-bytecode-cache-staleness` | Adjacent: it governs mutation harnesses that rewrite `.py` files, which insight 4's harness does | `related:` both ways; new page carries an edge row routing to it | + +**Conflicts flagged:** none. The one directive that needed adjusting +(`tests-that-cannot-fail`'s whole-suite row) was incomplete rather than +contradictory, so it was refined in place and routed onward, per +`wiki-ingest` step 4. + +**Health checks run on the checkout after the edits:** 183 pages, 0 duplicate +ids; 0 unresolved `[page-id]` references introduced (the 3 the scan reports are +pre-existing false positives — a regex character class in two testing pages and +the literal `[openai-compatible]` in an LLM page); 0 broken relative links from +any index; all 4 new pages 75–80 body lines (limit 120). + +## Open-PR check + +Listed with +`gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"`. +Four open heads: + +| PR | Head | Wiki paths touched | Overlap with this batch | +| --- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| #51 | `knowledge/dch0202-20260806-183029` | backend/{api-design,change-impact}, backend/index, debugging/hypothesis-testing, infrastructure/agent-orchestration ×2, qa/deliverables + qa/index, security/data/commit-identity-in-public-repos + security/index | None — no testing/quality, frontend, or python/language page in common | +| #50 | `knowledge/dch0202-20260806-172420` | testing/index, testing/quality/policy-at-several-return-sites (new), backend/change-impact/call-site-enumeration | **Nearest neighbour — examined in full.** Same underlying defect shape ("one of N sites silently stops being covered"), different subject: #50 covers _behavioral_ tests, one per success-return site of a handler, proven by reverting one site. Insight 2 covers _source-text_ regex guards where no behavioral seam is reachable, and the count-vs-anchor pattern choice. Neither carries the other's content | +| #49 | `knowledge/dch0202-rsquare-20260806-142309` (head ref deleted on remote; diffed via `refs/pull/49/head`) | testing/index + testing/quality ×8 (new: stale-artifact-baselines, unasserted-return-fields, value-preserving-refactor-assertions) | Same category, no shared trigger: baselines/return-field assertions/refactor-value assertions vs mutant triage and source-text wiring | +| #47 | `knowledge/dch0202-20260806-130040` | infrastructure/agent-orchestration, platforms/filesystems, testing/index, testing/quality/{guard-shape-vs-consequence, tests-that-cannot-fail} amendments | Touches two of the pages I amend. My edits are additive and in different regions (a `related:` id and one edge-row rewording); noted as a possible textual conflict for the owner to resolve at merge, not a content duplicate | + +**Per-candidate verdict:** 1 = **new**, 2 = **new**, 3 = **new**, 4 = **new**. +No candidate was folded or dropped — no open PR carries any of these four +insights. + +Note for the owner: #50, #49 and this PR all add pages under +`wiki/testing/quality` and all append a row to `wiki/testing/index.md`, so +whichever merges second will need the index rows rebased. The page files +themselves do not collide. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| # | Insight | Target | New category? | +| --- | -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | +| 1 | Surviving-mutant triage | `testing / quality` → `testing-quality-surviving-mutant-equivalence-triage` (new page) | No — `quality` already owns assertion strength and mutation verification | +| 2 | Source-text wiring assertions | `testing / quality` → `testing-quality-source-text-wiring-assertions` (new page) | No | +| 3 | Query state vs fetch state | `frontend / data-fetching` → `frontend-data-fetching-query-state-vs-fetch-state` (new page) + 1 edge row and a `related:` link on `frontend-data-fetching-async-ui-states` | No — `data-fetching` already owns in-UI fetching states | +| 4 | Locale-default text I/O encoding | `backend / python / language` → `backend-python-language-default-encoding-in-text-io` (new page) | No — `python/language` is described in `INDEX.md` as the home for "language traps" | + +**Why insight 4 went to `backend` and not `testing` or `platforms`.** The +directive changes Python source (`encoding=` at every text-mode call site) and +its test is a property of that language's tooling, so the routing protocol's +"own the artifact you will change" rule puts it in `backend/python`. It routes +upward to `platforms-environment-timezone-and-locale` for the general +hidden-environment-input case and to `testing-quality-tests-that-cannot-fail` +for proving the harness reddens. + +**Plumbing updated:** `wiki/testing/index.md` (+2 rows), +`wiki/frontend/index.md` (+1 row), `wiki/backend/python/index.md` (+1 row), +`log.md` (+1 ingest entry). `INDEX.md` unchanged — no new domain, and every +target domain's "route here when" line already covers these cases. diff --git a/log.md b/log.md index c930fc2..ad4a3cb 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] = n` / `toHaveLength(n)` count assertion over a call appearing at several sites stays green when the one site the guard was written for is deleted — enumerate the sites and bind each to an order anchor `A[\s\S]{0,N}?B` or a function-body slice, prove each by deleting only its own site, and run a reformat control), frontend/data-fetching/query-state-vs-fetch-state (a `data | undefined` component prop collapses TanStack Query's two orthogonal axes; a disabled or offline-paused query is `status: pending` with `isLoading === false` and `isError === false`, so "undefined means loading" renders a spinner no fetch will resolve — pass status+fetchStatus or an explicit union and test one case per cell), backend/python/language/default-encoding-in-text-io (a byte round-trip cannot prove an `encoding=` fix on a UTF-8 locale — run the real entry point under `-X warn_default_encoding -W always::EncodingWarning` and assert zero warning lines naming that file). Merged into existing: tests-that-cannot-fail (whole-suite edge row now routes surviving mutants through classification instead of reading them all as missing tests), harness-reverse-controls, behavior-not-implementation, guard-shape-vs-consequence, async-ui-states (+disabled/paused edge row), bytecode-cache-staleness, timezone-and-locale — related links both ways. All cited URLs opened this session; two local reproductions (CPython 3.14.6 EncodingWarning discriminator vs byte-identical round-trip; `@tanstack/query-core@5.100.14` queryObserver.js:308-332 `isLoading = isPending && isFetching`). diff --git a/wiki/backend/python/index.md b/wiki/backend/python/index.md index c0473f2..f9d1a25 100644 --- a/wiki/backend/python/index.md +++ b/wiki/backend/python/index.md @@ -30,3 +30,4 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [mutable-state-traps](language/mutable-state-traps.md) | State persists or leaks across calls/requests in a long-lived Python process — one user's data appears for another, values "remembered" between calls; loop-built callbacks all use the last value; reviewing function signatures (mutable defaults), class bodies (class attributes), or module-level objects for hidden sharing; choosing contextvars vs thread-locals for request context | | [bytecode-cache-staleness](language/bytecode-cache-staleness.md) | A script or harness rewrites `.py` files and re-runs them in a loop (mutation testing, edit/test/revert, codegen check, bisect) and the result stops tracking what is on disk — a revert that `git diff` reports clean still fails, or an injected change has no effect; choosing between clearing `__pycache__`, refreshing mtime, and hash-based `.pyc` (PEP 552); designing byte-length-preserving mutations | +| [default-encoding-in-text-io](language/default-encoding-in-text-io.md) | Python opens a text file without `encoding=` (`open`, `Path.read_text`, `subprocess` text mode) and you are adding the argument or writing the regression test that keeps it there; a file-writing bug reproduces on Windows, a `LANG=C` container, or a cp949/cp932 desktop but not on your machine; choosing a test discriminator that does not depend on the runner's locale | diff --git a/wiki/backend/python/language/bytecode-cache-staleness.md b/wiki/backend/python/language/bytecode-cache-staleness.md index c911edd..a7ddf1f 100644 --- a/wiki/backend/python/language/bytecode-cache-staleness.md +++ b/wiki/backend/python/language/bytecode-cache-staleness.md @@ -9,7 +9,7 @@ sources: - https://peps.python.org/pep-0552/ - https://docs.python.org/3/library/py_compile.html last_verified: 2026-08-04 -related: [testing-quality-harness-reverse-controls, testing-quality-tests-that-cannot-fail, backend-python-language-mutable-state-traps] +related: [backend-python-language-default-encoding-in-text-io, testing-quality-harness-reverse-controls, testing-quality-tests-that-cannot-fail, backend-python-language-mutable-state-traps] --- # Edited Python Source the Interpreter Keeps Ignoring diff --git a/wiki/backend/python/language/default-encoding-in-text-io.md b/wiki/backend/python/language/default-encoding-in-text-io.md new file mode 100644 index 0000000..a7c59b7 --- /dev/null +++ b/wiki/backend/python/language/default-encoding-in-text-io.md @@ -0,0 +1,98 @@ +--- +id: backend-python-language-default-encoding-in-text-io +domain: backend +category: language +applies_to: [python] +confidence: verified +sources: + - https://peps.python.org/pep-0597/ + - https://peps.python.org/pep-0686/ + - https://docs.python.org/3/library/functions.html +last_verified: 2026-08-07 +related: + [ + backend-python-language-bytecode-cache-staleness, + platforms-environment-timezone-and-locale, + testing-quality-tests-that-cannot-fail, + testing-quality-minimum-case-set, + ] +--- + +# Text I/O Whose Encoding Comes from the Machine's Locale + +## When this applies + +Python code opens a text file without `encoding=` — `open(p)`, `open(p, "w")`, +`Path.read_text()`, `csv`/`json` wrappers built on them — and you are adding the +argument, or writing the regression test that keeps it there. Also when a +file-writing bug reproduces on one machine (Windows, a `LANG=C` container, a +cp949/cp932 desktop) and not on yours. + +Timezone and locale as hidden inputs generally → +[platforms-environment-timezone-and-locale]. + +## Do this + +1. **Pass `encoding=` at every text-mode call site.** The default is the + machine's: "The default encoding is platform dependent (whatever + `locale.getencoding()` returns)". Choose the value from what the file is: + +| The file is | Pass | +| --------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | +| A format with a defined encoding (JSON, TOML, YAML, Markdown, source) | `encoding="utf-8"` | +| Written and read only by this program | `encoding="utf-8"` | +| Produced by a tool bound to the OS console encoding, deliberately | `encoding=locale.getencoding()`, stated explicitly so the dependency is visible | +| Raw bytes | Binary mode with no `encoding` — "For reading and writing raw bytes use binary mode and leave _encoding_ unspecified" | + +2. **Make the regression test run the real entry point under the interpreter's + own diagnostic, and assert zero warnings naming the file you fixed:** + + ```sh + python3 -X warn_default_encoding -W always::EncodingWarning + ``` + + `EncodingWarning` "is emitted when the `encoding` argument to `open()` is + omitted and the default locale-specific encoding is used", and the flag (or + `PYTHONWARNDEFAULTENCODING`) is what enables it. Filter the captured stderr + to the file under test by name, so unfixed call sites elsewhere in the + codebase do not redden this test. + +3. **Assert on the warning lines, not on the produced bytes.** The warning is + emitted at the call site regardless of what the locale happens to be, so it + is the same verdict on your laptop and in CI. + +4. **Keep the flag on the test invocation only.** Without `-X +warn_default_encoding` the warning is silent, so a suite that forgets the flag + passes on the reintroduced defect — assert the flag is present by requiring + the harness to redden on a deliberately unencoded `open()` before trusting it + ([testing-quality-tests-that-cannot-fail]). + +5. **Enable the flag repo-wide in CI once the call sites are clean**, so a new + omission is caught where it is written rather than at the next locale change. + +## Edge cases + +| Case | Then | +| ---------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The entry point is a library function, not a script | Run it through a one-line driver under the same flags; the warning is attributed to the frame that called `open()`, so the driver's own lines do not mask it | +| A dependency emits `EncodingWarning` from its own files | Filter by filename as in step 2 and record the dependency in the test's name, so the filter states what it is excluding | +| The code targets Python 3.15 or later, where UTF-8 mode is on by default (PEP 686) | Keep the explicit `encoding=`: the argument states the file's contract and is what makes the call correct under an inherited `PYTHONUTF8=0` or an older runtime | +| Running under `PYTHONUTF8=1` / UTF-8 mode already | The warning still fires on the omitted argument, so the test keeps working; the mode changes the value used, not whether the argument was passed | +| `subprocess` output is being decoded | The same default applies to its text mode — pass `encoding="utf-8"` there, and include it in the call-site sweep | +| The harness rewrites the file between runs to seed the missing-`encoding` mutation | Clear the bytecode cache between iterations ([backend-python-language-bytecode-cache-staleness]) | + +## Instead of + +| If you are about to | Do this instead | Why | +| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Prove the fix with a round-trip assertion (write non-ASCII text, read it back, compare) | Assert zero `EncodingWarning` lines naming the file, under `-X warn_default_encoding` | On a UTF-8 locale the encoded bytes are identical with and without the argument, so the round-trip passes on the defect; it can only fail on a machine you are not testing on | +| Assert the output file's declared charset (``, an XML declaration) | Assert the warning count | A declaration is a literal in the template — it is written correctly by code that encoded the body wrongly | +| Set `LANG`/`PYTHONUTF8` in the test environment to make the behavior deterministic | Fix the call sites and assert the warning | Pinning the environment makes the test pass by removing the input the defect depends on, so the defect ships and fails on the machines that do not inherit that environment | +| Read "it works on macOS and Linux" as evidence the encoding is right | Run the warning check | PEP 686: "many Python developers using Unix forget that the default encoding is platform dependent … Inconsistent default encoding causes many bugs"; "this change mostly affects Windows users" | + +## Sources + +- https://peps.python.org/pep-0597/ — `EncodingWarning` "is emitted when the `encoding` argument to `open()` is omitted and the default locale-specific encoding is used"; "The `-X warn_default_encoding` option and the `PYTHONWARNDEFAULTENCODING` environment variable are added. They are used to enable `EncodingWarning`"; "When the flag is set, `io.TextIOWrapper()`, `open()` and other modules using them will emit `EncodingWarning` when the `encoding` argument is omitted"; "Developers using macOS or Linux may forget that the default encoding is not always UTF-8" +- https://peps.python.org/pep-0686/ — enabling UTF-8 mode by default targets Python 3.15; "many Python developers using Unix forget that the default encoding is platform dependent. They omit to specify `encoding="utf-8"` … Inconsistent default encoding causes many bugs"; "Most Unix systems use UTF-8 locale … So this change mostly affects Windows users" +- https://docs.python.org/3/library/functions.html — `open()`: "The default encoding is platform dependent (whatever `locale.getencoding()` returns)"; "In text mode, if _encoding_ is not specified the encoding used is platform-dependent"; "For reading and writing raw bytes use binary mode and leave _encoding_ unspecified" +- Reproduction 2026-08-07 (CPython 3.14.6, macOS, `locale.getpreferredencoding(False) == 'UTF-8'`): a script with one `open(p, "w")` and one `open(p, "w", encoding="utf-8")` produced byte-identical output — a round-trip assertion cannot distinguish them. `python3 -X warn_default_encoding -W always::EncodingWarning script.py out.txt` emitted exactly one line, naming the unencoded call by file and line number; the same run without the flag emitted nothing diff --git a/wiki/frontend/data-fetching/async-ui-states.md b/wiki/frontend/data-fetching/async-ui-states.md index 636aaf1..9a10001 100644 --- a/wiki/frontend/data-fetching/async-ui-states.md +++ b/wiki/frontend/data-fetching/async-ui-states.md @@ -11,7 +11,7 @@ sources: - https://tanstack.com/query/latest/docs/framework/react/guides/optimistic-updates - https://react.dev/reference/react/Component last_verified: 2026-07-10 -related: [frontend-state-client-vs-server-state, frontend-data-fetching-race-conditions] +related: [frontend-state-client-vs-server-state, frontend-data-fetching-race-conditions, frontend-data-fetching-query-state-vs-fetch-state] --- # Designing Loading, Error, Empty, and Data States for an Async View @@ -59,6 +59,7 @@ Then apply these to the transitions between states: | List is empty because the user's filters excluded everything | Say so, and offer "clear filters" — the generic empty state ("add your first item") misleads | | Response resolves fast enough that the skeleton only flashes | Keep the reserved space but suppress indicator animation for sub-second responses — feedback that fast is distraction, not information | | Mutation has no inverse (send email, submit payment) | No optimistic update — render an explicit pending state until the server confirms | +| The query can be disabled (`enabled: false`) or paused offline, so it has no data and is not fetching | The four states above do not cover it — branch on the cache's status/fetchStatus pair ([frontend-data-fetching-query-state-vs-fetch-state]) | ## Instead of diff --git a/wiki/frontend/data-fetching/query-state-vs-fetch-state.md b/wiki/frontend/data-fetching/query-state-vs-fetch-state.md new file mode 100644 index 0000000..0c53ede --- /dev/null +++ b/wiki/frontend/data-fetching/query-state-vs-fetch-state.md @@ -0,0 +1,98 @@ +--- +id: frontend-data-fetching-query-state-vs-fetch-state +domain: frontend +category: data-fetching +applies_to: [react, tanstack-query, general] +confidence: verified +sources: + - https://tanstack.com/query/latest/docs/framework/react/guides/queries + - https://tanstack.com/query/latest/docs/framework/react/reference/useQuery + - https://tanstack.com/query/latest/docs/framework/react/guides/disabling-queries + - https://tanstack.com/query/latest/docs/framework/react/guides/network-mode +last_verified: 2026-08-07 +related: + [ + frontend-data-fetching-async-ui-states, + frontend-state-client-vs-server-state, + frontend-data-fetching-race-conditions, + testing-mocking-what-to-mock, + ] +--- + +# A Server-State Query That Has No Data and Is Not Loading + +## When this applies + +You are defining what a component receives from a server-state cache (TanStack +Query and equivalents) and are about to treat `data === undefined` as "loading". +Also when a view shows a permanent spinner with no error and no retry, or the +query it renders can be disabled (`enabled: false`, `skipToken`) or paused by +the network mode. + +Designing the loading / error / empty / data renderings themselves → +[frontend-data-fetching-async-ui-states]. + +## Do this + +1. **Take two independent inputs, not one.** The cache exposes them as separate + fields for this reason: "The `status` gives information about the `data`: Do + we have any or not? The `fetchStatus` gives information about the `queryFn`: + Is it running or not?" — and "all combinations for `status` and `fetchStatus` + [are] possible". A component prop of `data | undefined` collapses both axes + into one bit and cannot recover them. + +2. **Branch on the combination, and give every cell a rendering:** + +| status | fetchStatus | What it means | Render | +| --------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | +| `pending` | `fetching` | First fetch in flight — this is `isLoading`, defined as "`isFetching && isPending`" | Skeleton | +| `pending` | `idle` | Disabled/lazy query: "status === 'pending' and fetchStatus === 'idle'" with `enabled: false` | The pre-request state — the prompt, the disabled form, the "select a row" placeholder | +| `pending` | `paused` | Wanted to fetch, has no connection: "state: 'pending', but fetchStatus: 'paused' if they are mounting for the first time, and you have no network connection" | Offline notice plus a retry affordance | +| `error` | any | The attempt failed | Error message plus retry ([frontend-data-fetching-async-ui-states]) | +| `success` | `fetching` | Background refresh over existing data | The data, plus a subtle refresh indicator | +| `success` | `idle` | Settled | The data (or the empty state when it is an empty collection) | + +3. **Pass the discriminator down.** Give the presentational component either the + two fields or an explicit union (`{kind: 'idle' | 'loading' | 'paused' | +'error' | 'ready', …}`) built at the boundary that holds the query. The union + makes every unhandled state a type error instead of a blank screen. + +4. **Use `isLoading` for spinners and `isPending` for "no data yet".** The docs + state the split directly: lazy queries "will be in `status: 'pending'` right + from the start because `pending` means that there is no data yet … you likely + cannot use this flag to show a loading spinner". + +5. **Cover the disabled and paused cells in tests explicitly.** A test that mocks + the query hook supplies the flags by hand and therefore only ever produces + combinations its author already thought of — so the combination that ships the + bug is the one the suite never constructs. Write one case per row of the step-2 + table, taking the flag values from that table rather than from the component + ([testing-mocking-what-to-mock]). + +## Edge cases + +| Case | Then | +| ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The query has `initialData` or `placeholderData` | It starts at `status: 'success'`, so the pending rows never render — assert which data the user is seeing before treating success as authoritative | +| A disabled query has cached data from an earlier mount | It initializes as "status === 'success' or isSuccess" rather than pending; the idle-pre-request rendering does not apply | +| The component must trigger the fetch itself | Keep `enabled: false` rather than `skipToken`: "`refetch` from `useQuery` will not work with `skipToken`. Calling `refetch()` on a query that uses `skipToken` will result in a `Missing queryFn` error" | +| `select` narrows the data and returns `undefined` for a valid response | That is `success` with `data === undefined` — the sixth combination the one-bit contract also loses; assert on `status`, not on the value | +| The paused state is unreachable because `networkMode: 'always'` is set | Drop the paused row for that query and state the mode in the component's contract, so a later mode change re-opens the row deliberately | +| Several queries feed one view | Combine on the axes, not the values: pending if any is pending, paused if any is paused — a merged `data === undefined` cannot distinguish them | + +## Instead of + +| If you are about to | Do this instead | Why | +| ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Type the child's prop as `data \| undefined` and read `undefined` as "loading" | Pass `status` and `fetchStatus`, or an explicit state union built at the query boundary | A disabled or paused query is `pending` with `data === undefined`, `isLoading === false` and `isError === false`, so the child renders a spinner that no fetch will ever resolve | +| Show the spinner on `isPending` | Show it on `isLoading` (`isPending && isFetching`) and render the idle case separately | A lazy query is `pending` from the first render, so `isPending` puts a spinner on a query that was never requested | +| Add a timeout that turns a long spinner into an error | Render the `pending`/`idle` and `pending`/`paused` cells | The spinner is not slow, it is terminal — a timeout converts a missing state into a wrong one | +| Test the component by mocking the hook with `{data: undefined, isLoading: true}` and `{data: X}` | Drive one case per row of the step-2 table | Hand-written mocks reproduce the author's model of the states, so the combination that causes the bug is the one never constructed | + +## Sources + +- https://tanstack.com/query/latest/docs/framework/react/guides/queries — the `status` values (`pending` "The query has no data yet", `error`, `success`) and `fetchStatus` values (`fetching`, `paused` "The query wanted to fetch, but it is paused", `idle`); "Background refetches and stale-while-revalidate logic make all combinations for `status` and `fetchStatus` possible"; "The `status` gives information about the `data` … The `fetchStatus` gives information about the `queryFn`" +- https://tanstack.com/query/latest/docs/framework/react/reference/useQuery — `isLoading` "Is `true` whenever the first fetch for a query is in-flight. Is the same as `isFetching && isPending`"; `data` "Defaults to `undefined`"; `status` is `pending` "if there's no cached data and no query attempt was finished yet" +- https://tanstack.com/query/latest/docs/framework/react/guides/disabling-queries — a disabled query with no cached data is "status === 'pending' and fetchStatus === 'idle'"; "Lazy queries will be in `status: 'pending'` right from the start because `pending` means that there is no data yet … you likely cannot use this flag to show a loading spinner"; the `skipToken`/`refetch` incompatibility +- https://tanstack.com/query/latest/docs/framework/react/guides/network-mode — "Queries can be in `state: 'pending'`, but `fetchStatus: 'paused'` if they are mounting for the first time, and you have no network connection"; "it might not be enough to check for `pending` state to show a loading spinner" +- Source verification 2026-08-07 (`@tanstack/query-core@5.100.14`, `build/modern/queryObserver.js`): line 308 `const isPending = status === "pending"`, line 310 `const isLoading = isPending && isFetching`, line 332 `isPaused: newState.fetchStatus === "paused"` — the shipped derivation matches the reference, so `pending` + non-`fetching` yields `isLoading === false` with `data === undefined` diff --git a/wiki/frontend/index.md b/wiki/frontend/index.md index 4ca6c93..291a511 100644 --- a/wiki/frontend/index.md +++ b/wiki/frontend/index.md @@ -34,6 +34,7 @@ Match your situation to a "load when" line; load only matching pages. |------|-----------| | [race-conditions](data-fetching/race-conditions.md) | Repeated fetches with changing params can overlap (search-as-you-type, rapid tab/filter switches); UI intermittently shows results for a previous input; mutations race refetches | | [async-ui-states](data-fetching/async-ui-states.md) | Building any view backed by async data; users see blank screens, eternal spinners, or dead-end errors; reviewing loading/error/empty handling in UI code; deciding on skeletons vs spinners, retry affordances, empty states, background-refresh indication, or optimistic updates | +| [query-state-vs-fetch-state](data-fetching/query-state-vs-fetch-state.md) | Defining what a component receives from a server-state cache (TanStack Query and equivalents) and about to treat `data === undefined` as "loading"; a view shows a permanent spinner with no error and no retry; the query can be disabled (`enabled: false`, `skipToken`) or paused by the network mode; deciding what the presentational component's state prop should be | | [infinite-scroll](data-fetching/infinite-scroll.md) | Implementing infinite scroll or a load-more feed; an existing feed loses scroll position on back-navigation, duplicates/skips items, or spams page requests; choosing between infinite scroll and a load-more button | ## performance diff --git a/wiki/platforms/environment/timezone-and-locale.md b/wiki/platforms/environment/timezone-and-locale.md index 729de7c..618c01d 100644 --- a/wiki/platforms/environment/timezone-and-locale.md +++ b/wiki/platforms/environment/timezone-and-locale.md @@ -14,7 +14,7 @@ sources: - https://unicode.org/faq/casemap_charprop.html - https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap07.html last_verified: 2026-07-10 -related: [databases-schema-design-column-data-types, platforms-processes-background-services, platforms-environment-unicode-text-matching] +related: [backend-python-language-default-encoding-in-text-io, databases-schema-design-column-data-types, platforms-processes-background-services, platforms-environment-unicode-text-matching] --- # Timezone and Locale as Hidden Inputs to Date and Text Code diff --git a/wiki/testing/index.md b/wiki/testing/index.md index 5f3ed78..fb1d36e 100644 --- a/wiki/testing/index.md +++ b/wiki/testing/index.md @@ -34,6 +34,8 @@ Match your situation to a "load when" line; load only matching pages. | [spec-artifact-checks](quality/spec-artifact-checks.md) | Writing or reviewing an automated check that a mapping table covers every rule/field/enum case, or that ids resolve across documents; deciding whether a green check earned "verified" or only "present"; designing one negative control per check in a multi-check harness; parsing Markdown table rows programmatically in a doc-as-spec repo | | [schema-additions-under-a-golden-gate](quality/schema-additions-under-a-golden-gate.md) | Adding a node kind, variant, discriminator value, or field to a document format (IR, JSON Schema, spec artifact) whose only automated gate builds its negatives by mutating one committed golden example; the gate or the whole suite comes back green right after a schema change; deciding which negative each new schema keyword needs, and whether a green suite that never loads the schema is evidence at all | | [harness-reverse-controls](quality/harness-reverse-controls.md) | You built a harness that scores how well something is verified (mutation run, doc/spec gate suite, CI check matrix) and are about to cite its score in a commit, PR, README, or report; its verdicts come out uniform (every case caught, or every case green); deciding what control run proves the harness discriminates, how to score errored/never-ran cases, and what the harness's isolated working tree must contain | +| [surviving-mutant-equivalence-triage](quality/surviving-mutant-equivalence-triage.md) | A mutation run (PIT, Stryker, or a hand-seeded mutation) left a mutant alive on code you own and you are deciding what to change; a reviewer asks for a test covering a specific surviving mutant; a defensive branch carries a comment explaining why it is needed and its mutant survives; separating a missing test from an equivalent mutant from an uncovered line | +| [source-text-wiring-assertions](quality/source-text-wiring-assertions.md) | A test reads a source file as a string and asserts by regex that a call is present (cleanup in every handler, logging after each branch, teardown in each exit path) because the behavior has no reachable seam; such a guard is green while one of the call sites is gone; choosing between a count assertion, an order anchor, and a function-body slice | ## data diff --git a/wiki/testing/quality/behavior-not-implementation.md b/wiki/testing/quality/behavior-not-implementation.md index e1181df..7cbe4d0 100644 --- a/wiki/testing/quality/behavior-not-implementation.md +++ b/wiki/testing/quality/behavior-not-implementation.md @@ -9,7 +9,7 @@ sources: - https://abseil.io/resources/swe-book/html/ch12.html - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html last_verified: 2026-07-10 -related: [testing-quality-minimum-case-set, testing-mocking-what-to-mock, testing-quality-guard-shape-vs-consequence, backend-common-change-impact-call-site-enumeration] +related: [testing-quality-minimum-case-set, testing-mocking-what-to-mock, testing-quality-guard-shape-vs-consequence, testing-quality-source-text-wiring-assertions, backend-common-change-impact-call-site-enumeration] --- # Asserting Behavior Through the Public Interface diff --git a/wiki/testing/quality/guard-shape-vs-consequence.md b/wiki/testing/quality/guard-shape-vs-consequence.md index e3875e8..8e976c4 100644 --- a/wiki/testing/quality/guard-shape-vs-consequence.md +++ b/wiki/testing/quality/guard-shape-vs-consequence.md @@ -8,7 +8,7 @@ sources: - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html - https://pitest.org/ last_verified: 2026-08-04 -related: [testing-quality-tests-that-cannot-fail, testing-quality-behavior-not-implementation, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, qa-process-regression-scope] +related: [testing-quality-tests-that-cannot-fail, testing-quality-behavior-not-implementation, testing-quality-source-text-wiring-assertions, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, qa-process-regression-scope] --- # A Repo-Wide Guard That Fires on a Legitimate Artifact diff --git a/wiki/testing/quality/harness-reverse-controls.md b/wiki/testing/quality/harness-reverse-controls.md index aa62831..74965f5 100644 --- a/wiki/testing/quality/harness-reverse-controls.md +++ b/wiki/testing/quality/harness-reverse-controls.md @@ -13,7 +13,7 @@ sources: - https://testing.googleblog.com/2021/04/mutation-testing.html - https://docs.python.org/3/library/unittest.mock.html last_verified: 2026-08-04 -related: [testing-quality-tests-that-cannot-fail, testing-quality-minimum-case-set, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, backend-python-language-bytecode-cache-staleness, qa-exploratory-override-control-pairs] +related: [testing-quality-tests-that-cannot-fail, testing-quality-surviving-mutant-equivalence-triage, testing-quality-minimum-case-set, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, backend-python-language-bytecode-cache-staleness, qa-exploratory-override-control-pairs] --- # Citing a Verification Harness's Own Score diff --git a/wiki/testing/quality/source-text-wiring-assertions.md b/wiki/testing/quality/source-text-wiring-assertions.md new file mode 100644 index 0000000..e29efa9 --- /dev/null +++ b/wiki/testing/quality/source-text-wiring-assertions.md @@ -0,0 +1,103 @@ +--- +id: testing-quality-source-text-wiring-assertions +domain: testing +category: quality +applies_to: [general] +confidence: verified +sources: + - https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ + - https://pitest.org/quickstart/basic_concepts/ + - https://jestjs.io/docs/expect + - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions/Quantifier + - https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html +last_verified: 2026-08-07 +related: + [ + testing-quality-tests-that-cannot-fail, + testing-quality-behavior-not-implementation, + testing-quality-guard-shape-vs-consequence, + testing-quality-harness-reverse-controls, + testing-quality-surviving-mutant-equivalence-triage, + backend-common-change-impact-call-site-enumeration, + ] +--- + +# Asserting on Source Text That a Wiring Call Still Exists + +## When this applies + +A test reads a source file as a string and asserts by regex that some call is +present — a cleanup call in every handler, a logging call after each branch, a +teardown in each exit path — because the behavior has no reachable seam at this +test level. You are choosing the assertion shape, or such a guard is green while +one of the call sites is gone. + +Deciding whether a source-text assertion is warranted at all → +[testing-quality-behavior-not-implementation]. Guards that scan every shipped +artifact for a structural shape → [testing-quality-guard-shape-vs-consequence]. + +## Do this + +1. **Enumerate the call sites the guard is meant to protect, from the source, + before writing the pattern.** The defect this class of guard exists to catch + is "one of N sites was dropped", so the site list is the assertion's real + subject ([backend-common-change-impact-call-site-enumeration]). + +2. **Bind each occurrence to the context that must contain it, and assert one + pattern per site.** Pick the binding by how the sites are separated: + +| How the sites are separated | Assertion shape | +| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Each site follows a distinct preceding call or branch condition | Order anchor: `/[\s\S]{0,N}?/` — a bounded lazy quantifier, so the call must appear within N characters after that anchor and not anywhere else in the file | +| Each site lives in a separate named function | Slice the source to that function's body first (from its declaration to the next top-level declaration), then assert the call inside the slice | +| Sites differ only by an argument value | Assert the full call text including the argument (`setPendingEntry(null)` inside handler `X`), not the callee name | + +3. **Choose N from the enclosing block, not from the file.** Set the bound to + the largest legitimate distance between anchor and call in the current source + plus room for one added statement. A bound wide enough to span two sites lets + either one satisfy the other's assertion. + +4. **Prove each assertion by deleting exactly its own site and requiring exactly + that assertion to redden**, leaving the other sites intact. Deleting one call + is the standard mutation operator, not an ad-hoc edit: Stryker's Block + Statement mutator "removes the content of every block statement". + +5. **Run a semantics-preserving control and require green** — change a comment or + reformat the file. A source-text pattern is one whitespace assumption away + from asserting formatting, and the control is what separates "the guard reads + the wiring" from "the guard reads the layout" + ([testing-quality-harness-reverse-controls]). + +6. **Name each test after its site**, so a reviewer reading a failure knows which + call went missing rather than that "the count changed". + +## Edge cases + +| Case | Then | +| ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| The behavior is reachable through the public interface after all | Assert the behavior and delete the source-text guard — a text assertion passes on a call that is present and broken | +| The sites are generated from a template or macro | Assert on the generator's input at its one site, and add one behavior test on the generated output; per-site text assertions on generated files re-assert the generator | +| The anchor call itself is renamed in a refactor | The guard reddens on correct code — that is the coupling this guard buys; update anchor and call together, and re-run step 4 for each site | +| Two sites legitimately share one anchor (a branch and its else) | Anchor on the branch condition text instead of the shared call, so each arm has its own anchor | +| A site is added during review | The existing assertions staying green is the review finding — a per-site guard has no signal for a site that was never enumerated; re-run step 1 whenever the handler grows a branch | +| The pattern must survive a formatter that reflows lines | Match on the token sequence with `[\s\S]{0,N}?` between tokens rather than on a literal multi-line string, and keep the step-5 reformat control | +| A site's mutant survives despite the assertion | Classify it before strengthening the pattern ([testing-quality-surviving-mutant-equivalence-triage]) — the call may be redundant at that site | + +## Instead of + +| If you are about to | Do this instead | Why | +| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Assert `matches.length >= n` or `toHaveLength(n)` for a call that appears at several sites | Assert one anchored pattern per enumerated site | A lower bound is satisfied by the surviving sites, so deleting the one site the guard was written for keeps it green — the exact defect the guard exists to catch passes it | +| Raise a count assertion's threshold to `toHaveLength(exact)` after finding this gap | Anchor each occurrence to its own context | An exact count reddens when any site is added or removed, including legitimately, and still cannot say which site is missing | +| Match the call anywhere in the file (`/setPendingEntry\(null\)/`) | Bind it to the preceding call or the enclosing function slice | An unbound match is satisfied by any one of the sites, making N sites indistinguishable from one | +| Ship the anchored guard because the suite is green | Delete each site once and require its own assertion red | Narrowing a pattern is the easiest way to narrow it to nothing; a pattern that matches nothing and one that matches everything both read as green | +| Use a source-text guard as the primary coverage for the handler's logic | Keep it as a wiring check and test the behavior at the level that can reach it | Text guards are change-detector tests: they fail on refactors that preserve behavior and pass on a call whose implementation broke | + +## Sources + +- https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ — the Block Statement mutator "removes the content of every block statement", so deleting the statement at one site is a standard mutation operator rather than an ad-hoc edit (step 4) +- https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation was not detected by the covering test"; a per-site deletion that leaves the suite green is exactly this verdict for the site the guard names +- https://jestjs.io/docs/expect — `toHaveLength` asserts a `.length` value; applied to a match array it compares a total and carries no information about which element is missing +- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions/Quantifier — `{min,max}` "repeats an atom a minimum of `min` times and a maximum of `max` times"; adding `?` makes it non-greedy so "the quantifier will try to match as few times as possible" — the bounded lazy form is what limits an anchor's reach to one site +- https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — "you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again" — the cost a source-text guard accepts, and why it stays scoped to wiring +- Field measurement 2026-08-07 (rtb-unified, `apps/web` building-detail panel): `setPendingEntry(null)` appears at four sites (close, list-select, post-resolve, error branch). A `>= 3` count assertion stayed green after a mutant deleted the post-resolve site. Replacing it with the order anchor `resolveBuildingDetailEntry\([\s\S]{0,200}?setPendingEntry\(null\)` produced red on that same mutant, and a comment-only edit kept it green diff --git a/wiki/testing/quality/surviving-mutant-equivalence-triage.md b/wiki/testing/quality/surviving-mutant-equivalence-triage.md new file mode 100644 index 0000000..8a8c484 --- /dev/null +++ b/wiki/testing/quality/surviving-mutant-equivalence-triage.md @@ -0,0 +1,97 @@ +--- +id: testing-quality-surviving-mutant-equivalence-triage +domain: testing +category: quality +applies_to: [general] +confidence: verified +sources: + - https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/ + - https://pitest.org/quickstart/basic_concepts/ + - https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/ + - https://testing.googleblog.com/2021/04/mutation-testing.html +last_verified: 2026-08-07 +related: + [ + testing-quality-tests-that-cannot-fail, + testing-quality-harness-reverse-controls, + testing-quality-minimum-case-set, + testing-quality-behavior-not-implementation, + testing-quality-source-text-wiring-assertions, + backend-common-change-impact-call-site-enumeration, + ] +--- + +# A Surviving Mutant Before You Write a Test for It + +## When this applies + +A mutation run (PIT, Stryker, or a hand-seeded mutation) left a mutant alive on +code you own, and you are deciding what to change. Also when a reviewer asks for +a test to cover a specific surviving mutant, or a defensive branch you added has +a comment explaining why it is needed and its mutant survives. + +Building the mutation harness itself, or citing its score → +[testing-quality-harness-reverse-controls]. + +## Do this + +1. **Classify the survivor before writing anything.** A live mutant is one of + three things, and only one of them is a missing test. Run the mutated code + against the input the branch claims to guard, by hand, and read the result: + +| What the mutated code does on the guarded input | Class | Do | +| ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | +| Produces a different observable result than the original | Missing or weak test | Add the case that distinguishes them ([testing-quality-minimum-case-set]), then re-run the mutation and require red | +| Produces the same observable result for every reachable input | Equivalent mutant — the mutated code is redundant | Steps 2–4: delete the redundancy rather than testing it | +| The line is never executed by any test | PIT's separate `No coverage` state — "the same as Survived except there were no tests that exercised the line" | Add a test that reaches the line first; the kill/survive question is not answerable until then | + +2. **When the mutant is equivalent, find the condition that already absorbs it.** + Equivalence means some other expression makes the mutated one unobservable — + a later comparison, a type coercion, a caller-side check. Name that condition + explicitly; it is the reason the branch is dead. + +3. **Rewrite the code so the equivalent mutant cannot arise.** This is the + remedy Stryker documents: "try to rewrite the code so it won't occur". + Delete the redundant branch and keep the condition found in step 2 as the + single decision point. + +4. **Correct the justification comment in the same edit, using the mechanism you + just measured.** A branch removed in step 3 usually carries a comment saying + why it exists. That comment asserted a mechanism the equivalence just + disproved, so leaving it in place moves a false premise onto whichever + condition remains. Replace it with what the measurement showed, or delete it + when the remaining condition is self-evident. + +5. **Re-run the full suite and require the same pass count as before the edit.** + Deleting a genuinely redundant branch changes no test outcome; a suite that + moves means the branch was load-bearing and step 1 misclassified it — restore + and re-run step 1 with the input that changed. + +## Edge cases + +| Case | Then | +| ----------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The branch is unreachable through the public interface but reachable through another entry point | Not equivalent — it is uncovered from this level. Move the test to the level that reaches it ([testing-quality-behavior-not-implementation]) rather than deleting the branch | +| The mutated behavior differs only in a dimension the suite is not meant to cover (logging, metrics, timing) | PIT's second undetectable class — exclude that region from the mutation set instead of adding a test to chase it | +| The redundant branch exists for readability at a trust boundary (validating external input twice) | Keep it and record why in the comment as a deliberate defense-in-depth, not as a correctness claim; the mutant stays a known survivor | +| The equivalence holds only for the current caller set | Treat it as coverage, not equivalence: enumerate the call sites ([backend-common-change-impact-call-site-enumeration]); when a future caller could pass the absorbed input, the branch is live | +| Several mutants survive in the same function | Classify each one separately — a single function commonly carries one missing test and one equivalent mutant, and one verdict for all of them hides whichever is the other kind | +| The tool reports a 100% kill rate with no survivors at all | Read that as a harness signal, not a code signal, and run the no-op control ([testing-quality-harness-reverse-controls]) | +| The survivor is on a wiring call asserted by a source-text regex rather than by behavior | The count-style assertion is what let it live → [testing-quality-source-text-wiring-assertions] | + +## Instead of + +| If you are about to | Do this instead | Why | +| -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Read every surviving mutant as a test gap and write a case for it | Classify it against the step-1 table first | An equivalent mutant cannot be killed by any correct test, so the case you add asserts a behavior the code does not have and passes for every implementation | +| Chase a 100% mutation score by testing the survivors that resist | Rewrite the code so the equivalent mutant cannot arise, and report the score with the survivors classified | Stryker states there is no definitive way to detect equivalent mutants and that accepting a sub-100% score is the intended outcome | +| Delete a redundant branch and leave its explanatory comment on the remaining condition | Replace the comment with the mechanism the equivalence measurement showed | The comment stated why the deleted branch was necessary; equivalence proved that claim false, and the next reader refactors the surviving condition against it | +| Suppress or ignore the mutant in the tool's config to get the run green | Record the classification in the code (step 4) and leave the mutant visible | A suppression carries no reason, so the next person re-derives the same analysis; a corrected comment carries it | + +## Sources + +- https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/ — "There is no definitive way for Stryker to find and ignore them"; the remedy is "by finding these by hand, which is time consuming and try to rewrite the code so it won't occur, or accept that you won't make 100%"; the documented patterns include operations with neutral values, where several operators produce identical output +- https://pitest.org/quickstart/basic_concepts/ — "Not all mutations will behave differently than the unmutated class. These mutants are referred to as **equivalent mutations**"; "The resulting mutant behaves in exactly the same way as the original"; and the distinct verdicts "Survived: The mutation was not detected by the covering test" vs "No coverage: The same as Survived except there were no tests that exercised the line of code where the mutation was created" — the three-way split in step 1 +- https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/ — the mutant state set and `detected / valid` scoring, which is what makes classifying a survivor a prerequisite to reporting the number +- https://testing.googleblog.com/2021/04/mutation-testing.html — inserting faults and requiring test failure is the measurement; a fault that changes no observable behavior is not one +- Field measurement 2026-08-07 (rtb-unified, `apps/web` building-detail URL parsing): a mutant that deleted the empty-string guard on `?buildingId=` survived. Hand-running the guarded input showed `Number('') === 0`, which the following `parsed > 0` check already rejected — equivalent, not a gap. The branch's comment claimed "an empty string is otherwise read as 0", which the measurement disproved. Deleting the branch and rewriting the comment to the measured mechanism kept all 49 tests passing at the same count diff --git a/wiki/testing/quality/tests-that-cannot-fail.md b/wiki/testing/quality/tests-that-cannot-fail.md index b0534af..3bfa6cc 100644 --- a/wiki/testing/quality/tests-that-cannot-fail.md +++ b/wiki/testing/quality/tests-that-cannot-fail.md @@ -16,7 +16,7 @@ sources: - https://git-scm.com/docs/git-checkout - https://git-scm.com/docs/git-restore last_verified: 2026-08-05 -related: [testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] +related: [testing-quality-minimum-case-set, testing-quality-behavior-not-implementation, testing-mocking-what-to-mock, testing-async-async-testing, testing-quality-checks-that-cannot-pass, testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-surviving-mutant-equivalence-triage, testing-quality-source-text-wiring-assertions, testing-quality-schema-additions-under-a-golden-gate, testing-quality-differential-run-agreement, testing-quality-completion-predicates, testing-quality-guard-shape-vs-consequence, testing-quality-injected-clock-duration-assertions, testing-quality-write-path-assertions, backend-common-change-impact-call-site-enumeration, platforms-shells-portable-shell-scripts, qa-document-verification-spec-document-gates] --- # Proving a Test Can Fail @@ -83,7 +83,7 @@ suite reported as covered, or you are auditing a suspiciously green suite. | Case | Then | |------|------| | Mutating the code under test is impractical right now (slow build, shared branch) | Invert the expected value in the assertion instead and require red — this proves the assertion executes and compares, though not which code defects it catches | -| Auditing a whole suite, not one test | Run an automated mutation-testing tool (PIT, Stryker) and treat surviving mutants in changed code as missing or defective tests | +| Auditing a whole suite, not one test | Run an automated mutation-testing tool (PIT, Stryker), then classify each surviving mutant in changed code before writing a test for it — a missing test, an equivalent mutant, or an uncovered line ([testing-quality-surviving-mutant-equivalence-triage]) | | The mutation run is your own script rather than PIT/Stryker | Prove the harness discriminates before citing its score — a semantics-preserving no-op must survive ([testing-quality-harness-reverse-controls]) | | A test intentionally has no outcome assertion (smoke test: module loads, page renders) | Keep it only when the regression it guards manifests as a throw; name it as a smoke test so reviewers do not count it as behavior coverage | | The always-green test is a snapshot approved without reading | Snapshot rules → [testing-quality-behavior-not-implementation] | From 17571a8168f6973ba9cb3b91eec9c701f4f77a70 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=B5=9C=EC=98=81=EA=B8=B0?= Date: Fri, 7 Aug 2026 10:27:58 +0900 Subject: [PATCH 08/61] knowledge: fix 18 adversarial-review findings (misattributed citation, regex-reach claim, equivalence gate, missing state cells) --- .dev-loop/INGEST_REPORT.md | 107 ++++++++++++++---- log.md | 3 +- .../language/default-encoding-in-text-io.md | 27 +++-- .../query-state-vs-fetch-state.md | 18 +-- .../quality/guard-shape-vs-consequence.md | 2 +- .../quality/source-text-wiring-assertions.md | 91 ++++++++------- .../surviving-mutant-equivalence-triage.md | 94 ++++++++------- 7 files changed, 221 insertions(+), 121 deletions(-) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 21f6875..77806ef 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -18,8 +18,9 @@ comment that justified it, rather than adding a test. — "There is no definitive way for Stryker to find and ignore them"; the documented remedy is "by finding these by hand, which is time consuming and try to rewrite the code so it won't occur, or accept that you won't make - 100%". This is the primary support: the docs prescribe _rewriting the code_, - not adding a test. + 100%". This is the primary support, and both halves are load-bearing: the + docs name rewriting the code and accepting a classified survivor as the two + outcomes — neither of them is "add a test for it". - — "Not all mutations will behave differently than the unmutated class. These mutants are referred to as **equivalent mutations**"; "The resulting mutant behaves in exactly the same @@ -33,45 +34,56 @@ comment that justified it, rather than adding a test. faults and requiring failure is the measurement. **How verified.** The mutation-testing docs substantiate the classification and -the "rewrite the code" remedy directly. The comment-correction step is the -session's field observation, recorded as a dated field-measurement line in the -page's Sources rather than attributed to a doc. - -**Confidence: verified** (classification + remedy doc-backed; the +both remedies directly. The comment-correction step is the session's field +observation, recorded as a dated field-measurement line in the page's Sources +rather than attributed to a doc. After the adversarial pass, the page no longer +lets one hand-run input establish equivalence: the same Stryker sentence that +supports the remedy ("no definitive way … to find and ignore them") is what +makes a domain argument the required evidence for deleting a branch. + +**Confidence: verified** (classification + both remedies doc-backed; the comment-correction step carries its field measurement inline). ### 2. Anchor source-text wiring assertions per site instead of counting (`testing`) **Claim.** A guard that asserts by regex that a call is present, using `toHaveLength(n)` or `>= n` over match count, stays green when one of the N call -sites is deleted. Bind each occurrence to its own context — a bounded lazy order -anchor `A[\s\S]{0,N}?B`, or a function-body slice — and prove each by deleting -only its own site. +sites is deleted. Bind each occurrence to its own context — a bounded order +anchor `A[\s\S]{0,N}B` whose anchor occurs **exactly once** in the file, or a +function-body slice — and prove each by deleting only its own site. **Sources checked (all opened this session).** - - — the Block Statement mutator "removes the content of every block statement", - so deleting one call site is a standard mutation operator, not an ad-hoc edit. + — the Block Statement mutator "removes the content of every block statement". + Corrected after the adversarial pass: this empties a whole block rather than + removing one call, so the per-site deletion is a **hand-seeded** mutation and + the page now says so instead of claiming tool support it does not have. - — "'Survived' means the mutation was not detected by the covering test": the per-site deletion that leaves the suite green is exactly this verdict. - - — `{min,max}` "repeats an atom a minimum of `min` times and a maximum of `max` - times"; adding `?` makes it non-greedy so "the quantifier will try to match as - few times as possible". This is what makes the bounded lazy form limit an - anchor's reach to one site. + — documents `{min,max}` as bounded repetition and `?` as the non-greedy form + that "will try to match as few times as possible". Corrected after the + adversarial pass: an earlier draft presented MDN's `{min,max}` **table** as a + prose quotation, and claimed the lazy form limits the anchor's reach. Neither + holds — see the Node measurement below. - — `toHaveLength` compares a `.length` value; on a match array it is a total and carries no per-site information. - - — the refactor cost a source-text guard accepts, which is why the page scopes - it to wiring and routes behavior coverage elsewhere. + — cited for the change-detector category (the refactor cost a source-text + guard accepts), **without a quotation**: the sentence an earlier draft + attributed to this article is a reader comment, and the body was not + retrievable in full. See the cross-check table, row 1. **How verified.** The "a lower bound survives deleting one of N" property is -arithmetic and is stated as such. The mutation-operator and regex-semantics -claims are doc-quoted. The concrete red/green pair (count assertion green vs -anchored assertion red on the same mutant, comment-only control green) is the -session's field measurement, dated in the page's Sources. +arithmetic and is stated as such. The regex mechanism was **measured**, not +assumed: in Node, `/ANCHOR\([\s\S]{0,20}CALL\(/` and its lazy variant return +identical verdicts on four inputs (in range, call-before-anchor only, beyond the +bound, and call on both sides of the anchor), so the bound plus a once-occurring +anchor is what constrains the match. The concrete red/green pair (count +assertion green vs anchored assertion red on the same mutant, comment-only +control green) is the session's field measurement, dated in the page's Sources. **Confidence: verified.** @@ -119,7 +131,11 @@ code matches the reference, so `pending` + non-`fetching` yields **Claim.** On a UTF-8 locale, removing `encoding="utf-8"` from `open()` produces byte-identical output, so a round-trip regression test is green on the defect. Run the real entry point under `-X warn_default_encoding -W always::EncodingWarning` -and assert zero warning lines naming the file under test. +and assert zero warning lines naming the file under test. Scope correction from +the adversarial pass: this replaces the round-trip only for an *omitted* +argument — `EncodingWarning` never fires on an explicitly wrong value, so the +page keeps a value assertion (run under a non-UTF-8 locale) for the encodings +you set on purpose. **Sources checked (all opened this session).** @@ -150,6 +166,25 @@ the harness reddens on a deliberately unencoded `open()`). **Confidence: verified.** +## Adversarial cross-check (run before this PR was opened) + +Cross-Check: independent `claude` CLI (headless, `--permission-mode plan`) reviewed the wiki diff for fabricated citations, overreach, internal contradiction, bare prohibitions, and vague qualifiers — it returned 18 findings (4 critical, 9 warning, 5 info); every critical was re-verified by me against the primary source or by measurement, and all 18 were fixed before this PR was created. + +The four criticals were real, and two of them were citation defects: + +| # | Finding | Verified how | Fix | +|---|---|---|---| +| 1 | `source-text-wiring-assertions` quoted "you cannot safely refactor code if you know you need to adapt the tests afterwards…" as the Google Testing Blog article's own sentence | Re-fetched the page: the sentence is from a **reader comment dated 2015-02-04**, and its wording differs ("refactor **stuff**", "know **for sure**"). The article body was not retrievable in full, so no sentence from it is quotable | Quotation removed; the URL is now cited for the change-detector *category* only, with a note that nothing is quoted from it. **The same fabricated quote exists in the already-merged `testing-quality-guard-shape-vs-consequence`** — I inherited it from there rather than opening the source. That bullet is corrected in this PR with the correction stated inline | +| 2 | Step 4 claimed deleting one call site is Stryker's Block Statement mutator | The doc says it "removes the content of every block statement" — it empties a whole block, not one call | Reworded: the per-site deletion is a hand-seeded mutation, and the doc is cited for why tools do not generate it | +| 3 | The order-anchor rationale claimed a lazy quantifier limits the anchor's reach and that the call could "not [appear] anywhere else in the file" | Measured in Node: `{0,20}` and `{0,20}?` return identical verdicts on all four inputs, and a call appearing both before and after the anchor still matches | Rewritten around what actually constrains the match: the bound `N` **and an anchor that occurs exactly once**. Added an occurrence-count step, a "no unique anchor" row, and the measurement as a source. A separate MDN pseudo-quote (a table rendered as prose) was also removed | +| 4 | `surviving-mutant-equivalence-triage` authorised **deleting production code** on the basis of one hand-run input, while its own cited source says "There is no definitive way for Stryker to find and ignore them" | Read against the cited Stryker page | Step 1 now starts from the tool's `No coverage` verdict; step 2 requires a stated argument over the branch's whole input domain and routes to the missing-test row when that argument cannot be written; step 5 splits a moved pass count into behavior-test vs implementation-test causes | + +Warnings fixed: selective half-quote of the Stryker remedy; the universal claim that any change in pass count means misclassification; the missing `success` + `paused` cell (with a `status: 'error'` row that had collapsed the fetch axis into `any`, contradicting the page's own premise) in a table that step 5 makes a coverage contract; `applies_to: general` on a page whose every field name is TanStack-specific (now `[react, tanstack-query]`, with an edge row for single-axis caches); the "keep the flag on the test invocation only" step contradicting the next step's "enable it repo-wide"; and the claim that a round-trip test "can only fail on a machine you are not testing on" — plus the gap it hid, that `EncodingWarning` fires only on an *omitted* argument and says nothing about an explicitly wrong one (new step 6 keeps a value assertion for those). + +Info fixed: banned qualifiers "usually", "commonly", "generally" removed from directive sentences; a "sixth combination" ordinal that did not match its own table. + +Re-checked after the fixes: 183 pages / 0 duplicate ids, 0 unresolved `[page-id]` refs in the new pages, 0 broken index links, all four sections present in each page, bodies 83–94 lines (limit 120). + ## Existing-layer check Routed each candidate via `INDEX.md` → domain `index.md`, then read every page @@ -225,3 +260,29 @@ for proving the harness reddens. `wiki/frontend/index.md` (+1 row), `wiki/backend/python/index.md` (+1 row), `log.md` (+1 ingest entry). `INDEX.md` unchanged — no new domain, and every target domain's "route here when" line already covers these cases. + +## Decision Log (AI 생성) + +### 의도 — 무엇을 / 왜 + +- `~/.dev-loop/queue` 에 쌓인 pending 후보 4건을 검증 통과시켜 wiki 에 편입하는 것이 목적. 각 후보를 1차 출처(공식 문서)로 확인하고, 2건은 로컬에서 재현(CPython 3.14.6 EncodingWarning, `@tanstack/query-core@5.100.14` queryObserver.js)해 `confidence: verified` 근거를 만들었다. +- 4건 모두 **신규 페이지**로 라우팅했다. merge-before-create 를 먼저 적용했으나, 가장 가까운 기존 페이지(`tests-that-cannot-fail`)가 이미 ~100 body line 이라 절차를 덧붙이면 ≤120 규칙을 깬다. 대신 그 페이지의 "surviving mutant = missing test" edge 행을 **분류 우선**으로 정정하고 새 페이지로 라우팅하도록 고쳤다. +- PR 직전 독립 적대검증을 1회 돌렸고, 18건 전건을 반영했다. 특히 위조 인용 1건은 **기존 wiki 페이지에서 물려받은 것**이라 그 원본(`guard-shape-vs-consequence`)까지 같은 PR 에서 정정했다 — 알면서 거짓 귀속을 남길 수 없다고 판단. + +### 배제한 대안 — 무엇을 안 했나 / 왜 + +- **열려 있는 PR #50 에 fold 하지 않음.** #50 은 한 핸들러의 여러 success-return 지점을 *행위 테스트*로 덮는 내용이고, 이번 insight 2 는 행위 seam 이 없을 때 쓰는 *소스 텍스트* 가드의 count↔anchor 선택이다. 겹치는 것은 결함 형태("N개 중 하나가 조용히 빠짐")이지 기법이 아니라 별도 페이지로 두고 보고서에 근거를 남겼다. +- **insight 4 를 `testing/` 이 아니라 `backend/python/language/` 로.** 지시가 바꾸는 산출물이 Python 소스이고 판별자가 그 언어 도구의 성질이라, 라우팅 프로토콜의 "바꿀 artifact 를 소유한 도메인" 규칙을 따랐다. +- **round-trip 단정을 전면 금지하지 않음.** 적대검증이 짚은 대로 `EncodingWarning` 은 *인자 누락*만 잡는다 — 명시했지만 틀린 값(`encoding="latin-1"`)은 값 단정이 아니면 아무도 못 잡으므로 둘을 병행하게 했다. +- **merge 하지 않음.** knowledge-flush 는 PR-only 이고 승인은 레포 오너 몫이다. +- **[추정] 커밋 아이덴티티**: skill 지시(ambient git identity 상속)와 직전 flush(PR #49)의 선례에 맞춰 `최영기 ` 로 커밋했다. 이 레포는 public 이고, 마침 열려 있는 PR #51 이 "public 레포에는 forge no-reply 주소를 쓰라"는 페이지를 추가하는 중이라 상충 소지가 있다 — 바꿀지는 작성자 판단. + +### 리뷰어가 볼 곳 — 신뢰성 판단 포인트 + +- `wiki/testing/quality/surviving-mutant-equivalence-triage.md:49` (step 2) — 이 단계가 **운영 코드 분기 삭제**를 승인하는 게이트다. 도메인 논증 요구가 충분한 강도인지 봐 달라. +- `wiki/testing/quality/source-text-wiring-assertions.md:39` (step 2~3) — anchor 유일성 + bound N. 적대검증 전 버전은 lazy quantifier 가 reach 를 제한한다고 **틀리게** 적었다가 실측으로 뒤집힌 자리다. +- `wiki/testing/quality/guard-shape-vs-consequence.md` (Sources 마지막 bullet) — 이번 PR 범위 밖이지만 위조 인용을 정정한 out-of-band 수정. 되돌릴지 판단 필요. +- `wiki/frontend/data-fetching/query-state-vs-fetch-state.md` (step 2 표) — step 5 가 이 표를 테스트 커버리지 계약으로 못박으므로 빠진 셀이 곧 커버리지 구멍이다. 8행이 status × fetchStatus 를 다 덮는지 확인해 달라. +- `wiki/testing/index.md` — #50, #49 도 같은 파일에 행을 추가한다. 두 번째로 머지되는 쪽이 이 행을 rebase 해야 한다(페이지 파일 자체는 충돌 없음). + +> [추정] 표시 항목은 세션에 명시 근거가 없어 사후 재구성한 의도임 — 검증 필요 diff --git a/log.md b/log.md index ad4a3cb..d66c0cc 100644 --- a/log.md +++ b/log.md @@ -43,4 +43,5 @@ Append-only. Format: `## [YYYY-MM-DD] = n` / `toHaveLength(n)` count assertion over a call appearing at several sites stays green when the one site the guard was written for is deleted — enumerate the sites and bind each to an order anchor `A[\s\S]{0,N}?B` or a function-body slice, prove each by deleting only its own site, and run a reformat control), frontend/data-fetching/query-state-vs-fetch-state (a `data | undefined` component prop collapses TanStack Query's two orthogonal axes; a disabled or offline-paused query is `status: pending` with `isLoading === false` and `isError === false`, so "undefined means loading" renders a spinner no fetch will resolve — pass status+fetchStatus or an explicit union and test one case per cell), backend/python/language/default-encoding-in-text-io (a byte round-trip cannot prove an `encoding=` fix on a UTF-8 locale — run the real entry point under `-X warn_default_encoding -W always::EncodingWarning` and assert zero warning lines naming that file). Merged into existing: tests-that-cannot-fail (whole-suite edge row now routes surviving mutants through classification instead of reading them all as missing tests), harness-reverse-controls, behavior-not-implementation, guard-shape-vs-consequence, async-ui-states (+disabled/paused edge row), bytecode-cache-staleness, timezone-and-locale — related links both ways. All cited URLs opened this session; two local reproductions (CPython 3.14.6 EncodingWarning discriminator vs byte-identical round-trip; `@tanstack/query-core@5.100.14` queryObserver.js:308-332 `isLoading = isPending && isFetching`). +## [2026-08-07] ingest | knowledge-flush of 4 queued insights — 4 new pages, 7 existing pages cross-linked. New: testing/quality/surviving-mutant-equivalence-triage (classify a live mutant as missing-test / equivalent / no-coverage before writing anything; when equivalent, delete the redundant branch and correct the justification comment that the equivalence just disproved — Stryker's documented remedy is "try to rewrite the code so it won't occur"), testing/quality/source-text-wiring-assertions (a `>= n` / `toHaveLength(n)` count assertion over a call appearing at several sites stays green when the one site the guard was written for is deleted — enumerate the sites and bind each to a bounded order anchor that occurs exactly once in the file, or to a function-body slice; prove each by deleting only its own site, and run a reformat control. Measured: greedy vs lazy quantifiers give identical verdicts, so the bound and the anchor's uniqueness are what constrain the match), frontend/data-fetching/query-state-vs-fetch-state (a `data | undefined` component prop collapses TanStack Query's two orthogonal axes; a disabled or offline-paused query is `status: pending` with `isLoading === false` and `isError === false`, so "undefined means loading" renders a spinner no fetch will resolve — pass status+fetchStatus or an explicit union and test one case per cell), backend/python/language/default-encoding-in-text-io (a byte round-trip cannot prove an `encoding=` fix on a UTF-8 locale — run the real entry point under `-X warn_default_encoding -W always::EncodingWarning` and assert zero warning lines naming that file). Merged into existing: tests-that-cannot-fail (whole-suite edge row now routes surviving mutants through classification instead of reading them all as missing tests), harness-reverse-controls, behavior-not-implementation, guard-shape-vs-consequence, async-ui-states (+disabled/paused edge row), bytecode-cache-staleness, timezone-and-locale — related links both ways. All cited URLs opened this session; two local reproductions (CPython 3.14.6 EncodingWarning discriminator vs byte-identical round-trip; `@tanstack/query-core@5.100.14` queryObserver.js:308-332 `isLoading = isPending && isFetching`). +## [2026-08-07] revise | testing/quality/guard-shape-vs-consequence — corrected a misattributed citation found by the pre-PR adversarial pass: the sentence "you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again" was presented as the Google Testing Blog article's own, but re-fetching the page shows it is a reader comment (2015-02-04) with different wording ("refactor stuff", "know for sure"). The article body was not retrievable in full, so the bullet now cites the URL for the change-detector category without quoting it, and states the correction inline. The same quote had been copied into a new page in this flush before verification — the lesson being that a citation already present in the wiki is not a verified citation. diff --git a/wiki/backend/python/language/default-encoding-in-text-io.md b/wiki/backend/python/language/default-encoding-in-text-io.md index a7c59b7..e3b6917 100644 --- a/wiki/backend/python/language/default-encoding-in-text-io.md +++ b/wiki/backend/python/language/default-encoding-in-text-io.md @@ -28,7 +28,7 @@ argument, or writing the regression test that keeps it there. Also when a file-writing bug reproduces on one machine (Windows, a `LANG=C` container, a cp949/cp932 desktop) and not on yours. -Timezone and locale as hidden inputs generally → +Timezone and locale as hidden inputs across dates and text → [platforms-environment-timezone-and-locale]. ## Do this @@ -61,14 +61,23 @@ Timezone and locale as hidden inputs generally → emitted at the call site regardless of what the locale happens to be, so it is the same verdict on your laptop and in CI. -4. **Keep the flag on the test invocation only.** Without `-X -warn_default_encoding` the warning is silent, so a suite that forgets the flag - passes on the reintroduced defect — assert the flag is present by requiring - the harness to redden on a deliberately unencoded `open()` before trusting it - ([testing-quality-tests-that-cannot-fail]). +4. **Prove the check reddens before trusting it.** Without `-X + warn_default_encoding` the warning is silent, so a runner that drops the flag + reports green on the reintroduced defect and looks identical to a pass. Seed a + deliberately unencoded `open()` in the file under test, require red, then + restore ([testing-quality-tests-that-cannot-fail]). -5. **Enable the flag repo-wide in CI once the call sites are clean**, so a new - omission is caught where it is written rather than at the next locale change. +5. **Widen the flag from the one test invocation to the whole CI run once every + call site is clean**, so a new omission is caught where it is written rather + than at the next locale change. Until then the filter in step 2 is what keeps + the unfixed sites from reddening this test. + +6. **Keep a value assertion for the encodings you set explicitly.** + `EncodingWarning` fires only on an *omitted* argument, so it says nothing about + `encoding="latin-1"` or a deliberate `encoding=locale.getencoding()`. For those + call sites, assert the bytes the file should contain, and run that assertion + under a non-UTF-8 locale (`LANG=C`, or a cp949/cp932 job) where a wrong value + changes the output. ## Edge cases @@ -85,7 +94,7 @@ warn_default_encoding` the warning is silent, so a suite that forgets the flag | If you are about to | Do this instead | Why | | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Prove the fix with a round-trip assertion (write non-ASCII text, read it back, compare) | Assert zero `EncodingWarning` lines naming the file, under `-X warn_default_encoding` | On a UTF-8 locale the encoded bytes are identical with and without the argument, so the round-trip passes on the defect; it can only fail on a machine you are not testing on | +| Prove an omitted-`encoding` fix with a round-trip assertion alone (write non-ASCII text, read it back, compare) | Assert zero `EncodingWarning` lines naming the file, under `-X warn_default_encoding`, and keep the round-trip for the explicitly-set encodings (step 6) | On a UTF-8 locale the encoded bytes are identical with and without the argument, so the round-trip is green on the defect — it discriminates only on a runner whose locale encoding is not UTF-8, which is not the default on macOS or on most Linux CI images | | Assert the output file's declared charset (``, an XML declaration) | Assert the warning count | A declaration is a literal in the template — it is written correctly by code that encoded the body wrongly | | Set `LANG`/`PYTHONUTF8` in the test environment to make the behavior deterministic | Fix the call sites and assert the warning | Pinning the environment makes the test pass by removing the input the defect depends on, so the defect ships and fails on the machines that do not inherit that environment | | Read "it works on macOS and Linux" as evidence the encoding is right | Run the warning check | PEP 686: "many Python developers using Unix forget that the default encoding is platform dependent … Inconsistent default encoding causes many bugs"; "this change mostly affects Windows users" | diff --git a/wiki/frontend/data-fetching/query-state-vs-fetch-state.md b/wiki/frontend/data-fetching/query-state-vs-fetch-state.md index 0c53ede..66ce45d 100644 --- a/wiki/frontend/data-fetching/query-state-vs-fetch-state.md +++ b/wiki/frontend/data-fetching/query-state-vs-fetch-state.md @@ -2,7 +2,7 @@ id: frontend-data-fetching-query-state-vs-fetch-state domain: frontend category: data-fetching -applies_to: [react, tanstack-query, general] +applies_to: [react, tanstack-query] confidence: verified sources: - https://tanstack.com/query/latest/docs/framework/react/guides/queries @@ -23,8 +23,8 @@ related: ## When this applies -You are defining what a component receives from a server-state cache (TanStack -Query and equivalents) and are about to treat `data === undefined` as "loading". +You are defining what a component receives from a TanStack Query hook and are +about to treat `data === undefined` as "loading". Also when a view shows a permanent spinner with no error and no retry, or the query it renders can be disabled (`enabled: false`, `skipToken`) or paused by the network mode. @@ -34,8 +34,8 @@ Designing the loading / error / empty / data renderings themselves → ## Do this -1. **Take two independent inputs, not one.** The cache exposes them as separate - fields for this reason: "The `status` gives information about the `data`: Do +1. **Take two independent inputs, not one.** TanStack Query exposes them as + separate fields for this reason: "The `status` gives information about the `data`: Do we have any or not? The `fetchStatus` gives information about the `queryFn`: Is it running or not?" — and "all combinations for `status` and `fetchStatus` [are] possible". A component prop of `data | undefined` collapses both axes @@ -48,8 +48,11 @@ Designing the loading / error / empty / data renderings themselves → | `pending` | `fetching` | First fetch in flight — this is `isLoading`, defined as "`isFetching && isPending`" | Skeleton | | `pending` | `idle` | Disabled/lazy query: "status === 'pending' and fetchStatus === 'idle'" with `enabled: false` | The pre-request state — the prompt, the disabled form, the "select a row" placeholder | | `pending` | `paused` | Wanted to fetch, has no connection: "state: 'pending', but fetchStatus: 'paused' if they are mounting for the first time, and you have no network connection" | Offline notice plus a retry affordance | -| `error` | any | The attempt failed | Error message plus retry ([frontend-data-fetching-async-ui-states]) | +| `error` | `idle` | The attempt failed and nothing is retrying | Error message plus a retry affordance ([frontend-data-fetching-async-ui-states]) | +| `error` | `fetching` | A retry is already running over the failed state | Keep the error message, disable the retry affordance while it runs | +| `error` | `paused` | Failed, and the retry is waiting for a connection | Offline notice — a retry affordance here cannot run | | `success` | `fetching` | Background refresh over existing data | The data, plus a subtle refresh indicator | +| `success` | `paused` | Data is on screen and a background refetch is waiting for a connection | The data, plus an offline/stale indicator instead of the refresh indicator | | `success` | `idle` | Settled | The data (or the empty state when it is an empty collection) | 3. **Pass the discriminator down.** Give the presentational component either the @@ -76,8 +79,9 @@ Designing the loading / error / empty / data renderings themselves → | The query has `initialData` or `placeholderData` | It starts at `status: 'success'`, so the pending rows never render — assert which data the user is seeing before treating success as authoritative | | A disabled query has cached data from an earlier mount | It initializes as "status === 'success' or isSuccess" rather than pending; the idle-pre-request rendering does not apply | | The component must trigger the fetch itself | Keep `enabled: false` rather than `skipToken`: "`refetch` from `useQuery` will not work with `skipToken`. Calling `refetch()` on a query that uses `skipToken` will result in a `Missing queryFn` error" | -| `select` narrows the data and returns `undefined` for a valid response | That is `success` with `data === undefined` — the sixth combination the one-bit contract also loses; assert on `status`, not on the value | +| `select` narrows the data and returns `undefined` for a valid response | That is `status: 'success'` with `data === undefined` — a case outside the status × fetchStatus grid that the one-bit contract also loses; assert on `status`, not on the value | | The paused state is unreachable because `networkMode: 'always'` is set | Drop the paused row for that query and state the mode in the component's contract, so a later mode change re-opens the row deliberately | +| The cache in use is SWR, RTK Query, or Apollo rather than TanStack Query | Map its fields onto the two axes before applying the table — where a cache exposes no separate fetch axis, build the explicit union of step 3 from the fields it does expose, and keep the pre-request case distinct from loading | | Several queries feed one view | Combine on the axes, not the values: pending if any is pending, paused if any is paused — a merged `data === undefined` cannot distinguish them | ## Instead of diff --git a/wiki/testing/quality/guard-shape-vs-consequence.md b/wiki/testing/quality/guard-shape-vs-consequence.md index 8e976c4..6ffea97 100644 --- a/wiki/testing/quality/guard-shape-vs-consequence.md +++ b/wiki/testing/quality/guard-shape-vs-consequence.md @@ -78,6 +78,6 @@ Reviewing a guard that has never been red → [testing-quality-tests-that-cannot ## Sources -- https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — Alex Eagle, "Testing on the Toilet: Change-Detector Tests Considered Harmful" (2015-01-27): "Change-detector tests do not add clarity, and you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again." A shape-only guard that must be exempted for each new legitimate artifact is this failure mode at repo scope +- https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — Alex Eagle, "Testing on the Toilet: Change-Detector Tests Considered Harmful" (2015-01-27), cited for the change-detector category: a shape-only guard that must be exempted for each new legitimate artifact is that failure mode at repo scope. Correction 2026-08-07: an earlier revision of this bullet presented "you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again" as the article's own sentence. Re-fetching the page shows it is from a reader comment dated 2015-02-04 and its wording differs ("refactor stuff", "know for sure"); the article body was not retrievable in full, so nothing is quoted from it here - https://pitest.org/ — "Faults (or mutations) are automatically seeded into your code, then your tests are run. If your tests fail then the mutation is killed, if your tests pass then the mutation lived" — the basis for step 4's required-red fixture - Field evidence (linkly #35, 2026-08-04): `test_no_shipped_example_has_a_guarded_repository_call` asserted that no shipped `.lnpl` example contained a repository call under a guard. `examples/checkout.lnpl` legitimately added a `create` under `when stock > 0` — the issue's own reproduction shape — turning the guard permanently red. Re-expressing it as "a guarded call that could actually fail", with the conflict/miss decision taken from the production `_lnpl_ops` derivation via `seeded_entities`/`repository_calls`, returned the suite to `Ran 518 tests / OK` while a fixture holding a guarded-and-can-fail create still drove the guard red diff --git a/wiki/testing/quality/source-text-wiring-assertions.md b/wiki/testing/quality/source-text-wiring-assertions.md index e29efa9..5f5640f 100644 --- a/wiki/testing/quality/source-text-wiring-assertions.md +++ b/wiki/testing/quality/source-text-wiring-assertions.md @@ -41,26 +41,33 @@ artifact for a structural shape → [testing-quality-guard-shape-vs-consequence] 1. **Enumerate the call sites the guard is meant to protect, from the source, before writing the pattern.** The defect this class of guard exists to catch is "one of N sites was dropped", so the site list is the assertion's real - subject ([backend-common-change-impact-call-site-enumeration]). - -2. **Bind each occurrence to the context that must contain it, and assert one - pattern per site.** Pick the binding by how the sites are separated: - -| How the sites are separated | Assertion shape | -| --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Each site follows a distinct preceding call or branch condition | Order anchor: `/[\s\S]{0,N}?/` — a bounded lazy quantifier, so the call must appear within N characters after that anchor and not anywhere else in the file | -| Each site lives in a separate named function | Slice the source to that function's body first (from its declaration to the next top-level declaration), then assert the call inside the slice | -| Sites differ only by an argument value | Assert the full call text including the argument (`setPendingEntry(null)` inside handler `X`), not the callee name | - -3. **Choose N from the enclosing block, not from the file.** Set the bound to - the largest legitimate distance between anchor and call in the current source - plus room for one added statement. A bound wide enough to span two sites lets - either one satisfy the other's assertion. + subject ([backend-common-change-impact-call-site-enumeration]). Write the list + down: the assertion count comes from it, and step 6 re-runs against it. + +2. **Give each site an anchor that occurs exactly once in the file, and assert + one pattern per site.** A regex is satisfied by _any_ anchor–call pair that + fits its bound, so an anchor appearing at two sites lets either one satisfy + the other's assertion. Count the anchor's occurrences before using it: + +| How the sites are separated | Assertion shape | +| -------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Each site follows a call or branch condition whose text appears once in the file | Bounded order anchor: `/[\s\S]{0,N}/`, after confirming the anchor's occurrence count is 1 | +| Each site lives in a separate named function | Slice the source to that function's body (from its declaration to the next top-level declaration) and assert the call inside the slice — the slice bounds the search without needing a unique anchor | +| Sites share the callee and differ only by argument | Assert the full call text including the argument, inside the slice or after the unique anchor — the callee name alone is satisfied by any site | +| No anchor is unique and the sites are not separable into slices | The file gives the guard nothing to bind to: extract the sites into named functions first, or test the behavior at a level that reaches it | + +3. **Set the bound N from the distance the anchor and call actually have in the + current source, plus the length of one statement**, so a legitimately inserted + line does not redden the guard. The bound is what limits the anchor's reach: + measured 2026-08-07 in Node, `{0,20}` and `{0,20}?` return the same verdict on + every input — greedy versus lazy changes which match is reported, not whether + one exists, so a lazy quantifier adds no constraint. 4. **Prove each assertion by deleting exactly its own site and requiring exactly - that assertion to redden**, leaving the other sites intact. Deleting one call - is the standard mutation operator, not an ad-hoc edit: Stryker's Block - Statement mutator "removes the content of every block statement". + that assertion to redden**, leaving the other sites intact. This is a + hand-seeded mutation: Stryker's nearest operator, Block Statement, "removes + the content of every block statement" — it empties a whole block rather than + one call, so tools do not generate this edit for you. 5. **Run a semantics-preserving control and require green** — change a comment or reformat the file. A source-text pattern is one whitespace assumption away @@ -68,36 +75,42 @@ artifact for a structural shape → [testing-quality-guard-shape-vs-consequence] the wiring" from "the guard reads the layout" ([testing-quality-harness-reverse-controls]). -6. **Name each test after its site**, so a reviewer reading a failure knows which +6. **Re-run step 1 whenever the enclosing function grows a branch.** A per-site + guard has no signal for a site that was never enumerated, so the site list — + not the assertions — is what has to be kept current. + +7. **Name each test after its site**, so a reviewer reading a failure knows which call went missing rather than that "the count changed". ## Edge cases -| Case | Then | -| ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| The behavior is reachable through the public interface after all | Assert the behavior and delete the source-text guard — a text assertion passes on a call that is present and broken | -| The sites are generated from a template or macro | Assert on the generator's input at its one site, and add one behavior test on the generated output; per-site text assertions on generated files re-assert the generator | -| The anchor call itself is renamed in a refactor | The guard reddens on correct code — that is the coupling this guard buys; update anchor and call together, and re-run step 4 for each site | -| Two sites legitimately share one anchor (a branch and its else) | Anchor on the branch condition text instead of the shared call, so each arm has its own anchor | -| A site is added during review | The existing assertions staying green is the review finding — a per-site guard has no signal for a site that was never enumerated; re-run step 1 whenever the handler grows a branch | -| The pattern must survive a formatter that reflows lines | Match on the token sequence with `[\s\S]{0,N}?` between tokens rather than on a literal multi-line string, and keep the step-5 reformat control | -| A site's mutant survives despite the assertion | Classify it before strengthening the pattern ([testing-quality-surviving-mutant-equivalence-triage]) — the call may be redundant at that site | +| Case | Then | +| ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| The behavior is reachable through the public interface after all | Assert the behavior and delete the source-text guard — a text assertion passes on a call that is present and broken | +| The sites are generated from a template or macro | Assert on the generator's input at its one site, and add one behavior test on the generated output; per-site text assertions on generated files re-assert the generator | +| The anchor call itself is renamed in a refactor | The guard reddens on correct code — that is the coupling this guard buys; update anchor and call together, and re-run step 4 for each site | +| Two sites legitimately share one anchor (a branch and its else) | Anchor on each arm's own branch-condition text, or slice per arm; a shared anchor makes the two assertions interchangeable | +| The anchor's occurrence count rises from 1 to 2 in a later change | Both assertions became satisfiable by either site — re-run step 2 and pick a new anchor, then re-prove with step 4 | +| The pattern must survive a formatter that reflows lines | Match on the token sequence with `[\s\S]{0,N}` between tokens rather than on a literal multi-line string, and keep the step-5 reformat control | +| A site's mutant survives despite the assertion | Classify it before strengthening the pattern ([testing-quality-surviving-mutant-equivalence-triage]) — the call may be redundant at that site | ## Instead of -| If you are about to | Do this instead | Why | -| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Assert `matches.length >= n` or `toHaveLength(n)` for a call that appears at several sites | Assert one anchored pattern per enumerated site | A lower bound is satisfied by the surviving sites, so deleting the one site the guard was written for keeps it green — the exact defect the guard exists to catch passes it | -| Raise a count assertion's threshold to `toHaveLength(exact)` after finding this gap | Anchor each occurrence to its own context | An exact count reddens when any site is added or removed, including legitimately, and still cannot say which site is missing | -| Match the call anywhere in the file (`/setPendingEntry\(null\)/`) | Bind it to the preceding call or the enclosing function slice | An unbound match is satisfied by any one of the sites, making N sites indistinguishable from one | -| Ship the anchored guard because the suite is green | Delete each site once and require its own assertion red | Narrowing a pattern is the easiest way to narrow it to nothing; a pattern that matches nothing and one that matches everything both read as green | -| Use a source-text guard as the primary coverage for the handler's logic | Keep it as a wiring check and test the behavior at the level that can reach it | Text guards are change-detector tests: they fail on refactors that preserve behavior and pass on a call whose implementation broke | +| If you are about to | Do this instead | Why | +| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Assert `matches.length >= n` or `toHaveLength(n)` for a call that appears at several sites | Assert one anchored or sliced pattern per enumerated site | A lower bound is satisfied by the surviving sites, so deleting the one site the guard was written for keeps it green — the exact defect the guard exists to catch passes it | +| Raise the count assertion to an exact `toHaveLength(n)` after finding this gap | Keep the per-site assertions and re-run the enumeration (step 6) when the function grows a branch | An exact count reddens on legitimate additions and names no site; the per-site guard names the site, and step 6 is what covers additions | +| Match the call anywhere in the file (`/setPendingEntry\(null\)/`) | Bind it to a once-occurring anchor or to the enclosing function slice | An unbound match is satisfied by any one of the sites, making N sites indistinguishable from one | +| Add `?` to the quantifier to keep the anchor from reaching a later site | Set the bound N, and confirm the anchor occurs once | Laziness changes which match is reported, not whether the pattern matches; the reach is decided by N and by the anchor's uniqueness | +| Ship the anchored guard because the suite is green | Delete each site once and require its own assertion red | Narrowing a pattern is the easiest way to narrow it to nothing; a pattern that matches nothing and one that matches everything both read as green | +| Use a source-text guard as the primary coverage for the handler's logic | Keep it as a wiring check and test the behavior at the level that can reach it | Text guards fail on refactors that preserve behavior and pass on a call whose implementation broke | ## Sources -- https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ — the Block Statement mutator "removes the content of every block statement", so deleting the statement at one site is a standard mutation operator rather than an ad-hoc edit (step 4) +- https://stryker-mutator.io/docs/mutation-testing-elements/supported-mutators/ — the Block Statement mutator "removes the content of every block statement"; it empties a block rather than removing one call, which is why the per-site deletion in step 4 is hand-seeded rather than tool-generated - https://pitest.org/quickstart/basic_concepts/ — "'Survived' means the mutation was not detected by the covering test"; a per-site deletion that leaves the suite green is exactly this verdict for the site the guard names - https://jestjs.io/docs/expect — `toHaveLength` asserts a `.length` value; applied to a match array it compares a total and carries no information about which element is missing -- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions/Quantifier — `{min,max}` "repeats an atom a minimum of `min` times and a maximum of `max` times"; adding `?` makes it non-greedy so "the quantifier will try to match as few times as possible" — the bounded lazy form is what limits an anchor's reach to one site -- https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — "you cannot safely refactor code if you know you need to adapt the tests afterwards to get them passing again" — the cost a source-text guard accepts, and why it stays scoped to wiring -- Field measurement 2026-08-07 (rtb-unified, `apps/web` building-detail panel): `setPendingEntry(null)` appears at four sites (close, list-select, post-resolve, error branch). A `>= 3` count assertion stayed green after a mutant deleted the post-resolve site. Replacing it with the order anchor `resolveBuildingDetailEntry\([\s\S]{0,200}?setPendingEntry\(null\)` produced red on that same mutant, and a comment-only edit kept it green +- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Regular_expressions/Quantifier — documents `{min,max}` as a bounded repetition and `?` as the non-greedy form that "will try to match as few times as possible"; non-greediness governs how much the quantifier consumes, not whether the overall pattern matches (measured, step 3) +- https://testing.googleblog.com/2015/01/testing-on-toilet-change-detector-tests.html — Alex Eagle, "Testing on the Toilet: Change-Detector Tests Considered Harmful" (2015-01-27), cited for the change-detector category itself: a test coupled to source shape fails on behavior-preserving edits. The article body was not retrievable in full on 2026-08-07, so nothing here is quoted from it +- Measurement 2026-08-07 (Node): `/ANCHOR\([\s\S]{0,20}CALL\(/` and its lazy form `{0,20}?` returned identical verdicts on four inputs (anchor-then-call in range, call-before-anchor only, call beyond the bound, call both before and after the anchor) — the bound decides reach, and a call elsewhere in the file neither blocks nor is excluded by the pattern +- Field measurement 2026-08-07 (rtb-unified, `apps/web` building-detail panel): `setPendingEntry(null)` appears at four sites (close, list-select, post-resolve, error branch). A `>= 3` count assertion stayed green after a mutant deleted the post-resolve site. Binding it to the once-occurring anchor `resolveBuildingDetailEntry\(` within a 200-character bound produced red on that same mutant, and a comment-only edit kept it green diff --git a/wiki/testing/quality/surviving-mutant-equivalence-triage.md b/wiki/testing/quality/surviving-mutant-equivalence-triage.md index 8a8c484..286b9ab 100644 --- a/wiki/testing/quality/surviving-mutant-equivalence-triage.md +++ b/wiki/testing/quality/surviving-mutant-equivalence-triage.md @@ -36,36 +36,47 @@ Building the mutation harness itself, or citing its score → ## Do this 1. **Classify the survivor before writing anything.** A live mutant is one of - three things, and only one of them is a missing test. Run the mutated code - against the input the branch claims to guard, by hand, and read the result: - -| What the mutated code does on the guarded input | Class | Do | -| ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -| Produces a different observable result than the original | Missing or weak test | Add the case that distinguishes them ([testing-quality-minimum-case-set]), then re-run the mutation and require red | -| Produces the same observable result for every reachable input | Equivalent mutant — the mutated code is redundant | Steps 2–4: delete the redundancy rather than testing it | -| The line is never executed by any test | PIT's separate `No coverage` state — "the same as Survived except there were no tests that exercised the line" | Add a test that reaches the line first; the kill/survive question is not answerable until then | - -2. **When the mutant is equivalent, find the condition that already absorbs it.** - Equivalence means some other expression makes the mutated one unobservable — - a later comparison, a type coercion, a caller-side check. Name that condition - explicitly; it is the reason the branch is dead. - -3. **Rewrite the code so the equivalent mutant cannot arise.** This is the - remedy Stryker documents: "try to rewrite the code so it won't occur". - Delete the redundant branch and keep the condition found in step 2 as the - single decision point. - -4. **Correct the justification comment in the same edit, using the mechanism you - just measured.** A branch removed in step 3 usually carries a comment saying - why it exists. That comment asserted a mechanism the equivalence just - disproved, so leaving it in place moves a false premise onto whichever - condition remains. Replace it with what the measurement showed, or delete it - when the remaining condition is self-evident. - -5. **Re-run the full suite and require the same pass count as before the edit.** - Deleting a genuinely redundant branch changes no test outcome; a suite that - moves means the branch was load-bearing and step 1 misclassified it — restore - and re-run step 1 with the input that changed. + three things, and only one of them is a missing test. Start from the tool's + own verdict, then decide the remaining split by argument over the input + domain, not by trying one value: + +| Signal | Class | Do | +| --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- | +| The tool reports `No coverage` — "there were no tests that exercised the line of code where the mutation was created" | Uncovered line | Add a test that reaches the line, then re-run the mutation; the kill/survive question is not answerable until it is covered | +| The line is covered, and some input in the branch's domain makes original and mutant differ | Missing or weak test | Add that input as a case ([testing-quality-minimum-case-set]), then re-run the mutation and require red | +| The line is covered, and step 2 produces a proof that no input in the branch's domain makes them differ | Equivalent mutant — the branch is redundant | Steps 3–5 | + +2. **Prove equivalence over the domain, not over one input.** Name the condition + elsewhere in the code that absorbs the mutated branch — a later comparison, a + type coercion, a caller-side check — and state why it covers the branch's + whole input set (`Number(s) === 0` for every `s` the branch accepts is + rejected by a following `parsed > 0`). One value that agrees is consistent + with equivalence and does not establish it. When you cannot write that + argument, treat the mutant as the missing-test row and add the case: Stryker + states "There is no definitive way for Stryker to find and ignore them", so + the burden of proof sits on the deletion, not on keeping the branch. + +3. **Delete the redundant branch and keep the absorbing condition from step 2 as + the single decision point.** Stryker's guidance names two acceptable + outcomes — "The only solution is by finding these by hand, which is time + consuming and try to rewrite the code so it won't occur, or accept that you + won't make 100%" — so recording the mutant as a classified survivor is the + correct alternative when the branch stays for a reason in the edge table. + +4. **Correct the justification comment in the same edit, using the argument from + step 2.** When the branch removed in step 3 carries a comment saying why it + exists, that comment asserted a mechanism the equivalence proof contradicts, + so leaving it in place moves a false premise onto whichever condition remains. + Replace it with the absorbing condition you named, or delete it when that + condition is self-evident. + +5. **Re-run the full suite and read a changed pass count by what moved:** + +| After the deletion | Read it as | Do | +| --------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | --------------------------------------------------------------------------------- | +| Same pass count | The branch had no observable consequence the suite asserts | Keep the deletion | +| A behavior test reddens (asserting an input/output pair) | Step 2's domain argument is wrong — the branch is live | Restore it and re-classify with the input that failed | +| Only a test that names the branch itself reddens (source-shape guard, branch-coverage threshold, path snapshot) | The deletion is correct and the test asserted the implementation | Update that test to the new shape ([testing-quality-behavior-not-implementation]) | ## Edge cases @@ -73,25 +84,26 @@ Building the mutation harness itself, or citing its score → | ----------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | The branch is unreachable through the public interface but reachable through another entry point | Not equivalent — it is uncovered from this level. Move the test to the level that reaches it ([testing-quality-behavior-not-implementation]) rather than deleting the branch | | The mutated behavior differs only in a dimension the suite is not meant to cover (logging, metrics, timing) | PIT's second undetectable class — exclude that region from the mutation set instead of adding a test to chase it | -| The redundant branch exists for readability at a trust boundary (validating external input twice) | Keep it and record why in the comment as a deliberate defense-in-depth, not as a correctness claim; the mutant stays a known survivor | +| The redundant branch exists for readability at a trust boundary (validating external input twice) | Keep it and record why in the comment as a deliberate defense-in-depth, not as a correctness claim; the mutant stays a classified survivor | | The equivalence holds only for the current caller set | Treat it as coverage, not equivalence: enumerate the call sites ([backend-common-change-impact-call-site-enumeration]); when a future caller could pass the absorbed input, the branch is live | -| Several mutants survive in the same function | Classify each one separately — a single function commonly carries one missing test and one equivalent mutant, and one verdict for all of them hides whichever is the other kind | +| Several mutants survive in the same function | Classify each one separately — one verdict covering all of them hides whichever is the other kind | | The tool reports a 100% kill rate with no survivors at all | Read that as a harness signal, not a code signal, and run the no-op control ([testing-quality-harness-reverse-controls]) | | The survivor is on a wiring call asserted by a source-text regex rather than by behavior | The count-style assertion is what let it live → [testing-quality-source-text-wiring-assertions] | ## Instead of -| If you are about to | Do this instead | Why | -| -------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Read every surviving mutant as a test gap and write a case for it | Classify it against the step-1 table first | An equivalent mutant cannot be killed by any correct test, so the case you add asserts a behavior the code does not have and passes for every implementation | -| Chase a 100% mutation score by testing the survivors that resist | Rewrite the code so the equivalent mutant cannot arise, and report the score with the survivors classified | Stryker states there is no definitive way to detect equivalent mutants and that accepting a sub-100% score is the intended outcome | -| Delete a redundant branch and leave its explanatory comment on the remaining condition | Replace the comment with the mechanism the equivalence measurement showed | The comment stated why the deleted branch was necessary; equivalence proved that claim false, and the next reader refactors the surviving condition against it | -| Suppress or ignore the mutant in the tool's config to get the run green | Record the classification in the code (step 4) and leave the mutant visible | A suppression carries no reason, so the next person re-derives the same analysis; a corrected comment carries it | +| If you are about to | Do this instead | Why | +| -------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Read every surviving mutant as a test gap and write a case for it | Classify it against the step-1 table first | An equivalent mutant cannot be killed by any correct test, so the case you add asserts a behavior the code does not have and passes for every implementation | +| Declare equivalence because one input produced the same result | Write the step-2 domain argument, or classify it as a missing test | Agreement on one value is what both classes look like; the deletion in step 3 changes production code, so it needs the stronger claim | +| Chase a 100% mutation score by testing the survivors that resist | Take one of the two documented outcomes: rewrite the code so the mutant cannot arise, or record the survivor as classified and accept the score | Stryker states there is no definitive way to detect equivalent mutants and names accepting a sub-100% score as an acceptable outcome | +| Delete a redundant branch and leave its explanatory comment on the remaining condition | Replace the comment with the absorbing condition from step 2 | The comment stated why the deleted branch was necessary; the equivalence proof contradicts that claim, and the next reader refactors the surviving condition against it | +| Suppress or ignore the mutant in the tool's config to get the run green | Record the classification in the code (step 4) and leave the mutant visible | A suppression carries no reason, so the next person re-derives the same analysis; a corrected comment carries it | ## Sources -- https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/ — "There is no definitive way for Stryker to find and ignore them"; the remedy is "by finding these by hand, which is time consuming and try to rewrite the code so it won't occur, or accept that you won't make 100%"; the documented patterns include operations with neutral values, where several operators produce identical output -- https://pitest.org/quickstart/basic_concepts/ — "Not all mutations will behave differently than the unmutated class. These mutants are referred to as **equivalent mutations**"; "The resulting mutant behaves in exactly the same way as the original"; and the distinct verdicts "Survived: The mutation was not detected by the covering test" vs "No coverage: The same as Survived except there were no tests that exercised the line of code where the mutation was created" — the three-way split in step 1 +- https://stryker-mutator.io/docs/mutation-testing-elements/equivalent-mutants/ — "There is no definitive way for Stryker to find and ignore them"; "The only solution is by finding these by hand, which is time consuming and try to rewrite the code so it won't occur, or accept that you won't make 100%" — both halves of that sentence are load-bearing here: rewriting is one outcome, a classified survivor is the other +- https://pitest.org/quickstart/basic_concepts/ — "Not all mutations will behave differently than the unmutated class. These mutants are referred to as **equivalent mutations**"; "The resulting mutant behaves in exactly the same way as the original"; and the distinct verdicts "Survived: The mutation was not detected by the covering test" vs "No coverage: The same as Survived except there were no tests that exercised the line of code where the mutation was created" — the split in step 1 - https://stryker-mutator.io/docs/mutation-testing-elements/mutant-states-and-metrics/ — the mutant state set and `detected / valid` scoring, which is what makes classifying a survivor a prerequisite to reporting the number - https://testing.googleblog.com/2021/04/mutation-testing.html — inserting faults and requiring test failure is the measurement; a fault that changes no observable behavior is not one -- Field measurement 2026-08-07 (rtb-unified, `apps/web` building-detail URL parsing): a mutant that deleted the empty-string guard on `?buildingId=` survived. Hand-running the guarded input showed `Number('') === 0`, which the following `parsed > 0` check already rejected — equivalent, not a gap. The branch's comment claimed "an empty string is otherwise read as 0", which the measurement disproved. Deleting the branch and rewriting the comment to the measured mechanism kept all 49 tests passing at the same count +- Field measurement 2026-08-07 (rtb-unified, `apps/web` building-detail URL parsing): a mutant that deleted the empty-string guard on `?buildingId=` survived. The domain argument was that the guard's whole input set is strings that `Number()` maps to `0` or `NaN`, both of which the following `parsed > 0` rejects — so no accepted input distinguishes the two. The branch's comment claimed "an empty string is otherwise read as 0", which that argument contradicts. Deleting the branch and rewriting the comment left all 49 tests passing at the same count From 52edafee09e3570d4820db591622782a7e9aecce Mon Sep 17 00:00:00 2001 From: choiyounggi <74581798+choiyounggi@users.noreply.github.com> Date: Fri, 7 Aug 2026 14:42:43 +0900 Subject: [PATCH 09/61] knowledge: ingest 1 verified insight (MAPE-aligned point prediction, new backend/common/ml category) --- .dev-loop/INGEST_REPORT.md | 72 ++++++++----------- INDEX.md | 2 +- log.md | 1 + .../ml/mape-aligned-point-prediction.md | 61 ++++++++++++++++ wiki/backend/index.md | 8 ++- 5 files changed, 99 insertions(+), 45 deletions(-) create mode 100644 wiki/backend/common/ml/mape-aligned-point-prediction.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..075ca85 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,39 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 1 insight ingested (2 dropped as pending-duplicates of open PR #51) -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: 3 pending candidates across 2 session files. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +**Ingested — MAPE-aligned point prediction** (from `avm-hackathon-jycho` session): + +- **Claim:** A regression model trained to predict the conditional median (log target + L1 loss) and scored by MAPE is a structural overpredictor; the MAPE-optimal point prediction is the median of the 1/y-reweighted predictive distribution, which equals `median × exp(−σ²)` for a lognormal conditional. Correct per-row with `pred × exp(−λσ²)`, σ estimated from q16/q84 quantile spread, λ selected from 0.5 by multi-period holdout consistency rather than the theoretical 1.0. +- **Sources checked:** + - https://arxiv.org/abs/0912.0902 — Gneiting, "Making and Evaluating Point Forecasts" (JASA 106:746–762, 2011). Table 5: the Bayes rule under absolute percentage error is the β-median with β = −1 (median of the y⁻¹-reweighted predictive distribution). Confirmed via WebSearch against arxiv.org and the tandfonline JASA record. + - https://arxiv.org/abs/1605.02541 — de Myttenaere, Golden, Le Grand, Rossi, "Mean Absolute Percentage Error for regression models" (Neurocomputing 2016). Confirms MAPE-optimal regression ≡ 1/y-weighted MAE regression. The citation given in the raw candidate was checked and is real (also mirrored at hal.science/hal-01312590). + - Lognormal algebra re-derived independently: for Y ~ LN(μ, σ²), y⁻¹·f(y) normalizes to LN(μ − σ², σ²), whose median is exp(μ − σ²) = median(Y)·exp(−σ²). +- **Verification result:** the shrinkage mechanism and its direction are **verified** against the two papers plus the closed-form derivation. The practical λ = 0.5 starting point and the "quantile spread overestimates σ" claim rest on the session's holdout evidence only (Seoul commercial-building AVM, LightGBM + CatBoost: λ = 0.5 improved MAPE on all 4 holdout years, 19.18% → 18.86% excluding 2025 outliers; λ = 1.0 degraded 2 of 4 years) — that part is **field-tested** and the page's Sources section says so explicitly. Page frontmatter is `confidence: verified` for the sourced mechanism, with the field-tested scope of the λ practice stated inline. + +**Dropped candidates** (both from the linkly r3 orchestrate session) — see Open-PR check; neither was ingested, so no verification pass was spent upgrading them. ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routing went INDEX.md → backend → domain index. A full-text sweep of the checkout's `wiki/` for `mape`, `percentage error`, `lognormal`, `quantile`, `lightgbm`, `calibration` returned **zero hits** — nothing in any layer covers regression-model point-prediction calibration, so this is a create, not a merge. The closest existing category is `backend/common/llm`, whose pages cover *consuming* LLM completion APIs, not training/evaluating predictive models; I opened its nearest page to confirm the trigger space does not overlap (it gates on HTTP completion responses, not on metrics or point forecasts). + +Pages read: backend-common-llm-completion-response-validation + +No conflicts flagged. `related:` left empty — no genuinely adjacent page exists yet (the new category is a seed; future ML ingests should link here). Plumbing updated: `wiki/backend/index.md` gained a `### ml` section with a load-when line, the common-subtree route line and root `INDEX.md` backend line both mention MAPE-aligned point-prediction calibration, and `log.md` has the ingest entry. + +## Open-PR check + +Open `knowledge/*` heads listed via `gh pr list --search "head:knowledge/"`: #47 (`knowledge/dch0202-20260806-130040`), #49 (`knowledge/dch0202-rsquare-20260806-142309`), #50 (`knowledge/dch0202-20260806-172420`), #51 (`knowledge/dch0202-20260806-183029`), #52 (`knowledge/dch0202-rsquare-20260807-100149`). + +Per-candidate verdicts: + +- **Candidate 1 — worktree_escape guardrail escalation on cross-worktree reads** (linkly): **drop**. Fetched and diffed `origin/knowledge/dch0202-20260806-183029` (#51) against main: its `wiki/infrastructure/agent-orchestration/worktree-isolated-workers.md` additions already carry this insight in strictly better form — including the exact "budget the escalation round trip (read → approve → clear escalations/ → restart the watcher) and state in the worker's first brief that reads are approved" row. #51 additionally *corrects* the raw candidate: its local reproduction (guardrails 1.2.0) shows pure reads pass and the rule fires only when a main-root mention co-occurs with a write verb or an absolute-path redirect — the candidate's blanket "fires on read-only access" phrasing is the pre-correction version. Nothing unique to fold; retired as pending-duplicate. +- **Candidate 2 — Orca dispatch binding taxonomy (idle-prompt check; runtime_unavailable vs agent_unconfigured vs terminal_worktree_mismatch)** (linkly): **drop**. The same #51 diff's `wiki/infrastructure/agent-orchestration/pane-delivery-confirmation.md` additions carry all four rows (bind only on idle prompt because "done" is a report not the turn's end; wait-and-rebind for occupied runtime; close pane + new worker-mode agent for a dead one; always pass worktree with pane) citing the same three 2026-08-06 field incidents the candidate cites. Nothing unique to fold; retired as pending-duplicate. +- **Candidate 3 — MAPE-aligned point prediction** (avm-hackathon): **new**. No overlap with any open head — #47/#49/#50/#52 are testing/shell/encoding-themed and #51's wiki diff (11 files) touches no ML content. Ingested here. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +- **Candidate 3 → `backend/common/ml/mape-aligned-point-prediction.md`** (id `backend-common-ml-mape-aligned-point-prediction`), **new category `ml`** under backend/common. Justification for the new category: the harvested domain hint was `backend`, and the backend domain's 12 existing categories (api-design, auth, caching, change-impact, concurrency, errors, integrations, jobs, llm, orm, reliability, storage) all cover server-side application code concerns; none covers training or evaluating a predictive model. `llm` is the nearest name but its scope is consuming LLM completion APIs from server code — putting metric-aligned regression calibration there would corrupt its load-when gate. No other domain fits better (databases owns SQL/schema; qa owns release process). The category seeds with one page. +- Candidates 1–2: no routing — dropped as pending-duplicates of open PR #51 (see above); their queue rows are retired to `.processed.jsonl` so the auto-flush cannot re-surface them. diff --git a/INDEX.md b/INDEX.md index cb6cd09..8f617b7 100644 --- a/INDEX.md +++ b/INDEX.md @@ -10,7 +10,7 @@ follow the cross-pointers in their index or take the next matching seeded domain | Domain | Status | Route here when | |--------|--------|-----------------| | [databases](wiki/databases/index.md) | **seeded** | Designing schemas/tables/keys, choosing or evaluating indexes, writing or optimizing queries, choosing transaction/isolation behavior, surveying live data to derive a rule, verifying additive migrations | -| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, call-site enumeration before a contract change, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, consuming external-API responses, externally-owned defaults, object-storage references) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) | +| [backend](wiki/backend/index.md) | **seeded** | Server-side application code — language-agnostic (`common/`: API contracts, call-site enumeration before a contract change, idempotency, JWT, timeouts/retries, caching, jobs, transactions in app code, shared state/pools, errors, LLM completion validation & context budgeting, MAPE-aligned point-prediction calibration, consuming external-API responses, externally-owned defaults, object-storage references) plus stack subtrees: `java/` (JPA, Spring proxies, JVM threads/memory), `node/` (event loop, promises, runtime validation, shutdown), `python/` (GIL/asyncio, pydantic, WSGI/ASGI workers, language traps) | | [frontend](wiki/frontend/index.md) | **seeded** | Web UI code: state placement, rendering performance, in-UI data fetching (races, infinite scroll), auth token handling, forms, XSS-safe output, accessibility | | [infrastructure](wiki/infrastructure/index.md) | **seeded** | CI/CD pipelines, secrets in build/deploy, container image builds, rollout/rollback strategy, observability (logs/metrics/alerting), per-environment/path-valued config, multi-agent orchestration (worker liveness signals, shared run state, tmux pane delivery, completion gates, worktree-isolated workers) | | [testing](wiki/testing/index.md) | **seeded** | Writing or structuring automated tests: level choice, cases/assertions, test data, mock decisions, flaky tests (release-process quality → qa) | diff --git a/log.md b/log.md index c930fc2..eed6993 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] Date: Fri, 7 Aug 2026 15:43:58 +0900 Subject: [PATCH 10/61] knowledge: ingest 1 verified insight (warnings-as-errors gate vs intentional warnings); 2 dropped as in-flight duplicates --- .dev-loop/INGEST_REPORT.md | 128 ++++++++++++------ log.md | 1 + .../api-design/unenforced-declarations.md | 2 +- wiki/platforms/index.md | 2 +- ...diagnostics-without-a-failing-exit-code.md | 26 +++- 5 files changed, 106 insertions(+), 53 deletions(-) diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..72a257d 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,93 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state - -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +# Knowledge flush — 3 insight(s): 1 ingested, 2 dropped as in-flight duplicates ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +**1. Warnings-as-errors gates vs intentional-warning features (INGESTED, confidence: verified).** +Claim: before adopting a `-Werror`/`--strict`-style promotion, run the gate against a +*valid* input that legitimately warns (deprecation, accept-and-warn declaration) — the +usual two-way check (catches a bad file / passes a clean file) cannot see this third +input class, and when the platform's diagnostics have no severity tiers the gate and +the feature are mutually exclusive; record that as a platform defect. +Sources checked: +- https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html — fetched this session; + `-Werror=`/`-Wno-error=` per-warning promotion/exemption exists precisely so + specific warnings can be exempted from a blanket error gate ("more specific options + have priority over less specific ones"). +- https://rust-unofficial.github.io/patterns/anti_patterns/deny-warnings.html — fetched + this session; blanket `#![deny(warnings)]` is an anti-pattern because "APIs get + deprecated, so their use will emit a warning where before there was none"; the + recommended alternative is explicit lint selection deliberately excluding `deprecated`. +- Field reproduction (lnpl 0.2.0 QA re-measurement, 2026-08-05→07, evidence file + `qa/rerun/cases/batch-report/evidence/08-diag-channel.log` in the linkly repo): + unknown-verb → `--strict` rc=2 (caught); clean file → rc=0 (no false positive); + legitimate `on schedule` declaration → rc=2 anyway, via the accept-and-warn + "declared, not enforced" diagnostic. Mechanism doc-verified + locally reproduced → + **verified**. + +**2. worktree_escape read-only escalation round-trip (DROPPED — pending duplicate).** +Not re-verified here; the identical insight *with the same session evidence* (Wave 2 +worker's upstream-FINDINGS `awk`/`grep` verification and `git status` check each raising +`ask`/exit 5) is already carried by open PRs #47 and #51 on +`worktree-isolated-workers` — #51's version is strictly better (it names the actual rule +mechanism: main-root mention and write-verb/redirect matched independently over the +whole command string). Nothing unique remained to fold. + +**3. Orca dispatch-binding stage taxonomy (DROPPED — pending duplicate).** +Idle-prompt check before binding; `runtime_unavailable` → wait and bind a fresh unit; +`agent_unconfigured` → replace the agent; always pass `--worktree` with `--terminal`. +Open PR #51 already carries all four rows on `pane-delivery-confirmation`, including the +same three field observations (busy-bind pending→failed, dead-agent recovery, worktree +mismatch). Nothing unique remained to fold. ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Pages read: platforms-processes-tool-diagnostics-without-a-failing-exit-code, backend-common-api-design-unenforced-declarations, qa-process-release-gates, infrastructure-ci-cd-pipeline-structure, qa-exploratory-lowered-declaration-survival, infrastructure-agent-orchestration-worktree-isolated-workers, infrastructure-agent-orchestration-pane-delivery-confirmation + +- Routing candidates for insight 1 were qa/process (release-gates: release checklists — + wrong altitude), infrastructure/ci-cd (pipeline-structure: stage ordering — wrong + concern), and platforms/processes. `tool-diagnostics-without-a-failing-exit-code` + already owns this exact gate: its Do-5 recommends the `-Werror`/`--strict` promotion + and Do-6 proves three states (warning/clean/error). The insight is the missing fourth + state of that same adoption check → **merged** there (Do-5 caution + Do-6 fourth + control input + 1 edge-case row + 1 Instead-of row + 3 sources), no new page. +- No conflicts: the page's existing directives stand; the merge narrows when the + promotion switch is safe rather than contradicting it. +- Related-links added both ways with `backend-common-api-design-unenforced-declarations` + — its "accept and warn" shape is exactly the intentional diagnostic that collides + with a blanket gate. Platforms domain index "load when" line extended accordingly. +- Insights 2 and 3 were checked against `worktree-isolated-workers` and + `pane-delivery-confirmation` (merged state + open-PR diffs) — covered there, see + Open-PR check. + +## Open-PR check + +Open `knowledge/*` heads listed via `gh pr list` at flush time: +#55 (choiyounggi-20260807-144058), #52 (dch0202-rsquare-20260807-100149), +#51 (dch0202-20260806-183029), #50 (dch0202-20260806-172420), +#49 (dch0202-rsquare-20260806-142309), #47 (dch0202-20260806-130040). + +- Insight 1 (warnings-as-errors): diffed #51 and #47 fully; file lists of #55/#52/#50/#49 + checked via `git diff --name-only` / `gh api pulls/N/files` — none touches + `wiki/platforms/processes/tool-diagnostics-without-a-failing-exit-code.md` or carries + an overlapping trigger. Verdict: **new** (ingested). +- Insight 2 (worktree_escape read-only escalation): #47 adds the same directive and the + same session evidence to `worktree-isolated-workers`; #51 adds a refined version (rule + mechanism + budget-the-round-trip row). Verdict: **drop**. +- Insight 3 (dispatch-binding taxonomy): #51 adds all four rows + the same three field + observations to `pane-delivery-confirmation`. Verdict: **drop**. + +Note for review ordering: #47 and #51 both amend `worktree-isolated-workers`'s Do-this +table near the same rows — whichever merges second will need a trivial conflict +resolution (both versions are compatible; #51's is the more precise). ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +- Insight 1 → `platforms/processes/tool-diagnostics-without-a-failing-exit-code` + (merge, no new page, no new category). The harvested `domain: qa` hint was re-routed: + the merged wiki already holds the owning page for this gate under platforms/processes, + and merge-before-create outranks the hint (precedent: 2026-08-04 keg-only re-route in + log.md). qa/process/release-gates covers release checklists, not diagnostic-gate + adoption mechanics, so no qa page was created. +- Insights 2, 3 → no target; retired from the queue as pending duplicates of open PRs + #47/#51 (their would-have-been targets are the two agent-orchestration pages named + above, where the content already sits). diff --git a/log.md b/log.md index c930fc2..ac73d50 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] &1 >/dev/null) a hard gate**: `-Werror` (clang/gcc), `--max-warnings 0` (ESLint), `--strict` equivalents. Then the exit code carries the decision again and the wrapper stays a one-liner. Use stream capture when you must keep warnings non-fatal - for humans while still surfacing them to the loop. - -6. **Prove all three states before adopting the gate.** Run it on a file with a - warning, a clean file, and a file with a real error, and require the three - distinct outcomes above. Keep the control inputs in the repo so contributors + for humans while still surfacing them to the loop. Before promoting, check + the tool has per-diagnostic severity control (`-Werror=`/`-Wno-error=`, + explicit lint selection) — a blanket promotion also converts *intentional* + warnings (deprecations, accepted-but-unenforced declarations) into failures. + +6. **Prove all the states before adopting the gate.** Run it on a file with a + warning it should catch, a clean file, a file with a real error — and a + **valid file that legitimately warns** (a deprecation, a documented + declaration the runtime accepts-and-warns on), requiring a distinct decided + outcome for each. Keep the control inputs in the repo so contributors cannot drift the gate ([testing-quality-checks-that-cannot-pass]). ## Edge cases @@ -79,6 +86,7 @@ OUT=$(tool "$FILE" 2>&1 >/dev/null) | The tool offers `-Werror` / `--max-warnings 0` | Use it **in addition** — it converts the status, and the captured text is still what names which warning fired | | The wrapper runs under `set -e` | Command substitution failure inside `OUT=$(…)` is not suppressed by a condition context — assign first, test after, as above | | Warnings must not repeat on every run of an unchanged file | Hash `$OUT` per file and forward only on change; an unconditional exit 2 re-feeds the same text each time | +| The platform warns on valid, intended usage (deprecations, declarations it accepts but does not enforce) and offers **no per-diagnostic severity control** | The promotion switch and that feature are mutually exclusive — every legitimate use fails the gate, so adopting the feature means turning the gate off. Keep warnings non-fatal via stream capture with shape-matching, and record the missing severity tiers as a platform defect in its own right | ## Instead of @@ -89,6 +97,7 @@ OUT=$(tool "$FILE" 2>&1 >/dev/null) | Trust "the gate stayed quiet" as proof the file is clean | Run the gate once against a file you know produces a warning and require it to fire | A gate keyed on exit status is silent for both the clean case and the warning case | | Exit 1 with the diagnostics on stderr | Exit 2 | For `PostToolUse` only exit 2 shows stderr to Claude; other codes surface one line to the user as a hook error | | Put diagnostics on stdout and exit 0 | Print to stderr and exit 2 | On exit 0 `PostToolUse` stdout goes to the debug log, not the transcript — the model never sees it | +| Adopt `-Werror`/`--strict` after proving only "catches a bad file, passes a clean file" | Also run a valid file that legitimately warns | The two-way check cannot see the third input class; without diagnostic severity tiers, a typo-shaped no-op and an intentional declaration both return the same failing status | ## Sources @@ -98,3 +107,6 @@ OUT=$(tool "$FILE" 2>&1 >/dev/null) - https://code.claude.com/docs/en/hooks — exit 2: "stderr text is fed back to Claude as an error message"; `PostToolUse` cannot block ("the tool already ran") but shows stderr to Claude - Field reproduction 2026-08-05: compiler with warning input → exit 0 with 3 warnings on stderr; clean input → exit 0 with empty stderr; error → non-zero exit with errors; the three outcomes observed directly when the gate was tested - Local reproduction 2026-08-06 (macOS, Apple clang): `cc -Wall` on a snippet with an unused variable → exit 0, 157 bytes on stderr; `OUT=$(cc … 2>&1 >/dev/null)` captured the diagnostic while the reversed redirection order captured 0 chars +- https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html — `-Werror=` / `-Wno-error=`: per-warning error promotion and exemption; "more specific options have priority over less specific ones" +- https://rust-unofficial.github.io/patterns/anti_patterns/deny-warnings.html — blanket `#![deny(warnings)]` is an anti-pattern because "APIs get deprecated, so their use will emit a warning where before there was none"; recommends explicit lint selection that deliberately excludes `deprecated` +- Field reproduction 2026-08-05→07 (lnpl 0.2.0 QA re-measurement, `--strict`): unknown-verb file → rc=2 (caught), clean file → rc=0 (no false positive), and a legitimate `on schedule` declaration → rc=2 anyway, because the runtime's accept-and-warn "declared, not enforced" diagnostic has no severity class below the gate's threshold — the two-way check alone would have missed the collision From 1e227a27e8736c63f8782968e69d742d20d6a68c Mon Sep 17 00:00:00 2001 From: choiyounggi <74581798+choiyounggi@users.noreply.github.com> Date: Fri, 7 Aug 2026 16:44:21 +0900 Subject: [PATCH 11/61] knowledge: ingest 1 verified insight (SIGINT delivery to backgrounded processes under test) --- .dev-loop/INGEST_REPORT.md | 142 ++++++++++++------ log.md | 1 + .../non-interactive-cli-invocation.md | 2 +- wiki/testing/index.md | 1 + ...signal-delivery-to-a-process-under-test.md | 66 ++++++++ 5 files changed, 167 insertions(+), 45 deletions(-) create mode 100644 wiki/testing/strategy/signal-delivery-to-a-process-under-test.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..a2f42e1 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,107 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state - -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +# Knowledge flush — 3 insight(s): 1 ingested, 2 dropped as in-flight duplicates ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +**1. SIGINT delivery to `&`-backgrounded processes in non-interactive shells (INGESTED)** + +Claim: a test harness that starts a program with `cmd &` from a shell script and +sends `kill -INT "$pid"` never delivers an effective SIGINT — POSIX requires +job-control-disabled shells to start asynchronous-list commands with SIGINT and +SIGQUIT set to SIG_IGN, so the kill succeeds and the child discards the signal; +`wait` then hangs to the harness timeout and the failure reads like a product +bug. Deliver the signal from a subprocess driver instead (e.g. +`subprocess.Popen` + `send_signal(SIGINT)`), or `set -m` in the script, or use +SIGTERM when the contract is generic shutdown. + +Sources checked: +- https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html — + §2.11 Signals and Error Handling, verbatim: "If job control is disabled … + when the shell executes an asynchronous list, the commands in the list shall + inherit from the shell a signal action of ignored (SIG_IGN) for the SIGINT + and SIGQUIT signals." (fetched and grepped the spec text directly) +- https://www.gnu.org/software/bash/manual/bash.html#Signals — §3.7.6, verbatim: + "When job control is not in effect, asynchronous commands ignore SIGINT and + SIGQUIT in addition to these inherited handlers." +- https://docs.python.org/3/library/subprocess.html#subprocess.Popen.send_signal — + anchor existence verified by fetch. + +Local reproduction (2026-08-07, macOS): `sh|bash|zsh -c ' & wait'` all report SIG_IGN for the background child and the +default handler in the foreground; with `set -m` the background child reports +the default handler; SIGTERM reports SIG_DFL in both. Matches the harvested +field case (server binary: rc 143 timeout under `kill -INT`+`wait`, immediate +rc 0 under Popen+send_signal). + +Confidence: **verified**. + +**2. worktree_escape guardrail escalation budgeting (DROPPED — in-flight duplicate)** + +Claim: the worktree_escape guardrail escalates read-shaped cross-worktree +access, so orchestration briefs referencing other worktrees must budget the +escalation round trip. Not re-verified here — open PR #51 already carries this +exact insight with a stronger local reproduction (see Open-PR check). + +**3. Orca terminal bind-failure stage branching (DROPPED — in-flight duplicate)** + +Claim: check for the idle prompt before binding the next unit to a worker's +terminal; branch on the failure stage (runtime_unavailable → wait and rebind, +agent_unconfigured → replace the agent; always pass --worktree with +--terminal). Not re-verified here — open PR #51 already carries all four rows +with the same field evidence. ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed via INDEX.md → testing domain. Read the full testing domain index (all +"load when" lines) — no existing page covers signal delivery or process +lifecycle in tests; nearest neighbors (diagnosing-flaky-tests, async-testing, +test-level-choice) have non-overlapping triggers. Checked platforms as the +alternative home: background-services covers process persistence (nohup/launchd/ +systemd), non-interactive-cli-invocation covers prompt-capable CLIs hanging — +neither covers signal dispositions of `&` jobs, so this is a new trigger → +new page (merge-before-create satisfied: nothing to merge into). Repo-wide +`grep -ril sigint wiki/` confirmed no page mentions the mechanism. + +Created: `wiki/testing/strategy/signal-delivery-to-a-process-under-test.md` +(confidence verified, 3 sources + local repro). Updated `wiki/testing/index.md` +(strategy table row) and `log.md`. Related-links added both ways with +platforms-processes-non-interactive-cli-invocation (same interactive-vs- +automation divergence family). No conflicts with existing directives. + +Pages read: platforms-processes-background-services, platforms-processes-non-interactive-cli-invocation, infrastructure-agent-orchestration-pane-delivery-confirmation, infrastructure-agent-orchestration-worktree-isolated-workers + +## Open-PR check + +Open `knowledge/*` heads listed via `gh pr list` and each diffed against main +(`git diff origin/main origin/ -- wiki/`; fork heads #52/#49 fetched via +`pull//head`): + +- #56 (choiyounggi-20260807-153857), #55 (choiyounggi-20260807-144058), + #50 (dch0202-20260806-172420), #47 (dch0202-20260806-130040), + #52 (dch0202-rsquare-20260807-100149), #49 (dch0202-rsquare-20260806-142309): + no overlap with any of the 3 candidates (testing-quality, ML, platforms + pages; nothing touches signals or the two orchestration insights). +- #51 (dch0202-20260806-183029): **overlaps candidates 2 and 3 completely.** + Its worktree-isolated-workers.md diff contains the read-only escalation + budgeting row verbatim ("Budget the round trip … state in the worker's first + brief that reads are approved") plus a stronger reproduction showing pure + reads pass and ask fires only when a write verb/absolute redirect co-occurs + — a correction that supersedes candidate 2's broader claim. Its + pane-delivery-confirmation.md diff contains all of candidate 3: idle-prompt + check before binding, runtime-unavailable → wait, agent-unconfigured → + replace agent, pane/worktree mismatch → pass worktree, same i43/i45 field + evidence. + +Verdicts: candidate 1 → **new** (ingested here); candidate 2 → **drop** +(pending duplicate of #51, nothing unique to fold); candidate 3 → **drop** +(pending duplicate of #51, nothing unique to fold). No sibling duplicate PR +opened. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +- Candidate 1 → `testing/strategy/signal-delivery-to-a-process-under-test`. + Domain: the trigger is test-harness-shaped (harvester hint: testing) and the + testing INDEX line "writing or structuring automated tests" matches; category + `strategy` because the directive chooses the harness/driver structure, not an + assertion. No new category needed. The shell mechanism is cross-referenced to + platforms via the related-link rather than a second page (one case, one page). +- Candidates 2–3 → no routing; retired as pending duplicates of open PR #51. diff --git a/log.md b/log.md index c930fc2..c4c9365 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] & wait'` reports SIG_IGN for SIGINT in the background child and the default handler in the foreground; with `set -m` the background child reports the default handler; SIGTERM reports SIG_DFL in both. Field case same day: a server binary timed out (rc 143) under `kill -INT $pid; wait` from a script, and exited 0 immediately under `subprocess.Popen` + `send_signal(SIGINT)` From 459c37b41afe089eb4f610edfc71573593164aa6 Mon Sep 17 00:00:00 2001 From: choiyounggi <74581798+choiyounggi@users.noreply.github.com> Date: Fri, 7 Aug 2026 19:18:55 +0900 Subject: [PATCH 12/61] knowledge: ingest 1 verified insight (corpus sweep before a rejection rule); drop 2 in-flight duplicates of #51 --- .dev-loop/INGEST_REPORT.md | 181 +++++++++++++----- log.md | 1 + .../change-impact/call-site-enumeration.md | 2 +- .../corpus-sweep-before-a-rejection-rule.md | 88 +++++++++ wiki/backend/index.md | 1 + wiki/qa/process/regression-scope.md | 2 +- 6 files changed, 230 insertions(+), 45 deletions(-) create mode 100644 wiki/backend/common/change-impact/corpus-sweep-before-a-rejection-rule.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..0791aeb 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,148 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 3 insight(s): 1 ingested, 2 dropped as in-flight duplicates -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: `~/.dev-loop/queue/7947837a-…jsonl` (1 row), `~/.dev-loop/queue/f1a3ae46-…jsonl` (2 rows). ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### C1 — Bound a new rejection rule with a corpus sweep before writing production code (INGESTED) + +**Claim.** When adding a rule to a compiler/linter/parser/validator that will +start rejecting input the tool has been accepting silently, implement the +accept/reject predicate first as a throwaway script, run it over the entire +corpus, and record the reject count *and the rejected-path list* in the plan. +The rule is only settled when that list equals the set already known to be +defective. + +**Sources checked (all fetched this session):** + +- — the tool exists to answer + "Does the rule report the intended patterns? Does the rule falsely mark valid + patterns as errors?", because "the AST of Javascript and Typescript can cause + very unexpected results it is not enough to test the rule only against unit + tests and a small amount of repositories". Comparison mode reports "the exact + changes in ESLint reports their code changes introduced". +- — + lintcheck "Runs Clippy on a fixed set of crates read from + `lintcheck/lintcheck_crates.toml` and saves logs of the lint warnings into the + repo. We can then check the diff and spot new or disappearing warnings." + (fetched via `raw.githubusercontent.com`; the cited `blob/` URL is the same file.) +- — "Crater is a tool to run experiments + across parts of the Rust ecosystem. Its primary purpose is to detect regressions + in the Rust compiler, and it does this by building a large number of crates, + running their test suites and comparing the results between two versions of the + Rust compiler." Used specifically to measure the extent of breakage of a + potentially breaking change (e.g. a lint becoming deny-by-default) before it lands. + +**How verified / what is *not* sourced.** Three independent ecosystem tools +implement the same method — sweep the corpus, record verdicts, diff them — which +establishes the core directive. What the three tools do *not* establish is the +candidate's ordering refinement: they implement the rule in the production tree +first and then sweep. The "throwaway predicate *before* production code, and the +reject list must match the known-defective set" part rests on the session's own +reproducible field run (linkly #53: 148 sources swept — 40 `.lnpl` files + 108 +triple-quoted inline programs in tests — 2 rejects, both the QA probes the issue +named; one false positive in the first draft rule caught at plan time, where the +guard legitimately owned its block). The page's Sources section says this in a +final bullet rather than implying the ordering is doc-backed. + +**Confidence: `verified`** — method sourced to official tool docs, ordering +refinement backed by a reproducible measurement. Nothing was upgraded on the +strength of assertion alone. + +The candidate's own honest limitation is preserved as an edge-case row: the text +sweep could not index fixtures assembled with `.replace()`, so those 5 sites were +verified by hand. That partial-index failure is the same shape as the existing +`call-site-enumeration` page's keyword-search blind spot, which is why the two +are cross-linked. + +### C2 — guardrail `worktree_escape` fires on read-only cross-worktree access (DROPPED) + +Not ingested; superseded in flight (see Open-PR check). Verification note for the +record: the candidate's premise is **partially wrong**. PR #51 carries a local +reproduction (guardrails 1.2.0, macOS) showing that a pure read of a sibling +worktree passes — `grep`, `awk`, `cat`, `git -C … status` all returned clean — +and that `ask` fires only when a main-root mention survives the strip *and* a +write verb (`rm|mv|cp|tee|mkdir|touch|install|dd`) or a redirect to an absolute +path appears anywhere in the same command string. Ingesting this candidate as +written would have contradicted a better-evidenced page. + +### C3 — pane/dispatch binding failure taxonomy (DROPPED) + +Not ingested; superseded in flight. PR #51's +`infrastructure/agent-orchestration/pane-delivery-confirmation.md` already carries +all four of this candidate's rows (check for the idle prompt before binding; the +runtime-unavailable stage → wait and bind a fresh unit; the agent-unconfigured +stage → close the pane and create a new worker-mode agent; always pass the +worktree alongside the pane), plus an `Instead of` row for branching on the stage, +sourced to the same three field dispatches this candidate cites and corroborated +against the shipped `orca-worker-start.sh` comment. ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed C1 via `INDEX.md` → `backend` ("language-agnostic `common/`: … call-site +enumeration before a contract change"), then `wiki/backend/index.md` → +`change-impact`. Cross-checked `testing` and `qa` indexes before settling, since +the harvested hint said `testing`. + +Pages read: backend-common-change-impact-call-site-enumeration, testing-quality-guard-shape-vs-consequence, qa-process-regression-scope, backend-common-api-design-unenforced-declarations, testing-quality-checks-that-cannot-pass, infrastructure-agent-orchestration-worktree-isolated-workers, infrastructure-agent-orchestration-pane-delivery-confirmation + +Index "load when" lines were read for every page in `wiki/testing/index.md`, +`wiki/qa/index.md` and `wiki/backend/index.md`; the pages above are the ones whose +lines overlapped and were opened in full. + +**Overlaps found, and why none of them absorbed C1:** + +| Page | Overlap | Verdict | +|------|---------|---------| +| `backend-common-change-impact-call-site-enumeration` | Same category and the same "state the method next to the count / treat a search as a partial index / re-run after the edit" discipline — but its trigger is a *callee contract* change and its unit is a call site | Distinct trigger (input corpus vs call sites) → new page, cross-linked both ways | +| `testing-quality-guard-shape-vs-consequence` | Both concern a rule going red on a legitimate artifact | Its trigger is a guard that is **already red**; C1's is a rule that does not exist yet. Linked one-way from C1's "When this applies" and from the exemption-list `Instead of` row | +| `qa-process-regression-scope` | Both bound the blast radius of a change | That page picks what to *re-test* for a release; C1 sizes a rule before it is written. Linked both ways | +| `backend-common-api-design-unenforced-declarations` | Adjacent decision — whether unimplemented declarative input should reject/warn/ignore | C1 starts *after* "reject" was chosen. Linked one-way (that page is edited by two open PRs; no reciprocal edit, to avoid a third conflicting hunk) | +| `testing-quality-checks-that-cannot-pass` | Cited for the "sweep rejects zero inputs" edge case | Reference only | + +**Conflicts flagged:** none. No merged page carries this trigger — a grep over +`wiki/` for `corpus sweep|previously-accepted|tightening a validator|rejection +rule` returned nothing. + +**Merged vs created:** created 1 new page (no existing page shares the trigger, +so merge-before-create does not apply). No new category — `change-impact` already +exists and its scope covers this. + +**Related links added:** `call-site-enumeration` ← → new page; +`regression-scope` ← → new page. + +**Conflict note for the reviewer:** `wiki/backend/index.md` is also touched by open +PR #51 (which adds a `widening-a-closed-value-table` row to the same +`change-impact` table). Expect a small table-level conflict if both land; the two +rows are independent. + +## Open-PR check + +Listed with `gh pr list --repo choiyounggi/dev-loop --state open --search "head:knowledge/"`, +then fetched each as `pull//head` and diffed `origin/main..pr -- wiki/`. + +Open heads: **#57, #56, #55, #52, #51, #50, #49, #47.** + +| Candidate | Overlapping head(s) | Verdict | +|-----------|--------------------|---------| +| C1 corpus sweep before a rejection rule | none — additive-line scan for `corpus\|sweep\|reject\|previously-accepted\|lint rule\|new rule\|throwaway\|tighten` across all 8 heads matched only unrelated contexts (#52: a `subprocess` encoding row and a mutant-triage line; #51: a pane-bind row, a README number sweep, and the `widening-a-closed-value-table` index line) | **new** — ingested here | +| C2 guardrail read-only escalation | **#51** — `wiki/infrastructure/agent-orchestration/worktree-isolated-workers.md` adds the row "The escalation is read-only in substance and the coordinator must clear it → Budget the round trip (read the recorded escalation → approve → clear `escalations/` → restart the watcher) and state in the worker's first brief that reads are approved and only writes outside the worktree are refused", plus three rows and a reproduction that correct the candidate's premise | **drop** — pending duplicate with nothing unique to fold; the open PR's form is strictly better (it fixes the read-vs-write claim) | +| C3 pane binding failure taxonomy | **#51** — `wiki/infrastructure/agent-orchestration/pane-delivery-confirmation.md` adds all four rows plus an `Instead of` row and a field-observation source line covering the same three dispatches | **drop** — pending duplicate, nothing unique to fold | + +No sibling duplicate PR was opened, and nothing was pushed to #51's branch (it +already carries the content in equal-or-better form, so there was nothing to add). ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| Insight | Target | New category? | +|---------|--------|---------------| +| C1 | `backend` / `change-impact` / [`corpus-sweep-before-a-rejection-rule`](../wiki/backend/common/change-impact/corpus-sweep-before-a-rejection-rule.md) — id `backend-common-change-impact-corpus-sweep-before-a-rejection-rule` | No. `change-impact` already means "enumerate the affected set before changing a contract"; a validator that starts rejecting previously-accepted input is a narrowing of that contract, and the corpus is the affected set. `testing` (the harvested hint) was rejected — that domain is for writing automated tests, and this is a plan-time measurement; `qa/process` was rejected — its regression scoping decides what to *re-test*, not how to bound a rule that has not been written | +| C2 | — | dropped (in-flight duplicate of #51) | +| C3 | — | dropped (in-flight duplicate of #51) | + +Files changed: `wiki/backend/common/change-impact/corpus-sweep-before-a-rejection-rule.md` (new, 75 body lines), +`wiki/backend/index.md` (+1 routing row), `wiki/backend/common/change-impact/call-site-enumeration.md` +(+1 `related:` id), `wiki/qa/process/regression-scope.md` (+1 `related:` id), `log.md` (+1 entry). + +All three queue rows are retired — the ingested one and both drops — so nothing +re-crosses the auto-flush threshold. diff --git a/log.md b/log.md index c930fc2..e8c9894 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,4 @@ Append-only. Format: `## [YYYY-MM-DD] Date: Fri, 7 Aug 2026 21:40:11 +0900 Subject: [PATCH 13/61] knowledge: ingest 2 verified insight(s) --- .dev-loop/INGEST_REPORT.md | 189 ++++++++++++++---- log.md | 2 + .../exploratory/guard-true-path-coverage.md | 2 +- .../data/harness-vs-run-path-fixtures.md | 78 ++++++++ wiki/testing/data/test-data-and-isolation.md | 2 +- wiki/testing/index.md | 1 + .../quality/differential-run-agreement.md | 2 +- .../quality/harness-reverse-controls.md | 2 +- wiki/testing/quality/spec-artifact-checks.md | 36 +++- 9 files changed, 262 insertions(+), 52 deletions(-) create mode 100644 wiki/testing/data/harness-vs-run-path-fixtures.md diff --git a/.dev-loop/INGEST_REPORT.md b/.dev-loop/INGEST_REPORT.md index 55ccfd1..c5f5baf 100644 --- a/.dev-loop/INGEST_REPORT.md +++ b/.dev-loop/INGEST_REPORT.md @@ -1,53 +1,156 @@ -# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state +# Knowledge flush — 4 insight(s) -The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the -harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75 -unique pages, with the same insight landing at up to 3 different paths across -up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the -#6–#13 consolidation — this branch carries the reconciled end-state and the 15 -PRs are closed in its favor. +Queue drained: 4 pending candidates across 3 session files. **2 ingested, 2 dropped** +as in-flight duplicates of open PR #51. ## Verified best-practice -Every adopted page's sources were carried from its originating PR's flush, where -they were live-verified at flush time; no new URLs were introduced during -consolidation (checked mechanically: every `http(s)` URL in every merged page -appears in a source PR's diff; every added body line in amended pages traces to -a source PR hunk — orphan-line verification). Confidence fields were kept as the -originating flushes set them, except client-side-rate-limiting where the union -of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports -`verified` for the load-bearing claims. One subagent's fabricated content (12 -files matching neither main nor any PR, with invented source URLs) was detected -by the same verification and replaced with true PR content. +### 1. Harness-vs-run-path fixture synthesis (`cd4826874fdb6acc`) — **verified** + +**Claim.** When a spec/test harness and the production entry point each synthesize +the program's input from different declaration sets, a guard whose operand is absent +from the harness's narrower fixture evaluates not-true and the run reports +"guard false / step skipped, exit 0" — which reads as a defect in the program under +test but is an artifact of the harness's fixture scope. + +**Sources checked.** + +- https://www.postgresql.org/docs/current/functions-comparison.html — "Ordinary + comparison operators yield null (signifying "unknown"), not true or false, when + either input is null. For example, `7 = NULL` yields null"; and + `NULL::boolean IS TRUE` → `f`. An unknown operand lands on the not-true side, + where it is indistinguishable from a present-and-false operand. +- https://jqlang.org/manual/ — `.foo` on an object lacking that key produces `null`; + `if` takes the else branch "if A produces a value other than false or null". + Same collapse in a second, unrelated expression language. +- https://istqb-glossary.page/branch-coverage/ — "The percentage of branches that + have been exercised by a test suite" — grounds the "a skipping run is evidence + about the skip direction only" directive. + +**How verified.** Two reproductions run this session, not just cited: + +- Local (Python 3.14.6 + jq + `/usr/bin/jq`, macOS): one guard `shipment.ready` over a one-entity fixture + returned `False`, over the full fixture returned `True`, and the one-entity result + compared *equal* to the result on an explicit `{"shipment": {"ready": false}}` — + the two causes are one observable. `echo '{"order":{}}' | jq '.shipment.ready'` + → `null`, and both inputs took the `else` branch. +- Field (carried by the candidate, lnpl spec runner): the runner built its payload + from `sample_payload([first_entity])` while `run`/`diff`/mode-B used + `sample_payload(all_entities)`; injecting only the missing second-entity field + moved the same workflow from `STATUS: completed STEPS: 1 SKIPPED: 1` to + `STEPS: 2 SKIPPED: 0` with no other change. Three separately-filed major bugs + traced to that one narrowing. + +**Confidence: verified.** The semantic mechanism is doc-confirmed in two languages +and locally reproduced; the design directive (one synthesizer, explicit +author-overridable narrowing) is supported by the field before/after. Recorded as +`verified` with both reproductions written into the page's Sources so a reader can +re-run them. + +### 2. Negative-control anchor rot in a doc-as-spec gate (`52dcf45adfc479f9`) — **verified** + +**Claim.** When a gate's negative control builds its mutant by quoting the whole +target row — key cells and prose cells — an ordinary rewording of the prose stops +the match; the substitution returns the document unchanged rather than raising, the +check runs against the *original* and passes as it should, and the control reports +green while mutating nothing. Anchor on the key-column prefix and assert the +mutation applied. + +**Sources checked.** + +- https://docs.python.org/3/library/stdtypes.html — `str.replace`: "Return a copy of + the string with all occurrences of substring *old* replaced by *new*." The + documented signature carries no not-found error and no substitution count. (The + docs do not state the not-found case explicitly, so it was reproduced rather than + cited — see below.) `str.count` returns `0` for an absent substring. +- https://docs.python.org/3/library/re.html — `re.subn` "Perform the same operation + as `sub()`, but return a tuple `(new_string, number_of_subs_made)`" — the counted + form that makes "it applied" checkable without a separate assertion. +- Already-cited in the target page and re-used: https://pitest.org/ (a mutant is + killed when a test fails) and https://eslint.org/docs/latest/integrate/nodejs-api + (an `invalid` case must declare the errors it expects). + +**How verified.** Local reproduction 2026-08-07 (Python 3.14.6, macOS): a control +quoting `| security | jwt | rejects unsigned tokens |` against a document whose cell +had been reworded to `… at the edge` produced `mutant == doc` — mutation not +applied, no exception raised. The same mutation anchored on the prefix +`| security | jwt |` applied. `re.subn` reported `0` substitutions for the stale +anchor and `1` for the prefix pattern. Field origin: linkly +`impl/tests/test_enforcement_matrix.py::test_a_short_row_raises_rather_than_reading_as_empty_cells`, +where only `assertNotEqual(mutant, original, "the mutation did not apply")` +surfaced it; re-anchoring returned the suite to 37/37. + +**Confidence: verified.** + +### 3–4. The two dropped candidates + +`28fd6dfedea1b338` (worktree guardrail escalation on read-only cross-worktree +access) and `ba3b56aded3e6191` (orca terminal-binding failure taxonomy) were not +independently re-verified, because open PR #51 already carries both with *stronger* +evidence — see **Open-PR check**. Notably #51's reproduction **corrects** the +premise of `28fd6dfedea1b338`: the candidate asserts the guardrail fires on +read-only access, while #51 reproduced that plain reads of a sibling worktree +(`grep`, `awk`, `cat`, `git -C … status`) all pass, and the rule fires only when a +write verb or an absolute-path redirect co-occurs anywhere in the same command +string. Ingesting the candidate as written would have contradicted a better-sourced +open PR. ## Existing-layer check -- Merged-main near-dup scan before consolidation: pairwise Jaccard over - title + "When this applies" across all 141 merged pages → **0 flagged pairs**; - previously merged content carries no duplication. -- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one - canonical page each (rate limiting 8→1, call-site enumeration 7→folded into - the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1, - env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1, - harness-mediated results 2→1, leaked artifacts 2→1, orchestration category - naming unified). Three near-pairs kept distinct after trigger comparison, - with mutual `related:` links (differential setup vs interpretation; expansion - semantics vs off-switch design; import-time tactics vs level choice). -- 24 existing pages received union-merged amendments; additions already present - in main (from #16/#20) were skipped, and all non-canonical `related:` ids - were remapped to canonical page ids (post-merge broken-link scan: 0). +Routed via `INDEX.md` → `testing` (both ingested candidates concern automated test +code, not release process). Read `wiki/testing/index.md` and `wiki/qa/index.md` +in full, then every page whose "load when" line overlaps. + +Pages read: testing-quality-spec-artifact-checks, testing-quality-harness-reverse-controls, testing-quality-differential-run-agreement, testing-data-test-data-and-isolation, qa-exploratory-guard-true-path-coverage, backend-common-change-impact-call-site-enumeration + +Overlaps found and resolution: + +| Candidate | Page examined | Verdict | +|-----------|---------------|---------| +| Fixture parity | `testing-quality-differential-run-agreement` | Adjacent, not the same. It governs *citing an agree verdict* when the two sides model different state; this candidate is about the two sides being fed different *synthesized input*, and about attributing a skip. Cross-linked both ways, created new. | +| Fixture parity | `qa-exploratory-guard-true-path-coverage` | Closest existing page — it already says a guard-skipping run is evidence about the skip path only. It does **not** cover *why* the guard was false (an absent operand) or the fixture-parity fix. New page carries the causal/diagnostic half and defers the coverage rule to that page by id. Cross-linked both ways. | +| Fixture parity | `testing-data-test-data-and-isolation` | Owns fixture ownership/isolation; its closest row is "a factory whose shape depends on a value the test also passes to the code under test". Different case (a value vs. the declaration *scope*, and no diagnostic half). Cross-linked, created new rather than stretching that page's trigger. | +| Anchor rot | `testing-quality-spec-artifact-checks` | **Same trigger** — it already owns "one negative control per check in a doc-as-spec repo" and Markdown-table row parsing, but says nothing about how to *locate* the mutation site or that the control must prove it mutated. **Merged** (merge-before-create). | +| Anchor rot | `testing-quality-harness-reverse-controls` | Related but harness-level: it covers a *uniform* verdict across the whole harness ("every case survives → the harness never applied the mutation"). This candidate is a *per-control* rot that leaves the other controls working, so the harness-level uniformity signal never fires. Added the reciprocal `related:` link; content stayed in spec-artifact-checks. | + +Conflicts flagged: none among the ingested pages. One conflict *avoided* — see the +`28fd6dfedea1b338` note above (candidate premise vs. PR #51's reproduction); +resolved by dropping the candidate, not by overwriting anything. + +Related-links added (both directions): the new page ↔ `testing-data-test-data-and-isolation`, +`testing-quality-differential-run-agreement`, `qa-exploratory-guard-true-path-coverage`; +plus `testing-quality-harness-reverse-controls` → `testing-quality-spec-artifact-checks`. + +## Open-PR check + +Open `knowledge/*` heads listed at flush time: **#58, #57, #56, #55, #52, #51, #50, +#49, #47**. Fetched and diffed against `origin/main` for the heads whose titles +overlapped any candidate's trigger — #51 (`knowledge/dch0202-20260806-183029`), +#58 (`knowledge/choiyounggi-20260807-191239`), #57 (`knowledge/choiyounggi-20260807-163902`). + +| Candidate | Overlapping open PR | Verdict | +|-----------|--------------------|---------| +| `cd4826874fdb6acc` — harness fixture narrower than the run path | none (#58 touches `backend/common/change-impact`; #57 touches `testing/strategy/signal-delivery-to-a-process-under-test`; #51 touches orchestration/security/qa) | **new** — ingested | +| `52dcf45adfc479f9` — negative-control literal anchor rot | none (no open head touches `testing/quality/spec-artifact-checks.md`) | **new** — ingested (merged into the existing page) | +| `28fd6dfedea1b338` — worktree guardrail, read-only escalation | **#51**, `wiki/infrastructure/agent-orchestration/worktree-isolated-workers.md` (+4 rows, +1 dated local reproduction) | **drop** — the open PR carries the same directive ("Budget the round trip … and state in the worker's first brief that reads are approved") plus a reproduction that corrects the candidate's mechanism claim. Nothing unique to push. | +| `ba3b56aded3e6191` — orca terminal-binding taxonomy | **#51**, `wiki/infrastructure/agent-orchestration/pane-delivery-confirmation.md` (+4 Do rows, +1 Instead-of row, +1 dated field-observation source) | **drop** — 1:1 coverage of all four sub-claims (idle-prompt check before binding; worker-done is a report, not the end of the turn; runtime-unavailable → wait and bind a fresh unit; agent-unconfigured → close the pane and create a worker-mode agent; worktree passed alongside the pane). Nothing unique to push. | + +Both dropped candidates are the same pair that flushes **#56, #57 and #58** each +dropped as in-flight duplicates of #51 — they keep re-crossing the auto-flush +threshold because #51 has not merged. They are being retired to +`.processed.jsonl` in this flush (step 5), so they will not re-queue. ## Routing decision -- New categories: `infrastructure/agent-orchestration` (5 pages; unified the - competing `orchestration`/`agent-orchestration` names), `databases/data-survey` - (1), `qa/deliverables` (1). All other pages route into existing categories. -- Canonical-path decisions: rate limiting → `backend/common/reliability/` - (sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr - diagnostics → `platforms/processes/` (concern spans beyond shells); leaked - artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site - enumeration → the existing `backend/common/change-impact/` page. -- All 38 new pages listed in their domain indexes (nearest-index rule; backend - routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain - summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter, - ids, related-links, index coverage, size, qualifiers, staleness → 0 findings. +| Insight | Target | New category? | +|---------|--------|---------------| +| Harness-vs-run-path fixture synthesis | **new page** `testing/data/harness-vs-run-path-fixtures.md` (`testing-data-harness-vs-run-path-fixtures`) | No — `testing/data` already exists and owns fixture construction. The case is a distinct trigger from `test-data-and-isolation` (synthesis parity + skip attribution vs. ownership/isolation), so it is a page, not a row. | +| Negative-control anchor rot | **merged** into `testing/quality/spec-artifact-checks.md` — new `Do this` step 3 (anchor by key-column prefix + assert `mutant != original`; `re.subn` count as the alternative), 3 edge-case rows, 1 `Instead of` row, 2 sources, `last_verified` → 2026-08-07 | No | +| `28fd6dfedea1b338`, `ba3b56aded3e6191` | not routed — dropped to `.processed.jsonl` | — | + +Plumbing updated: `wiki/testing/index.md` (`## data` gains the new page with its +"load when" line), `log.md` (one `ingest` entry, one `dedup` entry). Page body +lengths: new page 65 lines, `spec-artifact-checks` 115 lines — both under the +120-line limit. All `related:` ids in every touched page resolve against `wiki/` +(checked mechanically). diff --git a/log.md b/log.md index c930fc2..11dc435 100644 --- a/log.md +++ b/log.md @@ -43,3 +43,5 @@ Append-only. Format: `## [YYYY-MM-DD]