Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 36 additions & 44 deletions .dev-loop/INGEST_REPORT.md
Original file line number Diff line number Diff line change
@@ -1,53 +1,45 @@
# Knowledge consolidation — 15 open PRs (#17–#40) → one reconciled state

The 15 open `knowledge/*` PRs (created 2026-08-04 → 2026-08-05, before the
harvest processed-store dedupe fix in #41) contained 123 file-versions of ~75
unique pages, with the same insight landing at up to 3 different paths across
up to 8 PRs. Per-PR review would re-import those duplicates, so — as with the
#6–#13 consolidation — this branch carries the reconciled end-state and the 15
PRs are closed in its favor.
# Knowledge flush — 3 insight(s): 2 ingested, 1 folded into open PR #80

## Verified best-practice

Every adopted page's sources were carried from its originating PR's flush, where
they were live-verified at flush time; no new URLs were introduced during
consolidation (checked mechanically: every `http(s)` URL in every merged page
appears in a source PR's diff; every added body line in amended pages traces to
a source PR hunk — orphan-line verification). Confidence fields were kept as the
originating flushes set them, except client-side-rate-limiting where the union
of provider-doc citations (Okta, Auth0, GitHub, OpenAI, RFC 6585) supports
`verified` for the load-bearing claims. One subagent's fabricated content (12
files matching neither main nor any PR, with invented source URLs) was detected
by the same verification and replaced with true PR content.
**1. pytest caplog log-message assertions must use `record.getMessage()` / `caplog.messages`, never `record.message % record.args`** — INGESTED, confidence: **verified**.
- Claim: pytest's capture handler formats each record at capture time, so `record.message` is already `msg % args`; re-applying `% record.args` raises `TypeError: not all arguments converted during string formatting` on any parameterized log call, while argument-less calls make the faulty pattern a silent no-op (which is how it survives in suites).
- Sources checked:
- https://docs.python.org/3/library/logging.html#logrecord-attributes — `message` "computed as msg % args … set when Formatter.format() is invoked"; `getMessage()` "merging any user-supplied arguments with the message".
- https://docs.pytest.org/en/stable/reference/reference.html#pytest.LogCaptureFixture.messages — "A list of format-interpolated log messages … all interpolated", recommended for exact comparisons.
- https://github.com/pytest-dev/pytest/blob/main/src/_pytest/logging.py — `LogCaptureHandler.emit` appends the record and calls `StreamHandler.emit` (which formats, setting `record.message`); verified against current source.
- How verified: fresh local reproduction (Python 3, stdlib logging with a formatting capture handler): parameterized record → `record.message` fully interpolated, `message % args` → the exact TypeError; argument-less record → silently passes. Matches the harvesting session's pytest run (`uv run --extra dev pytest tests/test_gh.py -q`, fail→pass flip after switching to `getMessage()`).

**2. A passing page-load login probe does not validate direct API auth under refresh-token cookie auth** — INGESTED, confidence: **verified**.
- Claim: browser navigation triggers the server's refresh middleware (short-lived access token rotated via Set-Cookie, persisted by the browser context), so a page-load "logged in" check passes while a script replaying a stored cookie jar sends the stale access token and fails; preflight the API's identity endpoint (GraphQL `currentUser`) with the operation's own client/cookie jar, and refresh + persist cookies on failure.
- Sources checked:
- https://github.com/velopert/velog-server/blob/master/src/lib/token.ts — read the actual middleware: `setTokenCookie` sets `access_token` maxAge 1h beside `refresh_token` maxAge 30d; `consumeUser` refreshes on expired/near-expiry access tokens and returns rotated cookies via Set-Cookie.
- How verified: primary-source code read (above) + session reproduction: `check-login.mjs` → `STATUS:LOGGED_IN` immediately followed by publish failure "Not logged in"; direct `v3.velog.io/graphql` `currentUser` with the same stored cookies → null; re-running the login flow (rewriting the cookie store) made the same publish succeed. Page generalizes the directive; velog specifics kept as the worked example and edge cases.

**3. tmux prompt injection: "delivered" return with the prompt stuck as `[Pasted text #N]` — send Enter separately and re-verify** — NOT re-ingested (fold, see Open-PR check). The insight is real and was independently reproduced this time (pdfsum1 run, 3 sessions), but an open PR already carries it in equal-or-better form.

## Existing-layer check

- Merged-main near-dup scan before consolidation: pairwise Jaccard over
title + "When this applies" across all 141 merged pages → **0 flagged pairs**;
previously merged content carries no duplication.
- Cross-PR dedup during consolidation: 10 duplicate clusters collapsed to one
canonical page each (rate limiting 8→1, call-site enumeration 7→folded into
the canonical merged in #20, stderr/exit-0 diagnostics 4→1, sysroot 2→1,
env-off-switch 2→1, completion predicates 2→1, robots.txt 2→1,
harness-mediated results 2→1, leaked artifacts 2→1, orchestration category
naming unified). Three near-pairs kept distinct after trigger comparison,
with mutual `related:` links (differential setup vs interpretation; expansion
semantics vs off-switch design; import-time tactics vs level choice).
- 24 existing pages received union-merged amendments; additions already present
in main (from #16/#20) were skipped, and all non-canonical `related:` ids
were remapped to canonical page ids (post-merge broken-link scan: 0).
Pages read: infrastructure-agent-orchestration-pane-delivery-confirmation, testing-quality-tests-that-cannot-fail, debugging-methodology-hypothesis-testing, infrastructure-agent-orchestration-control-signals-vs-primary-artifacts

- Routed via root `INDEX.md`, then read `wiki/infrastructure/index.md`, `wiki/testing/index.md`, `wiki/debugging/index.md` in full; grepped all of `wiki/` for `caplog|getMessage|LogRecord`, `refresh token|access token|cookie`, `pasted|send-keys|Enter`.
- **Candidate 1 (caplog):** no existing testing page covers log-content assertions (grep hit only backend/common/api-design/error-responses.md, unrelated). New trigger → new page `testing/quality/captured-log-message-assertions.md`. Related-link added both ways with `tests-that-cannot-fail` (the zero-arg latent-pass is exactly its "green history proves nothing" mechanism).
- **Candidate 2 (auth probe):** no debugging page covers a passing probe contradicting a failing operation; `hypothesis-testing`/`reproduce-first`/`verify-the-fix` own adjacent but distinct triggers. New trigger → new page `debugging/methodology/probe-path-vs-operation-path.md`. Related-links added both ways with `hypothesis-testing` and `control-signals-vs-primary-artifacts` (shared theme: a signal is evidence only about the path that produced it). No conflicts with existing directives found.
- **Candidate 3 (tmux):** `pane-delivery-confirmation` already owns the trigger (its multi-line-prompt edge row mandates separate body/submit sends); the specific `[Pasted text #N]` placeholder + separate-Enter recovery is carried by open PR #80's diff to that same page. Nothing to merge into the merged layer that #80 doesn't already add.
- Both domain `index.md` files gained accurate "load when" rows; `log.md` gained two ingest entries.

## Open-PR check

Listed 27 open `knowledge/*` heads (#47–#95) via `gh pr list --search "head:knowledge/"`, fetched all `origin/knowledge/*` refs, and grepped each `git diff origin/main...<head> -- wiki/` for overlap keywords (`pasted|send-keys|bracketed paste|caplog|getMessage|record.args|refresh token|access token|currentUser|login check|graphql`).

- **Candidate 1 (caplog): new.** No open head touches caplog/log-record assertion content (only incidental `send-keys`/rate-limit-token matches elsewhere).
- **Candidate 2 (auth probe): new.** Token-keyword hits in `dch0202-20260805-*` heads are GitHub OAuth rate-limit content, unrelated to probe-path divergence; no overlap.
- **Candidate 3 (tmux): fold into PR #80** (`knowledge/choiyounggi-20260812-234147`). Its diff adds to `pane-delivery-confirmation` the exact edge row — pane shows `❯ [Pasted text #3]` with no busy marker → "send `Enter` as its own `send-keys` call and re-read" — plus a 2026-08-12 field observation of the same rc=0/"delivered"-but-unsubmitted failure. My candidate's only unique contribution is a second independent reproduction (pdfsum1 run: 3 of 4 sessions stuck post-"delivered", one Enter each resolved), which I posted as a corroborating comment on #80 rather than opening a sibling duplicate. Candidate retired as folded.

## Routing decision

- New categories: `infrastructure/agent-orchestration` (5 pages; unified the
competing `orchestration`/`agent-orchestration` names), `databases/data-survey`
(1), `qa/deliverables` (1). All other pages route into existing categories.
- Canonical-path decisions: rate limiting → `backend/common/reliability/`
(sits beside timeouts-and-retries; 6 of 8 variants chose it); stderr
diagnostics → `platforms/processes/` (concern spans beyond shells); leaked
artifacts → `testing/data/artifact-leakage-from-a-suite`; call-site
enumeration → the existing `backend/common/change-impact/` page.
- All 38 new pages listed in their domain indexes (nearest-index rule; backend
routes via its python sub-index for bytecode-cache-staleness); INDEX.md domain
summaries updated for infrastructure/qa/databases. Full-wiki lint: frontmatter,
ids, related-links, index coverage, size, qualifiers, staleness → 0 findings.
| Insight | Target | New page? |
|---------|--------|-----------|
| caplog `getMessage()` assertions | `testing/quality/captured-log-message-assertions.md` (id `testing-quality-captured-log-message-assertions`) | Yes — existing `quality` category fits (assertion-correctness cluster); no new category needed |
| login-probe vs API-auth divergence | `debugging/methodology/probe-path-vs-operation-path.md` (id `debugging-methodology-probe-path-vs-operation-path`) | Yes — `methodology` fits (diagnosing contradictory evidence); considered `signals` but the page prescribes an investigation/verification method, not signal reading; no new category needed |
| tmux pasted-text Enter recovery | fold → open PR #80 (`infrastructure/agent-orchestration/pane-delivery-confirmation.md`) | No — corroborating evidence noted on #80; not re-ingested here |
2 changes: 2 additions & 0 deletions log.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,5 @@ Append-only. Format: `## [YYYY-MM-DD] <ingest|revise|lint|gap|contradiction|drif
## [2026-08-06] dedup | Cross-PR duplicate clusters collapsed: client-side rate limiting (8 PR versions at 3 paths → 1), call-site enumeration (7 versions at 7 paths → additions folded into the already-merged canonical page), stderr/exit-0 diagnostics (4 versions at 4 paths → 1), macOS sysroot (2→1), env-var off switch (2→1), completion predicates (2→1), robots.txt (2→1), harness-mediated results (2→1), leaked test artifacts (2→1), orchestration category naming (orchestration vs agent-orchestration → agent-orchestration). Kept distinct after trigger comparison: differential-testing vs differential-run-agreement (setup vs interpretation), unset-versus-empty-parameters vs env-var-off-switches (semantics vs design, cross-linked), import-time-side-effects vs test-level-choice edge (tactics vs level choice, cross-linked). Merged-main near-dup scan (Jaccard over title+trigger, 141 pages): 0 pairs — no duplication among previously merged content.
## [2026-08-06] ingest | Reconciled post-consolidation flush PRs #42–#43 (created against pre-#44 main). Folded unique content: platforms/filesystems/permissions-and-exec-bits (+`sh "$SCRIPT"` interpreter-invocation rows — stubs need no exec bit, avoids EDR chmod+x heuristics), platforms/shells/command-text-inspected-before-execution (+gate-AUTHOR step 8: parse all three POSIX quoting forms, expand only ~/$HOME/${HOME}; red-then-green bats evidence), testing/quality/checks-that-cannot-pass (+vacuously-green pre-implementation test row), platforms/processes/tool-diagnostics-without-a-failing-exit-code (+repeat-suppression edge, +clang reproduction source), infrastructure/ci-cd/secrets-handling (+independent credential channels row: gh token vs git SSH vs API token, git push --dry-run / author:@me probes), testing/mocking/what-to-mock (+ESM read-only-bindings DI rows, +PATH-emptied tripwire negative-proof). Dropped as duplicates of the #44 state: #42's rate-limit row (already in timeouts-and-retries + client-side-rate-limiting), #42's warning-only-diagnostics page (same trigger as tool-diagnostics-without-a-failing-exit-code), #42's leak-attribution row (already in test-data-and-isolation).
## [2026-08-12] revise | routing: disjoint scopes for the doc-gate cluster (testing/quality ↔ qa/document-verification) and the flaky pair (testing/flaky ↔ debugging/concurrency); INDEX backend LLM phrasing; databases→backup cross-pointer (#37)
## [2026-08-14] ingest | testing-quality-captured-log-message-assertions — assert captured log content via record.getMessage()/caplog.messages; record.message is pre-interpolated at capture, so `% record.args` re-application TypeErrors on parameterized calls (zero-arg calls mask the bug)
## [2026-08-14] ingest | debugging-methodology-probe-path-vs-operation-path — a passing precondition probe is evidence about the probe's path only; under refresh-token cookie auth a page-load login check diverges from direct API calls replaying a stored cookie jar — preflight the API's identity endpoint with the operation's own client
1 change: 1 addition & 0 deletions wiki/debugging/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ Match your situation to a "load when" line; load only matching pages.
| [reproduce-first](methodology/reproduce-first.md) | A bug is reported or behavior is wrong and you are about to investigate or fix; deciding what to capture when full reproduction is impossible (prod-only, timing-dependent, one-off crash) |
| [isolate-by-bisection](methodology/isolate-by-bisection.md) | A bug reproduces but its location is unknown; it worked before / works in env A but not env B / fails with one input but not another — binary-searching versions (git bisect), code paths, data, or environment diffs |
| [hypothesis-testing](methodology/hypothesis-testing.md) | You have a suspect cause and are about to "try a fix"; several suspects compete and you must pick what to test next; verifying that a fix that "worked" actually addressed the mechanism |
| [probe-path-vs-operation-path](methodology/probe-path-vs-operation-path.md) | A precondition probe (login status, health, connectivity) reports success while the operation it gates fails with an auth/permission error; a browser page-load login check gates direct API calls made with stored cookies; deciding what a preflight probe must exercise under refresh-token cookie auth |
| [verify-the-fix](methodology/verify-the-fix.md) | You believe a bug is fixed and are about to close or ship it; the bug "cannot be reproduced anymore" after changes; a previously fixed bug came back; deciding what must pass (repro re-run, both directions, regression test) and what to clean up before closing |

## signals
Expand Down
2 changes: 1 addition & 1 deletion wiki/debugging/methodology/hypothesis-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ sources:
- https://www.debuggingbook.org/html/Intro_Debugging.html
- https://sre.google/sre-book/effective-troubleshooting/
last_verified: 2026-07-10
related: [debugging-methodology-reproduce-first, debugging-methodology-isolate-by-bisection]
related: [debugging-methodology-reproduce-first, debugging-methodology-isolate-by-bisection, debugging-methodology-probe-path-vs-operation-path]
---

# Testing a Suspected Cause Before Changing Code
Expand Down
Loading
Loading