Repository navigation
autopilot 2026-10-12: unfiled follow-up notes #217
Description
Notes collected during the 2026-10-12 autopilot verification rounds that no PR or other issue carries yet:
- PORTING substitution map lacks rows for todolist->worklist and arena Phase C 'or capability tier' (pre-existing; from fix: isolate arena runs and clean up candidate artifacts #201 parity lane) -> fold into the lint/PORTING:25-26 PR
- Claude preflight reports a missing claude binary as a login failure (from fix(evals): handle login failures in live promise evaluations #198 consumer lane)
- ledger: duplicate nested field message lacks the holding object path; no .gitattributes eol=lf for ledger.json; no test for write_atomically cleanup/mode; test_static plain json.loads; unused import re in test_ledger.py (from fix(evals): enforce canonical promise ledger format #199 r2)
- codex listing: a literal </skills_instructions> in a skill description would end the listing early (latent, fix(evals): end the Codex skill listing at the closing tag #213); child rollouts anchor on Codex prose "### Available skills" and parse {} (latent, pre-existing)
- evals/promises/README.md:103 calls the second gap a "command chain"; the trigger is any failed shell call (see evals: oracle follow-ups from the #202 review #207 item 6).
- evals/promises/README.md:80 describes arena's per-run ${TMPDIR} path, which lands only with PR fix: isolate arena runs and clean up candidate artifacts #201 (stopped); main's arena SKILL.md still says /tmp/arena-/.
- PR sync: port upstream pstack through 1b6d44f (#570, #573-#575, #484, #486, #536) #196 and fix(verify-commands): flag an exit status masked by a tail pipe #200 round-2 notes: 5 of the owed live reruns are bound to the deferred verify-skill-run case; the install promise slash-skills-invocable needs install.py; owed feat(skills): add chat catch-up and refresh upstream workflows #190/sync: port upstream pstack through 1b6d44f (#570, #573-#575, #484, #486, #536) #196 live reruns have not run.
Activity
mdsmithaustin commentedon Oct 14, 2026
Issue217.2 is resolved by merged #247. The real CLI now reports a missing Claude executable as an installation prerequisite before starting cases. Installed logged-out behavior and authenticated dispatch remain unchanged. Independent current-trunk comparisons, focused regressions, all seven CI checks and all nine merge gates passed.
This issue remains open for native/install/Grok verification debt and the remaining tracked follow-ups. The current canonical test run still has three baseline NativeGit admission errors and one skip; no full-suite or native-model success is claimed. Failed-call oracle wording is routed to #207. The held new eval-interface proposals were not implemented.
mdsmithaustin commentedon Oct 16, 2026
Accepted bounded follow-up: retain installation, native, and original live-rerun obligations. Two focused model-free runs each executed 57 tests with 56 passes and one device-node fixture skip. These do not replace authenticated installation, native confinement, or live compliance checks. Read-only executable comparison reproduces the bundled ripgrep compatibility mismatch associated with the three historical NativeGit setup errors; those assertions remain unverified and the errors unwaived. No dependency admission, digest update, or supported-host claim was made. Missing-binary diagnosis and current path notes are addressed; failed-call recognition belongs to #207. Defer speculative parser, EOL, and diagnostic expansion.