[mdsmithaustin]

What Changed

• Stop harvesting Codex skill listings at </skills_instructions> or the next Markdown heading so subsequent instructions do not enter the listing.
• Extend regression coverage for usage sections and permission instructions after the skill listing.

Validation

Risk Assessment

✅ Low: The parser now cuts the Codex listing at whichever of </skills_instructions> and \n# comes first, as the intent requires. Both fixtures are non-vacuous: the post-tag - network: restricted line matches LISTED_SKILL and is caught by main's \n#-only cut, and the - Discovery: line would be caught by a tag-only cut. PORTING.md is untouched.

Testing

Targeted tests and four real-harvester ingestion scenarios passed using disposable protocol data. Base-parser and delimiter-mutation checks proved both regressions detect incorrect behavior. Inputs, traces, and comparison results were retained as evidence; transient files were removed. No UI was changed, and operator session data was not accessed.

• Live validation: ✅ go - 4 of 4 scenarios driven live against the product

Scenario	Result	Live	Evidence
Harvest an older rollout with a heading before the closing tag; exclude the How to use skills bullets	✅ pass	live	heading-before-tag-trace.json; listing-ingestion-results.json
Harvest a rollout with post-tag permissions and collaboration text; exclude the synthetic network bullet	✅ pass	live	tag-before-heading-trace.json; listing-ingestion-results.json
Harvest a rollout with a closing tag and no later heading; exclude trailing bullets	✅ pass	live	tag-only-trace.json; listing-ingestion-results.json
Harvest a heading-only listing containing a legitimate network skill; retain that skill and exclude bullets after the heading	✅ pass	live	valid-network-and-heading-only-trace.json; listing-ingestion-results.json

Evidence: Rollout ingestion results and base-parser comparison

[
  {
    "scenario": "heading-before-tag",
    "expected_from_intent": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works."
    },
    "actual_trace_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works."
    },
    "base_parser_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works."
    }
  },
  {
    "scenario": "tag-before-heading",
    "expected_from_intent": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works."
    },
    "actual_trace_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works."
    },
    "base_parser_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works.",
      "network": "restricted"
    }
  },
  {
    "scenario": "tag-only",
    "expected_from_intent": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works."
    },
    "actual_trace_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works."
    },
    "base_parser_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works.",
      "network": "restricted"
    }
  },
  {
    "scenario": "valid-network-and-heading-only",
    "expected_from_intent": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works.",
      "network": "Network diagnostics."
    },
    "actual_trace_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works.",
      "network": "Network diagnostics."
    },
    "base_parser_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works.",
      "network": "Network diagnostics."
    }
  },
  {
    "heading_guard_removed_listing": {
      "deslop": "Remove AI slop.",
      "how": "Use for how X works.",
      "Discovery": "the list above is the skills available."
    },
    "regression_rejects_mutant": true
  }
]

Evidence: Post-tag permissions scenario product trace

{
  "harness": "codex",
  "cli_version": "0.159.0",
  "model": null,
  "effort": null,
  "argv": [
    "codex",
    "go"
  ],
  "cwd": "~/.no-mistakes/worktrees/fbee15f974d9/01M4S4CB887KZ84A270Q91S1AE/.local-listing-test/tag-before-heading/w/project",
  "exit_code": 0,
  "duration_s": 0,
  "entry": "not-observed",
  "events": [
    {
      "kind": "user",
      "text": "go",
      "seq": 0,
      "turn": 0
    },
    {
      "kind": "text",
      "text": "Completed isolated rollout.",
      "seq": 1,
      "turn": 0
    }
  ],
  "files_read": [],
  "worklist": [],
  "spawns": [],
  "final_reply": "Completed isolated rollout.",
  "transcript_paths": [
    "~/.no-mistakes/worktrees/fbee15f974d9/01M4S4CB887KZ84A270Q91S1AE/.local-listing-test/tag-before-heading/transcripts/rollouts/rollout-tag-before-heading.jsonl"
  ],
  "x_binary": {
    "path": "unused-for-harvest",
    "source": "disposable-rollout-ingestion",
    "version": "0.159.0",
    "rejected": []
  },
  "x_turns": [
    {
      "index": null,
      "session_id": "tag-before-heading",
      "argv": [
        "codex",
        "go"
      ],
      "exit_code": 0,
      "timed_out": null,
      "duration_s": null
    }
  ],
  "x_entry_injections": [],
  "x_turn_entries": [],
  "x_todo_tools": "on",
  "x_token_usage": null,
  "x_compactions": [],
  "x_listed_skills": {
    "deslop": "Remove AI slop.",
    "how": "Use for how X works."
  },
  "x_subagents": []
}

Evidence: Older heading-first scenario product trace

{
  "harness": "codex",
  "cli_version": "0.159.0",
  "model": null,
  "effort": null,
  "argv": [
    "codex",
    "go"
  ],
  "cwd": "~/.no-mistakes/worktrees/fbee15f974d9/01M4S4CB887KZ84A270Q91S1AE/.local-listing-test/heading-before-tag/w/project",
  "exit_code": 0,
  "duration_s": 0,
  "entry": "not-observed",
  "events": [
    {
      "kind": "user",
      "text": "go",
      "seq": 0,
      "turn": 0
    },
    {
      "kind": "text",
      "text": "Completed isolated rollout.",
      "seq": 1,
      "turn": 0
    }
  ],
  "files_read": [],
  "worklist": [],
  "spawns": [],
  "final_reply": "Completed isolated rollout.",
  "transcript_paths": [
    "~/.no-mistakes/worktrees/fbee15f974d9/01M4S4CB887KZ84A270Q91S1AE/.local-listing-test/heading-before-tag/transcripts/rollouts/rollout-heading-before-tag.jsonl"
  ],
  "x_binary": {
    "path": "unused-for-harvest",
    "source": "disposable-rollout-ingestion",
    "version": "0.159.0",
    "rejected": []
  },
  "x_turns": [
    {
      "index": null,
      "session_id": "heading-before-tag",
      "argv": [
        "codex",
        "go"
      ],
      "exit_code": 0,
      "timed_out": null,
      "duration_s": null
    }
  ],
  "x_entry_injections": [],
  "x_turn_entries": [],
  "x_todo_tools": "on",
  "x_token_usage": null,
  "x_compactions": [],
  "x_listed_skills": {
    "deslop": "Remove AI slop.",
    "how": "Use for how X works."
  },
  "x_subagents": []
}

Evidence: Disposable ingestion driver

import json
import re
import shutil
import subprocess
import sys
import types
from pathlib import Path
from types import SimpleNamespace

sys.path.insert(0, str(Path('evals/promises').absolute()))
from harnesses import codex

workspace = Path.cwd()
evidence = Path('~/.no-mistakes/evidence/01M4S4CB887KZ84A270Q91S1AE')
entries = ('<skills_instructions>\n## Skills\n### Skill roots\n- `r1` = `/w/.agents/skills`\n'
           '### Available skills\n- deslop: Remove AI slop. (file: r1/deslop/SKILL.md)\n'
           '- how: Use for how X works. (file: r1/how/SKILL.md)\n')
expected = {'deslop': 'Remove AI slop.', 'how': 'Use for how X works.'}
after = ('<permissions instructions>\nFilesystem sandboxing defines which files can be read or written. '
         '`sandbox_mode` is `workspace-write`: The sandbox permits reading files, and editing files in `cwd` and `writable_roots`. '
         'Editing files in other directories requires approval. Network access is restricted.\n'
         '- network: restricted\nApproval policy is currently never. Do not provide the `sandbox_permissions` for any reason, commands will be rejected.\n'
         ' The writable roots are `/w/tmp`, `/w/relay`, `/w/relay/.git`, `/private/tmp`.\n</permissions instructions>'
         '<collaboration_mode># Collaboration Mode: Default\n\nYou are now in Default mode.\n\n'
         '## request_user_input availability\n\nUse the `request_user_input` tool only when it is listed.')
cases = [
    ('heading-before-tag', entries + '### How to use skills\n- Discovery: the list above is the skills available.\n</skills_instructions>\n- network: restricted', expected),
    ('tag-before-heading', entries + '</skills_instructions>' + after, expected),
    ('tag-only', entries + '</skills_instructions>\n- network: restricted', expected),
    ('valid-network-and-heading-only', entries + '- network: Network diagnostics. (file: r1/network/SKILL.md)\n# Instructions\n- bogus: outside listing', {**expected, 'network': 'Network diagnostics.'}),
]
base = types.ModuleType('codex_base')
exec(compile(subprocess.check_output(['git', 'show', '321ef775e3be7ff2e15d37fc3ebbe6457045bdf4:evals/promises/harnesses/codex.py']), '<base-codex>', 'exec'), base.__dict__)
report = []
for name, text, oracle in cases:
    root = workspace / '.local-listing-test' / name
    store = root / 'codex-home' / 'sessions' / '2026'
    store.mkdir(parents=True)
    project = root / 'w' / 'project'
    project.mkdir(parents=True)
    rows = [
        {'type': 'session_meta', 'payload': {'id': name, 'cwd': str(project), 'cli_version': '0.159.0'}},
        {'type': 'response_item', 'payload': {'type': 'message', 'role': 'developer', 'content': [{'type': 'input_text', 'text': text}]}},
        {'type': 'response_item', 'payload': {'type': 'message', 'role': 'user', 'content': [{'type': 'input_text', 'text': 'go'}]}},
        {'type': 'response_item', 'payload': {'type': 'message', 'role': 'assistant', 'content': [{'type': 'output_text', 'text': 'Completed isolated rollout.'}]}},
    ]
    data = ''.join(json.dumps(row) + '\n' for row in rows).encode()
    (store / f'rollout-{name}.jsonl').write_bytes(data)
    (root / 'launch.json').write_text(json.dumps({'path': 'unused-for-harvest', 'source': 'disposable-rollout-ingestion', 'version': '0.159.0', 'rejected': []}))
    run = SimpleNamespace(root=root, project=project, case={'turns': ['go']}, turns=[{'session_id': name, 'argv': ['codex', 'go'], 'exit_code': 0}])
    trace = codex.harvest(run)
    (evidence / f'{name}-trace.json').write_text(json.dumps(trace, indent=2) + '\n')
    shutil.copyfile(root / 'transcripts' / 'rollouts' / f'rollout-{name}.jsonl', evidence / f'{name}-input.jsonl')
    assert trace['x_listed_skills'] == oracle, (name, trace['x_listed_skills'], oracle)
    assert trace['final_reply'] == 'Completed isolated rollout.'
    old = base.harvest_rollout(data)['listed']
    if name in ('tag-before-heading', 'tag-only'):
        assert old == {**expected, 'network': 'restricted'}, old
    else:
        assert old == oracle, old
    report.append({'scenario': name, 'expected_from_intent': oracle, 'actual_trace_listing': trace['x_listed_skills'], 'base_parser_listing': old})
# Execute the restored regression against a tag-only delimiter to prove the heading guard is needed.
original = codex.LISTING_END
try:
    codex.LISTING_END = re.compile(r'</skills_instructions>')
    wrong = codex.harvest_rollout((evidence / 'heading-before-tag-input.jsonl').read_bytes())['listed']
    assert wrong == {**expected, 'Discovery': 'the list above is the skills available.'}, wrong
    report.append({'heading_guard_removed_listing': wrong, 'regression_rejects_mutant': wrong != expected})
finally:
    codex.LISTING_END = original
(evidence / 'listing-ingestion-results.json').write_text(json.dumps(report, indent=2) + '\n')
print(json.dumps(report, indent=2))

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

• Live validation: ✅ go - 4 of 4 scenarios driven live against the product

Scenario	Result	Live	Evidence
Harvest an older rollout with a heading before the closing tag; exclude the How to use skills bullets	✅ pass	live	heading-before-tag-trace.json; listing-ingestion-results.json
Harvest a rollout with post-tag permissions and collaboration text; exclude the synthetic network bullet	✅ pass	live	tag-before-heading-trace.json; listing-ingestion-results.json
Harvest a rollout with a closing tag and no later heading; exclude trailing bullets	✅ pass	live	tag-only-trace.json; listing-ingestion-results.json
Harvest a heading-only listing containing a legitimate network skill; retain that skill and exclude bullets after the heading	✅ pass	live	valid-network-and-heading-only-trace.json; listing-ingestion-results.json

• PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=evals/promises python3 -m unittest -v test_codex.SkillListing
• PYTHONDONTWRITEBYTECODE=1 python3 .local-listing-test/drive.py: executed the real Codex harvester against four disposable rollout stores without mocks; saved inputs and generated traces.
• Executed SkillListing against the base parser: the post-tag regression failed with the extra network skill.
• Executed the restored heading regression with the heading alternative removed in memory: it failed with the extra Discovery entry.
• Removed the disposable rollout stores and driver; git status --short confirmed a clean worktree.

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

[mdsmithaustin]

Verdict: PASS+NOTES
Head: 1c6e9ee
Patch-id: 8082c2394637a25d7b6a59a324bc6135e848b25e
Docs: pass

Round 1 swarm at this head: gates + red checks + corpus differential (sonnet@high), parser review and fixture review (opus@xhigh each; work model sonnet@high).

Gates pass (only known host noise). Each new test fails on main's parser or with its cut alternative removed. Across 1,832 saved Codex rollouts and 5,327 host rollouts, main's and head's parsers give identical listings, so no recorded verdict moves. Fixtures carry no secrets, emails, tokens or host paths. Merges clean with main, #203 and #205.

Notes (merge-time and follow-ups):

• The commit body says "Closes Restore the Codex heading-shape listing fixture and tidy PORTING.md:26 #195" while item 2 lands with refactor(tools): consolidate skill lint registries and checks #203. merge-gate takes the squash body from --body-file, so the merge writes "Refs Restore the Codex heading-shape listing fixture and tidy PORTING.md:26 #195" and Restore the Codex heading-shape listing fixture and tidy PORTING.md:26 #195 closes after refactor(tools): consolidate skill lint registries and checks #203.
• The invented - network: restricted bullet is labeled only in the PR body; the commit's "790 to 1,060" gap figure is not reproducible (measured 1,097 to 1,417). Both disappear with the squash body.
• Latent: a literal </skills_instructions> inside a skill description would end the listing early (no skill contains it); child rollouts anchor on Codex prose and parse an empty listing (pre-existing, no verdict reads child listings).