Skip to content

Take the CultureMech Edison sidecar-provenance fix, and stop requesting malformed citation sidecars #641

Description

@realmarcin

Propagation of two CultureMech fixes. CultureMech is the canonical hub for
vendored shared files, so both changes should be taken from there rather than
reimplemented.

1. _edison_capture.py attributes stale sidecars to the wrong task

capture_full_response builds its sidecar_files provenance block from a plain
.exists() sweep of the output directory:

"sidecar_files": _existing_sidecars(out_dir, stem),

Edison output stems are deterministic, so a rerun of the same record and job
lands in a directory that already holds the previous run's sidecars. A rerun
whose verbose fetch or list_files call fails therefore writes
agent_state_json: true into a meta stamped with a new task_id, pointing
at a trace belonging to the old one. For a field whose whole purpose is
provenance, an auditor following the meta reads the wrong trajectory.

Fix: track which keys the invocation actually wrote and report the AND of "we
wrote it" and "it is on disk". _existing_sidecars keeps its disk-snapshot
behaviour when the written-set is omitted, which is what enrich_edison_response
needs.

  • Canonical implementation and 6 regression tests: Report the sidecars this Edison run wrote, not the ones lying around (#288) CultureMech#291
  • Origin of the fix: TraitMech (which shipped it without tests)
  • Worth auditing your research/**/*-meta.yaml first. CultureMech audited 131
    and found 0 mis-attributions, so no corpus back-fix was needed there; yours may
    differ. Compare each meta's task_id against the task_id recorded inside
    the agent-state trace it claims.

Related latent hole found while reviewing: CultureBotAI/CultureMech#292
enrich_edison_response can restore a stale attribution, because
needs_enrichment decides what to refetch from bare disk existence.

2. research_community.py still requests --separate-citations

"--output",
    str(output_file),
    "--separate-citations",
    str(citations_file),

That sidecar is a regex over the report prose, not structured provider output,
and it is malformed. Evidence from two repositories:

  • TraitMech (ORNL Clostridium-Desulfovibrio-Geobacter trophic model: curate causal graph #249 there): 353 sidecars — 194 broken markdown-link tails,
    2,770 stray trailing commas, 332 of 353 duplicating a reference two or three
    times. TraitMech disabled it.
  • CultureMech: the single sidecar it ever produced re-emits the entire
    ~55-line rendered prompt as "Query", lists the bare string Na+ as entry 12 of
    27, and repeats 10.1101/2024.06.09.598106 three times over as entries 16, 17
    and 24, differing only in a trailing . or ,.

The report's own References section maps keys to DOIs and is what a curator
reads, so dropping this removes a broken duplicate, not a source.

CultureMech's artifact contract, which you can adopt verbatim:
docs/RESEARCH_ARTIFACT_CONTRACT.md in CultureBotAI/CultureMech#293.

artifact status
References section inside the report .md authoritative
client --separate-citations sidecar disabled
<stem>-citations.md (our own parser, Edison path) derived, for skimming

3. Optional: put _edison_capture.py under the vendored contract

scripts/check_vendored_sync.sh currently pins five files, none of which is
_edison_capture.py. This fix landing in three repos at three different times is
exactly the drift that check exists to prevent. Adding it requires the hub and
spoke copies to be byte-identical first; CultureMech#291 deliberately words its
new docstring domain-neutrally ("record", not "medium") so the file stays
adoptable verbatim.

Notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions