Skip to content

fix(cli): emit JSON envelope for local media extraction - #570

Closed
officialasishkumar wants to merge 1 commit into
steipete:mainfrom
officialasishkumar:fix/local-media-extract-json
Closed

officialasishkumar wants to merge 1 commit into
steipete:mainfrom
officialasishkumar:fix/local-media-extract-json

Conversation

@officialasishkumar

Copy link
Copy Markdown
Contributor

Summary

summarize <local audio/video> --extract --json exits 0 but writes the plain transcript (Transcript:\n…) to stdout, so anything that parses the documented JSON envelope fails. Only the --slides variant produced JSON, because it runs through the URL flow. The cause is in presentMediaFileResult: its extraction branch wrote extracted.content without checking JSON mode.

With --json, local media extraction now writes the standard envelope (input, env, extracted, prompt, llm, metrics, summary):

  • input uses the same file / asset-url shape as local media summaries (it reuses buildAssetJsonInput, which is now exported).
  • extracted is the full transcript result, so .extracted.content, .extracted.transcriptionProvider and the --timestamps segment fields look the same as on the URL and --slides paths.
  • prompt, llm and summary are null. metrics follows --metrics, as it does on the other extract-only JSON paths.

Output without --json is unchanged.

Fixes #562

Validation

  • Red/green: the new CLI regressions in tests/cli.asset.audio-file.test.ts run --extract --json on a local .wav and .mp4 with a stubbed ONNX transcriber. They fail on main because stdout is not JSON, and pass with this change. tests/cli.asset.media-file-options.test.ts also checks the new extraction args.
  • pnpm check: formatting, localization, lint, typecheck and coverage all pass. 605 test files passed, 30 skipped; 3,896 tests passed, 49 skipped.
  • Built CLI (node dist/cli.js a.wav --extract --json --transcriber parakeet …): stdout parses as one JSON value with input.kind: "file", extracted.content: "Transcript:\nhello transcript", extracted.transcriptionProvider: "onnx-parakeet", llm: null, summary: null. Running the same command without --json still prints the plain transcript.

No new options, configuration or dependencies.

`summarize <local audio/video> --extract --json` exited successfully but
wrote the plain transcript to stdout, so JSON consumers failed to parse
the result. The media presenter's extraction branch wrote
`extracted.content` directly without checking JSON mode; only the
`--slides` path (which goes through the URL flow) produced the envelope.

When `--json` is set, local media extraction now writes the standard
`input`/`env`/`extracted`/`prompt`/`llm`/`metrics`/`summary` envelope.
`input` uses the same file/asset-url shape as media summaries, and
`extracted` carries the full transcript result (content,
transcriptionProvider, transcript segments with `--timestamps`, etc.),
matching what the URL and `--slides` paths already expose. Plain text
extraction output is unchanged.

Fixes steipete#562
@clawsweeper

clawsweeper Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-11T12:03:17.532258Z
We couldn’t complete this review. Comment @codex review to try again.
c5ae93b PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@clawsweeper clawsweeper Bot added clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-environment Preliminary submission classification; full source review pending. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed clawsweeper:intake:bug Preliminary submission classification; full source review pending. clawsweeper:intake:needs-environment Preliminary submission classification; full source review pending. labels Nov 11, 2026
@clawsweeper

clawsweeper Bot commented Nov 11, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge.

What this changes

This PR emits the standard JSON envelope when extracting transcripts from local audio or video with --json.

Example: Run summarize a.wav --extract --json --metrics off.

  • Before: Stdout contains plain text such as Transcript:\nhello transcript, which a JSON parser rejects.
  • After: Stdout contains a JSON envelope with extracted.content containing the transcript and null prompt, llm, metrics, and summary fields.

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The focused patch appears correct and has meaningful regression coverage, but the supplied behavior evidence does not yet clear the external-PR proof gate.
Proof confidence 🦪 silver shellfish (2/6) Real behavior proof is necessary before merge. See Before merge.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Complete
User problem: Automation requesting JSON receives plain transcript text and fails to parse otherwise successful local-media extraction.
Reason: The change restores documented behavior using the existing flag and schema, without adding configuration or changing transcription.

Merge readiness

⛔ Blocked before merge - 1 item remains

Keep open: this PR fixes a documented JSON-output bug still present on main; no introduced correctness or security defect was found.

Priority: P2
Reviewed head: c5ae93bc4f5aa06a99e7e897990df8612c874858

Before merge

  • Add real behavior proof - The PR describes node dist/cli.js extracting a.wav with Parakeet and reports the resulting JSON fields plus unchanged text output, but supplies prose rather than captured terminal output, logs, or a linked artifact; attach a terminal screenshot or trace of that scenario and redact credentials, IPs, and non-public endpoints. After adding proof, update the PR body, then request a new review with @clawsweeper re-review. The PR author or someone with repository write access can request it.

Findings

None.

Tests

  • Missing end-to-end proof: Provide captured built-CLI output showing local-media --extract --json stdout parsing as one envelope; the body reports red/green regressions but no execution artifact was supplied.
Agent review details

How this fits together

The CLI routes acquired media through transcription, then passes the extracted result to the media presenter, which writes text or JSON to stdout.

flowchart LR
  A[Local audio or video] --> B[CLI input acquisition]
  B --> C[Media execution]
  C --> D[Transcript provider]
  D --> E[Media presenter]
  E --> F[JSON envelope or text stdout]
Loading

Technical review

Best possible solution:

Emit the existing JSON envelope in the media presenter while preserving full transcript metadata and ordinary text extraction.

Do we have a high-confidence way to reproduce the issue?

Current-main source independently establishes the failure, supported by the linked reporter’s release reproduction; no target code or tests were run during this read-only review.

Is this the best way to solve the issue?

The repair belongs in presentation because transcription already succeeds, and reusing the existing input helper preserves the established JSON contract without changing provider behavior.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against aaccb7c9c30e.

Provenance checked

  • Extract-only media presentation changes intended behavior with a stated reason (af1992f: Support --extract for media without invoking summarization; the later execution/presentation split preserved this behavior.)
  • Asset JSON input helper keeps the original intent (8e4ae40: Separate asset execution from presentation; the later resolved-media refactor narrowed its argument to byte-free presentation metadata.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

Likely related people:

  • Peter Steinberger: Raw commit 07ef3c1 adds src/run/flows/asset/media.ts:374 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 07ef3c1436da; files: src/run/flows/asset/media.ts)
  • Anton Novoselov: Raw commit af1992f adds src/run/flows/asset/media.ts:227 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: medium; commits: af1992f11e73; files: src/run/flows/asset/media.ts)

Preflight source

Source unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified.

Review metrics

Metric Value Why it matters
Change size Production +30/-3 lines; tests +65/-1 lines The growth is focused on JSON presentation and regression coverage for both local media routes.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #562
Summary: This PR is a candidate fix for the exact local-media JSON presentation defect documented by the canonical issue.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Labels

Label changes:

  • add P2: A supported output mode breaks JSON-consuming local-media automation, while transcription and plain-text access remain available.
  • add rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Label justifications:

  • P2: A supported output mode breaks JSON-consuming local-media automation, while transcription and plain-text access remain available.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Attach captured output from the reported built-CLI local-media extraction and JSON parsing.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed November 11, 2026, 7:16 AM ET / 12:16 UTC.

steipete added a commit that referenced this pull request Nov 14, 2026
Local audio/video extraction ignores `--json`, Markdown and transcription diagnostics misidentify their providers, and the Codex adapter pins an unavailable default while hiding backend failure reasons. Optional OpenRouter diagnostics can also wait beyond the request deadline, and Windows uninstall deletes launcher files even when Task Scheduler refuses deletion.

This consolidates and finishes #559, #560, #570, and #571, retaining their authors' fixes and adding truthful extraction provenance, supported Codex default selection, and regression coverage. It also corrects the YouTube fallback order and incorporates the small gateway documentation example from #568 through the existing OpenAI-compatible transport. Existing string-returning library converters remain compatible; explicitly requested Codex models remain explicit.

Validation:
- The new regression set fails on unmodified main across six test files, including denied task deletion, stalled HTTP response bodies, local media JSON, provenance, and Codex failure/default handling.
- Full `pnpm run build` and `pnpm run check` passed on Linux/Node 24: 605 test files passed, 3,909 tests passed, 49 skipped; coverage 94.52% lines and 85.67% branches.
- Independent Codex P0–P2 autoreview: scoped clean after fixing one stale test assertion.
- No version bump, release, deployment, or Chrome Web Store operation.

Addresses #561
Addresses #562
Addresses #563
Addresses #564
Addresses #565
Addresses #566
Addresses #567

Co-authored-by: Sebastien Tardif <[email protected]>
Co-authored-by: Vincent Peng <[email protected]>
Co-authored-by: Asish Kumar <[email protected]>
Co-authored-by: Nirupam Bhowmick <[email protected]>
@steipete

Copy link
Copy Markdown
Owner

Thanks for the focused contribution. Incorporated into #572 with co-author and changelog credit. The WAV and MP4 regressions fail on main and now pass. Local extract-only JSON uses the shared input shape and the documented envelope. Full build/check passed (3,909 tests), and #572 passed exact-head CI before landing. Closing this superseded branch.

@steipete steipete closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(cli): local audio/video extraction ignores --json

2 participants