Skip to content

fix(cli): run Python subprocesses in UTF-8 mode - #82

Merged
viktar-b merged 5 commits into
mainfrom
codex/windows-utf8
Sep 30, 2026
Merged

viktar-b merged 5 commits into
mainfrom
codex/windows-utf8

Conversation

@viktar-b

@viktar-b viktar-b commented Sep 30, 2026 •

Copy link
Copy Markdown
Owner

Why

Python can encode piped JSON with the Windows locale while the CLI expects UTF-8. Unicode calculation metadata then causes an encoding failure or invalid JSON. Start all CLI Python operations in UTF-8 mode.

Related to #77.

Scope

  • Route execution, dev definitions, bindings, and legacy export through a private runPythonProcess adapter.
  • Preserve interpreter selection, isolation, caller limits, schemas, and diagnostics. Decode legacy export buffers explicitly.
  • Exercise Unicode metadata, author names, public outputs, diagnostics, and directory names with the installed Python wheel.

Verification

  • The four regression cases fail on the base revision under ISO-8859-1 and pass with this change on macOS with Python 3.14.3.
  • CLI build, typecheck, and 75 package tests pass. Root TypeScript check and focused lint pass. CLI lint retains one existing warning and one existing informational finding.
  • Eight affected integration files pass all 46 tests, including dev server, HTML, and legacy PDF export.
  • Full npm run test:packages passes with fresh archives and a wheel, including 38 verification cases, 22 PDF cases, and generated-project browser/API/PDF/restart acceptance.
  • Native Windows proof passes all four Unicode cases with Python 3.11.9. The final main merge preserves the independently reviewed patch; current-head CI and CodeQL pass.

@vercel

vercel Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
calculation-source-object Ready Ready Preview Sep 30, 2026 9:47am UTC

Request Review

@viktar-b

Copy link
Copy Markdown
Owner Author

PASS+NOTES from an independent reviewer at head 1f40cd0770268f00a4462afe62fa7b023b1484ed, base 91282ab0836dfbba579b2f5e65c1d672119205a2. Stable patch ID: 15b833fafb82398ffe334f6d38f64a415b6bb44d.

No blocking findings. I reviewed all four callers, the private raw-buffer adapter, caller limits/isolation, legacy decoding, native-path test repair and both installed observer changes. The patch adds no comments or type suppressions. Scoped lint and diff checks passed.

Independent live proof:

  • All four protocol tests pass at this head with the installed wheel, both normally and under ISO-8859-1.
  • The original current-trunk execute invocation fails under that locale with UnicodeEncodeError; the UTF-8 invocation succeeds and preserves the exact reviewed Unicode title and author.
  • I inspected Windows run 36695880560. Checkout and runtime logs pin this head; all four tests pass on Windows Server 2025, Node 24.21.0, Python 3.11.9 and PowerShell 7.6.6.
  • I audited the owner's full installed-package receipts, including 38 verification and 22 PDF cases, verified 11 artifact hashes and matched all ten archived CLI build files to the reviewed build.

Notes: the PR's installed-package CI job is skipped by the existing classifier, so the audited full local run supplies that evidence. The PR description can now replace its native-pending line with the successful run. Refresh against current main before landing; retain this code verdict only if the stable patch ID is unchanged, with current-head CI and mergeability rechecked. Windows 11 desktop and human PDF inspection remain separate qualification work.

@viktar-b

Copy link
Copy Markdown
Owner Author

Root verification PASS at 02a73bceb5c6d9a951e8e657401837426a8a4bde against main 5b8c66f9a9546a693a0ebeffa44bddf4de2ec7d1.

The independent non-author verdict is recorded above. The refreshed diff retains stable patch ID 15b833fafb82398ffe334f6d38f64a415b6bb44d. Its base now includes the bindings and npm repairs. The owner rebuilt the updated Python wheel and passed all four Unicode cases in normal and ISO-8859-1 environments.

Native Windows proof passed all four Python operations at the reviewed patch. Independent review checked live before/after behavior and installed archive receipts, including 38 verification and 22 PDF cases. Current-head quality, isolation, dependency-review, PR-title and CodeQL pass. Installed-packages is skipped by the maintained CI policy; the full installed acceptance receipt was independently audited. GitHub reports CLEAN with no unresolved threads. Normal protected merge is authorized for this head only.

@viktar-b
viktar-b merged commit c710fef into main Sep 30, 2026
12 checks passed
@viktar-b
viktar-b deleted the codex/windows-utf8 branch September 30, 2026 09:52

This branch was successfully deployed

1 active deployment
Preview — 02a73bce Deployed Sep 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant