Skip to content

Fix bytecode version skew diagnostics - #4312

Closed
sxlijin wants to merge 1 commit into
canaryfrom
agent/bytecode-version-skew
Closed

sxlijin wants to merge 1 commit into
canaryfrom
agent/bytecode-version-skew

Conversation

@sxlijin

@sxlijin sxlijin commented Jul 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • wrap generated BAML bytecode in a versioned cross-language envelope
  • embed an independently parseable copy of the project baml.toml with the actual producer release and bytecode-format version
  • reject release, format, legacy, and malformed payload mismatches before deserializing Program
  • surface observed versions and concrete bridge-upgrade or regeneration guidance

Root cause

baml generate previously embedded a raw Borsh Program, while independently installed bridge runtimes deserialized it using their own schema without knowing which CLI release or format produced it. Version skew therefore surfaced as a low-level decode error or could be accepted without semantic compatibility evidence.

Compatibility policy

Generated bytecode is accepted only when the producer BAML release and bytecode-format version exactly match the consuming bridge. Moving channels are recorded as the concrete CLI release that performed generation. Legacy or unversioned payloads are rejected with regeneration guidance.

Validation

  • cargo test -p bex_project bytecode_compatibility_tests --no-default-features
  • cargo test -p baml_cli generate::tests --no-default-features
  • cargo test -p baml_cli generate_valid_project_returns_zero_exit_code --no-default-features
  • cargo test -p bridge_cffi bytecode_initializer --no-default-features
  • cargo check -p bridge_cffi -p sdk_test_harness_setup --no-default-features
  • cargo check -p bridge_python -p bridge_typescript --no-default-features
  • cargo check -p bridge_typescript_web --target wasm32-unknown-unknown --no-default-features
  • cargo clippy -p bex_project --no-default-features -- -D warnings
  • cargo clippy -p baml_cli -- -D warnings
  • cargo fmt --all --check
  • git diff --check

Summary by CodeRabbit

  • New Features

    • Generated bytecode now includes validated project configuration and format metadata.
    • Runtime loading validates bytecode compatibility and provides actionable guidance for invalid or outdated payloads.
  • Bug Fixes

    • Compatibility failures are now consistently reported as compilation or host errors across supported integrations.
    • C# generation uses compiled program data directly for more reliable runtime discovery.
  • Documentation

    • Updated bytecode initialization documentation to describe the versioned format and embedded metadata.

@vercel

vercel Bot commented Jul 31, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
beps Ready Ready Preview Jul 31, 2026 8:48pm
promptfiddle2 Ready Ready Preview Jul 31, 2026 8:48pm

Request Review

@coderabbitai

coderabbitai Bot commented Jul 31, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Generation now emits a versioned bytecode envelope with embedded, validated baml.toml metadata. Runtime loading validates compatibility before decoding. C# generation uses the compiled program directly, and compatibility errors use structured mappings.

Changes

Bytecode compatibility

Layer / File(s) Summary
Envelope contract and loading
baml_language/crates/bex_project/...
Defines the versioned envelope, embedded manifest metadata, serialization, compatibility checks, and envelope-based program loading.
Compatibility validation tests
baml_language/crates/bex_project/src/lib.rs
Tests envelope layout, round trips, and rejection of malformed, incompatible, truncated, and invalid metadata.
Manifest embedding and fixture serialization
baml_language/crates/baml_cli/..., baml_language/sdk_tests/harness_setup/...
CLI generation and SDK fixtures embed manifest metadata and use bex_project::serialize_bytecode.
Direct C# program input
baml_language/sdks/csharp/sdkgen_csharp/..., baml_language/crates/baml_cli/src/generate.rs
C# generation receives the compiled Program and derives callable identities without decoding serialized bytes.
Runtime compatibility errors
baml_language/crates/bridge_cffi/..., baml_language/crates/baml_lsp_server/src/playground_server.rs, baml_language/sdks/typescript/bridge_typescript_web/src/errors.rs
Compatibility failures map to structured host or compilation errors. Tests and initialization documentation cover the new envelope.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant BamlGenerate
  participant EmbeddedBamlToml
  participant SerializeBytecode
  participant CSharpGenerator
  BamlGenerate->>EmbeddedBamlToml: Read and validate baml.toml
  EmbeddedBamlToml-->>BamlGenerate: Return manifest with compatibility metadata
  BamlGenerate->>SerializeBytecode: Serialize compiled Program and manifest
  SerializeBytecode-->>BamlGenerate: Return versioned bytecode envelope
  BamlGenerate->>CSharpGenerator: Pass Program and serialized envelope
  CSharpGenerator->>CSharpGenerator: Derive callable identities from Program
Loading

Possibly related PRs

Suggested reviewers: hellovai

Poem

A rabbit packs the bytecode tight,
With manifest metadata bright.
The runtime checks each versioned seam,
C# reads the compiled dream.
Errors hop in structured style.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: improving diagnostics for bytecode version skew.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch agent/bytecode-version-skew

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@vercel
vercel Bot temporarily deployed to Preview – beps July 31, 2026 19:41 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 31, 2026 19:48 Inactive
@sxlijin
sxlijin force-pushed the agent/bytecode-version-skew branch from cee7fac to 1d5336b Compare July 31, 2026 19:51
@vercel
vercel Bot temporarily deployed to Preview – beps July 31, 2026 19:53 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 31, 2026 20:00 Inactive
@sxlijin
sxlijin force-pushed the agent/bytecode-version-skew branch from 1d5336b to ab33c73 Compare July 31, 2026 20:02
@vercel
vercel Bot temporarily deployed to Preview – beps July 31, 2026 20:04 Inactive
@blacksmith-sh

This comment has been minimized.

@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 31, 2026 20:11 Inactive
@github-actions

github-actions Bot commented Jul 31, 2026 •

Copy link
Copy Markdown

Binary size checks passed

✅ 7 passed

Artifact Platform File Gzip Gated on Baseline Delta Status
✅ baml-cli Linux 🔒 27.3 MB 11.6 MB file 27.4 MB -94.9 KB (-0.3%) OK
✅ packed-program Linux 🔒 17.7 MB 7.3 MB file 17.7 MB +27.2 KB (+0.2%) OK
✅ baml-cli macOS 🔒 21.2 MB 10.1 MB file 21.3 MB -101.1 KB (-0.5%) OK
✅ packed-program macOS 🔒 13.8 MB 6.4 MB file 13.8 MB -4.6 KB (-0.0%) OK
✅ baml-cli Windows 🔒 22.8 MB 10.3 MB file 23.0 MB -140.9 KB (-0.6%) OK
✅ packed-program Windows 🔒 14.8 MB 6.5 MB file 14.8 MB -12.2 KB (-0.1%) OK
✅ bridge_wasm WASM 16.8 MB 🔒 4.6 MB gzip 4.6 MB -41.5 KB (-0.9%) OK

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.


Generated by cargo size-gate · workflow run

@sxlijin
sxlijin marked this pull request as ready for review July 31, 2026 20:23
@sxlijin

sxlijin commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 31, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
baml_language/crates/bex_project/src/lib.rs (2)

117-121: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Couple the serde field name to BYTECODE_METADATA_TABLE.

Line 119 repeats the literal "__baml_codegen" that line 99 already defines as BYTECODE_METADATA_TABLE. serde(rename = ...) requires a literal, so the compiler cannot detect divergence. If the const changes, the producer writes the new table name and the runtime rejects every payload with a "missing table" error.

Add a unit test that pins the two together.

♻️ Suggested guard test
#[test]
fn metadata_table_name_matches_serde_rename() {
    let toml = format!("[{BYTECODE_METADATA_TABLE}]\nbaml_cli_version = \"x\"\nbytecode_format_version = 1\n");
    let parsed: EmbeddedBamlToml = toml::from_str(&toml).expect("parse");
    assert!(parsed.codegen.is_some());
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@baml_language/crates/bex_project/src/lib.rs` around lines 117 - 121, Keep
EmbeddedBamlToml’s serde rename synchronized with BYTECODE_METADATA_TABLE by
adding the suggested unit test in the existing test area. Construct TOML using
BYTECODE_METADATA_TABLE, deserialize it into EmbeddedBamlToml, and assert
parsed.codegen is present so future constant changes expose a mismatch.

206-226: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Pin the envelope prefix layout with a test.

Lines 206-210 assume the Borsh encoding places envelope_version at bytes 8..12. That holds only while magic is declared before envelope_version in BytecodeEnvelope. If a later change reorders the fields, this pre-check reads the baml_toml length prefix instead and rejects valid bytecode with a wrong envelope-version message. No test currently covers the byte offsets or an envelope-version mismatch.

Note also that lines 222-226 cannot be reached. Line 200 already proves the first 8 bytes equal BYTECODE_ENVELOPE_MAGIC, and Borsh decodes those same bytes into magic.

♻️ Suggested tests
#[test]
fn envelope_prefix_layout_is_stable() {
    let bytes = borsh::to_vec(&BytecodeEnvelope {
        magic: BYTECODE_ENVELOPE_MAGIC,
        envelope_version: BYTECODE_ENVELOPE_VERSION,
        baml_toml: String::new(),
        program: Vec::new(),
    })
    .expect("envelope");
    assert_eq!(&bytes[..8], &BYTECODE_ENVELOPE_MAGIC);
    assert_eq!(
        u32::from_le_bytes(bytes[8..12].try_into().expect("prefix")),
        BYTECODE_ENVELOPE_VERSION
    );
}

#[test]
fn rejects_unsupported_envelope_version() {
    let mut bytes = borsh::to_vec(&BytecodeEnvelope {
        magic: BYTECODE_ENVELOPE_MAGIC,
        envelope_version: BYTECODE_ENVELOPE_VERSION + 1,
        baml_toml: manifest(baml_version::CANONICAL_VERSION, BYTECODE_FORMAT_VERSION),
        program: Vec::new(),
    })
    .expect("envelope");
    let message = deserialize_bytecode(&bytes)
        .expect_err("envelope version mismatch")
        .to_string();
    assert!(message.contains("envelope version"), "{message}");
    bytes.truncate(10);
    let truncated = deserialize_bytecode(&bytes).expect_err("truncated");
    assert!(truncated.to_string().contains("truncated"), "{truncated}");
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@baml_language/crates/bex_project/src/lib.rs` around lines 206 - 226, Add
tests for deserialize_bytecode that serialize BytecodeEnvelope and assert the
magic occupies bytes 0..8 and envelope_version is encoded at bytes 8..12,
pinning the prefix layout. Add coverage for an unsupported envelope version and
a truncated prefix, asserting both produce the expected compatibility errors;
also remove or avoid relying on the unreachable envelope.magic validation after
the existing prefix check.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@baml_language/sdk_tests/harness_setup/src/csharp.rs`:
- Around line 119-126: The fixture bytecode paths currently serialize synthetic
manifests instead of preserving each fixture’s project manifest. In
baml_language/sdk_tests/harness_setup/src/csharp.rs lines 119-126, read the
fixture baml.toml, inject the reserved compatibility metadata into an in-memory
copy, and serialize it; apply the same manifest-copy change in
baml_language/sdk_tests/harness_setup/src/lib.rs lines 229-236, preserving all
existing project settings in both paths.

---

Nitpick comments:
In `@baml_language/crates/bex_project/src/lib.rs`:
- Around line 117-121: Keep EmbeddedBamlToml’s serde rename synchronized with
BYTECODE_METADATA_TABLE by adding the suggested unit test in the existing test
area. Construct TOML using BYTECODE_METADATA_TABLE, deserialize it into
EmbeddedBamlToml, and assert parsed.codegen is present so future constant
changes expose a mismatch.
- Around line 206-226: Add tests for deserialize_bytecode that serialize
BytecodeEnvelope and assert the magic occupies bytes 0..8 and envelope_version
is encoded at bytes 8..12, pinning the prefix layout. Add coverage for an
unsupported envelope version and a truncated prefix, asserting both produce the
expected compatibility errors; also remove or avoid relying on the unreachable
envelope.magic validation after the existing prefix check.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 10425171-d315-438a-828b-764bd712025a

📥 Commits

Reviewing files that changed from the base of the PR and between 098a8ce and ab33c73.

⛔ Files ignored due to path filters (1)
  • baml_language/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (15)
  • baml_language/crates/baml_cli/Cargo.toml
  • baml_language/crates/baml_cli/src/generate.rs
  • baml_language/crates/baml_lsp_server/src/playground_server.rs
  • baml_language/crates/bex_project/Cargo.toml
  • baml_language/crates/bex_project/src/lib.rs
  • baml_language/crates/bridge_cffi/src/baml_to_host.rs
  • baml_language/crates/bridge_cffi/src/ffi/runtime.rs
  • baml_language/crates/bridge_cffi/src/lib.rs
  • baml_language/sdk_tests/harness_setup/Cargo.toml
  • baml_language/sdk_tests/harness_setup/src/csharp.rs
  • baml_language/sdk_tests/harness_setup/src/lib.rs
  • baml_language/sdks/csharp/sdkgen_csharp/Cargo.toml
  • baml_language/sdks/csharp/sdkgen_csharp/src/lib.rs
  • baml_language/sdks/csharp/sdkgen_csharp/src/model.rs
  • baml_language/sdks/typescript/bridge_typescript_web/src/errors.rs
💤 Files with no reviewable changes (1)
  • baml_language/sdks/csharp/sdkgen_csharp/Cargo.toml

@sxlijin
sxlijin force-pushed the agent/bytecode-version-skew branch from ab33c73 to a3cf7ea Compare July 31, 2026 20:40
@vercel
vercel Bot temporarily deployed to Preview – beps July 31, 2026 20:41 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 July 31, 2026 20:49 Inactive

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
baml_language/sdk_tests/harness_setup/src/lib.rs (1)

150-180: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add focused unit coverage for manifest augmentation.

If coverage does not already exist, test missing baml.toml, malformed TOML, the reserved metadata table, and the exact injected version values. Keep these tests in the Rust module where possible.

As per coding guidelines, prefer Rust unit tests over integration tests where possible.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@baml_language/sdk_tests/harness_setup/src/lib.rs` around lines 150 - 180, Add
focused Rust unit tests for embedded_fixture_baml_toml covering a missing
baml.toml, malformed TOML, an existing reserved BYTECODE_METADATA_TABLE, and
successful augmentation with exact CANONICAL_VERSION and BYTECODE_FORMAT_VERSION
values. Keep the tests in the same Rust module and assert the relevant fallback
output or panic behavior without changing the production function.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@baml_language/sdk_tests/harness_setup/src/lib.rs`:
- Around line 150-180: Add focused Rust unit tests for
embedded_fixture_baml_toml covering a missing baml.toml, malformed TOML, an
existing reserved BYTECODE_METADATA_TABLE, and successful augmentation with
exact CANONICAL_VERSION and BYTECODE_FORMAT_VERSION values. Keep the tests in
the same Rust module and assert the relevant fallback output or panic behavior
without changing the production function.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 82512876-249b-45f6-845c-2e550ee43450

📥 Commits

Reviewing files that changed from the base of the PR and between ab33c73 and a3cf7ea.

⛔ Files ignored due to path filters (1)
  • baml_language/Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (15)
  • baml_language/crates/baml_cli/Cargo.toml
  • baml_language/crates/baml_cli/src/generate.rs
  • baml_language/crates/baml_lsp_server/src/playground_server.rs
  • baml_language/crates/bex_project/Cargo.toml
  • baml_language/crates/bex_project/src/lib.rs
  • baml_language/crates/bridge_cffi/src/baml_to_host.rs
  • baml_language/crates/bridge_cffi/src/ffi/runtime.rs
  • baml_language/crates/bridge_cffi/src/lib.rs
  • baml_language/sdk_tests/harness_setup/Cargo.toml
  • baml_language/sdk_tests/harness_setup/src/csharp.rs
  • baml_language/sdk_tests/harness_setup/src/lib.rs
  • baml_language/sdks/csharp/sdkgen_csharp/Cargo.toml
  • baml_language/sdks/csharp/sdkgen_csharp/src/lib.rs
  • baml_language/sdks/csharp/sdkgen_csharp/src/model.rs
  • baml_language/sdks/typescript/bridge_typescript_web/src/errors.rs
💤 Files with no reviewable changes (1)
  • baml_language/sdks/csharp/sdkgen_csharp/Cargo.toml
🚧 Files skipped from review as they are similar to previous changes (12)
  • baml_language/crates/baml_cli/Cargo.toml
  • baml_language/crates/bridge_cffi/src/baml_to_host.rs
  • baml_language/sdk_tests/harness_setup/src/csharp.rs
  • baml_language/sdks/csharp/sdkgen_csharp/src/model.rs
  • baml_language/crates/bridge_cffi/src/lib.rs
  • baml_language/sdk_tests/harness_setup/Cargo.toml
  • baml_language/crates/bex_project/Cargo.toml
  • baml_language/sdks/csharp/sdkgen_csharp/src/lib.rs
  • baml_language/crates/bridge_cffi/src/ffi/runtime.rs
  • baml_language/crates/baml_cli/src/generate.rs
  • baml_language/sdks/typescript/bridge_typescript_web/src/errors.rs
  • baml_language/crates/bex_project/src/lib.rs

This branch was previously deployed

2 inactive deployments
Preview – promptfiddle2 — a3cf7eaa Deployed Jul 31, 2026 by vercel[bot]
Preview – beps — a3cf7eaa Deployed Jul 31, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant