Skip to content

test(cli): preserve slashes in test names - #4402

Closed
sxlijin wants to merge 1 commit into
canaryfrom
sxlijin/B-360-test-name-slash-separator
Closed

sxlijin wants to merge 1 commit into
canaryfrom
sxlijin/B-360-test-name-slash-separator

Conversation

@sxlijin

@sxlijin sxlijin commented Aug 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • add an end-to-end regression for B-360 using the reported alpha/beta gamma test name
  • verify baml test --list preserves the literal slash in the canonical ID
  • verify executed PASS output preserves the literal slash and never renders it as ::

Context

The historical renderer split test paths on /, which made a user-provided slash look like the :: hierarchy separator. The canonical-ID path on canary now carries the complete test ID through rendering without splitting on /; this test locks that behavior at the CLI boundary.

Validation

  • cargo test -p baml_cli --test exit_code_e2e test_slash_in_name_is_rendered_verbatim -- --nocapture
  • cargo test -p baml_cli --test test_profiles_e2e failed_expansion_is_not_cached_and_literal_slash_selector_is_legal -- --nocapture
  • cargo fmt --check --package baml_cli
  • git diff --check

Summary by CodeRabbit

  • Tests
    • Added regression coverage ensuring test names containing / appear verbatim when listing tests and in execution results.

@linear

linear Bot commented Aug 13, 2026

Copy link
Copy Markdown

B-360

@vercel

vercel Bot commented Aug 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
beps Ready Ready Preview Aug 13, 2026 7:12pm
promptfiddle2 Ready Ready Preview Aug 13, 2026 7:12pm

Request Review

@coderabbitai

coderabbitai Bot commented Aug 13, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ee29bdee-f070-437b-9edd-a976ad2f2770

📥 Commits

Reviewing files that changed from the base of the PR and between f377531 and 0433dda.

📒 Files selected for processing (1)
  • baml_language/crates/baml_cli/tests/exit_code_e2e.rs

📝 Walkthrough

Walkthrough

Adds an end-to-end regression test for a test named alpha/beta gamma. The test verifies that baml test --list and executed test results preserve the slash in the test identifier.

Changes

Slash-Preserving Test Names

Layer / File(s) Summary
Slash rendering regression test
baml_language/crates/baml_cli/tests/exit_code_e2e.rs
Adds a test that verifies listing and execution output preserve alpha/beta gamma instead of converting / to ::.

Estimated code review effort: 2 (Simple) | ~10 minutes

Mergeability Score: 🔵 Low · up to 0433d

This PR adds CLI regression coverage for preserving literal slashes in test names without changing shipped runtime behavior. The targeted integration test could not run because the build environment could not execute the C compiler, so it should be rerun in a working environment before treating validation as complete.

Possibly related PRs

Suggested reviewers: aaronvg, hellovai, codeshaunted

Poem

A rabbit checks the test name bright,
With slash intact from left to right.
The list shows paths just as they came,
Execution echoes the same name.
No doubled colons hide the trail.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the regression test for preserving slashes in CLI test names.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch sxlijin/B-360-test-name-slash-separator

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vercel
vercel Bot temporarily deployed to Preview – beps August 13, 2026 19:05 Inactive
@github-actions

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 August 13, 2026 19:12 Inactive
@github-actions

Copy link
Copy Markdown

Binary size checks passed

✅ 7 passed

Artifact Platform File Gzip Gated on Baseline Delta Status
✅ baml-cli Linux 🔒 26.2 MB 11.1 MB file 27.4 MB -1.2 MB (-4.4%) OK
✅ packed-program Linux 🔒 16.9 MB 6.8 MB file 18.6 MB -1.7 MB (-8.9%) OK
✅ baml-cli macOS 🔒 20.4 MB 9.7 MB file 21.3 MB -913.9 KB (-4.3%) OK
✅ packed-program macOS 🔒 13.3 MB 6.0 MB file 14.5 MB -1.2 MB (-8.1%) OK
✅ baml-cli Windows 🔒 21.9 MB 9.9 MB file 23.0 MB -1.1 MB (-4.6%) OK
✅ packed-program Windows 🔒 14.1 MB 6.1 MB file 15.5 MB -1.4 MB (-9.1%) OK
✅ bridge_wasm WASM 15.5 MB 🔒 4.2 MB gzip 4.6 MB -437.3 KB (-9.5%) OK

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.


Generated by cargo size-gate · workflow run

sxlijin commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Closing this duplicate because the behavior was already fixed by #4076, which made literal / legal in test names and preserved canonical IDs in baml test output. B-360 has been linked to #4076 and marked Done.

@sxlijin sxlijin closed this Aug 13, 2026
@sxlijin
sxlijin deleted the sxlijin/B-360-test-name-slash-separator branch August 13, 2026 22:52

This branch was previously deployed

2 inactive deployments
Preview – promptfiddle2 — 0433ddad Deployed Aug 13, 2026 by vercel[bot]
Preview – beps — 0433ddad Deployed Aug 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant