Skip to content

[codex] Add render_null_as output format option - #3822

Merged
aaronvg merged 5 commits into
canaryfrom
codex/render-null-as
Jun 23, 2026
Merged

aaronvg merged 5 commits into
canaryfrom
codex/render-null-as

Conversation

@aaronvg

@aaronvg aaronvg commented Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds a render_null_as option to ctx.output_format(...) so nullable schema members can render a custom marker such as omit instead of null.

This is wired through both output-format implementations:

  • engine/baml-lib/jinja-runtime for the existing engine runtime and Jinja type checker
  • baml_language for the newer sys_llm renderer, legacy Jinja kwargs, typed ctx.output_format_with(...), and template type checking

Impact

Prompt authors can write:

{{ ctx.output_format(render_null_as='omit') }}

or, in prompt tags:

${ctx.output_format_with(render_null_as = "omit")}

and nullable fields render like string or omit, which gives models an explicit omit-key instruction surface.

Validation

  • cargo nextest run -p sys_llm output_format
  • cargo nextest run -p sys_jinja_types test_output_format
  • cargo nextest run -p bex_engine test_output_format_can_render_null_as_omit
  • cargo nextest run -p baml_tests prompt_output_format_with_can_render_null_as_omit
  • cargo nextest run -p baml_tests __baml_std__::test_03_hir
  • cargo nextest run -p baml_tests __baml_std__::test_06_codegen
  • cargo nextest run -p internal-baml-jinja output_format
  • cargo nextest run -p internal-baml-jinja-types test_output_format

Note

Low Risk
Additive, optional prompt-schema rendering change with no auth or response-parsing behavior; risk is mainly inconsistent wording if one runtime path were missed, which tests cover.

Overview
Adds optional render_null_as on parameterized output-format rendering so nullable schema text can use a custom token (e.g. omit) instead of null.

Exposed on Context.output_format_with(...) and Jinja ctx.output_format(render_null_as='…'), with the option threaded through sys_llm / sys_ops and the legacy engine/baml-lib/jinja-runtime renderer. Nullable unions, bare null types, and related media instructions now substitute the chosen label in rendered prompts.

Jinja type checkers accept none | string for the kwarg; new engine, baml_tests, and sys_llm tests assert schemas like nickname: string or omit,.

Reviewed by Cursor Bugbot for commit 515f318. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • New Features
    • Added optional render_null_as support to output-format schema rendering, letting you customize how null is worded for nullable fields (e.g., rendering as "omit" instead of the default).
    • Exposed render_null_as through template helpers and rendering APIs, with validation to accept only none/null or a string value.
  • Tests
    • Added/extended coverage to confirm custom null wording appears in rendered schemas and prompt instructions, and that invalid render_null_as values are rejected.

@vercel

vercel Bot commented Jun 22, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
beps Ready Ready Preview, Comment Jun 23, 2026 5:59pm
promptfiddle2 Ready Ready Preview, Comment Jun 23, 2026 5:59pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
promptfiddle Skipped Skipped Jun 23, 2026 5:59pm

Request Review

@coderabbitai

coderabbitai Bot commented Jun 22, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 1fa26735-4e7c-4c63-8c49-3c3ff6ab0fc1

📥 Commits

Reviewing files that changed from the base of the PR and between be2408c and 515f318.

📒 Files selected for processing (1)
  • engine/baml-lib/jinja-runtime/src/output_format/types.rs

📝 Walkthrough

Walkthrough

A new render_null_as optional parameter is added to the output-format schema rendering pipeline in both the engine and baml_language stacks. When set, it substitutes a caller-specified string (e.g., "omit") for the hardcoded "null" literal in generated output-format schemas. The change threads through type registration, Jinja kwarg parsing, RenderOptions struct, core rendering helpers, public APIs, and comprehensive tests across both implementations.

Changes

render_null_as option for output format rendering

Layer / File(s) Summary
RenderOptions struct and core null rendering
engine/baml-lib/jinja-runtime/src/output_format/types.rs, baml_language/crates/sys_llm/src/types/output_format.rs
RenderOptions gains render_null_as: RenderSetting<String> (defaulting to Auto) in both stacks. TypeValue::Null / RuntimeTy::Null rendering is redirected from the hardcoded "null" string to new render_null_type / rendered_null_type helpers that consult the option. media_output_instruction receives &RenderOptions to interpolate the configured null string in instruction text.
Engine null rendering in prompts and schemas
engine/baml-lib/jinja-runtime/src/output_format/types.rs
The engine implementation updates multiple rendering paths (auto_prefix, prompt/prefix generation for union null branches, inner_type_render) to use the new render_null_type helper instead of hardcoding "null". A new private helper render_null_type implements the dispatch logic.
Jinja kwarg parsing and public API wiring
engine/baml-lib/jinja-runtime/src/output_format/mod.rs, baml_language/crates/sys_llm/src/jinja/output_format_object.rs, baml_language/crates/sys_llm/src/lib.rs, baml_language/crates/sys_ops/src/lib.rs
Both OutputFormat::call implementations parse the new render_null_as kwarg (string or null), add it to the allowed-keywords list, and forward it into RenderOptions::new. A new helper parse_render_null_as_kwarg implements absent→Auto, null→Auto, string→Always semantics. The public render_output_format_content function and IoClassLlmContext::output_format_with gain a new Option<String> parameter.
Type system registration
engine/baml-lib/jinja/src/evaluate_type/types.rs, baml_language/crates/sys_jinja_types/src/evaluate_type/types.rs, baml_language/crates/baml_builtins2/baml_std/baml/ns_llm/llm_types.baml
PredefinedTypes::default in both stacks registers render_null_as as a `String
Tests
engine/baml-lib/jinja-runtime/src/output_format/types.rs, engine/baml-lib/jinja/src/evaluate_type/test_expr.rs, baml_language/crates/sys_llm/src/types/output_format.rs, baml_language/crates/sys_jinja_types/src/evaluate_type/test_expr.rs, baml_language/crates/bex_engine/tests/llm_render.rs, baml_language/crates/baml_tests/tests/prompt_tag_runtime.rs
Unit and integration tests verify that optional and media types render with the custom null token, that type-checking accepts render_null_as='omit' as Type::String and rejects integer values with a type error, and that end-to-end prompt assembly produces "string or omit" rather than "string or null". Existing tests are updated to pass the new parameter.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • BoundaryML/baml#3699: Modifies the same RenderOptions/render_type_hoisted/media_output_instruction paths in sys_llm for nullable union handling, directly touching the code this PR extends.

Poem

🐇 A null by any other name
Would render just the same—
Unless you pass render_null_as,
Then "omit" joins the game!
No more hardcoded "null" in sight,
The rabbit hops with pure delight. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title clearly and concisely describes the main change: adding a render_null_as output format option to customize null value rendering.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/render-null-as

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jun 22, 2026 •

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@vercel
vercel Bot temporarily deployed to Preview – beps June 22, 2026 18:04 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 June 22, 2026 18:11 Inactive
@github-actions

Copy link
Copy Markdown

No description provided.

@vercel
vercel Bot temporarily deployed to Preview – promptfiddle June 22, 2026 18:23 Inactive
@aaronvg
aaronvg marked this pull request as ready for review June 22, 2026 22:34
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle June 22, 2026 22:56 Inactive
@vercel
vercel Bot temporarily deployed to Preview – beps June 22, 2026 22:57 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 June 22, 2026 23:03 Inactive
@vercel
vercel Bot temporarily deployed to Preview – beps June 23, 2026 16:29 Inactive

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1299e67. Configure here.

@vercel
vercel Bot temporarily deployed to Preview – beps June 23, 2026 16:32 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 June 23, 2026 16:39 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle June 23, 2026 16:50 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle June 23, 2026 17:52 Inactive
@vercel
vercel Bot temporarily deployed to Preview – beps June 23, 2026 17:53 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 June 23, 2026 17:59 Inactive
@aaronvg
aaronvg added this pull request to the merge queue Jun 23, 2026
Merged via the queue into canary with commit e32d2df Jun 23, 2026
65 checks passed
@aaronvg
aaronvg deleted the codex/render-null-as branch June 23, 2026 19:49
@cursor cursor Bot mentioned this pull request Jun 23, 2026
meefs pushed a commit to meefs/baml that referenced this pull request Jun 23, 2026
Bump version to 0.223.0

❌ Typescript integ tests
❌ Python integ tests

Generated by bump-version script.

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Mechanical version and generated-client updates; no application logic
changes appear in this diff beyond release metadata.
> 
> **Overview**
> Releases **0.223.0** by aligning workspace and client package versions
from **0.222.0** to **0.223.0** (Rust workspace, `Cargo.lock`,
Go/Python/Ruby/TypeScript bindings, Zed extension, and integ-test
generators plus regenerated `baml_client` artifacts).
> 
> **CHANGELOG** and **fern/pages/changelog.mdx** document the
user-facing addition: **`render_null_as`** output format option (BoundaryML#3822),
bundled with this version bump.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
665d406. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Added `render_null_as` output format option

* **Chores**
  * Updated version to 0.223.0

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
wthrbtn added a commit to air-gapped/skills that referenced this pull request Jul 21, 2026
…ne trap

Classification: version-drift + new-feature + fresh
Sources:
  https://pypi.org/pypi/baml-py/json (0.223.0, 2026-06-23)
  https://registry.npmjs.org/@boundaryml/baml/latest (0.223.0)
  https://github.com/boundaryml/baml/blob/canary/fern/pages/changelog.mdx
  BoundaryML/baml#3822

- SDK 0.222.0 -> 0.223.0 across the header, GitHub, PyPI and npm rows.
- ctx.output_format(render_null_as="omit") added to SKILL.md's tunables
  (PR #3822, shipped 0.223.0). In scope: ctx.output_format is a named
  trigger in the skill's own description.
- Documented the two-version-line trap: the repo now cuts
  baml-language-0.NN.N releases (0.15.0, 2026-07-14) plus per-commit
  nightlies for the baml_language compiler workspace, so `gh release
  list` reports a 0.15.x nightly as isLatest. A future freshen reading
  that as "the version" would downgrade a 0.22x SDK pin.
- canary-features.md reframed: the 0.221.0 set has SHIPPED, so the load
  rule moves from audience-based ("only inside the BAML repo") to
  feature-based ("project pins baml-py>=0.221"). Closes a backlog item
  carried since 2026-05-28.
- Re-stamped 4 docs.boundaryml.com rows to 2026-07-21 (all fresh).
  WebFetch renders the Mintlify SPA pages, which the previous pass had
  recorded as impossible -- that blocker was wrong.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

This branch was previously deployed

3 inactive deployments
Preview – promptfiddle2 — 515f318a Deployed Jun 23, 2026 by vercel[bot]
Preview – beps — 515f318a Deployed Jun 23, 2026 by vercel[bot]
Preview – promptfiddle — 515f318a Deployed Jun 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant