Skip to content

fix(release): trust Homebrew formula before auditing it - #5546

Merged
codeshaunted merged 1 commit into
canaryfrom
avery/homebrew-tap-trust
Nov 6, 2026
Merged

codeshaunted merged 1 commit into
canaryfrom
avery/homebrew-tap-trust

Conversation

@codeshaunted

@codeshaunted codeshaunted commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Issue Reference

Failed canary release: https://github.com/BoundaryML/baml/actions/runs/42111122075/job/127577500967

Changes

  • Run brew trust --formula boundaryml/tap/baml before brew audit in the Homebrew publish job. Homebrew 7.0.0 made tap trust mandatory, and only brew install trusts a fully qualified formula automatically. The last real publish was on 2026-09-11, before 7.0.0, so this is the first wrapper release to hit the refusal.
  • Add brew trust and brew audit --strict to scripts/baml-test-homebrew-package. The dry run skipped audit, so it passed on the same runner image.
  • Move the dry-run script's injected keg_only and lowercase its reason. --strict rejected the old placement and capitalization.

Testing

Run locally on Homebrew 7.0.11 with the 0.2.6 artifacts from dry-run release 41981648965 (the 0.3.0 formula was not used):

  • Publish commands: brew tap boundaryml/tap, copy in the generated formula, brew audit --strict boundaryml/tap/baml. This reproduces the CI refusal (exit 1). After brew trust --formula boundaryml/tap/baml, the same audit exits 0.
  • Dry run: python3 scripts/baml-test-homebrew-package --formula <formula> --wrapper-dir <wrapper> exits 0, covering trust, audit, install, test, and linkage.

The publish job's install, test, and linkage steps were not run against the real tap.

Summary by CodeRabbit

  • Chores
    • Updated release checks to verify and audit the Homebrew package before installation and testing.
    • Improved consistency between automated release checks and local Homebrew package testing.

Homebrew 7 refuses to load formulae from untrusted taps, and only
`brew install` trusts a fully qualified formula automatically. The publish
job runs `brew audit` first, so it failed on the first wrapper release since
Homebrew 7.

Add `brew audit --strict` to the dry-run package test so it catches this
class of failure. Move its injected `keg_only` so the audit passes.
@vercel

vercel Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
developer-docs Ready Ready Preview Nov 5, 2026 11:33pm UTC

Request Review

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@coderabbitai

coderabbitai Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The Homebrew package test script updates where it inserts the formula's keg_only declaration. The script and release workflow now trust the formula before audit, installation, and testing.

Changes: Homebrew formula verification

Layer / File(s) Summary
Prepare and verify the formula
scripts/baml-test-homebrew-package, .github/workflows/release-baml-language.yml
The script inserts keg_only "release verification only" before the macOS block, then runs brew trust and brew audit --strict before installation. The release workflow runs brew trust before auditing, installing, and testing the formula.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Possibly related PRs

  • BoundaryML/baml#5483: Adds the Homebrew dry-run script and its release verification job, which this PR extends with formula trust and strict auditing.

Suggested reviewers: sxlijin

Merge Risk: Minimal · up to dbf1d

The formula is trusted before verification. No merge-blocking issue is evident.

Architecture risk: Low · up to dbf1d

The change affects 1 system.

Changed systems: scripts

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — scripts (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in scripts/baml-test-homebrew-package: The keg_only declaration is now inserted immediately before on_macos; the previous insertion directly after the formula class declaration, with differently capitalized text, was removed.
  • observed — Modified behavior in scripts/baml-test-homebrew-package: After committing the temporary tap, the script now trusts the formula and runs a strict Homebrew audit before installation.
  • observed — Modified behavior in .github/workflows/release-baml-language.yml: Adds a brew trust command for the tapped formula before the subsequent audit, installation, and tests.
Pre-merge checks: Passed 5

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly describes the main change: trust the Homebrew formula before auditing it.
Docstring Coverage Passed No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 0 files. (2 skipped: 2 unsupported.)

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.

A rabbit checks the formula’s name
Then trusts it before the audit game
The keg-only line moves into place
The tests proceed at hopping pace
All done, the rabbit leaves a trace

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c4b209f4-141d-4269-92a5-acc9669e4922
Commits

Reviewing files that changed from the base of the PR and between 1d9f45c and dbf1d2b.

Files selected for processing: 2

  • .github/workflows/release-baml-language.yml
  • scripts/baml-test-homebrew-package

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Binary size checks failed

❌ 2 violations · ✅ 1 passed

⚠️ Please fix the size gate issues or acknowledge them by updating baselines.

Artifact Platform File Gzip Gated on Baseline Delta Status
✅ baml-cli Linux 🔒 64.7 MB 27.5 MB file 73.1 MB -8.4 MB (-11.5%) OK
❌ packed-program Linux 🔒 39.1 MB 16.1 MB file 29.2 MB +10.0 MB (+34.3%) FAIL
❌ bridge_wasm WASM 26.0 MB 🔒 7.7 MB gzip 5.7 MB +2.0 MB (+35.7%) FAIL

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.

Details & how to fix

Violations:

  • packed-program (Linux) file_bytes: 39.1 MB exceeds limit of 30.1 MB (exceeded by +9.0 MB, policy: max_file_bytes)
  • packed-program (Linux) file_delta_pct: +34.3% exceeds limit of 3.0% (exceeded by +31.3pp, policy: max_delta_pct)
  • bridge_wasm (WASM) gzip_bytes: 7.7 MB exceeds limit of 5.9 MB (exceeded by +1.8 MB, policy: max_gzip_bytes)
  • bridge_wasm (WASM) gzip_delta_pct: +35.7% exceeds limit of 3.0% (exceeded by +32.7pp, policy: max_delta_pct)

Add/update baselines:

.ci/size-gate/wasm32-unknown-unknown.toml:

[artifacts.bridge_wasm]
file_bytes = 26012882
gzip_bytes = 7684668

.ci/size-gate/x86_64-unknown-linux-gnu.toml:

[artifacts.packed-program]
file_bytes = 39141528
gzip_bytes = 16087461

Generated by cargo size-gate · workflow run

@codeshaunted
codeshaunted added this pull request to the merge queue Nov 5, 2026
Merged via the queue into canary with commit 845c124 Nov 6, 2026
89 of 91 checks passed
@codeshaunted
codeshaunted deleted the avery/homebrew-tap-trust branch November 6, 2026 00:12
@rossirpaulo rossirpaulo mentioned this pull request Nov 6, 2026
meefs pushed a commit to meefs/baml that referenced this pull request Nov 6, 2026
Bumps the BAML language version from 0.24.0 to 0.25.0 with
`scripts/baml-language-version bump --minor`. Only version strings
change: 20 files, and `baml-language-version check` passes.

The 0.24.0 canary release ([run
42111122075](https://github.com/BoundaryML/baml/actions/runs/42111122075))
failed at "Publish Homebrew wrapper formula" with "Refusing to load
formula boundaryml/tap/baml from untrusted tap". BoundaryML#5546 fixed that.
Because of the failure:
- PyPI, npm and crates.io have 0.24.0.
- The Go SDK mirror, the `pkg.boundaryml.com` manifests and the
developer docs were not published.
- The canary channel was not promoted.

Rather than re-run 0.24.0, this releases 0.25.0 from current canary.
That includes BoundaryML#5546 and the `package_name` setting (BoundaryML#5532, BoundaryML#5533).

For the release oncall: the 0.3.0 wrapper got its GitHub release before
the Homebrew step failed. RELEASING.md says a language release
republishes the wrapper only when its version differs from the published
one, so check that this release still publishes the Homebrew formula.

🤖 Generated with [Claude Code](https://claude.com/claude-code)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Release**
* Updated BAML and its bridge packages and integrations to version
0.25.0.
  * The canary release channel remains unchanged.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Opus 5.5 <[email protected]>

This branch was successfully deployed

1 active deployment
Preview – developer-docs — dbf1d2bd Deployed Nov 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant