Skip to content

test(lsp): make didOpen filesystem assertion deterministic - #4626

Merged
hellovai merged 1 commit into
canaryfrom
codex/fix-did-open-discovery-race
Aug 28, 2026
Merged

hellovai merged 1 commit into
canaryfrom
codex/fix-did-open-discovery-race

Conversation

@hellovai

@hellovai hellovai commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Stack context

This PR targets canary and is the bottom layer of the active stack. #4593 is stacked on top of it. The change is independent of the compiler behavior in that PR and can land first.

Problem

did_open_never_reads_the_opened_file uses a real thread pool for filesystem discovery.

After h.open(...) queues discovery, a worker may immediately discover the project and read other.baml before the test checks that the read log is empty:

owner:  didOpen ── queue discovery ── return ── assert no reads
worker:                   └────────── read other.baml

The resulting failure does not mean that the opened main.baml buffer was read from disk. It only means that legitimate asynchronous sibling-file discovery won the scheduling race.

After this PR

This test supplies a manual executor for the I/O lane while retaining real thread pools for request and diagnostic work.

The sequence is now deterministic:

  1. Dispatch didOpen.
  2. Assert that synchronous handling performed no filesystem reads.
  3. Assert that exactly one discovery job was queued.
  4. Explicitly run the discovery job.
  5. Settle owner events and diagnostics.
  6. Preserve the existing assertion that discovery walked once and read only other.baml.

The production LSP implementation is unchanged.

Behavioral boundaries

Behavior Assertion
didOpen uses the editor buffer and performs no synchronous disk read Read log is empty before releasing I/O
Opening an unserved document queues project discovery Exactly one I/O job is parked
Discovery skips the opened document main.baml never appears in the read log
Discovery loads sibling source files Final read log is exactly other.baml
Diagnostics use the editor contents The opened document compiles without diagnostics

Validation

  • CI
  • Diff and whitespace checks
  • No production code changes

Stack created with GitHub Stacks CLI • Give Feedback 💬

Summary by CodeRabbit

  • Tests
    • Updated language-server protocol testing to verify that opening a document queues exactly one discovery job.
    • Added explicit execution and settling of the queued job for more reliable test behavior.

@vercel

vercel Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
beps Ready Ready Preview Aug 28, 2026 9:31pm
promptfiddle2 Ready Ready Preview Aug 28, 2026 9:31pm

Request Review

@coderabbitai

coderabbitai Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5154b095-a55e-4838-bdb8-8f78d3762327

📥 Commits

Reviewing files that changed from the base of the PR and between 4b17d9f and 007551a.

📒 Files selected for processing (1)
  • baml_language/crates/baml_lsp/tests/protocol.rs

Included review availability: Your plan provides up to 8 included reviews per hour; 4 remain after this review.


📝 Walkthrough

Walkthrough

The protocol test now uses explicit request, diagnostic, and manual I/O executors. It verifies that didOpen queues one discovery job and runs that job before the test continues.

Changes

Protocol test control

Layer / File(s) Summary
Manual I/O discovery execution
baml_language/crates/baml_lsp/tests/protocol.rs
The test configures explicit executors, asserts that didOpen queues one discovery job, and executes the job manually.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 00755

This PR makes the LSP filesystem assertion deterministic without changing production behavior or public interfaces. No actionable merge-blocking risk remains beyond normal checks and review.

Suggested reviewers: 2kai2kai2

Poem

A rabbit queued one task with care
Then ran it through the I/O air
The document opened bright
Discovery hopped into sight
And tests stayed neatly there

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: making the LSP test's didOpen filesystem assertion deterministic.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-did-open-discovery-race

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@vercel
vercel Bot temporarily deployed to Preview – beps August 28, 2026 21:18 Inactive
@hellovai
hellovai changed the base branch from codex/error-extraneous-throws-unknown to canary August 28, 2026 21:21
@hellovai
hellovai force-pushed the codex/fix-did-open-discovery-race branch from 889b9a9 to 007551a Compare August 28, 2026 21:21
@vercel
vercel Bot temporarily deployed to Preview – beps August 28, 2026 21:22 Inactive
@vercel
vercel Bot temporarily deployed to Preview – promptfiddle2 August 28, 2026 21:31 Inactive
@hellovai
hellovai added this pull request to the merge queue Aug 28, 2026
Merged via the queue into canary with commit 304898a Aug 28, 2026
40 checks passed
@hellovai
hellovai deleted the codex/fix-did-open-discovery-race branch August 28, 2026 22:01
pull Bot pushed a commit to rpatil524/baml that referenced this pull request Aug 29, 2026
…ML#4593)

## Stack context

[BoundaryML#4626](BoundaryML#4626), the independent
deterministic LSP test fix that formed the lower stack layer, is merged
into `canary`. [BoundaryML#4621](BoundaryML#4621),
whose pure-BAML compiler-test harness is used here, is also merged.
Review this PR relative to `canary`.

## Problem

BAML already infers the values that can escape through `throw`. An
explicit `throws` clause is a bound on those inferred values, but
`unknown` was treated as an automatic escape hatch from unused-contract
diagnostics.

That allowed declarations such as these to compile without explaining
the real behavior:

```baml
function DoesNotThrow() -> int throws unknown {
  1
}

function ThrowsOnlyInt() -> int throws unknown {
  throw 1
}
```

The first function does not throw. The second throws only `int`. In both
cases, `throws unknown` is broader than the implementation and hides
useful compiler information.

This precision requirement is specific to `unknown`. An ordinary
explicit bound may remain conservative: `throws A | B | C` is valid when
the function only throws `A | B`.

## After this PR

Both imprecise `unknown` declarations are rejected with E0097.

For a function that does not throw:

```text
`throws unknown` is unnecessary: BAML infers thrown types automatically, and this function does not throw. Remove the declaration; write an explicit `throws` type only to bound what may escape
```

For a function whose inferred thrown type is narrower:

```text
`throws unknown` is imprecise: this function only throws `int`. BAML infers thrown types automatically, so remove the declaration; write `throws int` only to explicitly bound what may escape
```

There are three intended remedies.

1. Let BAML infer the thrown type:

```baml
function ThrowsOnlyInt() -> int {
  throw 1
}
```

2. Keep an explicit, precise bound when the API contract should spell it
out:

```baml
function ThrowsOnlyInt() -> int throws int {
  throw 1
}
```

3. Keep `throws unknown` at a real unknown boundary:

```baml
function Rethrow(value: unknown) -> int throws unknown {
  throw value
}
```

## Behavioral boundaries

| Declaration and implementation | Result |
| --- | --- |
| `throws A \| B \| C`, only `A \| B` escapes | Compiles; ordinary
explicit bounds may be wider than inferred throws |
| `throws unknown`, no escaping throw | E0097: unnecessary |
| `throws unknown`, only `int` escapes | E0097: imprecise |
| `throws string \| unknown`, only `string` escapes | E0097: imprecise |
| `throws unknown`, an `unknown` value escapes directly | Compiles |
| `throws Alias`, where `type Alias = unknown`, and an unknown value
escapes | Compiles |
| `throws unknown`, unknown escapes through a called function | Compiles
|
| `throws string \| unknown`, both string and unknown can escape |
Compiles |
| `throws unknown \| UnusedError`, unknown escapes but `UnusedError`
does not | Compiles with the existing E0097 warning for the unused
concrete member |
| Signature-only interface requirement with `throws unknown` | Remains a
valid contract |

## Diagnostic edge cases

### Concrete superset bounds

The compiler does not require every concrete member of an explicit
`throws` union to appear in the current function body. A bound such as
`throws A | B | C` may intentionally reserve `C` for implementations or
future behavior even when inference currently finds only `A | B`.

`unknown` is different because it admits every value and therefore
erases all useful precision when no unknown-typed value can escape. Only
that unused open member becomes an error.

### Written union surfaces

Finalizing `unknown | SomeError` produces semantic `unknown`, which
would normally erase `SomeError`. The coverage check therefore preserves
the written surface for open unions so that an unused concrete member
still receives the existing warning.

Other contracts continue to use the finalized surface. This avoids false
unused-member warnings for types that reduce through projections or
aliases.

### Aliases and propagated throws

A direct `unknown`, an alias that resolves to `unknown`, and an
`unknown` value propagated through a call all satisfy the contract. The
diagnostic is based on the inferred escaping type, not only on a
syntactic `throw` in the current function.

### Compiler acceptance coverage

The former `ns_fixtures/ns_throws_unknown` corpus fixture has been
replaced with explicit `AssertCompiles` cases. Direct and aliased
unknown bounds, mixed unions, signature-only interface bounds, ordinary
unknown value flow, and callback error translation now state their
expected zero-error behavior at the public compiler boundary.

Iterator functions that either do not throw or fully handle their errors
no longer declare `throws unknown`.

The broader Rust integration-test programs also omit 177 executable
`throws unknown` convenience clauses. Signature-only interface
requirements, callback and function types, pattern checks, and the
dedicated precision tests retain their explicit open contracts.

## Diagnostic assertions

The public behavior is covered in pure BAML at:

```text
crates/baml_tests/baml_src/ns_compiler/ns_throws/ns_unknown_contracts/precision.baml
```

The tests use the shared `reflect.Package._compile` harness from BoundaryML#4621:

- `AssertRejected` and `ExactDiagnostic` lock the exact E0097 code and
message for direct, union, aliased, no-throw, and narrower-throw cases.
- `AssertCompiles` locks concrete superset bounds, accepted unknown
boundaries, transitive propagation, interface signatures, and callback
translation.
- A focused compiler test verifies that an unused concrete member beside
`unknown` remains a warning; successful `_compile` results intentionally
do not expose warnings.

## Validation

- [x] 9 pure-BAML `reflect.Package._compile` contract tests
- [x] Compiler/corpus snapshot coverage updated for the removed fixture
- [x] Mechanical audit of 177 executable test-helper migrations
- [x] Intentional signature and function-type `throws unknown` surfaces
preserved
- [x] Diff and whitespace checks
- [ ] CI after integration-test migration

---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added clearer diagnostics for missing required fields in class
constructors and object literals.
* Improved guidance when generic functions or methods are used without
specialization.
* Enhanced validation and messaging for `throws unknown` contracts,
including imprecise or unused error types.

* **Bug Fixes**
  * Prevented duplicate diagnostics for generated stream companions.
* Preserved warnings for unused concrete error types in open throws
contracts.
  * Improved handling of empty build metadata.

* **Tests**
* Added coverage for precise `throws unknown` behavior and updated
iterator exception scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: hellovai <[email protected]>

This branch was previously deployed

2 inactive deployments
Preview – promptfiddle2 — 007551a1 Deployed Aug 28, 2026 by vercel[bot]
Preview – beps — 007551a1 Deployed Aug 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant