Skip to content

feat(go): bring the Go SDK to parity with the JavaScript SDK - #1016

Open
0xdeafcafe wants to merge 176 commits into
mainfrom
feat/go-sdk-parity
Open

0xdeafcafe wants to merge 176 commits into
mainfrom
feat/go-sdk-parity

Conversation

@0xdeafcafe

@0xdeafcafe 0xdeafcafe commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This brings the Go Scenario SDK to parity with the JavaScript SDK. It supersedes #80: same goal, rebuilt on current main.

Core

  • Judge: two-phase, with a verdict per criterion (passed, failed, inconclusive), MinTurns/MaxTurns and additional context. Large traces get a structure-only view plus expand_trace/grep_trace, and traces can be fetched from LangWatch.
  • User simulator and script steps: match JS.
  • State views: ToolCalls, Transcript, Turns, Spans and the rest.
  • Events: sent to LangWatch in the same format as JS.
  • Tracing: set up once per process, safe with t.Parallel(). A "Scenario Run" span holds "Scenario Turn" spans and LLM spans for the built-in agents.
  • Embedding in a service: everything can be set per run (LangWatch config, ids, *http.Client), and results come back as values.

Modules

  • Agents and testing: evaluators, red-team (Crescendo and GOAT, plus the report), the Claude Code agent, browser tab handoff, and scenariotest for go test.
  • Voice (go/voice, its own module):
    • The runtime, script steps, interruptions, recording, and speech-to-text and text-to-speech.
    • All the effects.
    • Adapters for OpenAI Realtime, Gemini Live, ElevenLabs, composable, Twilio (including an already-accepted connection) and Pipecat.
    • Tunnels: cloudflared, with ngrok as its own module.
  • Providers: OpenAI, Anthropic, Gemini and Bedrock.
    • "provider/model" strings.
    • OpenAI-compatible endpoints: Azure, OpenRouter, Ollama, Groq and others.
    • Streaming helpers.
  • Build and release: per-module CI, a go.work workspace, a release-please component per module with linked versions, a live integration suite (manual workflow), and a Go-vs-JS benchmark harness (benchmarks/go-vs-js).

Performance

  • Judge digest: each span is rendered once and cached, JSON is re-encoded while scanning, and the span collector is sharded. A 10-turn fake run went from 12.7k to 1.8k allocations (1133 → 444 µs) before the last hot-spot pass.
  • Startup and voice: no work at init, and zero-allocation audio and WebSocket frame paths.
  • Upstream: the LangWatch Go SDK change in feat(sdk-go): send spans as OTLP/JSON so the root package no longer links gRPC langwatch#8578 removes gRPC from the import graph.

Test status (honest)

  • go vet passes in every module.
  • Each part passed go test -race before merging, up to the voice adapters and the first perf pass.
  • Not run yet: the tests of the last ten code-only lanes and of the merged result, because the dev machine was reserved for other work. This PR's CI is the first full run.
  • Not run: live tests against real providers and LangWatch.

Differences from JS

Listed in go/README.md under "Differences from the JavaScript SDK".

Review (CodeRabbit skipped this PR: 533 files)

Five reviewers split the PR by area: core, voice core, voice adapters, providers with the Claude Code agent, and CI/release/docs. Each finding was checked against the code, and every fix comes with a regression test. Fixed in the last 7 commits:

  • Core: AgentNonBlocking now picks the agent and takes its input before returning, as JS does. Previously a barge-in could reach the agent's input. Checkpoint criteria are merged only once. Clearing one run's spans no longer clears another run's spans in the same trace. A judge's own span collector and trace fetcher are cleared after each run. Evaluators still run when the run's context is cancelled.
  • Providers:
    • Gemini 3 tool calls keep their thought signature, so they are no longer rejected with a 400.
    • No empty text blocks are sent to Anthropic or Bedrock.
    • OpenAI streams whose chunk ids change keep every chunk.
    • Bedrock gets valid document names and formats, and tool choice none is honoured.
    • Missing media types are filled in.
    • Claude Code's whole process group is killed on stop.
  • Voice:
    • Local playback no longer blocks the turn, which had been inflating latency.
    • WebSocket close and pong are bounded.
    • Twilio restores only the voice URLs it actually read, and a failed embedded connect tears down.
    • cloudflared ignores the api.trycloudflare.com host.
    • Dial errors no longer print the signed URL's query string.
    • OpenAI Realtime and composable clean up properly on errors.
    • WAV headers are validated.
  • CI: go-live no longer passes its modules input through make, where it could run as shell code, and it now has a read-only token. go-ci reads needs from the environment. The workflow shape check covers go-ci. The changelog no longer links a 0.1.0 tag that was never pushed.
  • Docs: all 113 Go samples in the docs and README compile against the API.

Still open:

🤖 Generated with Claude Code

Extract LLM inference interface and message types into dedicated files.
Update agent interfaces and user simulator with refined API.
- Prefix event types with SCENARIO_ (SCENARIO_RUN_STARTED, etc.)
- Add ScenarioRunStatus type with uppercase values (SUCCESS, ERROR, FAILED)
- Rewrite eventalert with tmpdir file coordination across processes
- Show greeting banner only when API key is missing
- Use path-based watch URL format ({setUrl}/{batchRunId})
- Add SCENARIO_HEADLESS env var to suppress browser open
- Add SCENARIO_DISABLE_SIMULATION_REPORT_INFO env var to suppress banners
- Scope watch message per scenarioSetId
- Cache batch run ID per process with sync.Once
Add go.opentelemetry.io/otel, otel/sdk, otel/trace, otel/attribute,
otel/codes and github.com/langwatch/langwatch/sdk-go for full OTel
tracing integration.
- SpanCollector: implements sdktrace.SpanProcessor to collect spans,
  filters by thread ID with parent chain walking
- SpanDigestFormatter: renders spans as plain-text hierarchy with
  timestamps, durations, attributes, events, and error sections
- setupObservability: creates LangWatch exporter, TracerProvider, and
  SpanCollector; wires into global OTel provider
- Remove TracedInference and instrumentBuiltInAgents (replaced by OTel)
- Execution: create per-turn and per-agent spans with tracer, end spans
  at all exit paths
- JudgeAgent: build transcript from messages, include OTel trace digest
  in judge prompt alongside conversation transcript
- Runner: init observability when API key present, wire span collector
  to judge agents and execution, default endpoint to app.langwatch.ai,
  default SetID to "default", shutdown observability after run
Multi-provider inference abstraction supporting OpenAI, Anthropic,
Google Gemini, and AWS Bedrock with tool/function calling conversion.
LangWatch Work and others added 2 commits October 12, 2026 11:08
…passes

The passing judge ended the run on the first turn's judge step, so the
second turn, the one whose barge-in the test checks, never ran. The judge
now lets the run continue and the script ends with Succeed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
[]byte("v") inside the measured closure escapes through Write, so
AllocsPerRun counted the test's own conversion (3 per run, one per
context) rather than the connection's.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
0xdeafcafe added a commit to langwatch/langwatch that referenced this pull request Oct 12, 2026
…ity release candidate

Brings services/third_party/scenario-go to langwatch/scenario@c13d0cd4
(PR langwatch/scenario#1016, Go CI green). For the runner this adds:
non-blocking agent turns drained before a run finishes; a run that panics,
fails a test or is cancelled still finishes and sends RUN_FINISHED;
messages reported in index order; opt-in granular message and tool events;
LLM spans for the built-in judge and user simulator; tool-result error
flags; provider model strings; the judge digest cache and the sharded span
collector.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@langwatch-agent langwatch-agent added ci-green Latest run of every check is passing (checks API, not the legacy commit-status index) hound-checked Triaged by the pr-hound agent at the current head SHA labels Oct 12, 2026
LangWatch Work and others added 7 commits October 12, 2026 18:27
The executor dropped the first len(delivered) entries of the agent's
pending queue after the call. A rollback or ClearPendingMessages during
the call, followed by a broadcast, made it drop messages the agent never
saw. Each queued message now carries its broadcast sequence number and
the call drops only those broadcast up to the moment it read the queue.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
OnStart recorded every started span id per trace, released only by
ClearThread for traces tied to a thread, so traces of unrelated
instrumentation grew the collector without bound. A trace whose spans
all ended with none tied to a thread (and no fetcher claim) is now
listed as idle and the oldest past 4096 are forgotten; scenario traces
stay until ClearThread, so the remote fetcher still tells the process's
own spans apart.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…parses

Parser.str returned its one scratch buffer for every escaped string, so
a later escaped key or value overwrote the event name or streamSid Parse
held. Each of the two now unescapes into its own reused buffer; the
unescaped path is unchanged and escaped frames still do not allocate
once the buffers grew.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The test let a 5 ms polling goroutine race the drain's 100 ms tail
silence and the next user turn's reconcile, and failed under parallel
-race load. The burst's first chunk now reaches the drain ceiling on its
own, so the drain stops without a wall-clock wait, and a script step
holds the next user turn until the transport has queued the whole burst.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…s without a stop

mediaStreamLoop flushed the 100 ms inbound batch only on a stop frame,
so a socket that closed or failed without one lost up to 100 ms of the
caller's audio. The loop now flushes on every exit of a socket that is
the call's transport, before the end-of-call chunk.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
With SetReadLimit(-1) a peer's 63-bit frame length went straight into
slices.Grow and panicked the process. A message past MaxMessageSize
(1 GiB) now closes the connection with 1009 whatever the read limit, and
a large frame grows the buffer in 1 MiB steps as its bytes arrive, so
an announced length alone allocates nothing.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
postWithRetry always ran under context.Background(), so after a drain
timeout the bus goroutine kept posting and retrying the queued events
for minutes after Run returned. Delivery now runs under the bus's
context, cancelled when the drain ends: the post in flight and its
backoff stop, and the events still queued are dropped with one warning.
Each attempt is also bounded by its own timeout. During a normal run
nothing changes: no event is dropped and retries post the same payload,
keeping the original timestamp.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
LangWatch Work and others added 10 commits October 12, 2026 19:54
- go-live: pass the modules input to the script as arguments instead of
  through make, which pasted it into the recipe as shell code; read-only
  token and no persisted checkout credentials.
- go-ci: read toJSON(needs) from the environment, since the module list
  comes from directory names a pull request controls.
- validate-aggregator-workflows: check go-ci like the other CI workflows.
- CHANGELOG: the 0.1.0 heading linked a tag that was never pushed.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…checks

- Local playback queues audio to a writer goroutine instead of blocking the
  turn on ffmpeg, and Close is bounded.
- WebSocket pong and close echo get their own deadline; Close no longer
  waits forever behind a stuck write; the fallback transport lifts a
  deadline once its operation ends.
- ParseWAV rejects zero sample rate, channels or bit depth.
- A cancelled ffmpeg run reports the context error.
- Speech gate: a negative hangover or pre-roll turns it off.
- voicetest honours the receive timeout.
- apihttp caps the buffer it sizes from Content-Length.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ria once

- AgentNonBlocking and ScheduleAgentTurnNonBlocking pick the agent, open
  the turn and snapshot its input before returning, as in JS; only the
  agent call runs in the background, so a barge-in lands on the next turn.
- Only the call whose verdict set the result post-processes it, and the
  final merge runs once, so checkpoint criteria are no longer duplicated.
- Clearing one run's spans no longer forgets another run's spans in a
  shared trace.
- A judge's own span collector and trace fetcher are cleared after a run.
- Evaluators run under the cleanup context, so a cancelled run still
  evaluates.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…, URL redaction

- cloudflared: skip the api.trycloudflare.com host it logs on failure,
  and fail if the process exited.
- Twilio: restore only voice URLs that were actually read, and every
  callee dialled on the connection; an embedded connect that never gets
  its start frame tears down instead of staying connected.
- OpenAI Realtime: a failed session.update closes the socket; a second
  Connect closes the first.
- Composable: a reply joins the history only once it was spoken.
- WebSocket dial errors no longer print the signed URL's query string.
- ElevenLabs: the ceiling timeout message matches the receive logic.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- Gemini: keep thought signatures on replayed function calls (cached per
  call, with Google's documented placeholder as fallback) and on streamed
  text, so Gemini 3 tool loops are not rejected.
- Anthropic and Bedrock: no empty text blocks for empty tool results or
  system messages.
- OpenAI streaming: chunks whose id changes mid-stream are kept; tool-call
  index -1 is call_0 in events and in the final message alike.
- Bedrock: valid document names and explicit document and image formats;
  tool choice none leaves the tools out when the history allows it.
- Missing media types are filled in instead of sent empty; a user message
  with only unsupported parts is not sent as an empty array.
- Claude Code: on stop, the whole process group is killed after the grace
  period, including children that ignore SIGTERM.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A scripted user line needs an agent with the user role in the agents list,
on the Go and the JS SDK alike, so the cold start failed on Go ("no agent
with this role was found") and would have on JS. Both sides now list a user
simulator over the scripted model next to the echo agent; it is never asked
for a message, so the work stays the same.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…arity rows

AI SDK 6 prints a "system messages in messages" console.warn on every
generateText call of the JS user simulator and judge (20 lines per run,
not covered by AI_SDK_LOG_WARNINGS), which would add terminal I/O to the JS
side only. The harness now drops exactly that message, counts it and
reports the count per run.

The parity note and README now say what the span and event byte rows show:
Go 61 spans (run, turns, agent calls, one chat span per user simulator and
judge model call), JS AI SDK 81 (two AI SDK spans per such call), JS direct
41 (none); event bytes match exactly for Go and JS direct, and the AI SDK
column is a little longer because its answer text is a list of parts. The
README no longer claims that Go's model calls start no spans.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The micro section always said "median of 5 runs of about 1 s", also for a
QUICK run; it now uses the BENCH_COUNT and benchtime run.sh passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Full run at commit 7ea2329 on an M1 Max with FORCE=1: the 1-minute load
average was 7.63 at the start and 9.50 at the end (other work on the
machine), so the absolute numbers are noisy; the report carries the
warning. Messages (40) and events (42) per run match between Go and both
JS columns.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
0xdeafcafe added a commit to langwatch/langwatch that referenced this pull request Oct 13, 2026
…ueue dead workers, upgrade pacing, upgradelab) (#8580)

* fix(eventing): renew an outbox lease while its handler runs, so a crashed worker's intent is taken again within one short lease

The dispatcher renews each delivery's lease every third of its length, fenced
on the lease token; a refused renewal means another dispatcher took the
message, and the handler's outcome is then dropped and reported as fenced.
The default lease is now 2 minutes. The system-migrations pass drops its
hour-long lease for the default, keeping its hourly re-drive. Both process
stores gain renewLease.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(eventing): the stuck-drain alarm judges a drain by time since its last progress, so a drain still renewing its lease is never taken for stuck

The dispatcher records its last lease, settled delivery or successful renewal;
the outbox worker's watchdog re-arms while that is within the threshold and
abandons the drain only after a full threshold with no progress.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(system-migrations): the finished-fleet fixture carries its report

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test: fixtures follow the backfill limit, the pass's queue reader, the project page read and step definitions in after

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(group-queue): a job claimed by a worker that dies is run again instead of lost

A claim now keeps the job's body, and its drained batch siblings with their
original scores, in the group's claimed record under a per-claim token, rather
than deleting them. Completion, retry, hand-back, blocking and dead-lettering
delete the record. A later claim that finds the active key gone and the record
still there puts the jobs back at the front of the group in order; the poison
guard counts the death on that claim. Every outcome script compares the token,
so a slow former owner whose claim was recovered changes nothing and never
double-completes. Keys are new and scripts content-addressed, so previous-release
workers keep their exact behaviour. Spec: packages/group-queue/specs/claimed-job-recovery.feature.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(group-queue): drain, dead-letter and replay know a dead worker's claimed record

An operator drain deletes the group's claimed record with its staged jobs; a
move to the dead-letter queue carries a dead worker's claimed jobs into it in
order (a live worker's claim is dropped with its active key, as before); a
dead-lettered job leaves its claim's record so a death before completion cannot
bring it back. A projection replay treats a group holding a claimed record as
in flight. Spec: packages/group-queue/specs/claimed-job-recovery.feature.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(upgradelab): kill-mid-step waits 25 minutes after the step so a dead worker's claimed jobs can come back

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgradelab): -clickhouse-base alone keeps its server; only the URLs no flag named come from haven

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(adr-034): a re-run of a dead worker's job may double-count a rollup bucket; accepted as noise

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgradelab): UD-1 waits for head's worker to boot before the kill window opens and prints its progress

On a loaded host the worker took ten minutes to write its first line, so the drill sat silent inside
the 25-minute kill window. It now waits up to 30 minutes for the boot and prints each phase.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgradelab): the browser walk accepts the boot screen's health probe cancelled by the walk's next page

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgradelab): the SSO cookie check waits for the boot screen to leave before it judges the page

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgrade,group-queue): every background step and the identity backfill pace by their own outstanding jobs, so the queue stays bounded at any size; a listed run's retried group keeps its place

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(trace): the trace-limit check logs a busy or unreachable ClickHouse as a warning

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(haven,ui): haven up --hmr runs the plain Vite dev server; Vite's bundled dev mode and its refresh plugin are gone

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* ci(publish-docker-private): ref- tags only, branch or tag of this repository only, build commit stamped

Review fixes: a custom tag could replace a release version tag; a raw sha or
refs/pull/* could build a commit that lives only in a fork; the image carried no
build commit for the Upgrades page; a failed cache write could fail the run.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(scenario): regenerate the module READMEs after the child channel move

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(scenario): the span metrics lane reads trace's summary at the span's instant

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ci,upgradelab): publish-docker-private keeps an explicit refs/heads/ or refs/tags/ prefix; the SSO cookie check fails when the boot screen never leaves

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ci): deployment-impact heading prints literally, boxd refresh uses the workspace root, e2e-ci uses setup-pnpm-node

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(docker,ciguard): pin Go 1.27.2 in the image builds to match go.work; exempt vendored scenario-go from the go version guard

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ci): app-ci upgrades the database with BUILD_TIME unset, drops the test-unit steps package-suites duplicates, and lets every lint check report

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(group-queue): a former owner that dead-letters or parks a recovered job leaves the group's failures, attempt count and claim marker to the recovering worker

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(system-migrations): a paced pass fails on a tenant read or claim error and leaves only the listed queue's own refusals to the next pass

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* ci(go): derive build, test, vet and lint packages from go.work so every Go package runs

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ci): repoint the gitleaks allowlist to the restructured paths and drop entries whose files or values are gone

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(authz): admission writes already scoped by organization go through the tenant guard; only the platform-wide standing read skips it

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(authz): the access listing carries cappedBySeat, and the batch reads only the permitted projects' kind for the aggregate gate

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(api): the live cleanup asks for organization rows only when it seeded some, so a failed boot reports its own cause instead of a tenancy-guard refusal

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* style: format the twelve files the lint job's oxfmt check flagged

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(ops): regenerate the ops module READMEs

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* chore(openapi): regenerate the SDK clients and docs reference from the current routes

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(spec-binding): bind the shutdown, dynamic-import and capacity-tier scenarios by their spec names

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(self-hosting): say what the upgrade does not do instead of a bare negative

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(governance): the cost-attribution repository's client type declares the batch read, so the batched config read typechecks

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(organization): a failed audit-log row uses the list table's danger tone instead of style props the row does not accept

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(governance): the backend test's policy repository implements the batched read, and the billing-fact page fake handles an absent limit

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(seedgen): the organization fixture answers grantsConfirmed, as the contract now requires

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(role): GET /api/v1/roles with no builtIn filter lists custom roles only, as in 3.20.1; the bare /api/roles twin answers the same

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* build(scenariorunner): update the vendored Go scenario SDK to the parity release candidate

Brings services/third_party/scenario-go to langwatch/scenario@c13d0cd4
(PR langwatch/scenario#1016, Go CI green). For the runner this adds:
non-blocking agent turns drained before a run finishes; a run that panics,
fails a test or is cancelled still finishes and sends RUN_FINISHED;
messages reported in index order; opt-in granular message and tool events;
LLM spans for the built-in judge and user simulator; tool-result error
flags; provider model strings; the judge digest cache and the sharded span
collector.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* chore(tooling): delete visualdiff, diffsuite, lane-diff and the fuzz and interaction tools

apidiff stays, with diffkit and openapidiff it imports. Its deferred-scenario
spec moves to specs/tooling/apidiff-self-hosted-pass.feature.

* fix(eventing): an outbox handler silent for five minutes loses its lease

The dispatcher renews a delivery's lease only while the handler shows progress
through heartbeat(). Past outbox.maxSilenceMs (default 5 minutes, never below
the process manager's lease) renewal stops, the lease lapses and the message is
redelivered. The system migrations pass sets one hour.

* docs(plans): spin-up lessons cite local session records as plain text, not machine paths

* feat(api): projects, user-avatar, langy and elevenlabs answer at /api/v1, the bare path kept as a twin

Callers in the SDKs, MCP server and docs move to /api/v1. OpenAPI clients and
docs regenerate, including the custom-roles-only listing.

* fix(apidiff): wait for the seeded project, run main's worker and flush Redis between runs

The fixtures waited only for the api, so they ran before haven's seed and got
401. Main's monolith lane needs WORKERS_IN_PROCESS=1. Destroyed stacks left
their Redis databases, so a rerun dropped the fixture trace as a duplicate.

* ci(gitleaks): allowlist the invented test values and digests the architecture merge brought in

* fix(instant-eval): a query identity ClickHouse refuses answers instant_eval_not_enabled, not 500

* fix(group-queue): a worker that lost its claim can no longer dead-letter or re-stage the recovering worker's jobs

The dead-letter and stage scripts check the claim token before any write, the
same check the drain uses.

* test(role): both role listings assert the full response body

* fix(docker): Go images copy the vendored scenario SDK before downloading modules

* fix(docker): the app image runs the prepare scripts per package, without nx, which the image does not carry

* ci(migration-compat): run main's live api suites in one worker so their upgrade passes don't queue past the setup timeout

* ci(go): the large-plan memory test runs without the race detector, in its own step

* fix(go): clear the golangci-lint findings in the sims, services and tools, and resync the feature-parity lists

* fix(apidiff): clear the golangci-lint findings

* refactor(apidiff): split the run-flag parsing and boot planning into helpers under the lint limits

* fix(group-queue): a worker that lost its claim keeps the payload lease the recovering worker relies on

The failure paths complete first and release the lease only when the claim was
still theirs, the same order as the success path.

* fix(group-queue): a lost claim on the block path reports no dead payloads, as the dead-letter path already does

* test(e2e): agent testing, automations, navigation, guardrail and docs-link specs assert the branch's deliberate layout

* fix(ci): clear the upgradelab lint findings and give gateway-matrix's upgrade its ClickHouse

* fix(haven): the monolith lane hosts its workers, destroy waits for the launcher before flushing, stacks may create LangWatchQL functions

apidiff drops its monolith-worker workaround.

* fix(eventing): the outbox silence limit is a flat five minutes; long-running process managers send heartbeats

Topic clustering keeps 15 minutes (one langevals call can run 12) and the
system migrations pass keeps an hour (one tenant's migration reports no
progress).

* test: bind the unbound scenarios, retarget stale annotations, and fix two stale unit tests

* feat(auth): only the session sign-up started skips the secure-account offer

Auth marks an account's first session (new Session.startedBySignUp column) and
lists it on the browser-session entry; the user module reads it instead of a
60-second window, so a sign-in seconds after sign-up gets the offer.

* fix(upgradelab): the SSO cookie check records the page's console, errors, failed requests and DOM, so a blank page names its cause

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(system-migrations): every tenant migration on the worker is paced on its own list by default (5,000 of its own jobs), so an undeclared one can't flood the queue

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgradelab): the SSO cookie check waits for the page to draw, not only for the boot screen to leave

The final build's boot screen leaves when the theme's style lands in the root, a moment
before the page; the check judged that gap as a blank page.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ui): the boot screen stays until the app draws a real element, so the theme's style and script no longer leave a blank gap

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(system-migrations,authz): a tenant migration can wait for queue room inside one organisation; the authz grant import sends a big organisation's grants 500 at a time

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(group-queue): a listed run forgets each settled group from its set, baseline, candidates and live keys, so its Redis memory stays at its outstanding groups

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* perf(user): one upgrade step records every account's created event and re-states its standing, a page of 2,000 at a time with one batch send

The two steps each sent one command per account and waited for it, so 200k accounts took over 45 minutes.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* perf(eventing,user): a pipeline reads many of its own aggregates in one ClickHouse query; the user upgrade step reads a page's standing history in one read

Each page of 2,000 accounts made 2,000 reads queued behind the worker's ClickHouse cap, about 2 to 4 minutes a page at 200k users.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(eventing): the many-aggregate read brackets its tenant and aggregate lists, the form the ClickHouse tenant guard recognises

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(browser): online evaluations, experiments and datasets empty states offer set-up-with-AI again, as on main

* refactor(eventing): a delivery's progress is any call through its module's dependencies, not explicit heartbeats

The worker wraps each module's store clients, repositories, channels and peer
Apis so a call inside an outbox delivery marks that delivery's progress. The
explicit heartbeat calls threaded through 18 process managers are removed.

* fix: regenerate the LWQL Prisma manifest and query fixture, register gq_jobs_recovered_total for hot reload, and update four stale unit tests

* fix(eventing): lock and claim renewal timers start outside the delivery, so a hung replay or migration pass still loses its lease

The system migrations pass drops its one-hour limit: every tenant step works
through tracked calls.

* docs: regenerate the user module and packages READMEs

* ci(gateway-matrix): give the upgrade its Redis as well as its ClickHouse

* fix(apidiff): re-sign the branch licence for old main without organizationId, and mask time-derived virtual-key prefixes

* fix(ci): gateway-matrix sets the api-key pepper and makes its credentials key before the upgrade runs

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(go): CI go lint and tests pass again

- seedgen keeps its Bsize conversion (int64 on Linux, uint32 on darwin) with an explained nolint
- featureparity's inert list matches the Node tool (id-index-reads is planned, not built)
- haven's process supervisor sums a group's RSS from ps -axo pgid=,rss=; Linux ps -g selects by session, so the memory ceiling never fired there

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* chore(langy): regenerate the feature map for the /api/v1/projects path

* test(api): the npx entry-point test reads langwatch.ts, which now hosts the worker half

* fix(make): the service target runs under bash, so its pipefail works on CI runners whose sh is dash

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(architecture-enforcer): the shared Prisma table list, the lint rule records and the coordinator protocol test match today's data

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(architecture-enforcer): annotations sit on their tests, and the ClickHouse, api-package, typecheck-script and contract-edge lists match today's tree

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix: browser-lane packages declare playwright, analytics links go through the host Link, and the enforcer tests match the dev launcher and the one rule that accepts a disable

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(deleted-spellings): the feature-flag module keeps its own use-feature-flag file, and noMembers matches only the deleted helper's import or call

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ci): gateway-matrix names its public base URL, which the hosted MCP endpoint needs to boot

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(apidiff): run stacks start the analytics stand-in, and analytics scenarios are deferred where no stand-in answers

* fix(docker): hoist @opentelemetry/api so the image's SDK working copy resolves its peer

Without it every source-run process in the app image that imports the SDK
fails with ERR_MODULE_NOT_FOUND; the chart's LWQL access job was the first.

* fix(group-queue): a listed run counts as stopped only after its work stands still for 5 minutes, not 60 seconds, so a busy worker no longer fails healthy background steps

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgradelab): the Ops > Upgrades check waits for the boot screen to leave before it signs in, as the SSO check does

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(architecture): one kind of event; an upgrade step's events reach every consumer and the queue carries the volume

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(group-queue): a listed run's list stays open while it registers work, open lists take turns, and the listed consumer claims again as each of its jobs finishes

Its follow-on jobs then run ahead of a backlog for as long as the run lasts: the list lapsed an hour after opening, the first open list took every slot, and a full listed consumer slept up to 5 s.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* chore(claude): coordinator, manifest and handoff folders keep their READMEs and templates; dated working files stay local and are ignored

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(skills): frontend, design-system and tooling skills teach today's component APIs, haven commands and sims; one way to drive a browser

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(skills): backend, eventing and upgrade skills teach today's shapes and the 2026-10-12 rulings (one kind of event, one pass per table, reruns through the step, own-queue pacing)

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgrade,system-migrations): background steps, paced passes and the pass request wait 5 minutes for their queue work to move, matching the group queue

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(skills): new skills for authorization and tenancy, platform primitives, system migrations, upgradelab and browser telemetry; the tenant-id lint hint shows the bracketed tenant set the guard accepts

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(api-key): a retried run-key mint waits on the key its first attempt wrote instead of minting another, so it converges once the grants projection catches up

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(project,organization,annotation): one project facts step walks projects once with batch sends; organisation presence sends per page; the five duplicate backfill tasks are gone (reruns go through the step in Ops)

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor: overlays, cards and copy buttons move to the design system's callable shapes; module code uses the house logger, nowInstant, ksuid and a contract error

Design system: studio-* imports, Drawer.Root and Dialog.Root move to Drawer and Dialog where the size and padding scales match; Card.Root and SettingsCard move to Card; the trace dialog wrapper and two local copy controls are gone.
Helpers: browser console calls use createLogger, repository clocks use nowInstant, idempotency keys use ksuid, and the scenario mapping refusal is ScenarioRunRejectedError.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(upgradelab): the Ops > Upgrades check's screenshot waits up to a minute and never fails the check

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(trace): the onboarding journey config imports the browser logger it uses

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test: module tests live in colocated __tests__, and 24 installation tests boot through bootInstalledProcess with testPeer

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs: the record names definePipeline's real package and drops two landed rows from §16; CLAUDE.md says the root scripts run through Nx

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* docs(architecture): a request waiting on its own projection waits at most 10 seconds, then answers pending; no queue jumping, no inline folds

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(agent): connected-agent state goes through ConnectedAgentStateRepository (redis and memory twins) and relay messages through ConnectedAgentRelayChannel; services no longer touch the Redis session store

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* perf(group-queue): a listed run registers each page's groups in two calls instead of two per job, listed jobs coalesce like any other, and a throughput bench measures listed runs

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* test(design-system): the dialog backdrop scenarios bind to the design system's Dialog

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(langy): local control goes through call, user-wait and control-request repositories (redis and memory twins) and a workspace channel; the skip policy's publish moves out of rules into a service

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* feat(secrets,redis-client,api-key): mintOnce shares one sealed credential across servers; run keys and agent sandbox keys use it, so a run gets exactly one key

The pending-key table is gone. Also lands the authz engine and identity sign-in switch tables and authz's new dependencies, ahead of their code.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* perf(clickhouse-client,eventing,group-queue): an insert may declare a tenant set, event appends waiting on one organisation's server fold into shared inserts (each still flushed, a failed one retried alone), and queue slots default to 1,000

Bench on the real append path: 1,706 to 22,019 users/min, 54,000 to 270 ClickHouse inserts for 6,000 users.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* feat(authz,workflow): the grant read-your-writes wait is 10 s everywhere; the run-key mint's pending answer is retryable, and the studio shows waiting and resends instead of failing

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* refactor(authz,identity,ops): the engine and sign-in switches are module-owned state, rolled back by an Ops event; the five legacy tenant migrations are declared tenant steps, Ops copies their old state twice and its Migrations page reads the framework state

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ops,authz): during the rollout Ops writes moved tenant-migration state to both tables, the re-copy applies withdrawals made on the older page, and the authz tests expect the tenant guard fragment, the seat cap and the lineage reads

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
@langwatch-agent

Copy link
Copy Markdown
Contributor

Human Review Brief

Caution

This is a whole Go SDK, 82,000 lines across 563 files, with new CI, live-test and publish workflows. One review cannot cover it.

It brings the Go scenario SDK to parity with JavaScript: two-phase judging with per-criterion verdicts, the event format, tracing, a voice module with six adapters and two tunnels, and four model providers. It supersedes the closed #80.

The parts that bite after merge are the ones that publish and the ones that talk to LangWatch: the release-please config, go-publish.yml, and the event sender. The low-risk check fails on it because GitHub will not diff more than 300 files; #1018 fixes that check.

Mode Deep Review. An 82,000-line, 563-file new SDK with its own release and publish workflows.
Issue No linked issue.
State CI green on the latest run of every check. 0 unresolved threads. MERGEABLE, merge state BLOCKED. Reviewer: drewdrewthis.
Evidence Benchmarks against the JS SDK and new CI and live-test workflows. See the PR body for test counts.
Where to look Review in slices: .github/workflows/go-*.yml and the release-please changes first, then the event sender against the backend contract, then providers and voice.
🗺️ Never opened this module? Start here · ~3 min

scenario ships SDKs in JavaScript and Python, and now Go. Each SDK sends run events to LangWatch in one shared format, so a Go run shows in the same UI as a JS one.

⚖️ What approving this makes permanent · ~1 min
  1. Go is a first-class SDK, released and published from this repo.

  2. Voice and ngrok are separate Go modules.

❓ Worth asking the author · ~1 min
  1. Can the review be split by module, with a named reviewer per slice?

  2. Who owns the Go SDK after merge?

✅ Why the evidence is good, and what each file is · ~1 min

Benchmarks against the JS SDK and new CI and live-test workflows. See the PR body for test counts.

File + / - What it is
.github/workflows/go-ci.yml 193 / 0 Substance
.github/workflows/go-live.yml 147 / 0 Substance
.github/workflows/go-publish.yml 142 / 0 Substance
.release-please-config.json 123 / 1 Substance
.release-please-manifest.json 8 / 1 Substance
Makefile 143 / 1 Substance
benchmarks/go-vs-js/.gitignore 2 / 0 Substance
benchmarks/go-vs-js/go/cmd/coldstart/main.go 26 / 0 Substance
benchmarks/go-vs-js/go/cmd/scenario/main.go 177 / 0 Substance
benchmarks/go-vs-js/go/fake/agent.go 70 / 0 Substance
benchmarks/go-vs-js/go/fake/llm.go 42 / 0 Substance
benchmarks/go-vs-js/go/fake/run.go 84 / 0 Substance
benchmarks/go-vs-js/go/fake/script.go 64 / 0 Substance
benchmarks/go-vs-js/go/fake/sink.go 46 / 0 Substance
benchmarks/go-vs-js/go/go.mod 38 / 0 Substance
benchmarks/go-vs-js/go/go.sum 63 / 0 Substance
benchmarks/go-vs-js/go/stats/stats.go 78 / 0 Substance
benchmarks/go-vs-js/js/build-internals.mjs 69 / 0 Substance
benchmarks/go-vs-js/js/coldstart-driver.mjs 50 / 0 Substance
benchmarks/go-vs-js/js/coldstart.cjs 75 / 0 Substance
benchmarks/go-vs-js/js/coldstart.mjs 13 / 0 Substance
benchmarks/go-vs-js/js/lib/bench.mjs 50 / 0 Substance
benchmarks/go-vs-js/js/lib/fake-model.mjs 208 / 0 Substance
benchmarks/go-vs-js/js/lib/paths.mjs 86 / 0 Substance
benchmarks/go-vs-js/js/lib/setup.mjs 109 / 0 Substance
benchmarks/go-vs-js/js/lib/sink.mjs 32 / 0 Substance
benchmarks/go-vs-js/js/lib/stats.mjs 86 / 0 Substance
benchmarks/go-vs-js/js/micro.mjs 112 / 0 Substance
benchmarks/go-vs-js/js/report.mjs 253 / 0 Substance
benchmarks/go-vs-js/js/scenario.mjs 115 / 0 Substance
benchmarks/go-vs-js/js/voice.mjs 61 / 0 Substance
benchmarks/go-vs-js/run.sh 184 / 0 Substance
go/agent.go 170 / 0 Substance
go/agents/claudecode/adapter.go 288 / 0 Substance
go/agents/claudecode/doc.go 48 / 0 Substance
go/agents/claudecode/errors.go 119 / 0 Substance
go/agents/claudecode/lines.go 92 / 0 Substance
go/agents/claudecode/process_other.go 26 / 0 Substance
go/agents/claudecode/process_unix.go 136 / 0 Substance
go/agents/claudecode/prompt.go 184 / 0 Substance
go/agents/claudecode/run.go 179 / 0 Substance
go/agents/claudecode/skills.go 152 / 0 Substance
go/agents/claudecode/streamjson.go 275 / 0 Substance
go/agents/claudecode/transcript.go 429 / 0 Substance
go/agui.go 214 / 0 Substance
go/config.go 159 / 0 Substance
go/configure.go 166 / 0 Substance
go/connected_agent.go 94 / 0 Substance
go/emitter.go 238 / 0 Substance
go/evaluations.go 63 / 0 Substance
RELEASE_PLEASE.md 145 / 1 Docs
benchmarks/go-vs-js/README.md 242 / 0 Docs
benchmarks/go-vs-js/results.md 93 / 0 Docs
docs/docs/pages/advanced/custom-observability.mdx 82 / 7 Docs
docs/docs/pages/advanced/evaluators.mdx 89 / 8 Docs
docs/docs/pages/advanced/red-teaming.mdx 351 / 34 Docs
docs/docs/pages/advanced/red-teaming/goat.mdx 85 / 10 Docs
docs/docs/pages/advanced/red-teaming/quick-start.mdx 76 / 2 Docs
docs/docs/pages/advanced/red-teaming/report.mdx 33 / 0 Docs
docs/docs/pages/advanced/remote-traces.mdx 223 / 4 Docs
docs/docs/pages/agent-integration/claude-code.mdx 152 / 18 Docs
docs/docs/pages/basics/configuration.mdx 81 / 17 Docs
docs/docs/pages/basics/judge-agent.mdx 113 / 3 Docs
docs/docs/pages/basics/scripted-simulations.mdx 546 / 16 Docs
docs/docs/pages/basics/test-runner-integration.mdx 52 / 1 Docs
docs/docs/pages/basics/user-simulator.mdx 80 / 0 Docs
docs/docs/pages/basics/writing-scenarios.mdx 243 / 1 Docs
docs/docs/pages/introduction/getting-started.mdx 116 / 1 Docs
docs/docs/pages/testing-guides/fixtures.mdx 59 / 1 Docs
docs/docs/pages/voice/adapters/composable.mdx 27 / 3 Docs
docs/docs/pages/voice/adapters/elevenlabs.mdx 55 / 12 Docs
docs/docs/pages/voice/adapters/gemini-live.mdx 17 / 4 Docs
docs/docs/pages/voice/adapters/openai-realtime.mdx 19 / 3 Docs
docs/docs/pages/voice/adapters/pipecat.mdx 22 / 2 Docs
docs/docs/pages/voice/adapters/twilio.mdx 59 / 1 Docs
docs/docs/pages/voice/getting-started.mdx 107 / 2 Docs
docs/docs/pages/voice/happy-path-elevenlabs.mdx 111 / 3 Docs
docs/docs/pages/voice/recipes/effects.mdx 46 / 16 Docs
docs/docs/pages/voice/recipes/interrupt.mdx 51 / 4 Docs
docs/docs/pages/voice/recipes/multi-turn.mdx 23 / 0 Docs
docs/docs/pages/voice/recipes/observability.mdx 66 / 13 Docs
go/CHANGELOG.md 13 / 0 Docs
go/README.md 640 / 0 Docs
benchmarks/go-vs-js/go/fake/script_test.go 73 / 0 Tests
benchmarks/go-vs-js/go/stats/stats_test.go 20 / 0 Tests
benchmarks/go-vs-js/go/voicebench/voice_test.go 123 / 0 Tests
benchmarks/go-vs-js/inpkg/zz_govsjs_bench_test.go 77 / 0 Tests
go/agents/claudecode/adapter_test.go 701 / 0 Tests
go/agents/claudecode/bench_test.go 88 / 0 Tests
go/agents/claudecode/e2e_test.go 42 / 0 Tests
go/agents/claudecode/lifecycle_test.go 304 / 0 Tests
go/agents/claudecode/lines_test.go 52 / 0 Tests
go/agents/claudecode/logs_test.go 72 / 0 Tests
go/agents/claudecode/main_test.go 339 / 0 Tests
go/agents/claudecode/prompt_test.go 76 / 0 Tests
go/agents/claudecode/skills_test.go 120 / 0 Tests
go/agents/claudecode/streamjson_test.go 335 / 0 Tests
go/agui_test.go 70 / 0 Tests
go/bench_test.go 219 / 0 Tests
go/configure_headless_test.go 13 / 0 Tests

Note

The event format must match what LangWatch reads, or Go runs will not show up.

@langwatch-agent langwatch-agent added review: deep PR Hound review mode hound-escalated Dispatched once to the code-review agent by pr-branch-hound labels Oct 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-green Latest run of every check is passing (checks API, not the legacy commit-status index) hound-checked Triaged by the pr-hound agent at the current head SHA hound-escalated Dispatched once to the code-review agent by pr-branch-hound review: deep PR Hound review mode

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants