Repository navigation
feat(go): bring the Go SDK to parity with the JavaScript SDK - #1016
0xdeafcafe wants to merge 176 commits into
Conversation
Extract LLM inference interface and message types into dedicated files. Update agent interfaces and user simulator with refined API.
- Prefix event types with SCENARIO_ (SCENARIO_RUN_STARTED, etc.)
- Add ScenarioRunStatus type with uppercase values (SUCCESS, ERROR, FAILED)
- Rewrite eventalert with tmpdir file coordination across processes
- Show greeting banner only when API key is missing
- Use path-based watch URL format ({setUrl}/{batchRunId})
- Add SCENARIO_HEADLESS env var to suppress browser open
- Add SCENARIO_DISABLE_SIMULATION_REPORT_INFO env var to suppress banners
- Scope watch message per scenarioSetId
- Cache batch run ID per process with sync.Once
Add go.opentelemetry.io/otel, otel/sdk, otel/trace, otel/attribute, otel/codes and github.com/langwatch/langwatch/sdk-go for full OTel tracing integration.
- SpanCollector: implements sdktrace.SpanProcessor to collect spans, filters by thread ID with parent chain walking - SpanDigestFormatter: renders spans as plain-text hierarchy with timestamps, durations, attributes, events, and error sections - setupObservability: creates LangWatch exporter, TracerProvider, and SpanCollector; wires into global OTel provider - Remove TracedInference and instrumentBuiltInAgents (replaced by OTel)
- Execution: create per-turn and per-agent spans with tracer, end spans at all exit paths - JudgeAgent: build transcript from messages, include OTel trace digest in judge prompt alongside conversation transcript - Runner: init observability when API key present, wire span collector to judge agents and execution, default endpoint to app.langwatch.ai, default SetID to "default", shutdown observability after run
Multi-provider inference abstraction supporting OpenAI, Anthropic, Google Gemini, and AWS Bedrock with tool/function calling conversion.
…passes The passing judge ended the run on the first turn's judge step, so the second turn, the one whose barge-in the test checks, never ran. The judge now lets the run continue and the script ends with Succeed. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
[]byte("v") inside the measured closure escapes through Write, so
AllocsPerRun counted the test's own conversion (3 per run, one per
context) rather than the connection's.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ity release candidate Brings services/third_party/scenario-go to langwatch/scenario@c13d0cd4 (PR langwatch/scenario#1016, Go CI green). For the runner this adds: non-blocking agent turns drained before a run finishes; a run that panics, fails a test or is cancelled still finishes and sends RUN_FINISHED; messages reported in index order; opt-in granular message and tool events; LLM spans for the built-in judge and user simulator; tool-result error flags; provider model strings; the judge digest cache and the sharded span collector. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The executor dropped the first len(delivered) entries of the agent's pending queue after the call. A rollback or ClearPendingMessages during the call, followed by a broadcast, made it drop messages the agent never saw. Each queued message now carries its broadcast sequence number and the call drops only those broadcast up to the moment it read the queue. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
OnStart recorded every started span id per trace, released only by ClearThread for traces tied to a thread, so traces of unrelated instrumentation grew the collector without bound. A trace whose spans all ended with none tied to a thread (and no fetcher claim) is now listed as idle and the oldest past 4096 are forgotten; scenario traces stay until ClearThread, so the remote fetcher still tells the process's own spans apart. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…parses Parser.str returned its one scratch buffer for every escaped string, so a later escaped key or value overwrote the event name or streamSid Parse held. Each of the two now unescapes into its own reused buffer; the unescaped path is unchanged and escaped frames still do not allocate once the buffers grew. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The test let a 5 ms polling goroutine race the drain's 100 ms tail silence and the next user turn's reconcile, and failed under parallel -race load. The burst's first chunk now reaches the drain ceiling on its own, so the drain stops without a wall-clock wait, and a script step holds the next user turn until the transport has queued the whole burst. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…s without a stop mediaStreamLoop flushed the 100 ms inbound batch only on a stop frame, so a socket that closed or failed without one lost up to 100 ms of the caller's audio. The loop now flushes on every exit of a socket that is the call's transport, before the end-of-call chunk. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
With SetReadLimit(-1) a peer's 63-bit frame length went straight into slices.Grow and panicked the process. A message past MaxMessageSize (1 GiB) now closes the connection with 1009 whatever the read limit, and a large frame grows the buffer in 1 MiB steps as its bytes arrive, so an announced length alone allocates nothing. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
postWithRetry always ran under context.Background(), so after a drain timeout the bus goroutine kept posting and retrying the queued events for minutes after Run returned. Delivery now runs under the bus's context, cancelled when the drain ends: the post in flight and its backoff stop, and the events still queued are dropped with one warning. Each attempt is also bounded by its own timeout. During a normal run nothing changes: no event is dropped and retries post the same payload, keeping the original timestamp. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- go-live: pass the modules input to the script as arguments instead of through make, which pasted it into the recipe as shell code; read-only token and no persisted checkout credentials. - go-ci: read toJSON(needs) from the environment, since the module list comes from directory names a pull request controls. - validate-aggregator-workflows: check go-ci like the other CI workflows. - CHANGELOG: the 0.1.0 heading linked a tag that was never pushed. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…checks - Local playback queues audio to a writer goroutine instead of blocking the turn on ffmpeg, and Close is bounded. - WebSocket pong and close echo get their own deadline; Close no longer waits forever behind a stuck write; the fallback transport lifts a deadline once its operation ends. - ParseWAV rejects zero sample rate, channels or bit depth. - A cancelled ffmpeg run reports the context error. - Speech gate: a negative hangover or pre-roll turns it off. - voicetest honours the receive timeout. - apihttp caps the buffer it sizes from Content-Length. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ria once - AgentNonBlocking and ScheduleAgentTurnNonBlocking pick the agent, open the turn and snapshot its input before returning, as in JS; only the agent call runs in the background, so a barge-in lands on the next turn. - Only the call whose verdict set the result post-processes it, and the final merge runs once, so checkpoint criteria are no longer duplicated. - Clearing one run's spans no longer forgets another run's spans in a shared trace. - A judge's own span collector and trace fetcher are cleared after a run. - Evaluators run under the cleanup context, so a cancelled run still evaluates. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…, URL redaction - cloudflared: skip the api.trycloudflare.com host it logs on failure, and fail if the process exited. - Twilio: restore only voice URLs that were actually read, and every callee dialled on the connection; an embedded connect that never gets its start frame tears down instead of staying connected. - OpenAI Realtime: a failed session.update closes the socket; a second Connect closes the first. - Composable: a reply joins the history only once it was spoken. - WebSocket dial errors no longer print the signed URL's query string. - ElevenLabs: the ceiling timeout message matches the receive logic. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
… too Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
- Gemini: keep thought signatures on replayed function calls (cached per call, with Google's documented placeholder as fallback) and on streamed text, so Gemini 3 tool loops are not rejected. - Anthropic and Bedrock: no empty text blocks for empty tool results or system messages. - OpenAI streaming: chunks whose id changes mid-stream are kept; tool-call index -1 is call_0 in events and in the final message alike. - Bedrock: valid document names and explicit document and image formats; tool choice none leaves the tools out when the history allows it. - Missing media types are filled in instead of sent empty; a user message with only unsupported parts is not sent as an empty array. - Claude Code: on stop, the whole process group is killed after the grace period, including children that ignore SIGTERM. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
A scripted user line needs an agent with the user role in the agents list,
on the Go and the JS SDK alike, so the cold start failed on Go ("no agent
with this role was found") and would have on JS. Both sides now list a user
simulator over the scripted model next to the echo agent; it is never asked
for a message, so the work stays the same.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…arity rows AI SDK 6 prints a "system messages in messages" console.warn on every generateText call of the JS user simulator and judge (20 lines per run, not covered by AI_SDK_LOG_WARNINGS), which would add terminal I/O to the JS side only. The harness now drops exactly that message, counts it and reports the count per run. The parity note and README now say what the span and event byte rows show: Go 61 spans (run, turns, agent calls, one chat span per user simulator and judge model call), JS AI SDK 81 (two AI SDK spans per such call), JS direct 41 (none); event bytes match exactly for Go and JS direct, and the AI SDK column is a little longer because its answer text is a list of parts. The README no longer claims that Go's model calls start no spans. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
The micro section always said "median of 5 runs of about 1 s", also for a QUICK run; it now uses the BENCH_COUNT and benchtime run.sh passes. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Full run at commit 7ea2329 on an M1 Max with FORCE=1: the 1-minute load average was 7.63 at the start and 9.50 at the end (other work on the machine), so the absolute numbers are noisy; the report carries the warning. Messages (40) and events (42) per run match between Go and both JS columns. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
…ueue dead workers, upgrade pacing, upgradelab) (#8580) * fix(eventing): renew an outbox lease while its handler runs, so a crashed worker's intent is taken again within one short lease The dispatcher renews each delivery's lease every third of its length, fenced on the lease token; a refused renewal means another dispatcher took the message, and the handler's outcome is then dropped and reported as fenced. The default lease is now 2 minutes. The system-migrations pass drops its hour-long lease for the default, keeping its hourly re-drive. Both process stores gain renewLease. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(eventing): the stuck-drain alarm judges a drain by time since its last progress, so a drain still renewing its lease is never taken for stuck The dispatcher records its last lease, settled delivery or successful renewal; the outbox worker's watchdog re-arms while that is within the threshold and abandons the drain only after a full threshold with no progress. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(system-migrations): the finished-fleet fixture carries its report Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test: fixtures follow the backfill limit, the pass's queue reader, the project page read and step definitions in after Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(group-queue): a job claimed by a worker that dies is run again instead of lost A claim now keeps the job's body, and its drained batch siblings with their original scores, in the group's claimed record under a per-claim token, rather than deleting them. Completion, retry, hand-back, blocking and dead-lettering delete the record. A later claim that finds the active key gone and the record still there puts the jobs back at the front of the group in order; the poison guard counts the death on that claim. Every outcome script compares the token, so a slow former owner whose claim was recovered changes nothing and never double-completes. Keys are new and scripts content-addressed, so previous-release workers keep their exact behaviour. Spec: packages/group-queue/specs/claimed-job-recovery.feature. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(group-queue): drain, dead-letter and replay know a dead worker's claimed record An operator drain deletes the group's claimed record with its staged jobs; a move to the dead-letter queue carries a dead worker's claimed jobs into it in order (a live worker's claim is dropped with its active key, as before); a dead-lettered job leaves its claim's record so a death before completion cannot bring it back. A projection replay treats a group holding a claimed record as in flight. Spec: packages/group-queue/specs/claimed-job-recovery.feature. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(upgradelab): kill-mid-step waits 25 minutes after the step so a dead worker's claimed jobs can come back Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgradelab): -clickhouse-base alone keeps its server; only the URLs no flag named come from haven Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(adr-034): a re-run of a dead worker's job may double-count a rollup bucket; accepted as noise Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgradelab): UD-1 waits for head's worker to boot before the kill window opens and prints its progress On a loaded host the worker took ten minutes to write its first line, so the drill sat silent inside the 25-minute kill window. It now waits up to 30 minutes for the boot and prints each phase. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgradelab): the browser walk accepts the boot screen's health probe cancelled by the walk's next page Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgradelab): the SSO cookie check waits for the boot screen to leave before it judges the page Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgrade,group-queue): every background step and the identity backfill pace by their own outstanding jobs, so the queue stays bounded at any size; a listed run's retried group keeps its place Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(trace): the trace-limit check logs a busy or unreachable ClickHouse as a warning Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(haven,ui): haven up --hmr runs the plain Vite dev server; Vite's bundled dev mode and its refresh plugin are gone Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * ci(publish-docker-private): ref- tags only, branch or tag of this repository only, build commit stamped Review fixes: a custom tag could replace a release version tag; a raw sha or refs/pull/* could build a commit that lives only in a fork; the image carried no build commit for the Upgrades page; a failed cache write could fail the run. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(scenario): regenerate the module READMEs after the child channel move Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(scenario): the span metrics lane reads trace's summary at the span's instant Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(ci,upgradelab): publish-docker-private keeps an explicit refs/heads/ or refs/tags/ prefix; the SSO cookie check fails when the boot screen never leaves Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(ci): deployment-impact heading prints literally, boxd refresh uses the workspace root, e2e-ci uses setup-pnpm-node Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(docker,ciguard): pin Go 1.27.2 in the image builds to match go.work; exempt vendored scenario-go from the go version guard Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(ci): app-ci upgrades the database with BUILD_TIME unset, drops the test-unit steps package-suites duplicates, and lets every lint check report Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(group-queue): a former owner that dead-letters or parks a recovered job leaves the group's failures, attempt count and claim marker to the recovering worker Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(system-migrations): a paced pass fails on a tenant read or claim error and leaves only the listed queue's own refusals to the next pass Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * ci(go): derive build, test, vet and lint packages from go.work so every Go package runs Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(ci): repoint the gitleaks allowlist to the restructured paths and drop entries whose files or values are gone Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(authz): admission writes already scoped by organization go through the tenant guard; only the platform-wide standing read skips it Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(authz): the access listing carries cappedBySeat, and the batch reads only the permitted projects' kind for the aggregate gate Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(api): the live cleanup asks for organization rows only when it seeded some, so a failed boot reports its own cause instead of a tenancy-guard refusal Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * style: format the twelve files the lint job's oxfmt check flagged Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(ops): regenerate the ops module READMEs Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * chore(openapi): regenerate the SDK clients and docs reference from the current routes Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(spec-binding): bind the shutdown, dynamic-import and capacity-tier scenarios by their spec names Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(self-hosting): say what the upgrade does not do instead of a bare negative Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(governance): the cost-attribution repository's client type declares the batch read, so the batched config read typechecks Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(organization): a failed audit-log row uses the list table's danger tone instead of style props the row does not accept Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(governance): the backend test's policy repository implements the batched read, and the billing-fact page fake handles an absent limit Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(seedgen): the organization fixture answers grantsConfirmed, as the contract now requires Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(role): GET /api/v1/roles with no builtIn filter lists custom roles only, as in 3.20.1; the bare /api/roles twin answers the same Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * build(scenariorunner): update the vendored Go scenario SDK to the parity release candidate Brings services/third_party/scenario-go to langwatch/scenario@c13d0cd4 (PR langwatch/scenario#1016, Go CI green). For the runner this adds: non-blocking agent turns drained before a run finishes; a run that panics, fails a test or is cancelled still finishes and sends RUN_FINISHED; messages reported in index order; opt-in granular message and tool events; LLM spans for the built-in judge and user simulator; tool-result error flags; provider model strings; the judge digest cache and the sharded span collector. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * chore(tooling): delete visualdiff, diffsuite, lane-diff and the fuzz and interaction tools apidiff stays, with diffkit and openapidiff it imports. Its deferred-scenario spec moves to specs/tooling/apidiff-self-hosted-pass.feature. * fix(eventing): an outbox handler silent for five minutes loses its lease The dispatcher renews a delivery's lease only while the handler shows progress through heartbeat(). Past outbox.maxSilenceMs (default 5 minutes, never below the process manager's lease) renewal stops, the lease lapses and the message is redelivered. The system migrations pass sets one hour. * docs(plans): spin-up lessons cite local session records as plain text, not machine paths * feat(api): projects, user-avatar, langy and elevenlabs answer at /api/v1, the bare path kept as a twin Callers in the SDKs, MCP server and docs move to /api/v1. OpenAPI clients and docs regenerate, including the custom-roles-only listing. * fix(apidiff): wait for the seeded project, run main's worker and flush Redis between runs The fixtures waited only for the api, so they ran before haven's seed and got 401. Main's monolith lane needs WORKERS_IN_PROCESS=1. Destroyed stacks left their Redis databases, so a rerun dropped the fixture trace as a duplicate. * ci(gitleaks): allowlist the invented test values and digests the architecture merge brought in * fix(instant-eval): a query identity ClickHouse refuses answers instant_eval_not_enabled, not 500 * fix(group-queue): a worker that lost its claim can no longer dead-letter or re-stage the recovering worker's jobs The dead-letter and stage scripts check the claim token before any write, the same check the drain uses. * test(role): both role listings assert the full response body * fix(docker): Go images copy the vendored scenario SDK before downloading modules * fix(docker): the app image runs the prepare scripts per package, without nx, which the image does not carry * ci(migration-compat): run main's live api suites in one worker so their upgrade passes don't queue past the setup timeout * ci(go): the large-plan memory test runs without the race detector, in its own step * fix(go): clear the golangci-lint findings in the sims, services and tools, and resync the feature-parity lists * fix(apidiff): clear the golangci-lint findings * refactor(apidiff): split the run-flag parsing and boot planning into helpers under the lint limits * fix(group-queue): a worker that lost its claim keeps the payload lease the recovering worker relies on The failure paths complete first and release the lease only when the claim was still theirs, the same order as the success path. * fix(group-queue): a lost claim on the block path reports no dead payloads, as the dead-letter path already does * test(e2e): agent testing, automations, navigation, guardrail and docs-link specs assert the branch's deliberate layout * fix(ci): clear the upgradelab lint findings and give gateway-matrix's upgrade its ClickHouse * fix(haven): the monolith lane hosts its workers, destroy waits for the launcher before flushing, stacks may create LangWatchQL functions apidiff drops its monolith-worker workaround. * fix(eventing): the outbox silence limit is a flat five minutes; long-running process managers send heartbeats Topic clustering keeps 15 minutes (one langevals call can run 12) and the system migrations pass keeps an hour (one tenant's migration reports no progress). * test: bind the unbound scenarios, retarget stale annotations, and fix two stale unit tests * feat(auth): only the session sign-up started skips the secure-account offer Auth marks an account's first session (new Session.startedBySignUp column) and lists it on the browser-session entry; the user module reads it instead of a 60-second window, so a sign-in seconds after sign-up gets the offer. * fix(upgradelab): the SSO cookie check records the page's console, errors, failed requests and DOM, so a blank page names its cause Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(system-migrations): every tenant migration on the worker is paced on its own list by default (5,000 of its own jobs), so an undeclared one can't flood the queue Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgradelab): the SSO cookie check waits for the page to draw, not only for the boot screen to leave The final build's boot screen leaves when the theme's style lands in the root, a moment before the page; the check judged that gap as a blank page. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(ui): the boot screen stays until the app draws a real element, so the theme's style and script no longer leave a blank gap Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(system-migrations,authz): a tenant migration can wait for queue room inside one organisation; the authz grant import sends a big organisation's grants 500 at a time Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(group-queue): a listed run forgets each settled group from its set, baseline, candidates and live keys, so its Redis memory stays at its outstanding groups Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * perf(user): one upgrade step records every account's created event and re-states its standing, a page of 2,000 at a time with one batch send The two steps each sent one command per account and waited for it, so 200k accounts took over 45 minutes. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * perf(eventing,user): a pipeline reads many of its own aggregates in one ClickHouse query; the user upgrade step reads a page's standing history in one read Each page of 2,000 accounts made 2,000 reads queued behind the worker's ClickHouse cap, about 2 to 4 minutes a page at 200k users. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(eventing): the many-aggregate read brackets its tenant and aggregate lists, the form the ClickHouse tenant guard recognises Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(browser): online evaluations, experiments and datasets empty states offer set-up-with-AI again, as on main * refactor(eventing): a delivery's progress is any call through its module's dependencies, not explicit heartbeats The worker wraps each module's store clients, repositories, channels and peer Apis so a call inside an outbox delivery marks that delivery's progress. The explicit heartbeat calls threaded through 18 process managers are removed. * fix: regenerate the LWQL Prisma manifest and query fixture, register gq_jobs_recovered_total for hot reload, and update four stale unit tests * fix(eventing): lock and claim renewal timers start outside the delivery, so a hung replay or migration pass still loses its lease The system migrations pass drops its one-hour limit: every tenant step works through tracked calls. * docs: regenerate the user module and packages READMEs * ci(gateway-matrix): give the upgrade its Redis as well as its ClickHouse * fix(apidiff): re-sign the branch licence for old main without organizationId, and mask time-derived virtual-key prefixes * fix(ci): gateway-matrix sets the api-key pepper and makes its credentials key before the upgrade runs Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(go): CI go lint and tests pass again - seedgen keeps its Bsize conversion (int64 on Linux, uint32 on darwin) with an explained nolint - featureparity's inert list matches the Node tool (id-index-reads is planned, not built) - haven's process supervisor sums a group's RSS from ps -axo pgid=,rss=; Linux ps -g selects by session, so the memory ceiling never fired there Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * chore(langy): regenerate the feature map for the /api/v1/projects path * test(api): the npx entry-point test reads langwatch.ts, which now hosts the worker half * fix(make): the service target runs under bash, so its pipefail works on CI runners whose sh is dash Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(architecture-enforcer): the shared Prisma table list, the lint rule records and the coordinator protocol test match today's data Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(architecture-enforcer): annotations sit on their tests, and the ClickHouse, api-package, typecheck-script and contract-edge lists match today's tree Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix: browser-lane packages declare playwright, analytics links go through the host Link, and the enforcer tests match the dev launcher and the one rule that accepts a disable Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(deleted-spellings): the feature-flag module keeps its own use-feature-flag file, and noMembers matches only the deleted helper's import or call Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(ci): gateway-matrix names its public base URL, which the hosted MCP endpoint needs to boot Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(apidiff): run stacks start the analytics stand-in, and analytics scenarios are deferred where no stand-in answers * fix(docker): hoist @opentelemetry/api so the image's SDK working copy resolves its peer Without it every source-run process in the app image that imports the SDK fails with ERR_MODULE_NOT_FOUND; the chart's LWQL access job was the first. * fix(group-queue): a listed run counts as stopped only after its work stands still for 5 minutes, not 60 seconds, so a busy worker no longer fails healthy background steps Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgradelab): the Ops > Upgrades check waits for the boot screen to leave before it signs in, as the SSO check does Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(architecture): one kind of event; an upgrade step's events reach every consumer and the queue carries the volume Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(group-queue): a listed run's list stays open while it registers work, open lists take turns, and the listed consumer claims again as each of its jobs finishes Its follow-on jobs then run ahead of a backlog for as long as the run lasts: the list lapsed an hour after opening, the first open list took every slot, and a full listed consumer slept up to 5 s. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * chore(claude): coordinator, manifest and handoff folders keep their READMEs and templates; dated working files stay local and are ignored Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(skills): frontend, design-system and tooling skills teach today's component APIs, haven commands and sims; one way to drive a browser Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(skills): backend, eventing and upgrade skills teach today's shapes and the 2026-10-12 rulings (one kind of event, one pass per table, reruns through the step, own-queue pacing) Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgrade,system-migrations): background steps, paced passes and the pass request wait 5 minutes for their queue work to move, matching the group queue Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(skills): new skills for authorization and tenancy, platform primitives, system migrations, upgradelab and browser telemetry; the tenant-id lint hint shows the bracketed tenant set the guard accepts Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(api-key): a retried run-key mint waits on the key its first attempt wrote instead of minting another, so it converges once the grants projection catches up Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(project,organization,annotation): one project facts step walks projects once with batch sends; organisation presence sends per page; the five duplicate backfill tasks are gone (reruns go through the step in Ops) Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor: overlays, cards and copy buttons move to the design system's callable shapes; module code uses the house logger, nowInstant, ksuid and a contract error Design system: studio-* imports, Drawer.Root and Dialog.Root move to Drawer and Dialog where the size and padding scales match; Card.Root and SettingsCard move to Card; the trace dialog wrapper and two local copy controls are gone. Helpers: browser console calls use createLogger, repository clocks use nowInstant, idempotency keys use ksuid, and the scenario mapping refusal is ScenarioRunRejectedError. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(upgradelab): the Ops > Upgrades check's screenshot waits up to a minute and never fails the check Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(trace): the onboarding journey config imports the browser logger it uses Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test: module tests live in colocated __tests__, and 24 installation tests boot through bootInstalledProcess with testPeer Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs: the record names definePipeline's real package and drops two landed rows from §16; CLAUDE.md says the root scripts run through Nx Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(architecture): a request waiting on its own projection waits at most 10 seconds, then answers pending; no queue jumping, no inline folds Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(agent): connected-agent state goes through ConnectedAgentStateRepository (redis and memory twins) and relay messages through ConnectedAgentRelayChannel; services no longer touch the Redis session store Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * perf(group-queue): a listed run registers each page's groups in two calls instead of two per job, listed jobs coalesce like any other, and a throughput bench measures listed runs Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(design-system): the dialog backdrop scenarios bind to the design system's Dialog Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(langy): local control goes through call, user-wait and control-request repositories (redis and memory twins) and a workspace channel; the skip policy's publish moves out of rules into a service Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * feat(secrets,redis-client,api-key): mintOnce shares one sealed credential across servers; run keys and agent sandbox keys use it, so a run gets exactly one key The pending-key table is gone. Also lands the authz engine and identity sign-in switch tables and authz's new dependencies, ahead of their code. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * perf(clickhouse-client,eventing,group-queue): an insert may declare a tenant set, event appends waiting on one organisation's server fold into shared inserts (each still flushed, a failed one retried alone), and queue slots default to 1,000 Bench on the real append path: 1,706 to 22,019 users/min, 54,000 to 270 ClickHouse inserts for 6,000 users. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * feat(authz,workflow): the grant read-your-writes wait is 10 s everywhere; the run-key mint's pending answer is retryable, and the studio shows waiting and resends instead of failing Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * refactor(authz,identity,ops): the engine and sign-in switches are module-owned state, rolled back by an Ops event; the five legacy tenant migrations are declared tenant steps, Ops copies their old state twice and its Migrations page reads the framework state Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(ops,authz): during the rollout Ops writes moved tenant-migration state to both tables, the re-copy applies withdrawals made on the older page, and the authz tests expect the tenant guard fragment, the seat cap and the lineage reads Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Human Review BriefCaution This is a whole Go SDK, 82,000 lines across 563 files, with new CI, live-test and publish workflows. One review cannot cover it. It brings the Go scenario SDK to parity with JavaScript: two-phase judging with per-criterion verdicts, the event format, tracing, a voice module with six adapters and two tunnels, and four model providers. It supersedes the closed #80. The parts that bite after merge are the ones that publish and the ones that talk to LangWatch: the release-please config,
🗺️ Never opened this module? Start here · ~3 minscenario ships SDKs in JavaScript and Python, and now Go. Each SDK sends run events to LangWatch in one shared format, so a Go run shows in the same UI as a JS one. ⚖️ What approving this makes permanent · ~1 min
❓ Worth asking the author · ~1 min
✅ Why the evidence is good, and what each file is · ~1 minBenchmarks against the JS SDK and new CI and live-test workflows. See the PR body for test counts.
Note The event format must match what LangWatch reads, or Go runs will not show up. |
Summary
This brings the Go Scenario SDK to parity with the JavaScript SDK. It supersedes #80: same goal, rebuilt on current
main.Core
MinTurns/MaxTurnsand additional context. Large traces get a structure-only view plusexpand_trace/grep_trace, and traces can be fetched from LangWatch.ToolCalls,Transcript,Turns,Spansand the rest.SCENARIO_SEND_GRANULAR_EVENTS/WithGranularEventsForRun), following the contract verified with the backend (fix(scenarios): reconcile live previews with stored messages langwatch#8570 and feat(javascript): opt-in granular message events and live adapter progress #1015).t.Parallel(). A "Scenario Run" span holds "Scenario Turn" spans and LLM spans for the built-in agents.*http.Client), and results come back as values.Modules
scenariotestforgo test.go/voice, its own module):"provider/model"strings.go.workworkspace, a release-please component per module with linked versions, a live integration suite (manual workflow), and a Go-vs-JS benchmark harness (benchmarks/go-vs-js).Performance
Test status (honest)
go vetpasses in every module.go test -racebefore merging, up to the voice adapters and the first perf pass.Differences from JS
Listed in
go/README.mdunder "Differences from the JavaScript SDK".Review (CodeRabbit skipped this PR: 533 files)
Five reviewers split the PR by area: core, voice core, voice adapters, providers with the Claude Code agent, and CI/release/docs. Each finding was checked against the code, and every fix comes with a regression test. Fixed in the last 7 commits:
AgentNonBlockingnow picks the agent and takes its input before returning, as JS does. Previously a barge-in could reach the agent's input. Checkpoint criteria are merged only once. Clearing one run's spans no longer clears another run's spans in the same trace. A judge's own span collector and trace fetcher are cleared after each run. Evaluators still run when the run's context is cancelled.go-liveno longer passes its modules input through make, where it could run as shell code, and it now has a read-only token.go-cireadsneedsfrom the environment. The workflow shape check covers go-ci. The changelog no longer links a 0.1.0 tag that was never pushed.Still open:
go-completeis not a required check in branch protection yet, so Go CI can't block a merge.SendDTMFin b-leg may end the originating call once the digits finish. This is the same in JS and needs a live call to confirm.evaluatecheck is the low-risk workflow crashing on PRs over 300 files. fix(ci): treat PRs over the 300-file diff limit as oversized in the low-risk check #1018 fixes it, and the check re-runs once fix(ci): treat PRs over the 300-file diff limit as oversized in the low-risk check #1018 is on main.🤖 Generated with Claude Code