Repository navigation
feat(javascript)!: move to AI SDK 7 - #1024
0xdeafcafe wants to merge 1 commit into
Conversation
Judge and user simulator send their system prompts as `instructions`; telemetry uses the stable `telemetry` option; examples register the @ai-sdk/otel integration; docs updated for AI SDK 7. BREAKING CHANGE: @langwatch/scenario now requires `ai` >= 7 and Node.js >= 22.12. Projects on AI SDK 6 must stay on @langwatch/scenario 1.x. AI SDK 7 emits no spans until `registerTelemetry(new OpenTelemetry())` from @ai-sdk/otel is called.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Automated low-risk assessment This PR was evaluated against the repository's Low-Risk Pull Requests procedure and does not qualify as low risk.
This PR requires a manual review before merging. |
Why
AI SDK 7 refuses
{ role: "system" }messages insidemessagesat runtime, so scenario's judge and user simulator break for anyone on AI SDK 7, and the LangWatch monorepo cannot move to AI SDK 7 until scenario does (langwatch/langwatch#8722). Rewrite of #929 on current main. Draft placeholder: it depends on alangwatchSDK release carrying AI SDK 7 span support (langwatch/langwatch#8706).What changed
instructionsin the judge (decision and verdict calls) and the user simulator, rather thanallowSystemInMessages, so the requests stay valid without opting out of AI SDK 7's check. The judge's discovery recap slice moves from.slice(2)to.slice(1)because the system message is no longer element 0.allowSystemInMessages: trueis kept in three places where a system turn is real transcript content: author-scriptedmessage({ role: "system" })steps reaching the user simulator; the red-team attacker history, whose[SCORE]/[BACKTRACKED]/[INJECTED]markers are positional system turns; and the composable voice agent when it speaks first (AI SDK 7 refuses an emptymessages, so the system prompt cannot be hoisted out). Requests there are identical to AI SDK 6.experimental_telemetrybecomestelemetry; examples registerregisterTelemetry(new OpenTelemetry())from@ai-sdk/otelin a vitest setup file; docs show per-call metadata viaruntimeContext+includeRuntimeContext.ai^7.0.130 (peer>=7.0.0),@ai-sdk/openai^4.0.86, Node>=22.12(AI SDK 7 is ESM-only; the CJS build needsrequire(esm)). TODO:langwatch^1.20.0 is a placeholder; swap for the real release once feat: support AI SDK 7 spans in the SDK, trace ingestion and docs langwatch#8706 ships.Test plan
Not yet run:
pnpm installcannot resolve until thelangwatchrelease exists. Then, injavascript/:New:
specs/ai-sdk-7-system-prompts.feature,src/agents/__tests__/user-simulator-instructions.test.ts; the judge custom-prompt test asserts no system-role message.How I can prove I was successful
No playable artefact yet; see Test plan. Once green: a scenario run on AI SDK 7, traced into LangWatch.
Anything surprising?
setupScenarioTracingregisternew OpenTelemetry()(adds@ai-sdk/otel, risk of duplicate spans if the user also registers), or leave it to users (current state, documented).finish_testis missing:completion.toolCallsnow spans every step, sotoolCalls[0]picks the first step's call, not the last..slice(1)change has no dedicated test yet.javascript/package-lock.jsonsits besidepnpm-lock.yaml; decide whether to keep regenerating it.