Skip to content

feat(javascript)!: move to AI SDK 7 - #1024

Draft
0xdeafcafe wants to merge 1 commit into
mainfrom
feat/ai-sdk-7-v2
Draft

0xdeafcafe wants to merge 1 commit into
mainfrom
feat/ai-sdk-7-v2

Conversation

@0xdeafcafe

Copy link
Copy Markdown
Collaborator

Why

AI SDK 7 refuses { role: "system" } messages inside messages at runtime, so scenario's judge and user simulator break for anyone on AI SDK 7, and the LangWatch monorepo cannot move to AI SDK 7 until scenario does (langwatch/langwatch#8722). Rewrite of #929 on current main. Draft placeholder: it depends on a langwatch SDK release carrying AI SDK 7 span support (langwatch/langwatch#8706).

What changed

  • System prompts go as top-level instructions in the judge (decision and verdict calls) and the user simulator, rather than allowSystemInMessages, so the requests stay valid without opting out of AI SDK 7's check. The judge's discovery recap slice moves from .slice(2) to .slice(1) because the system message is no longer element 0.
  • allowSystemInMessages: true is kept in three places where a system turn is real transcript content: author-scripted message({ role: "system" }) steps reaching the user simulator; the red-team attacker history, whose [SCORE]/[BACKTRACKED]/[INJECTED] markers are positional system turns; and the composable voice agent when it speaks first (AI SDK 7 refuses an empty messages, so the system prompt cannot be hoisted out). Requests there are identical to AI SDK 6.
  • experimental_telemetry becomes telemetry; examples register registerTelemetry(new OpenTelemetry()) from @ai-sdk/otel in a vitest setup file; docs show per-call metadata via runtimeContext + includeRuntimeContext.
  • Versions: ai ^7.0.130 (peer >=7.0.0), @ai-sdk/openai ^4.0.86, Node >=22.12 (AI SDK 7 is ESM-only; the CJS build needs require(esm)). TODO: langwatch ^1.20.0 is a placeholder; swap for the real release once feat: support AI SDK 7 spans in the SDK, trace ingestion and docs langwatch#8706 ships.

Test plan

Not yet run: pnpm install cannot resolve until the langwatch release exists. Then, in javascript/:

pnpm install && pnpm run lint && pnpm run typecheck && pnpm test
pnpm run build && pnpm run smoke:dist
pnpm -F vitest-examples typecheck

New: specs/ai-sdk-7-system-prompts.feature, src/agents/__tests__/user-simulator-instructions.test.ts; the judge custom-prompt test asserts no system-role message.

How I can prove I was successful

No playable artefact yet; see Test plan. Once green: a scenario run on AI SDK 7, traced into LangWatch.

Anything surprising?

  • Scenario's own LLM calls stop being traced by default: AI SDK 7 emits no spans until an integration is registered. Open decision: have setupScenarioTracing register new OpenTelemetry() (adds @ai-sdk/otel, risk of duplicate spans if the user also registers), or leave it to users (current state, documented).
  • Judge fallback when finish_test is missing: completion.toolCalls now spans every step, so toolCalls[0] picks the first step's call, not the last.
  • The .slice(1) change has no dedicated test yet.
  • javascript/package-lock.json sits beside pnpm-lock.yaml; decide whether to keep regenerating it.

Judge and user simulator send their system prompts as `instructions`;
telemetry uses the stable `telemetry` option; examples register the
@ai-sdk/otel integration; docs updated for AI SDK 7.

BREAKING CHANGE: @langwatch/scenario now requires `ai` >= 7 and
Node.js >= 22.12. Projects on AI SDK 6 must stay on @langwatch/scenario 1.x.
AI SDK 7 emits no spans until `registerTelemetry(new OpenTelemetry())`
from @ai-sdk/otel is called.
@coderabbitai

coderabbitai Bot commented Oct 14, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Automated low-risk assessment

This PR was evaluated against the repository's Low-Risk Pull Requests procedure and does not qualify as low risk.

The PR changes how the code integrates with external AI SDKs (moves system-role messages into top-level instructions, adds telemetry registration, and updates ai and provider SDK versions), and also bumps Node engine and package dependencies. These are changes to third‑party integrations and runtime behavior, which are explicitly excluded from low‑risk changes and therefore require normal review.

This PR requires a manual review before merging.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant