fix(node): Use context-scoped suppression for LangChain + AI provider span deduplication

.agents/skills/add-ai-integration/SKILL.md

@@ -64,8 +64,8 @@ Reference: `packages/node/src/integrations/tracing/vercelai/`
 
 **Use when:** SDK has no native OTel support (OpenAI, Anthropic, Google GenAI)
 
-1. **Core:** Create `instrument{Provider}Client()` in `packages/core/src/tracing/{provider}/index.ts` — Proxy to wrap client methods, create spans manually
-2. **Node.js `instrumentation.ts`:** Patch module exports, wrap client constructor. Check `_INTERNAL_shouldSkipAiProviderWrapping()` for LangChain compatibility.
+1. **Core:** Create `instrument{Provider}Client()` in `packages/core/src/tracing/{provider}/index.ts` — Proxy to wrap client methods, create spans manually. As the lowest-level integration, bail out of span creation when `_INTERNAL_isAiProviderSpanSuppressed()` returns `true` (see Provider Span Suppression).
+2. **Node.js `instrumentation.ts`:** Patch module exports, wrap client constructor.
 3. **Node.js `index.ts`:** Export integration function using `generateInstrumentOnce()` helper
 
 Reference: `packages/node/src/integrations/tracing/openai/`
@@ -75,10 +75,17 @@ Reference: `packages/node/src/integrations/tracing/openai/`
 **Use when:** SDK provides lifecycle hooks (LangChain, LangGraph)
 
 1. **Core:** Create `create{Provider}CallbackHandler()` — implement SDK's callback interface, create spans in callbacks
-2. **Node.js `instrumentation.ts`:** Auto-inject callbacks by patching runnable methods. Disable underlying AI provider wrapping.
+2. **Node.js `instrumentation.ts`:** Auto-inject callbacks by patching runnable methods. Wrap the underlying call in `_INTERNAL_withSuppressedAiProviderSpans()` so the lower-level provider SDK doesn't create duplicate spans (see Provider Span Suppression).
 
 Reference: `packages/node/src/integrations/tracing/langchain/`
 
+## Provider Span Suppression
+
+When a higher-level integration (LangChain, LangGraph) drives a provider SDK, both would create spans for one call. Coordinate via the scope-based helpers in `packages/core/src/tracing/ai/suppression.ts`:
+
+- **Higher-level integration:** wrap the underlying call in `_INTERNAL_withSuppressedAiProviderSpans(() => ...)`.
+- **Lowest-level integration (talks to the provider SDK):** bail out of span creation when `_INTERNAL_isAiProviderSpanSuppressed()` is `true`, including embeddings paths that call the SDK internally.
+
 ## Auto-Instrumentation (Node.js)
 
 **Mandatory** for Node.js AI integrations. OTel only patches when the package is imported (zero cost if unused).
@@ -99,6 +106,7 @@ Reference: `packages/node/src/integrations/tracing/langchain/`
 2. Set `SEMANTIC_ATTRIBUTE_SENTRY_ORIGIN = 'auto.ai.{provider}'` (alphanumerics, `_`, `.` only)
 3. Truncate large data with helper functions from `utils.ts`
 4. `gen_ai.invoke_agent` for parent ops, `gen_ai.chat` for child ops
+5. Lowest-level provider integrations must check `_INTERNAL_isAiProviderSpanSuppressed()` before creating spans (see Provider Span Suppression)
 
 ## Checklist
 

.cursor/BUGBOT.md

@@ -45,6 +45,7 @@ Unless explicitly noted (e.g. in the `Testing Conventions` section), only flag t
   - Only consider calling `captureException` if the instrumentation prevents errors from bubbling up (e.g. by swallowing them in a `try/catch` or an error event listener). Doing so is generally discouraged — prefer to let the error propagate instead.
   - Flag any instrumentation that swallows errors without calling `captureException`, and any instrumentation that calls `captureException` even though the error would still bubble up to the user (which causes double-reporting).
 - When calling `generateInstrumentationOnce`, the passed in name MUST match the name of the integration that uses it. If there are multiple instrumentations, they need to follow the pattern `${INSTRUMENTATION_NAME}.some-suffix`.
+- For AI provider instrumentations (to avoid duplicate spans when a higher-level integration like LangChain drives a provider SDK): flag any provider span-creation path (including embeddings methods that call the SDK internally) that does not bail out when `_INTERNAL_isAiProviderSpanSuppressed()` is `true`, and flag any process-global suppression mechanism (use `_INTERNAL_withSuppressedAiProviderSpans` instead, which is scoped).
 - Flag any unguarded `debug.log` / `debug.warn` / `debug.error` call in SDK source. The convention is the short-circuit form `DEBUG_BUILD && debug.log(...)` (not `if (DEBUG_BUILD) { ... }` wrapping). Without the `DEBUG_BUILD` gate the message text ships in production bundles and bloats bundle size.
 - Flag direct `console.log` / `console.warn` / `console.error` / `console.info` / `console.debug` calls in SDK source. The accepted patterns are:
   - The SDK's `debug` logger (gated with `DEBUG_BUILD && debug.*`) for SDK-internal diagnostics.

dev-packages/node-integration-tests/suites/tracing/langchain/scenario-openai-before-langchain.mjs

@@ -38,9 +38,7 @@ async function run() {
   const baseURL = `http://localhost:${server.address().port}`;
 
   await Sentry.startSpan({ op: 'function', name: 'main' }, async () => {
-    // EDGE CASE: Import and instantiate Anthropic client BEFORE LangChain is imported
-    // This simulates the timing issue where a user creates an Anthropic client in one file
-    // before importing LangChain in another file
+    // Direct Anthropic call made BEFORE LangChain is imported/used
     const { default: Anthropic } = await import('@anthropic-ai/sdk');
     const anthropicClient = new Anthropic({
       apiKey: 'mock-api-key',
@@ -55,8 +53,8 @@ async function run() {
       max_tokens: 100,
     });
 
-    // NOW import LangChain - at this point it will mark Anthropic to be skipped
-    // But the client created above is already instrumented
+    // Import and use LangChain - its own instrumentation records the span and suppresses the nested
+    // Anthropic SDK span only for the duration of the call.
     const { ChatAnthropic } = await import('@langchain/anthropic');
 
     // Create a LangChain model - this uses Anthropic under the hood
@@ -73,8 +71,8 @@ async function run() {
     // Use LangChain - this will be instrumented by LangChain integration
     await langchainModel.invoke('LangChain Anthropic call');
 
-    // Create ANOTHER Anthropic client after LangChain was imported
-    // This one should NOT be instrumented (skip mechanism works correctly)
+    // Direct Anthropic call made AFTER LangChain was used - still instrumented, since suppression
+    // is scoped to the LangChain call and does not leak to direct provider calls.
     const anthropicClient2 = new Anthropic({
       apiKey: 'mock-api-key',
       baseURL,

dev-packages/node-integration-tests/suites/tracing/langchain/test.ts

@@ -248,31 +248,29 @@ describe('LangChain integration', () => {
   );
 
   createEsmTests(__dirname, 'scenario-openai-before-langchain.mjs', 'instrument.mjs', (createRunner, test) => {
-    test('demonstrates timing issue with duplicate spans', async () => {
+    test('suppresses provider spans inside LangChain calls but keeps direct calls', async () => {
       await createRunner()
         .ignore('event')
         .expect({ transaction: { transaction: 'main' } })
         .expect({
           span: container => {
-            expect(container.items).toHaveLength(2);
-            const anthropicSpan = container.items.find(
-              span =>
-                span.attributes['sentry.origin'].value ===
-                (isOrchestrionEnabled() ? 'auto.ai.orchestrion.anthropic' : 'auto.ai.anthropic'),
+            expect(container.items).toHaveLength(3);
+
+            const anthropicOrigin = isOrchestrionEnabled() ? 'auto.ai.orchestrion.anthropic' : 'auto.ai.anthropic';
+
+            // Both direct Anthropic calls (before and after the LangChain import) are instrumented.
+            // Suppression is scoped to the LangChain call, so direct calls keep their spans.
+            const anthropicSpans = container.items.filter(
+              span => span.attributes['sentry.origin'].value === anthropicOrigin,
             );
-            expect(anthropicSpan).toBeDefined();
-            expect(anthropicSpan!.name).toBe('chat claude-3-5-sonnet-20241022');
+            expect(anthropicSpans).toHaveLength(2);
 
-            // LangChain call is instrumented by LangChain.
-            const langchainSpan = container.items.find(
+            // The LangChain call produces exactly one LangChain span; the nested Anthropic call is suppressed.
+            const langchainSpans = container.items.filter(
               span => span.attributes['sentry.origin'].value === 'auto.ai.langchain',
             );
-            expect(langchainSpan).toBeDefined();
-            expect(langchainSpan!.name).toBe('chat claude-3-5-sonnet-20241022');
-
-            // Third call (not present): Direct Anthropic call made AFTER LangChain import
-            // is NOT instrumented, which demonstrates the skip mechanism works for NEW
-            // clients. We should only have ONE Anthropic span (the first one), not two.
+            expect(langchainSpans).toHaveLength(1);
+            expect(langchainSpans[0]!.name).toBe('chat claude-3-5-sonnet-20241022');
           },
         })
         .start()

dev-packages/node-integration-tests/suites/tracing/langchain/v1/scenario-openai-before-langchain.mjs

@@ -38,9 +38,7 @@ async function run() {
   const baseURL = `http://localhost:${server.address().port}`;
 
   await Sentry.startSpan({ op: 'function', name: 'main' }, async () => {
-    // EDGE CASE: Import and instantiate Anthropic client BEFORE LangChain is imported
-    // This simulates the timing issue where a user creates an Anthropic client in one file
-    // before importing LangChain in another file
+    // Direct Anthropic call made BEFORE LangChain is imported/used
     const { default: Anthropic } = await import('@anthropic-ai/sdk');
     const anthropicClient = new Anthropic({
       apiKey: 'mock-api-key',
@@ -55,8 +53,8 @@ async function run() {
       max_tokens: 100,
     });
 
-    // NOW import LangChain - at this point it will mark Anthropic to be skipped
-    // But the client created above is already instrumented
+    // Import and use LangChain - its own instrumentation records the span and suppresses the nested
+    // Anthropic SDK span only for the duration of the call.
     const { ChatAnthropic } = await import('@langchain/anthropic');
 
     // Create a LangChain model - this uses Anthropic under the hood
@@ -73,8 +71,8 @@ async function run() {
     // Use LangChain - this will be instrumented by LangChain integration
     await langchainModel.invoke('LangChain Anthropic call');
 
-    // Create ANOTHER Anthropic client after LangChain was imported
-    // This one should NOT be instrumented (skip mechanism works correctly)
+    // Direct Anthropic call made AFTER LangChain was used - still instrumented, since suppression
+    // is scoped to the LangChain call and does not leak to direct provider calls.
     const anthropicClient2 = new Anthropic({
       apiKey: 'mock-api-key',
       baseURL,

dev-packages/node-integration-tests/suites/tracing/langchain/v1/test.ts

@@ -279,29 +279,29 @@ conditionalTest({ min: 20 })('LangChain integration (v1)', () => {
     'scenario-openai-before-langchain.mjs',
     'instrument.mjs',
     (createRunner, test) => {
-      test('demonstrates timing issue with duplicate spans', async () => {
+      test('suppresses provider spans inside LangChain calls but keeps direct calls', async () => {
         await createRunner()
           .ignore('event')
           .expect({ transaction: { transaction: 'main' } })
           .expect({
             span: container => {
-              expect(container.items).toHaveLength(2);
-              const anthropicSpan = container.items.find(
-                span =>
-                  span.attributes['sentry.origin'].value ===
-                  (isOrchestrionEnabled() ? 'auto.ai.orchestrion.anthropic' : 'auto.ai.anthropic'),
+              expect(container.items).toHaveLength(3);
+
+              const anthropicOrigin = isOrchestrionEnabled() ? 'auto.ai.orchestrion.anthropic' : 'auto.ai.anthropic';
+
+              // Both direct Anthropic calls (before and after the LangChain import) are instrumented.
+              // Suppression is scoped to the LangChain call, so direct calls keep their spans.
+              const anthropicSpans = container.items.filter(
+                span => span.attributes['sentry.origin'].value === anthropicOrigin,
               );
-              expect(anthropicSpan).toBeDefined();
-              expect(anthropicSpan!.name).toBe('chat claude-3-5-sonnet-20241022');
+              expect(anthropicSpans).toHaveLength(2);
 
-              const langchainSpan = container.items.find(
+              // The LangChain call produces exactly one LangChain span; the nested Anthropic call is suppressed.
+              const langchainSpans = container.items.filter(
                 span => span.attributes['sentry.origin'].value === 'auto.ai.langchain',
               );
-              expect(langchainSpan).toBeDefined();
-              expect(langchainSpan!.name).toBe('chat claude-3-5-sonnet-20241022');
-
-              // Third call (not present): Direct Anthropic call made AFTER LangChain import
-              // is NOT instrumented, demonstrating the skip mechanism works for NEW clients.
+              expect(langchainSpans).toHaveLength(1);
+              expect(langchainSpans[0]!.name).toBe('chat claude-3-5-sonnet-20241022');
             },
           })
           .start()

packages/core/src/shared-exports.ts

@@ -73,10 +73,9 @@ export {
   installedIntegrations,
 } from './integration';
 export {
-  _INTERNAL_skipAiProviderWrapping,
-  _INTERNAL_shouldSkipAiProviderWrapping,
-  _INTERNAL_clearAiProviderSkips,
-} from './utils/ai/providerSkip';
+  _INTERNAL_isAiProviderSpanSuppressed,
+  _INTERNAL_withSuppressedAiProviderSpans,
+} from './tracing/ai/suppression';
 export { filterKeyValueData as _INTERNAL_filterKeyValueData } from './utils/data-collection/filterKeyValueData';
 export { filterCookies as _INTERNAL_filterCookies } from './utils/data-collection/filterCookies';
 export { filterQueryParams as _INTERNAL_filterQueryParams } from './utils/data-collection/filterQueryParams';

packages/core/src/tracing/ai/suppression.ts

@@ -0,0 +1,27 @@
+import { getCurrentScope, withScope } from '../../currentScopes';
+import type { Scope } from '../../scope';
+
+const SUPPRESS_AI_PROVIDER_SPANS_KEY = '__SENTRY_SUPPRESS_AI_PROVIDER_SPANS__';
+
+/**
+ * Check if AI provider spans should be suppressed in the current scope.
+ *
+ * @internal
+ */
+export function _INTERNAL_isAiProviderSpanSuppressed(): boolean {
+  return getCurrentScope().getScopeData().sdkProcessingMetadata[SUPPRESS_AI_PROVIDER_SPANS_KEY] === true;
+}
+
+/**
+ * Execute a callback with AI provider spans suppressed in the current scope.
+ * This is used by higher-level integrations (like LangChain) to prevent
+ * duplicate spans from underlying AI provider instrumentations.
+ *
+ * @internal
+ */
+export function _INTERNAL_withSuppressedAiProviderSpans<T>(callback: () => T): T {
+  return withScope((scope: Scope) => {
+    scope.setSDKProcessingMetadata({ [SUPPRESS_AI_PROVIDER_SPANS_KEY]: true });
+    return callback();
+  });
+}

packages/core/src/tracing/anthropic-ai/index.ts

@@ -20,6 +20,7 @@ import {
   GEN_AI_RESPONSE_TOOL_CALLS_ATTRIBUTE,
   GEN_AI_SYSTEM_ATTRIBUTE,
 } from '../ai/gen-ai-attributes';
+import { _INTERNAL_isAiProviderSpanSuppressed } from '../ai/suppression';
 import type { InstrumentedMethodEntry } from '../ai/utils';
 import {
   buildMethodPath,
@@ -262,6 +263,10 @@ function instrumentMethod<T extends unknown[], R>(
 ): (...args: T) => R | Promise<R> {
   return new Proxy(originalMethod, {
     apply(target, thisArg, args: T): R | Promise<R> {
+      if (_INTERNAL_isAiProviderSpanSuppressed()) {
+        return Reflect.apply(target, thisArg, args);
+      }
+
       const operationName = instrumentedMethod.operation || 'unknown';
       const requestAttributes = extractRequestAttributes(args, methodPath, operationName);
       const model = requestAttributes[GEN_AI_REQUEST_MODEL_ATTRIBUTE] ?? 'unknown';

packages/core/src/tracing/google-genai/index.ts

@@ -27,6 +27,7 @@ import {
   GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE,
   GEN_AI_USAGE_TOTAL_TOKENS_ATTRIBUTE,
 } from '../ai/gen-ai-attributes';
+import { _INTERNAL_isAiProviderSpanSuppressed } from '../ai/suppression';
 import type { InstrumentedMethodEntry } from '../ai/utils';
 import { stringify } from '../../utils/string';
 import {
@@ -281,6 +282,10 @@ function instrumentMethod<T extends unknown[], R>(
 
   return new Proxy(originalMethod, {
     apply(target, _, args: T): R | Promise<R> {
+      if (_INTERNAL_isAiProviderSpanSuppressed()) {
+        return Reflect.apply(target, _, args);
+      }
+
       const operationName = instrumentedMethod.operation || 'unknown';
       const params = args[0] as Record<string, unknown> | undefined;
       const requestAttributes = extractRequestAttributes(operationName, params, context);

packages/core/src/tracing/langchain/embeddings.ts

@@ -11,6 +11,7 @@ import {
   GEN_AI_REQUEST_MODEL_ATTRIBUTE,
   GEN_AI_SYSTEM_ATTRIBUTE,
 } from '../ai/gen-ai-attributes';
+import { _INTERNAL_withSuppressedAiProviderSpans } from '../ai/suppression';
 import { resolveAIRecordingOptions } from '../ai/utils';
 import { LANGCHAIN_ORIGIN } from './constants';
 import type { LangChainOptions } from './types';
@@ -85,12 +86,17 @@ export function instrumentEmbeddingMethod(
           attributes: attributes as Record<string, SpanAttributeValue>,
         },
         () => {
-          return Reflect.apply(target, thisArg, args).then(undefined, error => {
-            captureException(error, {
-              mechanism: { handled: false, type: 'auto.ai.langchain' },
-            });
-            throw error;
-          });
+          // `embedQuery`/`embedDocuments` call the underlying provider SDK (e.g. openai) internally, so
+          // suppress that SDK's own instrumentation for this call to avoid a duplicate span.
+          return _INTERNAL_withSuppressedAiProviderSpans(() => Reflect.apply(target, thisArg, args)).then(
+            undefined,
+            error => {
+              captureException(error, {
+                mechanism: { handled: false, type: 'auto.ai.langchain' },
+              });
+              throw error;
+            },
+          );
         },
       );
     },

packages/core/src/tracing/openai/index.ts

@@ -15,6 +15,7 @@ import {
   GEN_AI_SYSTEM_ATTRIBUTE,
   GEN_AI_SYSTEM_INSTRUCTIONS_ATTRIBUTE,
 } from '../ai/gen-ai-attributes';
+import { _INTERNAL_isAiProviderSpanSuppressed } from '../ai/suppression';
 import type { InstrumentedMethodEntry } from '../ai/utils';
 import { stringify } from '../../utils/string';
 import {
@@ -151,6 +152,10 @@ function instrumentMethod<T extends unknown[], R>(
   options: OpenAiOptions,
 ): (...args: T) => Promise<R> {
   return function instrumentedCall(...args: T): Promise<R> {
+    if (_INTERNAL_isAiProviderSpanSuppressed()) {
+      return originalMethod.apply(context, args);
+    }
+
     const operationName = instrumentedMethod.operation || 'unknown';
     const requestAttributes = extractRequestAttributes(args, operationName);
     const model = (requestAttributes[GEN_AI_REQUEST_MODEL_ATTRIBUTE] as string) || 'unknown';