Repository navigation
Conversation
A trace processor for @openai/agents that sends each agent run to telemetry.dev as one trace: agents, turns, model calls with token usage, tool calls, and the commands a sandbox agent ran, with their exit codes and the sandbox backend. instrumentSandboxClient records the provider's sandbox id so the provider's own sandbox events can join the trace. Only model-call spans carry usage; task, turn, and agent spans repeat the totals of their calls. The root of a trace ends once the trace and all of its spans have ended, because other processors can delay span ends past the trace end, and a run that throws never ends its trace.
…ning The sandbox example used an undefined agent, and a plain Agent ignores the sandbox client, so it now builds a SandboxAgent. The double-count warning now also covers instrumentOpenAI(), which instruments every OpenAI client in the process.
The intro said each run is one trace. Runs that share a groupId land in one session trace, a run started inside an active span joins that span's trace, and sessionMode process groups runs without a groupId.
Match the docs guide: cover propagateAttributes() sessions and set sessionMode on the processor.
Name propagateAttributes() and process sessions, and the user ID and metadata that propagateAttributes() sends.
Test the rules the README states through a real Runner: groupId and propagateAttributes() sessions share one session trace with groupId winning, a run inside an active span joins that span, and process sessions group runs without a session. In the SDK, test that a root inside a span of the host's own context manager joins that span.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
@telemetry-dev/openai-agents, a trace processor for the OpenAI Agents SDK (@openai/agents0.21 and later). The SDK has its own tracing model and no OpenTelemetry export, so the processor maps each Agents trace to OTel spans through@telemetry-dev/sdk: agents, turns, model calls, tool calls, handoffs, guardrails, and thesandbox.*spans of sandbox agents. It's registered withaddTraceProcessor, which keeps the OpenAI Traces exporter, orsetTraceProcessorsto replace it.Only model calls (
chatspans) carry token usage. The task, turn, and agent spans repeat the totals of their calls, so sending them too would count each call two or three times. Chat Completions calls keep their usage only in the stored completion, so the processor reads it from there.Sandbox spans get
sandbox.operationandsandbox.backend. Theexec_commandandwrite_stdintool spans getprocess.exit.code, read from the header lineProcess exited with code Nin the tool output, since the JS SDK puts no exit code on its spans. A non-zero exit is recorded but doesn't mark the span as an error, because commands likegrepexit 1 without failing.instrumentSandboxClientwrites the provider's sandbox id (E2B'ssandboxId, for example) onto the session span, and the run's command spans copy it assandbox.id, so the provider's own sandbox events can be joined to the trace. The command and working directory ofsandbox.execare input content: they're sent only withcaptureInputon and go throughmask.The trace lifecycle is where I'd look most closely. The Agents SDK ends a trace only when its run returns, fires span starts and ends without awaiting them, and calls each processor in turn, so a slow processor registered first delivers span ends after the trace end. The root span therefore ends once the trace and every one of its spans have ended. A run that throws gets its root ended after 60 seconds without new spans, or at the next
forceFlush. Finished traces are kept for that long so late spans still join their root.shutdown, which the SDK also calls on SIGTERM, ends open spans but keeps their state, so a server that keeps draining requests still gets connected traces.Before the first release,
@telemetry-dev/openai-agentsneeds to exist on npm with its GitHub Actions trusted publisher set, asRELEASING.mddescribes. Otherwise the release'snpm publishfails. The lockfile adds the package's dependencies; pnpm also moves the sharedwspeer from 8.21.3 to 8.22.1 (the Agents SDK needs 8.22) and re-resolves@types/nodeunder protobufjs.The package tests run real agents against the local sandbox, with the OpenAI HTTP calls faked:
vp test runinpackages/openai-agentspasses 19 tests, and each new assertion fails when its check is removed.