Instrument the experience learning loop (story 4.1) - #14
Merged
Merged
Conversation
Add OpenTelemetry-compatible spans and metrics to the fourteen operations of the experience learning loop, emitted through System.Diagnostics only. The library emits; the host owns exporters. No project gains a PackageReference: ActivitySource and Meter ship in the net10.0 shared framework, so all four exact-pinned dependency-boundary assertions stay byte-identical and OpenTelemetry remains forbidden in Core and Abstractions. Traces carry run, attempt, record, event and feedback identifiers plus the host's correlation id. Metrics carry only bounded dimensions: operation, outcome, error class, and whether the call was nested inside another instrumented operation. Task text, record payload, private reasoning and raw exception messages never reach telemetry on either channel, and a test sweeps every span tag and every measurement for a planted marker to keep that honest. Exception objects are never handed to telemetry. A failing span records the exception's type name and a four-value error class; a driver message that could quote SQL text and parameters is never exported. Instrumentation cannot affect the caller: tagging and metric writes sit outside the guarded region and swallow internally, so a disposed Meter or a throwing listener can never turn a committed lifecycle transition into a reported failure. With no listener registered, operations allocate no Activity and return identical results. A deadline this library imposes classifies as Timeout, not as Cancelled, so a hung embedding provider inside the post-commit indexing hook raises a failure count instead of looking like a caller who asked to stop. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds OpenTelemetry-compatible spans and metrics to the fourteen operations of the experience learning loop, using
System.Diagnosticsonly. The library emits; the host owns exporters.No project gains a dependency
ActivitySourceandMetership in thenet10.0shared framework, so nothing was added to any.csprojand nopackages.lock.jsonmoved. All four exact-pinned dependency-boundary assertions pass unedited, andOpenTelemetrystays on the forbidden list for Core and Abstractions — the forbidden name is the SDK, and this uses the BCL API. A new boundary test pins the MAF adapter's package set the same way, so a second exporter pipeline can't be added quietly.What it emits
Spans carry run, attempt, record, event and feedback identifiers plus the host's correlation id, on both the returning and the faulted path. Metrics carry four bounded dimensions and nothing else:
operation,outcome,error.class, andnested— whether the call came from the host or from inside another instrumented operation.sum by (operation)filtered tonested=falseanswers "what did the host ask for"; unfiltered answers "what did the library do".What it will not emit
Task text, record payload, attempt inputs and results, private reasoning, and raw exception messages never reach telemetry on either channel. A test drives the whole loop with a marker planted in the task description, an attempt result, an attempt error, reflection text, evidence detail and lifecycle reasons, then sweeps every span tag and every measurement for it.
Exception objects are never handed to telemetry — no
AddException, noRecordException, noex.Message. A driver message can quote SQL text and parameters, so a failing span records the exception's type name and a four-value error class instead.Both channels have an exact allow-list on their key sets, so adding a fifth metric dimension or an undocumented span attribute fails a test rather than shipping.
Safety properties
Instrumentation cannot affect the caller. Tagging and metric writes sit outside the guarded region and swallow internally, so a disposed
Meteror a throwing listener can never turn a committed lifecycle transition into a reported failure.A deadline this library imposes is a
Timeout, not aCancelled. Classification distinguishes the host's token from an operation's own budget, so a hung embedding provider inside the post-commit indexing hook raises a failure count rather than looking like a caller who asked to stop.No listener means no cost and no difference. With nothing subscribed, operations allocate no
Activityand return results compared whole against the instrumented run.No duplicate MAF spans. The capture wrapper still only reads
Activity.Current; a test asserts the exact span multiset around a wrapped agent invocation, and that capture still harvests the ambient trace id intoProvenance.CorrelationId.Review
Two reviewers, then a fix pass, then a second amendment. 21 findings triaged; the ones worth naming:
Cancelled— the one class documented "normally not alertable".nestedexists.The verification-gap reviewer ran 27 mutations against the original tests; 10 survived, including making the entire failure-recording path a no-op for 12 of 13 operations. Every one of those now fails.
Tests: 989 → 1096, zero warnings under
TreatWarningsAsErrors.🤖 Generated with Claude Code