Skip to content

Instrument the experience learning loop (story 4.1) - #14

Merged
fabbrik merged 1 commit into
mainfrom
feat/4-1-telemetry
Sep 22, 2026
Merged

fabbrik merged 1 commit into
mainfrom
feat/4-1-telemetry

Conversation

@fabbrik

@fabbrik fabbrik commented Sep 22, 2026

Copy link
Copy Markdown
Owner

Adds OpenTelemetry-compatible spans and metrics to the fourteen operations of the experience learning loop, using System.Diagnostics only. The library emits; the host owns exporters.

No project gains a dependency

ActivitySource and Meter ship in the net10.0 shared framework, so nothing was added to any .csproj and no packages.lock.json moved. All four exact-pinned dependency-boundary assertions pass unedited, and OpenTelemetry stays on the forbidden list for Core and Abstractions — the forbidden name is the SDK, and this uses the BCL API. A new boundary test pins the MAF adapter's package set the same way, so a second exporter pipeline can't be added quietly.

What it emits

Spans carry run, attempt, record, event and feedback identifiers plus the host's correlation id, on both the returning and the faulted path. Metrics carry four bounded dimensions and nothing else: operation, outcome, error.class, and nested — whether the call came from the host or from inside another instrumented operation. sum by (operation) filtered to nested=false answers "what did the host ask for"; unfiltered answers "what did the library do".

What it will not emit

Task text, record payload, attempt inputs and results, private reasoning, and raw exception messages never reach telemetry on either channel. A test drives the whole loop with a marker planted in the task description, an attempt result, an attempt error, reflection text, evidence detail and lifecycle reasons, then sweeps every span tag and every measurement for it.

Exception objects are never handed to telemetry — no AddException, no RecordException, no ex.Message. A driver message can quote SQL text and parameters, so a failing span records the exception's type name and a four-value error class instead.

Both channels have an exact allow-list on their key sets, so adding a fifth metric dimension or an undocumented span attribute fails a test rather than shipping.

Safety properties

Instrumentation cannot affect the caller. Tagging and metric writes sit outside the guarded region and swallow internally, so a disposed Meter or a throwing listener can never turn a committed lifecycle transition into a reported failure.

A deadline this library imposes is a Timeout, not a Cancelled. Classification distinguishes the host's token from an operation's own budget, so a hung embedding provider inside the post-commit indexing hook raises a failure count rather than looking like a caller who asked to stop.

No listener means no cost and no difference. With nothing subscribed, operations allocate no Activity and return results compared whole against the instrumented run.

No duplicate MAF spans. The capture wrapper still only reads Activity.Current; a test asserts the exact span multiset around a wrapped agent invocation, and that capture still harvests the ambient trace id into Provenance.CorrelationId.

Review

Two reviewers, then a fix pass, then a second amendment. 21 findings triaged; the ones worth naming:

  • Instrumentation was inside the try/catch that reports failure, so a throwing tag expression would have reported an already-durable lifecycle commit as failed.
  • A library-imposed indexing timeout classified as Cancelled — the one class documented "normally not alertable".
  • The first fix for metric double-counting removed the failure signal for all work done inside finalization. That was the wrong trade and is why nested exists.

The verification-gap reviewer ran 27 mutations against the original tests; 10 survived, including making the entire failure-recording path a no-op for 12 of 13 operations. Every one of those now fails.

Tests: 989 → 1096, zero warnings under TreatWarningsAsErrors.

🤖 Generated with Claude Code

Add OpenTelemetry-compatible spans and metrics to the fourteen operations of
the experience learning loop, emitted through System.Diagnostics only. The
library emits; the host owns exporters. No project gains a PackageReference:
ActivitySource and Meter ship in the net10.0 shared framework, so all four
exact-pinned dependency-boundary assertions stay byte-identical and
OpenTelemetry remains forbidden in Core and Abstractions.

Traces carry run, attempt, record, event and feedback identifiers plus the
host's correlation id. Metrics carry only bounded dimensions: operation,
outcome, error class, and whether the call was nested inside another
instrumented operation. Task text, record payload, private reasoning and raw
exception messages never reach telemetry on either channel, and a test sweeps
every span tag and every measurement for a planted marker to keep that honest.

Exception objects are never handed to telemetry. A failing span records the
exception's type name and a four-value error class; a driver message that
could quote SQL text and parameters is never exported.

Instrumentation cannot affect the caller: tagging and metric writes sit
outside the guarded region and swallow internally, so a disposed Meter or a
throwing listener can never turn a committed lifecycle transition into a
reported failure. With no listener registered, operations allocate no Activity
and return identical results.

A deadline this library imposes classifies as Timeout, not as Cancelled, so a
hung embedding provider inside the post-commit indexing hook raises a failure
count instead of looking like a caller who asked to stop.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@fabbrik
fabbrik merged commit 588a5fa into main Sep 22, 2026
1 check passed
@fabbrik
fabbrik deleted the feat/4-1-telemetry branch September 22, 2026 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant