[mthines]

Why

The Playwright gates (PR, staged build, production) leave nothing behind once a run ends except a CI log. As traces, their durations, flakes and failures can be charted and searched across runs, next to the CLI/REST/MCP smokes they run alongside.

What changed

• src/support/otel-reporter.ts: a Playwright reporter ported from dash0/tests/e2e/config/otel-reporter.ts.
  • One root span per test attempt, each its own trace, with test.case.* semconv and lorekit.e2e.{project,status,retry,flaky,duration_ms,tags,skip_reason}.
  • A child span per step (WHEN/THEN steps, actions, expects, hooks). Fixture steps are folded away, and steps are capped at 250 per test.
  • An unexpected failure is an ERROR span with an exception event.
• Exporter reused, not copied. Unlike the dash0 version, which uses NodeSDK, the reporter exports through the repo's existing scripts/telemetry/otlp-export.mjs and the CLI's resolveTelemetryConfig, as the eval exporter does:
  • same token priority, Dash0-Dataset routing and opt-outs;
  • service.name=e2e and the hard-coded deployment.environment.name=test;
  • no new dependencies.
    It sends once, from onEnd, and a failed export never fails the run. E2E_OTEL_OUT=<file> writes the payload to a file.
• Privacy. Playwright titles fill(x) as Fill "x", which for the sign-in step is the test account's password. Values of Fill/Type/Insert/Press/Key up/Key down steps are redacted to "…". The same values are scrubbed from error text, where a failed action's call log echoes them as fill("x"), and the configured secrets (E2E_USER_PASSWORD, LOREKIT_TESTING_USER_PW, E2E_SUPABASE_SERVICE_ROLE_KEY) are blanked from it. A fill that timed out on a disabled input, run through the reporter, leaves the password nowhere in the payload. stdout, stderr, attachment paths, username and hostname are not sent (the dash0 version records them). Absolute paths in error messages and stack traces are rewritten, the repo root to relative and the home directory to ~, so a failed local run does not ship the username. otel-spans.test.ts pins all of this, including buildRedactions (the list the reporter builds from the configured secrets, the repo root and the home directory).
• CI. The Playwright steps of web-e2e (ci.yml) and e2e-web-staged (deploy.yml) get the existing LOREKIT_TELEMETRY_TOKEN plus a LOREKIT_CORRELATION_ID. smoke-production already has both job-wide.
• Project graph. project.json gains implicitDependencies: cli, because the reporter imports the CLI's telemetry config. The test target also lists {workspaceRoot}/scripts/telemetry/**/* as an input, alongside the inputs Nx infers, so a change to the shared encoder runs otel-spans.test.ts.
• Docs. docs/otel.md (a new section and a service.name inventory row), docs/smoke-tests.md, and CLAUDE.md's package map.

How to verify

• pnpm nx typecheck web-e2e, test and lint all pass.
• nx show projects --affected --files=scripts/telemetry/otlp-export.mjs returns web-e2e.
• A deliberately failing test run through the reporter with E2E_OTEL_OUT: the exported stack reads packages/web-e2e/src/…, with no /Users/ path left.
• Ran the public suite against https://www.lorekit.io with a telemetry token: otel: exported 415 e2e spans for 35 test attempts. The spans show up in Dash0 (dev, default dataset) under service.name=e2e, nested correctly.
• actionlint: no new findings in the touched workflows.

Export follows the CLI's resolveTelemetryConfig: it is on when a credential resolves (LOREKIT_TELEMETRY_TOKEN, OTEL_EXPORTER_OTLP_HEADERS) or an explicit OTEL_EXPORTER_OTLP_ENDPOINT is set (which needs no credential), so a local run with any of those sends too; LOREKIT_TELEMETRY=off opts out.

User-facing docs/llms.txt: none applied, because this is test tooling and contributor docs only.

[dash0-dev]

Darkplane Automerge did not approve this pull request. It scored 66.14 of 100, and 75 was required.

• 🔴 Change footprint scored 33.
  This is a hand-written feature, not a mechanical edit. It adds about 620 lines across new files (otel-reporter.ts, otel-spans.ts, otel-modules.ts, otel-spans.test.ts) and a Playwright reporter hook. It also touches deploy.yml and ci.yml to pass LOREKIT_TELEMETRY_TOKEN, and project.json gains a new implicit dependency on cli. Risk is limited because it is e2e-only code, the reporter swallows export errors so it can't fail a run, and it reuses existing exporter modules. I did not read otel-spans.ts or otel-spans.test.ts.

The other 2 criteria

• 🟢 Blast radius scored 80.
  The change is confined to the web-e2e test package, docs, and env additions in ci.yml and deploy.yml. otel-reporter.ts catches export failures and logs a line instead of failing the run, and it sends nothing without a telemetry token. The deploy gate workflow is touched and the LOREKIT_TELEMETRY_TOKEN secret reaches more steps. project.json also adds a cli implicit dependency, and the reporter imports from scripts/telemetry and packages/cli. A revert is simple.
• 🟢 Evidence quality scored 85.
  The PR adds otel-spans.test.ts (+213 lines), which covers span building, redaction and buildRedactions. The substantive CI checks pass, including Web — Playwright e2e, Typecheck, Test & Lint (affected), and Integration smoke. Several automated review rounds, including one blocking password-leak finding, were fixed and tested. Remaining gaps: the reviews and the author's verification run are the author's own claims, and the reporter's real export path is not covered by automated tests.

_{An AI evaluator produced these scores from the diff, the repository history, and the pull request text. The pull request text is author-controlled, so treat the scores as signals rather than proof. Reviewed for commit 31d5e51.}

[mthines]

✅ No issues found

The new commit moves the redaction list into a tested pure function, buildRedactions, with no behaviour change. One advisory doc-wording note on the correlation key.

_{1 advisory finding below the confidence bar — see Less certain below.}

Checked: 11 of 11 changed files read · 0 of 3 dependent files traced · 1 possible issue → 0 confirmed → 0 posted

Progress: open review threads 3 → 1 → 1 → 0 → 0 · blocking 1 → 0 → 0 → 0 → 0 across the last 5 reviews

What this change reaches — 11 changed exports · 3 dependent files · 0 checked · 3 not checked

flowchart LR
  subgraph pr["Changed in this PR"]
    s1["OtlpEncoder<br/>added"]:::changed
    s2["buildRedactions<br/>added"]:::changed
    s3["buildTestSpans<br/>added"]:::changed
    s4["loadOtlpExport<br/>added"]:::changed
    s5["loadTelemetryConfig<br/>added"]:::changed
    s6["OtlpSpan<br/>added"]:::changed
    smore["+5 more changed exports<br/>listed below"]:::changed
  end
  s1c1["packages/web-e2e/src/support/ · 2 files<br/>✓ checked"]:::ok
  s1c2["packages/web-e2e/src/otel-spans.test.ts<br/>✓ checked"]:::ok
  s2c1["packages/web-e2e/src/otel-spans.test.ts<br/>✓ checked"]:::ok
  s2c2["packages/web-e2e/src/support/otel-reporter.ts<br/>✓ checked"]:::ok
  s3c1["packages/web-e2e/src/otel-spans.test.ts<br/>✓ checked"]:::ok
  s3c2["packages/web-e2e/src/support/otel-reporter.ts<br/>✓ checked"]:::ok
  s4c1["packages/web-e2e/src/otel-spans.test.ts<br/>✓ checked"]:::ok
  s4c2["packages/web-e2e/src/support/otel-reporter.ts<br/>✓ checked"]:::ok
  s5c1["packages/web-e2e/src/otel-spans.test.ts<br/>✓ checked"]:::ok
  s5c2["packages/web-e2e/src/support/otel-reporter.ts<br/>✓ checked"]:::ok
  s6c1["packages/web-e2e/src/support/ · 2 files<br/>? not checked"]:::unknown
  s1 --> s1c1
  s1 --> s1c2
  s2 --> s2c1
  s2 --> s2c2
  s3 --> s3c1
  s3 --> s3c2
  s4 --> s4c1
  s4 --> s4c2
  s5 --> s5c1
  s5 --> s5c2
  s6 -.-> s6c1
  classDef changed fill:#eef2ff,stroke:#6366f1,color:#1e1b4b
  classDef ok fill:#e7f6ec,stroke:#16a34a,color:#14532d
  classDef partial fill:#fef9c3,stroke:#ca8a04,color:#713f12
  classDef unknown fill:#f3f4f6,stroke:#9ca3af,stroke-dasharray:4 3,color:#374151
  classDef bad fill:#fde8e8,stroke:#dc2626,color:#7f1d1d
  classDef warn fill:#fff4e5,stroke:#d97706,color:#78350f
  classDef imports fill:#f8fafc,stroke:#64748b,stroke-dasharray:4 3,color:#1e293b

Loading

• OtlpEncoder (packages/web-e2e/src/support/otel-spans.ts) — added change · 3 consumer files · 3 verified unaffected
• buildRedactions (packages/web-e2e/src/support/otel-spans.ts) — added change · 2 consumer files · 2 verified unaffected
• buildTestSpans (packages/web-e2e/src/support/otel-spans.ts) — added change · 2 consumer files · 2 verified unaffected
• loadOtlpExport (packages/web-e2e/src/support/otel-modules.ts) — added change · 2 consumer files · 2 verified unaffected
• loadTelemetryConfig (packages/web-e2e/src/support/otel-modules.ts) — added change · 2 consumer files · 2 verified unaffected
• OtlpSpan (packages/web-e2e/src/support/otel-spans.ts) — added change · 2 consumer files · 2 not traced
• MAX_STEP_SPANS (packages/web-e2e/src/support/otel-spans.ts) — added change · 1 consumer file · 1 not traced
• ResultLike (packages/web-e2e/src/support/otel-spans.ts) — added change · 1 consumer file · 1 not traced
• StepLike (packages/web-e2e/src/support/otel-spans.ts) — added change · 1 consumer file · 1 not traced
• stepSpanName (packages/web-e2e/src/support/otel-spans.ts) — added change · 1 consumer file · 1 verified unaffected
• TestLike (packages/web-e2e/src/support/otel-spans.ts) — added change · 1 consumer file · 1 not traced

Less certain (1) — advisory, below the confidence bar

• docs/otel.md:358 — suggestion: web-e2e sends web-e2e-<run> while ci.yml's smokes send ci-<run>, and the staged job runs no smokes. Say the key is per job, shared only in smoke-production. (confidence 85)

Review details

Gate	Status	Details
Description vs. code	✅	Description matches the diff, including the buildRedactions extraction and the credential-free endpoint case
Prior review feedback	✅	Earlier review comments are resolved.
Documentation	✅	docs/otel.md, docs/smoke-tests.md and CLAUDE.md cover the reporter, its redaction and how export is enabled
Self-review signals	✅	No debug logs, leftover TODOs, or unreviewed stubs.
Code review	✅	The multi-lens review found no blocking issues.

Found

Quality — produced 1 → posted inline 0 · cleared 0 · carried forward 0 · deferred 0 · below-bar 1
Optimality — ran — approach optimal: buildRedactions extraction makes the reporter's list unit-testable with no new surface
Standards — ran — 1 governing doc (CLAUDE.md), 48 normative lines, 0 findings
Measurability — ran — gate 1 quiet: changed paths are test tooling, none classify web/mobile/api/worker

Run

full · 54 lines in delta · tier deep · depth checkout
Integrations — not activated — no manifest or lockfile in the diff
CI — still pending: Benchmark harness tests, Darkplane auto-approval, Deploy-scope resolver (unit), Edge typecheck (deno check), Migration order (no out-of-order prefixes), +7 more · 2 passed
Memories — no relevance rules or lessons consulted

^{Nothing to report — severity, 0 files skipped.}

^{pr-reviewer · commit 31d5e51 · full review · how these findings are produced · updated Oct 18, 2026 3:26pm UTC}