From ae0a6b04f542845b45d9de3e0ac4361cbab4964a Mon Sep 17 00:00:00 2001 From: Jamie Date: Wed, 26 Aug 2026 18:41:32 -0400 Subject: [PATCH 1/2] Document canonical Lighthouse operations --- AGENTS.md | 41 +++++- CHANGELOG.md | 11 ++ OPERATIONS.md | 290 ++++++++++++++++++++++++++++++++++++++ PHASE2_ANALYTICS_NOTES.md | 2 + PHASE3_ANALYTICS_NOTES.md | 2 + README.md | 18 +++ SOT.md | 29 +++- package-lock.json | 4 +- package.json | 2 +- plan.md | 2 + 10 files changed, 389 insertions(+), 12 deletions(-) create mode 100644 OPERATIONS.md diff --git a/AGENTS.md b/AGENTS.md index 211113d..a24602a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -33,20 +33,44 @@ Authority sources in descending order: - Do not print or commit secret values. - Cross-repository contracts with Agent Smith, BUS Core, buscore-site, or the leads database must be documented when touched. +## Canonical Analytics Access and Diagnostics + +For any Lighthouse alert, WATCH, analytics question, report failure, service-health question, or production-access request: + +1. Read `SOT.md`, then `OPERATIONS.md`, then the relevant `CHANGELOG.md` entry and contract/fixture. +2. Treat `OPERATIONS.md` as the canonical procedure for choosing a diagnostic surface and classifying its side effects. It is subordinate to `SOT.md` and cannot authorize behavior absent from the SOT. +3. Default to local zero-mutation diagnosis. Production, Cloudflare, Discord, GitHub, D1, or other external access requires explicit scope approval. +4. If the approved endpoint, credential source, account context, or tool access is missing, report `ACCESS_BLOCKED`. Do not improvise an endpoint, credential, direct SQL query, or alternate service. +5. Do not fan out across every endpoint. Start with supplied evidence; when live access is approved, use the stored-data `GET /report?view=ceo` diagnostic first and narrow from there. + +Operational constraints: + +- `GET /report?view=source_health` is ingestion-integrity evidence, not service-probe truth. +- `WATCH` is not synonymous with outage; Agent Smith owns WATCH/ALERT/UNAVAILABLE wording, while Lighthouse owns facts and availability. +- The current `ADMIN_TOKEN` is broad: it protects report reads and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Possession does not authorize writes. +- Many GET/HEAD surfaces change evidence. Bare/fleet/site reports refresh stored traffic; report failures can increment errors; artifact HEAD records raw/HEAD truth; update, redirect, artifact, telemetry, and admin-write routes are not passive probes. +- Lighthouse, Agent Smith, BUS Core, buscore-site, and tgc-site are separate failure domains. Do not infer a cross-service outage from one unavailable layer. + --- ## 2. Mandatory Change Bundle -For any non-trivial change, the following **must** be updated together in the same change set: +For any non-trivial change, the following **must** be updated together in the same change set unless the owner-approved documentation-only exception below applies: - **Code** - **SOT.md** - **CHANGELOG.md** -- **Version** (in `package.json`) +- **Version** (aligned in `package.json` and `package-lock.json`) This is a requirement, not a suggestion. -Failure to update all four components is a policy violation. +Failure to satisfy either the complete bundle or every condition of the documentation-only exception is a policy violation. + +### Owner-Approved Documentation-Only Exception + +A non-trivial documentation-only change may omit the **Code** component only with explicit owner approval and only when it governs or documents existing runtime behavior without changing it. The change must still update `SOT.md`, `CHANGELOG.md`, `package.json`, and `package-lock.json` together. `CHANGELOG.md` must state that no runtime code, endpoint, response contract, auth, configuration, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior changed. + +This exception does not apply when runtime or contract implementation is required. Any such change requires the complete **Code + SOT + CHANGELOG + Version** bundle. Never add a dummy code edit merely to satisfy the normal bundle. --- @@ -157,6 +181,9 @@ Any agent performing modification work must report: - **Version bump** — Old → New - **Unresolved drift** — Any conflicts or mismatches found - **Blocked items** — Work requiring explicit human approval +- **Diagnostic access** — None / local only / approved external reads, with the surface named +- **Production interaction** — Yes/No +- **Evidence side effects** — Yes/No; name any request that could refresh, count, persist, archive, or otherwise change evidence Agents must not complete work if any mandatory component is skipped. @@ -240,14 +267,14 @@ Repository-wide integration rule for Lighthouse-tracked public sites: ## Summary **Before making changes:** -1. Read SOT.md and CHANGELOG.md -2. Understand current documented behavior +1. Read SOT.md, OPERATIONS.md, and CHANGELOG.md +2. Understand current documented behavior and the diagnostic side-effect class **When making changes:** -1. Update code +1. Update code, unless the explicit owner-approved documentation-only exception applies 2. Update SOT.md 3. Update CHANGELOG.md -4. Bump version in `package.json` +4. Bump and align the version in `package.json` and `package-lock.json` 5. Report all changes explicitly **Golden rule:** diff --git a/CHANGELOG.md b/CHANGELOG.md index 2b985c7..fde541a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,16 @@ # Changelog +## [1.29.3] - 2026-08-26 + +- Added `OPERATIONS.md` as the canonical Lighthouse access and diagnostic runbook, including ownership boundaries, resource and credential maps, a morning-triage sequence, source/probe interpretation, and standard incident output. +- Routed agents and operators through the SOT and runbook before live access, with an explicit `ACCESS_BLOCKED` outcome when approved endpoint, credential, account, or tool access is unavailable. +- Documented that the shared `ADMIN_TOKEN` is broad and protects both report reads and administrative writes. +- Classified report, manifest, update, redirect, artifact, telemetry, scheduled, D1, deployment, and administrative surfaces by their possible evidence/state side effects. +- Marked the Phase 2, Phase 3, and policy-alignment documents as scoped historical or policy references rather than current operations authority. +- Reconciled cross-repository producer baselines to released BUS Core `1.4.2` and the merged buscore-site `1.4.2` release sync while retaining Agent Smith `0.25.2` as shipped authority; recorded the unresolved BUS Core restore/import signal authority drift rather than legitimizing it. +- Added the explicit owner-approved documentation-only bundle exception: code may be omitted only when runtime and contract behavior do not change; SOT, changelog, package, and lockfile remain mandatory, and dummy code changes are forbidden. +- No Worker code, endpoint, response contract, auth, configuration, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior changed. The deployed Worker remains Lighthouse 1.29.2 (`f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`) with CEO contract `1.1`; version 1.29.3 identifies this repository documentation release. + ## [1.29.2] - 2026-08-10 - Repaired the lead-endpoint health check so its safe GET probe accepts `405 Method Not Allowed` even when the live endpoint omits an `Allow` header; `404` and other failures still fail, and Lighthouse never creates a synthetic lead. diff --git a/OPERATIONS.md b/OPERATIONS.md new file mode 100644 index 0000000..da15d5e --- /dev/null +++ b/OPERATIONS.md @@ -0,0 +1,290 @@ +# Lighthouse Operations and Diagnostics + +- Status: current operational runbook +- Scope: Lighthouse analytics access, evidence interpretation, and incident diagnosis +- Runtime baseline: deployed Lighthouse Worker `1.29.2`; repository documentation release `1.29.3`; CEO report and metric-definition contract `1.1` +- Last reconciled: 2026-08-26 +- Live Lighthouse endpoint and Cloudflare control-plane verification during this documentation change: not performed + +## Authority and Purpose + +Use this file to locate the correct diagnostic surface without rediscovering the system or probing every endpoint. + +Authority order remains: + +1. `SOT.md` — intended and shipped Lighthouse behavior. +2. `CHANGELOG.md` — shipped-change history. +3. `src/index.ts` and `contracts/` — implementation and machine-readable contracts. +4. `OPERATIONS.md` — canonical access and diagnostic procedure, subordinate to the sources above. +5. `README.md` — implementation overview and setup. + +If this runbook conflicts with `SOT.md`, stop and report the conflict. Do not improvise a route, credential, query, storage meaning, or status interpretation. + +## Safety Default + +The default diagnostic posture is local and zero-mutation: + +- Read repository documentation, contracts, fixtures, configuration names, and code. +- Inspect local Git state without changing it. +- Do not contact production, Cloudflare, Discord, GitHub, Airtable, or another service unless the user has authorized that scope. +- Do not run migrations, deployments, scheduled handlers, retention jobs, report snapshots, notes, campaign writes, telemetry submissions, or release downloads during passive diagnosis. +- Never print, persist, or commit secret values, raw lead data, identifiers, IP material, or other private payloads. + +If the approved endpoint, credential source, account context, or tool authorization is unavailable, record `ACCESS_BLOCKED` and stop that diagnostic branch. Do not substitute guessed URLs, unrelated credentials, direct D1 queries, or broader probes. + +## System Ownership + +| System | Owns | Does not prove | +|---|---|---| +| Lighthouse | Analytics ingestion, aggregate storage, source availability/freshness/coverage, scheduled service-probe evidence, protected report payloads, and CEO contract `1.1` | Agent Smith delivery, Discord delivery, or producer-side transmission success | +| Agent Smith | Report-mode selection, strict CEO validation, status/trust wording, daily/weekly/monthly presentation, Discord delivery orchestration/attempts, and monthly archive attempts | That every Lighthouse scheduled task ran, every source is fresh, Discord received a message, the watch channel is configured, posting permission exists, or a monthly archive write succeeded merely because Smith's private `/health` command responds | +| BUS Core | Optional, fail-soft product-event production; default-config startup/manual reads of Lighthouse `/update/check`; and user-triggered staging that re-reads the configured manifest URL without analytics query parameters before GETting the manifest-declared artifact | Lighthouse acceptance, availability, producer authenticity, or completed staging; the canonical client does not call Lighthouse `/manifest/core/stable.json` or `/download/latest` | +| `buscore-site` | BUS Core public-site event production; `dev_mode`/`noAnalytics` suppression; early-access and Managed BUS routes; Turnstile/KV controls; and D1 lead writes | Lighthouse acceptance or persistence after a fail-soft browser send | +| `tgc-site` | Consent-gated TGC event and Cloudflare Web Analytics production; `dev_mode`, privacy-signal, origin, and `test_mode` controls; and the static `/api/intake` client | Lighthouse acceptance or persistence after a fail-soft browser send, or proven backend delivery for `/api/intake` | +| `tgc-ops` | Pointer-first cross-asset documentation in principle | Current analytics truth: its telemetry/dependency records contain known stale claims and must be reconciled before use | + +Lighthouse remains independently runnable. A failure in one producer, consumer, optional binding, or presentation layer is not automatically a Lighthouse outage. + +## Canonical Locations + +| Need | Canonical location | +|---|---| +| Governance and approval boundaries | `AGENTS.md` | +| Shipped behavior, routes, auth, storage, scheduling | `SOT.md` | +| Diagnostic sequence and side-effect classification | `OPERATIONS.md` | +| Shipped history | `CHANGELOG.md` | +| Runtime bindings and cron | `wrangler.toml` | +| Route implementation | `src/index.ts` | +| D1 schema history | `migrations/` | +| CEO response contract | `contracts/ceo-v1/report.schema.json` | +| BUS Core product telemetry contract | `contracts/buscore-product-telemetry-v1.json` and `src/productTelemetry.ts` | +| Representative CEO states | `contracts/ceo-v1/*.json` | +| Metric meanings | `SOT.md`, `README.md`, `BUS_CORE_TRAFFIC_TRUTH.md`, and `TGC_SITE_ANALYTICS_POLICY.md` | +| Historical Phase 2/3 implementation record | `PHASE2_ANALYTICS_NOTES.md` and `PHASE3_ANALYTICS_NOTES.md`; never use alone as current operations authority | +| Agent Smith mode, commands, status, schedules, and delivery | `../Agent_Smith/SOT.md`, `../Agent_Smith/CHANGELOG.md`, `../Agent_Smith/wrangler.toml`, `../Agent_Smith/CONTRACTS.md`, `../Agent_Smith/BUS_CORE_REPORTING_CONTRACT.md`, and `../Agent_Smith/src/commands/` | +| BUS Core product producer | `../TGC-BUS-Core/SOT.md`, `../TGC-BUS-Core/OPERATIONS.md`, `../TGC-BUS-Core/CHANGELOG.md`, `../TGC-BUS-Core/core/telemetry/client.py`, `../TGC-BUS-Core/core/services/update.py`, `../TGC-BUS-Core/core/services/update_stage.py`, and `../TGC-BUS-Core/core/api/routes/telemetry.py` | +| BUS Core site producer | `../buscore-site/SOT.md`, `../buscore-site/CHANGELOG.md`, `../buscore-site/SITE_ANALYTICS_IMPLEMENTATION.md`, `../buscore-site/manifest/core/stable.json`, `../buscore-site/assets/js/site-analytics.js`, and `../buscore-site/tests/browser/deploy-analytics.test.mjs` | +| TGC site producer | `../tgc-site/SOT.md`, `../tgc-site/contracts/lighthouse-analytics-contract.md`, `../tgc-site/assets/js/telemetry.js`, and `../tgc-site/tests/telemetry-payload.test.js` | + +Sibling paths above describe the audited local checkout layout. If a repository is absent, stop that cross-repository branch rather than searching unrelated locations. A checked-in buscore-site manifest is a repository release projection that can lag production; it is not live endpoint proof. BUS Core facts were last reconciled against released version `1.4.2`, and buscore-site source facts against its merged `1.4.2` release-sync change, on 2026-08-26. Agent Smith facts remain reconciled against shipped version `0.25.2`; an unshipped local `0.25.3` documentation bundle is not current authority. Older Agent Smith SOT paragraphs that call `/report` a raw diagnostic product are superseded: in current production `ceo_v1`, the private `/report` command renders the CEO business/decision product. The retained `formatDiagnosticReport()` is test-only and has no command/runtime route. + +## Deployed Resource Map + +The following identifiers are checked into `wrangler.toml`: + +| Binding or trigger | Configured resource | Purpose | +|---|---|---| +| Worker name | `buscore-lighthouse` | Cloudflare Worker service name | +| `DB` | D1 database `buscore-lighthouse` | Primary Lighthouse aggregates and bounded evidence | +| `BUSCORE_LEADS_DB` | D1 database `buscore-leads` | Optional read binding for aggregate inquiry reporting | +| `MANIFEST_R2` | R2 bucket `bus-core` | Stable manifest and versioned release artifacts | +| Cron | `5 0 * * *` | Daily traffic capture, rollup, GitHub snapshot, service probes, and retention | + +The checked-in production consumer endpoint is `https://lighthouse.buscore.ca/report` in `../Agent_Smith/wrangler.toml`. Lighthouse's own `wrangler.toml` does not declare the custom hostname/route, so the consumer configuration is the current repository-visible pointer, not independent deployment proof. A control-plane read is required to verify route attachment; do not infer a `workers.dev` URL from the Worker name. + +`tgc-ops` is not yet safe as an analytics source of truth: its current records include stale future-tense BUS Core telemetry claims and a reversed Lighthouse/buscore-site dependency. Use owning repositories until that separate repository is explicitly reconciled. + +## Credentials and Access Boundaries + +| Name | Where used | Boundary | +|---|---|---| +| `ADMIN_TOKEN` | Lighthouse runtime | Exact-match credential accepted through `X-Admin-Token` | +| `LIGHTHOUSE_ADMIN_TOKEN` | Agent Smith runtime | Consumer-side name for the Lighthouse admin credential | +| `LIGHTHOUSE_REPORT_URL` | Agent Smith runtime | Canonical report endpoint; checked-in production value points to `https://lighthouse.buscore.ca/report` | +| `CF_API_TOKEN` and `CF_ZONE_TAG` | Lighthouse scheduled traffic capture | Cloudflare GraphQL traffic source, not report authentication | +| `GITHUB_REPO` and optional `GITHUB_TOKEN` | Lighthouse scheduled GitHub snapshot/probe configuration | Repository selection and optional API quota; not Lighthouse report authentication | +| `TELEMETRY_RATE_LIMIT_SECRET` | Lighthouse ingestion/release counting | Keys rotating abuse-control identifiers; never a diagnostic credential | + +`ADMIN_TOKEN` is not a read-only credential. The same token authorizes protected report reads and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Possession of the token does not authorize those writes. Do not paste it into commands, logs, chat, files, or screenshots. + +## Diagnostic Access Classes + +### Class 0 — local zero-mutation + +Safe default: + +- Read the authority files and CEO fixtures. +- Inspect `wrangler.toml`, route code, tests, and migrations. +- Run local searches and `git status`. +- Review a supplied alert/report without contacting any service. + +This class changes neither local tracked files nor external state. + +### Class 1 — approved control-plane reads + +Cloudflare deployment, route, binding, cron, log, or D1 metadata reads may establish infrastructure state without intentionally changing Lighthouse application data. They still require explicit authorization, a valid account context, and the least-privileged supported command. A Cloudflare/Wrangler access failure is infrastructure evidence, not a Lighthouse application failure. + +Do not use direct D1 SQL as a fallback merely because a protected report cannot be accessed. SQL can bypass report semantics, expose data outside the contract, or be accidentally executed as a write. + +### Class 2 — approved read-mostly report calls + +These protected views skip the best-effort Cloudflare traffic refresh and read currently persisted evidence: + +| Request | Primary use | +|---|---| +| `GET /report?view=ceo` | First diagnostic read; strict CEO `1.1` facts, windows, source state, limitations, failures, and latest service probes | +| `GET /report?view=source_health` | Ingestion integrity by tracked site; not service-probe truth | +| `GET /report?view=asset` | Stored Phase 2 rollup, GitHub snapshot, service checks, and campaign evidence | +| `GET /report?view=tgc` | Detailed TGC compatibility diagnostics | +| `GET /report?view=monthly` | Historical monthly asset/scoring compatibility surface | + +These are read-mostly, not guaranteed zero-write: if report assembly throws, Lighthouse best-effort increments `metrics_daily.errors` before returning `503 report_unavailable`. + +Agent Smith surfaces inherit downstream behavior: + +| Smith surface | Current behavior and diagnostic boundary | +|---|---| +| Private `/health` Discord command | Fetches the active Lighthouse report and publicly GETs the stable manifest. A manifest miss/read error or CEO report-assembly failure can therefore alter `metrics_daily.errors`. It does not validate cron execution, watch-channel configuration, Discord posting permission/receipt, TGC freshness, or monthly archive success. An unsigned HTTP GET to the Smith Worker is not this command and proves no health. | +| Private `/report` Discord command | In production `ceo_v1`, fetches `GET /report?view=ceo` and renders the CEO business/decision product. It is not a raw diagnostic command and never falls back to legacy. | +| Private `/tgc` Discord command | Fetches the stored-data `view=tgc` compatibility diagnostic. It remains owner/channel gated and is not used for CEO scheduled facts. | + +Missing `DISCORD_WATCH_CHANNEL_ID` skips scheduled posting. Post failures are logged rather than converted into delivery proof, and a monthly archive attempt can occur even after a Discord post failure. An archived snapshot is therefore not proof of Discord delivery. + +### Class 3 — evidence-mutating or explicitly mutating surfaces + +Do not call these during passive diagnosis: + +| Surface | Side effect or risk | +|---|---| +| Bare `GET /report`, `view=fleet`, `view=site` | Performs a best-effort previous-completed-day Cloudflare traffic capture/upsert before report assembly | +| `GET /manifest/core/stable.json` | A missing object or read error attempts a best-effort `metrics_daily.errors` increment; successful GET is non-mutating | +| `HEAD /manifest/core/stable.json` | Does not increment the Lighthouse error counter; it still contacts production and is reserved for an explicitly approved liveness check | +| `GET /update/check` | Can increment qualified update-check aggregates and rate-control state; failures can increment errors | +| `GET /download/latest` | A successful redirect schedules a best-effort increment of `buscore_download_intent_daily.successful_redirects`; the `302` does not prove persistence. Failures attempt a best-effort error increment | +| `GET /releases/:filename` | Can change raw/success/cache/rate/source-credit and qualified-download evidence and transfers artifact content | +| `HEAD /releases/:filename` | Records raw/HEAD artifact truth even though it does not count a full response, source credit, or download | +| `POST /metrics/pageview` and `POST /metrics/event` | Return `204` after body capture, then asynchronously validate, rate-limit, and persist. The HTTP response is not acceptance or persistence proof; invalid pageview bodies can still write dropped-invalid evidence | +| `POST /telemetry/v1/events` | After basic content-type/size checks, mutates product-telemetry rate-control state before parsing the payload. Valid events then persist deduplication/aggregate state and return an event-ID acknowledgement. The receiver is public and has no diagnostic authentication | +| `POST /campaign`, `POST /notes`, `POST /report/snapshot` | Explicit protected writes | +| Scheduled handler, D1 command, migration, secret operation, deployment, release | Operational mutation requiring separate explicit approval | + +The existence of a route in `README.md` or `SOT.md` is not authorization to probe it. + +## Canonical Morning Diagnostic Sequence + +Use this order for a Lighthouse `WATCH`, `ALERT`, unavailable report, or analytics concern: + +1. **Preserve the initiating evidence.** Record the exact message, timestamp, timezone, delivery lane, requested window, and whether it came from Agent Smith `/health`, `/report`, a scheduled Discord message, Cloudflare, or another surface. +2. **Classify the layer before probing.** Separate report preparation, Lighthouse facts, scheduled service probes, source ingestion, Agent Smith presentation, Discord delivery, and Cloudflare access. +3. **Read local authority.** Read `AGENTS.md`, `SOT.md`, this runbook, the relevant changelog entry, and the CEO schema/fixture matching the observed state. If `source_health` or supplied evidence identifies a producer, also read that producer's canonical files listed above. +4. **Confirm authorization and access material.** Use only the approved production endpoint and an owner-approved, non-echoing credential mechanism. Agent Smith's Cloudflare runtime binding is not a user-facing diagnostic credential source. Never reveal the credential. If the endpoint, mechanism, or authorization is unavailable, report `ACCESS_BLOCKED` and continue only with local evidence. +5. **Read the CEO view first only when both production access and a safe credential mechanism are approved.** Fetch exactly `GET https://lighthouse.buscore.ca/report?view=ceo` with the `X-Admin-Token` header without placing the token literal in command arguments, files, logs, chat, or screenshots. Check HTTP status, JSON parse, `report_contract_version`, `metric_definition_version`, `generated_at`, exact windows, `sources`, `details.service_probes`, and `limitations` before interpreting totals. +6. **Narrow to one secondary view only when evidence requires it.** Use `source_health` for producer-ingestion integrity, `asset` for stored probe/GitHub/rollup detail, or `tgc` for the TGC compatibility diagnostic. Do not fan out across every surface. +7. **Check Agent Smith separately.** Confirm configured mode and active lane. Production is checked in as `ceo_v1`. A Smith `/health` response can show its configuration and CEO readiness; it does not independently prove Discord delivery, Lighthouse cron completion, TGC source freshness, or monthly archive success. +8. **Correlate by timestamp and ownership.** Compare the alert time, CEO `generated_at`, each source's `data_through`, probe `checked_at` values, and the applicable complete or partial window. Do not compare partial today with a complete day as though they were equivalent. +9. **Report diagnosis with confidence and gaps.** State what is proven, what is inferred, what is unavailable, whether any diagnostic action could have changed evidence, and the smallest next check requiring approval. + +Do not start by calling bare reports, update checks, download redirects, artifact GET/HEAD routes, POST routes, D1 SQL, or deployment commands. + +## Evidence Map + +| CEO source | Backing evidence | What it means | +|---|---|---| +| `artifact_delivery` | `artifact_traffic_daily` and related bounded delivery aggregates | Worker-observed artifact response evidence; not completed transfer, person, or installation | +| `update_checks` | `metrics_daily` and `release_update_checks_daily` | Qualified release-route requests; not active users or authentic-client proof | +| `product_telemetry` | `buscore_product_events_daily` plus bounded event-ID deduplication | Accepted allowlisted events are aggregated once per event ID; duplicate IDs are acknowledged but not re-counted. The canonical BUS Core client transmits only after local enablement plus disclosure acknowledgement, but this public unauthenticated receiver cannot prove sender provenance or consent. Native posts use `Content-Type: application/json` and `User-Agent: BUS-Core/` as honest transport identity only; the user agent is not authentication, a user/device identifier, or sender-provenance proof, and Lighthouse does not persist it as product identity. No persistent installation identity is stored | +| `buscore_site` | `site_events_raw` plus `buscore_download_intent_daily` | Accepted production BUS Core page views and rate-bounded canonical artifact-click interest from its definition boundary | +| `tgc_site` | `site_events_raw` for `tgc_site` | Consented, allowlisted TGC site events with bounded sanitized detail | +| `voluntary_inquiries` | Aggregate reads from optional `BUSCORE_LEADS_DB` | Voluntary inquiry totals and fixed privacy-safe attribution buckets; never raw lead data | +| `lighthouse_errors` | `metrics_daily.errors` | Best-effort counter updated only by selected route failures: manifest GET, `/update/check`, `/download/latest`, and report assembly. It is not a complete Worker error log: manifest HEAD, artifact-route failures, product-ingest failures, and scheduled-probe failures do not increment it | +| `service_probes` | Latest active rows in `health_checks` | Scheduled liveness evidence for six named surfaces | + +`GET /report?view=source_health` is ingestion-integrity evidence. It reports recent accepted signals and persisted drop counters by tracked site. It is not the service-probe table and must not be used as proof that the manifest, artifact, lead, GitHub, or site endpoints are currently reachable. + +Update-check aggregates include only GET requests whose query contains exactly one each of `current_version`, `channel`, and `first_check`; whose channel and versions are valid and plausible (`current_version >= 1.4.0` and not newer than the selected manifest release); and whose unsuppressed client IP, configured rate-limit secret, and two-per-IP/day gate permit counting. Successful manifest reads that fail any gate—including BUS Core staging's parameterless manifest read—are not counted. + +For producer-side BUS Core product-telemetry truth, use the protected read-only `GET /app/telemetry/status` only with an authorized local BUS Core session and `settings.read` permission. It exposes enabled, pending, acknowledged, rejected, dead-letter, and last-delivery state. Do not use BUS Core `/transparency.report` or the Home “Telemetry Off” card as telemetry-health evidence. A queued retry does not self-wake when `next_attempt_at` arrives; delivery resumes only when a later emit or startup flush starts the worker, so an aging pending item does not by itself prove an ongoing Lighthouse outage. + +Producer-specific comparisons are not one-to-one: + +- BUS Core site emission is suppressed by `dev_mode` or `localStorage.noAnalytics === "1"` and deduplicates same-path page views within three seconds. It has no producer-side origin guard or `test_mode` label, so an unsuppressed local/staged page can attempt the production endpoint even though Lighthouse may reject it. +- TGC emission requires current optional-analytics consent, honors GPC/DNT and `dev_mode`, refuses non-production origins, and can label `test_mode` traffic for report exclusion. A quiet `tgc_site` source can therefore reflect consent/suppression rather than emitter or Lighthouse failure. +- BUS Core's primary `/download/latest` CTA intentionally emits no `download_click`; only exact versioned Lighthouse artifact links qualify. Website click evidence and artifact delivery should not be expected to match. +- Early-access success may emit `early_access_submit_success`, but suppression can prevent it. Managed BUS success emits no dedicated Lighthouse success event. BUS Core lead rows are unique by email and shared across both forms, so row totals are not submission totals. +- A TGC intake endpoint failure emits both `form_submit_failure` and `form_submit_fallback` for one attempt; do not sum them as two failed inquiries. The current TGC form reports category `contact` even when a service is preselected. +- A true return from `navigator.sendBeacon()` proves only browser queue acceptance. Neither it nor a swallowed/fail-soft fetch result proves Lighthouse HTTP acceptance or persistence. + +## Scheduled Service-Probe Truth + +The current scheduled probe set is: + +| Target | Safe scheduled check | Passing boundary | +|---|---|---| +| `site_home` | Public GET of BUS Core home | `2xx` or `3xx` | +| `site_downloads` | Public GET of BUS Core downloads page | `2xx` or `3xx` | +| `manifest` | Public `HEAD /manifest/core/stable.json` | `200` metadata response | +| `release_artifact` | Bound manifest lookup followed by public HEAD of its exact canonical artifact | `200` and positive `Content-Length` | +| `lead_endpoint` | GET-only request to the early-access endpoint | Lighthouse currently accepts `2xx` or `405 Method Not Allowed` and never POSTs a synthetic lead. The current buscore-site route is expected to return `405` to GET; investigate a `2xx` as possible routing drift | +| `github_release` | Public HEAD of the configured repository's latest-release page | `200` or a same-repository non-empty release-tag redirect | + +The Lighthouse daily cron is `5 0 * * *` (00:05 UTC). Agent Smith's checked-in delivery crons are 13:00 UTC daily, 14:00 UTC on Mondays, and 15:00 UTC on the first of each month. These are separate jobs: correlate their timestamps, and do not treat a missing Smith post as proof that the earlier Lighthouse cron failed. Probe rows are bounded to about 90 days. Each Lighthouse probe and scheduled writer is fail-soft; one failure does not prove that the rest of the run failed. + +These are BUS Core-oriented liveness probes. They do not probe `truegoodcraft.ca`, either browser emitter, Cloudflare Web Analytics injection, TGC `/api/intake`, Managed BUS intake, or browser-to-Lighthouse delivery. A passing manifest HEAD proves route/object metadata only, not valid manifest JSON, required fields, CORS, or browser hydration. A passing early-access GET/405 proves only that the route/method boundary responded; it does not exercise Turnstile, KV rate control, D1 lead storage, production POST success, Managed BUS, or TGC intake. + +## Vocabulary and Interpretation + +| Term | Operational meaning | +|---|---| +| `WATCH` | Agent Smith found evidence needing attention or follow-up. It is not, by itself, a current outage. | +| `ALERT` | Agent Smith found current failure evidence requiring action. Identify the exact source/probe and timestamp. | +| `OK` | Agent Smith found no current actionable condition for the report product. Activity may still be zero or absent. | +| `UNAVAILABLE` | The requested report product could not be prepared or validated. It does not authorize fallback or prove every Lighthouse function is down. | +| System Health `OK` / `DEGRADED` | Agent Smith's private `/health` command summary, not a Lighthouse response-contract status. | +| CEO rollout readiness `READY` / `NOT READY` | Agent Smith's configured CEO lane readiness, separate from System Health and active-delivery status. | +| Active report delivery `OK` / `unavailable` | Agent Smith's active-lane fetch/validation result at command time; it is not Discord scheduled-delivery receipt. | +| `ACCESS_BLOCKED` | The diagnostic could not reach an approved endpoint/control plane or lacked approved credentials. This is an operator-access result, not a service-health result. | +| Source `available` | The source query/binding succeeded. Zero may be a real observed zero only when the metric and window support it. | +| Source `unavailable` | The source could not be queried or its binding is absent. Dependent values must be `null`, not zero. | +| Freshness `unknown` / `source_history_missing` | No trustworthy watermark exists yet. This is not the same as a failed current endpoint. | +| Freshness `stale` / `source_data_stale` | The latest stored evidence is older than the contract threshold. It may indicate an emitter, ingestion, schedule, or access issue; narrow the layer. | +| Coverage `partial` | The source does not establish complete coverage for the whole requested window, often because it is sparse or began later. Partial coverage is expected for multiple current sources and is not automatically an error. | + +Additional failure boundaries: + +- HTTP `401 unauthorized` from `/report` proves credential mismatch/missing configuration at that request boundary; it does not prove the Worker is down. +- HTTP `503 report_unavailable` means report assembly threw and may have incremented `metrics_daily.errors`. +- A Cloudflare/Wrangler authorization or account-context error—including the previously observed `7403` during a control-plane read—is classified as access/tooling failure until independent service evidence says otherwise. The number alone is not a Lighthouse application status. +- Producer-side `sendBeacon`/`fetch` behavior, whether apparently queued or failed, does not create delivery proof. Lighthouse acceptance must be established from Lighthouse evidence. +- A missing optional `BUSCORE_LEADS_DB` affects inquiry-dependent fields only; it must not make unrelated Lighthouse sources unavailable. + +## Standard Diagnostic Report + +Every diagnosis should state: + +- Initiating evidence and timestamp/timezone. +- Environment and endpoint used, without secret values. +- Authorization scope and access class. +- Agent Smith mode/lane if relevant. +- Lighthouse HTTP and contract versions if read. +- Exact report windows and whether each is complete. +- Source availability, freshness, coverage, `data_through`, and reason code. +- Latest named service-probe states and `checked_at` times. +- Proven impact, likely layer, and alternative explanations. +- Whether any diagnostic request could have changed evidence. +- Confidence level, remaining unknowns, and the smallest approval-gated next action. + +Never report a WATCH as an outage, an access failure as application failure, a sparse zero as confirmed inactivity, a download response as a person/installation/completed transfer, an update check as an active user, or `source_health` as endpoint liveness. + +## Known Gaps — Do Not Work Around + +- Lighthouse has no least-privilege report-read credential. The current `ADMIN_TOKEN` also authorizes three write routes. +- This repository has no owner-approved, non-echoing production report helper or canonical operator secret-retrieval mechanism. Until one is separately designed and approved, lack of a safe mechanism is `ACCESS_BLOCKED`. +- Lighthouse's `wrangler.toml` does not declare the checked-in consumer hostname, so local configuration alone cannot prove the production route attachment. +- `tgc-ops` analytics/dependency records are stale and cannot yet serve as the trusted cross-repository entry point. +- BUS Core `1.4.2` has an explicit producer-side code/SOT authority conflict: its code emits repeatable `restore_attempted`, `restore_completed`, `import_completed`, and `import_failed` events, and the current Lighthouse contract accepts them, but they remain outside BUS Core's SOT-authorized signal set. Lighthouse acceptance does not resolve producer authority. Treat their presence as known drift, not approval or a new metric definition; use BUS Core's SOT, operations runbook, and changelog as authority pending separate resolution. +- Agent Smith exposes no raw diagnostic command. Its retained diagnostic formatter is test-only; current private `/report` is the active business/decision product. +- Browser producers are fail-soft and provide no end-to-end delivery receipt. +- Agent Smith has no independent Discord receipt ledger; a post attempt, log line, or archived monthly snapshot is not receipt proof. + +Any helper, token split, route/config reconciliation, cross-repository index repair, receipt mechanism, or automated drift check is a separate future change requiring its owning repository's approval and governance bundle. + +## Approval Boundaries + +Separate approval is required before: + +- Any production or external-service request. +- Any Cloudflare control-plane, log, or D1 access. +- Any endpoint in Class 3. +- Any POST, scheduled invocation, direct SQL, migration, secret operation, deployment, release, commit, push, or cross-repository edit. + +When authorization covers only read-only diagnosis, stop before the first action that can mutate evidence or configuration and ask for the narrower additional approval. diff --git a/PHASE2_ANALYTICS_NOTES.md b/PHASE2_ANALYTICS_NOTES.md index e8e1d24..2b2ecb7 100644 --- a/PHASE2_ANALYTICS_NOTES.md +++ b/PHASE2_ANALYTICS_NOTES.md @@ -1,5 +1,7 @@ # Phase 2 Analytics Foundation — Implementation Notes (Lighthouse) +> **Historical implementation record.** This file preserves the Phase 2 delivery context and is not the current operational runbook. For current behavior use `SOT.md`; for access, side effects, and diagnosis use `OPERATIONS.md`. In particular, current scheduled liveness uses non-counted public manifest HEAD, exact canonical artifact HEAD, GET-only lead probing, and the public GitHub latest-release page as defined by the current SOT. + Scope: Phase 2 of `BUS-Core-Analytics-Plan.md`. Lighthouse (data layer) only. No Phase 3, no scoring, no monthly asset brief, no AI, no BUS Core Core changes, no invasive telemetry, no PII, no Agent Smith outbound changes. diff --git a/PHASE3_ANALYTICS_NOTES.md b/PHASE3_ANALYTICS_NOTES.md index 9660222..249ea3d 100644 --- a/PHASE3_ANALYTICS_NOTES.md +++ b/PHASE3_ANALYTICS_NOTES.md @@ -1,5 +1,7 @@ # Phase 3 Analytics — Implementation Notes (Lighthouse + Agent Smith) +> **Historical implementation record.** This file preserves the Phase 3 delivery context and is not the current operational runbook. Use `SOT.md` and `OPERATIONS.md` for current Lighthouse behavior and diagnostics, and Agent Smith's current authority for its active report mode and presentation. `POST /report/snapshot` is a write. Do not infer Agent Smith's active production lane from the historical monthly workflow described here. + Scope: Phase 3 of `BUS-Core-Analytics-Plan.md`. Monthly Asset Brief + deterministic scoring + report archival + operator notes. No Phase 4, no new telemetry, no AI, no BUS Core Core change, no public dashboards, no PII. diff --git a/README.md b/README.md index 4b8c963..592d9c3 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,13 @@ # buscore-lighthouse +## 1.29.3 operations documentation release + +Repository version 1.29.3 adds the canonical operations and diagnostics runbook and a narrow owner-approved documentation-only governance path. It changes no Worker code, route, contract, auth, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior. The deployed Worker remains Lighthouse 1.29.2 (`f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`) with CEO report and metric-definition contract `1.1`; this repository release does not authorize or require a Worker deployment. + +## Operations and diagnostics + +Use [`OPERATIONS.md`](OPERATIONS.md) as the canonical runbook for Lighthouse alerts, analytics diagnosis, endpoint selection, credential boundaries, and request side effects. Read it after `SOT.md` and before contacting production or Cloudflare. `PHASE2_ANALYTICS_NOTES.md` and `PHASE3_ANALYTICS_NOTES.md` are historical implementation records, not current runbooks. + ## 1.29.2 service-probe truth repair Version 1.29.2 keeps the non-persisting lead liveness check on GET but recognizes `405 Method Not Allowed` as a healthy method boundary even when the endpoint omits `Allow`. GitHub release liveness uses the public latest-release page rather than the quota-limited unauthenticated REST API and validates same-repository release-tag redirects. CEO contract `1.1`, stored metrics, report windows, ingestion, auth, retention, and cron cadence are unchanged; no migration or secret change was required. It is deployed as Cloudflare Worker version `f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`. @@ -222,6 +230,8 @@ dev_mode=1; Domain=.buscore.ca; Path=/; Max-Age=31536000; SameSite=Lax; Secure | GET | `/report` | Return protected aggregate report; legacy BUS Core output includes literal `product_telemetry` windows when migration 0013 is available | | GET | `/report?view=ceo` | Return the protected CEO contract and metric-definition version `1.1` with exact windows and per-source availability; no legacy view is changed | +This table documents route behavior; it does not authorize production probing. Some GET and HEAD routes refresh, count, persist, or otherwise alter evidence. Consult `OPERATIONS.md` before using any route diagnostically. + Notes: - Successful `/manifest/core/stable.json` requests are uncounted. A genuine GET miss/error increments `metrics_daily.errors`; a HEAD miss/error does not. - `/download/latest` never increments `downloads` directly. @@ -724,6 +734,10 @@ Required bindings/secrets: - `CF_ZONE_TAG` (required for scheduled Buscore traffic capture) - `TELEMETRY_RATE_LIMIT_SECRET` (required in production for keyed standardized-site and BUS Core product-telemetry minute controls and qualified update/artifact daily controls) - `BUSCORE_LEADS_DB` (optional external D1 read binding for aggregate BUS Core operator reporting and CEO voluntary-inquiry totals) +- `GITHUB_REPO` (optional; defaults to `True-Good-Craft/TGC-BUS-Core` for the scheduled GitHub snapshot and latest-release probe) +- `GITHUB_TOKEN` (optional secret; raises scheduled GitHub API snapshot rate limits and is not required by the public latest-release HEAD probe) + +`ADMIN_TOKEN` is a broad administrative credential, not a read-only token. It protects report reads and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Do not expose its value or treat report-read approval as write approval. No new bindings or secrets are introduced by pageview ingestion. @@ -744,6 +758,10 @@ Traffic capture notes: - Lighthouse validates that the response includes a numeric daily request `count` field; if missing/undefined/non-numeric, the run is treated as failed and the row is skipped. - Bare `/report`, `view=fleet`, and `view=site` perform one best-effort refresh capture for the previous completed UTC day before assembly. Stored-data views, including `view=ceo`, skip that external refresh. +## Provisioning is not diagnosis + +The setup commands below create resources, apply migrations, set secrets, start a local runtime, or deploy code. They are provisioning and development procedures, not passive diagnostic steps. Do not run them during read-only incident diagnosis or against remote resources without explicit approval. + ## Setup ### 1. Prerequisites diff --git a/SOT.md b/SOT.md index 3f96a23..72684f4 100644 --- a/SOT.md +++ b/SOT.md @@ -1,5 +1,11 @@ # Lighthouse — Source of Truth +## Canonical operations and diagnostics — v1.29.3 documentation release + +Version 1.29.3 establishes `OPERATIONS.md` as the canonical runbook, subordinate to this SOT, for choosing Lighthouse diagnostic surfaces, identifying credential and ownership boundaries, classifying evidence side effects, and reporting `ACCESS_BLOCKED` when approved access is unavailable. It also marks the Phase 2, Phase 3, and policy-alignment documents as historical or scoped references rather than current incident authority. + +This is an owner-approved documentation-only governance release. It changes no Worker source code or deployed behavior: no endpoint, response contract, auth, configuration, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior changes. The repository and lockfile version advance because the documented operator workflow changed. No Worker deployment is required or authorized by this release; the deployed runtime remains Lighthouse 1.29.2, Cloudflare Worker version `f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`, with CEO report and metric-definition contract `1.1`. + ## Service-probe truth repair — v1.29.2 deployed Version 1.29.2 removes two false-positive service failures without weakening the checked surfaces. The `lead_endpoint` probe remains a GET-only, non-persisting liveness check and accepts either a successful `2xx` response or `405 Method Not Allowed`; a 405 proves that the route exists and enforces its method boundary even when the response omits an `Allow` header. Lighthouse never submits a synthetic lead. @@ -119,9 +125,25 @@ Additional constraints: - External services may call Lighthouse or consume Lighthouse outputs, but no Lighthouse core feature may require those services to be up. - Proposed features that create hard runtime dependencies on external products are out of scope unless reworked to preserve independent operation. +### Operational Diagnostics Authority + +`OPERATIONS.md` is the canonical access and diagnostic runbook for the shipped behavior in this SOT. It is subordinate to this SOT and may not introduce or authorize a route, credential, query, storage meaning, or status that is not grounded here and in current code. + +Operational access rules: + +- Default diagnosis is local and zero-mutation. Any production, Cloudflare, Discord, GitHub, D1, or other external interaction requires explicit scope approval. +- When an approved endpoint, credential source, account context, or tool authorization is unavailable, the result is `ACCESS_BLOCKED`, not a Lighthouse outage. Agents must not substitute guessed URLs, unrelated credentials, direct D1 SQL, or broader probes. +- Protected `view=ceo`, `view=tgc`, `view=source_health`, `view=asset`, and `view=monthly` skip the best-effort traffic refresh and are the read-mostly report surfaces. They are not guaranteed zero-write because a report-assembly failure best-effort increments `metrics_daily.errors`. +- Bare `/report`, `view=fleet`, and `view=site` perform a best-effort previous-completed-day traffic capture/upsert before assembly. Public manifest GET failures, update checks, download redirects, artifact requests, telemetry submissions, admin POST routes, scheduled work, D1 operations, migrations, deployments, and releases can also mutate evidence or state as specified by their route contracts. +- `HEAD /manifest/core/stable.json` does not increment `metrics_daily.errors`. `HEAD /releases/:filename` still records raw/HEAD artifact truth and therefore is not a zero-mutation diagnostic. +- `view=source_health` is telemetry-ingestion integrity, not endpoint liveness. Scheduled `health_checks` and CEO `details.service_probes` are the service-probe evidence. +- Agent Smith owns WATCH/ALERT/UNAVAILABLE and report-delivery wording. Lighthouse owns facts, availability, exact windows, source state, limitations, and scheduled probe rows. A WATCH is not automatically an outage. +- Known producer-side drift must be reported rather than normalized. BUS Core `1.4.2` emits repeatable `restore_attempted`, `restore_completed`, `import_completed`, and `import_failed` events that the current Lighthouse contract accepts, but those events remain outside BUS Core's SOT-authorized signal set. Lighthouse acceptance does not grant producer authority; BUS Core's SOT governs pending a separate resolution. +- Lighthouse, Agent Smith, BUS Core, buscore-site, and tgc-site remain independent failure domains; one unavailable producer, consumer, optional binding, or presentation layer must not be promoted into an unsupported cross-service diagnosis. + ## 1a. Phase 2 Analytics Foundation (v1.17.0) -Phase 2 of `BUS-Core-Analytics-Plan.md`. Additive, aggregate/operator-only, no PII, no new user telemetry. Lighthouse remains the data layer and still does not post to Discord. See `PHASE2_ANALYTICS_NOTES.md`. +Phase 2 of `BUS-Core-Analytics-Plan.md`. Additive, aggregate/operator-only, no PII, no new user telemetry. Lighthouse remains the data layer and still does not post to Discord. `PHASE2_ANALYTICS_NOTES.md` is a historical implementation record; use this SOT and `OPERATIONS.md` for current behavior and diagnostics. Four additive D1 tables and their scheduled writers: - `daily_rollup` — one aggregate row per completed UTC day. Writer runs in the daily cron for the **previous completed UTC day** (never partial-day). Reuses existing report query helpers; `wqpi = artifact_downloads + attributed_leads` (same definition as the Phase 1 brief). Missing inputs (e.g. no `BUSCORE_LEADS_DB`) are stored `null`, never faked. `return_rate` is stored `null` (a 7-day windowed metric, not an honest single-day value). Idempotent: `INSERT ... ON CONFLICT(day) DO UPDATE`, `day` is PRIMARY KEY. @@ -148,7 +170,7 @@ Configuration additions (both optional): `GITHUB_REPO` (defaults to `True-Good-C ## 1b. Phase 3 Analytics: Monthly Asset Brief, Scoring, Archival (v1.18.0) -Phase 3 of `BUS-Core-Analytics-Plan.md`. Additive, aggregate-only, no PII, no new telemetry, no AI. Lighthouse remains the data/scoring layer and still does not post to Discord (Agent Smith posts and archives). See `PHASE3_ANALYTICS_NOTES.md`. +Phase 3 of `BUS-Core-Analytics-Plan.md`. Additive, aggregate-only, no PII, no new telemetry, no AI. Lighthouse remains the data/scoring layer and still does not post to Discord (Agent Smith posts and archives). `PHASE3_ANALYTICS_NOTES.md` is a historical implementation record; use this SOT and `OPERATIONS.md` for current behavior and diagnostics. Two additive D1 tables (migration `0011_add_phase3_report_and_notes.sql`): - `report_snapshots(id, generated_at, kind, status, wqpi, summary_json, narrative)` — dated archive of each generated brief. Aggregate only, indefinite retention. @@ -437,6 +459,8 @@ Required bindings/secrets used by code: - `CF_ZONE_TAG` — required for the approved daily Buscore traffic capture job. - `TELEMETRY_RATE_LIMIT_SECRET` — required in production; keys scope-separated standardized-site and BUS Core product-telemetry identifiers that rotate by UTC minute and qualified update-check/artifact-request identifiers that use UTC-day buckets. Update-check and artifact-request counting fail closed when this secret is absent. A random per-isolate fallback remains local-development compatibility for product telemetry only, not the production contract. - `BUSCORE_LEADS_DB` — optional external read binding used only for aggregate BUS Core operator reporting and CEO voluntary-inquiry totals; Lighthouse core operation and all unrelated report sources remain available when it is absent. +- `GITHUB_REPO` — optional configured repository slug for the scheduled GitHub snapshot and latest-release probe; defaults to `True-Good-Craft/TGC-BUS-Core` when absent. +- `GITHUB_TOKEN` — optional secret used by the scheduled GitHub API snapshot to raise rate limits. It is not required by the public latest-release HEAD probe; absence degrades unavailable snapshot fields to `null` rather than fabricated values. Not used by current code: @@ -648,6 +672,7 @@ Rules: - Lighthouse must not combine `anon_user_id` with `ip_hash` or `user_agent_hash` into synthetic identity. - Traffic capture uses Cloudflare aggregate analytics only; no raw request logging is introduced outside the documented narrow pageview ingestion path. - `/report` is protected by `X-Admin-Token` exact match to `env.ADMIN_TOKEN`. +- `ADMIN_TOKEN` is a broad administrative credential, not a read-only diagnostic token. The same exact-match credential protects `GET /report` and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Authorization to read a report does not imply authorization to call those writes. ## 8. Explicit Non-Features diff --git a/package-lock.json b/package-lock.json index 7d82d57..14db6a0 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "buscore-lighthouse", - "version": "1.29.2", + "version": "1.29.3", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "buscore-lighthouse", - "version": "1.29.2", + "version": "1.29.3", "license": "ISC", "devDependencies": { "@cloudflare/workers-types": "^4.20260305.0", diff --git a/package.json b/package.json index 3740cdb..64543f7 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "buscore-lighthouse", - "version": "1.29.2", + "version": "1.29.3", "description": "Standalone deterministic metrics worker: manifest proxy + fixed daily counters + protected on-demand reporting.", "scripts": { "dev": "wrangler dev", diff --git a/plan.md b/plan.md index 90d2d4f..f387d3b 100644 --- a/plan.md +++ b/plan.md @@ -4,6 +4,8 @@ Date: 2026-04-10 Scope: Lighthouse repo only Status: Active baseline for future Lighthouse policy-conformance work +This file is a Lighthouse policy-alignment baseline, not an operations or incident-response runbook. Use `OPERATIONS.md` for current access, endpoint side effects, and diagnostic sequence. Nothing in this plan authorizes production access, probing, writes, migrations, deployment, or cross-repository changes. + ## Mission Constraints - Preserve BUS Core as the grandfathered `legacy_hybrid` exception. From 16bd4176b732cebdfe4e238f4a498670f0c30b64 Mon Sep 17 00:00:00 2001 From: Jamie Date: Wed, 26 Aug 2026 18:53:27 -0400 Subject: [PATCH 2/2] Document Cloudflare preview publication path --- AGENTS.md | 2 ++ CHANGELOG.md | 3 ++- OPERATIONS.md | 19 ++++++++++++++++++- README.md | 4 ++-- SOT.md | 5 +++-- 5 files changed, 27 insertions(+), 6 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index a24602a..10c4d72 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -28,6 +28,7 @@ Authority sources in descending order: ## Owner Approval and Operational Safety - Do not commit unless Jamie/the user explicitly approves. +- Do not push or merge unless Jamie/the user explicitly approves. This repository is connected to Cloudflare Workers Builds: the 1.29.3 review-branch push uploaded a preview Worker version, and a push to the Cloudflare-configured production branch may promote an active deployment independently of the checked-in GitHub Actions gate. - Do not deploy, run `wrangler deploy`, apply D1 migrations, rotate secrets, perform destructive operations, or publish releases unless Jamie/the user explicitly approves. - When code depends on a D1 schema change, a migration requires explicit approval and remote verification before any Worker deployment. - Do not print or commit secret values. @@ -49,6 +50,7 @@ Operational constraints: - `WATCH` is not synonymous with outage; Agent Smith owns WATCH/ALERT/UNAVAILABLE wording, while Lighthouse owns facts and availability. - The current `ADMIN_TOKEN` is broad: it protects report reads and the mutating `POST /campaign`, `POST /notes`, and `POST /report/snapshot` routes. Possession does not authorize writes. - Many GET/HEAD surfaces change evidence. Bare/fleet/site reports refresh stored traffic; report failures can increment errors; artifact HEAD records raw/HEAD truth; update, redirect, artifact, telemetry, and admin-write routes are not passive probes. +- Git publication is not zero-mutation. The 1.29.3 review-branch push triggered Cloudflare Workers Builds, uploaded a Worker preview version, and created preview URLs; future non-production behavior depends on external integration settings. The checked-in workflow's release gate does not govern that separate integration; treat a merge or production-branch push as potentially production-deploying until the Cloudflare build settings are explicitly verified. - Lighthouse, Agent Smith, BUS Core, buscore-site, and tgc-site are separate failure domains. Do not infer a cross-service outage from one unavailable layer. --- diff --git a/CHANGELOG.md b/CHANGELOG.md index fde541a..3aa9a59 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,10 +6,11 @@ - Routed agents and operators through the SOT and runbook before live access, with an explicit `ACCESS_BLOCKED` outcome when approved endpoint, credential, account, or tool access is unavailable. - Documented that the shared `ADMIN_TOKEN` is broad and protects both report reads and administrative writes. - Classified report, manifest, update, redirect, artifact, telemetry, scheduled, D1, deployment, and administrative surfaces by their possible evidence/state side effects. +- Recorded the separately connected Cloudflare Workers Builds path: publishing the review branch automatically uploaded preview Worker version `33f4db25-9faf-435d-a83e-83d7d1c17eac` and created version/branch preview URLs. The Cloudflare check classified the upload as a preview and did not report an active-production promotion; active-production state was not independently control-plane verified. The checked-in GitHub Actions release gate does not govern this integration, so a merge remains potentially production-deploying until the Cloudflare production-branch and deploy-command settings are explicitly verified or the owner accepts that consequence. - Marked the Phase 2, Phase 3, and policy-alignment documents as scoped historical or policy references rather than current operations authority. - Reconciled cross-repository producer baselines to released BUS Core `1.4.2` and the merged buscore-site `1.4.2` release sync while retaining Agent Smith `0.25.2` as shipped authority; recorded the unresolved BUS Core restore/import signal authority drift rather than legitimizing it. - Added the explicit owner-approved documentation-only bundle exception: code may be omitted only when runtime and contract behavior do not change; SOT, changelog, package, and lockfile remain mandatory, and dummy code changes are forbidden. -- No Worker code, endpoint, response contract, auth, configuration, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior changed. The deployed Worker remains Lighthouse 1.29.2 (`f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`) with CEO contract `1.1`; version 1.29.3 identifies this repository documentation release. +- No Worker code, endpoint, response contract, auth, configuration, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior changed. The last production deployment recorded in repository history is Lighthouse 1.29.2 (`f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`) with CEO contract `1.1`; version 1.29.3 identifies this repository documentation release. The automatic preview upload above is external publication evidence, not proof of current active-production state. ## [1.29.2] - 2026-08-10 diff --git a/OPERATIONS.md b/OPERATIONS.md index da15d5e..c0eb18a 100644 --- a/OPERATIONS.md +++ b/OPERATIONS.md @@ -2,9 +2,10 @@ - Status: current operational runbook - Scope: Lighthouse analytics access, evidence interpretation, and incident diagnosis -- Runtime baseline: deployed Lighthouse Worker `1.29.2`; repository documentation release `1.29.3`; CEO report and metric-definition contract `1.1` +- Runtime baseline: last production deployment recorded in repository history is Lighthouse Worker `1.29.2`; repository documentation release `1.29.3`; CEO report and metric-definition contract `1.1`; current active-production state not independently control-plane verified - Last reconciled: 2026-08-26 - Live Lighthouse endpoint and Cloudflare control-plane verification during this documentation change: not performed +- Repository-publication evidence: the PR branch push triggered Cloudflare Workers Builds, whose check classified uploaded version `33f4db25-9faf-435d-a83e-83d7d1c17eac` as a preview, reported version/branch preview URLs, and did not report an active-production promotion; this is not independent proof of current active-production state ## Authority and Purpose @@ -27,6 +28,7 @@ The default diagnostic posture is local and zero-mutation: - Read repository documentation, contracts, fixtures, configuration names, and code. - Inspect local Git state without changing it. - Do not contact production, Cloudflare, Discord, GitHub, Airtable, or another service unless the user has authorized that scope. +- Do not treat `git push` as local-only. This repository's Cloudflare Workers Builds integration uploaded a preview Worker version for the 1.29.3 review-branch push; future branch and production-branch behavior depends on external build settings. - Do not run migrations, deployments, scheduled handlers, retention jobs, report snapshots, notes, campaign writes, telemetry submissions, or release downloads during passive diagnosis. - Never print, persist, or commit secret values, raw lead data, identifiers, IP material, or other private payloads. @@ -82,6 +84,17 @@ The following identifiers are checked into `wrangler.toml`: The checked-in production consumer endpoint is `https://lighthouse.buscore.ca/report` in `../Agent_Smith/wrangler.toml`. Lighthouse's own `wrangler.toml` does not declare the custom hostname/route, so the consumer configuration is the current repository-visible pointer, not independent deployment proof. A control-plane read is required to verify route attachment; do not infer a `workers.dev` URL from the Worker name. +## Repository Publication and Deployment Paths + +| Path | Proven or repository-visible behavior | Operational boundary | +|---|---|---| +| `.github/workflows/governance.yml` | Runs dependency installation, typechecking, and the full test suite for every PR and `main` push | Validation only; it does not deploy | +| `.github/workflows/deploy.yml` | Triggers on `main` push or manual dispatch; both its validation gate and downstream deploy run only for manual dispatch or the workflow's explicit commit-message release opt-in | This release gate governs only the checked-in deploy workflow | +| Cloudflare Workers Builds, non-production branch | The 1.29.3 review-branch push uploaded a Worker version and produced version/branch preview URLs; the check classified it as a preview and did not report an active-production promotion | This is external preview state created by `git push`, not independent proof of current active-production state. Do not call a preview URL during passive diagnosis because its effective bindings and request side effects have not been separately verified | +| Cloudflare Workers Builds, configured production branch | Cloudflare-managed production branch and deploy-command settings are outside this repository | A merge or production-branch push is potentially active-production-deploying even if the GitHub Actions deploy job skips; require explicit owner approval and either control-plane verification or explicit acceptance of that consequence | + +The absence of Cloudflare deployment secrets in GitHub Actions does not disable Cloudflare Workers Builds, which uses separately managed integration credentials. Do not describe the checked-in workflow as the sole deployment control. + `tgc-ops` is not yet safe as an analytics source of truth: its current records include stale future-tense BUS Core telemetry claims and a reversed Lighthouse/buscore-site dependency. Use owning repositories until that separate repository is explicitly reconciled. ## Credentials and Access Boundaries @@ -156,6 +169,8 @@ Do not call these during passive diagnosis: | `POST /metrics/pageview` and `POST /metrics/event` | Return `204` after body capture, then asynchronously validate, rate-limit, and persist. The HTTP response is not acceptance or persistence proof; invalid pageview bodies can still write dropped-invalid evidence | | `POST /telemetry/v1/events` | After basic content-type/size checks, mutates product-telemetry rate-control state before parsing the payload. Valid events then persist deduplication/aggregate state and return an event-ID acknowledgement. The receiver is public and has no diagnostic authentication | | `POST /campaign`, `POST /notes`, `POST /report/snapshot` | Explicit protected writes | +| `git push` to a non-production branch | Connected Cloudflare Workers Builds can upload a preview Worker version and create preview URLs, as observed for the 1.29.3 review branch; the check did not report an active-production promotion, but active-production state was not independently verified | +| Merge or push to the Cloudflare-configured production branch | May run the integration's externally configured deploy command and promote the active production deployment independently of the checked-in GitHub Actions gate | | Scheduled handler, D1 command, migration, secret operation, deployment, release | Operational mutation requiring separate explicit approval | The existence of a route in `README.md` or `SOT.md` is not authorization to probe it. @@ -270,6 +285,7 @@ Never report a WATCH as an outage, an access failure as application failure, a s - Lighthouse has no least-privilege report-read credential. The current `ADMIN_TOKEN` also authorizes three write routes. - This repository has no owner-approved, non-echoing production report helper or canonical operator secret-retrieval mechanism. Until one is separately designed and approved, lack of a safe mechanism is `ACCESS_BLOCKED`. - Lighthouse's `wrangler.toml` does not declare the checked-in consumer hostname, so local configuration alone cannot prove the production route attachment. +- Cloudflare Workers Builds production-branch, build-command, and deploy-command settings are not checked into this repository and were not control-plane verified for this release. The observed branch-preview upload proves the integration is active, so the GitHub Actions gate cannot be used as sole merge-safety evidence. - `tgc-ops` analytics/dependency records are stale and cannot yet serve as the trusted cross-repository entry point. - BUS Core `1.4.2` has an explicit producer-side code/SOT authority conflict: its code emits repeatable `restore_attempted`, `restore_completed`, `import_completed`, and `import_failed` events, and the current Lighthouse contract accepts them, but they remain outside BUS Core's SOT-authorized signal set. Lighthouse acceptance does not resolve producer authority. Treat their presence as known drift, not approval or a new metric definition; use BUS Core's SOT, operations runbook, and changelog as authority pending separate resolution. - Agent Smith exposes no raw diagnostic command. Its retained diagnostic formatter is test-only; current private `/report` is the active business/decision product. @@ -285,6 +301,7 @@ Separate approval is required before: - Any production or external-service request. - Any Cloudflare control-plane, log, or D1 access. - Any endpoint in Class 3. +- Any branch push or PR merge; a branch push can create Cloudflare preview state and did so for the 1.29.3 review branch, while a merge is potentially active-production-deploying until the integration settings are verified. - Any POST, scheduled invocation, direct SQL, migration, secret operation, deployment, release, commit, push, or cross-repository edit. When authorization covers only read-only diagnosis, stop before the first action that can mutate evidence or configuration and ask for the narrower additional approval. diff --git a/README.md b/README.md index 592d9c3..1b19eb9 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ## 1.29.3 operations documentation release -Repository version 1.29.3 adds the canonical operations and diagnostics runbook and a narrow owner-approved documentation-only governance path. It changes no Worker code, route, contract, auth, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior. The deployed Worker remains Lighthouse 1.29.2 (`f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`) with CEO report and metric-definition contract `1.1`; this repository release does not authorize or require a Worker deployment. +Repository version 1.29.3 adds the canonical operations and diagnostics runbook and a narrow owner-approved documentation-only governance path. It changes no Worker code, route, contract, auth, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior. The last production deployment recorded in repository history is Lighthouse 1.29.2 (`f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`) with CEO report and metric-definition contract `1.1`; active-production state was not independently control-plane verified during this documentation release. This release does not authorize or require an active-production promotion. Publishing the review branch did cause the separately connected Cloudflare Workers Builds integration to upload a version that its check classified as a preview and for which it reported version/branch preview URLs, as recorded in `CHANGELOG.md` and `OPERATIONS.md`. ## Operations and diagnostics @@ -36,7 +36,7 @@ Production Worker 1.29.0 narrows the consented `site_key=tgc_site` lane to aggre The server accepts page views, selected commercial/contact/outbound interest, form start/attempt/outcome, and sanitized errors. Version 1.29.1 accepts a producer-supplied coarse viewport label or, for rolling compatibility, exact lowercase `WIDTHxHEIGHT`; exact dimensions are immediately normalized by width to `small` below 768, `medium` from 768 through 1199, or `large` from 1200 upward, and only that bucket is stored. Event-specific value sanitization turns recognized form, error, and outbound values into bounded categories, unrecognized non-empty values into `other`, and discards absent/blank values or values on the remaining TGC events. Lighthouse discards visitor/session fields and rejects the superseded lifecycle, field-level form, scroll/engagement/section, and first-party web-vital event families. The existing TGC report shape remains available for rollback and historical rows; Smith's CEO lane uses page views and voluntary inquiries only. Raw TGC events are retained for 90 days; other standardized-site raw events retain the general 30-day policy, and rotating keyed rate identifiers are retained for two days. See `TGC_SITE_ANALYTICS_POLICY.md` for the current boundaries. -Production deploys use the gated `.github/workflows/deploy.yml`: full tests first, then Wrangler only on manual dispatch or an explicitly marked release merge. Migrations remain separate and are never implied by deployment. +The repository-visible `.github/workflows/governance.yml` runs `npm ci`, typechecking, and the full test suite for every PR and `main` push. Separately, `.github/workflows/deploy.yml` triggers on `main` pushes and manual dispatch, but its validation gate and downstream Wrangler deployment run only for manual dispatch or an explicitly marked release commit. Neither workflow is the only Cloudflare path: the separately connected Cloudflare Workers Builds integration uploaded a version from the 1.29.3 review branch that its check classified as a preview, and it can deploy from its configured production branch. Those production-branch and deploy-command settings are not checked into this repository and were not read from the control plane during the 1.29.3 documentation release. Treat a merge to `main` as potentially production-deploying until those settings are explicitly verified or the owner accepts that consequence. Migrations remain separate and are never implied by deployment. BUS Core artifact delivery and demand semantics are defined in `BUS_CORE_TRAFFIC_TRUTH.md`. Version 1.25.0 keeps downloads public while separating raw Worker traffic, successful artifact responses, privacy-preserving daily client-network buckets, probable-human intent proxies, confirmed product telemetry, and leads. Migration `0014_add_artifact_traffic_truth.sql` was applied remotely before the 2026-07-18 v1.25.0 deployment. diff --git a/SOT.md b/SOT.md index 72684f4..351ae1c 100644 --- a/SOT.md +++ b/SOT.md @@ -4,7 +4,7 @@ Version 1.29.3 establishes `OPERATIONS.md` as the canonical runbook, subordinate to this SOT, for choosing Lighthouse diagnostic surfaces, identifying credential and ownership boundaries, classifying evidence side effects, and reporting `ACCESS_BLOCKED` when approved access is unavailable. It also marks the Phase 2, Phase 3, and policy-alignment documents as historical or scoped references rather than current incident authority. -This is an owner-approved documentation-only governance release. It changes no Worker source code or deployed behavior: no endpoint, response contract, auth, configuration, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior changes. The repository and lockfile version advance because the documented operator workflow changed. No Worker deployment is required or authorized by this release; the deployed runtime remains Lighthouse 1.29.2, Cloudflare Worker version `f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`, with CEO report and metric-definition contract `1.1`. +This is an owner-approved documentation-only governance release. It changes no Worker source code or deployed behavior: no endpoint, response contract, auth, configuration, binding, storage, retention, schedule, integration, migration, secret, or deployment behavior changes. The repository and lockfile version advance because the documented operator workflow changed. No active-production Worker promotion is required or authorized by this release. The last production deployment recorded in repository history is Lighthouse 1.29.2, Cloudflare Worker version `f07d4af2-a8d6-4df6-adfa-aad7eb9f578d`, with CEO report and metric-definition contract `1.1`; active-production state was not independently control-plane verified during this documentation release. Publishing the review branch caused the separately connected Cloudflare Workers Builds integration to upload version `33f4db25-9faf-435d-a83e-83d7d1c17eac`; its check classified the upload as a preview, reported version/branch preview URLs, and did not report an active-production promotion. That check is not proof of current active-production state and does not authorize a merge. ## Service-probe truth repair — v1.29.2 deployed @@ -58,7 +58,7 @@ The server enforces the TGC event allowlist, production-origin match, path/URL c Lighthouse remains the source of truth. Agent Smith may present this protected aggregate view through `/tgc`. Airtable may receive curated periodic KPI/campaign/content/experiment summaries later, but must not receive raw events or stable identifiers. -Worker 1.29.1 was deployed on `2026-08-09T16:41:20.004814Z` as Cloudflare Version ID `ee320e1a-9ceb-4d88-a848-fd7ae0e9e3bc`; it supersedes the earlier 1.29.0 deployment. The repository's normal production path is `.github/workflows/deploy.yml`, which runs the complete typecheck/test gate and deploys only on manual dispatch or an explicitly marked main-branch commit. Because that repository currently has no Cloudflare credential secrets configured, the owner-approved 1.29.1 release used authenticated local Wrangler after the same gate; no secret value changed. Wrangler deployment preserves separately provisioned Worker secrets. Schema migrations remain a separate, explicit operation. Migration 0015 was remotely verified before Worker 1.27.0 deployment; 1.29.0 and 1.29.1 required no migration. +Worker 1.29.1 was deployed on `2026-08-09T16:41:20.004814Z` as Cloudflare Version ID `ee320e1a-9ceb-4d88-a848-fd7ae0e9e3bc`; it supersedes the earlier 1.29.0 deployment. The repository-visible `.github/workflows/governance.yml` runs dependency installation, typechecking, and the full test suite for every PR and `main` push. Separately, `.github/workflows/deploy.yml` triggers on `main` pushes and manual dispatch, but its complete typecheck/test gate and downstream deploy run only for manual dispatch or an explicitly marked main-branch commit. A separate Cloudflare Workers Builds Git integration also exists outside those workflows and uses Cloudflare-managed build settings and credentials: the 1.29.3 review-branch push was observed on 2026-08-26 to upload a version that its check classified as a preview and for which it reported version/branch preview URLs; the check did not report an active-production promotion, and active-production state was not independently control-plane verified. A push to the integration's configured production branch may run its configured deploy command and promote the active deployment. The production branch and deploy command are not checked into this repository and were not control-plane verified for the 1.29.3 documentation release, so the GitHub Actions release gate must not be treated as the sole production control. The owner-approved 1.29.1 release used authenticated local Wrangler after the same test gate; no secret value changed. Wrangler deployment preserves separately provisioned Worker secrets. Schema migrations remain a separate, explicit operation. Migration 0015 was remotely verified before Worker 1.27.0 deployment; 1.29.0 and 1.29.1 required no migration. ## BUS Core traffic truth and bounded delivery work — v1.25.0 deployed @@ -138,6 +138,7 @@ Operational access rules: - `HEAD /manifest/core/stable.json` does not increment `metrics_daily.errors`. `HEAD /releases/:filename` still records raw/HEAD artifact truth and therefore is not a zero-mutation diagnostic. - `view=source_health` is telemetry-ingestion integrity, not endpoint liveness. Scheduled `health_checks` and CEO `details.service_probes` are the service-probe evidence. - Agent Smith owns WATCH/ALERT/UNAVAILABLE and report-delivery wording. Lighthouse owns facts, availability, exact windows, source state, limitations, and scheduled probe rows. A WATCH is not automatically an outage. +- Repository publication is operational state, not a zero-mutation diagnostic. Connected Cloudflare Workers Builds uploaded a non-production preview version from the 1.29.3 review branch independently of `.github/workflows/deploy.yml`; future branch behavior depends on external integration settings. Because its production branch and deploy command are control-plane configuration rather than repository state, a merge or production-branch push must be treated as potentially active-production-deploying until explicitly verified and approved. - Known producer-side drift must be reported rather than normalized. BUS Core `1.4.2` emits repeatable `restore_attempted`, `restore_completed`, `import_completed`, and `import_failed` events that the current Lighthouse contract accepts, but those events remain outside BUS Core's SOT-authorized signal set. Lighthouse acceptance does not grant producer authority; BUS Core's SOT governs pending a separate resolution. - Lighthouse, Agent Smith, BUS Core, buscore-site, and tgc-site remain independent failure domains; one unavailable producer, consumer, optional binding, or presentation layer must not be promoted into an unsupported cross-service diagnosis.