ci: create the orgs-smoke user on preview automatically by mthines · Pull Request #702 · mthines/lorekit

mthines · 2026-10-18T12:07:01Z

Why

The orgs REST smoke on preview has been skipped: sign-in fails with invalid_credentials. rebuild-preview.yml runs supabase db reset --linked every week, which wipes the fixed smoke user, and the user only came back if someone re-seeded it by hand.

What changed

smoke-preview in deploy.yml and preview.yml gets a new step, Ensure the fixed smoke user exists on preview, right before the JWT mint. It runs the existing idempotent seed-smoke-user.mjs, which signs in first and only uses the Auth admin API when that fails: the user is missing, or its password has drifted from LOREKIT_SMOKE_PASSWORD.
preview.yml's seed job (/preview --seed) gets the same step before it seeds demo data. It runs in parallel with smoke, so it can't count on smoke's step having run first.
rebuild-preview.yml: re-seeds the user right after db reset, replacing the old 're-seed out-of-band' notice.
The service-role key is the preview environment's SUPABASE_SERVICE_ROLE_KEY (already set there) and is set only on those steps. Each step refuses outright when the project ref is production's. Production is never seeded by CI.
Seeding is best-effort, like the mint: a failure warns, and the orgs suite skips that run.
Docs: docs/deployment.md (the orgs-smoke secrets table now lists SUPABASE_SERVICE_ROLE_KEY; rebuild, 'Seed the orgs-smoke user' and the demo-data reseed), the preview.yml secrets header, and the headers of seed-smoke-user.mjs and seed-preview.mjs.

This reverses the earlier 'never seed in CI' choice, at the maintainer's request: with a weekly reset, a manual seed can't keep the suite running.

How to verify

actionlint on all three workflows: no new findings (only the two SC2129 style notes already on main).
After merge, the deploy run's smoke-preview should log seeding smoke user … or ok — smoke user already seeded, and the orgs suite should run instead of skipping.

User-facing docs/llms.txt: none applied, because this is CI wiring and contributor docs only.

rebuild-preview.yml's db reset wipes the fixed smoke user every week, and the orgs REST smoke then skipped until someone re-seeded it from an admin shell. smoke-preview (deploy.yml, preview.yml) now runs the idempotent seed-smoke-user.mjs before its JWT mint, and rebuild-preview re-runs it right after the reset. The service-role key is the preview environment's, set on that one step, and the step refuses the production ref.

dash0-dev · 2026-10-18T12:08:15Z

Darkplane Automerge did not approve this pull request. It scored 50.49 of 100, and 75 was required.

🟡 Blast radius scored 66.
The change only touches CI workflows (deploy.yml, preview.yml, rebuild-preview.yml), docs and script comments. It adds best-effort, idempotent steps with a production-ref guard, so a revert undoes it. The cost is that the preview SUPABASE_SERVICE_ROLE_KEY now reaches steps in preview.yml that run PR-head code, though only for authorized commenters and the preview environment. I did not see the truncated rest of the docs/deployment.md diff.
🟡 Evidence quality scored 50.
The change is CI workflow wiring, scripts and docs, and no new tests cover it. Relevant checks pass, including the Preview-seed dataset/SQL builders (unit) test that covers seed-preview.mjs, and many other checks are neutral. The only review signal is the author's own automated reviewer, whose findings were addressed. Actual behaviour is untested: the PR says to verify it after merge, and its actionlint run isn't shown in the checks.
🟡 Change footprint scored 35.
The same "ensure smoke user" step is copied into deploy.yml, preview.yml and rebuild-preview.yml, and each copy passes SUPABASE_SERVICE_ROLE_KEY to a CI step. That reverses a documented policy of keeping the key out of CI, though a production-ref guard and a preview-environment scope limit the exposure. The scripts seed-smoke-user.mjs and seed-preview.mjs only change comments or messages, and the rest is docs. The change is small and repetitive but hand-written, and it touches deploy and secret-handling workflows. I read the diff for seed-smoke-user.mjs but not the truncated part of the docs diff or the seed-preview.mjs diff.

_{An AI evaluator produced these scores from the diff, the repository history, and the pull request text. The pull request text is author-controlled, so treat the scores as signals rather than proof. Reviewed for commit f4b7438.}

mthines · 2026-10-18T13:20:39Z

✅ No issues found

The new commit fixes the last open thread: the demo-data reseed note and seed-preview.mjs's header now say CI re-creates the smoke user. Both claims match the workflows. No new findings.

Checked: 2 of 2 changed files read · no possible issues found

Progress: open review threads 1 → 1 → 0 across the last 3 reviews

What this change reaches — 1 changed file imported by 1 file

flowchart LR
  subgraph pr["Changed in this PR"]
    m1["scripts/seed/seed-preview.mjs<br/>file changed"]:::changed
  end
  m1i1["scripts/seed/seed-preview.test.mjs<br/>imports this file"]:::imports
  m1 -.-> m1i1
  classDef changed fill:#eef2ff,stroke:#6366f1,color:#1e1b4b
  classDef ok fill:#e7f6ec,stroke:#16a34a,color:#14532d
  classDef partial fill:#fef9c3,stroke:#ca8a04,color:#713f12
  classDef unknown fill:#f3f4f6,stroke:#9ca3af,stroke-dasharray:4 3,color:#374151
  classDef bad fill:#fde8e8,stroke:#dc2626,color:#7f1d1d
  classDef warn fill:#fff4e5,stroke:#d97706,color:#78350f
  classDef imports fill:#f8fafc,stroke:#64748b,stroke-dasharray:4 3,color:#1e293b

scripts/seed/seed-preview.mjs — changed file · imported by 1 file, not traced

Review details

Gate	Status	Details
Description vs. code	✅	Description matches the diff: smoke-user seed steps in three workflows plus the doc updates.
Prior review feedback	✅	Earlier review comments are resolved.
Documentation	✅	deployment.md, the preview.yml header and both script headers describe the new seeding.
Self-review signals	✅	No debug logs, leftover TODOs, or unreviewed stubs.
Code review	✅	The multi-lens review found no blocking issues.

Found

Quality — produced 0 → posted inline 0 · cleared 0 · carried forward 0 · deferred 0 · below-bar 0
Standards — ran — delta files (docs/deployment.md, scripts/seed/seed-preview.mjs): 0 findings

Run

incremental · 11 lines in delta · tier standard · depth checkout
Integrations — none
CI — still pending: Preview-seed dataset/SQL builders (unit), Typecheck, Test & Lint (affected), Darkplane auto-approval · 3 passed
Memories — no relevance rules or lessons consulted

^{Nothing to report — optimality (skipped), measurability (skipped), severity, 0 files skipped.}

^{pr-reviewer · commit f4b7438 · incremental review, delta since 1425405 · how these findings are produced · updated Oct 18, 2026 1:27pm UTC}

…e-role key for the orgs smoke

github-actions · 2026-10-18T13:28:02Z

✅ Preview deploy succeeded

Branch: ci/auto-seed-preview-smoke-user @ c7d6600

🔗 Open preview → https://lorekit-kqn0bqwui-mads-thines-projects.vercel.app

Step	Result
API (Supabase edge functions + migrations)	✅ success
Web (Vercel preview)	✅ success
Smoke tests	✅ success
Seed demo data	❌ failure

Endpoints:

View workflow run

mthines commented Oct 18, 2026

View reviewed changes

Comment thread docs/deployment.md

ci: ensure the smoke user before /preview --seed; document the servic…

1425405

…e-role key for the orgs smoke

mthines commented Oct 18, 2026

View reviewed changes

Comment thread docs/deployment.md

docs: the demo-data reseed no longer waits on a manual smoke-user seed

f4b7438

mthines merged commit 6c4913a into main Oct 18, 2026
20 checks passed

mthines deleted the ci/auto-seed-preview-smoke-user branch October 18, 2026 13:28

mthines mentioned this pull request Oct 18, 2026

fix(smoke): seed the preview smoke user with an sb_secret_ service-role key #704

Merged

Navigation Menu

Repository navigation

ci: create the orgs-smoke user on preview automatically - #702

ci: create the orgs-smoke user on preview automatically#702
mthines merged 3 commits into
mainfrom
ci/auto-seed-preview-smoke-user

mthines commented Oct 18, 2026 •

edited

Loading

Uh oh!

dash0-dev Bot commented Oct 18, 2026 •

edited

Loading

Uh oh!

mthines commented Oct 18, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mthines commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

How to verify

Uh oh!

dash0-dev Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mthines commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ No issues found

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 18, 2026

✅ Preview deploy succeeded

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mthines commented Oct 18, 2026 •

edited

Loading

dash0-dev Bot commented Oct 18, 2026 •

edited

Loading

mthines commented Oct 18, 2026 •

edited

Loading