Skip to content

Create every per-Project secret in the cluster: generated keys, database credentials and OIDC clients #444

Open
8 of 10 issues completed

Description

@ExtraToast

Problem Statement

Moving a Project onto the estate path today means a person running estate/bootstrap/cluster.sh with a Vault token, once per Project and again after every restore. The script generates session and HMAC keys, creates a fixed Postgres login per Project and writes it to Vault, and copies each OIDC client secret into auth-api's document and checks the copies match. Every one of those steps is knowable from the project files, yet none of it happens in the cluster, so each move is blocked on a manual run and each new stage makes the script longer.

Solution

The render creates every per-Project secret the project files make knowable, in the cluster, before the release that needs it:

  1. Generated keys. A key declared on a kv grant may say how to generate it. A per-Project seed Job writes any generated key that is missing, under the Project's own Vault prefix, and never overwrites one.
  2. Database credentials. An edge to a Postgres provider with credentials: gets its database and a Vault database role. The Process receives rotating credentials as environment through VSO (Deliver every secret through VSO, as env or file #378), and no fixed login is created.
  3. OIDC clients. An edge to auth declares the client it signs in with. The client secret is generated under the consuming Project's prefix, and auth registers every declared client and reads its secret from there.

After this, a Project moves with a project file and a release. cluster.sh keeps only what is once per cluster: Flux's deploy key, the render pull secret, the policy job's role and the bootstrap CRDs.

User Stories

  1. As an Application author, I want to declare that a session key is generated, so that I never create it by hand.
  2. As an Application author, I want a generated key to keep its value across releases, restarts and re-renders, so that sessions survive a deploy.
  3. As an Application author, I want to say how long a generated key is and how it is encoded, so that it fits what my app parses.
  4. As an operator, I want a generated key that already exists never to be overwritten, so that a seed run can never log everyone out or break a signature.
  5. As an operator, I want the seed Job to be able to create keys only under its own Project's prefix, so that a compromised seed cannot touch another Project's secrets.
  6. As an operator, I want the seed to run before the release that needs its keys, so that a Process never starts without them.
  7. As an operator, I want a seed failure to hold the release, as a failed prepare step does, so that a half-seeded Project never serves.
  8. As an Application author, I want my database created for me from my edge to Postgres, so that I declare it once and never run SQL by hand.
  9. As an Application author, I want rotating database credentials as environment variables, so that my app needs no Vault client.
  10. As an operator, I want no fixed database password anywhere, so that a leaked credential expires on its own.
  11. As an operator, I want an existing database adopted rather than recreated, so that moving a Project off the old path keeps its data and its owner role.
  12. As an Application author, I want to declare my OIDC client on my edge to auth, so that signing in works on the first release.
  13. As an operator, I want one copy of each client secret, so that auth and the app can never disagree.
  14. As auth's maintainer, I want auth to learn every declared client from the render, so that adding an app needs no change to auth's own files.
  15. As an operator, I want a restored Vault without these secrets to get them back on the next release, so that a restore needs no per-Project script.
  16. As an operator, I want composition to refuse a generated key that another Project also writes, so that two seeds never fight over one value.
  17. As an operator, I want cluster.sh reduced to the once-per-cluster stages, so that I run it when building a cluster and never when moving a Project.

Implementation Decisions

  • Generated keys live on the grant. A kv grant's keys entry may be an object with generate: {bytes, encoding}, where encoding is hex or base64. A generated key must be under the Project's own prefix, secret/data/<project>-system/; anywhere else is refused. A plain string key stays as today.
  • One seed Job per Project, rendered like a prepare Process and ordered before every other prepare step and before any new version starts. Its Vault policy allows create on the Project's prefix and read there, and nothing else. It writes a key only when it is absent; a present key, of any value, is left alone. It needs no image of the Project's own.
  • Database credentials finish Deliver every secret through VSO, as env or file #378. The render already derives a Vault database role per consuming edge. A Job owned by the Postgres provider's Project creates the consumer's database and owner role when absent, and adopts an existing database by name. VSO's dynamic secret delivers the credential as environment.
  • The OIDC client is declared on the edge to auth, with its redirect path. Its secret is a generated key under the consumer's prefix, so the seed Job creates it. auth receives a derived read grant on every declared client's secret path and the list of clients as rendered configuration; how auth-api consumes that list (a prepare step that registers clients, or reading the paths at start) is decided in its ticket.
  • Composition refuses a generated key outside its Project's prefix, two Projects generating the same path, and an OIDC client declared on an edge to a provider that is not an auth provider.
  • Both implementations and the oracles change together while emf/ exists. Spec chapters 10, 16, 20, 30 and 55 change, and the decisions get ADRs.
  • The Estate repository drops cluster.sh's per-Project stages once each part has landed, and the estate-dashboard and tribelt move off their fixed logins.

Testing Decisions

  • Each new field gets a worked example with its rendered tree, reviewed by hand, and a refusal example per refusal.
  • The seed Job's policy is tested on the rendered object: create on the prefix, nothing elsewhere.
  • Whether the seed overwrites is tested on the Job's script against a fake Vault: absent keys are written, present keys untouched.
  • The live check for each part is one Project moved with no cluster.sh run: tribelt off its fixed login, and a new Project with a generated session key and an OIDC client.

Out of Scope

  • Secrets a person supplies (GitHub tokens, third-party API keys, GitHub App keys). They stay manual for now; a dashboard page for them is a later effort.
  • The once-per-cluster bootstrap stages.
  • Rotating a generated key; a generated key keeps its first value.

Further Notes

Part of #195. Builds on #378, which this makes unblocked work rather than a prerequisite to wait on.

Activity

  1. added
    type: featureNew user-facing or operator-facing capability.
    area: deployhomelab-deploy, deploy-v2, manifests, and rollout flow.
    on Oct 15, 2026
  2. ExtraToast commented on Oct 15, 2026

    @ExtraToast
    ContributorAuthor

    Decided while splitting this into tickets:

  3. added
    status: ready-for-agentFully specified; an agent may take it with nobody watching.
    and removed
    status: ready-for-agentFully specified; an agent may take it with nobody watching.
    on Oct 20, 2026
  4. 3 remaining items

  5. added
    status: ready-for-agentFully specified; an agent may take it with nobody watching.
    and removed
    status: ready-for-agentFully specified; an agent may take it with nobody watching.
    on Oct 24, 2026
  6. ExtraToast commented on Oct 29, 2026

    @ExtraToast
    ContributorAuthor

    Picked up from the ready-for-agent backlog and skipped: nothing in this epic is agent work right now.

    State on 2026-10-29:

    Question for the maintainer: should this epic drop status: ready-for-agent and stay open until #450 lands, or do you want #460 triaged to ready-for-agent so the next agent has something to start here?

  7. ExtraToast commented on Oct 29, 2026

    @ExtraToast
    ContributorAuthor

    Decision (maintainer, 2026-10-29): #460 (back-channel logout) is triaged for an agent now. #450 waits for the identity cutover.

  8. added
    status: ready-for-agentFully specified; an agent may take it with nobody watching.
    and removed
    status: ready-for-agentFully specified; an agent may take it with nobody watching.
    on Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: deployhomelab-deploy, deploy-v2, manifests, and rollout flow.status: ready-for-agentFully specified; an agent may take it with nobody watching.type: featureNew user-facing or operator-facing capability.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions