feat: shared cluster in previews-shared scopes secrets per env

.gitignore

@@ -7,6 +7,7 @@
 *.md
 !/README.md
 !/CHANGELOG.md
+!/aws/*/README.md
 docs/
 runbooks/
 examples/

CHANGELOG.md

@@ -1,3 +1,11 @@
+# 0.2.0
+
+Modules: `aws-ec2-vault-cluster` 0.2.0, `aws-vault-access` 0.3.0. Needs `vault-admin` 0.3.0 and a fresh node AMI. Nothing changes for a cluster outside the shared previews env.
+
+* A cluster launched in the shared previews env (Nullstone env type `PreviewsSharedEnv`) is shared: output `shared = true`, and every tenant belongs to an env. Tenant roles are `<env>.<tenant>`, secrets live under `kv/data/envs/<env>/customers/<tenant>/`, and an app's broker can only log in as its own env's tenants. The cluster function binds each app role to its env through an identity entity; provisioning does the same for tenant roles.
+* `aws-vault-access` passes the app's env to the function on a shared cluster and injects `VAULT_ENV` (empty on an unshared cluster).
+* `vault-utils tenants create|destroy` take `--env` (required on a shared cluster, refused elsewhere). `tenants list` shows the env. New `envs list` and `envs destroy <env> --yes [--purge-secrets]`. `vault-utils env` prints `SHARED_ENVS=true` for a shared cluster.
+
 # 0.1.2
 
 * Node AMI serves the Vault web UI at `$VAULT_ADDR/ui`. Needs a fresh AMI.

README.md

@@ -161,6 +161,9 @@ Run from `local/` unless noted. Destructive commands require `--yes`.
 | `docker compose run --rm -e VAULT_TOKEN=... bootstrap tenants list` | no | List tenants; flags one missing a reader or writer role |
 | `docker compose run --rm -e VAULT_TOKEN=... bootstrap tenants destroy <id> --yes` | yes (access) | Revoke access; secrets kept |
 | `docker compose run --rm -e VAULT_TOKEN=... bootstrap tenants destroy <id> --yes --purge-secrets` | yes | Also destroy secret versions |
+| `vault-utils tenants create <id> --env <env>` | no | Onboard a tenant for one env (shared cluster; see [Shared cluster](#shared-cluster)) |
+| `vault-utils envs list` | no | Envs with tenants (shared cluster) |
+| `vault-utils envs destroy <env> --yes [--purge-secrets]` | yes | Offboard every tenant of an env; purge also destroys its secrets |
 | `docker compose run --rm bootstrap snapshot take` | no | Raft snapshot plus SHA-256 |
 | `docker compose run --rm bootstrap snapshot restore <file> --yes` | yes | Replaces all Vault state |
 | `go test -short ./...` | no | Unit tests (tenant ID, policy lint, render, compose lint) — repo root |
@@ -187,7 +190,34 @@ kv/metadata/customers/{tenant_id}/*
 | Tenant policies | `tenant-reader`, `tenant-writer`, `tenant-database` | Static, written at bootstrap. The tenant comes from the login role name (`{{identity.entity.aliases.<accessor>.metadata.role_name}}`), so nothing is written per tenant. |
 | Broker policies | `apps-reader`, `apps-writer` | Read `role-id` and mint `secret-id` for any onboarded tenant on one mount. No KV, database, or sys access. |
 
-Tenant IDs: `^[a-z0-9]([a-z0-9-]{1,30}[a-z0-9])$` (3-32 characters). Rejected: `/`, `..`, `*`, `sys`, `data`, `root`, and similar reserved names.
+Tenant IDs: `^[a-z0-9]([a-z0-9-]{1,30}[a-z0-9])$` (3-32 characters). Rejected: `/`, `..`, `*`, `sys`, `data`, `root`, `envs`, and similar reserved names.
+
+### Shared cluster
+
+A cluster launched in the stack's shared previews env (Nullstone env type `PreviewsSharedEnv`) serves every preview env. Its output `shared` is true, its nodes run with `SHARED_ENVS=true`, and every tenant belongs to one env:
+
+```
+kv/data/envs/{env}/customers/{tenant_id}/*
+kv/metadata/envs/{env}/customers/{tenant_id}/*
+```
+
+| Piece | Shared cluster |
+|---|---|
+| Tenant role name | `{env}.{tenant_id}` on the same two mounts. Env names use the tenant ID character class, so the `.` is unambiguous. |
+| Tenant policies | Env and tenant come from entity metadata (`{{identity.entity.metadata.env}}`, `...tenant`). `tenants create --env` writes the roles, one entity per role with that metadata, and an alias on the role's `role_id`. Nothing per env. |
+| Broker policies | `auth/approle-<kind>/role/{{identity.entity.metadata.env}}.*` with every parameter denied: an app mints logins only for its own env and cannot change a role. The env comes from the entity the cluster function writes on the app's auth role (see [App access](#app-access)). |
+| Provisioning, apps-auth | May create entities and aliases; `policies` and `disabled` are refused, groups and OIDC are denied. |
+
+A cluster anywhere else is unshared and renders the 0.1.x policies unchanged. Converting a cluster between the two is not supported: launch a new one.
+
+```bash
+vault-utils tenants create acme-corp --env pr-123
+vault-utils tenants list
+vault-utils envs list
+vault-utils envs destroy pr-123 --yes --purge-secrets
+```
+
+`--env` is required on a shared cluster and refused on an unshared one; `vault-utils env` exports `SHARED_ENVS` so the CLI knows which it is talking to. `envs destroy` offboards every tenant of the env; `--purge-secrets` also destroys `kv/metadata/envs/<env>` and needs break-glass.
 
 ```bash
 export VAULT_ADDR=http://127.0.0.1:8200
@@ -429,20 +459,40 @@ On boot, `vault-configure.service` runs after cloud-init, writes `/etc/vault.d/c
 
 Connect `vault` to the cluster. The app module must expose `security_group_id` and its IAM role name.
 
-`role_name` is optional. If empty, the Vault role is `<app-name>-<resource-suffix>`. `access` is `reader` (default) or `writer`; the capability binds the matching broker policy (`apps-reader` or `apps-writer`) and nothing else. The app receives `VAULT_ADDR`, `VAULT_ROLE`, `VAULT_TLS_SERVER_NAME` (empty unless the NLB terminates TLS), and `VAULT_TENANT_MOUNT`. During apply the capability calls the cluster function with `method = "aws"` and the app IAM role ARN; the function binds only that principal to that role. The same function serves GCP service accounts (`method = "gcp"`) for a future GCP cluster module.
+`role_name` is optional. If empty, the Vault role is `<app-name>-<resource-suffix>`. `access` is `reader` (default) or `writer`; the capability binds the matching broker policy (`apps-reader` or `apps-writer`) and nothing else. The app receives `VAULT_ADDR`, `VAULT_ROLE`, `VAULT_TLS_SERVER_NAME` (empty unless the NLB terminates TLS), `VAULT_TENANT_MOUNT`, and `VAULT_ENV`. During apply the capability calls the cluster function with `method = "aws"` and the app IAM role ARN; the function binds only that principal to that role. The same function serves GCP service accounts (`method = "gcp"`) for a future GCP cluster module.
+
+On a shared cluster (output `shared`), the capability also sends the app's Nullstone env name. The function then gives the auth role an identity entity with that env, which the broker policies template, and `VAULT_ENV` carries it to the app. On an unshared cluster `VAULT_ENV` is empty and the function never touches identity. Leave `role_name` empty on a shared cluster: a fixed name collides across preview envs and the function refuses the second binding.
 
-The app does not receive a Vault token. At startup it logs in with its IAM role, then for every request it logs in again as one tenant:
+The app does not receive a Vault token. At startup it logs in with its IAM role, then for every request it logs in again as one tenant. The role is `$TENANT`, or `$VAULT_ENV.$TENANT` when `VAULT_ENV` is set; the KV path is `customers/$TENANT/...`, or `envs/$VAULT_ENV/customers/$TENANT/...`:
 
 ```bash
 vault login -method=aws role="$VAULT_ROLE"
 
-ROLE_ID=$(vault read -field=role_id auth/$VAULT_TENANT_MOUNT/role/$TENANT/role-id)
-SECRET_ID=$(vault write -f -field=secret_id auth/$VAULT_TENANT_MOUNT/role/$TENANT/secret-id)
+ROLE="${VAULT_ENV:+$VAULT_ENV.}$TENANT"
+ROLE_ID=$(vault read -field=role_id auth/$VAULT_TENANT_MOUNT/role/$ROLE/role-id)
+SECRET_ID=$(vault write -f -field=secret_id auth/$VAULT_TENANT_MOUNT/role/$ROLE/secret-id)
 TENANT_TOKEN=$(vault write -field=token auth/$VAULT_TENANT_MOUNT/login role_id=$ROLE_ID secret_id=$SECRET_ID)
 ```
 
 The broker token reads no secrets. The tenant token is good for that tenant only, for 15 minutes. A login as any other role is denied. The operator token is not injected.
 
+### Shared cluster in Nullstone
+
+Launch the cluster block in the stack's shared previews env (named `previews-shared` below). In `.nullstone/previews.yml`, point each app's capability at it; everywhere else the app uses its own env's cluster:
+
+```yaml
+version: "0.1"
+
+apps:
+  api:
+    capabilities:
+      vault:
+        connections:
+          vault: previews-shared.vault
+```
+
+Create the preview env's tenants with `vault-utils tenants create <id> --env <env>` and remove them with `vault-utils envs destroy <env> --yes --purge-secrets` when the env goes away.
+
 ## Security
 
 - Host ports bind to `127.0.0.1` only

aws/aws-ec2-vault-cluster/README.md

@@ -0,0 +1,81 @@
+# aws-ec2-vault-cluster
+
+Self-hosted Vault CE on an EC2 Auto Scaling Group: Raft storage, KMS auto-unseal, internal NLB on `vault.internal`, S3 snapshots, platform tokens in Secrets Manager. Apps connect through `nullstone/aws-vault-access`. Operations, backup, and break-glass are in the [repo README](../../README.md).
+
+## Connections
+
+| Name | Contract | Purpose |
+|---|---|---|
+| `network` | `network/aws/vpc` | VPC, private subnets, internal zone |
+| `snapshots_bucket` | `datastore/aws/s3` | Raft snapshots and the init claim |
+| `unseal_key` | `datastore/aws/kms` | Auto-unseal key (not the bucket key) |
+| `subdomain` (optional) | `subdomain/aws/route53` | User-facing name; the NLB terminates TLS only with this |
+
+## Variables
+
+| Name | Default | Purpose |
+|---|---|---|
+| `cluster_size` | `1` | Odd number of nodes |
+| `instance_type` | `t3.micro` | |
+| `ami`, `ami_owner` | latest `nullstone-vault` AMI from `522657839841` | Set `ami_owner = "self"` when this account bakes it |
+| `backup_schedule` | empty | Cron for S3 snapshots; empty disables |
+| `protect_platform_secrets` | `true` | `prevent_destroy` on the token secrets; set false before destroying the workspace |
+
+## Shared cluster
+
+Launched in the stack's shared previews env (Nullstone env type `PreviewsSharedEnv`), the cluster is shared: output `shared` is true and the nodes run with `SHARED_ENVS=true`. Every tenant then belongs to an env. Anywhere else the cluster is unshared and nothing below changes from 0.1.x. A cluster cannot switch modes; launch a new one.
+
+## Where a tenant's secrets live
+
+| Cluster | KV path | AppRole role name | Database roles |
+|---|---|---|---|
+| Unshared | `kv/data/customers/{tenant}/*` | `{tenant}` | `tenant-{tenant}-readonly`, `-readwrite` |
+| Shared | `kv/data/envs/{env}/customers/{tenant}/*` | `{env}.{tenant}` | `tenant-{env}.{tenant}-readonly`, `-readwrite` |
+
+KV v2: `metadata/`, `delete/`, `undelete/`, and `destroy/` mirror the `data/` tree. Tenant IDs and env names match `^[a-z0-9]([a-z0-9-]{1,30}[a-z0-9])$`; neither contains `.`, so the role name splits unambiguously.
+
+## How a tenant token is scoped
+
+The tenant policies (`tenant-reader`, `tenant-writer`, `tenant-database`) are static, written once at bootstrap, and never name a tenant. A Vault ACL template fills the tenant in from the identity of the token that logged in:
+
+- Unshared: the AppRole role name (`{{identity.entity.aliases.<accessor>.metadata.role_name}}`), which only provisioning can set.
+- Shared: entity metadata `env` and `tenant` (`{{identity.entity.metadata.env}}`), written by provisioning when it creates the tenant's roles and aliased on each role's `role_id`.
+
+Apps cannot reach the identity API, so neither value can be forged. A token whose entity lacks the metadata matches no path. Cross-tenant, cross-env, parent, wildcard, and traversal requests return 403; a 200 there is a breach.
+
+## Apps
+
+An app never holds a token that spans tenants.
+
+1. It logs in with its IAM role (`vault login -method=aws role=$VAULT_ROLE`) and gets a broker token holding `apps-reader` or `apps-writer`. That token reads no secrets.
+2. For each request it mints a one-time secret-id for one tenant's role on `$VAULT_TENANT_MOUNT` (`approle-reader` or `approle-writer`) and logs in again. The result is a 15 minute token for that tenant only.
+
+On a shared cluster the broker policy is templated on the app entity's `env`, which the cluster function writes when `aws-vault-access` binds the app: `auth/approle-<kind>/role/{{identity.entity.metadata.env}}.*` with every parameter denied. The app can mint logins only for its own env's tenants and cannot alter a role. `VAULT_ENV` tells the app its env; it is empty on an unshared cluster, where the role is `$TENANT` and the path `customers/$TENANT`.
+
+## Humans
+
+There is no per-person login. A human uses a platform token from Secrets Manager, or the token `vault login` saved, with `vault-utils` (`vault-utils env --org --stack --env --block` prints `VAULT_ADDR`, `VAULT_TLS_SERVER_NAME`, and `SHARED_ENVS`):
+
+| Token | Can | Cannot |
+|---|---|---|
+| Provisioning (`provisioning_secret_arn`) | `tenants create / list / destroy`, `envs list / destroy`; on a shared cluster also the tenant entities. Mint any tenant's login. | Read tenant KV with its own token, write policies, give an entity policies, touch groups or OIDC |
+| Operator (`operator_secret_arn`) | Health, mounts, snapshots, start generate-root | Read tenant KV, restore |
+| Break-glass root | Restore, purge, read tenant data in an incident | Kept: minted from operator plus recovery keys, two people, revoked after one action |
+
+The function's `apps-auth` token writes app auth roles and, on a shared cluster, app entities. It cannot read tenant secrets.
+
+Per-tenant CLI on a shared cluster takes `--env`; on an unshared one `--env` is refused.
+
+## Outputs
+
+`vault_addr`, `user_vault_addr`, `tls_server_name`, `vault_fqdn`, `user_fqdn`, `vault_api_port`, `nlb_security_group_id`, `admin_function_name`, `shared`, `operator_secret_arn`, `provisioning_secret_arn`, `db_hostname`, `db_port`, `db_endpoint`, `private_urls`, `public_urls`, plus node identifiers (`role_name`, `instance_profile_name`, `security_group_id`, `autoscaling_group_name`, `ami_id`).
+
+## Testing
+
+```bash
+tofu fmt -check
+tofu init -backend=false
+tofu test
+```
+
+`tofu test` mocks every provider. A real plan needs a Nullstone workspace and AWS credentials.

aws/aws-ec2-vault-cluster/admin.tf

@@ -1,6 +1,6 @@
 module "vault_admin" {
   source  = "api.nullstone.io/nullstone/aws-vault-admin/aws"
-  version = "~> 0.2.2"
+  version = "~> 0.3.0"
 
   name             = "${local.resource_name}-apps-auth"
   tags             = local.tags

aws/aws-ec2-vault-cluster/nullstone.tf

@@ -28,3 +28,14 @@ locals {
   block_name    = data.ns_workspace.this.block_name
   resource_name = "${data.ns_workspace.this.block_ref}-${random_string.resource_suffix.result}"
 }
+
+data "ns_env" "this" {
+  stack_id = data.ns_workspace.this.stack_id
+  env_id   = data.ns_workspace.this.env_id
+}
+
+# A cluster launched in the stack's shared previews env serves every preview env and keeps their secrets
+# apart (envs/<env>/customers/<tenant>). Anywhere else it is one env's cluster with the unscoped layout.
+locals {
+  shared = data.ns_env.this.type == "PreviewsSharedEnv"
+}

aws/aws-ec2-vault-cluster/outputs.tf

@@ -102,3 +102,8 @@ output "public_urls" {
   value       = local.public_urls
   description = "list(string) ||| A list of URLs accessible to the public"
 }
+
+output "shared" {
+  value       = local.shared
+  description = "bool ||| True when this cluster runs in the shared previews env (Nullstone env type PreviewsSharedEnv) and scopes every tenant's secrets under envs/<env>/."
+}

aws/aws-ec2-vault-cluster/outputs.tftest.hcl

@@ -11,6 +11,13 @@ mock_provider "ns" {
     }
   }
 
+  mock_data "ns_env" {
+    defaults = {
+      name = "prod"
+      type = "PipelineEnv"
+    }
+  }
+
   mock_data "ns_connection" {
     defaults = {
       outputs = {
@@ -121,3 +128,44 @@ run "uses_https_with_a_subdomain" {
     error_message = "With TLS, vault.internal needs the user-facing name for certificate verification."
   }
 }
+
+run "is_not_shared_in_a_pipeline_env" {
+  command = plan
+
+  assert {
+    condition     = output.shared == false && strcontains(base64decode(aws_launch_template.this.user_data), "SHARED_ENVS=false")
+    error_message = "A cluster in any env but the shared previews env is unshared and tells its nodes so."
+  }
+}
+
+run "is_not_shared_in_a_preview_env" {
+  command = plan
+
+  override_data {
+    target = data.ns_env.this
+    values = {
+      type = "PreviewEnv"
+    }
+  }
+
+  assert {
+    condition     = output.shared == false
+    error_message = "A preview env's own cluster is unshared."
+  }
+}
+
+run "is_shared_in_the_shared_previews_env" {
+  command = plan
+
+  override_data {
+    target = data.ns_env.this
+    values = {
+      type = "PreviewsSharedEnv"
+    }
+  }
+
+  assert {
+    condition     = output.shared == true && strcontains(base64decode(aws_launch_template.this.user_data), "SHARED_ENVS=true")
+    error_message = "A cluster in the shared previews env is shared and its nodes scope tenants per env."
+  }
+}

aws/aws-ec2-vault-cluster/templates/user-data.sh.tpl

@@ -13,6 +13,7 @@ VAULT_APPS_AUTH_SECRET_ARN=${apps_auth_secret_arn}
 SNAPSHOT_BUCKET=${snapshot_bucket_name}
 SNAPSHOT_PREFIX=${snapshot_prefix}
 BACKUP_SCHEDULE="${backup_schedule}"
+SHARED_ENVS=${shared}
 AUDIT_LOG_PATH=/opt/vault/audit/audit.log
 AWS_REGION=${aws_region}
 UNSEAL_KMS_KEY_ARN=${unseal_kms_key_arn}

aws/aws-ec2-vault-cluster/user-data.tf

@@ -11,6 +11,7 @@ locals {
     snapshot_bucket_name    = local.snapshot_bucket_name
     snapshot_prefix         = local.snapshot_prefix
     backup_schedule         = var.backup_schedule
+    shared                  = local.shared
     vault_api_addr          = "https://${trimsuffix(aws_route53_record.vault.fqdn, ".")}:${local.vault_api_port}"
   })
 }