Skip to content

container-registry: replace distribution with zot - #1053

Merged
sbee-mq[bot] merged 7 commits into
mainfrom
zot
Nov 13, 2026
Merged

sbee-mq[bot] merged 7 commits into
mainfrom
zot

Conversation

@mulatta

@mulatta mulatta commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Distribution needed docker_auth for token auth and a daily offline
garbage-collection window. zot authenticates directly, collects
garbage online, hot-reloads ACLs, deduplicates blobs, and ships a web
UI.

zot runs on psi with local storage, since psi moves nearly every image
and the campus network drops fast inbound TCP downloads such as R2
blob redirects. eta keeps public TLS and proxies over wg-admin. The R2
bucket and docker_auth are removed; the seven existing images were
copied with their digests verified.

The registry becomes private. Members sign in through authentik and use
UI-issued API keys for docker; non-admins see only research/ and
infra/, and researchers cannot overwrite published tags. The CI bot and
a break-glass admin stay in htpasswd. Apply terraform/authentik before
deploying psi, since zot discovers the issuer at startup.

@mulatta
mulatta force-pushed the zot branch 5 times, most recently from 1143978 to b71142e Compare November 13, 2026 03:56
Build the zot web UI from source instead of embedding the prebuilt release tarball. The vite output is byte-identical to the upstream commit-ce3819c zui.tgz, so the registry serves the same assets while the UI stays auditable and patchable.

Change-Id: Id08ca81190fbe8cb1d14cceb8f6dcfcd58eff77c
Build the full extension set with the source-built UI so the registry on eta can replace distribution and docker_auth with OIDC, group ACLs, API keys, and retention.

Change-Id: Id176a0bfc567e4a3ce0f1de7550d46c431aff66b
Keep runtime credentials out of the store and validate configuration before activation. A stable /etc path lets ACL and retention changes hot-reload without interrupting registry clients; other settings still need a restart.

Retention reloads only while the running policy already needs metaDB. Docker-built images need http.compat docker2s2 because zot accepts only OCI manifests by default.

The service runs as a static user so local storage outside StateDirectory, such as a separate data array, can be owned by it.

Change-Id: I11b1966b5696b2762011929254f85be1d37d6a95
Distribution needed docker_auth for token auth and an offline garbage-collection window that stopped the registry daily. zot authenticates directly, collects garbage online, hot-reloads ACL changes, deduplicates blobs, and ships a web UI.

zot runs on psi and stores images on its HDD array. psi runs k3s and the release runner, so it pulls and pushes nearly every image, and the campus network drops fast inbound TCP downloads such as R2 blob redirects. Local storage keeps those transfers on the host and lets zot read blobs directly for scanning. eta keeps public TLS and proxies over wg-admin, whose UDP transport is not affected by the drops; psi resolves the public name to its own vhost so local clients need no mirror configuration.

The registry becomes private. Lab members sign in through authentik and use UI-issued API keys for docker and skopeo, so access follows authentik groups instead of shared passwords. Repositories are limited to research/ and infra/ so later per-project grants need only a longer pattern. Researchers may push but not overwrite tags, keeping published versions immutable; the CI bot keeps overwrite rights for its BuildKit cache tags.

Existing bcrypt hashes become the htpasswd file, keeping CI credentials and a break-glass admin that works without authentik. Docker media types stay accepted so default docker and skopeo pushes and Docker Hub mirrors keep working. Images are reproducible, so the store is not backed up.

Change-Id: I35de548f7dc8ddf111f4771efc763d053b75802a
zot now authenticates registry clients directly, leaving no users of the token server.

Change-Id: I99b989f60b532de3e43ca6822f20e70456be6d57
zot now stores images on psi, so the bucket and its scoped token have no users. The bucket is already empty.

Change-Id: Ifd31a82e85ae7f6d0febc62e7638066a1ab05750
zot signs lab members in through authentik and maps the groups claim directly to repository permissions, so the provider sends plain group names. The application is limited to lab members because zot grants nothing to other authenticated users anyway.

Change-Id: I7e2da7730f8c72d1b3fed3b669bd78307f67c378
@mulatta
mulatta enabled auto-merge (rebase) November 13, 2026 06:01
@sbee-mq
sbee-mq Bot merged commit 44887c4 into main Nov 13, 2026
8 checks passed
@sbee-mq
sbee-mq Bot deleted the zot branch November 13, 2026 06:01
@sbee-mq sbee-mq Bot added mq/waiting and removed mq/waiting labels Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant