fix: pin CNPG postgres to 18.4 (PG 18.6 segfaults the 0.113.0 DocumentDB extension; unblock E2E) + govulncheck

[WentingWu666666]

Problem

The TEST - E2E workflow started failing deterministically on 2026-08-13 and stayed red on main afterwards. The E2E data job failed specs ($inc, deleteMany, $or) with a generic Mongo (InternalError), while every baked DocumentDB / gateway / operator artifact was byte-identical across the last green (Aug 11) and first red (Aug 13) runs.

Root cause

The DocumentDB CRD defaults spec.image.postgres to the floating tag ghcr.io/cloudnative-pg/postgresql:18-minimal-trixie. Each E2E job spins up a fresh kind cluster and pulls whatever that tag currently points to. CloudNative-PG rolled the tag on 2026-08-13:

Build (immutable dated tag)	PG version	Timestamp (UTC)	Note
18.4-202608030910-minimal-trixie	18.4	Aug 3	what the last green run (Aug 11) used
18.4-202608130737-minimal-trixie	18.4	Aug 13 07:37	last 18.4
18.6-202608131513-minimal-trixie	18.6	Aug 13 15:13	first-known-bad (first red run was Aug 13 16:20)

The 18.4 → 18.6 jump is the regression. The PostgreSQL backend segfaults inside the DocumentDB extension on 18.6. The postgres logs show the crashing query is a DocumentDB insert and, because restart_after_crash is off, the crash takes the whole instance down:

client backend (PID 208) was terminated by signal 11: Segmentation fault
    detail: Failed process was running: SELECT * FROM documentdb_api.insert($1, $2, $3, NULL)
terminating any other active server processes
shutting down because "restart_after_crash" is off

That crash is the real, common failure across all red runs. Clients only see the downstream symptom — usually a generic Mongo (InternalError) when their backend dies mid-request, and in one run a SCRAM-SHA-256 "sasl conversation error" from a connection whose backend was gone. The earlier SCRAM hypothesis was wrong: SCRAM was an occasional symptom, not the cause.

Isolating the culprit (repro matrix)

The 0.113.0 extension .deb was held constant while the PostgreSQL image was swapped:

Extension	PostgreSQL	Image / userland	Result
0.113.0 deb	18.4	CNPG trixie	✅ works
0.113.0 deb	18.6	CNPG trixie	💥 segfault
0.113.0 deb	18.6	non-CNPG Debian + PGDG	💥 segfault

The only variable that flips pass → fail is the PostgreSQL minor (18.4 → 18.6); image vendor and userland drop out. This rules out a bad CNPG image build and the operator's own extension-image packaging (Dockerfile_extension / ImageVolume lib-bundling) — the raw deb crashes on 18.6 regardless of vendor.

The fault is a product bug in the extension exposed by a PG 18.6 planner change (fixed upstream in 3555c7d): PG 18.6's executor-pruning rework added an unguarded list_nth on fdwPrivLists in ExecInitModifyTable (18.4 only read it behind an ri_FdwRoutine != NULL guard). The extension's hand-built local-shard insert plan (CreateLocalShardInsertPlan) left that list NIL, so the second insert into a collection dereferences NULL and segfaults. It's a source logic bug — not an ABI/recompile issue.

Upstream status: the fix has been merged into documentdb/documentdb main, but is not yet in any released deb (latest release is v0.114-0, which predates the fix). Until a release carrying the fix ships, PostgreSQL must stay pinned to 18.4 — see follow-ups.

Fix

Pin the CRD default to the version-constrained 18.4 tag:

ghcr.io/cloudnative-pg/postgresql:18.4-minimal-trixie

Staying on the 18.4 minor avoids the 18.6 insert segfault, while the rolling tag still receives CNPG's Debian/PGDG security rebuilds — so the user-facing default isn't frozen on a single dated image (see review discussion). This is a stopgap, not the destination — it holds PostgreSQL at 18.4 until a DocumentDB release carrying the 18.6 fix ships, at which point the default reverts to the floating 18-minimal-trixie tag.

Changes

• operator/src/api/preview/documentdb_types.go — updated +kubebuilder:default marker (source of truth) + explanatory comment.
• Regenerated CRDs (make manifests generate): operator/src/config/crd/bases/documentdb.io_dbs.yaml and operator/documentdb-helm-chart/crds/documentdb.io_dbs.yaml.

Validation

• Full E2E suite (data, upgrade, smoke, lifecycle, scale, tls, feature, backup, cluster-replication, ImageVolume) is green on this PR.
• go test ./internal/cnpg/... passes.

Follow-ups (out of scope)

• Un-pin, gated on an upstream release. When documentdb/documentdb cuts a release (v0.115-0+) containing the fix now on main: bump documentDbVersion, verify the new deb on the latest PG 18.6 (canary/local repro across the 18.x range), then revert this pin back to floating 18-minimal-trixie. Tracked separately so the bridge doesn't become permanent.
• De-float the version pairing. The operator floats three independent references to 18-minimal-trixie (the postgres pod, the extension-image build base, and the deb's compile-time minor); they drifted apart, which is how this reached users. Long term, bump them as a matched set (or validate the postgres major at admission via the existing webhook).
• Add a canary CI job that runs the smoke suite against the latest 18-minimal-trixie so the next upstream minor roll is caught before it reaches production, while the 18.4 tag protects the default path.
• Consider disabling fail-fast on the E2E matrix so one job failure doesn't cancel the whole workflow.

---

Also in this PR: govulncheck fix

The govulncheck workflow was failing on 4 called standard-library vulnerabilities, all fixed in go1.26.6:

• GO-2026-6218 — quadratic complexity in net/url resolvePath
• GO-2026-6090 — unbounded post-handshake messages in crypto/tls
• GO-2026-5972 — recursion depth in encoding/asn1
• GO-2026-5026 — punycode label handling via golang.org/x/net/idna

Bumped the go directive from 1.26.5 → 1.26.6 in every scanned module (operator/src, documentdb-kubectl-plugin, operator/cnpg-plugins/sidecar-injector, test/e2e, test/longhaul, test/shared) and the pinned GO_VERSION in test-unit.yml / test-unit-coverage.yml. Also made those two workflows shellcheck-clean (SC2046/SC2086) since the actionlint gate lints any workflow a PR touches. Verified locally: govulncheck reports No vulnerabilities found for documentdb-kubectl-plugin and operator/src under go1.26.6.

[documentdb-triage-tool]

🤖 Auto-triaged by documentdb-triage-tool.

Applied: go, bug
Project fields suggested: Component api · Priority P1 · Effort M · Status In Progress
Confidence: 0.85 (mixed)

Reasoning

component from path globs (api, manifests); effort from diff stats (24+3 LOC, 3 files); LLM: Pins a floating upstream image tag to a known-good digest to unblock a deterministically failing E2E pipeline on main; touches API types, manifests, and CRDs but the change itself is minimal.

If a label is wrong, remove it manually and ping @patty-chow so the rules can be tuned. The bot will not re-label items that already have component labels.

[Copilot]

Pull request overview

Pins the DocumentDB CRD default PostgreSQL image to a last-known-good CNPG Postgres 18.4 digest to make CI (and new clusters) reproducible and to avoid the PG 18.6 regression that triggers segfaults in the currently released DocumentDB extension; additionally bumps Go patch version to address govulncheck failures and updates the unit-test workflows accordingly.

Changes:

• Pin spec.image.postgres default from floating 18-minimal-trixie to an immutable 18.4-...@sha256:... digest in the API marker and regenerated CRDs.
• Bump go directives across modules from 1.26.5 → 1.26.6.
• Update unit-test / unit-coverage GitHub Actions workflows to use Go 1.26.6 and improve shell robustness (avoid SC2046/SC2086 via arrays + quoting).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
operator/src/api/preview/documentdb_types.go	Updates the kubebuilder default marker and documents the rationale for digest pinning.
operator/src/config/crd/bases/documentdb.io_dbs.yaml	Regenerated CRD schema reflecting the pinned Postgres image default and expanded description.
operator/documentdb-helm-chart/crds/documentdb.io_dbs.yaml	Regenerated Helm-packaged CRD mirroring the pinned default for chart installs.
operator/src/go.mod	Bumps operator module Go directive to 1.26.6 for vulnfixes.
documentdb-kubectl-plugin/go.mod	Bumps kubectl plugin module Go directive to 1.26.6.
operator/cnpg-plugins/sidecar-injector/go.mod	Bumps sidecar-injector module Go directive to 1.26.6.
test/e2e/go.mod	Bumps E2E test module Go directive to 1.26.6.
test/longhaul/go.mod	Bumps longhaul test module Go directive to 1.26.6.
test/shared/go.mod	Bumps shared test utilities module Go directive to 1.26.6.
.github/workflows/test-unit.yml	Updates CI Go version and makes package list handling safer via mapfile + arrays.
.github/workflows/test-unit-coverage.yml	Updates CI Go version and improves shell safety/quoting for coverage and diff coverage steps.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[xgerman]

The immutable digest is appropriate for reproducible CI, but I do not think it should be the CRD default for user deployments. CNPG rebuilds its images with Debian/PGDG security updates; pinning this digest permanently freezes the default on the August 3 image and will miss future security fixes unless this PR is manually refreshed.

Could the default use the version-constrained rolling tag instead?

ghcr.io/cloudnative-pg/postgresql:18.4-minimal-trixie

That keeps PostgreSQL at 18.4 to avoid the known 18.6 regression while allowing CNPG to publish refreshed 18.4 images with security updates. The immutable digest can remain in a dedicated E2E/reproducibility fixture if needed.

[guanzhousongmicrosoft]

new version 117 should fix the issue, see this commit:
documentdb/documentdb@3555c7d#diff-20d7a707a9dea4dc6f525e4f008f2e8c4975fe6664d1a0cc2438a04ea3948ece

[xgerman]

The digest pin makes this the production default indefinitely, so CNPG Debian/PGDG security rebuilds will not reach new deployments unless we manually update this CRD. Could we use the version-constrained rolling tag ghcr.io/cloudnative-pg/postgresql:18.4-minimal-trixie here instead? It keeps PG 18.4 while allowing refreshed 18.4 images; reserve the immutable digest for the E2E fixture if reproducibility is required.

[WentingWu666666]

Good call — thanks. Switched the CRD default to the version-constrained rolling tag:

ghcr.io/cloudnative-pg/postgresql:18.4-minimal-trixie

This keeps PostgreSQL on the 18.4 minor (avoiding the 18.6 insert segfault) while still letting CNPG publish refreshed 18.4 images with Debian/PGDG security updates, so the user-facing default isn't frozen on a single dated image.

I kept it simple and didn't add a separate immutable-digest E2E fixture for now — the E2E suite runs against the CRD default, and 18.4 is the safe minor. We can add a pinned-digest fixture later if we want byte-for-byte reproducible CI.

This is still a stopgap: once a DocumentDB release carrying the PG 18.6 fix (documentdb/documentdb@3555c7d) ships, the default reverts to the floating 18-minimal-trixie tag. Done in 0b79d76.