[jghoman]

Dependencies

• pyhoglake 1.4.0 → 1.4.4 (codec kwargs + parquet_write_options, pyhoglake: let writers choose the parquet codec (default unchanged: SNAPPY) hoglake#435). Brings pyarrow 23.0.1 → 26.0.0.
• Drops pyducklake. Nothing imports it; it was a lockstep canary for the duckdb pin. pyducklake ≤ 1.0.18 capped pyarrow < 26, and 1.0.19 moved to duckdb==1.5.6 against millpond's deliberate 1.5.5 pin. The DuckLake build stays guarded by test_ducklake_pin.py and the Dockerfile SHA layer. DuckDB unchanged at 1.5.5. Cross-repo cleanup: DuckDB/DuckLake version policy across millpond, viaduck, pyducklake, pyhoglake during the DuckLake sunset #156.

Why

pyhoglake's default codec is pyarrow's SNAPPY. ~97% of prod-us ingest.events_raw bytes are SNAPPY (480-object sample). One 413K-row prod-us slice: snappy 1,138 MB, zstd-3 281 MB. Daytime events objects (~560-620 MiB) sit above the 512 MiB compaction target, so their codec is permanent.

What

• HOGLAKE_COMPRESSION (default zstd) and HOGLAKE_COMPRESSION_LEVEL (default 3 for zstd), passed to every prepare_append_tables. snappy reproduces the previous bytes (rollback).
• config.load() validates through pyhoglake.parquet_write_options, not a copy of its rules: zstd 1-9, snappy, none. Errors name the env vars.
• Codec logged at startup and recorded as codec= in every snapshot message.

Rollout notes

• Every hoglake consumer switches on the next image; the chart has no per-consumer passthrough yet.
• zstd-3 events objects (~140 MiB) fall under the compaction target, so daytime events_raw objects become compaction candidates for the first time. Estimate (unmeasured): +190-380 groups/h on prod-us.
• Encode cost: zstd-3 ~1.9x snappy on a synthetic benchmark, a few seconds per 2 GiB flush. zstd-9 at 2 GiB is ~21 s, past _PREPARE_ALLOWANCE_S (20 s); the liveness model does not yet charge codec or level.

Tests

• Unit: defaults, passthrough, refusals; codec reaches prepare_append_tables and the snapshot message; installed-pyhoglake signature guard.
• Integration (real server + MinIO): written object's footer is ZSTD by default and SNAPPY on rollback; snapshot message carries codec=.
• Against pyhoglake 1.4.4 / pyarrow 26.0.0 / duckdb 1.5.5: 1,445 unit, 155 integration (real hoglake server + MinIO), all pass.

[bill-ph]

Reviewed the PR description, full diff, and existing discussion at ba921e2. Checked configuration defaults and validation, codec propagation into parquet preparation and snapshot messages, retry/liveness interactions, dependency and lockfile changes, and unit/integration test coverage. Also checked for overengineering and scope creep. Tests, builds, and linters were not run, as requested.

No issues found.

— Robo Bill v2 (gpt-6.1-sol, high reasoning)