[Algunenano]

Updates contrib/zstd to a rebase of the ClickHouse/zstd fork onto upstream dev (49cf5179, 2026-10-05; the old fork was 199 non-merge commits behind), plus pending upstream PRs and one new patch of ours. Fork branch: ClickHouse/dev-20261021 (8388bfae). The overlap-copy patches of the old fork are kept; its unwind-info patches are upstream already.

Changes on top of upstream dev:

• Decompress: optimize sequence decompression (fast-path inlining, ABI scalarization, branch elision) meta/zstd#4751: sequence decoding. Most of the decompression gain on x86.
• dfast: pipelined extDict block compressor meta/zstd#4764: double-fast compressor, the strategy of ZSTD(3).
• opt: tabulate the optimal parser's per-code price components meta/zstd#4732: optimal parser (ZSTD(16) and above).
• decompress: preserve the frame pointer in the Huffman x86-64 asm meta/zstd#4811: keeps the frame pointer in the Huffman x86-64 asm, so profilers can unwind through it. Neutral for speed.
• legacy/v0.5: fix streaming regression from the wrap history fix meta/zstd#4816: fixes a regression of upstream dev in streaming decompression of legacy v0.5 frames (Destination buffer is too small once a frame crosses its window).
• New: a branchless copy of matches with an offset below 16 (NEON tbl, SSSE3 pshufb, AVX2 vpshufb). It replaces ZSTD_overlapCopy8, whose branch on offset < 8 mispredicts on numeric columns where offsets like 4, 8 and 12 alternate.

Compressed output is byte-identical to the current fork at levels 1, 3, 6, 9 and 16, and the checksum of all columns of ClickBench hits is the same after decompression.

ClickBench hits (100M rows in one ZSTD(3) part, no columns cache, median of 20 hot runs, non-LTO local build) on AMD EPYC 9R14 (Zen 4), speedup vs current master:

Query	16 threads on 8 cores	64 threads on 32 cores
SELECT count() FROM hits WHERE NOT ignore(SearchPhrase)	1.22x	1.19x
same with URL	1.14x	1.12x
same with Title	1.17x	1.13x
same with ClientIP	1.21x	1.14x
same with RegionID	1.14x	1.13x
same with UserID	1.08x	1.03x
SELECT sum(cityHash64(*)) FROM hits	1.06x	1.06x
SELECT COUNT(DISTINCT SearchPhrase) FROM hits	1.12x	1.11x

zstd CLI on the same data, single core, ZSTD(3), ClickHouse block sizes, vs current master: compression 1.018x, decompression 1.175x (geomean over 7 numeric columns and the separate characters and sizes streams of 4 String columns). The short-offset copy alone, vs the same set without it: ClientIP 1.15x, UserID 1.12x, RegionID 1.085x, ResolutionWidth 1.065x, and the characters streams of long strings 1-2% slower; in ClickHouse it is 1.12x on the ClientIP scan, 1.08x on UserID, 1.06x on RegionID, 0.98x on URL and Title, 1.01x on sum(cityHash64(*)) and 1.02x on COUNT(DISTINCT SearchPhrase) (16 threads). On Graviton4 it is +5.6% CLI decompression geomean, ClientIP +45%, UserID +14%, and 1.20x on the ClientIP scan in ClickHouse.

Earlier measurements of the same set without the short-offset copy, vs current master:

• Intel Xeon 6975P-C: SearchPhrase / URL / Title scans 1.24x / 1.17x / 1.20x with 16 threads on 8 cores, COUNT(DISTINCT SearchPhrase) 1.12x. With ZSTD(3), the INSERT INTO ... SELECT of the whole table uses 2% less CPU and OPTIMIZE FINAL of 5 parts (20M rows) takes 36.5 s instead of 37.5 s.
• Graviton4 (c8g.16xlarge): string scans 1.02x to 1.04x, the rest unchanged, insert and merge unchanged.
• Per PR, CLI: Decompress: optimize sequence decompression (fast-path inlining, ABI scalarization, branch elision) meta/zstd#4751 is +9% decompression on Intel and neutral on Graviton4; dfast: pipelined extDict block compressor meta/zstd#4764 is +2.7% ZSTD(3) compression on Intel and neutral on Graviton4; opt: tabulate the optimal parser's per-code price components meta/zstd#4732 is +13% (Intel) and +17% (Graviton4) at ZSTD(16).

Evaluated and not included:

• lazy: speed up the row matchfinder (levels 5-12) by 2-8%, identical output meta/zstd#4841: only helps levels 5-12 on x86. It also miscounts match lengths on big-endian targets, which corrupts data silently.
• decompress: keep the repeat-offset triple in registers on x86-64 meta/zstd#4733: +4% alone on x86, and slower combined with Decompress: optimize sequence decompression (fast-path inlining, ABI scalarization, branch elision) meta/zstd#4751.
• Add more prefetching to decompression. meta/zstd#3828 and FSE: fuse symbol spreading with table building on the fast path meta/zstd#4836: no gain on top of Decompress: optimize sequence decompression (fast-path inlining, ABI scalarization, branch elision) meta/zstd#4751.
• Optimize ZSTD compression level 6 parameters meta/zstd#4646: compresses worse at level 6.
• AArch64 only: AArch64: Improve sequence decoding meta/zstd#4813 (0.98x decompression), huf: Tune X1/X2 decoder selection for AArch64 meta/zstd#4803 and [AArch64] Optimize ZSTD_isRLE() with 64-byte NEON vector unrolling meta/zstd#4736 (neutral), Improving AArch64 Performance with SVE for Decompression Functions meta/zstd#4715 (no SVE code with our flags).

Related: #124405

Changelog category (leave one):

• Performance Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Update zstd to the latest upstream development version with pending performance improvements. Decompression of ZSTD(3) data is up to 20% faster on x86.

---

Workflow [PR]
Sync PR [sync-upstream/pr/127206]

[clickhouse-gh]

Workflow [PR], commit [50e6d81]

Summary: ❌

• Performance Comparison: Performance dashboard

job_name	test_name	status	info	comment
La Casa Del Dolor (amd_tsan)		FAIL
	ThreadSanitizer: data race (STID: 3437-4705)	FAIL	cidb

---

AI Review

Summary

Updates zstd to upstream development code with compression and decompression optimizations, including a custom SIMD copy for short match offsets. No actionable findings remain at 50e6d816baa75c49e4e0fc922d97d0f999332ac3; the earlier big-endian and legacy-streaming issues are addressed.

Final Verdict

Status: ✅ Approve

[clickhouse-gh]

Build profile diff (arm_release)

Comparing 50e6d816b with master 77c29e32d (stripped binary size, per-symbol sizes and ThinLTO time; compile times per translation unit against the most recent warmup build that recompiled it).

✅ No significant changes.

Binary sizes

programs/clickhouse-stripped: smaller than the master baseline by the known offset between the two builds, so the difference is not shown. A delta that differs from the offset by more than 50% of it is shown, in either direction.

The official master build is compiled with -g and a pull request build is not, and XRay counts debug instructions towards its instrumentation threshold, so master instruments thousands of functions more and its binary is ~0.4% larger no matter what the pull request does.

Object file sizes

18 object files changed (+86.26 KiB total), 0 added.

Object file	Master	PR	Δ
contrib/zstd-cmake/CMakeFiles/_zstd.dir/__/zstd/lib/compress/zstd_double_fast.c.o	56.91 KiB	117.44 KiB	+60.54 KiB (+106.38%)
contrib/zstd-cmake/CMakeFiles/_zstd.dir/__/zstd/lib/decompress/zstd_decompress_block.c.o	47.63 KiB	69.63 KiB	+22.00 KiB (+46.19%)

719 more object files are built by the master warmup baseline only (it builds every object-file target, a pull request build only clickhouse-bundle) and not compared.

Compile time of recompiled translation units

42 translation units recompiled, 26 s compile time in total, 42 of them have a recent master baseline.

Median compile-time ratio to the baselines is ×1.30 (machine-speed difference or a change affecting every TU); per-TU deltas below are relative to that ratio.
The matched translation units cost +9.8 s (+60%) in total before that adjustment.

Job report

[Algunenano]

Failures are unrelated and fixed in #127186