Skip to content

pclnpost: physically compact the deduplicated funcinfo carrier - #2267

Open
zhouguangyuan0718 wants to merge 2 commits into
xgo-dev:mainfrom
zhouguangyuan0718:codex/pclnpost-compact-sections
Open

pclnpost: physically compact the deduplicated funcinfo carrier#2267
zhouguangyuan0718 wants to merge 2 commits into
xgo-dev:mainfrom
zhouguangyuan0718:codex/pclnpost-compact-sections

Conversation

@zhouguangyuan0718

@zhouguangyuan0718 zhouguangyuan0718 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Why this is still needed

Current main still rewrites the compact prebuilt functab into a buffer sized to the original llgo_funcinfo_entry section and only zeroes its unused tail. LTO inline copies are logically deduplicated, but the original section and file space remain allocated.

Main has removed the old stub metadata model, so this revision deliberately keeps only one disposable entry carrier. If the compact table does not fit that carrier, rewriting fails closed and the existing runtime fallback remains authoritative.

Summary

  • place the ELF entry carrier immediately before .bss and the Mach-O carrier in an isolated __LLGO segment
  • physically remove the unused carrier suffix while preserving virtual addresses
  • update ELF program/section headers and Mach-O load-command/linkedit offsets under strict supported-layout contracts
  • stage, reopen and verify the complete image, re-sign an originally signed Mach-O, then atomically replace the executable
  • leave the original binary byte-for-byte unchanged on unsupported layouts or any rewrite/sign/verification failure

Validation

  • go test ./internal/pclnpost ./internal/build
  • go test -race ./internal/pclnpost
  • focused internal/pclnpost coverage: 86.1%
  • Darwin/arm64 Full-LTO texttemplate: 2,576,608 -> 2,366,320 bytes; __LLGO.filesize: 245,760 -> 32,768 bytes
  • Darwin compacted binary runs, passes codesign --verify --strict, and caller/logging acceptance tests pass
  • Linux/amd64 Full-LTO println: 62,688 -> 61,616 bytes; final writable PT_LOAD.p_filesz: 0x9a0 -> 0x570
  • Linux compacted binary runs and reopens with the shortened entry section

No Go-version compatibility workaround is included in this PR.

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: pclnpost physical carrier compaction

This is a well-engineered change: the transactional stage → sync → (sign) → verify → atomic rename → dir-fsync flow in replaceBinary, the fail-closed shape validation, and the post-edit re-parse with elf.NewFile/macho.NewFile are all solid, and the added tests assert real physical shrink and that a failed verify leaves the original intact. Doc comments for Rewrite, writeBack, and the package were correctly updated away from the old "performs no writes" language.

The findings below are hardening gaps against malformed/crafted binary layouts and one ELF-alignment correctness concern. raw here is the freshly linked output, so these are robustness rather than remote-exploit issues, but the Mach-O path already guards several of them and the ELF path does not — worth closing the asymmetry.

Key items (inline):

  • patchMachOFileOffsets second load-command walk lacks the bounds/cmdsz-minimum checks the parser applies (compact.go:247).
  • ELF oldEnd is never bounds-checked against len(raw) before copy(out[lastUsed:], out[oldEnd:]), unlike the Mach-O path (compact.go:385/447).
  • ELF cut point lastUsed is only 8-byte aligned; shifting later non-PT_LOAD segments by an unaligned removed can break p_offset ≡ p_vaddr (mod p_align) — Mach-O page-aligns but ELF does not (compact.go:404).
  • Design doc / funcInfoSiteLayoutArgs comment say carriers link "after .data" but the script inserts INSERT AFTER .got.plt.

Additional lower-priority notes (not inline):

  • parseELFLayout computes shoff+shnum*shentsz / strOff+strSize from 64-bit header fields; a near-2^64 shoff/strOff can wrap and pass the > len(raw) guard, then panic on indexing (compact.go:466/469/475). Prefer overflow-safe comparison (shoff > len(raw) first, then compare the remainder).
  • machoPageSize infers page size from ARM64 cputype only; other targets fall back to 4K. Fine for Apple Silicon today, but a comment noting the assumption would help.
  • writeBack recomputes physicalRemoved from os.Stat and clamps negatives, discarding the authoritative removed returned by compactCarrier (write.go:255). Since BytesRemoved is diagnostic-only, returning removed directly (or asserting equality) is more robust.
  • Memory/CPU (link-time, single-shot, so Low): the full binary is duplicated ~3x (info.raw, raw at write.go:160, out via append in compactCarrier) and fully parsed ~3x (compactCarrier re-parse + load(staged) in verify). Acceptable as defense-in-depth; one of the redundant full-buffer copies could be dropped if link-time footprint ever matters.

View job run

Comment thread internal/pclnpost/compact.go Outdated
Comment thread internal/pclnpost/compact.go
Comment thread internal/pclnpost/compact.go Outdated
Comment thread internal/build/build.go Outdated
@github-actions

github-actions Bot commented Aug 2, 2026

Copy link
Copy Markdown

LLGo baseline benchmarks

ac39ffc8c6ba | workflow run | long-term charts

Program measurements

Platform Workload File size vs base Build vs base Run vs base
Linux cprintf 18656 B +0.0% 291.374 ms -0.8% (better) 1.262 ms -5.3% (better)
Linux fmtprintf 1870272 B -0.6% (better) 2.598 s -1.3% (better) 3.207 ms -2.2% (better)
Linux println 67992 B -0.7% (better) 289.543 ms +2.3% (worse) 1.591 ms -0.3% (better)
macOS cprintf 101072 B +19.4% (worse) 472.435 ms +27.1% (worse) 4.555 ms +3.7% (worse)
macOS fmtprintf 1889248 B +0.0% 2.190 s -37.5% (better) 11.442 ms -14.3% (better)
macOS println 137616 B +13.5% (worse) 428.811 ms +8.1% (worse) 4.419 ms +21.7% (worse)
Core language and compiler benchmarks
Platform Benchmark ns/op vs base
Linux BenchmarkLookupPCRandom 13.410 ns/op +0.5% (worse)
Linux BenchmarkMergeCompilerFlags 149.400 ns/op -0.8% (better)
Linux BenchmarkMergeLinkerFlags 94.590 ns/op +0.3% (worse)
Linux BenchmarkChannelBuffered 33.390 ns/op +0.1% (worse)
Linux BenchmarkChannelHandoff 27299 ns/op -2.4% (better)
Linux BenchmarkDefer 45.810 ns/op -1.6% (better)
Linux BenchmarkDirectCall 1.557 ns/op +0.0%
Linux BenchmarkGlobalRead 1.556 ns/op +0.0%
Linux BenchmarkGlobalWrite 2.487 ns/op +0.1% (worse)
Linux BenchmarkGoroutine 41958 ns/op +31.1% (worse)
Linux BenchmarkInterfaceCall 7.893 ns/op +1.4% (worse)
Linux BenchmarkRuntimeGetG 1.875 ns/op +0.3% (worse)
macOS BenchmarkLookupPCRandom 11.510 ns/op -20.9% (better)
macOS BenchmarkMergeCompilerFlags 113.200 ns/op -19.2% (better)
macOS BenchmarkMergeLinkerFlags 65.840 ns/op -31.4% (better)
macOS BenchmarkChannelBuffered 23.750 ns/op -33.8% (better)
macOS BenchmarkChannelHandoff 7327 ns/op -11.1% (better)
macOS BenchmarkDefer 29.130 ns/op -33.5% (better)
macOS BenchmarkDirectCall 1.240 ns/op +1.6% (worse)
macOS BenchmarkGlobalRead 0.948 ns/op -21.4% (better)
macOS BenchmarkGlobalWrite 1.108 ns/op -20.9% (better)
macOS BenchmarkGoroutine 28737 ns/op +3.8% (worse)
macOS BenchmarkInterfaceCall 5.545 ns/op -12.5% (better)
macOS BenchmarkRuntimeGetG 1.987 ns/op -15.3% (better)

Compared with d600859372ce measured in the same runner job.

@codecov

codecov Bot commented Aug 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.36697% with 42 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/pclnpost/write.go 53.57% 7 Missing and 6 partials ⚠️
internal/build/build.go 64.51% 8 Missing and 3 partials ⚠️
internal/pclnpost/compact.go 96.91% 6 Missing and 5 partials ⚠️
internal/pclnpost/external.go 61.11% 4 Missing and 3 partials ⚠️

📢 Thoughts on this report? Let us know!

@zhouguangyuan0718
zhouguangyuan0718 force-pushed the codex/pclnpost-compact-sections branch 2 times, most recently from 875db43 to d3f8e86 Compare August 2, 2026 16:11
@zhouguangyuan0718
zhouguangyuan0718 force-pushed the codex/pclnpost-compact-sections branch from d3f8e86 to 80f64c0 Compare August 12, 2026 02:58
@zhouguangyuan0718 zhouguangyuan0718 changed the title pclnpost: physically compact deduplicated funcinfo carriers pclnpost: physically compact the deduplicated funcinfo carrier Aug 12, 2026
@zhouguangyuan0718
zhouguangyuan0718 force-pushed the codex/pclnpost-compact-sections branch from ac39ffc to e767561 Compare August 12, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant