build: prototype package-level deadcode planning for ThinLTO

[luoliwoshang]

Summary

This PR prototypes a ThinLTO-compatible deadcode pipeline for LLGo's Go method-table pruning.

The main architectural change is to separate the work into three stages:

1. each package produces its normal LLVM module plus LLGo Meta information;
2. a global Meta planner computes which method-table slots must remain live;
3. each package module is rewritten in place before its ThinLTO bitcode summary is emitted, and LLVM ThinLTO performs the remaining cross-module optimization.

This keeps LLGo's language-specific reachability knowledge in the planner while handing the rewritten program to the standard ThinLTO pipeline. It also establishes a boundary where the current planner can be replaced or extended later without changing the package rewrite/link flow.

This is intentionally a prototype. It validates the architecture and size behavior first; it does not yet implement MethodByName string propagation, archive/cache integration, or ThinLTO backend caching.

Motivation

The existing -deadcodedrop path materializes strong method-table overrides in the entry module. That approach works for the normal link pipeline, but it does not compose correctly with ThinLTO.

With:

-lto=thin -deadcodedrop

LLVM 19.1.7 previously crashed in:

FunctionImportGlobalProcessing::processGlobalForThinLTO

The strong override is also an awkward long-term integration point: the replacement global is detached from the package module that owns the original weak_odr global, its COMDAT, and its ThinLTO summary identity.

The goal of this experiment is therefore not to teach the existing override mechanism about ThinLTO. Instead, it makes LLGo's planner produce a global liveness plan, applies that plan to the owning package modules, and then lets ThinLTO analyze the resulting modules normally.

Previous strong-override design

The old non-ThinLTO path remains available and unchanged.

It computes dead method slots and emits same-name strong globals in the entry module to override the package-owned weak method tables at link time. ThinLTO sees both the original package definition and the late replacement through its module-summary/global-resolution machinery. That mismatch is the source of the LLVM crash seen in this experiment.

The new ThinLTO path deliberately does not emit those entry-module overrides.

Proposed pipeline

When both -lto=thin and -deadcodedrop are enabled, the build now performs:

package compilation
    -> collect package LLVM modules and Meta
    -> merge Meta into a global summary
    -> build one global deadcode.Plan
    -> rewrite each owning package module in place
    -> emit rewritten ThinLTO bitcode and module summaries
    -> build temporary linker archives
    -> run the normal ThinLTO link

The plan currently contains the live method slots for each type:

type Plan struct {
    LiveSlots map[string][]int
}

deadcode.BuildPlan uses the existing reachability analysis. deadcode.Analyze is retained as a compatibility wrapper.

The important boundary is that the package contributes Meta and owns its LLVM globals, while the planner makes one whole-program decision. The planner implementation can evolve independently; for example, later work could add reflection/string-flow facts or consume additional LLVM analysis without restoring the strong-override design.

Implementation

Global planning

internal/deadcode now exposes BuildPlan, which converts the merged global Meta summary and root set into an explicit reusable plan.

Package-local rewrite

internal/dcepass.RewriteTypeMethodTables applies the global plan to each package module in place.

For dead method slots, it replaces IFn/TFn targets with runtime.unreachableMethod. The original method-table global remains in its owning module, including its:

• weak_odr linkage;
• COMDAT membership;
• module identity used by ThinLTO;
• ABI-compatible initializer layout.

No same-name strong duplicate is added to the entry module.

Build integration

The experimental path is enabled only for the combination:

-lto=thin -deadcodedrop

For this mode, package bitcode/archive emission is delayed until all package Meta has been merged and the global plan is available. Each package module is rewritten first, then its ThinLTO bitcode and summary are generated from the rewritten IR.

Other build configurations continue to use their existing paths.

Size-level linker compatibility

ld64.lld accepts numeric values for --lto-O0..3 and rejects --lto-Oz. ltoLinkerOptFlag now emits the numeric linker flag only for O0 through O3; Os and Oz omit it, matching Clang driver behavior while retaining the size-oriented LLGo pre-link pipeline.

Why rewrite before emitting ThinLTO summaries?

ThinLTO decisions are driven by per-module summaries. Rewriting after summary emission would leave LLVM analyzing stale references: the summary could claim a method target is live even though the IR was later changed to runtime.unreachableMethod.

Emitting the summary from the already-rewritten package module ensures that symbol resolution, importing, internalization, and backend DCE all see the same graph.

Correctness validation

The ThinLTO + deadcode combination now completes and runs correctly for the interface/reflection cases used to exercise method-table reachability:

globaldce_interface_matrix
globaldce_interface_slots
globaldce_reflect_method
globaldce_reflect_type_method
globaldce_typeid_dce
globaldce_unexported_method_identity

A small interface experiment also confirms that pruning reaches the final binary while preserving behavior:

Metric	ThinLTO baseline	ThinLTO + planner DCE
File size	122,112 B	121,328 B
__text	0x51a4	0x506c
Drop symbols	3	0

Program output is identical.

Four-demo size experiment

Environment:

macOS arm64
LLVM 19.1.7
LLGO_BUILD_CACHE=off
-a (force package rebuild)

All 12 final binaries exited with status 0, and every output matched the previous DCE reference byte for byte.

Binary size

The no-DCE baseline is the normal non-ThinLTO build. Previous DCE is the existing non-ThinLTO strong-override path. The two new columns use the package-level planner and ThinLTO rewrite.

Demo	No-DCE baseline	Previous DCE	ThinLTO O2 + DCE	O2 vs previous	ThinLTO Oz + DCE	Oz vs previous	Oz vs O2
goimporter-1389	5342.0 KiB	3881.6 KiB	4150.3 KiB	+268.7 KiB / +6.92%	4238.9 KiB	+357.3 KiB / +9.21%	+88.7 KiB / +2.14%
embedunexport-1598	3904.5 KiB	2514.5 KiB	2653.9 KiB	+139.5 KiB / +5.55%	2689.3 KiB	+174.9 KiB / +6.95%	+35.4 KiB / +1.33%
mimeheader	2201.0 KiB	1613.3 KiB	1504.2 KiB	-109.1 KiB / -6.76%	1527.9 KiB	-85.5 KiB / -5.30%	+23.6 KiB / +1.57%
gotypes	3938.2 KiB	3269.1 KiB	3594.7 KiB	+325.6 KiB / +9.96%	3696.3 KiB	+427.2 KiB / +13.07%	+101.6 KiB / +2.83%

A positive value means the first column in the comparison is larger. mimeheader is smaller than the previous DCE in both ThinLTO modes, but Oz is larger than ThinLTO O2 in all four demos.

Build time

Each value is the median wall time (real) of three sequential forced rebuilds for one demo. The compiler binary build is excluded. The rounds were ordered differently to reduce warm-cache and thermal-order effects.

Demo	Previous DCE	ThinLTO O2 + DCE	ThinLTO Oz + DCE	O2 vs previous	Oz vs previous
goimporter-1389	37.59 s	47.52 s	45.84 s	+9.93 s / +26.4%	+8.25 s / +21.9%
embedunexport-1598	27.66 s	39.94 s	40.41 s	+12.28 s / +44.4%	+12.75 s / +46.1%
mimeheader	28.66 s	38.84 s	37.32 s	+10.18 s / +35.5%	+8.66 s / +30.2%
gotypes	28.35 s	41.05 s	40.80 s	+12.70 s / +44.8%	+12.45 s / +43.9%
All four, per-round sum median	122.26 s	168.41 s	161.91 s	+46.15 s / +37.7%	+39.65 s / +32.4%

ThinLTO is currently slower because this prototype disables package cache in the ThinLTO + deadcode mode, delays package emission, rewrites every package module, regenerates its bitcode summary, and then performs the ThinLTO link.

Section attribution

The incremental Oz size is not explained by __text alone:

Demo	O2 __text	Oz __text	Oz minus O2	O2 __llgo_fie	Oz __llgo_fie	FIE delta
goimporter-1389	1120.3 KiB	1111.2 KiB	-9.1 KiB	282.9 KiB	365.4 KiB	+82.6 KiB
embedunexport-1598	581.8 KiB	582.8 KiB	+1.0 KiB	133.2 KiB	176.0 KiB	+42.9 KiB
mimeheader	389.6 KiB	390.8 KiB	+1.3 KiB	81.5 KiB	107.7 KiB	+26.2 KiB
gotypes	1029.0 KiB	1040.6 KiB	+11.6 KiB	254.0 KiB	332.3 KiB	+78.4 KiB

__llgo_fie contains the funcinfo entry-site records emitted inside function bodies. ThinLTO can duplicate those records through inline copies. The link-phase pclnpost rewrite deduplicates the logical table in place, but it does not shrink the already allocated Mach-O section, so its zero-filled tail still contributes to the file size. The Oz pipeline produced more such records than O2, which dominates the Oz size increase in these binaries.

The final text-symbol counts also show that Oz is not simply retaining all code:

Demo	ThinLTO O2 text symbols	ThinLTO Oz text symbols
goimporter-1389	2028	2091
embedunexport-1598	1064	1097
mimeheader	508	535
gotypes	1868	1918

Interpretation

The planner itself still prunes method targets correctly. The size comparison is affected by three independent factors:

1. ThinLTO O2 changes the baseline through cross-package import and inlining, so it is not directly comparable to the old non-ThinLTO DCE pipeline.
2. The current LLGo funcinfo site representation makes inline copies consume __llgo_fie section capacity, and the in-place post-link rewrite cannot return that capacity to the Mach-O file.
3. ld64.lld accepts only numeric --lto-O0..3; there is no --lto-Oz backend flag. The Oz experiment therefore applies thinlto-pre-link<Oz> and size-oriented front-end IR optimization, while the ThinLTO backend keeps its numeric default optimization level.

This means the current Oz result is a useful diagnostic experiment, not yet an end-to-end size-optimized ThinLTO mode. The next size experiments should tune ThinLTO import/inlining and funcinfo-section emission directly.

Known limitations

• Package cache use is temporarily disabled for ThinLTO + deadcode.
• The prototype still creates temporary archives to match the linker's existing input contract, but it does not read or write cached package archives in this mode.
• Reusing one in-memory package module for multiple entry points/plans is not supported; a second link cannot reconstruct the original method table after the first rewrite.
• ThinLTO backend caching is not wired up yet.
• Oz currently controls the LLGo pre-link pipeline; an end-to-end size-oriented ThinLTO backend mode is not wired up.
• The planner still uses the current Meta reachability algorithm.
• MethodByName string/control-flow propagation is out of scope for this experiment.
• Archive and cache design needs to be revisited before enabling this path as the default.

Follow-ups

• Evaluate -Oz and size-oriented ThinLTO import/inlining thresholds.
• Make package bitcode immutable or reloadable so multiple entry-point plans can be applied independently.
• Define a cache key for the global plan and rewritten package bitcode.
• Wire ThinLTO backend cache directories into the build.
• Extend planner inputs when reflection/string-flow analysis is ready.
• Revisit temporary archive construction once the architecture is validated.

Tests

Passed:

go test ./internal/crosscompile
go test ./internal/deadcode ./internal/dcepass
go test ./internal/build -run "Test(ThinLTODeadcodeEnabled|DeadcodeDropEnabled|ApplyDeadcode)" -v

The four-demo runtime/build experiment also passed with identical output for all 12 binaries.

A full go test ./internal/build run was started but entered an existing long-running build test path and was interrupted after the targeted tests passed. The earlier full-suite attempt in this checkout also has the known test/go analyzer panic, cl matrix segfault, and cl timeout described above.

[github-actions]

LLGo baseline benchmarks

ffdedb0cf167 | workflow run | long-term charts

Program measurements

Platform	Workload	File size	vs base	Build	vs base	Run	vs base
Linux	cprintf	18544 B	+0.0%	307.055 ms	+0.4% (worse)	1.280 ms	-4.5% (better)
Linux	fmtprintf	1877280 B	-0.0% (better)	3.180 s	+0.2% (worse)	3.340 ms	+1.3% (worse)
Linux	println	68096 B	+0.0%	301.830 ms	-2.0% (better)	1.575 ms	-6.2% (better)
macOS	cprintf	84672 B	+0.0%	591.493 ms	+7.8% (worse)	3.961 ms	-4.5% (better)
macOS	fmtprintf	1888208 B	+0.0%	3.882 s	-6.6% (better)	17.864 ms	+16.8% (worse)
macOS	println	121200 B	+0.0%	440.414 ms	-10.1% (better)	6.276 ms	+21.9% (worse)

Core language and compiler benchmarks

Platform	Benchmark	ns/op	vs base
Linux	BenchmarkLookupPCRandom	13.340 ns/op	+0.8% (worse)
Linux	BenchmarkMergeCompilerFlags	152.100 ns/op	-0.1% (better)
Linux	BenchmarkMergeLinkerFlags	95.830 ns/op	-2.6% (better)
Linux	BenchmarkChannelBuffered	33.750 ns/op	-0.1% (better)
Linux	BenchmarkChannelHandoff	28087 ns/op	+8.7% (worse)
Linux	BenchmarkDefer	50.400 ns/op	-0.5% (better)
Linux	BenchmarkDirectCall	1.557 ns/op	-0.1% (better)
Linux	BenchmarkGlobalRead	1.558 ns/op	+0.1% (worse)
Linux	BenchmarkGlobalWrite	2.489 ns/op	+0.1% (worse)
Linux	BenchmarkGoroutine	32954 ns/op	+2.9% (worse)
Linux	BenchmarkInterfaceCall	7.785 ns/op	-0.1% (better)
Linux	BenchmarkRuntimeGetG	1.873 ns/op	+0.2% (worse)
macOS	BenchmarkLookupPCRandom	14.580 ns/op	+11.6% (worse)
macOS	BenchmarkMergeCompilerFlags	171.100 ns/op	-11.8% (better)
macOS	BenchmarkMergeLinkerFlags	87.940 ns/op	-11.8% (better)
macOS	BenchmarkChannelBuffered	24.900 ns/op	-0.2% (better)
macOS	BenchmarkChannelHandoff	8573 ns/op	+2.1% (worse)
macOS	BenchmarkDefer	34.600 ns/op	-22.3% (better)
macOS	BenchmarkDirectCall	1.165 ns/op	+2.8% (worse)
macOS	BenchmarkGlobalRead	1.162 ns/op	+0.9% (worse)
macOS	BenchmarkGlobalWrite	1.168 ns/op	-3.1% (better)
macOS	BenchmarkGoroutine	46606 ns/op	-22.9% (better)
macOS	BenchmarkInterfaceCall	4.946 ns/op	-3.3% (better)
macOS	BenchmarkRuntimeGetG	2.276 ns/op	-12.3% (better)

Compared with c9515d8cb9cf measured in the same runner job.

[codecov]

Codecov Report

❌ Patch coverage is 54.78261% with 52 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/build/build.go	18.51%	39 Missing and 5 partials ⚠️
internal/dcepass/dcepass.go	83.33%	4 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!