Repository navigation
Build an executable from an object per source - #118
Conversation
talac rebuilt everything on every run. --incremental=<directory> builds one object per source through the source partitions the backend already had, plus one for the process entry, and keeps them in the directory. An object is known by the Xmm of the functions of its source, the signatures of what they call in other sources, and what else decides it: the compiler itself, the target and the optimization level. The Xmm is what the frontend and the Xpp and Xmm passes left, so a comment or a moved line lowers nothing, a changed body lowers its source, and a changed signature also lowers its callers. Only the sources that changed are lowered, on several threads, each in an LLVM context of its own. A key is made from functions that do not form a module, so the wire gets EncodeFunctions, which verifies nothing and cannot be decoded. An entry object has no functions of its own, which Options::entry_only says. The frontend still reads every source, and symbols are numbered across the program, so a new function lowers the sources after it again. LLVM sees one source at a time, so calls between sources are not inlined. Both are in Documents/CLI.md.
Sources without declarations share the same functions, none, so their keys were equal: two workers wrote one staged file and the linker saw the object twice. The key now covers the name of the source. The workers ran on the default thread stack, 512 KiB on macOS, while LLVM recurses over the code it lowers. They now use llvm::thread with 8 MiB, which also drops std::jthread, absent from Apple libc++.
|
Self-review of the incremental build. Checked and sound. Found and fixed in the last commit.
Known limits, not defects. Windows locally: |
Symbol ids are numbered across the program and are part of the native name, and generated spellings such as $closure12 carry them too, so one new method changed the objects of the sources after it. BuildObjects now derives each id from the source, the spelling without those digits and which of that spelling the function is. References are rewritten before the keys are made, so a call that moves to another function still recompiles its caller. Measured on 13 sources and about 4800 methods: a new method recompiles 1 source instead of 9.
|
Self-review of the follow-up commit (stable symbols).
Windows locally: |
|
Correction to the self-review and to commit 2ba95cf: " |
First step of incremental compilation.
--incremental=<directory>keeps one object per source, keyed on the Xmm of the source and the signatures of what it calls, and lowers only what changed, in parallel.Measured on a generated program of 13 sources and about 4,800 methods, with a fastbuild talac on Windows, so absolute times are pessimistic: a changed body lowers 1 of 13 sources, a comment or a moved line 0 of 13, a new method in the third file 9 of 13.
The same measurement shows the larger part of the time is before LLVM: about 6 s in the frontend and about 9 s from Core to Xmm in a fastbuild, against 3 to 5 s in LLVM and the link. So this is not the whole job: the next steps are symbols that keep their number when another function appears, and reuse in the frontend.