Repository navigation
Latest master toolchains are broken when linking LLVM: "__morestack: invalid needed version 2" #81554
Description
When I install the latest master toolchains via rustup-toolchain-install-master and try to build Miri with them, linking fails:
error: linking with `cc` failed: exit code: 1
|
[...]
= note: /usr/bin/ld: /home/r/.rustup/toolchains/miri/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-11-rust-1.51.0-nightly.so: __morestack: invalid needed version 2
/usr/bin/ld: /home/r/.rustup/toolchains/miri/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-11-rust-1.51.0-nightly.so: error adding symbols: bad value
collect2: error: ld returned 1 exit status
I did some bisection, and it looks like the problem started happening with #81489.
Cc @rust-lang/infra
Activity
RalfJung commentedon Jan 30, 2021
The error is actually already mentioned in #81489. However, I am building Miri with cargo, so I don't know how I'd "use LLD instead".
[-]Latest master toolchains are broken[/-][+]Latest master toolchains are broken when linking LLVM: "__morestack: invalid needed version 2"[/+]Mark-Simulacrum commentedon Jan 30, 2021
Hm, we probably need to revert that PR. I did not realize we'd need lld downstream too, I thought that was only on our builder.
cc @nikic @cuviper
cuviper commentedon Jan 30, 2021
Yes, a revert seems appropriate. We're going to have to figure out where that's actually coming from. The only change I saw between 10 and 11 by grepping
morestackwas the removal of llgo, but maybe there's some bad remnant.nikic commentedon Jan 30, 2021
Yeah, I thought this issue would be limited to the rustc build, but it makes sense that it can also affect project linking against rustc shared objects.
I've looked a bit into what the root cause is, and what I have so far as that the symbol comes from https://github.com/llvm/llvm-project/blob/b10d445307a0f3c7e5522836b4331090aacaf349/llvm/lib/ExecutionEngine/RuntimeDyld/RTDyldMemoryManager.cpp#L217 and is present as a weak symbol both when compiling with clang-10 and clang-11. The only difference I see from
objdump -Tis this:So apparently
__morestackwas treated as a symbol of theLLVM_11library, while now it's treated as a symbol ofBase-- I'm not sure what that actually means, but presumably that refers to the compiler runtime libraries. My guess is that previously it just ended up being nulled out as a missing weak symbol, but now it's actually being found and linking fails because ... reasons. I'm not good at linkers :)Edit:
readelf -Wsshows these as:RalfJung commentedon Jan 30, 2021
@nikic can you prepare a revert PR?
nikic commentedon Jan 30, 2021
@RalfJung Done in #81565.
If nobody has an idea on how to avoid the linker error, two more options would be to comment out the symbol in our llvm-project fork, or to avoid upgrading clang altogether and just build both Python 2 and Python 3 for that image.
nagisa commentedon Jan 30, 2021
The issue is probably going to be in how exactly the objects that comprise libLLVM.so are compiled, not linked. I suspect
lldjust ends up ignoring the malformed object or otherwise not reporting the issue, but the object and the relevant symbols ultimately end up in thesoanyway.Alas, reasons these obscure failures can occur are many – I've seen them occur from things as trivial as forgetting to
-fPIC.The question is: is there perhaps a way to disable ExecutionEngine entirely? Intuitively nothing we do should depend on that code. And if not, I'm comfortable patching out
__morestackin our fork as we get closer to Feb 11 and continue having trouble pinpointing exact cause of this problem.Auto merge of rust-lang#81565 - nikic:revert, r=nagisa
nikic commentedon Jan 30, 2021
We normally don't link ExecutionEngine, but dist-x86_64-linux in particular builds LLVM with ThinLTO, in which case we link libLLVM.so as shared, instead of statically linking individual components into rustc_llvm.
nikic commentedon Jan 30, 2021
It seems to be possible to configure the components in libLLVM.so using
LLVM_DYLIB_COMPONENTS, but I'm not sure to what degree we can restrict those (not sure if this job also ships LLVM tools).nikic commentedon Jan 31, 2021
So, I think we could have used
LLVM_DYLIB_COMPONENTSto exclude unnecessary parts from libLLVM.so, as LLVM tools can still statically link the parts not in the dylib. But in the end, we can't use the option anyway due to https://bugs.llvm.org/show_bug.cgi?id=48974.matthiaskrgr commentedon Mar 6, 2021
Looks like I'm getting this locally when trying to bootstrap rustc (while building
rustc_driver), might happen since the llvm upgrade but I'm not 100% sure.Are there some workarounds I can put into my
config.tomlfor this?I'm using lld to link.
Backtrace
RalfJung commentedon Mar 6, 2021
@matthiaskrgr which commit are you trying to build?
matthiaskrgr commentedon Mar 6, 2021
@RalfJung 51748a8 latest git master
RalfJung commentedon Mar 6, 2021
Probably best to open a new issue then; this one has been closed and the steps to reproduce it still work without error -- you might be having a different problem.
matthiaskrgr commentedon Mar 6, 2021
Opened #82839
aaronpuchert commentedon Apr 10, 2021
Did anyone file an upstream bug report about this? I'm observing it outside of Rust, but with a similar setup (libLLVM.so with lld+ThinLTO, then linking it into a binary with ld.bfd). If not, I'd try to build a reproducer.
aaronpuchert commentedon Apr 10, 2021
I've opened bug 49915 for this.
aaronpuchert commentedon May 12, 2021
The bug has been fixed, the fix will be in LLVM 12.0.1.
nikic commentedon May 13, 2021
Thanks! We can try updating clang again once the 12.0.1 tag is there.