Repository navigation
libstd: Drop RHEL 5 support, Increase Minimum kernel version to 2.6.27 #62516
Description
The docs currently claim that the minimum Linux kernel version for libstd is 2.6.18 (released September 20th, 2006). This is because RHEL 5 used that kernel version. However, RHEL 5 entered ELS on March 31, 2017.
Should we continue to support RHEL 5 for libstd, or should we increase the minimum Linux Kernel version to 2.6.27 (2nd LTS) or 2.6.32 (RHEL 6, 3rd LTS)? List of the relevant kernel versions.
Even bumping the min-version to 2.6.27 would allow us to remove most of the Linux-specific hacks in libstd. Example: the O_CLOEXEC code.
Reacted by Nick Paladino, Hadrien G., Artyom Pavlov, Ralf Jung, Slanterns and runiq
Reacted by Nick Paladino
Activity
josephlr commentedon Jul 29, 2019
Given that RHEL is the only reason we keep comically old kernel versions around, I would propose that Rust only support RHEL until Maintenance Support ends. This is (essentially) what we already did for RHEL 4. Rust never supported RHEL 4, and set its minimum supported Linux Kernel version back when RHEL 5 still had maintenance support.
It would be nice to get someone from Red Hat or a RHEL customer to comment. This policy would allow us to increment the minimum kernel from 2.6.18 to 2.6.32 (and remove a lot of Linux-specific hacks).
Note that RHEL has a weird (and very long) support system for RHEL 4, 5, and 6 (sources: dates and service details). It has 5 major steps:
Current status of RHEL versions:
tesuji commentedon Jul 29, 2019
cc @cuviper which I think they could be related.
newpavlov commentedon Jul 29, 2019
It also may be worth to drop support of Windows XP and Vista (especially considering that panics are broken on XP since June 2016, see: #34538). Previously it was discussed here.
cc @steveklabnik
briansmith commentedon Jul 29, 2019
Besides
O_CLOEXEC, which other things would we be able to clean up?josephlr commentedon Jul 29, 2019
Good question! There are 6 workarounds in
libstdfor older Linux versions (that I could find). Increasing the minimum version to 2.6.32 (aka 3rd Kernel LTS, aka RHEL 6) would fix 5 of them. Code links are inline:File::open_cuses theO_CLOEXECflag withopen(2)(mentioned above, added in 2.6.23)FileDesc::duplicateusesF_DUPFD_CLOEXECwithfcntl(2)(added in 2.6.24, there is also the mention of a bug occuring on "some linux kernel at some point")pipe::anon_pipeusespipe2(2)to atomically set theO_CLOEXECflag on the pipe fds (added in 2.6.27)Socket::new_rawusesSOCK_CLOEXECwithsocket(2)(added in 2.6.27)Socket::acceptusesaccept4(2)to permit use ofSOCK_CLOEXEC(added in 2.6.28)fs::copyusescopy_file_range(2)(added in 4.5, not fixed by this proposal)As you can see, the workarounds fixed by this proposal all have a similar flavor.
cuviper commentedon Jul 29, 2019
I am Red Hat's maintainer for Rust on RHEL -- thanks @lzutao for the cc. I try to keep an eye on new issues, but this one slipped past me.
Red Hat only ships the Rust toolchain to customers for RHEL 7 and RHEL 8. If our customers would like to use Rust on older RHEL, they can do so via
rustup, and we'll support them in the same way we would for any other third-party software.Internally we do also build and use
rustcon RHEL 6, mostly because it's needed to ship Firefox updates. This is where it gets a little hairy, because each RHEL N is bootstrapped on RHEL N-1 and often kept that way -- meaning a lot of our RHEL 6 builders are still running on a RHEL 5 kernel. I would have to apply local workarounds if upstream Rust loses the ability to run on 2.6.18 kernels. We prefer to keep fixes upstream as much as possible, both for open sharing and to avoid bitrot.Is there much of a benefit to removing the current workarounds? I agree all of that
CLOEXECcode is annoying and cumbersome, but it's already written, and as far as I know it hasn't been a maintenance burden. Do you see otherwise?If there are any known issues with such Linux compatibility code, I am definitely willing to take assignment for fixing them.
alex commentedon Jul 29, 2019
(Not a Rust maintainer, but I'm a Rust user and maintain a lot of other OSS software so I have well-developed feelings around supporting Very Old Software :-))
Do you have any sense of on what timeline Rust would be able to drop support for 2.6.18 kernels without causing you pain?
In general I don't think people mind supporting configurations that have users and are painful to work around, but needing to support them for forever is a bitter pill to swallow! Particularly as they get harder and harder to test over time (already I have no idea how to test on really old kernels besides building it myself). So if there was an estimate "we'd like to be able to support this until Q2 2020", even if it's not set in stone, I think that would be very helpful!
nikic commentedon Jul 29, 2019
@cuviper The other benefit would be that Rust wouldn't have to use CentOS 5 for CI, which means we don't have to patch LLVM (and Emscripten LLVM) to compile on those systems. Of course that's also fairly limited in scope.
cuviper commentedon Jul 29, 2019
@alex
As soon as we stop shipping Firefox and Thunderbird updates on RHEL 6, I won't need Rust there anymore. AFAIK this does correspond to the end of Maintenance Support, approximately November 30, 2020. Then I'll be on to RHEL 7 builders as my minimum, probably still with some RHEL 6 2.6.32 kernels involved.
@nikic
It should be fine for me if we update CI to CentOS 6. This is mostly concerned with how the release binaries are linked to GLIBC symbol versions, which is all in userspace. It's a broader community question whether any other Rust users still care about running on RHEL or CentOS 5.
(Small caveat - glibc support for a symbol can still return
ENOSYS, as noted forpipe2.)Rollup merge of rust-lang#63099 - josephlr:vxworks, r=alexcrichton
josephlr commentedon Jul 29, 2019
@cuviper I noticed that Red Hat also has Extended Life Cycle Support (ELS) for RHEL 6 until June 30, 2024. Will you need RHEL 5 to work during this time? I don't know how ELS works with Firefox updates. Also, is there any reason RHEL 4 support wasn't an issue prior to March 31, 2017 (while RHEL 5 was still normally supported)?
This issue came up for me when dealing with opening files in rust-random/getrandom#58 , see #62082 (comment) for more info. No single RHEL 5 issue is that bad, it's mainly just the sum of a bunch of tiny issues.
cuviper commentedon Jul 29, 2019
I'm not on that team, but AFAIK we don't ship Firefox updates for ELS. The last build I see for RHEL 5 was 45.8.0 on 2017-03-08. Maybe there could be an exception for a severe security issue, but I really doubt we'd rebase to newer Firefox for that, which means Rust requirements won't change.
New Firefox ESR versions do require a newer Rust toolchain too, which is generally why we have to keep up. Otherwise we could just freeze some older compat rustc while upstream moves on.
Rust wasn't required until Firefox 52.
5 remaining items
Rollup merge of rust-lang#63099 - josephlr:vxworks, r=alexcrichton
newpavlov commentedon Jul 30, 2019
I think we should not officially support retired OS versions (well, maybe with some grace period), so I prefer this option. Supporting 14 year old kernels seems a bit too much to me.
@cuviper
Is it possible to apply kernel patches to those builders to add support for functionality like CLOEXEC? Or build Firefox separately on RHEL 6?
cuviper commentedon Jul 30, 2019
@briansmith
I think it won't be fruitful for us to debate the merits of stable enterprise kernels, but no, this is not a possibility. An alternative take is "Red Hat should be responsible for the work in Rust to maintain the support they want" -- here I am, ready and willing.
@newpavlov
I definitely can't change those kernels. The ones that are stuck on N-1 are precisely to avoid rocking the boat, and backporting features is a big change. Some of our arches do eventually update the builders to their matching N kernel, but some don't, and I don't know all the reasons.
If the
CLOEXECworkarounds are removed, then I will have to reapply them in our own builds. This assumes I keep using only our own binaries -- if I ever have to re-bootstrap from upstream binaries for stage0, I would also have to use some interception (likeLD_PRELOAD) to hack in the workarounds. This is all possible, but much worse than the status quo of maintaining support here.josephlr commentedon Aug 1, 2019
@cuviper I think this sounds reasonable. We aren't going to be able to update these kernels, it's just a question of when we would drop support for these OSes. Especially given that we wont need RHEL 5 CI support, I think that leaving things as they are is fine.
I opened this issue to verify two things:
It looks like both these things are true. We should remove RHEL 5 workarounds after November 30, 2020. This issue can be postponed until then.
EDIT: For our use case in
getrandom, we were able to add compatibility for RHEL 5 using two lines of code.RalfJung commentedon Aug 27, 2019
See rust-random/getrandom#58 for a recent case where supporting such ancient kernels required extra manual work (fixed link to the 2 lines mentioned by @josephlr above).
I am not sure how often that comes up, but probably the hardest part is to even notice that it happens -- and even then those codepaths will likely linger untested.
cuviper commentedon Aug 27, 2019
Yes, but this will still be true in general. If the CI kernel is newer than whatever kernel baseline we choose, it will be possible for newer syscalls to pass undetected. I don't know if we have any control over that -- can we choose a particular VM image?
So that leaves us with a stated policy of support. The more that developers are aware of such issues, the better, but I'm the one most likely to notice when my build actually fails. If I come back with a fix, I at least need the project to be receptive.
I haven't been in the habit of building nightly on our RHEL6 builders, but maybe I should! Catching this early is better for everyone involved...
josephlr commentedon Aug 27, 2019
That seems reasonable. How quick is the Rust related bootstrapping? Could it be run automatically (or more frequently)?
This is an interesting point. If I hadn't made that change,
getrandomwould have kept working on RHEL 5, it just would have leaked an FD. I'm not sure how we would even test for this sort of thing. This isn't a RHEL 5 specific concern, I just don't know how Rust tests these types of resource leak issues in general.cuviper commentedon Aug 27, 2019
About 2 hours. Automation is harder, since I believe our build system needs my credentials, but I'll look into what is possible here.
Yeah, that's a lot more subtle than an
ENOSYS!