prevent unwinding past FFI boundaries in code generation

It's undefined to unwind past an FFI boundary such as a pub extern "C" fn. Code generation should automatically insert a landing pad doing an abort. This will eliminate the class of memory safety errors resulting from unwinding into C from Rust. LLVM will be able to optimize it out if it is being caught and handled explicitly, such as to translate into an error code for C.

EDIT: Mentoring instructions can be found here.

Assigning P-high, not 1.0.

Triage: I'm not aware of a code change here, I believe that we are expecting people to handle this themselves with catch_panic or whatever it's called.

/cc @rust-lang/libs @rust-lang/lang is that the only solution we are pursing here, or do we plan on doing it automatically eventually?

@steveklabnik

@steveklabnik IIRC, there's still work to do to ensure that you correctly get an abort if you don't recover from the panic before hitting the boundary. I believe that's what this issue is about.

@ranma42

cc @ranma42

@brson

Has there been any work done on this? Automatically inserting catch_unwind statements at FFI boundaries that automatically abort on uncaught panics seems like a straightforward way to close this soundness hole. I see where @brson silently adjusted the tags, so maybe the core team reviewed it back in August.

@coder543

UPDATE: This comment appears to have been a bit confused! See revised instructions below.

No work has been done on this to my knowledge, but I am in favor of the solution you proposed @coder543, and would be happy to work with you on implementing it. If nothing else, it would help us gain experience with overhead etc.

I think that the strategy we would use is to modify the code that generates MIR (our mid-level intermediate IR) for calls, which is found in this file here. You can see that it generates the code to execute upon unwind right here, on this line -- currently, this is always cleanup code. We could modify it to inspect the type of the value being invoked (and, in particular, its ABI) and generate alternate unwind code that aborts (I'm not sure the best way to encode an abort right now though).

The function being called is stored in the fun variable, which is an Operand<'tcx>. Operands support a ty method (defined here). In principle, I suppose, we could call it like fun.ty(&this.local_decls, this.hir.tcx()) -- that method is only meant to be called after MIR construction is complete, but I think it will actually work just fine. (Otherwise, we could extract the type of the function being called from the HAIR, which is the intermediate representation we are building things from; but that's just a touch more involved.)

The main question then is how to generate MIR that aborts. Right now I think we have no way to encode an abort into the IR. We could probably add an ABORT terminator (i.e., a new variant of the MIR data structure TerminatorKind, found in src/librustc/mir/mod.rs).

So, if you were going to implement this, I think we would do the following steps:

Add an ABORT terminator to TerminatorKind and work through the various implications. That could be a PR by itself, although it'd be presently unused.
Inspect the types of callees as described above, find those with C ABIs, and generate an ABORT terminator instead.

That might be two PRs, or maybe one. To get started, one might also skip the first step, and just experiment with having the compiler print out a match when it finds a C ABI function, and then worry about how to handle it.

@iainnicol

@iainnicol

My use case is that I would like to be able to catch Rust panics from the C/C++ side.

I have two concerns about this approach: 1) does it really work in practice, and if so what kind of exception is a Rust exception in C++ and 2) why can't you use catch_unwind?

@nikomatsakis

@nikomatsakis or anyone who would like to provide mentoring/review/feedback:

So I made some progress today! (Yay!) As can be seen in diwic@729092c

It's probably not ready for PR yet, could use some mentoring / feedback:

Should tests be added and if so how can one test that an abort has happened? As of now I have only manually inspected mir and llvm-ir to see that it looks correct.
I'm not sure why my is_foreign_item row is not working, I think that would be better if we could use it
For all places where the compiler complained about a missing TerminatorKind::Abort branch, I basically added one that did the same as TerminatorKind::Resume (except for code generation). This might need to be double checked due to behavioral differences (after all, resume resumes and abort aborts).
Not sure what to do on the mir inlining pass. But it does not seem to happen in my small tests, maybe this is something experimental which is not enabled by default?

@diwic

@diwic

Should tests be added and if so how can one test that an abort has happened? As of now I have only manually inspected mir and llvm-ir to see that it looks correct.

Yes! Tests should definitely be added. One way to do this is to write a run-pass test that invokes itself using Command, passing some arguments and then checking the return code. sigpipe-should-be-ignored.rs is an example of a test that uses that technique.

I'm not sure why my is_foreign_item row is not working, I think that would be better if we could use it

Can you say a bit more, I'm not sure what that means?

For all places where the compiler complained about a missing TerminatorKind::Abort branch

OK, I'll double-check when you open a PR.

Not sure what to do on the mir inlining pass. But it does not seem to happen in my small tests, maybe this is something experimental which is not enabled by default?

It is experimental and does not happen by default. You can enable it with -Zmir-opt-level=2. It would be great if you did test it -- I think you should be able to "link up" the abort block from the inlined function into the abort block in the caller, presumably handling it in a similar fashion to how resume is handled or something...I'll have to go look in more depth.

In general, it'd be great if you wanted to open a PR and we can discuss in more depth there! Feel free to tag it with [WIP] in the subject line, and be sure to put r? @nikomatsakis so that it gets assigned to me.

Second attempt here: #46833

Mir: Abort on nounwind ABIs

Mir: fixup nits in previous commit (f536143)

@diwic

Thank you so much for pushing on this old bug @diwic!

@iainnicol

@iainnicol

Would it be possible to opt out of the abort generation? Perhaps by tagging the function with the existing #[unwind] attribute?

JFTR, as implemented, tagging the function with the #[unwind] attribute does opt out of the abort generation.

re-opening as 1.24.1 removed this behavior, even though it's expected to come back soon.

rlua should be added to https://github.com/rust-lang/rust/blob/master/src/tools/cargotest/main.rs

Closing this in favor of #52652 (which is more recent and active).

Oops, sorry I didn’t find this when opening #52652 !

Navigation Menu

Repository navigation

prevent unwinding past FFI boundaries in code generation #18510

22 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

prevent unwinding past FFI boundaries in code generation #18510

Description

Activity

pnkfelix commented on Nov 6, 2014

steveklabnik commented on Dec 31, 2015

aturon commented on Dec 31, 2015

ranma42 commented on Dec 31, 2015

coder543 commented on Jul 10, 2017

nikomatsakis commented on Jul 17, 2017

22 remaining items

diwic commented on Oct 7, 2017

diwic commented on Oct 7, 2017

nikomatsakis commented on Nov 22, 2017

diwic commented on Dec 19, 2017

bstrie commented on Dec 25, 2017

diwic commented on Dec 25, 2017

steveklabnik commented on Mar 2, 2018

nox commented on Mar 2, 2018

Mark-Simulacrum commented on Jul 29, 2018

SimonSapin commented on Jul 29, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions