Repository navigation
#[repr(transparent)] for Box<T>? #52976
Description
#43036 added #[repr(transparent)] with one of the goals being safe FFI interaction for newtypes, and added that attribute to bunch of built-in types such as NonZero, Waker, Unique etc.
One use-case that could be really helpful for FFI though is adding it to Box<T>. I haven't found any previous discussions on pros and cons of doing so, so apologies if this is way off and there are obvious reasons why it can't be such, but as far as I can tell, this is not a breaking change neither from API nor from memory representation perspective.
Looking from definition, Box is just a wrapper around Unique which is already #[repr(transparent)] and wraps NonNull which is also #[repr(transparent)] which, in turn, wraps raw pointer which is FFI-safe.
Adding this attribute to Box would make it transparent wrapper all the way down around a raw pointer, and so would allow to do FFI interactions by simply leaking Box as return type in allocator functions and accepting it as-is in deallocator function without manual into_raw and from_raw conversions.
Aside from ergonomics improvement (allocation / deallocation is pretty common operation in cdylibs), this would also allow code to be more self-documenting as just from looking at rustdoc you would be able to tell which pointers are owned and will be allocated or consumed by Rust side.
Counter-argument might be that one can implement custom transparent CBox on top of NonNull, but this involves reimplementing many APIs and guarantees Box already provides e.g. Unique is not exposed at the moment, and then you also have various standard traits that "just work" with Box, so if it would be possible to avoid reimplementing all of that with no obvious downsides, it might be still useful to shared library authors.
Activity
mjbshaw commentedon Aug 5, 2018
How does this interact with
Drop? And how are foreign functions expected to allocate (using Rust's allocator) the boxed memory?CryZe commentedon Aug 5, 2018
They don't. I already do this in my C API even though it's not entirely defined behavior at the moment: https://github.com/LiveSplit/livesplit-core/blob/3ac0834f70a3963f1b7e4fec84b0803c51e6349e/capi/src/fuzzy_list.rs#L12-L24
RReverser commentedon Aug 5, 2018
Yeah, what @CryZe said. This is not for importing foreign functions, this is for easily and safely exposing Rust functions that allocate+return+deallocate
Boxvia C API.glandium commentedon Aug 5, 2018
How would this interact with #50882?
RReverser commentedon Aug 5, 2018
Sadly, they wouldn't be compatible when allocator is not zero-sized. I'm surprised
Boxis going to store allocator by value (it seems so from the change?) even though in theory same allocator state might be reused for many allocations and must be shared between them... Also it seems quite inefficient for most libs/apps out there that use just one allocator. I guess I'm going to need to dig into that PR deeper tomorrow when I'm not asleep.sfackler commentedon Aug 5, 2018
The allocator value that would be used in most cases is
Global, a zero sized type.glandium commentedon Aug 5, 2018
Nothing in
Box<T, A: Alloc>(Unique<T>, A)says the allocator is stored by value. It all depends whatAllocis impl'ed for. It could be a ZST, likeGlobal, or a reference (impl Alloc for &MyAlloc).RReverser commentedon Aug 6, 2018
@sfackler The question is more whether this would be guaranteed for any allocators (I suspect not). In that case,
repr(transparent)would indeed not work, but I suspect there are other parts of compiler that currently rely on lang item behindBoxbeing just a pointer...CryZe commentedon Aug 6, 2018
RReverser commentedon Aug 6, 2018
@CryZe
Sorry, I didn't get it - instead of what?
CryZe commentedon Aug 6, 2018
PhantomData<A>instead of an embedded A, which could not be zero sized.RReverser commentedon Aug 6, 2018
Agreed. IIUC the motivation for storing
Ais to give flexibility and allow several instances of the same allocator, each with its own state, but this looks like quite a niche usecase to be covered by stdlib.RReverser commentedon Aug 6, 2018
Also, I suppose, if allocator does want to share some extra data, it can always allocate pointer with larger size and store that metadata (or a pointer to one) alongside actual data in the heap.
sfackler commentedon Aug 6, 2018
Why does Box need to differ from unique_ptr in this respect?
RReverser commentedon Aug 6, 2018
Do you mean C++ unique_ptr? Why would Box be similar to it? For one, Rust already has lots of differences with C++, including in memory management, and in particular availability of zero-sized types which is what @CryZe and me are suggesting to use for custom allocators.
16 remaining items
gnzlbg commentedon Jul 10, 2019
@RReverser
@Lokathor pointed out, that this "implementation detail" is "documented" in the
repr(transparent)RFC:Box<T>is not user code, but part of the implementation, so it could rely on this, and document that, at least forBox<T>this will always hold.RReverser commentedon Jul 10, 2019
Fine by me - not the outcome I initially hoped for, but it does the trick and reaches same goal anyway :)
SimonSapin commentedon Jul 25, 2019
The goal of using standard library containers with different allocators, including non-zero-size handles, is part of an accepted RFC that precedes the new transparency guarantee being proposed here: https://rust-lang.github.io/rfcs/1398-kinds-of-allocators.html#what-about-standard-library-containers
Ideally we would have
Box<T, A>isrepr(transparent)if and only ifAis zero-size, but as far as I know there is no way to express that in the language today.gnzlbg commentedon Jul 25, 2019
@SimonSapin
This PR solves that: #62514
Once we stabilize Box<T, A> we just need to clarify that that only holds if A is zero-sized, and that's pretty much it AFAICT.
We plan to submit an RFC to guarantee this for all types for which this holds as part of the UCGs RFC. There is a PR in the UCG repo with proposed wording for that as well: rust-lang/unsafe-code-guidelines#164 (is blocked on some nit-picking on how to abbreviate one-aligned ZSTs but there is consensus that we want to guarantee that). So when that happens the documentation comment guaranteeing this wouldn't be technically necessary anymore, since it would be a given, but it would obviously be documentation worth having.
crlf0710 commentedon Aug 17, 2019
I think #62514 is only adding comments, and there seems to be needing language level mechanism if we want to move forward.
gnzlbg commentedon Aug 17, 2019
The comment there guarantees that box can be used in C FFI. Isn't that enough to resolve this?
crlf0710 commentedon Aug 17, 2019
Is it? From FFI ABI perspective, adding those comments doesn't turn ABI for
Boxfrom representing astructto representing apointer, which is all#[repr(transparent)]all about, does it?CryZe commentedon Aug 17, 2019
As long as it is guaranteed and there is some test verifying that it is the case, it doesn't need a special attribute.
gnzlbg commentedon Aug 17, 2019
Box is as special as
&mut Tori32, the compiler already sets the ABI of Box to that of a single pointer. The only thing the comment does is guarantee to users that this will never change (otherwise that would be a breaking change).crlf0710 commentedon Aug 17, 2019
ok, i guess that's the magic of
lang_itemthen. thank you for the elaboration.Rollup merge of rust-lang#62514 - stephaneyfx:box-ffi, r=nikomatsakis
Mark-Simulacrum commentedon Dec 23, 2019
We now have some extensive documentation about use of Box with FFI and generally the properties we guarantee: https://doc.rust-lang.org/nightly/std/boxed/index.html#memory-layout. Closing.