Repository navigation
Representation of bool, integers and floating points #9
Description
This issue is to discuss the memory layout for integral and floating point types:
boolu8..u128,i8..i128usize,isizef32,f64
For the most part, these are relatively uncontroversial. However, there are some interesting things worth discussing:
- Unlike other types, there are no
#[repr(C)]vs#[repr(Rust)]variants here. The size is always fixed and well-defined across FFI boundaries. The types map to their corresponding integral types in the surrounding C ABI. - Prior discussions ([#46156][], [#46176][]) documented bool as a single byte that is either 0 or 1.
- How is
usizeintended to be defined on various platforms?- the native size of a pointer?
- the max of various other considerations?
- other edge cases to consider?
- Rust currently states that the maximum size of any single value must fit in with
isize
- Can we say a bit more about why? (e.g., ensuring that "pointer diff" is representable - Do we want to discuss signaling NaN at all? Specifically: why is it potentially of concern, and are there things that unsafe authors or other folks need to be aware of? (@gankro, for example, wrote that "NaN masking is unnecessary from LLVM's perspective", but I don't really know what that means. =)
Activity
joshtriplett commentedon Aug 30, 2018
Don't forget
u128andi128.Another topic: Does FFI code need to use types like
libc::uint64_t, or is it safe to just useu64?nikomatsakis commentedon Aug 30, 2018
@joshtriplett
Edited, thanks!
Good question! I presume it does not, but I'd be curious if there is another side to the discussion.
*Related note: clearly
usizeand C'sunsizedare not equivalent, but it's worth stating this explicitly.Gankra commentedon Aug 30, 2018
Much of the rust ecosystem (e.g. webrender and therefore firefox) assumes uint64_t and u64 are the same ABI-wise, and I'm unaware of any reason to prevent that assumption.
I believe pointer size is the correct definition but I haven't read the link, so grain of salt
There's no good reason, it's just because llvm has a quirky definition of in-bounds pointer calculations
Signaling NaNs only merit discussion insofar as the IEEE spec defines some random operations to act differently on them. (e.g.
max(sNaN, x) != max(qNaN, x), although iirc this example is regarded as a mistake and is intended to be changed)Signaling itself is, I think, largely a failed experiment and worth ignoring (soft cc on @stephentyrone in case i'm misremembering)
hanna-kruppe commentedon Aug 30, 2018
Wrt signaling NaNs, it's more of a question for the (now deferred, cc #8) discussion of valid values. There's a persistent rumor (including, at times, among LLVM contributors) that handling an sNaN or doing certain operations on it will cause a trap or is undefined behavior in LLVM. This is not the case, but I've encountered enough people thinking it's true that I think it would be best explicitly state that signaling NaN are perfectly fine, and thus that all bit patterns are valid floats.
gnzlbg commentedon Aug 30, 2018
+1 From the point-of-view of just layout, SNaNs are not really that interesting and the easiest thing is to just allow them. AFAIK
f32::from_bits(u32)is safe and stable and works for all bit patterns, so we can't really do much about this anyways without potentially breaking some code.nikomatsakis commentedon Aug 31, 2018
@gankro
The link is to a comment from @gnzlbg and states:
nikomatsakis commentedon Aug 31, 2018
@gankro
Do you think we should write down that this is something that is presently true but which may be changed in the future (so unsafe code should not rely on it being true)? It seems like it might affect quite a bit how one writes code.
nikomatsakis commentedon Aug 31, 2018
(In particular, it seems to imply that it is safe to use
isizefor "pointer offset" within any one value, which is otherwise not necessarily true, right?)Gankra commentedon Aug 31, 2018
I don't think we can ever change it since it's baked into ptr::offset. If we did it would be in a way where negative offsets were a valid very-large-positive offset, so isize would still "work" but be weird.
Gankra commentedon Aug 31, 2018
also fwiw I think gcc also gets sad with huge offsets
gnzlbg commentedon Aug 31, 2018
I think that the bare minimal guarantee here is that the Rust extern "C" function declarations need to use types that match in size and alignment with the types of the C function declaration.
That is, if C uses
uint64_t, then you can useu64,libc::uint64_t, or even i64. In particular,libctypes are not special. When C usesunsigned, then you need to use a 32 or 64 bit type (or something else) depending on the platform you are targeting. Thelibc::uinttype does this correctly for you, but you don't have to use that.That would be the bare minimum, and I think that would already be ok since we are just passing bags of bytes here and it is all unsafe anyways. @mw might know whether this can result in any issues due to, e.g., cross-language inlining.
If we wanted to extend this minimum, we could map the C types to the Rust types, e.g. saying that if a C's function declaration uses
uint64_tthen Rust extern "C" declaration must use a 64-bit wide unsigned integer type. Or if it usesunsignedthat the Rust extern "C" declaration must use an unsigned integer type of the same width. But this opens many questions, e.g., isstruct A(u64)a 64-bit unsigned integer type that I can use where C usesuint64_t? What if I applyrepr(transparent)to it? I'd rather avoid all this.I think if we can get by with only the size and alignment requirements, we should. If someone then uses a
&Tas the return type of an extern "C" function, and the function happens to create an invalid value, then that's UB but that would be covered by a different part of the unsafe code guidelines.36 remaining items
avadacatavra commentedon Oct 18, 2018
should this discussion deal with all scalar types (aka should we include characters in this discussion?)
strega-nil commentedon Oct 18, 2018
@avadacatavra
char [C++]is equivalent to eitheri8/u8(although you can use either for ABI compat);char [Rust]is not ABI compatible with anything (although I'd argue it'd be useful to be ABI compatible withchar32_t) (note: that type only exists in C++).People have argued that they shouldn't be ABI compatible, since
char32_tdoesn't have the correctness guarantees Rust'schardoes; I would argue that it's the same idea as C-likeenums in Rust vsenums in C++.briansmith commentedon Oct 21, 2018
In https://gankro.github.io/blah/rust-layouts-and-abis/#the-layoutsabis-of-builtins, it would be useful to define the ABI correspondence for function parameters
x: &[T; n]and CT x[static n]andT *x. ring is one crate that depends on this correspondence.Gankra commentedon Oct 21, 2018
@briansmith I believe that is implicit in pointer ABI matching and array layout matching. I'm not aware of any system under-which the ABI of a pointer depends on the pointee's type, and array types in function parameters are just sugar for pointers.
gnzlbg commentedon Nov 2, 2018
Summarizing the discussion about
usize/isizeso far, we have already committed to these having the same size as a native pointer and changing that at this point would be a big breaking change.The representation of
usizedetermines many things, like:offseted,size_of/size_of_valreturnusize),[T; N: usize]),usize/isizein C FFI are compatible with C'suintptr_t/intptr_t.etc. We should document these, but they don't change
usize's representation so we don't have to document all of these things right now.This definition would also limit the problematic platforms to those that either do not have a native pointer size (can't think of any) or those that have multiple native pointer sizes (near and far pointers in segmented architectures). I'd say it's ok to worry about them when someone tries to add support for them (for segmented archs one could pick one of the pointer types as "native" and add newer types for the rest).
gnzlbg commentedon Nov 4, 2018
@sfackler
The merged PR specifies
boolto be of size1which is not the same as being C-compatible [0] . The intent seems to have been to guarantee compatibility with C's_Booltype, e.g., @withoutboats called this out in the discussion here:and with more rationale here:
We have to decide whether we want
boolto have the same representation as C's_Booltype (C FFI safe), or whether we want to makeboolhave size1(C FFI unsafe?). We could also makeboolhave size1and be C FFI safe by trading out support for platforms in which_Booldoes not have size1.[0] The MSVC2012 docs mention that MSVC <= 4.2
booltype is 4 bytes wide - link .gnzlbg commentedon Nov 5, 2018
FWIW, this is what @gankro 's document documents (https://gankro.github.io/blah/rust-layouts-and-abis/):
Gankra commentedon Nov 5, 2018
Just to add more context:
MSVC 5.0 fixed the sizeof bool in 1997, meaning this was fixed before windows 98 was released.
Considering windows xp is a tier 3 (cross-compile only) platform because it's so janky and officially unsupported by the vendor, I find it dubious that we have interest in supporting compatibility with pre-windows-98 systems.
Gankra commentedon Nov 7, 2018
Great presentation by JF Bastien on efforts to standardize int/bool repr in C++20:
talk: https://www.youtube.com/watch?v=JhUxIVf1qok
paper: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0907r4.html
TL;DR:
[-]Representation of integers and floating points[/-][+]Representation of bool, integers and floating points[/+]gnzlbg commentedon Mar 14, 2019
I've went through the reference and all the comments:
The reference is not super clear about whether
i64maps toint64_tand what that implies (2's complement, etc.). From the text and the comments here, this was the intent (e.g. one does not have to uselibc::int64_ton C FFI, but can just usei64). So I've just sent a PR updating the reference to be more clear about this: Be more explicit about the layout guarantees of integer and floating-point types #98I've also opened an issue on the
libcrepository about this, and plan to tackle this for thelibcandctycrates: Deprecate the fixed-width integer aliases libc#1304The reference is not super clear about the layout of raw pointers to sized types beyond them being
1word long and whether they can be used in C FFI. Of course they can, but I've opened an issue about this to update the reference: Are raw pointers to sized types usable in C FFI ? #99If you feel I missed anything, it would be better to open an specific issue to discuss that.