Tracking Issue for pointer_bytes_offsets

Feature gates:

#![feature(pointer_byte_offsets)]

#![feature(const_pointer_byte_offsets)]

This is a tracking issue for the pointer_byte_offsets raw pointer conveniences like ptr.byte_add(offset)

Public API

impl<T: ?Sized> *const T {
    // feature gates `pointer_byte_offsets` and `const_pointer_byte_offsets`
    pub const unsafe fn byte_offset(self, count: isize) -> Self;
    pub const unsafe fn byte_add(self, count: usize) -> Self;
    pub const unsafe fn byte_sub(self, count: usize) -> Self;

    pub const fn wrapping_byte_offset(self, count: isize) -> Self;
    pub const fn wrapping_byte_add(self, count: usize) -> Self;
    pub const fn wrapping_byte_sub(self, count: usize) -> Self;

    pub const unsafe fn byte_offset_from<U: ?Sized>(self, origin: *const U) -> isize;
}

// ... and the same for` *mut T`

Steps / History

Implementation: Add convenience byte offset/check align functions to pointers #95643
Making byte_offset_from more generic: Make pointer::byte_offset_from more generic #103489
Final comment period (FCP): Tracking Issue for pointer_bytes_offsets #96283 (comment)
Stabilization PR: Stabilize [const_]pointer_byte_offsets #116205

Unresolved Questions

Should these operations actually accomodate DSTs? This seems deeply semantically dubious/broken but idk, you can plausibly use them right if you are extremely careful and understand the implications of stacked borrows for slices/projections.

@RalfJung

To clarify the open question's concerns a bit: if you make a ref-slice and then cast it to a raw-slice, offsetting the raw pointer will not change the slice's len, but you can't actually turn the offset slice back into a ref, because (at least under the strictest rules) the original ref-slice has provenance over only its exact range, and so derived pointers cannot access outside the range. Creating the offset ref-slice is trying to assert provenance over additional bytes you don't have permission to access.

Similarly, trait-object-refs will make you sad if you do that.

This can only be soundly used (meaningfully) if you from_raw_parts the raw-DST from a raw pointer with provenance over the whole "array" you want to stride through.

So like, in general you need to write something like (pseudocode of the general pattern):

let array = [...];

// Steal metadata from an example or make the metadata from whole cloth
let dst = &array[something];
let metadata = get_metadata(dst);

// Now build a raw DST from a pointer to the whole array
let raw_array_ptr = array.as_ptr();
let raw_dst = ptr::from_raw_parts(raw_array_ptr, metadata);

// Yay this actually can be strided through the whole array now
let next_dst = raw_dst.byte_add(offset_to_next_dst);

(CC @RalfJung since this is an interesting stacked borrows example, no need to reply if there's nothing more to say.)

@Gankra

@Gankra actually, I think allowing these methods take pointers to DST unlocks a lot of abilities for custom DSTs. In particular byte_sub is required to implement Arc::from_raw (as it accepts a possibly fat pointer to T that needs to be offseted back to ArcInner<T>).

I want to propose to stabilize these methods! (can someone open an fcp?)

Stabilization report

Implementation History

Initially these methods were implemented in Add convenience byte offset/check align functions to pointers #95643
The only unresolved question raised since then is whatever to allow calling these methods on pointers to unsized types. I propose to resolve this to "yes, we should actually", this will unlock a lot of options for working with unsized types (see expitience report for an example).

API Summary

impl *const T {
    pub const unsafe fn byte_offset(self, count: isize) -> Self;
    pub const unsafe fn byte_add(self, count: usize) -> Self;
    pub const unsafe fn byte_sub(self, count: usize) -> Self;

    pub const fn wrapping_byte_offset(self, count: isize) -> Self;
    pub const fn wrapping_byte_add(self, count: usize) -> Self;
    pub const fn wrapping_byte_sub(self, count: usize) -> Self;

    pub const unsafe fn byte_offset_from<U: ?Sized>(self, origin: *const U) -> isize;
}

// ... and the same for` *mut T`

Experience Report

#99113, #100819 and #100030 refactored some std code into using these methods simplifying the code.

These methods allow custom Arc implementations to work with unsized types just as well as std::arc::Arc, namely they allow to offset a pointer by <size of arc metadata> while keeping pointer metadata, which is required for example for Arc::from_raw.

This also allows to polyfil with_metadata_of by using wrapping_byte_offset, which is again very useful when working with DSTs.

Pondering: does byte_offset_from really need its argument to be a *const T? Arguably it doesn't care what type you pass it, so it could be *const U instead (or *const impl Sized, I suppose).

That would allow things like ptr::addr_of!(x.b).byte_offset_from(ptr::addr_of!(x)) to compile without needing an arbitrary choice of cast.

EDIT: this was updated in #103489

Rollup merge of rust-lang#103489 - WaffleLapkin:byte_offset_from_you,…

I have a question regarding the offset_from family of functions, stabilized or not. In all those cases where it is safe to use, why not use as integer casts, ignoring the fact that conversion from bytes to units of T might be required?

I just had the situation today, where I needed the offset between 2 bytes slices (one is a subslice of the other) and failed to see why I should use the unsafe offset_from. In either case, I have to be able to reason about why my code is correct, but not having to reason about why my code doesn't invoke UB seemed like a +1 for integer casts. Obviously this is just one example, and certainly not "all those cases".

But I'm wondering, what are safe examples in which I should use offset_from as opposed to integer casts?

offset_from (and particularly sub_ptr) allows LLVM to optimize more based on the unsafety. If your code is not a place where you get a material advantage from those extra optimizations, then by all means use the safe phrasings. (sub_ptr is very important for slice::Iter::len, but there's very little code that's anywhere close to as perf-critical as that.)

Another advantage of offset_from is that it can be used in const fn.

For some time we had safe variants of offset_from, maybe it is worth thinking about bringing them back.

Rollup merge of #103489 - WaffleLapkin:byte_offset_from_you, r=scottmcm

rust: sync: add `Arc::{from_raw, into_raw}`

rust: upgrade to Rust 1.75.0

*RFL import: rust: upgrade to Rust 1.75.0

rust: upgrade to Rust 1.75.0

Navigation Menu

Repository navigation

Tracking Issue for pointer_bytes_offsets #96283

Public API

Steps / History

Unresolved Questions

Stabilization report

Implementation History

API Summary

Experience Report

70 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Tracking Issue for pointer_bytes_offsets #96283

Description

Public API

Steps / History

Unresolved Questions

Activity

Gankra commented on Apr 21, 2022

WaffleLapkin commented on Jul 10, 2022

WaffleLapkin commented on Sep 4, 2022

Stabilization report

Implementation History

API Summary

Experience Report

scottmcm commented on Oct 24, 2022

dennis-hamester commented on Jan 20, 2023

scottmcm commented on Jan 21, 2023

RalfJung commented on Jan 23, 2023

70 remaining items

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions