How does MMIO memory fit into the memory model, and can reference point to it?

This question is intended to replace #33 and #265, and ask the general question of whether or not we can have a type, either user-defined or language-provided, that, when wrapped in a shared reference, guarantees that no accesses are introduced at the operational semantics level that are not part of the original program (and thus, if volatile accesses only are used, an implementation won't introduce any speculative reads).
If such a type can exist, the second question is what is the operational semantics of retagging as a shared reference to such a type.

I think the question whether we "can" have is fairly straight-forward -- yes, we can totally do that. I think it wouldn't be too hard either, operationally speaking. For instance we could say that &T where T contains a VolatileCell is exempt from all forms of dereferencability requirements. Most likely those requirements will stem from the aliasing model, not any kind of validity of value representations, but at least in Tree Borrows it would be trivial to skip any kind of "fake read" that might otherwise check a reference for being dereferenceable. (In fact right now Tree Borrows already skips the fake read for &!Frozen, and relies on other parts of Miri checking dereferenceability, but in MiniRust that would not work.)

For UnsafeCell there is a lot of demand for more precise tracking, reflexing in the opsem that only parts of memory allow shared mutation. Does that even make sense for VolatileCell? When doing MMIO, I assume it's always the entire memory block that needs to be treated volatile, and there's no other non-volatile memory behind the same reference?

Is this something that is actually needed?

The main use case is making "register layout structs" in Peripheral Access Crates (PACs):

struct MyRegisterBlock {
   foo: VolatileCell<u32>,
   bar: VolatileCell<u32>,
   baz: VolatileCell<u32>,
}
struct VolatileCell<T: Copy>(UnsafeCell<T>);
impl<T> VolatileCell<T> {
   fn read(&self) -> T { self.0.get().read_volatile() }
   fn write(&self, val: T) { self.0.get().write_volatile(val) }
}

let regs: &MyRegisterBlock = &*(0x4000_0000 as *const MyRegisterBlock);
regs.foo.write(0x42);

However, you can accomplish the same thing by staying entirely within raw pointers:

struct MyRegisterBlock(*mut u32);
impl MyRegisterBlock {
   fn foo(&self) -> Reg<u32> { Reg(self.0.add(0)) }
   fn bar(&self) -> Reg<u32> { Reg(self.0.add(1)) }
   fn baz(&self) -> Reg<u32> { Reg(self.0.add(2)) }
}
struct Reg<T: Copy>(*mut T);
impl<T> Reg<T> {
   fn read(&self) -> T { self.0.read_volatile() }
   fn write(&self, val: T) { self.0.write_volatile(val) }
}

let regs: MyRegisterBlock = MyRegisterBlock(0x4000_0000 as _);
regs.foo().write(0x42);

Pros and cons:

The latter clearly avoids all issues with speculative reads, since it never creates references.
They're both equally ergonomic to use. The only change is .foo vs .foo().
The former is shorter to write: you get rustc to lay out the struct for you, instead of having to do pointer maths. However, I don't think this is an advantage, because PACs are usually autogenerated with tools like svd2rust or chiptool, rarely written by hand.
The struct approach breaks down anyway if registers overlap (can happen if same address has different meaning based on some mode, or if the same reg can be accessed as e.g. u16 and u32). svd2rust has code to detect that, and fall back to generating code similar to the raw pointer approach.

So overall, I don't think there's any reason to prefer the struct approach.

in Embassy we've been using PACs generated by chiptool, which generates code using raw pointers, out of concerns for the dereferenceable / speculative reads issue. It's been working well so far, we haven't found anything that it can't do that the struct approach could.

So IMO it's not worth it to add exceptions to the memory model to support usages like VolatileCell. It's not needed, you can do everything fine without.

@Lokathor

I'm an outsider here and totally happy with not having VolatileCell, if the domain experts say that the status quo is totally sufficient.

@Lokathor has opinions on this topic, IIRC.

I specifically would like VolatileCell as it does make exposing MMIO via magic symbols (rather than magic addresses) a bit more ergonomic IMO.

@phil-opp

My opinions are:

The voladdress crate has types that store usize+PhantomData<T> because that makes it easy to work with in const contexts. It turns the usize into an actual pointer at the last possible moment, allowing almost everything except the read and write ops themselves to be const fn.
My crate works extremely well for me personally. I'm entirely satisfied with it. If you want to find someone who needs more from the language than what is currently possible you'll have to look around. Perhaps @phil-opp has a strong opinion? They're in charge of the volatile crate and have been tinkering with the design of a next version for a while now I think.

@Dirbaio I think what people "really want" is something that also has magical field projection so that you can do code like this:

#[repr(C)]
pub struct Controls {
  display_control: u16,
  display_status: u16,
  vcount: u16,
}

pub const MMIO: &VolatileCell<Controls> = whatever_expression_here!(0x0400_0000);

fn main() {
  // magical field projection from `&VolatileCell<Controls>` into `&VolatileCell<u16>`
  let display: &VolatileCell<u16> = &MMIO.display_control;

  // call some method to assign some value
  display.write(1);

  // or we can chain the expression
  let y: u16 = MMIO.vcount.read();
}

I think without some sort of field projection thing added to the language it's less compelling, though possibly still useful even then. Also note that the above example doesn't bother with when things are unsafe or not, which things are readable or writable, etc. There's a lot of design that can go into an mmio abstraction type.

@chorman0773 can you say more about that linker stuff? I think you told me once but I've forgotten if you did, and more details about that might make a more compelling case for needing VolatileCell or might explain what constraints might apply.

can you say more about that linker stuff

Yes. In SNES-Dev, I use linker scripts to define access to the MMIO registers. For stuff that is specific to SNES-Dev, the address is even actualy known at compile time, as it is generally assigned by the linker itself, and given a special type in the mapping table.

That part sounds similar to GBA dev, but with GBA it's generally done with just const values, no linker involved. I'm unclear on why you'd want to have the linker involved at all.

@Dirbaio

@Dirbaio in your example, MyRegisterBlock is no longer a ZST. Can that approach work without holding on to the pointer?

Thanks for the mention!

As far as I know, volatile cells are still commonly used in the Rust embedded an OS ecosystems. For example:

Many popular projects still depend on the volatile_register crate, even though it is not suitable for MMIO.
- The cortex-m uses volatile_register in it's public API, e.g. for accessing SysTick.
- Also used in riscv crate, but I think only internally.
The Tock project defines its own VolatileCell type.
The Theseus project also defines a custom cell-like type based on an older version of the volatile crate.

Most of these projects are aware that this approach is not sound, but still haven't changed their code. I assume that it's a mix of personal preference (structs are much easier to write than doing manual pointer arithmetic) and migration cost (requires breaking changes across the entire ecosystem). So I think that there is definitely demand for a sound VolatileCell type.

@Dirbaio

@Dirbaio

The former is shorter to write: you get rustc to lay out the struct for you, instead of having to do pointer maths. However, I don't think this is an advantage, because PACs are usually autogenerated with tools like svd2rust or chiptool, rarely written by hand.

I agree that it's less of an issue when the pointer arithmetic is generated by tools. However, I think most people write things by hand first and then create tools to autogenerate the boilerplate code later using the same design. So the design that is easier to write by hand is often used for code generation too. For example, I think svd2rust generates code that is based on the volatile_register crate, at least for Cortex M.

The voladdress crate has types that store usize+PhantomData<T> because that makes it easy to work with in const contexts. It turns the usize into an actual pointer at the last possible moment, allowing almost everything except the read and write ops themselves to be const fn.

That is not compatible with CHERI, right? For CHERI you did at least have to store a raw pointer to preserve the full capability.

@thejpster

@thejpster: in your example, MyRegisterBlock is no longer a ZST. Can that approach work without holding on to the pointer?

in the struct approach, you use &MyRegisterBlock, which is pointer-sized.
in the raw pointer approach, you use MyRegisterBlock, which is pointer-sized.

a HAL wouldn't store a MyRegisterBlock, just like it doesn't store a &MyRegisterBlock with today's PACs. They store zero-sized owned singletons instead, that can be deref'd / converted to a &MyRegisterBlock/MyRegisterBlock. Both approaches are equally memory-efficient and generate the same ASM (I've personally checked with svd2rust vs chiptool)

@phil-opp Most of these projects are aware that this approach is not sound, but still haven't changed their code. I assume that it's a mix of personal preference (structs are much easier to write than doing manual pointer arithmetic) and migration cost (requires breaking changes across the entire ecosystem). So I think that there is definitely demand for a sound VolatileCell type.

from the Embedded WG's perspective, we were definitely aware of this and studying possible solutions (it's one of the reasons chiptool exists for example). The push for it has essentially stopped once rust-lang/rust#98017 was merged, which meant VolatileCell is no longer unsound in the current implementation. It was not an "okay, now let's push to make VolatileCell sound in the memory model" decision, it was more like "okay, it's less urgent now, and we don't have much manpower so let's leave it for now".

Moving to pointers is still something I'd personally like to see done, due to the other advantages I mention in this thread. (this is my personal opinion though, not the WG's)

@phil-opp: I agree that it's less of an issue when the pointer arithmetic is generated by tools. However, I think most people write things by hand first and then create tools to autogenerate the boilerplate code later using the same design. So the design that is easier to write by hand is often used for code generation too. For example, I think svd2rust generates code that is based on the volatile_register crate, at least for Cortex M.

my experience is the contrary, everyone starts adding support for a new chip by grabbing a .svd file from the vendor and running it through svd2rust. .svd files are widely available: ARM forces vendors to provide .svd's and it's become some sort of de-facto standard so many risc-v vendors also offer them.

I disagree reg structs are easier to write by hand, too. Documentation always says "register FOO is at offset 0x4c". Translating that to pointer math is trivial, you just add 0x4c. Translating that to a struct you find yourself counting registers manually to infer at which offset each field is, and manually calculating sizes of dummy padding hole arrays.

I think people write registers structs simply because that's how it's always been done in C. There's a good reason to use structs in C: you can use fields with regs->foo syntax, but not methods with regs->foo() syntax. So you can't do the "raw pointers + methods" approach in C. People come to embedded Rust come from C, so that's the first thing they tried in Rust, it somewhat worked, and then it stuck despite the soundness concerns.

That is not compatible with CHERI, right? For CHERI you did at least have to store a raw pointer to preserve the full capability.

You may be right. I've never seen a hardware manual for any CHERI device.

The push for it has essentially stopped once rust-lang/rust#98017 was merged, which meant VolatileCell is no longer unsound in the current implementation.

That is concernying from a t-opsem perspective, this is certainly not the outcome we hoped for when resolving the Arc issue that way. :/

Anyway, from my perspective -- what's missing here is a VolatileCell RFC. This doesn't seem like a gap in the language that needs solving to gain some crucial expressiveness, so I don't have writing such an RFC anywhere on my own roadmap. t-opsem would be involved only insofar as we'd make the memory model actually support the desired semantics, but until such an RFC is accepted, I think this is S-status-not-opsem.

since we don't have volatile atomic operations, you'd need a non-Copy VolatilePtrMut in order to build abstractions that allow safe volatile writes.

The current implementation guarantees volatile ops up to a single word are compiled to a single LDR or STR instruction, so they're already atomic. I'm not sure if rust opsem formally guarantees this or not? if not, it should. Otherwise pretty much all existing MMIO code is broken.

So, as long as you only write words (which is usually the case when you do MMIO) you don't need exclusive access to make it sound, it's not UB. (with some caveats, you need extra conditions so it's not UB from the hardware, such as making accesses in uncahced device memory to avoid cache coherency issues. If you're doing MMIO you already fulfill these conditions)

From the std docs:

Just like in C, whether an operation is volatile has no bearing whatsoever on questions involving concurrent access from multiple threads. Volatile accesses behave exactly like non-atomic accesses in that regard. In particular, a race between a write_volatile and any other operation (reading or writing) on the same location is undefined behavior.

Not sure if this can be removed and things will "just work"

Yeah those docs are kinda old and conservatively designed, they're more like a lower bound on what's possible to be confident about right now.

We could definitely work to improve the situation.

It's probably worth noting as an aside that it may be desirable to create an exclusive borrow of a MMIO register for API design reasons (statically enforcing invariants etc.)

Honestly we should make creation of the VolatileWhatever be unsafe so that usage can be kept safe as often as possible. That plus an atomic flag or whatever and you've got your "lock this device" api designed already.

The current implementation [guarantees](https://llvm.org/docs/LangRef.html#volatile-memory-accesses) volatile ops up to a single word are compiled to a single LDR or STR instruction, so they're already atomic.

You misunderstood what "atomic" means. It is not about how many asm instructions are emitted, it is about how it interacts with the Rust memory model. There have been plenty of discussions around volatile atomic on this issue tracker, please have a look at the context here.

I meant "single ldr/str" instruction does make it "atomic" in the sense as "two racing word-sized reads/writes to the same memory address are not UB". So, similar to Relaxed atomics. This guarantee is enough to make register access APIs safe without needing exclusive/mutable references or owned singletons.

and about "how it interacts with the Rust memory model", the answer is just "it doesn't"? you never mix volatile accesses and regular accesses to MMIO registers. If you do want it to interact then yes you do need "real" volatile atomics. (for example do regular writes to a buffer then do an volatile atomic Release write to a DMA register to ensure DMA start happens after the buffer is filled)

@bravely-beep

Yeah those docs are kinda old and conservatively designed, they're more like a lower bound on what's possible to be confident about right now.

We could definitely work to improve the situation.

The docs still reflect the current status though. We'd have to change our LLVM codegen to make these operations actually atomic, but that doesn't work with our current API -- my hope is that we can use bytewise atomic volatile memcpy once rust-lang/rfcs#3301 becomes a thing.

I meant "single ldr/str" instruction does make it "atomic" in the sense as "two racing word-sized reads/writes to the same memory address are not UB". So, similar to Relaxed atomics.

What the hardware does is not the point. These operations are not marked as atomic in the memory model so they must not race.

It may be possible to carve out an exception for locations which are never pointed-to by a reference and never accessed using non-volatile loads/stores. (Not sure if we have to worry about inbounds pointer arithmetic.) We don't have a guarantee from LLVM for this case AFAIK though. And as @bravely-beep noted, such accesses can still not establish synchronization the way atomic accesses can.

At this point, don't we already have a way to disable dereferenceable, MaybeDangling?

Can't VolatileCell just be implemented as a repr(transparent) wrapper on MaybeDangling once the latter is implemented?

Assume these properties:

No magic conversion of dereferences into volatile accesses. You would have to explicitly call load() and store() methods. These would start by casting self to a raw pointer, then perform the corresponding volatile ops.
There could also be field projection methods. These would start by casting self to a raw pointer, then compute &raw self_raw.foo, then cast back to &VolatileCell of the field type.
None of the methods would ever create references directly to the interior. They would only create &VolatileCell references.
&mut VolatileCell is never exposed to safe Rust code, at least not covering actual MMIO regions (though the user could create their own VolatileCell in normal memory and take a mutable reference to it; this is harmless).

I don't see where this fails, but maybe I'm missing something.

At this point, don't we already have a way to disable dereferenceable, MaybeDangling?

I mean, you could use MaybeDangling<&mut Mmio> (once that type is implemented), but at that point NonNull<Mmio> seems simpler?

Can't VolatileCell just be implemented as a repr(transparent) wrapper on MaybeDangling once the latter is implemented?

No, that's not at all the same thing. MaybeDangling<T> removes the dereferenceability required from references in T; VolatileCell<T> would remove dereferenceability for references to this type.

@Darksonn

We didn't touch much on this in the volatile 2.0 meeting at the all-hands, but I did talk about it briefly with @Darksonn, and our consensus was that if we can have field projections (which we want anyway!), then there's no strong need for VolatileCell -- people can then write a VolatileRef wrapper around a raw pointer with reasonable ergonomics, if that's the best API for their use-case.

If using a "VolatileRef" wrapper is the recommended route here, it's worth considering the ergonomics of reborrowing. My main use-case for a potential &mut VolatileCell<T> is that it would allow easy automatic reference reborrowing. Reborrows could be implemented on a wrapper type but they'd have to be manually-called methods, which significantly hurts ergonomics.

FWIW, I've gone with "use procmacro to make functions that do field projections to make MMIO structs work", and Google went with "use macro! to do field protections to make MMIO structs work".

Both seem workable.

See https://docs.rs/derive-mmio and https://docs.rs/safe-mmio.

It has occurred to me that with #256 (which we plan to support anyway, most likely) you can implement VolatileCell today, as:

pub struct VolatileCell<T>(PhantomData<T>);

That is, a ZST. It's still a reference and the compiler may read from it, but only zero bytes so not affecting MMIO. With #256, we can also read from it (using volatile access of course) by converting &self into a raw pointer.

Navigation Menu

Repository navigation

How does MMIO memory fit into the memory model, and can reference point to it? #411

28 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

How does MMIO memory fit into the memory model, and can reference point to it? #411

Description

Activity

RalfJung commented on Jun 6, 2023

Dirbaio commented on Jun 6, 2023

RalfJung commented on Jun 6, 2023

chorman0773 commented on Jun 6, 2023

Lokathor commented on Jun 7, 2023

chorman0773 commented on Jun 7, 2023

Lokathor commented on Jun 7, 2023

thejpster commented on Jun 7, 2023

phil-opp commented on Jun 7, 2023

phil-opp commented on Jun 7, 2023

bjorn3 commented on Jun 7, 2023

Dirbaio commented on Jun 7, 2023

Lokathor commented on Jun 7, 2023

RalfJung commented on Jun 7, 2023

28 remaining items

Dirbaio commented on Jan 8, 2025

bravely-beep commented on Jan 8, 2025

Lokathor commented on Jan 8, 2025

bravely-beep commented on Jan 8, 2025

Lokathor commented on Jan 8, 2025

RalfJung commented on Jan 8, 2025

Dirbaio commented on Jan 8, 2025

RalfJung commented on Jan 8, 2025

comex commented on Jan 12, 2025

RalfJung commented on Jan 14, 2025

RalfJung commented on May 20, 2025

bravely-beep commented on Jul 2, 2025

thejpster commented on Jul 3, 2025

ChayimFriedman2 commented on Jul 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions