Skip to content

mutable_transmutes lint should catch transmutes from a type without interior mutability to one with #111229

Open

Description

@glandium

View all comments

I'm filing this as a regression, although the summary is worded in a feature request-ish way.

So, the regression itself is this: code that transmutes from a type without interior mutability to one with interior mutability leads to really bad outcomes after the bump to LLVM 16, because of the changes in llvm/llvm-project@01859da. This is a pattern that happens in the wild, probably mostly around FFI. At least, that's how it happens in the Firefox codebase.

Code

Here is a reduced testcase using a similar pattern to what Firefox is using:

pub struct Foo(std::cell::UnsafeCell<usize>);

pub struct Bar([u8; 0]);

pub fn foo(f: &Bar) {
    unsafe {
        let f = std::mem::transmute::<&Bar, &Foo>(f);
        *(f.0.get()) += 1;
    }
}

With rustc up to 1.69.0 in --release mode, this produces the following IR:

define void @_ZN10playground3foo17hc0e352349f95f9ecE(ptr noalias nocapture noundef nonnull readonly align 1 %f) unnamed_addr #0 {
start:
  %0 = load i64, ptr %f, align 8, !noundef !2
  %1 = add i64 %0, 1
  store i64 %1, ptr %f, align 8
  ret void
}

With rustc 1.70.0-beta.2 in --release mode, this produces the following IR:

define void @_ZN10playground3foo17h2141d3a0b5fe8d73E(ptr noalias nocapture noundef nonnull readonly align 1 %f) unnamed_addr #0 {
start:
  ret void
}

Note how everything is gone because the input pointer is marked as noalias readonly, and thus the code is not expected to change what it points to, so it's all removed. This is the typical example of undefined behavior leading to the optimizer doing unexpected things. I'm not arguing that there isn't undefined behavior. The undefined behavior existed before. But with LLVM 16, now the undefined behavior is actively dangerous.

Now, to come back to the feature-request-y summary I wrote, the following code does not compile:

pub struct Bar([u8; 0]);

pub fn foo(f: &Bar) {
    unsafe {
        let _ = std::mem::transmute::<&Bar, &mut Bar>(f);
    }
}

The produced error is:

error: transmuting &T to &mut T is undefined behavior, even if the reference is unused, consider instead using an UnsafeCell

I would argue that the code that is now compiled to nothing should also produce a similar error, and that error should be shipped in 1.70.0.

Cc: @pcwalton, @nikic

Activity

  1. added
    C-bugCategory: This is a bug.
    regression-untriagedUntriaged performance or correctness regression.
    on May 5, 2023
  2. added
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on May 5, 2023
  3. apiraino commented on May 17, 2023

    @apiraino
    Contributor

    WG-prioritization assigning priority (Zulip discussion).

    @rustbot label -I-prioritize +P-high +T-compiler +A-LLVM +regression-from-stable-to-beta

  4. added
    A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.
    P-highHigh priority
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    and removed
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on May 17, 2023
  5. ChayimFriedman2 commented on May 17, 2023

    @ChayimFriedman2
    Contributor

    @rustbot claim

  6. glandium commented on Jul 14, 2023

    @glandium
    ContributorAuthor

    FWIW, we've hit yet another case of such a transmute that a lint would have made obvious in the first place...

  7. glandium commented on Jul 14, 2023

    @glandium
    ContributorAuthor

    While trying to find workarounds, I also figured out that the same footgun can be triggered with "manual" transmutes too... e.g.

    let f = std::ptr::NonNull::from(f).cast::<Foo>().as_ref();
    

    or

    let f = (f as *const Bar as *const Foo).as_ref().unwrap();
    

    I don't know if they exist in the wild, but a lint would sure be useful for those too.

  8. 24 remaining items

  9. ChayimFriedman2 commented on Nov 30, 2024

    @ChayimFriedman2
    Contributor

    @dev-ardi I already have a working PR for this (#128351), don't know why I was unassigned.

  10. jieyouxu commented on Dec 1, 2024

    @jieyouxu
    Member

    My apologizes, I have no idea why I unassigned when I even did a try job on the PR (#128351)

  11. added
    A-lintsArea: Lints (warnings about flaws in source code) such as unused_mut.
    and removed
    A-lintsArea: Lints (warnings about flaws in source code) such as unused_mut.
    on Dec 21, 2024
  12. apiraino commented on Jun 19, 2026

    @apiraino
    Contributor

    Visited during T-compiler P-high review#t-compiler/meetings > [triage] 2026-06-19 P-high review @ 💬

    Downgrading and checking status of #143343

    @rustbot label -P-high +P-medium

  13. added and removed
    P-highHigh priority
    on Jun 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-LLVMArea: Code generation parts specific to LLVM. Both correctness bugs and optimization-related issues.A-lintsArea: Lints (warnings about flaws in source code) such as unused_mut.C-bugCategory: This is a bug.P-mediumMedium priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions