Skip to content

transmute in safe code via a dyn type that rustc incorrectly believes to be impossible #164521

Open

Description

@jsnell

In the following code:

#![forbid(unsafe_code)]
#![allow(dead_code)]
// `Sub` has a where clause that holds for `S` but not for `dyn Sub<..>` itself.
// Codegen then takes `dyn Sub<..>` for a type that "can never be constructed"
// and stops computing vtable slots for it.
trait Holds {}
impl<'a> Holds for &'a S {}

trait A<T> { fn a(&self, t: T) -> T; }
trait B<T, U> { fn b(&self, t: T) -> U; }
trait Sub<T, U>: A<T> + B<T, U> where for<'a> &'a Self: Holds {}

struct S;
impl<T> A<T> for S { fn a(&self, t: T) -> T { t } }
impl<T, U> B<T, U> for S { fn b(&self, _: T) -> U { unreachable!() } }
impl<T, U> Sub<T, U> for S {}

// Should load the `B` vptr out of the `Sub` vtable. Keeps the `Sub` vtable instead.
#[inline(never)]
fn upcast<'a, T, U>(x: &'a dyn Sub<T, U>) -> &'a dyn B<T, U> { x }

// Slot 3 of the `Sub` vtable is `A::a`, so `.b(t)` runs `A::a` and hands `t` back as a `U`.
fn transmute<T, U>(t: T) -> U { upcast::<T, U>(&S).b(t) }

pub fn main() {
    let x = 5u8;
    let addr: usize = transmute(&x);
    println!("&u8 as usize: {addr:#x}");
    let r: &u64 = transmute(0x10usize);
    println!("{r}"); // reads address 0x10
}

I expected to see this happen: a runtime panic

Instead, this happened: segmentation fault from dereferencing 0x10

Related, but less exciting and with a different root cause:

#![allow(dead_code)]
// Second route, no upcast: a call of a supertrait method on `dyn Sub`.
// The first slot of `A` in the vtable is reported as 0, so `a` (third method) is
// looked up in slot 2, which holds the alignment of `S`.
trait Holds {}
impl<'a> Holds for &'a S {}
trait A { fn a0(&self) {} fn a1(&self) {} fn a(&self) -> u32; }
trait Sub: A where for<'a> &'a Self: Holds {}
struct S;
impl A for S { fn a(&self) -> u32 { 7 } }
impl Sub for S {}
#[inline(never)]
fn call(x: &dyn Sub) -> u32 { x.a() }
pub fn main() {
    println!("{}", call(&S));
}

I expected to see this happen: prints 7

Instead, this happened: segmentation fault

Meta

Nightly: https://rust.godbolt.org/z/M8Wxz6qjh
1.99.0: https://rust.godbolt.org/z/nxM1We8G5

This bisects to 1.86.0, Claude is guessing it's due to #136311 based on the title, I have not verified that.

Activity

  1. added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Oct 18, 2026
  2. clubby789 commented on Oct 18, 2026

    @clubby789
    Contributor

    Confirmed manually that this is #136311

  3. added
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/Soundness
    on Oct 18, 2026
  4. added
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on Oct 18, 2026
  5. added
    A-dyn-traitArea: trait objects, vtable layout
    T-typesRelevant to the types team, which will review and decide on the PR/issue.
    and removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Oct 18, 2026
  6. jieyouxu commented on Oct 19, 2026

    @jieyouxu
  7. zachs18 commented on Oct 20, 2026

    @zachs18
    Contributor

    You can also get an ICE out of this if you remove #[inline(never)] from upcast with optimizations enabled. (godbolt link)

    @rustbot label +I-ICE

    repro that causes internal compiler error
    #![forbid(unsafe_code)]
    #![allow(dead_code)]
    // `Sub` has a where clause that holds for `S` but not for `dyn Sub<..>` itself.
    // Codegen then takes `dyn Sub<..>` for a type that "can never be constructed"
    // and stops computing vtable slots for it.
    trait Holds {}
    impl<'a> Holds for &'a S {}
    
    trait A<T> { fn a(&self, t: T) -> T; }
    trait B<T, U> { fn b(&self, t: T) -> U; }
    trait Sub<T, U>: A<T> + B<T, U> where for<'a> &'a Self: Holds {}
    
    struct S;
    impl<T> A<T> for S { fn a(&self, t: T) -> T { t } }
    impl<T, U> B<T, U> for S { fn b(&self, _: T) -> U { unreachable!() } }
    impl<T, U> Sub<T, U> for S {}
    
    // Should load the `B` vptr out of the `Sub` vtable. Keeps the `Sub` vtable instead.
    //#[inline(never)]
    fn upcast<'a, T, U>(x: &'a dyn Sub<T, U>) -> &'a dyn B<T, U> { x }
    
    // Slot 3 of the `Sub` vtable is `A::a`, so `.b(t)` runs `A::a` and hands `t` back as a `U`.
    fn transmute<T, U>(t: T) -> U { upcast::<T, U>(&S).b(t) }
    
    pub fn main() {
        let x = 5u8;
        let addr: usize = transmute(&x);
        println!("&u8 as usize: {addr:#x}");
        let r: &u64 = transmute(0x10usize);
        println!("{r}"); // reads address 0x10
    }
    thread 'rustc' (4) panicked at /rustc-dev/b6eeb93c992027a03e1fd120b6652546396b142c/compiler/rustc_const_eval/src/interpret/cast.rs:500:21:
    assertion failed: &vtable_entries[..vtable_entries_b.len()] == vtable_entries_b
    stack backtrace:
       0:     0x717f66689c16 - <<std[2197041a2d5adeae]::sys::backtrace::BacktraceLock>::print::DisplayBacktrace as core[b29223e0788b9091]::fmt::Display>::fmt
       1:     0x717f66e104e9 - core[b29223e0788b9091]::fmt::write
       2:     0x717f64a8a8bc - <std[2197041a2d5adeae]::sys::stdio::unix::Stderr as core[b29223e0788b9091]::io::write::Write>::write_fmt
       3:     0x717f6665a94a - std[2197041a2d5adeae]::panicking::default_hook::{closure#0}
       4:     0x717f6667d595 - std[2197041a2d5adeae]::panicking::default_hook
       5:     0x717f65406b29 - std[2197041a2d5adeae]::panicking::update_hook::<alloc[65095dd98ba263e9]::boxed::Box<rustc_driver_impl[fa7a7ce11be7f74c]::install_ice_hook::{closure#1}>>::{closure#0}
       6:     0x717f6667da32 - std[2197041a2d5adeae]::panicking::panic_with_hook
       7:     0x717f6665aa34 - std[2197041a2d5adeae]::panicking::panic_handler::{closure#0}
       8:     0x717f666510a9 - std[2197041a2d5adeae]::sys::backtrace::__rust_end_short_backtrace::<std[2197041a2d5adeae]::panicking::panic_handler::{closure#0}, !>
       9:     0x717f6665c72d - __rustc[53ccf4bf3c630fc5]::rust_begin_unwind
      10:     0x717f624531ec - core[b29223e0788b9091]::panicking::panic_fmt
      11:     0x717f6364ee92 - core[b29223e0788b9091]::panicking::panic
      12:     0x717f68f37d11 - <rustc_const_eval[c1b6fa63f3b4b4eb]::interpret::eval_context::InterpCx<rustc_const_eval[c1b6fa63f3b4b4eb]::const_eval::dummy_machine::DummyMachine>>::unsize_into_ptr::<rustc_const_eval[c1b6fa63f3b4b4eb]::interpret::place::MPlaceTy>.cold
      13:     0x717f67b0526a - <rustc_mir_transform[2eae737c95da95e8]::gvn::VnState>::eval_to_const
      14:     0x717f67b89262 - <rustc_mir_transform[2eae737c95da95e8]::gvn::VnState as rustc_middle[981f6d45772a0e11]::mir::visit::MutVisitor>::visit_assign
      15:     0x717f67b874bc - <rustc_mir_transform[2eae737c95da95e8]::gvn::GVN as rustc_mir_transform[2eae737c95da95e8]::pass_manager::MirPass>::run_pass
      16:     0x717f66e059ea - rustc_mir_transform[2eae737c95da95e8]::pass_manager::run_passes_inner
      17:     0x717f66fe191e - rustc_mir_transform[2eae737c95da95e8]::optimized_mir
      18:     0x717f66fe0b5d - rustc_query_impl[dce42a22f9092267]::query_vtables::optimized_mir::invoke_provider_fn::__rust_begin_short_backtrace
      19:     0x717f6732701c - rustc_query_impl[dce42a22f9092267]::execution::try_execute_query::<rustc_middle[981f6d45772a0e11]::query::caches::DefIdCache<rustc_middle[981f6d45772a0e11]::query::erase::ErasedData<[u8; 8usize]>>, false>
      20:     0x717f67326b32 - rustc_query_impl[dce42a22f9092267]::query_vtables::optimized_mir::execute_query_non_incr::__rust_end_short_backtrace
      21:     0x717f6754dd4b - rustc_mir_transform[2eae737c95da95e8]::cross_crate_inline::cross_crate_inlinable
      22:     0x717f6754fa39 - rustc_query_impl[dce42a22f9092267]::query_vtables::cross_crate_inlinable::invoke_provider_fn::__rust_begin_short_backtrace
      23:     0x717f67550309 - rustc_query_impl[dce42a22f9092267]::execution::try_execute_query::<rustc_middle[981f6d45772a0e11]::query::caches::DefIdCache<rustc_middle[981f6d45772a0e11]::query::erase::ErasedData<[u8; 1usize]>>, false>
      24:     0x717f6754ffd8 - rustc_query_impl[dce42a22f9092267]::query_vtables::cross_crate_inlinable::execute_query_non_incr::__rust_end_short_backtrace
      25:     0x717f6755a8b6 - rustc_passes[7b71efe79532f104]::reachable::recursively_reachable
      26:     0x717f6756007d - rustc_passes[7b71efe79532f104]::reachable::reachable_set
      27:     0x717f67fe74aa - rustc_query_impl[dce42a22f9092267]::query_vtables::reachable_set::invoke_provider_fn::__rust_begin_short_backtrace
      28:     0x717f67fea517 - rustc_query_impl[dce42a22f9092267]::execution::try_execute_query::<rustc_middle[981f6d45772a0e11]::query::caches::SingleCache<rustc_middle[981f6d45772a0e11]::query::erase::ErasedData<[u8; 8usize]>>, false>
      29:     0x717f67fea024 - rustc_query_impl[dce42a22f9092267]::query_vtables::reachable_set::execute_query_non_incr::__rust_end_short_backtrace
      30:     0x717f67fae9ca - <rustc_metadata[142077cb525e8060]::rmeta::encoder::EncodeContext>::encode_crate_root
      31:     0x717f683ece07 - rustc_metadata[142077cb525e8060]::rmeta::encoder::encode_metadata::{closure#3}
      32:     0x717f683eeb06 - rustc_metadata[142077cb525e8060]::rmeta::encoder::encode_metadata
      33:     0x717f680e71a5 - rustc_metadata[142077cb525e8060]::fs::encode_and_write_metadata
      34:     0x717f680e596f - <rustc_interface[ee6c17263c420aba]::queries::Linker>::codegen_and_build_linker
      35:     0x717f680cbd0c - rustc_interface[ee6c17263c420aba]::interface::run_compiler::<(), rustc_driver_impl[fa7a7ce11be7f74c]::compiler_entrypoint::{closure#0}>::{closure#2}
      36:     0x717f68185724 - std[2197041a2d5adeae]::sys::backtrace::__rust_begin_short_backtrace::<rustc_interface[ee6c17263c420aba]::util::run_in_thread_with_globals<rustc_interface[ee6c17263c420aba]::util::run_in_thread_pool_with_globals<rustc_interface[ee6c17263c420aba]::interface::run_compiler<(), rustc_driver_impl[fa7a7ce11be7f74c]::compiler_entrypoint::{closure#0}>::{closure#2}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}, ()>
      37:     0x717f681854b3 - <std[2197041a2d5adeae]::thread::lifecycle::spawn_unchecked<rustc_interface[ee6c17263c420aba]::util::run_in_thread_with_globals<rustc_interface[ee6c17263c420aba]::util::run_in_thread_pool_with_globals<rustc_interface[ee6c17263c420aba]::interface::run_compiler<(), rustc_driver_impl[fa7a7ce11be7f74c]::compiler_entrypoint::{closure#0}>::{closure#2}, ()>::{closure#0}, ()>::{closure#0}::{closure#0}, ()>::{closure#1} as core[b29223e0788b9091]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
      38:     0x717f6817db2d - <std[2197041a2d5adeae]::sys::thread::unix::Thread>::new::thread_start
      39:     0x717f60e9cb84 - <unknown>
      40:     0x717f60f29ecc - <unknown>
      41:                0x0 - <unknown>
    
    error: the compiler unexpectedly panicked. This is a bug
    
    note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md
    
    note: please make sure that you have updated to the latest nightly
    
    note: please attach the file at `/app/rustc-ice-2026-10-20T04_48_35-2.txt` to your bug report
    
    note: rustc 1.101.0-nightly (b6eeb93c9 2026-10-18) running on x86_64-unknown-linux-gnu
    
    note: compiler flags: -C debuginfo=2 -C llvm-args=--x86-asm-syntax=intel --crate-type rlib -C opt-level=3 -C debuginfo=none
    
    query stack during panic:
    #0 [optimized_mir] optimizing MIR for `main`
    #1 [cross_crate_inlinable] whether the item should be made inlinable across crates
    ... and 1 other queries... use `env RUST_BACKTRACE=1` to see the full query stack
    Compiler returned: 101
  8. added
    I-ICEIssue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️
    and removed
    I-prioritizeIssue needs a team member to assess the impact. Will be replaced by P-{low,medium,high,critical}
    on Oct 20, 2026
  9. rustbot commented on Oct 21, 2026

    @rustbot
    Collaborator

    Assigning P-medium (discussion on Zulip)

  10. moved this to open/unblocked in T-types unsound issueson Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-dyn-traitArea: trait objects, vtable layoutA-type-systemArea: Type systemC-bugCategory: This is a bug.I-ICEIssue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️I-unsoundIssue: A soundness hole (worst kind of bug), see: https://en.wikipedia.org/wiki/SoundnessP-mediumMedium priorityS-has-bisectionStatus: A bisection has been found for this issueT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.T-typesRelevant to the types team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions