Skip to content

Normalize the field type when computing repr(simd) layout - #6

Open
rynewang wants to merge 1 commit into
mainfrom
fix/simd-layout-normalize-len
Open

rynewang wants to merge 1 commit into
mainfrom
fix/simd-layout-normalize-len

Conversation

@rynewang

@rynewang rynewang commented Nov 1, 2026

Copy link
Copy Markdown
Owner

For review before sending upstream to rust-lang/rust.

Fixes rust-lang#138008. Adds the missing regression test for rust-lang#149156.

Problem

The #[repr(simd)] branch of layout_of_uncached (compiler/rustc_ty_utils/src/layout.rs) matched on the unnormalized field type (.skip_norm_wip()). With

const C: usize = 16;
#[repr(simd)]
pub struct Foo([u8; C]);

the length is an unevaluated alias const, so extract_const_value hits the ConstKind::Alias arm and returns LayoutError::Unknown(Foo). As a result:

  • pub unsafe fn foo(a: Foo) {} ICEs in the mono ABI check: fn_abi_of_instance(foo, []) failed: Layout(Unknown(Foo))
  • size_of::<Foo>() fails const eval (E0080)
  • the same struct written with [u8; 16] works fine

Bisected to rust-lang#137852 (by matthiaskrgr on the issue).

Fix

Normalize the field type with try_normalize_erasing_regions(cx.typing_env, ..) before inspecting it. A normalization failure becomes LayoutError::NormalizationFailure, the same way the pointee-metadata path handles it.

Tests

Verified on a stage1 build:

  • The new simd test fails without the patch and passes with it.
  • tests/crashes, tests/ui/{abi,repr,const-generics,simd,layout,limits}: 1970 tests, 0 failures.
  • ./x fmt --check and ./x test tidy are clean.

🤖 Generated with Claude Code

The SIMD branch of `layout_of_uncached` read the array length from the
unnormalized field type, so `#[repr(simd)] struct Foo([u8; C])` with
`const C: usize = 16` saw an unevaluated alias const and returned
`LayoutError::Unknown`. That ICEd in the mono ABI check
(`fn_abi_of_instance(foo, []) failed: Layout(Unknown(Foo))`) and made
`size_of::<Foo>()` fail const eval, while `[u8; 16]` worked.

Normalize the field type first, reporting `NormalizationFailure` if
that fails.

Also add a `-Zno-codegen` regression test for rust-lang#149156, which was fixed
by rust-lang#162057 but had no test.

Fixes rust-lang#138008
Fixes rust-lang#149156

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ICE: fn_abi_of_instance(foo, []) failed: Layout(Unknown(Foo))

1 participant