Skip to content

assertion failed: trying to encode a dep node twice #150018

Closed

Description

@zetanumbers

As I've suspected #149916 revealed a bug in rustc's query system on multiple threads. I was trying to build rustc itself with the nightly toolchain as stage0 after doing git checkout build fails while trying to compile rustc_lint_defs.

Error

thread 'rustc' (199561) panicked at compiler/rustc_query_system/src/dep_graph/graph.rs:1392:9:
assertion `left == right` failed: trying to encode a dep node twice
  left: 64535
 right: 4294967295
backtrace
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<u32, u32>
   4: rustc_query_system::query::plumbing::try_execute_query::<rustc_query_impl::DynamicConfig<rustc_data_structures::vec_cache::VecCache<rustc_span::def_id::LocalDefId, rustc_middle::query::erase::Erased<[u8; 8]>, rustc_query_system::dep_graph::graph::DepNodeIndex>, false, false, false>, rustc_query_impl::plumbing::QueryCtxt, true>
   5: rustc_query_impl::plumbing::force_from_dep_node::<rustc_query_impl::DynamicConfig<rustc_data_structures::vec_cache::VecCache<rustc_span::def_id::LocalDefId, rustc_middle::query::erase::Erased<[u8; 8]>, rustc_query_system::dep_graph::graph::DepNodeIndex>, false, false, false>>
   6: <rustc_query_impl::plumbing::query_callback<rustc_query_impl::query_impl::mir_drops_elaborated_and_const_checked::QueryType>::{closure#0} as core::ops::function::FnOnce<(rustc_middle::ty::context::TyCtxt, rustc_query_system::dep_graph::dep_node::DepNode, rustc_query_system::dep_graph::serialized::SerializedDepNodeIndex)>>::call_once
   7: <rustc_query_system::dep_graph::graph::DepGraphData<rustc_middle::dep_graph::DepsType>>::try_mark_previous_green::<rustc_query_impl::plumbing::QueryCtxt>
   8: <rustc_query_system::dep_graph::graph::DepGraphData<rustc_middle::dep_graph::DepsType>>::try_mark_previous_green::<rustc_query_impl::plumbing::QueryCtxt>
   9: <rustc_query_system::dep_graph::graph::DepGraphData<rustc_middle::dep_graph::DepsType>>::try_mark_previous_green::<rustc_query_impl::plumbing::QueryCtxt>
  10: <rustc_query_system::dep_graph::graph::DepGraphData<rustc_middle::dep_graph::DepsType>>::try_mark_previous_green::<rustc_query_impl::plumbing::QueryCtxt>
  11: <rustc_query_system::dep_graph::graph::DepGraphData<rustc_middle::dep_graph::DepsType>>::try_mark_green::<rustc_query_impl::plumbing::QueryCtxt>
  12: rustc_query_system::query::plumbing::ensure_must_run::<rustc_query_impl::DynamicConfig<rustc_query_system::query::caches::DefIdCache<rustc_middle::query::erase::Erased<[u8; 32]>>, false, false, false>, rustc_query_impl::plumbing::QueryCtxt>
  13: <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners::<rustc_hir_analysis::check_crate::{closure#2}>::{closure#0}
  14: rustc_data_structures::sync::parallel::par_slice::par_rec::<&rustc_span::def_id::LocalDefId, rustc_data_structures::sync::parallel::par_for_each_in<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_hir_analysis::check_crate::{closure#2}>::{closure#0}>::{closure#0}::{closure#0}>
  15: <rustc_thread_pool::job::StackJob<rustc_thread_pool::latch::SpinLatch, rustc_thread_pool::join::join_context<rustc_thread_pool::join::join::call<(), rustc_data_structures::sync::parallel::par_slice::par_rec<&rustc_span::def_id::LocalDefId, rustc_data_structures::sync::parallel::par_for_each_in<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_hir_analysis::check_crate::{closure#2}>::{closure#0}>::{closure#0}::{closure#0}>::{closure#1}>::{closure#0}, rustc_thread_pool::join::join::call<(), rustc_data_structures::sync::parallel::par_slice::par_rec<&rustc_span::def_id::LocalDefId, rustc_data_structures::sync::parallel::par_for_each_in<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_hir_analysis::check_crate::{closure#2}>::{closure#0}>::{closure#0}::{closure#0}>::{closure#2}>::{closure#0}, (), ()>::{closure#0}::{closure#0}, ()> as rustc_thread_pool::job::Job>::execute
  16: <rustc_thread_pool::registry::WorkerThread>::wait_or_steal_until_cold
  17: <rustc_thread_pool::registry::ThreadBuilder>::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: please make sure that you have updated to the latest nightly

note: please attach the file at `/home/huawei/dev/rust/rustc-ice-2025-12-15T10_49_29-199385.txt` to your bug report

note: compiler flags: --crate-type lib -C opt-level=3 -C embed-bitcode=no -C debuginfo=line-tables-only -C debug-assertions=on -C incremental=[REDACTED] -C symbol-mangling-version=v0 -Z annotate-moves -Z unstable-options -Z macro-backtrace -C split-debuginfo=off -C force-frame-pointers=true -C llvm-args=-import-instr-limit=10 -C link-args=-Wl,-z,origin -C link-args=-Wl,-rpath,$ORIGIN/../lib -Z on-broken-pipe=kill -C lto=off -Z threads=24 -Z binary-dep-depinfo -Z tls-model=initial-exec -Z force-unstable-if-unmarked

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [mir_drops_elaborated_and_const_checked] elaborating drops for `<impl at compiler/rustc_lint_defs/src/lib.rs:557:1: 557:10>::default_fields_for_macro`
#1 [analysis] running analysis passes on crate `rustc_lint_defs`
end of query stack
there was a panic while trying to force a dep node
try_mark_green dep node stack:
#0 mir_for_ctfe(rustc_lint_defs[912f]::{impl#9}::default_fields_for_macro)
#1 eval_to_allocation_raw(6a497d626948f9b4-2f772ebace64724b)
#2 eval_to_const_value_raw(6a497d626948f9b4-2f772ebace64724b)
#3 eval_static_initializer(rustc_lint_defs[912f]::builtin::UNCONDITIONAL_PANIC)
end of try_mark_green dep node stack
error: could not compile `rustc_lint_defs` (lib)

It is surprisingly persistent as and I was unable to get it go away just by retriggering the incremental compilation using touch src/lib.rs but deleting incremental folder fixes this per usual after some rebuilds goes away. We can infer from the error that compiler finds itself in a situation where it tries to mark one dep node as both green and red.

Rust version

Stage 0:

rustc 1.94.0-nightly (0208ee09b 2025-12-14)
binary: rustc
commit-hash: 0208ee09be465f69005a7a12c28d5eccac7d5f34
commit-date: 2025-12-14
host: x86_64-unknown-linux-gnu
release: 1.94.0-nightly
LLVM version: 21.1.5

Steps to reproduce

Do a clean build x build library --stage 1 at 52fda2e, then git checkout to ba45f0b and without cleaning try rebuilding it the same way again. And don't forget to set RUSTFLAGS_BOOTSTRAP to -Zthreads=n where n > 1.

Run ./x b rustc in a clean git worktree of rust-lang/rust with rust.incremental = true and rust.rustflags = "-Zthreads=8" set in bootstrap.toml. Then do a change from below diff to rustc source code:

diff --git a/compiler/rustc_data_structures/src/lib.rs b/compiler/rustc_data_structures/src/lib.rs
index 4467a281118..e79312458b3 100644
--- a/compiler/rustc_data_structures/src/lib.rs
+++ b/compiler/rustc_data_structures/src/lib.rs
@@ -162,3 +162,8 @@ fn fmt(&self, f: &mut ::std::fmt::Formatter<'_>) -> ::std::fmt::Result {
         }
     };
 }
+
+#[derive(Clone, Copy)]
+pub struct BadStruct {
+    int: u64,
+}

Finally run ./x b rustc again. It may sometimes not reproduce, in such case restore changes and ./x b rustc, then apply change and ./x b rustc until ICE is reproduced.

Activity

  1. added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Dec 15, 2025
  2. zetanumbers commented on Dec 15, 2025

    @zetanumbers
    ContributorAuthor

    If I do the these steps enough times without replacing stage0 I get #141540.

  3. zetanumbers commented on Dec 15, 2025

    @zetanumbers
    ContributorAuthor

    I've also got "assertion failed: tried to mark a red node as green" while messing around with the compiler. It is the same bug but it behaves a bit differently due to different orders of atomic ops. And its apparently the source of #148295.

    thread 'rustc' (433110) panicked at compiler/rustc_query_system/src/dep_graph/graph.rs:1367:17:
    assertion `left != right` failed: tried to mark a red node as green
      left: 4294967294
     right: 4294967294
    
    backtrace
    stack backtrace:
       0: __rustc::rust_begin_unwind
       1: core::panicking::panic_fmt
       2: core::panicking::assert_failed_inner
       3: core::panicking::assert_failed::<u32, u32>
       4: <rustc_query_system::dep_graph::graph::DepGraphData<rustc_middle::dep_graph::DepsType>>::try_mark_previous_green::<rustc_query_impl::plumbing::QueryCtxt>
       5: <rustc_query_system::dep_graph::graph::DepGraphData<rustc_middle::dep_graph::DepsType>>::try_mark_previous_green::<rustc_query_impl::plumbing::QueryCtxt>
       6: <rustc_query_system::dep_graph::graph::DepGraphData<rustc_middle::dep_graph::DepsType>>::try_mark_previous_green::<rustc_query_impl::plumbing::QueryCtxt>
       7: rustc_query_system::query::plumbing::ensure_must_run::<rustc_query_impl::DynamicConfig<rustc_data_structures::vec_cache::VecCache<rustc_span::def_id::LocalDefId, rustc_middle::query::erase::Erased<[u8; 0]>, rustc_query_system::dep_graph::graph::DepNodeIndex>, false, false, false>, rustc_query_impl::plumbing::QueryCtxt>
       8: <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners::<rustc_interface::passes::run_required_analyses::{closure#1}::{closure#0}>::{closure#0}
       9: rustc_data_structures::sync::parallel::par_slice::par_rec::<&rustc_span::def_id::LocalDefId, rustc_data_structures::sync::parallel::par_for_each_in<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_interface::passes::run_required_analyses::{closure#1}::{closure#0}>::{closure#0}>::{closure#0}::{closure#0}>
      10: <rustc_thread_pool::job::StackJob<rustc_thread_pool::latch::SpinLatch, rustc_thread_pool::join::join_context<rustc_thread_pool::join::join::call<(), rustc_data_structures::sync::parallel::par_slice::par_rec<&rustc_span::def_id::LocalDefId, rustc_data_structures::sync::parallel::par_for_each_in<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_interface::passes::run_required_analyses::{closure#1}::{closure#0}>::{closure#0}>::{closure#0}::{closure#0}>::{closure#1}>::{closure#0}, rustc_thread_pool::join::join::call<(), rustc_data_structures::sync::parallel::par_slice::par_rec<&rustc_span::def_id::LocalDefId, rustc_data_structures::sync::parallel::par_for_each_in<&rustc_span::def_id::LocalDefId, &[rustc_span::def_id::LocalDefId], <rustc_middle::ty::context::TyCtxt>::par_hir_body_owners<rustc_interface::passes::run_required_analyses::{closure#1}::{closure#0}>::{closure#0}>::{closure#0}::{closure#0}>::{closure#2}>::{closure#0}, (), ()>::{closure#0}::{closure#0}, ()> as rustc_thread_pool::job::Job>::execute
      11: <rustc_thread_pool::registry::WorkerThread>::wait_or_steal_until_cold
      12: <rustc_thread_pool::registry::ThreadBuilder>::run
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    
    error: the compiler unexpectedly panicked. this is a bug.
    
    note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md
    
    note: please make sure that you have updated to the latest nightly
    
    note: please attach the file at `/home/huawei/dev/rust/rustc-ice-2025-12-15T12_09_14-432905.txt` to your bug report
    
    note: compiler flags: --crate-type lib -C opt-level=3 -C embed-bitcode=no -C debuginfo=line-tables-only -C debug-assertions=on -C incremental=[REDACTED] -C symbol-mangling-version=v0 -Z annotate-moves -Z unstable-options -Z macro-backtrace -C split-debuginfo=off -C force-frame-pointers=true -C llvm-args=-import-instr-limit=10 -C link-args=-Wl,-z,origin -C link-args=-Wl,-rpath,$ORIGIN/../lib -Z on-broken-pipe=kill -C lto=off -Z threads=24 -Z binary-dep-depinfo -Z tls-model=initial-exec -Z force-unstable-if-unmarked
    
    note: some of the compiler flags provided by cargo are hidden
    
    query stack during panic:
    #0 [analysis] running analysis passes on crate `rustc_ast_pretty`
    end of query stack
    error: could not compile `rustc_ast_pretty` (lib)
    
  4. added
    I-ICEIssue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    A-incr-compArea: Incremental compilation
    C-bugCategory: This is a bug.
    on Dec 15, 2025
  5. petrochenkov commented on Dec 19, 2025

    @petrochenkov
    Contributor

    Issue debugged, a solution is proposed in #150156.

  6. removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Dec 22, 2025
  7. luxalpa commented on Dec 29, 2025

    @luxalpa

    Are there any workarounds other than doing cargo clean? Do we know which nightly / toolchain version/date this originally appeared in?

  8. TeamDman commented on Jan 5, 2026

    @TeamDman

    Are there any workarounds other than doing cargo clean?

    I usually

    rm -r .\target\debug\incremental\
  9. zetanumbers commented on Jan 12, 2026

    @zetanumbers
    ContributorAuthor

    Are there any workarounds other than doing cargo clean?

    You can kinda brute-force it by doing your cargo build command again and again. At least that works when compiling rustc.

    Do we know which nightly / toolchain version/date this originally appeared in?

    This exact issue appeared sometime around nightly-2025-12-13. However downgrading would only get you either #142778 or #141540. The latter issue will corrupt your incremental cache and so brute-force won't be able to help you any longer.

  10. mischnic commented on Jan 15, 2026

    @mischnic

    I'm also seeing these two quite frequently (nightly 2026-01-03)

    thread 'rustc' panicked at /rustc-dev/7ecabfaaf1a58172cdd71f7661cf6589ddf7928f/compiler/rustc_query_system/src/dep_graph/graph.rs:1367:17:
    assertion `left != right` failed: tried to mark a red node as green
      left: 4294967294
     right: 4294967294
    
    thread 'rustc' panicked at /rustc-dev/7ecabfaaf1a58172cdd71f7661cf6589ddf7928f/compiler/rustc_query_system/src/dep_graph/graph.rs:1392:9:
    assertion `left == right` failed: trying to encode a dep node twice
      left: 180797
     right: 4294967295
    

    a few full logs from yesterday:

    Archiv.zip

  11. luxalpa commented on Jan 18, 2026

    @luxalpa

    You can kinda brute-force it by doing your cargo build command again and again. At least that works when compiling rustc.

    I am on rustc version: 1.94.0-nightly (4f14395 2025-12-22), bruteforcing it does not seem to work for me, or at least not for the first 7 times. It seems like I have to completely rebuild it which sucks since it takes like 10 minutes.

  12. zetanumbers commented on Jan 18, 2026

    @zetanumbers
    ContributorAuthor

    You can kinda brute-force it by doing your cargo build command again and again. At least that works when compiling rustc.

    I am on rustc version: 1.94.0-nightly (4f14395 2025-12-22), bruteforcing it does not seem to work for me, or at least not for the first 7 times. It seems like I have to completely rebuild it which sucks since it takes like 10 minutes.

    In that case rm -r target/debug/incremental should be your best second option until #150156 is merged.

  13. added 2 commits that reference this issue on Feb 10, 2026
    0c09117
    9e79395
  14. added 4 commits that reference this issue on Feb 13, 2026
    298b33d
    210c39b
    7194876
    ca5e141
  15. added a commit that references this issue on Aug 7, 2026
    a75abf5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-incr-compArea: Incremental compilationA-parallel-compilerArea: parallel compilerC-bugCategory: This is a bug.I-ICEIssue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions