Segfault in omni.syntheticdata during the post-process graph tick (Isaac Sim 6.1.0) #905

Description

Rarely, the app segfaults inside libomni.syntheticdata.plugin.so while SyntheticData._post_process_graph_tick evaluates its graph. It runs headless, with cameras and annotators. I've seen it three times:

• at the first renders after the timeline starts (1 in 11 starts);
• at a render after reloading a stage (1 in 70+ runs);
• mid-run in Isaac Lab 3.0.0rc1, 32 envs with two cameras, on a normal step (1 in 4 runs).

From the minidumps: all three crash at the same instruction (plugin offset 0x137f0d, mov 0x10(%r13),%rcx), in a loop over 32-byte records. The array's pointer and count don't agree: twice the pointer was NULL with a count of 3, and once it was valid with a count of about 250 million. That suggests the tick reads a stale or freed object. The last command before each crash was CreateAttrCommand on state:_sdp_intergraph_downstream_node_handles_.

I can't trigger it on demand. 240 runs aimed at the first two cases didn't crash.

Isaac Sim version

6.1.0 (VERSION: 6.1.0-rc.26+release.49347.2d230af4.gl), Kit 110.3.0, omni.syntheticdata 0.6.17

Operating System (OS)

Ubuntu 24.04

GPU Name

RTX 4090

GPU Driver and CUDA versions

Driver 595.99.02, CUDA 13.2 (the first crash was on driver 595.91.07)