Skip to content

Standalone Python process remains alive after SimulationApp.close() returns (Isaac Sim 5.1.0.0) #877

Closed

Description

Summary

A bounded signal-free standalone Python run using Isaac Sim 5.1.0.0 / Kit 107.3.3 returned from the project wrapper's IsaacApp.close(), persisted its final report, and returned 0 from worker main. The child process nevertheless remained alive until the parent deadline and was terminated for cleanup. Natural process exit was not observed; the cause is unresolved. This report does not claim an SDK defect.

No task-specific USD scene, robot, camera, or dataset was created. The SDK default startup created an empty stage. This reduced run shows that camera construction is not necessary for this observed process-survival symptom; it does not prove the earlier camera qualification had the same cause.

Environment and launch

  • Isaac Sim distribution/application: 5.1.0.0 / 5.1.0
  • Kit: 107.3.3+production.229672.69cbf6ad.gl
  • CPython 3.11.15; Ubuntu 22.04.5 LTS, kernel 6.8.0-138-generic
  • RTX 4080 SUPER, driver 580.178.04, Vulkan
  • Headless standalone launch; fast_shutdown=False; child-only OMNICLIENT_HUB_MODE=disabled; working directory set to repository root
  • The run used the pinned bootstrap and worker linked below. The short code sketch is an illustration of the measured path, not a separately tested SDK-only reproducer.
# Illustrative sketch of the measured project-wrapper path; not independently tested.
from navfm.isaac.app import IsaacApp, SimConfig

app = IsaacApp(SimConfig(headless=True), fast_shutdown=False)
app.start()
app.close()

Observed event sequence

Event Elapsed seconds
Project IsaacApp wrapper constructor returned 0.025
IsaacApp.start() returned 5.756
IsaacApp.close() entered 5.761
IsaacApp.close() returned 35.810
Final worker report persisted 35.819
Worker main returned 0 35.821
Parent deadline reached 59.998
Cleanup SIGTERM requested 60.001
Child reaped with exit -15 60.167

IsaacApp.__init__ is the project wrapper constructor, not the SDK SimulationApp constructor. SDK construction occurs inside IsaacApp.start() and was not separately bracketed. Worker return, process exit, and SDK cleanup are distinct; natural process exit and complete SDK cleanup were not observed.

Sanitized startup excerpts:

| Driver Version: 580.178.04 | Graphics API: Vulkan
| 0 | NVIDIA GeForce RTX 4080 SUPER | ...
| OS: 22.04.5 LTS (Jammy Jellyfish) ubuntu, Version: 22.04.5, Kernel: 6.8.0-138-generic
[5.757s] Simulation App Shutting Down

Immutable source and evidence references

The sanitized evidence identities and SDK-source hash are recorded in the issue package. Raw logs and runtime evidence are not attached or publicly linked; they remain available locally if a support engineer requests a specific artifact.

Questions

  1. Is natural process exit expected after SimulationApp.close() returns with fast_shutdown=False in this standalone 5.1.0.0 / Kit build?
  2. Is there a supported shutdown or cleanup sequence for a child where close() returns and worker main returns but the process remains alive?
  3. Which specific diagnostic artifact or additional observation would best distinguish Python interpreter shutdown from native Kit/framework teardown or another remaining process-lifetime condition?

No workaround is proposed. The run did not test camera qualification, navigation, or a task-specific scene.

Activity

  1. vick-yu commented on Oct 13, 2026

    @vick-yu

    Thanks for the careful writeup and the immutable evidence package -- the
    fsync-backed lifecycle markers make this much easier to analyze than the
    typical "my script hangs" report.

    To answer your three questions directly:

    1. Is natural process exit expected after SimulationApp.close() with fast_shutdown=False?

    Not guaranteed in Isaac Sim 5.1.0, and this is a known architectural limitation
    of that version.

    In 5.1.0, close() with fast_shutdown=False calls
    shutdown_and_release_framework(), which performs extension-by-extension
    teardown and then returns control to Python. After that point, CPython's
    interpreter shutdown proceeds normally -- but native Kit and Carbonite worker
    threads (the tasking pool, renderer threads, etc.) are not joined by the
    Python side before returning
    . These are non-daemon C threads. CPython's
    atexit and threading cleanup wait for daemon threads, but non-daemon
    native C threads that are not tracked by Python's threading module are left
    to finish on their own schedule.

    In your measurement, the process survived ~24 seconds after main() returned.
    That interval corresponds to Kit's native teardown completing independently of
    the Python runtime -- this is the framework cleaning up after itself, but doing
    so asynchronously from Python's perspective.

    The 5.1 documentation describes the option as:

    fast_shutdown (bool) -- True to exit process immediately, false to shutdown
    each extension.

    "Shutdown each extension" refers to the extension teardown sequence, not a
    guarantee that the process exits immediately after close() returns.

    2. Is there a supported cleanup or shutdown sequence for this case?

    Yes. The supported workaround in 5.1.0 is to use fast_shutdown=True
    (the default) if bounded process exit is a hard requirement for your supervisor.
    With fast_shutdown=True, Kit calls quickReleaseFrameworkAndTerminate
    internally, which terminates the process immediately after the framework is
    released -- your supervisor will see a clean exit within seconds.

    If you need graceful extension teardown and bounded process exit, the
    cleanest approach in 5.1.0 is to add an explicit os._exit(0) after
    close() returns:

    import os
    app.close()
    # Give native threads a brief grace period, then force exit.
    os._exit(0)

    os._exit() bypasses CPython's atexit handlers and terminates immediately,
    which is safe after close() has already finished its cleanup sequence.

    Note for Isaac Sim 6.0.0 and later: this behavior is improved. The 6.0+
    SimulationApp switches from shutdown_and_release_framework() to
    app.shutdown() (which handles fast_shutdown internally and avoids a GIL
    deadlock that could occur in the older path). It also registers an atexit
    handler so the SDK can clean up even if close() is never called explicitly.
    If you are able to move to 6.0.0 or later, the post-close process-survival
    window is substantially reduced even with fast_shutdown=False.

    3. Which artifact would best distinguish the remaining process-lifetime condition?

    A strace -p <pid> -e trace=futex,nanosleep,epoll_wait -f snapshot taken in
    the ~24-second window between main() returning and process exit would show
    which syscalls the surviving native threads are blocked on. If you see futex
    waits with a consistent caller (e.g., carb.tasking thread joining its pool),
    that pinpoints the specific subsystem keeping the process alive.

    Alternatively, attaching gdb -p <pid> and running info threads followed
    by thread apply all bt at that moment would give the full native thread
    backtraces and identify any threads that have not yet exited.

    In practice, for production use, the os._exit(0) pattern or fast_shutdown=True
    is the pragmatic solution rather than diagnosing the specific native teardown
    sequencing.


    The issue package you provided is thorough and the observation is accurate.
    This is a real limitation of the 5.1.0 teardown path, not a user error.

  2. vick-yu commented on Oct 27, 2026

    @vick-yu

    Hi @FrankAsanteVanLaarhoven — thanks again for the exceptionally thorough writeup and the immutable evidence package. We haven't heard back since our reply explaining the non-daemon native thread behavior and the available workarounds (fast_shutdown=True, sys.exit() after close()). We'll go ahead and close this out. If you're still hitting this or have follow-up findings, you can reopen it directly via the 'Reopen issue' button, or open a new issue for 6.0.0+ referencing this one. Thanks for contributing such a well-documented report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions