Skip to content

Add Python 3.15 and 3.15t support - #4555

Merged
zcbenz merged 7 commits into
ml-explore:mainfrom
XXXXRT666:python-3.15-support
Sep 29, 2026
Merged

zcbenz merged 7 commits into
ml-explore:mainfrom
XXXXRT666:python-3.15-support

Conversation

@XXXXRT666

Copy link
Copy Markdown
Contributor
  • ☑️ I understand it is strictly prohibited to use AI to write PR description
  • AI usage disclosure: All tests are generated by AI, and AI helps me fix bugs

Add Python 3.15 and 3.15t to the release wheel matrix. Update nanobind from 3.0.1 to 3.1.0 and align the extension build configuration and documentation with free-threaded support.

Freeze bound types on Python 3.15 to enable Python immutable type optimizations. This prevents runtime changes to attributes on those types.

Tests on Python 3.15t also exposed two lifetime issues:

  • Transform callbacks were stored in nanobind function objects, which are immortal in free threaded builds. The wrapper now owns the C++ callback directly through a gc nanobind class instance, allowing captured objects and compiled function caches to be released.
  • The random state sentinel could access the Python runtime during static destruction and crash at shutdown when first initialized on a worker thread. Keep its existing process lifetime reference without a reference count operation during static destruction.

@zcbenz zcbenz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running test_bindings.py with debug build DEBUG=1 setup.py build_ext -i gets critical error:

Critical nanobind error: attempted to change the reference count of a Python object or to call into Python while the GIL was not held!

@zcbenz

zcbenz commented Sep 28, 2026

Copy link
Copy Markdown
Member

There is also a weird error when building wheel https://github.com/ml-explore/mlx/actions/runs/36394870740/job/108838778494:

  /usr/bin/ccache /usr/bin/c++ -DNB_DOMAIN=mlx -Dcore_EXPORTS -I/home/runner/work/_temp/uv-python-dir/cpython-3.15.0rc2-linux-x86_64-gnu/include/python3.15 -I/home/runner/work/mlx/mlx/build/temp.linux-x86_64-cpython-315/mlx.core/_deps/nanobind-src/include -I/home/runner/work/mlx/mlx -O3 -DNDEBUG -std=gnu++20 -flto=auto -fno-fat-lto-objects -fPIC -fvisibility=hidden -fno-stack-protector -ffunction-sections -fdata-sections -Werror -MD -MT python/src/CMakeFiles/core.dir/device.cpp.o -MF python/src/CMakeFiles/core.dir/device.cpp.o.d -o python/src/CMakeFiles/core.dir/device.cpp.o -c /home/runner/work/mlx/mlx/python/src/device.cpp
  In file included from /home/runner/work/_temp/uv-python-dir/cpython-3.15.0rc2-linux-x86_64-gnu/include/python3.15/Python.h:14,
                   from _deps/nanobind-src/include/nanobind/nb_python.h:21,
                   from _deps/nanobind-src/include/nanobind/nanobind.h:32,
                   from /home/runner/work/mlx/mlx/python/src/device.cpp:6:
  /home/runner/work/_temp/uv-python-dir/cpython-3.15.0rc2-linux-x86_64-gnu/include/python3.15/pyconfig.h:2067: error: "_POSIX_C_SOURCE" redefined [-Werror]
   2067 | #define _POSIX_C_SOURCE 202405L
        | 
  In file included from /usr/include/x86_64-linux-gnu/c++/11/bits/os_defines.h:39,
                   from /usr/include/x86_64-linux-gnu/c++/11/bits/c++config.h:586,
                   from /usr/include/c++/11/utility:68,
                   from /usr/include/c++/11/optional:36,
                   from /home/runner/work/mlx/mlx/python/src/device.cpp:3:
  /usr/include/features.h:290: note: this is the location of the previous definition
    290 | # define _POSIX_C_SOURCE        200809L
        | 

@XXXXRT666

Copy link
Copy Markdown
Contributor Author

structure_sentinel() still stored an owning static nb::object, which performed a DECREF after shutting down

Some translation units included standard headers before Python.h. This caused glibc to define some macros before Python 3.15's pyconfig.h defined them with different values

This 2 problems are all fixed now

@zcbenz zcbenz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

XXXXRT666 and others added 7 commits September 29, 2026 17:03
Let garbage-collected wrappers own C++ callbacks so compiled caches and captured objects are released. Keep the random state sentinel alive without a Python reference-count operation during static destruction.
Keep the structure sentinel alive without Python reference-count operations during static destruction. Add a subprocess regression test for compiled-function shutdown.
Include Python headers before standard headers in binding translation units to avoid feature macro redefinition errors with Python 3.15.

This branch was successfully deployed

1 active deployment
dry-run — 9d316f72 Deployed Sep 29, 2026 by zcbenz via Publish mlx-cuda to PyPI #291
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants