gh-152754: Fix crash when an os.scandir iterator is shared between threads

[nascheme]

The issue was a race between ScandirIterator_iternext() and ScandirIterator.close(): one thread could be inside readdir() or FindNextFileW() while another closed the underlying directory stream or Windows handle.

Changes

• Add a PyMutex and atomic closed state to ScandirIterator in all builds.
• Serialize access to the native directory stream or handle.
• Make close() mark the iterator closed immediately without waiting for an in-progress directory read. The reading thread performs the native close afterward when necessary.
• Copy native directory-entry data while holding the mutex, then create Python objects and report errors after releasing it.
• Keep Py_BEGIN_ALLOW_THREADS around readdir(), FindNextFileW(), closedir(), and FindClose().
• Avoid incorrectly emitting ResourceWarning when opening the directory fails.
• Add regression tests for concurrent iteration, iteration racing with close(), and failed iterator creation.
• Document the behavior of sharing a scandir() iterator between threads.

Threading semantics

Sharing an os.scandir() iterator between threads will no longer corrupt the iterator or crash. Concurrent calls may consume entries in any order, so which thread receives each entry remains unspecified. Calling close() while another thread is iterating marks the iterator closed immediately and ends iteration early.

A PyMutex is used rather than a critical section because the protection must remain active across Py_BEGIN_ALLOW_THREADS while the native directory operation runs without the GIL.

• Issue: Segfault between os.scandir iterator's close() and __next__() on the dirp handle #152754

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #34039922 | 📁 Comparing 854b5b9 against main (bc31217)

  🔍 Preview build  

2 files changed

± library/os.html
± whatsnew/changelog.html

[nascheme]

Note, I revised this PR to no longer try to be thread-safe with multiple threads using the same iterator. That seems a pretty unlikely usage scenario. Instead, this avoids crashing and/or memory corruption.