Skip to content

Native Windows support (MinGW-w64, ch3:sock, single node) - #8007

Open
yorickreum wants to merge 10 commits into
pmodels:mainfrom
yorickreum:windows-native
Open

Native Windows support (MinGW-w64, ch3:sock, single node)#8007
yorickreum wants to merge 10 commits into
pmodels:mainfrom
yorickreum:windows-native

Conversation

@yorickreum

@yorickreum yorickreum commented Oct 12, 2026

Copy link
Copy Markdown

Follow-up to #7280. This makes MPICH build and run natively on Windows with MSYS2/MinGW-w64 (UCRT64) and the existing autotools build, using ch3:sock:

./autogen.sh && ./configure --with-device=ch3:sock --disable-romio --without-hwloc

Native Windows builds (no Cygwin) produce libmpi-0.dll/libpmpi-0.dll. Programs built with MSVC can link to them through the C ABI.

What changes:

  • mpl: portable socket layer. MPL_sock_* (read/write/close/poll/accept/set_nonblock/pair/get_error) are inline wrappers on POSIX. On Windows they use Winsock and translate WSAGetLastError() into errno, so the callers' error handling stays the same. MPL_sock_poll ignores negative fds like poll() (WSAPoll reports them as POLLNVAL). MPL_sock_pair is a loopback socket pair that verifies its peer.
  • mpl: Windows shared memory with named file mappings. The old NT backend no longer compiled.
  • ch3/sock, PMI client, spawn/join use the MPL socket layer. On Windows, PMI is finalized eagerly (an atexit handler in a DLL runs after Winsock is gone), and singleton init is not available.
  • config: MinGW support (Winsock libraries, -no-undefined, weak symbols off since weak aliases are not exported from DLLs, Winsock-aware probes).
  • ch3/sock: listen only on loopback when the advertised interface is a loopback address. This avoids a firewall prompt per program on Windows.
  • pm/winexec: a small single-node mpiexec (CreateProcess, PMI-1 over a loopback port, job object, line-wise output forwarding like hydra, MPIEXEC_TIMEOUT). It is the default process manager on MinGW.
  • binding/fortran: libmpifort is static on MinGW, because mpif.h common blocks (MPI_BOTTOM, MPI_IN_PLACE) cannot be shared between a DLL and the program.

Linux builds are unaffected: everything is behind _WIN32 or MinGW-only configure branches.

Limitations / open questions

  • Single node only; no ROMIO, hwloc or C++ bindings yet.
  • MPI_Comm_spawn, name publishing and singleton init are not supported by winexec (it replies with an error).
  • winexec has its own small PMI-1 server instead of reusing src/pm/util/pmiserv.c, which depends on fork. Is reusing it preferred?
  • ch3:sock was the shortest path (as suggested in Building MPICH on Windows #7280). Would ch4 be preferred long term?
  • Fortran (gfortran, static libmpifort): 269 of 274 tests pass. The profiling tests fail (overriding MPI routines collides with the static archive), and so does allocmemf, which the test makefiles do not build by default.

Testing

  • Windows 11, MSYS2 UCRT64 (gcc 16): test/mpi directories basic, init, group, pt2pt, comm, coll, attr, datatype, errhan, info, rma, topo, mpi_t, threads, session: 789 of 790 pass with the default 180 s time limit (typename needs Fortran). Some RMA tests are slow over loopback TCP, e.g. rma/large_acc_flush_local takes about 70 s.
  • A subset of 352 tests on a GitHub Actions Windows runner: all pass.
  • Linux (WSL Ubuntu 26.04), this branch: ch3:sock 792 of 793 (typename, built without Fortran), ch4:ofi with Fortran 1092 of 1092.

Author Checklist

  • Provide Description
  • Commits Follow Good Practice
  • Passes All Tests (Jenkins)
  • Contribution Agreement

@yorickreum
yorickreum marked this pull request as ready for review October 12, 2026 17:45
@hzhou-muse

hzhou-muse commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Linux impact review

PR 8007 is low-risk for Linux builds. The branch builds cleanly on Linux and the tests I ran pass. One correction to the description: Linux is not completely unaffected. One behavior change and one header change reach Linux, and both look fine.

What I checked

  • CI: the automatic checks pass on a8bae68, including mpich-warnings, which is a Linux build.
  • Local build: ch3:sock (--disable-fortran --disable-romio) on Linux. It finished with 3 warnings, all in code this PR doesn't touch (json-c, alltoallw). The default process manager is still hydra, and winexec is not built.
  • Tests: spawn/join, spawn1, spawninfo1, disconnect_reconnect and pt2pt/pingping all report "No Errors". join runs through the rewritten MPIR_fd_send/MPIR_fd_recv in spawn_impl.c.

Changes that reach Linux

  1. ch3:sock can now listen on loopback only (2db492f). If MPICH_INTERFACE_HOSTNAME (or MPICH_INTERFACE_HOSTNAME_R_<rank>) is 127.x or localhost, ranks listen on 127.0.0.1 instead of all interfaces. I confirmed this from /proc/net/tcp:

    MPICH_INTERFACE_HOSTNAME rank listener
    unset 0.0.0.0 (same as before)
    127.0.0.1 127.0.0.1
    localhost 127.0.0.1

    This is fine, since a loopback interface only ever worked on a single node. When the variable is set, each listen now also does a name lookup. It reads the environment variables directly rather than the CVAR, which matches the existing MPIDI_CH3I_Sock_get_host_description. The PR description should mention this change rather than say everything is behind _WIN32.

  2. mpl.h now pulls in more system headers. mpl_sock.h is included by mpl.h, so every file that includes mpl.h (MPICH itself, hydra, PMI) now gets <unistd.h>, <fcntl.h> and <poll.h>, plus small inline wrappers. These are standard POSIX headers, so Linux, macOS and FreeBSD should be fine.

  3. Same behavior on Linux:

    • The new MPL_sock_* wrappers call the same functions as before: read/write/close/poll/accept, fcntl for non-blocking mode, and getsockopt(SO_ERROR) for socket errors.
    • MPL_sock_pair is still pipe().
    • send(fd, ..., 0) became write(), which behaves the same on a socket.
    • The removed separate checks for a failed F_GETFL still report through the same **sock|poll|nonblock error.
    • The new HAVE_DLOPEN guard in mpir_init.c is set by the top-level configure (AC_SEARCH_LIBS([dlopen])), so the MPIX_Init_fortran lookup still runs on Linux.
    • The new MPL_HAVE_SYS_MMAN_H guard in mpl_trmem.c is defined by MPL's configure, so mmap tracing is unchanged.
    • Linux has mmap, so the Windows shared-memory option in aclocal_shm.m4 is never reached.
    • The MPII_Bsend_data_t size check now passes the include directory with -I instead of an absolute path. It works the same.

@hzhou

hzhou commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

@yorickreum is it feasible to set up github action for the windows testing?

Yorick Reum added 10 commits October 19, 2026 14:08
Yorick Reum
mpl: add a portable socket layer with a Winsock implementation
f0fdd54 
Add MPL_sock_{read,write,close,poll,accept,set_nonblock,set_cloexec,pair}
and MPL_sock_init. On POSIX they are thin inline wrappers. On Windows they
use Winsock and translate WSAGetLastError() into errno, so callers can keep
testing errno against EAGAIN, EINPROGRESS etc. MPL_sock_poll ignores
negative fds like poll() does (WSAPoll reports them as POLLNVAL), and
MPL_sock_pair replaces pipe() with a connected loopback socket pair.

mpl_sockaddr uses GetAdaptersAddresses instead of getifaddrs on Windows,
and MPL_large_readv/writev use WSARecv/WSASend. mpl_iov.h provides
struct iovec where <sys/uio.h> does not exist.
Yorick Reum
mpl: update the Windows shared memory backend
2f3dc0a 
The NT shm backend no longer compiled against the current MPL_shm
interface. Rewrite it with named file mappings. Unlike POSIX, a mapping
disappears with its last handle, so the local handle is kept open until
the shm handle is finalized. Select it automatically when neither mmap
nor SysV shm is available but windows.h is, and only use mmap in the
memory tracer when it exists.
Yorick Reum
misc: portability fixes for native Windows
266626d 
Use strerror_s and _putenv_s where strerror_r and setenv do not exist,
only include dlfcn.h when dlopen is available, use MPL_MAX instead of the
non-standard MAX macro, and skip SIGUSR1 where it is not defined.
Yorick Reum
ch3/sock, pmi: use the MPL socket layer
16366a7 
Use the MPL_sock_* functions instead of read/write/close/fcntl/poll/pipe
on sockets in ch3:sock, the PMI client and the spawn/join code, so they
also work with Winsock. The wakeup pipe of the sock channel becomes a
socket pair, which WSAPoll can wait on.
Yorick Reum
pmi: Windows specifics
738566c 
- PMI singleton init (fork/exec of mpiexec) is not available on Windows.
- Finalize PMI right away on Windows: an atexit handler registered from
  a DLL runs at DLL detach, when Winsock may already be gone, so the
  deferred PMI_Finalize could not reach the process manager.
Yorick Reum
config: support native Windows (MinGW-w64) builds
365fa7a 
Link Winsock and the IP helper library and pass -no-undefined so libtool
builds DLLs. Disable weak symbols: depending on the binutils version the
"#pragma weak" checks may pass, but weak aliases are not exported from a
DLL, so the MPI_* entry points would be missing; build the separate
libpmpi instead. ch3:sock gets poll() from WSAPoll, and the socklen_t and
gethostname probes include the Winsock headers. The MPII_Bsend_data_t
size probe passes the include directory with -I, since a native compiler
cannot open the absolute MSYS path that was used in the #include.
Yorick Reum
ch3/sock: listen on loopback if the interface is the loopback address
1e555f9 
When MPICH_INTERFACE_HOSTNAME is 127.0.0.1 or localhost, other processes
can only reach us through loopback anyway, so do not listen on all
interfaces. Besides not exposing the port, this avoids a firewall prompt
on Windows for every new program.
Yorick Reum
pm/winexec: add a single-node process manager for Windows
c3b9d87 
mpiexec starts the ranks with CreateProcess, serves PMI-1 over a loopback
port (PMI_PORT/PMI_ID), and places the ranks in a job object so they are
killed together. Like hydra, it forwards the output of the ranks line by
line and connects stdin to rank 0 only. It honors MPIEXEC_TIMEOUT,
defaults the ranks to loopback communication, and adds
<prefix>\Library\bin to their PATH when installed in a Python
environment. It is the default process manager on MinGW
(--with-pm=winexec).
Yorick Reum
binding/fortran: build libmpifort as a static library on MinGW
14eb572 
mpif.h and "use mpi" define MPI_BOTTOM, MPI_IN_PLACE etc. as common
blocks, which libmpifort recognizes by address. A DLL and the program
using it each get their own copy of a common block, so these constants
were not recognized (e.g. MPI_Allreduce with MPI_IN_PLACE gave wrong
results). Link the thin Fortran layer statically, as it is done for a
similar reason on Darwin with -flat_namespace; libmpi stays a DLL.

As a consequence, intercepting MPI routines from Fortran by defining
them in the program (test/mpi/f*/profile) results in duplicate symbols.
Yorick Reum
ci: add a Windows build and test workflow
0e03ccd 
Build MPICH natively on Windows with MSYS2 (UCRT64), ch3:sock and the
winexec process manager on a GitHub-hosted runner, and run a subset of
test/mpi (basic, init, group, pt2pt, comm, coll). The job fails if any
test fails, since runtests itself exits with 0 in that case.
@yorickreum

yorickreum commented Oct 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

@yorickreum is it feasible to set up github action for the windows testing?

Added a workflow. MSYS2/UCRT64 on a hosted Windows runner, ch3:sock + winexec, and the basic/init/group/pt2pt/comm/coll subset of test/mpi (352 tests). It fails if any test fails. Total run time is about 45 minutes, mostly the build. I am happy to reduce run time in a follow-up commit by just adding ccache.
It currently runs on PRs and pushes to main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yorickreum @hzhou-muse @hzhou