[unRekable]

What changed and why

A screen share or camera on Windows now asks Media Foundation for the machine's own hardware H.264 encoder first: crates/mediad/src/mf_encoder.rs enumerates the hardware MFTs (MFTEnumEx), drives the first one that takes NV12 and answers H.264 as an async transform, and normalizes the output to the Annex B stream the SFU and the OpenH264 decoder already take. A machine that lists no hardware encoder, or one that refuses its setup, falls back to the OpenH264 runtime download exactly as before, with its words untouched; on a machine where the hardware takes the job, the download is skipped. One encode surface (Coded), two backends behind it.

Review changes in this push:

• The H.264 output type is negotiated first, before the NV12 input type — the hardware MFTs want it that way and refuse an input type while the output is unset (MF_E_TRANSFORM_TYPE_NOT_SET).
• The wire profile is held to the promise: what the transform kept is read back (MF_MT_MPEG2_PROFILE) and must be baseline (66) or constrained baseline (256). A driver that kept Main or High, or names no profile, is refused in its own words and Coded falls back to OpenH264. Nothing is retried without the profile, so what goes on the wire is always what the SFU was told to expect.
• A hardware failure mid-share no longer ends the share: one frame's error in Coded::encode swaps the backend to OpenH264 (loaded on demand) with a forced keyframe and the track keeps going. Until the OpenH264 download lands, the frame is skipped and the next one tries again.
• A frame is a keyframe only when its sample says so by value: MFSampleExtension_CleanPoint is read (GetUINT32) and must be non-zero, not merely present.
• The first frame may wait up to 120 ms for the encoder's warmup; every later frame waits at most 10 ms, so a driver holding frames back (lookahead, slow driver) never stalls the capture thread. Bits not out in time go out on a later frame's pump, as a skip already does.
• COM hygiene: every IMFActivate from MFTEnumEx is released (in enumerate and at open), the chosen activate is ShutdownObject after ActivateObject, and MFStartup/CoInitializeEx are balanced by a guard the encoder holds — Media Foundation after the transform, the apartment last.
• The available() probe runs once on a thread of its own, so the answer is the machine's and not the asking thread's apartment; a probe that never really ran (its thread would not start) is not cached.

One encode surface (Coded), two backends behind it; the asks of the MFT are narrowed to a small trait (Mft) with the machine's transform handed over by a Source, which is what lets the tests below run without hardware.

Type

• feature
• fix
• perf
• refactor
• chore
• docs

Platforms

• windows
• macos
• linux
• android
• ios

Proof

Scenario: e2e/scenarios/hardware-encoder-fallback — its driver runs, per test, the Media Foundation encoder's unit tests against the fake-MFT seam (no devices, no encoder hardware, no network): the encoder selection, the H.264 output type refused, a driver that kept Main/High or named no profile (refused in its own words, Coded settling on OpenH264), keyframe detection requiring CleanPoint non-zero, the frame loop with its key/tune asks, the close, the probe on its own thread, and the pure NV12 and Annex B helpers. On the base it fails: the module does not exist, so cargo reports no matching tests.

What this scenario cannot say: that a real NVIDIA/Intel/AMD MFT negotiates output-first, keeps constrained baseline, and survives a whole call. That needs the lab recording a real share on a machine with a hardware MFT, which is the released-client proof this scenario points to. Until then, the CI runners prove the decision, the refusal and the fallback paths, and the OpenH264 fallback keeps its own covered path.

Compatibility

No protocol change: the same H.264 constrained baseline goes out, Annex B, one layer — and now enforced rather than assumed, since a driver that would send anything else is refused and OpenH264 takes over. An older server and an older client keep working.

Risk and rollback

If a hardware MFT misbehaves on a machine, the encoder returns its own truthful words and Coded falls back to OpenH264 — at open, and now also mid-share; the fallback is the previous behavior verbatim. Revert the commit to return entirely to OpenH264.

Release note

Screen shares and cameras on Windows are encoded by the machine's own hardware encoder, with the OpenH264 download as the fallback.

[itz4blitz]

Draft, so this is early feedback, but I would not merge it as it stands. Thanks for taking this on, the shape (one Coded surface, MF behind it, OpenH264 as the fallback) is the right one.

Blocking

1. pr-gate is red and it is real, not flaky: "A feature needs an e2e scenario". Per docs/quality-gate.md a feature needs an e2e/scenarios/<id> in this PR. If the lab cannot exercise real encoder hardware, the scenario can still drive a share on Windows and assert the fallback path and the Annex B output; say so under Proof rather than leaving it as "none yet".
2. Types are set in the wrong order in Encoder::open. SetInputType is called before SetOutputType. Media Foundation's H.264 encoders (and the NVENC/QSV/AMF MFTs) generally want the output type first and reject the input type with MF_E_TRANSFORM_TYPE_NOT_SET. As written I expect many machines to fall straight back to OpenH264, which defeats the feature. Please swap them and, ideally, check it on real NVIDIA/Intel/AMD hardware.
3. The profile fallback quietly changes what goes on the wire. If the encoder refuses ConstrainedBase/level, open retries with no profile and only logs at debug. The PR body and 06-calls.md both promise constrained baseline, but the SFU and receivers get whatever the driver picks (Main/High). Either fail over to OpenH264 in that case, or read the negotiated profile back and reject non-baseline.
4. A failure after open is fatal. ProcessInput/ProcessOutput errors (MF_E_TRANSFORM_STREAM_CHANGE, NEED_MORE_INPUT, a driver reset, GPU removed) come back as Err and nothing falls back to OpenH264 mid-share. Since needs_openh264() now skips the early load_api(), a failure here also ends up as the share dying rather than switching codecs. A runtime failover in Coded::encode (swap Backend to OpenH264 on error and force a keyframe) would make "falls back exactly as before" actually true.

Should fix

5. MFSampleExtension_CleanPoint is checked with .is_ok(). That tells you the attribute exists, not that it is TRUE. Read the value and require it to be non-zero, otherwise a delta frame can be tagged as a key frame.
6. encode can block the capture thread up to 120 ms per frame when the encoder is holding frames back (lookahead or a slow driver). At 15 to 60 fps that stalls capture. Consider a much shorter wait, or never waiting after the first frame.
7. Leaks: in enumerate and open, only entry 0 of the MFTEnumEx array is taken; the other IMFActivates are never released, and enumerate releases none. Also ShutdownObject on the chosen activate is never called, and MFStartup/CoInitializeEx are never balanced. Small, but easy to get right.
8. available() is cached for the whole process and is first evaluated on whatever thread calls it. A RPC_E_CHANGED_MODE on that one thread would disable hardware encoding for the rest of the run. Do the probe on a dedicated thread, or do not cache an error caused by the thread's apartment.

Tests

The unit tests cover nv12, annex_b, and the packing helpers well. Nothing covers Encoder::open/encode, key-frame detection, or the fallback after a refusal (a small trait or closure seam for the MFT would let you test those without hardware).

Also: the PR body has literal backslashes where the code spans should be. Not a blocker.