diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 3333551..df2ba75 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -74,6 +74,7 @@ included at compile time, so there is one copy of every kernel for all targets. | `gen_tex.rs` | the texture-path generator with modes (realtime, video with lookahead, cadence) that the Stream wraps | | `gmhs.rs` | the repair stage and the cadence statistics and composite | | `interop.rs` | Windows: a D3D11 shared texture into the Vulkan device without a copy | +| `vulkan_interop.rs` | Windows, Linux: frames from another Vulkan device through exported buffers and semaphores | | `api.rs` | the CPU convenience: `api::Engine`, `api::Generator`, frames as bytes | | Other crates and directories | Role | @@ -278,7 +279,8 @@ Native (`examples/player`, the `interop` module): | Piece | Does | |---|---| | ticks | Windows: DWM vblank timing with a PLL that keeps the phase; elsewhere the window system's frame callbacks | -| `interop::d3d11_shared_texture(engine, handle, &desc)` | a D3D11 shared texture (screen capture) into the Vulkan device without a copy, then `add_input_texture` on a session or a stream; C: `fh_session_import_d3d11` / `fh_stream_import_d3d11`. DX12 import, Vulkan external memory on Linux and IOSurface on macOS are open | +| `interop::d3d11_shared_texture(engine, handle, &desc)` | a D3D11 shared texture (screen capture) into the Vulkan device without a copy, then `add_input_texture` on a session or a stream; C: `fh_session_import_d3d11` / `fh_stream_import_d3d11`. DX12 import and IOSurface on macOS are open | +| `vulkan_interop` | a host's own Vulkan device (an emulator, a game) on the same GPU: frames in and out through buffers it exports, ordered by timeline semaphores in both directions; the engine keeps its own device and queue. C: `fh_vk_*`, `fh_stream_copy_in` / `fh_stream_copy_out` | | presentation | the host's swapchain; the engine hands back the output texture, the host blits or composites it | A plugin for OBS or a media framework is a native host with frames arriving as textures or buffers: the same import diff --git a/CHANGELOG.md b/CHANGELOG.md index 23444e2..10c20bc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,23 @@ follow semantic versioning as described in [CONTRIBUTING.md](CONTRIBUTING.md). ## [Unreleased] +## [0.4.0] - 2026-11-16 + +### Added + +- Frames from a host's own Vulkan device (Windows, Linux): `vulkan_interop` imports buffers and semaphores the host + exports on the same GPU, `copy_in` copies a frame into an input slot after a semaphore wait and `copy_out` copies an + output slot back before a signal, with no copy through the CPU. C: `fh_engine_vulkan_interop`, `fh_vk_row_pitch`, + `fh_vk_buffer_import` / `_destroy`, `fh_vk_semaphore_import` / `_destroy`, `fh_vk_signal`, `fh_stream_copy_in` / + `fh_stream_copy_out`, `fh_session_copy_in` / `fh_session_copy_out`; C++ `fh::VkBuffer`, `fh::VkSemaphore`, + `Stream::copyIn` / `copyOut`. The engine's Vulkan device enables the external memory and semaphore extensions + whenever the adapter has them. + +### Fixed + +- `tools/build_web.sh` remaps source paths, so the browser module no longer carries the builder's home directory in + its panic locations. + ## [0.3.7] - 2026-11-10 ### Changed diff --git a/Cargo.lock b/Cargo.lock index a7ca4e7..edbd23e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -548,8 +548,9 @@ dependencies = [ [[package]] name = "flyhigh-capi" -version = "0.3.7" +version = "0.4.0" dependencies = [ + "anyhow", "flyhigh-engine", "serde_json", "wgpu", @@ -557,7 +558,7 @@ dependencies = [ [[package]] name = "flyhigh-cli" -version = "0.3.7" +version = "0.4.0" dependencies = [ "anyhow", "bytemuck", @@ -572,9 +573,10 @@ dependencies = [ [[package]] name = "flyhigh-engine" -version = "0.3.7" +version = "0.4.0" dependencies = [ "anyhow", + "ash", "bytemuck", "flyhigh-options", "half", @@ -588,7 +590,7 @@ dependencies = [ [[package]] name = "flyhigh-options" -version = "0.3.7" +version = "0.4.0" dependencies = [ "proc-macro2", "quote", @@ -597,7 +599,7 @@ dependencies = [ [[package]] name = "flyhigh-player" -version = "0.3.7" +version = "0.4.0" dependencies = [ "anyhow", "flyhigh-engine", @@ -608,7 +610,7 @@ dependencies = [ [[package]] name = "flyhigh-web" -version = "0.3.7" +version = "0.4.0" dependencies = [ "console_error_panic_hook", "flyhigh-engine", diff --git a/Cargo.toml b/Cargo.toml index 6ca6ccc..8c71b16 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -5,7 +5,7 @@ members = ["crates/flyhigh-options", "crates/flyhigh-engine", "crates/flyhigh-ca default-members = ["crates/flyhigh-options", "crates/flyhigh-engine", "crates/flyhigh-capi", "crates/flyhigh-cli", "examples/player"] [workspace.package] -version = "0.3.7" +version = "0.4.0" edition = "2021" license = "Apache-2.0" repository = "https://github.com/getflyhigh/flyhigh-engine" diff --git a/bindings/cpp/flyhigh.hpp b/bindings/cpp/flyhigh.hpp index 48b4344..070b821 100644 --- a/bindings/cpp/flyhigh.hpp +++ b/bindings/cpp/flyhigh.hpp @@ -211,6 +211,16 @@ class Stream { } void warmUp() { detail::check(fh_stream_warm_up(h_.get(), e_), "fh_stream_warm_up"); } void reset() { detail::check(fh_stream_reset(h_.get()), "fh_stream_reset"); } + /// A frame from the host's Vulkan device into input slot `slot`, after `wait` reaches `value`. + void copyIn(uint32_t slot, const FhVkBuffer* b, uint32_t bytesPerRow, const FhVkSemaphore* wait = nullptr, + uint64_t value = 0) { + detail::check(fh_stream_copy_in(h_.get(), e_, slot, b, bytesPerRow, wait, value), "fh_stream_copy_in"); + } + /// Output slot `out` into the host's buffer, then `signal` at `value`. + void copyOut(uint32_t out, const FhVkBuffer* b, uint32_t bytesPerRow, const FhVkSemaphore* signal = nullptr, + uint64_t value = 0) const { + detail::check(fh_stream_copy_out(h_.get(), e_, out, b, bytesPerRow, signal, value), "fh_stream_copy_out"); + } std::string report() const { return detail::json([&](char* b, size_t n) { return fh_stream_report_json(h_.get(), b, n); }, "fh_stream_report_json"); } @@ -221,6 +231,31 @@ class Stream { uint32_t w_, hgt_; }; +/// Memory exported by the host's Vulkan device (fh_vk_buffer_import). +class VkBuffer { +public: + VkBuffer(const Engine& e, const FhVkMemory& m, uint64_t size) : h_(fh_vk_buffer_import(e.get(), &m, size)) { + if (!h_) throw detail::last("fh_vk_buffer_import"); + } + FhVkBuffer* get() const { return h_.get(); } + +private: + Handle h_; +}; + +/// A semaphore exported by the host's Vulkan device (fh_vk_semaphore_import). +class VkSemaphore { +public: + VkSemaphore(const Engine& e, intptr_t handle, bool timeline) + : h_(fh_vk_semaphore_import(e.get(), handle, timeline ? 1u : 0u)) { + if (!h_) throw detail::last("fh_vk_semaphore_import"); + } + FhVkSemaphore* get() const { return h_.get(); } + +private: + Handle h_; +}; + /// What to present on this display tick and what to generate (ARCHITECTURE.md); milliseconds of the host's /// monotonic clock. class Pacer { diff --git a/crates/flyhigh-capi/Cargo.toml b/crates/flyhigh-capi/Cargo.toml index 4423b33..6f9e7f2 100644 --- a/crates/flyhigh-capi/Cargo.toml +++ b/crates/flyhigh-capi/Cargo.toml @@ -14,3 +14,4 @@ crate-type = ["cdylib", "staticlib", "rlib"] flyhigh-engine.workspace = true serde_json.workspace = true wgpu.workspace = true +anyhow.workspace = true diff --git a/crates/flyhigh-capi/include/flyhigh.h b/crates/flyhigh-capi/include/flyhigh.h index acf9829..c0e6e4c 100644 --- a/crates/flyhigh-capi/include/flyhigh.h +++ b/crates/flyhigh-capi/include/flyhigh.h @@ -7,8 +7,8 @@ * FhStream frames with timestamps in, intervals out (fh_stream_*): the ring, the pair classifier, cadence * FhPacer what to present on this display tick and what to generate (fh_pacer_*): pure logic, no clock * FhGenerator the CPU convenience (fh_generator_*): source frames in, the frames to display out, in order - * Frames cross this ABI as RGBA8 bytes (width * height * 4); GPU textures come in through the interop calls - * (fh_session_import_d3d11, fh_stream_import_d3d11). + * Frames cross this ABI as RGBA8 bytes (width * height * 4); GPU frames come in through the interop calls + * (fh_session_import_d3d11, fh_stream_import_d3d11; fh_vk_* from the host's own Vulkan device). * Single-threaded use per handle. Errors: a call returns NULL or -1 and fh_last_error() holds the message. * Options: always start from the fh_*_options_default() struct and change what you need. */ @@ -262,6 +262,34 @@ int fh_generator_push(FhGenerator* generator, const uint8_t* rgba, size * NULL), 0 = queue empty, -1 = error */ int fh_generator_pop(FhGenerator* generator, uint8_t* rgba, size_t cap, float* phase, int* real); +/* ---- Vulkan interop: frames from the host's own Vulkan device (Windows, Linux) ---- + * The host exports memory and semaphores on the same GPU (compare its deviceUUID with fh_engine_vulkan_interop's): + * NT handles on Windows (OPAQUE_WIN32), file descriptors on Linux (OPAQUE_FD; owned by the engine once imported). + * Frames cross as RGBA8 rows of fh_vk_row_pitch(width) bytes in buffers: a dedicated allocation for a buffer with + * TRANSFER_SRC | TRANSFER_DST usage, imported with the exporter's allocation size and memory type index. + * copy_in waits on a semaphore before the engine reads the frame, copy_out signals one after the engine wrote it. + * Values: a timeline semaphore takes the value to wait for / signal (above 0); a binary one takes 0 and needs its + * signal submitted before the wait. */ +typedef struct FhVkBuffer FhVkBuffer; +typedef struct FhVkSemaphore FhVkSemaphore; +typedef struct FhVkMemory { + intptr_t handle; + uint64_t allocation_size; + uint32_t memory_type_index; +} FhVkMemory; + +int fh_engine_vulkan_interop(const FhEngine* engine, uint8_t* uuid); /* 1 supported (uuid: 16 bytes or NULL), 0 not, -1 */ +uint32_t fh_vk_row_pitch(uint32_t width); +FhVkBuffer* fh_vk_buffer_import(const FhEngine* engine, const FhVkMemory* memory, uint64_t size); +void fh_vk_buffer_destroy(FhVkBuffer* buffer); +FhVkSemaphore* fh_vk_semaphore_import(const FhEngine* engine, intptr_t handle, uint32_t timeline); +void fh_vk_semaphore_destroy(FhVkSemaphore* semaphore); /* waits for the engine's queue */ +int fh_vk_signal(const FhEngine* engine, const FhVkSemaphore* semaphore, uint64_t value); /* nothing copied */ +int fh_stream_copy_in(FhStream* stream, const FhEngine* engine, uint32_t slot, const FhVkBuffer* buffer, uint32_t bytes_per_row, const FhVkSemaphore* wait, uint64_t value); +int fh_stream_copy_out(const FhStream* stream, const FhEngine* engine, uint32_t out, const FhVkBuffer* buffer, uint32_t bytes_per_row, const FhVkSemaphore* signal, uint64_t value); +int fh_session_copy_in(FhSession* session, const FhEngine* engine, uint32_t slot, const FhVkBuffer* buffer, uint32_t bytes_per_row, const FhVkSemaphore* wait, uint64_t value); +int fh_session_copy_out(const FhSession* session, const FhEngine* engine, uint32_t out, const FhVkBuffer* buffer, uint32_t bytes_per_row, const FhVkSemaphore* signal, uint64_t value); + #ifdef __cplusplus } #endif diff --git a/crates/flyhigh-capi/src/lib.rs b/crates/flyhigh-capi/src/lib.rs index 19507cc..ca469d6 100644 --- a/crates/flyhigh-capi/src/lib.rs +++ b/crates/flyhigh-capi/src/lib.rs @@ -6,14 +6,17 @@ //! - core.rs: `fh_engine_*` (the device), `fh_model_*`, `fh_session_*` (two frames in, one frame out) //! - stream.rs: `fh_stream_*` (frames with timestamps in, intervals out) //! - pacer.rs: `fh_pacer_*` (what to present on this tick, what to generate) +//! - vulkan.rs: `fh_vk_*`, `fh_*_copy_in` / `fh_*_copy_out` (frames from the host's own Vulkan device) //! - here: `fh_generator_*`, the CPU convenience over `api::Generator`, and `fh_last_error` mod core; mod pacer; mod stream; +mod vulkan; pub use core::*; pub use pacer::*; pub use stream::*; +pub use vulkan::*; use flyhigh_engine::api::{Generator, Mode, Options, OutFrame}; use flyhigh_engine::{Engine, Model}; diff --git a/crates/flyhigh-capi/src/vulkan.rs b/crates/flyhigh-capi/src/vulkan.rs new file mode 100644 index 0000000..9103c6a --- /dev/null +++ b/crates/flyhigh-capi/src/vulkan.rs @@ -0,0 +1,359 @@ +//! `fh_vk_*`, `fh_stream_copy_*`, `fh_session_copy_*`: frames from the host's own Vulkan device through exported +//! memory and semaphores (`vulkan_interop`). Windows and Linux; elsewhere every call fails with a message. +use crate::{guard, set_err}; +use flyhigh_engine::stream::Stream; +use flyhigh_engine::{Engine, Session}; + +#[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] +use flyhigh_engine::vulkan_interop::{self as vi, ExternalMemory, Semaphore}; + +#[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] +const UNSUPPORTED: &str = "Vulkan interop is built for Windows and Linux only"; + +/// Exported memory: the handle (an NT handle on Windows, a file descriptor on Linux) and the exporter's allocation. +#[repr(C)] +pub struct FhVkMemory { + pub handle: isize, + pub allocation_size: u64, + pub memory_type_index: u32, +} + +pub struct FhVkBuffer(wgpu::Buffer); + +pub struct FhVkSemaphore { + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + inner: Semaphore, +} + +macro_rules! live { + ($p:expr) => { + if $p.is_null() { + set_err("null argument"); + return -1; + } + }; +} + +/// 1 when the engine's device takes memory and semaphores from another Vulkan device (then `uuid`, if not null, +/// receives its 16-byte deviceUUID to match the host's GPU), 0 when it cannot, -1 on error. +/// # Safety +/// `e` is a live handle; `uuid` is null or points to 16 writable bytes. +#[no_mangle] +pub unsafe extern "C" fn fh_engine_vulkan_interop(e: *const Engine, uuid: *mut u8) -> i32 { + guard(-1, || { + live!(e); + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + { + let e = unsafe { &*e }; + if !vi::supported(e) { + return 0; + } + if !uuid.is_null() { + let Some(id) = vi::device_uuid(e) else { return 0 }; + unsafe { std::ptr::copy_nonoverlapping(id.as_ptr(), uuid, 16) }; + } + 1 + } + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + { + let _ = uuid; + 0 + } + }) +} + +/// Bytes per row of an RGBA8 frame `width` wide in an interop buffer (a multiple of 256). +#[no_mangle] +pub extern "C" fn fh_vk_row_pitch(width: u32) -> u32 { + (width * 4).next_multiple_of(wgpu::COPY_BYTES_PER_ROW_ALIGNMENT) +} + +/// Imports `size` bytes of exported memory as a buffer (a dedicated allocation for a buffer with TRANSFER_SRC | +/// TRANSFER_DST usage on the host). NULL on error. A file descriptor belongs to the engine once this succeeds. +/// # Safety +/// `e` is a live handle; `m` points to a valid `FhVkMemory`. +#[no_mangle] +pub unsafe extern "C" fn fh_vk_buffer_import(e: *const Engine, m: *const FhVkMemory, size: u64) -> *mut FhVkBuffer { + guard(std::ptr::null_mut(), || { + if e.is_null() || m.is_null() { + set_err("null argument"); + return std::ptr::null_mut(); + } + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + { + let m = unsafe { &*m }; + let memory = ExternalMemory { + handle: m.handle, + allocation_size: m.allocation_size, + memory_type_index: m.memory_type_index, + }; + match vi::import_buffer(unsafe { &*e }, &memory, size) { + Ok(b) => Box::into_raw(Box::new(FhVkBuffer(b))), + Err(err) => { + set_err(format!("{err:#}")); + std::ptr::null_mut() + } + } + } + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + { + let _ = size; + set_err(UNSUPPORTED); + std::ptr::null_mut() + } + }) +} + +/// # Safety +/// `b` is null or a handle from `fh_vk_buffer_import` that is not used afterwards. +#[no_mangle] +pub unsafe extern "C" fn fh_vk_buffer_destroy(b: *mut FhVkBuffer) { + guard((), || { + if !b.is_null() { + drop(unsafe { Box::from_raw(b) }); + } + }) +} + +/// Imports an exported semaphore; `timeline` 1 for a timeline semaphore, 0 for a binary one. NULL on error. +/// # Safety +/// `e` is a live handle. +#[no_mangle] +pub unsafe extern "C" fn fh_vk_semaphore_import(e: *const Engine, handle: isize, timeline: u32) -> *mut FhVkSemaphore { + guard(std::ptr::null_mut(), || { + if e.is_null() { + set_err("null argument"); + return std::ptr::null_mut(); + } + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + { + match vi::import_semaphore(unsafe { &*e }, handle, timeline != 0) { + Ok(inner) => Box::into_raw(Box::new(FhVkSemaphore { inner })), + Err(err) => { + set_err(format!("{err:#}")); + std::ptr::null_mut() + } + } + } + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + { + let _ = (handle, timeline); + set_err(UNSUPPORTED); + std::ptr::null_mut() + } + }) +} + +/// Waits for the engine's queue to finish with the semaphore, then destroys it. +/// # Safety +/// `s` is null or a handle from `fh_vk_semaphore_import` that is not used afterwards. +#[no_mangle] +pub unsafe extern "C" fn fh_vk_semaphore_destroy(s: *mut FhVkSemaphore) { + guard((), || { + if !s.is_null() { + drop(unsafe { Box::from_raw(s) }); + } + }) +} + +/// Signals `s` (at `value` for a timeline semaphore, 0 for a binary one) once every earlier submit of the engine is +/// done, with nothing copied. 0 / -1. +/// # Safety +/// `e` and `s` are live handles. +#[no_mangle] +pub unsafe extern "C" fn fh_vk_signal(e: *const Engine, s: *const FhVkSemaphore, value: u64) -> i32 { + guard(-1, || { + live!(e); + live!(s); + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + { + match vi::signal(unsafe { &*e }, unsafe { &(*s).inner }, value) { + Ok(()) => 0, + Err(err) => { + set_err(format!("{err:#}")); + -1 + } + } + } + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + { + let _ = value; + set_err(UNSUPPORTED); + -1 + } + }) +} + +#[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] +unsafe fn semaphore<'a>(s: *const FhVkSemaphore, value: u64) -> Option<(&'a Semaphore, u64)> { + (!s.is_null()).then(|| (unsafe { &(*s).inner }, value)) +} + +#[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] +fn status(r: anyhow::Result<()>) -> i32 { + match r { + Ok(()) => 0, + Err(err) => { + set_err(format!("{err:#}")); + -1 + } + } +} + +fn check_slot(slot: u32, count: usize, what: &str) -> bool { + if slot as usize >= count { + set_err(format!("bad {what} slot {slot} (there are {count})")); + return false; + } + true +} + +/// Copies a frame from `b` (rows of `bytes_per_row`) into input slot `slot`, after `wait` (null: no wait) reaches +/// `value` (0 for a binary semaphore). 0 / -1. +/// # Safety +/// `s`, `e` and `b` are live handles; `wait` is null or a live handle. +#[no_mangle] +pub unsafe extern "C" fn fh_stream_copy_in( + s: *mut Stream, + e: *const Engine, + slot: u32, + b: *const FhVkBuffer, + bytes_per_row: u32, + wait: *const FhVkSemaphore, + value: u64, +) -> i32 { + guard(-1, || { + live!(s); + live!(e); + live!(b); + let s = unsafe { &*s }; + if !check_slot(slot, s.inputs(), "input") { + return -1; + } + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + { + let tex = s.input_texture(slot as usize); + status(vi::copy_in(unsafe { &*e }, unsafe { &(*b).0 }, bytes_per_row, tex, unsafe { + semaphore(wait, value) + })) + } + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + { + let _ = (bytes_per_row, wait, value); + set_err(UNSUPPORTED); + -1 + } + }) +} + +/// Copies output slot `out` into `b`, then signals `signal` (null: none) at `value` (0 for a binary semaphore) once +/// this and every earlier submit of the engine is done. 0 / -1. +/// # Safety +/// `s`, `e` and `b` are live handles; `signal` is null or a live handle. +#[no_mangle] +pub unsafe extern "C" fn fh_stream_copy_out( + s: *const Stream, + e: *const Engine, + out: u32, + b: *const FhVkBuffer, + bytes_per_row: u32, + signal: *const FhVkSemaphore, + value: u64, +) -> i32 { + guard(-1, || { + live!(s); + live!(e); + live!(b); + let s = unsafe { &*s }; + if !check_slot(out, s.outputs(), "output") { + return -1; + } + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + { + let tex = s.output_texture(out as usize); + status(vi::copy_out(unsafe { &*e }, tex, unsafe { &(*b).0 }, bytes_per_row, unsafe { + semaphore(signal, value) + })) + } + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + { + let _ = (bytes_per_row, signal, value); + set_err(UNSUPPORTED); + -1 + } + }) +} + +/// `fh_stream_copy_in` for a session's input slot. +/// # Safety +/// As `fh_stream_copy_in`. +#[no_mangle] +pub unsafe extern "C" fn fh_session_copy_in( + s: *mut Session, + e: *const Engine, + slot: u32, + b: *const FhVkBuffer, + bytes_per_row: u32, + wait: *const FhVkSemaphore, + value: u64, +) -> i32 { + guard(-1, || { + live!(s); + live!(e); + live!(b); + let s = unsafe { &*s }; + if !check_slot(slot, s.inputs(), "input") { + return -1; + } + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + { + let tex = s.input_texture(slot as usize); + status(vi::copy_in(unsafe { &*e }, unsafe { &(*b).0 }, bytes_per_row, tex, unsafe { + semaphore(wait, value) + })) + } + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + { + let _ = (bytes_per_row, wait, value); + set_err(UNSUPPORTED); + -1 + } + }) +} + +/// `fh_stream_copy_out` for a session's output slot. +/// # Safety +/// As `fh_stream_copy_out`. +#[no_mangle] +pub unsafe extern "C" fn fh_session_copy_out( + s: *const Session, + e: *const Engine, + out: u32, + b: *const FhVkBuffer, + bytes_per_row: u32, + signal: *const FhVkSemaphore, + value: u64, +) -> i32 { + guard(-1, || { + live!(s); + live!(e); + live!(b); + let s = unsafe { &*s }; + if !check_slot(out, s.outputs(), "output") { + return -1; + } + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + { + let tex = s.output_texture(out as usize); + status(vi::copy_out(unsafe { &*e }, tex, unsafe { &(*b).0 }, bytes_per_row, unsafe { + semaphore(signal, value) + })) + } + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + { + let _ = (bytes_per_row, signal, value); + set_err(UNSUPPORTED); + -1 + } + }) +} diff --git a/crates/flyhigh-capi/tests/core.rs b/crates/flyhigh-capi/tests/core.rs index 4fd1c80..c7645d9 100644 --- a/crates/flyhigh-capi/tests/core.rs +++ b/crates/flyhigh-capi/tests/core.rs @@ -190,3 +190,20 @@ fn a_panic_inside_a_call_becomes_an_error_not_an_abort() { assert_eq!(r, -1); assert!(last_error().contains("boom"), "{}", last_error()); } + +#[test] +fn vulkan_interop_calls_check_their_arguments_over_c() { + assert_eq!(flyhigh::fh_vk_row_pitch(1), 256); + assert_eq!(flyhigh::fh_vk_row_pitch(1920), 7680); + assert_eq!(flyhigh::fh_vk_row_pitch(1366), 5632); + let memory = flyhigh::FhVkMemory { handle: 0, allocation_size: 0, memory_type_index: 0 }; + assert!(unsafe { flyhigh::fh_vk_buffer_import(std::ptr::null(), &memory, 1024) }.is_null()); + assert!(last_error().contains("null"), "{}", last_error()); + assert!(unsafe { flyhigh::fh_vk_semaphore_import(std::ptr::null(), 0, 1) }.is_null()); + assert_eq!(unsafe { flyhigh::fh_engine_vulkan_interop(std::ptr::null(), std::ptr::null_mut()) }, -1); + assert_eq!(unsafe { flyhigh::fh_vk_signal(std::ptr::null(), std::ptr::null(), 1) }, -1); + unsafe { + flyhigh::fh_vk_buffer_destroy(std::ptr::null_mut()); + flyhigh::fh_vk_semaphore_destroy(std::ptr::null_mut()); + } +} diff --git a/crates/flyhigh-engine/Cargo.toml b/crates/flyhigh-engine/Cargo.toml index 7e5189d..0ea2383 100644 --- a/crates/flyhigh-engine/Cargo.toml +++ b/crates/flyhigh-engine/Cargo.toml @@ -20,6 +20,10 @@ libm.workspace = true half.workspace = true anyhow.workspace = true +# vulkan_interop.rs: memory and semaphores of another Vulkan device (the version wgpu-hal uses) +[target.'cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))'.dependencies] +ash = "0.38" + # interop.rs: D3D11 shared textures into the Vulkan device (the handle type; the version wgpu-hal uses) [target.'cfg(windows)'.dependencies] windows = { version = "0.62", features = ["Win32_Foundation", "Win32_System_LibraryLoader"] } diff --git a/crates/flyhigh-engine/src/gpu/device.rs b/crates/flyhigh-engine/src/gpu/device.rs index a36a8f7..42c0af0 100644 --- a/crates/flyhigh-engine/src/gpu/device.rs +++ b/crates/flyhigh-engine/src/gpu/device.rs @@ -160,24 +160,27 @@ impl Gpu { // fused4 (coop-only microbench kernel) stages ~23 KB in shared; the WebGPU default is 16 KB and the portable // kernels fit in it, so ask for 32 KB only where the adapter has it let wg_storage = alim.max_compute_workgroup_storage_size.clamp(16384, 32768); - let (device, queue) = adapter - .request_device(&wgpu::DeviceDescriptor { - label: Some("flyhigh"), - required_features: feats, - required_limits: wgpu::Limits { - max_compute_workgroup_storage_size: wg_storage, - ..wgpu::Limits::default() - }, - memory_hints: if opts.compact_memory { - wgpu::MemoryHints::MemoryUsage - } else { - wgpu::MemoryHints::Performance - }, - experimental_features: unsafe { wgpu::ExperimentalFeatures::enabled() }, - trace: wgpu::Trace::Off, - }) - .await - .map_err(|e| anyhow::anyhow!("no wgpu device: {e:?}"))?; + let desc = wgpu::DeviceDescriptor { + label: Some("flyhigh"), + required_features: feats, + required_limits: wgpu::Limits { max_compute_workgroup_storage_size: wg_storage, ..wgpu::Limits::default() }, + memory_hints: if opts.compact_memory { + wgpu::MemoryHints::MemoryUsage + } else { + wgpu::MemoryHints::Performance + }, + experimental_features: unsafe { wgpu::ExperimentalFeatures::enabled() }, + trace: wgpu::Trace::Off, + }; + // hosts on their own Vulkan device hand frames over through external memory and semaphores + #[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] + let opened = crate::vulkan_interop::open_device(&adapter, &desc); + #[cfg(not(all(any(windows, target_os = "linux"), not(target_arch = "wasm32"))))] + let opened: Option> = None; + let (device, queue) = match opened { + Some(opened) => opened?, + None => adapter.request_device(&desc).await.map_err(|e| anyhow::anyhow!("no wgpu device: {e:?}"))?, + }; let gpu = Gpu::assemble(instance, adapter, device, queue, opts, caps); if verbose { if let Some(c) = gpu.shader_compiler { diff --git a/crates/flyhigh-engine/src/lib.rs b/crates/flyhigh-engine/src/lib.rs index 2f47c85..f342a7a 100644 --- a/crates/flyhigh-engine/src/lib.rs +++ b/crates/flyhigh-engine/src/lib.rs @@ -15,6 +15,8 @@ pub mod net; pub mod ops; pub mod options; pub mod session; +#[cfg(all(any(windows, target_os = "linux"), not(target_arch = "wasm32")))] +pub mod vulkan_interop; // Every type and function keeps its pre-split path at the crate root. #[cfg(not(target_arch = "wasm32"))] diff --git a/crates/flyhigh-engine/src/vulkan_interop.rs b/crates/flyhigh-engine/src/vulkan_interop.rs new file mode 100644 index 0000000..1fb472d --- /dev/null +++ b/crates/flyhigh-engine/src/vulkan_interop.rs @@ -0,0 +1,336 @@ +//! Frames from a host's own Vulkan device (an emulator, a game, a capture) on the engine's Vulkan device, with no copy +//! through the CPU. The host allocates exportable memory and semaphores and hands their handles over: an NT handle on +//! Windows (`VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT`, the same for semaphores), a file descriptor elsewhere +//! (`OPAQUE_FD`). The host keeps a Windows handle and may close it after the import; a file descriptor belongs to the +//! engine once the import succeeds. +//! +//! Frames cross as buffers, not images: an image's layout and compression are private to the device that made it, so +//! two devices sharing one need matching create parameters and layout handoffs on every frame, while a buffer is the +//! same bytes on both sides. Rows are RGBA8, `bytes_per_row` a multiple of 256 (`row_pitch`). The copies in and out +//! cost about 0.05 ms per 1080p frame each. +//! +//! Ordering is the semaphores' job. The host signals after writing a frame and `copy_in` makes the engine's next +//! submit wait for it; `copy_out` signals after the engine has written its frame, and the host waits before reading. +//! A binary semaphore needs its signal submitted before the wait (value 0); a timeline semaphore waits for a value. +use crate::Engine; +use anyhow::{bail, ensure, Context, Result}; +use ash::vk; +use wgpu::hal::api::Vulkan; + +#[cfg(windows)] +const MEMORY_HANDLE: vk::ExternalMemoryHandleTypeFlags = vk::ExternalMemoryHandleTypeFlags::OPAQUE_WIN32; +#[cfg(not(windows))] +const MEMORY_HANDLE: vk::ExternalMemoryHandleTypeFlags = vk::ExternalMemoryHandleTypeFlags::OPAQUE_FD; +#[cfg(windows)] +const SEMAPHORE_HANDLE: vk::ExternalSemaphoreHandleTypeFlags = vk::ExternalSemaphoreHandleTypeFlags::OPAQUE_WIN32; +#[cfg(not(windows))] +const SEMAPHORE_HANDLE: vk::ExternalSemaphoreHandleTypeFlags = vk::ExternalSemaphoreHandleTypeFlags::OPAQUE_FD; + +#[cfg(windows)] +const EXTENSIONS: [&std::ffi::CStr; 2] = + [ash::khr::external_memory_win32::NAME, ash::khr::external_semaphore_win32::NAME]; +#[cfg(not(windows))] +const EXTENSIONS: [&std::ffi::CStr; 2] = [ash::khr::external_memory_fd::NAME, ash::khr::external_semaphore_fd::NAME]; + +/// The bytes per row of an RGBA8 frame `width` wide in an interop buffer. +pub fn row_pitch(width: u32) -> u32 { + (width * 4).next_multiple_of(wgpu::COPY_BYTES_PER_ROW_ALIGNMENT) +} + +/// The size of an interop buffer for an RGBA8 frame. +pub fn buffer_size(width: u32, height: u32) -> u64 { + row_pitch(width) as u64 * height as u64 +} + +/// Opens the device with the external memory and semaphore extensions when the adapter has them, else `None` and the +/// caller requests the device as usual. +pub(crate) fn open_device( + adapter: &wgpu::Adapter, + desc: &wgpu::DeviceDescriptor<'_>, +) -> Option> { + if adapter.get_info().backend != wgpu::Backend::Vulkan { + return None; + } + let open = unsafe { + let hal = adapter.as_hal::()?; + if !EXTENSIONS.iter().all(|e| hal.physical_device_capabilities().supports_extension(e)) { + return None; + } + hal.open_with_callback( + desc.required_features, + &desc.required_limits, + &desc.memory_hints, + Some(Box::new(|args: wgpu::hal::vulkan::CreateDeviceCallbackArgs<'_, '_, '_>| { + for e in EXTENSIONS { + if !args.extensions.contains(&e) { + args.extensions.push(e); + } + } + })), + ) + }; + let open = match open { + Ok(open) => open, + Err(e) => return Some(Err(anyhow::anyhow!("opening the Vulkan device: {e:?}"))), + }; + Some(unsafe { adapter.create_device_from_hal(open, desc) }.map_err(|e| anyhow::anyhow!("no wgpu device: {e:?}"))) +} + +fn has_extensions(engine: &Engine) -> bool { + let device = &engine.gpu().device; + match unsafe { device.as_hal::() } { + Some(hal) => EXTENSIONS.iter().all(|e| hal.enabled_device_extensions().contains(e)), + None => false, + } +} + +/// Whether this engine's device can take buffers and semaphores from another Vulkan device. +pub fn supported(engine: &Engine) -> bool { + has_extensions(engine) +} + +/// The engine's `VkPhysicalDeviceIDProperties::deviceUUID`: memory and semaphores are shared only between devices on +/// the same GPU and driver, so a host compares it with its own before exporting anything. +pub fn device_uuid(engine: &Engine) -> Option<[u8; 16]> { + let device = &engine.gpu().device; + let hal = unsafe { device.as_hal::() }?; + let instance = hal.shared_instance().raw_instance(); + let mut id = vk::PhysicalDeviceIDProperties::default(); + let mut props = vk::PhysicalDeviceProperties2::default().push_next(&mut id); + unsafe { instance.get_physical_device_properties2(hal.raw_physical_device(), &mut props) }; + Some(id.device_uuid) +} + +/// Exported memory as the host allocated it. Vulkan requires the import to repeat the exporter's allocation size and +/// memory type index (the devices share one physical device, so the indices agree); the allocation is a dedicated one +/// for a buffer with `TRANSFER_SRC | TRANSFER_DST` usage. +#[derive(Clone, Copy, Debug)] +pub struct ExternalMemory { + pub handle: isize, + pub allocation_size: u64, + pub memory_type_index: u32, +} + +/// Imports `size` bytes of another device's memory as a buffer the engine copies frames to and from. +pub fn import_buffer(engine: &Engine, memory: &ExternalMemory, size: u64) -> Result { + ensure!(memory.handle != 0 && memory.handle != -1, "memory handle is not valid"); + ensure!(size > 0, "buffer size is 0"); + ensure!(supported(engine), "this device cannot import Vulkan memory"); + let device = &engine.gpu().device; + let hal_buffer = unsafe { + let hal = device.as_hal::().context("the engine device is not Vulkan")?; + let raw = hal.raw_device(); + let mut external = vk::ExternalMemoryBufferCreateInfo::default().handle_types(MEMORY_HANDLE); + let info = vk::BufferCreateInfo::default() + .size(size) + .usage(vk::BufferUsageFlags::TRANSFER_SRC | vk::BufferUsageFlags::TRANSFER_DST) + .sharing_mode(vk::SharingMode::EXCLUSIVE) + .push_next(&mut external); + let buffer = raw.create_buffer(&info, None).context("vkCreateBuffer")?; + let device_memory = match import_memory(raw, buffer, memory) { + Ok(device_memory) => device_memory, + Err(e) => { + raw.destroy_buffer(buffer, None); + return Err(e); + } + }; + if let Err(e) = raw.bind_buffer_memory(buffer, device_memory, 0) { + raw.destroy_buffer(buffer, None); + raw.free_memory(device_memory, None); + bail!("vkBindBufferMemory: {e}"); + } + wgpu::hal::vulkan::Buffer::from_raw_managed(buffer, device_memory, 0, size) + }; + Ok(unsafe { + device.create_buffer_from_hal::( + hal_buffer, + &wgpu::BufferDescriptor { + label: Some("fh_vk_imported"), + size, + usage: wgpu::BufferUsages::COPY_SRC | wgpu::BufferUsages::COPY_DST, + mapped_at_creation: false, + }, + ) + }) +} + +unsafe fn import_memory(raw: &ash::Device, buffer: vk::Buffer, memory: &ExternalMemory) -> Result { + let req = unsafe { raw.get_buffer_memory_requirements(buffer) }; + ensure!( + req.memory_type_bits & (1 << memory.memory_type_index) != 0, + "memory type {} cannot back a transfer buffer here", + memory.memory_type_index + ); + ensure!( + memory.allocation_size >= req.size, + "allocation of {} bytes is smaller than the {} the buffer needs", + memory.allocation_size, + req.size + ); + let mut dedicated = vk::MemoryDedicatedAllocateInfo::default().buffer(buffer); + #[cfg(windows)] + let mut import = + vk::ImportMemoryWin32HandleInfoKHR::default().handle_type(MEMORY_HANDLE).handle(memory.handle as vk::HANDLE); + #[cfg(not(windows))] + let mut import = vk::ImportMemoryFdInfoKHR::default().handle_type(MEMORY_HANDLE).fd(memory.handle as i32); + let alloc = vk::MemoryAllocateInfo::default() + .allocation_size(memory.allocation_size) + .memory_type_index(memory.memory_type_index) + .push_next(&mut import) + .push_next(&mut dedicated); + unsafe { raw.allocate_memory(&alloc, None) }.context("importing the memory (vkAllocateMemory)") +} + +/// A semaphore of another device, imported into the engine's device. +pub struct Semaphore { + raw: vk::Semaphore, + timeline: bool, + device: wgpu::Device, +} + +impl Semaphore { + pub fn is_timeline(&self) -> bool { + self.timeline + } + + fn value(&self, value: u64) -> Result> { + match (self.timeline, value) { + (true, 0) => bail!("a timeline semaphore needs a value above 0"), + (true, v) => Ok(Some(v)), + (false, 0) => Ok(None), + (false, _) => bail!("a binary semaphore takes value 0"), + } + } +} + +impl Drop for Semaphore { + fn drop(&mut self) { + // the queue may still hold a wait or a signal on it + let _ = self.device.poll(wgpu::PollType::wait_indefinitely()); + if let Some(hal) = unsafe { self.device.as_hal::() } { + unsafe { hal.raw_device().destroy_semaphore(self.raw, None) }; + } + } +} + +/// Imports the semaphore behind `handle`; `timeline` says which kind the host created. +pub fn import_semaphore(engine: &Engine, handle: isize, timeline: bool) -> Result { + ensure!(handle != 0 && handle != -1, "semaphore handle is not valid"); + ensure!(supported(engine), "this device cannot import Vulkan semaphores"); + let device = &engine.gpu().device; + let raw = unsafe { + let hal = device.as_hal::().context("the engine device is not Vulkan")?; + let raw = hal.raw_device(); + let instance = hal.shared_instance().raw_instance(); + let mut kind = vk::SemaphoreTypeCreateInfo::default().semaphore_type(if timeline { + vk::SemaphoreType::TIMELINE + } else { + vk::SemaphoreType::BINARY + }); + let semaphore = raw.create_semaphore(&vk::SemaphoreCreateInfo::default().push_next(&mut kind), None)?; + #[cfg(windows)] + let imported = ash::khr::external_semaphore_win32::Device::new(instance, raw).import_semaphore_win32_handle( + &vk::ImportSemaphoreWin32HandleInfoKHR::default() + .semaphore(semaphore) + .handle_type(SEMAPHORE_HANDLE) + .handle(handle as vk::HANDLE), + ); + #[cfg(not(windows))] + let imported = ash::khr::external_semaphore_fd::Device::new(instance, raw).import_semaphore_fd( + &vk::ImportSemaphoreFdInfoKHR::default() + .semaphore(semaphore) + .handle_type(SEMAPHORE_HANDLE) + .fd(handle as i32), + ); + if let Err(e) = imported { + raw.destroy_semaphore(semaphore, None); + bail!("importing the semaphore: {e}"); + } + semaphore + }; + Ok(Semaphore { raw, timeline, device: device.clone() }) +} + +fn check_rows(texture: &wgpu::Texture, buffer: &wgpu::Buffer, bytes_per_row: u32) -> Result<()> { + ensure!(texture.format() == wgpu::TextureFormat::Rgba8Unorm, "interop frames are rgba8unorm"); + ensure!( + bytes_per_row.is_multiple_of(wgpu::COPY_BYTES_PER_ROW_ALIGNMENT) && bytes_per_row >= texture.width() * 4, + "bytes_per_row {bytes_per_row}: at least {} and a multiple of 256", + texture.width() * 4 + ); + ensure!( + buffer.size() >= bytes_per_row as u64 * texture.height() as u64, + "buffer of {} bytes is smaller than {} rows of {bytes_per_row}", + buffer.size(), + texture.height() + ); + Ok(()) +} + +fn layout(bytes_per_row: u32, height: u32) -> wgpu::TexelCopyBufferLayout { + wgpu::TexelCopyBufferLayout { offset: 0, bytes_per_row: Some(bytes_per_row), rows_per_image: Some(height) } +} + +/// Copies a frame from `buffer` into `texture` (a stream or session input), after `wait` when given. The wait gates +/// this submit; everything later that reads the texture is ordered after the copy on the queue. +pub fn copy_in( + engine: &Engine, + buffer: &wgpu::Buffer, + bytes_per_row: u32, + texture: &wgpu::Texture, + wait: Option<(&Semaphore, u64)>, +) -> Result<()> { + check_rows(texture, buffer, bytes_per_row)?; + let gpu = engine.gpu(); + let mut enc = gpu.device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("fh_vk_in") }); + enc.copy_buffer_to_texture( + wgpu::TexelCopyBufferInfo { buffer, layout: layout(bytes_per_row, texture.height()) }, + texture.as_image_copy(), + texture.size(), + ); + if let Some((semaphore, value)) = wait { + let value = semaphore.value(value)?; + let queue = unsafe { gpu.queue.as_hal::() }.context("the engine queue is not Vulkan")?; + queue.add_wait_semaphore(semaphore.raw, value, vk::PipelineStageFlags::TRANSFER); + } + gpu.queue.submit([enc.finish()]); + Ok(()) +} + +/// Copies `texture` (a generated output, or an input to pass a real frame through) into `buffer`, then signals +/// `signal` when given, once this and every earlier submit of the engine is done. +pub fn copy_out( + engine: &Engine, + texture: &wgpu::Texture, + buffer: &wgpu::Buffer, + bytes_per_row: u32, + signal: Option<(&Semaphore, u64)>, +) -> Result<()> { + check_rows(texture, buffer, bytes_per_row)?; + let gpu = engine.gpu(); + let mut enc = gpu.device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("fh_vk_out") }); + enc.copy_texture_to_buffer( + texture.as_image_copy(), + wgpu::TexelCopyBufferInfo { buffer, layout: layout(bytes_per_row, texture.height()) }, + texture.size(), + ); + if let Some((semaphore, value)) = signal { + let value = semaphore.value(value)?; + let queue = unsafe { gpu.queue.as_hal::() }.context("the engine queue is not Vulkan")?; + queue.add_signal_semaphore(semaphore.raw, value); + } + gpu.queue.submit([enc.finish()]); + Ok(()) +} + +/// Signals `semaphore` once every submit so far is done, with nothing copied: for a frame the host shows without +/// anything from the engine, so its wait still pairs with a signal. +pub fn signal(engine: &Engine, semaphore: &Semaphore, value: u64) -> Result<()> { + let value = semaphore.value(value)?; + let gpu = engine.gpu(); + let queue = unsafe { gpu.queue.as_hal::() }.context("the engine queue is not Vulkan")?; + queue.add_signal_semaphore(semaphore.raw, value); + let enc = gpu.device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("fh_vk_signal") }); + gpu.queue.submit([enc.finish()]); + Ok(()) +} diff --git a/crates/flyhigh-engine/tests/interop_vulkan.rs b/crates/flyhigh-engine/tests/interop_vulkan.rs new file mode 100644 index 0000000..bbebd62 --- /dev/null +++ b/crates/flyhigh-engine/tests/interop_vulkan.rs @@ -0,0 +1,372 @@ +// Frames from another Vulkan device: a host device of its own writes two frames into exported buffers, signals a +// timeline semaphore, the engine copies them in after the wait, generates the in-between, copies it out and signals; +// the host waits and reads it back. It must match the engine's own upload path byte for byte. Ignored by default; needs +// a Vulkan GPU (`cargo test --release --test interop_vulkan -- --ignored`). +#![cfg(any(windows, target_os = "linux"))] +use ash::vk; +use flyhigh_engine::vulkan_interop::{self, buffer_size, row_pitch}; +use flyhigh_engine::{Engine, EngineOptions, Model, Session, SessionOptions}; +use std::path::PathBuf; + +const W: u32 = 200; +const H: u32 = 120; + +fn frame(dx: f32) -> Vec { + let mut v = vec![0u8; (W * H * 4) as usize]; + for y in 0..H { + for x in 0..W { + let u = (x as f32 - dx) * 0.07; + let w = y as f32 * 0.05; + let o = ((y * W + x) * 4) as usize; + v[o] = (128.0 + 100.0 * u.sin()) as u8; + v[o + 1] = (128.0 + 100.0 * (u * 0.7 + w).cos()) as u8; + v[o + 2] = (128.0 + 90.0 * (u + w).sin()) as u8; + v[o + 3] = 255; + } + } + v +} + +fn pitched(rgba: &[u8]) -> Vec { + let pitch = row_pitch(W) as usize; + let mut v = vec![0u8; pitch * H as usize]; + for y in 0..H as usize { + let row = W as usize * 4; + v[y * pitch..y * pitch + row].copy_from_slice(&rgba[y * row..(y + 1) * row]); + } + v +} + +fn unpitched(bytes: &[u8]) -> Vec { + let pitch = row_pitch(W) as usize; + let row = W as usize * 4; + (0..H as usize).flat_map(|y| bytes[y * pitch..y * pitch + row].iter().copied()).collect() +} + +#[cfg(windows)] +const MEMORY_HANDLE: vk::ExternalMemoryHandleTypeFlags = vk::ExternalMemoryHandleTypeFlags::OPAQUE_WIN32; +#[cfg(not(windows))] +const MEMORY_HANDLE: vk::ExternalMemoryHandleTypeFlags = vk::ExternalMemoryHandleTypeFlags::OPAQUE_FD; +#[cfg(windows)] +const SEMAPHORE_HANDLE: vk::ExternalSemaphoreHandleTypeFlags = vk::ExternalSemaphoreHandleTypeFlags::OPAQUE_WIN32; +#[cfg(not(windows))] +const SEMAPHORE_HANDLE: vk::ExternalSemaphoreHandleTypeFlags = vk::ExternalSemaphoreHandleTypeFlags::OPAQUE_FD; + +struct Host { + _entry: ash::Entry, + instance: ash::Instance, + physical: vk::PhysicalDevice, + device: ash::Device, + queue: vk::Queue, + pool: vk::CommandPool, +} + +struct HostBuffer { + buffer: vk::Buffer, + memory: vk::DeviceMemory, + allocation_size: u64, + memory_type_index: u32, +} + +impl Host { + fn new(uuid: [u8; 16]) -> Option { + let entry = unsafe { ash::Entry::load() }.ok()?; + let app = vk::ApplicationInfo::default().api_version(vk::API_VERSION_1_2); + let instance = + unsafe { entry.create_instance(&vk::InstanceCreateInfo::default().application_info(&app), None) }.ok()?; + let physical = unsafe { instance.enumerate_physical_devices() }.ok()?.into_iter().find(|&p| { + let mut id = vk::PhysicalDeviceIDProperties::default(); + let mut props = vk::PhysicalDeviceProperties2::default().push_next(&mut id); + unsafe { instance.get_physical_device_properties2(p, &mut props) }; + id.device_uuid == uuid + })?; + let family = unsafe { instance.get_physical_device_queue_family_properties(physical) }.iter().position(|f| { + f.queue_flags.contains(vk::QueueFlags::TRANSFER) || f.queue_flags.contains(vk::QueueFlags::GRAPHICS) + })? as u32; + // software drivers may import but not export these, or not timeline semaphores + let mut timeline_kind = vk::SemaphoreTypeCreateInfo::default().semaphore_type(vk::SemaphoreType::TIMELINE); + let semaphore_query = vk::PhysicalDeviceExternalSemaphoreInfo::default() + .handle_type(SEMAPHORE_HANDLE) + .push_next(&mut timeline_kind); + let mut semaphore_props = vk::ExternalSemaphoreProperties::default(); + unsafe { + instance.get_physical_device_external_semaphore_properties(physical, &semaphore_query, &mut semaphore_props) + }; + let buffer_query = vk::PhysicalDeviceExternalBufferInfo::default() + .usage(vk::BufferUsageFlags::TRANSFER_SRC | vk::BufferUsageFlags::TRANSFER_DST) + .handle_type(MEMORY_HANDLE); + let mut buffer_props = vk::ExternalBufferProperties::default(); + unsafe { instance.get_physical_device_external_buffer_properties(physical, &buffer_query, &mut buffer_props) }; + let exportable = semaphore_props + .external_semaphore_features + .contains(vk::ExternalSemaphoreFeatureFlags::EXPORTABLE | vk::ExternalSemaphoreFeatureFlags::IMPORTABLE) + && buffer_props + .external_memory_properties + .external_memory_features + .contains(vk::ExternalMemoryFeatureFlags::EXPORTABLE | vk::ExternalMemoryFeatureFlags::IMPORTABLE); + if !exportable { + return None; + } + #[cfg(windows)] + let extensions = + [ash::khr::external_memory_win32::NAME.as_ptr(), ash::khr::external_semaphore_win32::NAME.as_ptr()]; + #[cfg(not(windows))] + let extensions = [ash::khr::external_memory_fd::NAME.as_ptr(), ash::khr::external_semaphore_fd::NAME.as_ptr()]; + let mut timeline = vk::PhysicalDeviceTimelineSemaphoreFeatures::default().timeline_semaphore(true); + let queue_info = [vk::DeviceQueueCreateInfo::default().queue_family_index(family).queue_priorities(&[1.0])]; + let info = vk::DeviceCreateInfo::default() + .queue_create_infos(&queue_info) + .enabled_extension_names(&extensions) + .push_next(&mut timeline); + let device = unsafe { instance.create_device(physical, &info, None) }.ok()?; + let queue = unsafe { device.get_device_queue(family, 0) }; + let pool = unsafe { + device.create_command_pool(&vk::CommandPoolCreateInfo::default().queue_family_index(family), None) + } + .ok()?; + Some(Host { _entry: entry, instance, physical, device, queue, pool }) + } + + fn memory_type(&self, bits: u32, flags: vk::MemoryPropertyFlags) -> u32 { + let types = unsafe { self.instance.get_physical_device_memory_properties(self.physical) }; + (0..types.memory_type_count) + .find(|&i| bits & (1 << i) != 0 && types.memory_types[i as usize].property_flags.contains(flags)) + .expect("memory type") + } + + fn buffer(&self, size: u64, exported: bool) -> HostBuffer { + let mut external = vk::ExternalMemoryBufferCreateInfo::default().handle_types(MEMORY_HANDLE); + let mut info = vk::BufferCreateInfo::default() + .size(size) + .usage(vk::BufferUsageFlags::TRANSFER_SRC | vk::BufferUsageFlags::TRANSFER_DST); + if exported { + info = info.push_next(&mut external); + } + let buffer = unsafe { self.device.create_buffer(&info, None) }.unwrap(); + let req = unsafe { self.device.get_buffer_memory_requirements(buffer) }; + let flags = if exported { + vk::MemoryPropertyFlags::DEVICE_LOCAL + } else { + vk::MemoryPropertyFlags::HOST_VISIBLE | vk::MemoryPropertyFlags::HOST_COHERENT + }; + let mut export = vk::ExportMemoryAllocateInfo::default().handle_types(MEMORY_HANDLE); + let mut dedicated = vk::MemoryDedicatedAllocateInfo::default().buffer(buffer); + let memory_type_index = self.memory_type(req.memory_type_bits, flags); + let mut alloc = + vk::MemoryAllocateInfo::default().allocation_size(req.size).memory_type_index(memory_type_index); + if exported { + alloc = alloc.push_next(&mut export).push_next(&mut dedicated); + } + let memory = unsafe { self.device.allocate_memory(&alloc, None) }.unwrap(); + unsafe { self.device.bind_buffer_memory(buffer, memory, 0) }.unwrap(); + HostBuffer { buffer, memory, allocation_size: req.size, memory_type_index } + } + + fn export_memory(&self, b: &HostBuffer) -> vulkan_interop::ExternalMemory { + vulkan_interop::ExternalMemory { + handle: self.export_handle(b), + allocation_size: b.allocation_size, + memory_type_index: b.memory_type_index, + } + } + + fn export_handle(&self, b: &HostBuffer) -> isize { + #[cfg(windows)] + { + let ext = ash::khr::external_memory_win32::Device::new(&self.instance, &self.device); + let info = vk::MemoryGetWin32HandleInfoKHR::default().memory(b.memory).handle_type(MEMORY_HANDLE); + unsafe { ext.get_memory_win32_handle(&info) }.unwrap() + } + #[cfg(not(windows))] + { + let ext = ash::khr::external_memory_fd::Device::new(&self.instance, &self.device); + let info = vk::MemoryGetFdInfoKHR::default().memory(b.memory).handle_type(MEMORY_HANDLE); + unsafe { ext.get_memory_fd(&info) }.unwrap() as isize + } + } + + fn timeline(&self) -> (vk::Semaphore, isize) { + let mut kind = vk::SemaphoreTypeCreateInfo::default().semaphore_type(vk::SemaphoreType::TIMELINE); + let mut export = vk::ExportSemaphoreCreateInfo::default().handle_types(SEMAPHORE_HANDLE); + let info = vk::SemaphoreCreateInfo::default().push_next(&mut kind).push_next(&mut export); + let semaphore = unsafe { self.device.create_semaphore(&info, None) }.unwrap(); + #[cfg(windows)] + let handle = { + let ext = ash::khr::external_semaphore_win32::Device::new(&self.instance, &self.device); + let info = vk::SemaphoreGetWin32HandleInfoKHR::default().semaphore(semaphore).handle_type(SEMAPHORE_HANDLE); + unsafe { ext.get_semaphore_win32_handle(&info) }.unwrap() + }; + #[cfg(not(windows))] + let handle = { + let ext = ash::khr::external_semaphore_fd::Device::new(&self.instance, &self.device); + let info = vk::SemaphoreGetFdInfoKHR::default().semaphore(semaphore).handle_type(SEMAPHORE_HANDLE); + unsafe { ext.get_semaphore_fd(&info) }.unwrap() as isize + }; + (semaphore, handle) + } + + fn write(&self, b: &HostBuffer, bytes: &[u8]) { + unsafe { + let p = self.device.map_memory(b.memory, 0, bytes.len() as u64, vk::MemoryMapFlags::empty()).unwrap(); + std::ptr::copy_nonoverlapping(bytes.as_ptr(), p as *mut u8, bytes.len()); + self.device.unmap_memory(b.memory); + } + } + + fn read(&self, b: &HostBuffer, len: usize) -> Vec { + let mut v = vec![0u8; len]; + unsafe { + let p = self.device.map_memory(b.memory, 0, len as u64, vk::MemoryMapFlags::empty()).unwrap(); + std::ptr::copy_nonoverlapping(p as *const u8, v.as_mut_ptr(), len); + self.device.unmap_memory(b.memory); + } + v + } + + /// Copies `copies` (from, to) on the GPU after `wait` and signals `signal` on `semaphore`, then returns. + fn submit( + &self, + copies: &[(&HostBuffer, &HostBuffer)], + size: u64, + semaphore: vk::Semaphore, + wait: u64, + signal: u64, + ) { + unsafe { + let cmd = self + .device + .allocate_command_buffers( + &vk::CommandBufferAllocateInfo::default().command_pool(self.pool).command_buffer_count(1), + ) + .unwrap()[0]; + self.device.begin_command_buffer(cmd, &vk::CommandBufferBeginInfo::default()).unwrap(); + for (from, to) in copies { + self.device.cmd_copy_buffer(cmd, from.buffer, to.buffer, &[vk::BufferCopy::default().size(size)]); + } + let barrier = vk::MemoryBarrier::default() + .src_access_mask(vk::AccessFlags::TRANSFER_WRITE) + .dst_access_mask(vk::AccessFlags::HOST_READ | vk::AccessFlags::MEMORY_READ); + self.device.cmd_pipeline_barrier( + cmd, + vk::PipelineStageFlags::TRANSFER, + vk::PipelineStageFlags::HOST | vk::PipelineStageFlags::ALL_COMMANDS, + vk::DependencyFlags::empty(), + &[barrier], + &[], + &[], + ); + self.device.end_command_buffer(cmd).unwrap(); + let waits = if wait > 0 { vec![semaphore] } else { vec![] }; + let wait_values = [wait]; + let stages = [vk::PipelineStageFlags::TRANSFER]; + let signals = [semaphore]; + let signal_values = [signal]; + let mut values = vk::TimelineSemaphoreSubmitInfo::default() + .wait_semaphore_values(&wait_values[..waits.len()]) + .signal_semaphore_values(&signal_values); + let cmds = [cmd]; + let submit = vk::SubmitInfo::default() + .wait_semaphores(&waits) + .wait_dst_stage_mask(&stages[..waits.len()]) + .command_buffers(&cmds) + .signal_semaphores(&signals) + .push_next(&mut values); + self.device.queue_submit(self.queue, &[submit], vk::Fence::null()).unwrap(); + } + } + + fn wait(&self, semaphore: vk::Semaphore, value: u64) { + let semaphores = [semaphore]; + let values = [value]; + let info = vk::SemaphoreWaitInfo::default().semaphores(&semaphores).values(&values); + unsafe { self.device.wait_semaphores(&info, 5_000_000_000) }.expect("the engine signals within 5 s"); + } +} + +impl Drop for Host { + fn drop(&mut self) { + unsafe { + let _ = self.device.device_wait_idle(); + self.device.destroy_command_pool(self.pool, None); + } + } +} + +#[test] +#[ignore] +fn frames_from_another_vulkan_device_come_back_generated() { + let engine = match Engine::new(&EngineOptions { backend: Some(wgpu::Backends::VULKAN), ..EngineOptions::default() }) + { + Ok(e) => e, + Err(e) => { + eprintln!("skipped: no Vulkan engine ({e:#})"); + return; + } + }; + if !vulkan_interop::supported(&engine) { + eprintln!("skipped: {} has no external memory and semaphore extensions", engine.info().name); + return; + } + let uuid = vulkan_interop::device_uuid(&engine).expect("a Vulkan engine has a device UUID"); + let Some(host) = Host::new(uuid) else { + eprintln!("skipped: no host Vulkan device on the engine's GPU"); + return; + }; + let size = buffer_size(W, H); + let (f0, f1) = (frame(0.0), frame(8.0)); + let staging = [host.buffer(size, false), host.buffer(size, false)]; + host.write(&staging[0], &pitched(&f0)); + host.write(&staging[1], &pitched(&f1)); + let shared = [host.buffer(size, true), host.buffer(size, true), host.buffer(size, true)]; + let (semaphore, semaphore_handle) = host.timeline(); + + let imported: Vec = shared + .iter() + .map(|b| vulkan_interop::import_buffer(&engine, &host.export_memory(b), size).expect("import buffer")) + .collect(); + let sem = vulkan_interop::import_semaphore(&engine, semaphore_handle, true).expect("import semaphore"); + assert!(sem.is_timeline()); + + let model = + Model::from_dir(&PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("..").join("..").join("models")).unwrap(); + let mut s = Session::new(&engine, &model, &SessionOptions::new(W, H)).unwrap(); + let (a, b, out) = (s.add_input(), s.add_input(), s.add_output()); + + // the engine's waits go in before the host's signal: a timeline semaphore allows it + let pitch = row_pitch(W); + vulkan_interop::copy_in(&engine, &imported[0], pitch, s.input_texture(a), Some((&sem, 1))).unwrap(); + vulkan_interop::copy_in(&engine, &imported[1], pitch, s.input_texture(b), Some((&sem, 1))).unwrap(); + host.submit(&[(&staging[0], &shared[0]), (&staging[1], &shared[1])], size, semaphore, 0, 1); + s.pair(a, b).unwrap(); + s.mid(0.5, out).unwrap(); + vulkan_interop::copy_out(&engine, s.output_texture(out), &imported[2], pitch, Some((&sem, 2))).unwrap(); + let readback = host.buffer(size, false); + host.submit(&[(&shared[2], &readback)], size, semaphore, 2, 3); + host.wait(semaphore, 3); + let got = unpitched(&host.read(&readback, size as usize)); + + let mut reference = Session::new(&engine, &model, &SessionOptions::new(W, H)).unwrap(); + let (ra, rb, rout) = (reference.add_input(), reference.add_input(), reference.add_output()); + reference.write_rgba(ra, &f0).unwrap(); + reference.write_rgba(rb, &f1).unwrap(); + reference.generate(ra, rb, 0.5, rout).unwrap(); + let want = reference.read_rgba(rout).unwrap().block(engine.gpu()).unwrap(); + assert_eq!(got.len(), want.len()); + let differing = got.iter().zip(&want).filter(|(x, y)| x != y).count(); + eprintln!("{}: in-between through another Vulkan device, {differing} bytes differ", engine.info().name); + assert_eq!(differing, 0, "the interop path gives the same frame as the upload path"); + + assert!(vulkan_interop::copy_in(&engine, &imported[0], 100, s.input_texture(a), None).is_err()); + assert!(vulkan_interop::copy_in(&engine, &imported[0], pitch, s.input_texture(a), Some((&sem, 0))).is_err()); + + drop(sem); + drop(imported); + unsafe { + let _ = host.device.device_wait_idle(); + host.device.destroy_semaphore(semaphore, None); + for b in shared.iter().chain(&staging).chain([&readback]) { + host.device.destroy_buffer(b.buffer, None); + host.device.free_memory(b.memory, None); + } + } +} diff --git a/docs/api-c.md b/docs/api-c.md index f3a948c..b0514f6 100644 --- a/docs/api-c.md +++ b/docs/api-c.md @@ -5,7 +5,7 @@ is `include/flyhigh.h`. C++, C#, Python and other FFI hosts call the same functi layers are opaque handles: `FhEngine` (the device), `FhModel`, `FhSession` (two frames in, one frame at phase `t` out), `FhStream` (frames with timestamps in, intervals out), `FhPacer` (what to present on this tick, what to generate) and `FhGenerator` (the CPU convenience: source frames in, the frames to display out, in order). Frames cross -the ABI as RGBA8 bytes (`width * height * 4`); GPU textures come in through the interop calls (`fh_session_import_d3d11`, `fh_stream_import_d3d11`). Single-threaded use per +the ABI as RGBA8 bytes (`width * height * 4`); GPU frames come in through the interop calls (`fh_session_import_d3d11`, `fh_stream_import_d3d11`; [Vulkan interop](#vulkan-interop) from the host's own Vulkan device). Single-threaded use per handle; a failed call returns `NULL` or `-1` and `fh_last_error()` has the message. Options always start from the `fh_*_options_default()` struct. All options and defaults: [options.md](options.md). @@ -74,6 +74,48 @@ times in host ms), `fh_stream_set_mouse_captured(st, 0 / 1)`, and at present tim with `output` 1) turned by the mouse motion since, into output slot `out`; 1 = present `out`, 0 = present the frame as it is ([integration guide](integration.md#input-latency-the-camera-warp)). +## Vulkan interop + +For a host with its own Vulkan device (an emulator, a game, a capture on Vulkan) on Windows and Linux: frames cross +on the GPU through memory and semaphores the host exports, with no copy through the CPU. The engine stays on its own +device (its own queue, its tensor-core kernels); the two devices must be on the same GPU and driver. + +```c +uint8_t uuid[16]; +if (fh_engine_vulkan_interop(engine, uuid) != 1) { /* the engine's device cannot import: another path */ } +/* compare uuid with the host's VkPhysicalDeviceIDProperties::deviceUUID; pick the engine's GPU with the + adapter option ("10de:2803") when there are several */ + +uint32_t pitch = fh_vk_row_pitch(width); /* bytes per row, a multiple of 256 */ +FhVkMemory m = { handle, allocation_size, memory_type_index }; /* the host's exported allocation */ +FhVkBuffer* in = fh_vk_buffer_import(engine, &m, (uint64_t)pitch * height); +FhVkSemaphore* ready = fh_vk_semaphore_import(engine, ready_handle, 1); /* 1 = timeline */ + +/* host: copy the frame into the buffer (RGBA8 rows of pitch bytes), signal `ready` to v on its queue */ +fh_stream_copy_in(stream, engine, slot, in, pitch, ready, v); /* the engine's copy waits for v */ +fh_stream_push(stream, engine, slot, pts, &push); +fh_stream_generate(stream, engine, 0.5f, out); +fh_stream_copy_out(stream, engine, out, out_buffer, pitch, done, w); /* signals `done` to w after it */ +/* host: its copy out of out_buffer waits for `done` to reach w */ +``` + +- The host allocates each buffer as a dedicated allocation (`VkMemoryDedicatedAllocateInfo`) for a buffer with + `TRANSFER_SRC | TRANSFER_DST` usage, exported as `OPAQUE_WIN32` (an NT handle) on Windows or `OPAQUE_FD` on Linux, + and passes its allocation size and memory type index: Vulkan requires the import to repeat them. Its device needs + `VK_KHR_external_memory_win32` / `_fd` and `VK_KHR_external_semaphore_win32` / `_fd`. +- Buffers rather than images: an image's layout and compression belong to the device that made it; a buffer is the + same bytes on both. The copies cost about 0.05 ms per 1080p frame each. +- A Windows handle stays the host's (close it after the import); a file descriptor belongs to the engine once the + import succeeds. +- Semaphores: a timeline semaphore takes the value to wait for or signal (above 0), and the engine may wait before + the host has submitted the signal. A binary one takes 0, and its signal must be submitted before the engine's wait. + One semaphore per direction keeps each signalled by one queue only, so the values never have to be ordered across + the two queues. +- `fh_vk_signal` signals with nothing copied. `fh_vk_semaphore_destroy` waits for the engine's queue first, so a + pending wait or signal never outlives the semaphore. `fh_session_copy_in` / `fh_session_copy_out` do the same for + a session's slots. +- Rust: `vulkan_interop::{supported, device_uuid, import_buffer, import_semaphore, copy_in, copy_out, signal}`. + ## Pacer (`fh_pacer_*`) The engine's scheduler for a host with its own display loop ([ARCHITECTURE.md](../ARCHITECTURE.md)): pure logic, no clock, diff --git a/docs/api-rust.md b/docs/api-rust.md index b331d83..54872e4 100644 --- a/docs/api-rust.md +++ b/docs/api-rust.md @@ -46,7 +46,9 @@ let rgba = s.read_rgba(out)?.block(engine.gpu())?; // or present s.ou `mid_views` take `(id, &TextureView)` pairs; the view behind an id must never change. `add_input_texture(texture)` makes a texture the host owns an input slot (the frame size, rgba8 or bgra8, sampleable); on Windows `interop::d3d11_shared_texture(&engine, handle, &SharedTexture::rgba8(w, h))` imports a D3D11 shared texture into - the Vulkan device without a copy for exactly that. + the Vulkan device without a copy for exactly that. A host on its own Vulkan device (Windows, Linux) hands frames over + through `vulkan_interop`: exported buffers (`import_buffer`) and timeline semaphores (`import_semaphore`), with + `copy_in` into an input slot after a wait and `copy_out` from an output slot before a signal ([C API](api-c.md#vulkan-interop)). ## Stream diff --git a/docs/integration.md b/docs/integration.md index 16b3b71..96ea23d 100644 --- a/docs/integration.md +++ b/docs/integration.md @@ -6,6 +6,7 @@ |---|---| | CPU bytes (decoded video, screenshots, any language) | the Core through [C](api-c.md) (`fh_session_*`) or [Rust](api-rust.md) (`Session::write_rgba` / `read_rgba`), or the CPU convenience `api::Generator` / `fh_generator_*`: RGBA8 in, RGBA8 out, one upload and one readback per frame | | GPU textures in a Rust / wgpu app | [`Session`](api-rust.md) on your own views, or the [`Stream`](api-rust.md#stream) when frames carry timestamps: no CPU copies; `examples/player/src/main.rs` | +| GPU images on your own Vulkan device (an emulator, a game) | [Vulkan interop](api-c.md#vulkan-interop): frames in and out through buffers you export, ordered by semaphores; the engine keeps its own device on the same GPU | | A `