Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
87e2a73
README: projects using FlyHigh, full list in docs/projects.md
MONZikWasTaken Nov 9, 2026
42134c2
Merge release into dev
MONZikWasTaken Nov 9, 2026
99d2b6c
Static protection keeps line art on pans; smoother pacing at x5, even…
MONZikWasTaken Nov 9, 2026
6e3d57f
Exact modes: a tick shows a source frame only where it lands on it
MONZikWasTaken Nov 9, 2026
a590894
ARCHITECTURE: wrap the planner paragraph
MONZikWasTaken Nov 9, 2026
cd33fd3
Tick lock: frames held to their whole number of ticks; Auto sizes by …
MONZikWasTaken Nov 9, 2026
dc67e11
v0.3.5
MONZikWasTaken Nov 9, 2026
2edbb40
Merge v0.3.5 into dev
MONZikWasTaken Nov 9, 2026
0d7ef18
Auto on the exact grid where no factor is even; the display's rate wi…
MONZikWasTaken Nov 9, 2026
d7c6667
Video mode: in-betweens on their own pair's span; CostSampler retries…
MONZikWasTaken Nov 9, 2026
9cec9df
options.md: Auto, the headroom and fill_display as they are now
MONZikWasTaken Nov 9, 2026
f52abbb
Auto: a penalty that does not lower the stall load is lifted (the sta…
MONZikWasTaken Nov 9, 2026
b77e494
v0.3.6
MONZikWasTaken Nov 9, 2026
542e290
Merge v0.3.6 into dev
MONZikWasTaken Nov 9, 2026
8dad44a
A change after a held frame is shown as drawn (animation on twos and …
MONZikWasTaken Nov 9, 2026
f4ab629
v0.3.7
MONZikWasTaken Nov 10, 2026
14beaf9
Merge v0.3.7 into dev
MONZikWasTaken Nov 10, 2026
6ef9ece
Browser module build remaps source paths
MONZikWasTaken Nov 11, 2026
d8028c1
Vulkan interop: frames from a host's own Vulkan device
MONZikWasTaken Nov 16, 2026
00fb03c
v0.4.0
MONZikWasTaken Nov 16, 2026
f590729
Merge v0.4.0 into dev
MONZikWasTaken Nov 16, 2026
d58c62d
v0.4.1: static protection by default, nearest-source regions as an op…
MONZikWasTaken Nov 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -498,8 +498,8 @@ its 8 x 8 px flow grid:
per-pixel choice between the frames was tried and read as patches.

Both are measured with the local contrast an in-between keeps against the real frame, not PSNR or LPIPS, which reward
the blend that reads as fog. Video mode keeps the plain render. `FH_STILL=0` and `FH_NEAR=0` turn them off for
comparisons.
the blend that reads as fog. Video mode keeps the plain render. The nearest-source regions are an option (`nearest_source`, off by default);
`FH_STILL=0` turns the still test off for comparisons.

## Cadence (video mode)

Expand Down
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,26 @@ follow semantic versioning as described in [CONTRIBUTING.md](CONTRIBUTING.md).

## [Unreleased]

## [0.4.1] - 2026-11-17

### Changed

- Static protection is on by default (frames 8, tolerance 4) in every API: `SessionOptions`, `Options`, the C
defaults and the browser module, where `staticProtection: false` turns it off. A still overlay over a moving world
(HUD text, a crosshair, a bar) was painted over by the world the flow brought from both sides: the opaque overlay of
a synthetic 1080p HUD clip scored 14 to 16 dB and scores 24.5 to 29.5 dB once the protection has its history.
Captured 120 fps video without overlays (UVG, 14 clips) is unchanged (-0.00 dB).
- Nearest-source regions are off by default and became an option: `SessionOptions::nearest_source`, `nearestSource`
in the browser module, `FH_NEAR=1` in the environment for the C API. On the same 14 clips they cost 0.53 dB and
gained no sharpness (0.772 against 0.775 of the ground truth's local contrast); on ground-truth triplets the
realtime defaults now score ATD 31.66, CS2 29.32, natural 38.18 dB (0.3.7: 30.62, 28.83, 37.32).

### Fixed

- `VideoSource` (browser adapters) works in a browser without `importExternalTexture` (Firefox 141 and 142): the
frame goes through a 2D canvas there instead of failing with "importExternalTexture is not a function". That path
costs a CPU copy per frame; `VideoSource.path` tells a host which one is in use (`copy`, `external`, `canvas`).

## [0.4.0] - 2026-11-16

### Added
Expand Down
12 changes: 6 additions & 6 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ members = ["crates/flyhigh-options", "crates/flyhigh-engine", "crates/flyhigh-ca
default-members = ["crates/flyhigh-options", "crates/flyhigh-engine", "crates/flyhigh-capi", "crates/flyhigh-cli", "examples/player"]

[workspace.package]
version = "0.4.0"
version = "0.4.1"
edition = "2021"
license = "Apache-2.0"
repository = "https://github.com/getflyhigh/flyhigh-engine"
Expand Down
6 changes: 4 additions & 2 deletions bindings/python/flyhigh.py
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,9 @@ def _session_options(width: int, height: int, internal=None, k: int = 1, repair:
o.k = k
o.repair = repair
o.render = _RENDER[render]
if static_protection:
if static_protection is False:
o.keep_static = 0
elif static_protection:
o.keep_static = 1
if isinstance(static_protection, dict):
o.static_frames = int(static_protection.get("frames", o.static_frames))
Expand All @@ -343,7 +345,7 @@ def _session_options(width: int, height: int, internal=None, k: int = 1, repair:

class Session:
"""The plans of a model for one frame size: Session(engine, model, width, height, internal="auto"|"frame"|(w, h),
k=1, repair=1, render="model"|"bilinear"|"bicubic", static_protection=None|True|{"frames", "tolerance"},
k=1, repair=1, render="model"|"bilinear"|"bicubic", static_protection=None|True|False|{"frames", "tolerance"},
fast_render=False, linear_t=False, keep_maps=False, protection_mask=False, tune=None)."""

def __init__(self, engine: Engine, model: Model, width: int, height: int, **options):
Expand Down
2 changes: 1 addition & 1 deletion crates/flyhigh-capi/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ pub extern "C" fn fh_options_default(width: u32, height: u32) -> FhOptions {
repair: o.repair,
cadence: 0,
render_sampling: 0,
keep_static: 0,
keep_static: 1,
static_frames: o.static_frames,
static_tolerance: o.static_tolerance,
fast_render: 0,
Expand Down
2 changes: 1 addition & 1 deletion crates/flyhigh-capi/tests/defaults.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ fn options_default_matches_rust() {
(c.width, c.height, c.k, c.internal_w, c.internal_h, c.mode, c.repair, c.cadence, c.render_sampling),
(1920, 1080, 1, 856, 480, 0, DEFAULT_REPAIR, 0, 0)
);
assert_eq!((c.keep_static, c.fast_render, c.linear_t), (0, 0, 0));
assert_eq!((c.keep_static, c.fast_render, c.linear_t), (1, 0, 0));
let s = fh_options_default(640, 480);
assert_eq!((s.internal_w, s.internal_h), (0, 0)); // 0, 0 = the frame size
}
5 changes: 3 additions & 2 deletions crates/flyhigh-engine/src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ pub struct Options {
/// render source sampler: None = the model's (manifest), Some(true) bicubic, Some(false) bilinear
pub cubic: Option<bool>,
/// static protection: pixels equal in both source frames (HUD, burnt-in subtitles, logos) are kept as they are
/// instead of being warped with the motion behind them (default off)
/// instead of being warped with the motion behind them (default on)
pub keep_static: bool,
/// static protection: unchanged source frames before a pixel is protected (default 8)
pub static_frames: u32,
Expand All @@ -121,7 +121,7 @@ impl Options {
repair: DEFAULT_REPAIR,
cadence: false,
cubic: None,
keep_static: false,
keep_static: true,
static_frames: crate::DEFAULT_STATIC_FRAMES,
static_tolerance: crate::DEFAULT_STATIC_TOLERANCE,
fast_render: false,
Expand All @@ -147,6 +147,7 @@ impl Options {
fast_render: self.fast_render,
linear_t: self.linear_t,
protection_mask: false,
near: crate::options::near_default(),
}
}
}
Expand Down
7 changes: 5 additions & 2 deletions crates/flyhigh-engine/src/gen_tex.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ pub struct TexOptions {
pub fast_render: bool,
pub linear_t: bool,
pub protection_mask: bool,
/// nearest-source regions (`NetOpts::near`, realtime mode)
pub near: bool,
}
impl TexOptions {
/// defaults: k 1, internal size `default_internal`, realtime, repair `DEFAULT_REPAIR`, no cadence, model sampler, no static protection
Expand All @@ -77,12 +79,13 @@ impl TexOptions {
repair: crate::DEFAULT_REPAIR,
cadence: false,
cubic: None,
keep_static: false,
keep_static: true,
static_frames: crate::DEFAULT_STATIC_FRAMES,
static_tolerance: crate::DEFAULT_STATIC_TOLERANCE,
fast_render: false,
linear_t: false,
protection_mask: false,
near: crate::options::near_default(),
}
}
}
Expand Down Expand Up @@ -211,7 +214,7 @@ impl GenTex {
// games (realtime); video and anime keep the soft blend until it is measured there
decisive: crate::options::decisive_default() && opts.mode == TexMode::Realtime,
still: crate::options::still_default() && opts.mode == TexMode::Realtime,
near: crate::options::near_default() && opts.mode == TexMode::Realtime,
near: opts.near && opts.mode == TexMode::Realtime,
};
let mut sess = Vec::new();
for _ in 0..n_sess {
Expand Down
14 changes: 9 additions & 5 deletions crates/flyhigh-engine/src/options.rs
Original file line number Diff line number Diff line change
Expand Up @@ -236,11 +236,12 @@ pub struct NetOpts {
pub fn decisive_default() -> bool {
std::env::var("FH_BLEND").is_ok_and(|v| v.trim() == "decisive")
}
/// Nearest-source regions for hosts (Session, Stream realtime); FH_NEAR=0 turns them off for comparisons. On a flick the
/// network cannot follow, the blend of two misplaced copies reads as fog: the nearer frame, warped, reads as a short
/// hold.
/// The default of `SessionOptions::nearest_source`: off, FH_NEAR=1 turns it on. On a flick
/// the network cannot follow, the blend of two misplaced copies reads as fog and the nearer frame, warped, reads as a
/// short hold; measured on captured 120 fps video (FlyHigh-Bench, UVG, 14 clips) the regions cost 0.53 dB and gain no
/// sharpness, so they wait for a test that separates the flicks they help from the motion they hurt.
pub fn near_default() -> bool {
std::env::var("FH_NEAR").map_or(true, |v| v.trim() != "0")
std::env::var("FH_NEAR").is_ok_and(|v| v.trim() == "1")
}
/// The still test for hosts (Session, Stream realtime); FH_STILL=0 turns it off for comparisons. The model's reference
/// render (`NetOpts::new`, the parity fixtures) has none.
Expand Down Expand Up @@ -324,7 +325,10 @@ mod tests {
}
let session = &groups[1].options;
assert_eq!(session.iter().find(|o| o.name == "k").map(|o| o.default.as_str()), Some("1"));
assert_eq!(session.iter().find(|o| o.name == "static_protection").map(|o| o.default.as_str()), Some("off"));
assert_eq!(
session.iter().find(|o| o.name == "static_protection").map(|o| o.default.as_str()),
Some("frames 8, tolerance 4")
);
}

#[test]
Expand Down
12 changes: 10 additions & 2 deletions crates/flyhigh-engine/src/session.rs
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,12 @@ pub struct SessionOptions {
/// Works with or without `static_protection`; a rewritten mask applies from the next pair.
#[option(values = "on, off", c = "protection_mask (0, 1)")]
pub protection_mask: bool,
/// Where the two warped frames disagree over a whole region (a flick the network could not follow), the
/// in-between takes the nearer source frame alone there: a short hold instead of the fog of two misplaced
/// copies. Off by default: on captured 120 fps video it costs 0.53 dB and gains no sharpness; a host whose
/// viewers prefer the hold on fast turns turns it on. The environment variable `FH_NEAR=1` sets the default to on.
#[option(default = "off", values = "on, off", c = "none (FH_NEAR=1 in the environment)")]
pub nearest_source: bool,
/// Kernel choices from an earlier autotune on this device at this size (the text of a `TuneTable`); without one
/// the engine tunes at the first build, or uses the fixed routing when the autotune is off.
#[option(default = "none", values = "an autotune table", c = "tune (text, NULL = none)")]
Expand All @@ -129,11 +135,12 @@ impl SessionOptions {
k: 1,
repair: crate::DEFAULT_REPAIR,
render: RenderSampling::Model,
static_protection: None,
static_protection: Some(StaticProtection::default()),
fast_render: false,
linear_t: false,
keep_maps: false,
protection_mask: false,
nearest_source: crate::options::near_default(),
tune: None,
}
}
Expand Down Expand Up @@ -192,7 +199,7 @@ impl SessionOptions {
protection_mask: self.protection_mask,
decisive: crate::options::decisive_default(),
still: crate::options::still_default(),
near: crate::options::near_default(),
near: self.nearest_source,
}
}

Expand Down Expand Up @@ -220,6 +227,7 @@ impl SessionOptions {
fast_render: self.fast_render,
linear_t: self.linear_t,
protection_mask: self.protection_mask,
near: self.nearest_source,
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion crates/flyhigh-engine/tests/defaults.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ fn generator_defaults() {
let o = Options::new(1920, 1080);
assert_eq!((o.k, o.repair, o.mode, o.cadence, o.cubic), (1, DEFAULT_REPAIR, Mode::Realtime, false, None));
assert_eq!(o.internal, Some((856, 480)));
assert!(!o.keep_static && !o.tex_options().keep_static && !TexOptions::new(64, 64).keep_static);
assert!(o.keep_static && o.tex_options().keep_static && TexOptions::new(64, 64).keep_static);
assert_eq!(Options::new(800, 600).internal, None);
let t = o.tex_options();
assert_eq!((t.w, t.h, t.iw, t.ih, t.k, t.repair), (1920, 1080, 856, 480, 1, 1));
Expand Down
1 change: 1 addition & 0 deletions crates/flyhigh-engine/tests/golden_plans.rs
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ fn dump_video(e: &Engine) -> String {
let mut o = TexOptions::new(1280, 720);
o.mode = TexMode::Video;
o.cadence = true;
o.keep_static = false;
let g = GenTex::new(&e.gpu, &e.blob, &e.ops, o, None).expect("video session");
let d = |s: &SessionExt| format!("# pair\n{}# mid\n{}", plan_dump(&s.plan_pair), plan_dump(&s.plan_mid));
assert_eq!(g.sess.len(), 3);
Expand Down
17 changes: 16 additions & 1 deletion crates/flyhigh-web/js/adapters.js
Original file line number Diff line number Diff line change
Expand Up @@ -71,10 +71,14 @@ function draw(device, pipe, bindGroup, view) {

/** Writes a <video>'s current frame into a stream's input slot: copyExternalImageToTexture where the browser takes a
* video source and the frame has the slot's size, an importExternalTexture render pass into the slot otherwise (Firefox
* rejects video sources; a frame of another size is scaled by the pass, the copy is 1:1 only). */
* rejects video sources; a frame of another size is scaled by the pass, the copy is 1:1 only). A browser with neither
* (Firefox before external textures) gets the frame through a 2D canvas, which costs a CPU copy per frame; `path`
* names the one in use: 'copy', 'external' or 'canvas'. */
export class VideoSource {
constructor(device, stream) {
this.device = device; this.stream = stream; this.copyWorks = !FIREFOX; this.pipe = null; this.sampler = null; this.views = new Map();
this.external = typeof device.importExternalTexture === 'function'; this.canvas = null; this.ctx = null;
this.path = this.copyWorks ? 'copy' : this.external ? 'external' : 'canvas';
}
write(slot, video) {
const tex = this.stream.inputTexture(slot);
Expand All @@ -85,6 +89,17 @@ export class VideoSource {
this.copyWorks = false;
}
}
if (!this.external) {
if (!this.canvas || this.canvas.width !== tex.width || this.canvas.height !== tex.height) {
this.canvas = new OffscreenCanvas(tex.width, tex.height);
this.ctx = this.canvas.getContext('2d', { alpha: false });
}
this.ctx.drawImage(video, 0, 0, tex.width, tex.height);
this.device.queue.copyExternalImageToTexture({ source: this.canvas }, { texture: tex }, [tex.width, tex.height]);
this.path = 'canvas';
return;
}
this.path = 'external';
const d = this.device;
if (!this.pipe) {
this.sampler = d.createSampler({ magFilter: 'linear', minFilter: 'linear' });
Expand Down
12 changes: 9 additions & 3 deletions crates/flyhigh-web/src/core.rs
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,8 @@ impl EngineOptionsJs {
}

/// `{ width, height, internal: 'auto' | 'frame' | [w, h], k, repair, render: 'model' | 'bilinear' | 'bicubic',
/// staticProtection: false | true | { frames, tolerance }, fastRender, linearT, keepMaps, protectionMask, tune }`.
/// staticProtection: false | true | { frames, tolerance }, fastRender, linearT, keepMaps, protectionMask,
/// nearestSource, tune }`.
#[derive(Deserialize)]
#[serde(deny_unknown_fields, rename_all = "camelCase")]
struct SessionOptionsJs {
Expand All @@ -178,6 +179,8 @@ struct SessionOptionsJs {
#[serde(default)]
protection_mask: Option<bool>,
#[serde(default)]
nearest_source: Option<bool>,
#[serde(default)]
tune: Option<String>,
}

Expand Down Expand Up @@ -206,8 +209,10 @@ impl SessionOptionsJs {
Some(r) => return Err(JsValue::from_str(&format!("render '{r}' is not model, bilinear or bicubic"))),
};
o.static_protection = match self.static_protection {
None | Some(serde_json::Value::Null) | Some(serde_json::Value::Bool(false)) => None,
Some(serde_json::Value::Bool(true)) => Some(StaticProtection::default()),
Some(serde_json::Value::Bool(false)) => None,
None | Some(serde_json::Value::Null) | Some(serde_json::Value::Bool(true)) => {
Some(StaticProtection::default())
}
Some(serde_json::Value::Object(m)) => {
let d = StaticProtection::default();
Some(StaticProtection {
Expand All @@ -227,6 +232,7 @@ impl SessionOptionsJs {
o.linear_t = self.linear_t.unwrap_or(false);
o.keep_maps = self.keep_maps.unwrap_or(false);
o.protection_mask = self.protection_mask.unwrap_or(false);
o.nearest_source = self.nearest_source.unwrap_or(false);
o.tune = self.tune.as_deref().filter(|t| !t.is_empty()).and_then(flyhigh_engine::tune_table_from);
o.validate().map_err(err)?;
Ok(o)
Expand Down
4 changes: 2 additions & 2 deletions docs/api-js.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ console.log(engine.info(), model.info()); // { name,

Session options: `{ width, height, internal: 'auto' | 'frame' | [w, h], k, repair, render: 'model' | 'bilinear' |
'bicubic', staticProtection: false | true | { frames, tolerance }, fastRender, linearT, keepMaps, protectionMask,
tune }`. Only `width` and `height` are required; the defaults are those of [options.md](options.md) (`internal` auto =
nearestSource, tune }`. Only `width` and `height` are required; the defaults are those of [options.md](options.md) (`internal` auto =
the 480-line rung above 960 px, `k` 1, `repair` 1).

## `FhSession`: two frames in, one frame out
Expand Down Expand Up @@ -132,7 +132,7 @@ and nothing the engine should know about ([ARCHITECTURE.md](../ARCHITECTURE.md))
| `pacerDefaults()` | the Pacer options this browser needs: Firefox gets `min_present_interval_ms: 7` (its compositor stalls below that) |
| `new ReadbackNudge(device).wrap(promise)` | Firefox resolves `mapAsync` only on GPU polls; empty submits while a readback is pending bring it from 70 to 100 ms down to 5. Wrap `classify`, `stats`, `cost`, `readRgba` |
| `await warmUp(device, sessionOrStream)` | compiles every kernel and waits for the GPU before the first frame |
| `new VideoSource(device, stream).write(slot, video)` | a `<video>` frame into an input slot: `pushVideo` where the browser takes it, an `importExternalTexture` render pass into the slot where it does not (Firefox) |
| `new VideoSource(device, stream).write(slot, video)` | a `<video>` frame into an input slot: `pushVideo` where the browser takes it, an `importExternalTexture` render pass into the slot where it does not (Firefox), a 2D canvas copy where neither exists (Firefox 141 and 142, slow); `path` names the one in use: `copy`, `external`, `canvas` |
| `new CanvasPresenter(device, canvas, stream).present(frame)` | the frame a tick names (`{ source }` or `{ output }`) onto the canvas |
| `new TickSource(pacer, onTick).start()` | `requestAnimationFrame` as the Pacer's ticks; a tick later than 50 ms after the previous one is reported as a host stall |
| `new CostSampler(pacer, nudge).push(stream, slot, pts)` / `.generate(stream, request, out)` | `push` and `generate` with a timed run every thirtieth call and on every probe, the GPU milliseconds fed to `costSample` |
Expand Down
Loading
Loading