From 11713c28f0a16cbdb7f7382842869546c0618b57 Mon Sep 17 00:00:00 2001 From: Leonie Monigatti Date: Mon, 2 Nov 2026 11:09:58 +0100 Subject: [PATCH 1/3] Document configurable video frame rate and frame limit for LFM3-VL-2B and LFM3-Omni-2B --- lfm/models/lfm3-omni-2b.mdx | 8 ++++++++ lfm/models/lfm3-vl-2b.mdx | 9 +++++++++ 2 files changed, 17 insertions(+) diff --git a/lfm/models/lfm3-omni-2b.mdx b/lfm/models/lfm3-omni-2b.mdx index 3f66a074..7141018b 100644 --- a/lfm/models/lfm3-omni-2b.mdx +++ b/lfm/models/lfm3-omni-2b.mdx @@ -164,3 +164,11 @@ LFM3-Omni-2B is Liquid AI's first omni model: a single end-to-end model that und ``` + +## Video Frame Rate and Frame Limit + +By default, LFM3-Omni-2B samples video at 2 fps, with at least 4 and at most 384 frames per video. Both are configurable: each frame adds vision tokens, so a lower frame rate or frame limit uses less context and memory and lowers latency, while a higher one preserves more detail. Set them according to your latency and quality requirements. These settings do not change the soundtrack: the model always hears all of it. + +- **Transformers:** set the frame rate per call with `processor.apply_chat_template(..., processor_kwargs={"fps": 1})`. With the soundtrack, use `processor_kwargs={"use_audio_in_video": True, "fps": 1}`. Set the frame limit when you load the processor: `AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128)`. +- **SGLang:** change `fps`, `min_frames`, and `max_frames` under `video_processor` in `processor_config.json` in a local copy of the checkpoint. +- **llama.cpp:** always uses 2 fps and 4 to 384 frames for this model. `--video-fps` has no effect. diff --git a/lfm/models/lfm3-vl-2b.mdx b/lfm/models/lfm3-vl-2b.mdx index 81949392..210b0339 100644 --- a/lfm/models/lfm3-vl-2b.mdx +++ b/lfm/models/lfm3-vl-2b.mdx @@ -203,3 +203,12 @@ LFM3-VL-2B thinks in a `` block before it answers. Reasoning can be long, ## Video Input Images and videos are processed at their native resolution, and every 2×2 block of patches is merged into one language token. Videos are sampled at 2 fps, consecutive frames are paired, and each frame is prefixed with its timestamp. See [Vision Capabilities](/lfm/key-concepts/vision-capabilities#video-understanding) for a video example. + +### Frame Rate and Frame Limit + +By default, LFM3-VL-2B samples video at 2 fps, with at least 4 and at most 768 frames per video. Both are configurable: each frame adds vision tokens, so a lower frame rate or frame limit uses less context and memory and lowers latency, while a higher one preserves more detail. Set them according to your latency and quality requirements. + +- **Transformers:** set the frame rate per call with `processor.apply_chat_template(..., processor_kwargs={"fps": 1})`. Set the frame limit when you load the processor: `AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128)`. +- **vLLM:** set a server default in `--media-io-kwargs`, for example `'{"video": {"video_backend": "qwen3_vl", "fps": 1, "max_frames": 128}}'`. Override it per request with `extra_body={"media_io_kwargs": {"video": {"fps": 1, "max_frames": 128}}}`. +- **SGLang:** change `fps`, `min_frames`, and `max_frames` under `video_processor` in `processor_config.json` in a local copy of the checkpoint. +- **llama.cpp:** always uses 2 fps and 4 to 768 frames for this model. `--video-fps` has no effect. From eaef98b998615392c8003cf890c2f72e8abd9a68 Mon Sep 17 00:00:00 2001 From: Leonie Monigatti Date: Tue, 10 Nov 2026 09:42:43 +0100 Subject: [PATCH 2/3] Use "audio track" instead of "soundtrack" on the LFM3-Omni-2B page --- lfm/models/lfm3-omni-2b.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/lfm/models/lfm3-omni-2b.mdx b/lfm/models/lfm3-omni-2b.mdx index 7141018b..fd09999c 100644 --- a/lfm/models/lfm3-omni-2b.mdx +++ b/lfm/models/lfm3-omni-2b.mdx @@ -19,7 +19,7 @@ LFM3-Omni-2B is Liquid AI's first omni model: a single end-to-end model that und | Parameters | 2.17B | | Context Length | 32K tokens | | Architecture | LFM3 backbone, LFM3 vision encoder, LFM3 causal audio encoder, and LiquidCodec audio tokenizer | -| Input | Text, images, video (with or without its soundtrack), audio | +| Input | Text, images, video (with or without its audio track), audio | | Output | Text, speech, or interleaved text and speech (24kHz) | | Text Languages | English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish | | Speech Languages | Arabic, Danish, German, Greek, British English, American English, Finnish, French, Hebrew, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish, Swedish, Turkish, Chinese; female and male voices | @@ -34,7 +34,7 @@ LFM3-Omni-2B is Liquid AI's first omni model: a single end-to-end model that und - Video frames and their soundtrack, interleaved in time + Video frames and their audio track, interleaved in time @@ -132,7 +132,7 @@ LFM3-Omni-2B is Liquid AI's first omni model: a single end-to-end model that und ]) ``` - **Video with Soundtrack:** + **Video with Audio Track:** Set `load_audio_from_video=True` and `use_audio_in_video=True` to interleave the video's audio track with its frames. Without the two flags, only the frames are used. @@ -157,7 +157,7 @@ LFM3-Omni-2B is Liquid AI's first omni model: a single end-to-end model that und - SGLang serves LFM3-Omni-2B with text output. Text, image, video (optionally with its soundtrack, via `use_audio_in_video`), and audio inputs are supported. + SGLang serves LFM3-Omni-2B with text output. Text, image, video (optionally with its audio track, via `use_audio_in_video`), and audio inputs are supported. ```bash python -m sglang.launch_server --model-path LiquidAI/LFM3-Omni-2B @@ -167,8 +167,8 @@ LFM3-Omni-2B is Liquid AI's first omni model: a single end-to-end model that und ## Video Frame Rate and Frame Limit -By default, LFM3-Omni-2B samples video at 2 fps, with at least 4 and at most 384 frames per video. Both are configurable: each frame adds vision tokens, so a lower frame rate or frame limit uses less context and memory and lowers latency, while a higher one preserves more detail. Set them according to your latency and quality requirements. These settings do not change the soundtrack: the model always hears all of it. +By default, LFM3-Omni-2B samples video at 2 fps, with at least 4 and at most 384 frames per video. Both are configurable: each frame adds vision tokens, so a lower frame rate or frame limit uses less context and memory and lowers latency, while a higher one preserves more detail. Set them according to your latency and quality requirements. These settings do not change the audio track: the model always hears all of it. -- **Transformers:** set the frame rate per call with `processor.apply_chat_template(..., processor_kwargs={"fps": 1})`. With the soundtrack, use `processor_kwargs={"use_audio_in_video": True, "fps": 1}`. Set the frame limit when you load the processor: `AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128)`. +- **Transformers:** set the frame rate per call with `processor.apply_chat_template(..., processor_kwargs={"fps": 1})`. With the audio track, use `processor_kwargs={"use_audio_in_video": True, "fps": 1}`. Set the frame limit when you load the processor: `AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128)`. - **SGLang:** change `fps`, `min_frames`, and `max_frames` under `video_processor` in `processor_config.json` in a local copy of the checkpoint. - **llama.cpp:** always uses 2 fps and 4 to 384 frames for this model. `--video-fps` has no effect. From 41ebeb509407dd2ed0314fa35d102d4bbe4cafa7 Mon Sep 17 00:00:00 2001 From: Leonie Monigatti Date: Tue, 10 Nov 2026 09:42:43 +0100 Subject: [PATCH 3/3] Explain why llama.cpp frame settings for LFM3 models can't be changed --- lfm/models/lfm3-omni-2b.mdx | 2 +- lfm/models/lfm3-vl-2b.mdx | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lfm/models/lfm3-omni-2b.mdx b/lfm/models/lfm3-omni-2b.mdx index fd09999c..623d0622 100644 --- a/lfm/models/lfm3-omni-2b.mdx +++ b/lfm/models/lfm3-omni-2b.mdx @@ -171,4 +171,4 @@ By default, LFM3-Omni-2B samples video at 2 fps, with at least 4 and at most 384 - **Transformers:** set the frame rate per call with `processor.apply_chat_template(..., processor_kwargs={"fps": 1})`. With the audio track, use `processor_kwargs={"use_audio_in_video": True, "fps": 1}`. Set the frame limit when you load the processor: `AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128)`. - **SGLang:** change `fps`, `min_frames`, and `max_frames` under `video_processor` in `processor_config.json` in a local copy of the checkpoint. -- **llama.cpp:** always uses 2 fps and 4 to 384 frames for this model. `--video-fps` has no effect. +- **llama.cpp:** always uses 2 fps and 4 to 384 frames for this model. These values are fixed in llama.cpp itself, so `--video-fps` has no effect, and editing `processor_config.json` before conversion doesn't help either: the GGUF conversion rejects other values. diff --git a/lfm/models/lfm3-vl-2b.mdx b/lfm/models/lfm3-vl-2b.mdx index 210b0339..415305c8 100644 --- a/lfm/models/lfm3-vl-2b.mdx +++ b/lfm/models/lfm3-vl-2b.mdx @@ -211,4 +211,4 @@ By default, LFM3-VL-2B samples video at 2 fps, with at least 4 and at most 768 f - **Transformers:** set the frame rate per call with `processor.apply_chat_template(..., processor_kwargs={"fps": 1})`. Set the frame limit when you load the processor: `AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128)`. - **vLLM:** set a server default in `--media-io-kwargs`, for example `'{"video": {"video_backend": "qwen3_vl", "fps": 1, "max_frames": 128}}'`. Override it per request with `extra_body={"media_io_kwargs": {"video": {"fps": 1, "max_frames": 128}}}`. - **SGLang:** change `fps`, `min_frames`, and `max_frames` under `video_processor` in `processor_config.json` in a local copy of the checkpoint. -- **llama.cpp:** always uses 2 fps and 4 to 768 frames for this model. `--video-fps` has no effect. +- **llama.cpp:** always uses 2 fps and 4 to 768 frames for this model. These values are fixed in llama.cpp itself, so `--video-fps` has no effect, and editing `processor_config.json` before conversion doesn't help either: the GGUF conversion rejects other values.