Document video frame rate and frame limit for LFM3-VL-2B and LFM3-Omni-2B

lfm/models/lfm3-omni-2b.mdx

@@ -1,11 +1,11 @@
 ---
 title: "LFM3-Omni-2B"
 description: "2B omni model that understands text, images, video, and audio, and answers in text, speech, or both"
 ---
 
 <a href="/lfm/models/omni-models" className="back-button">← Back to Omni Models</a>
 
 LFM3-Omni-2B is Liquid AI's first omni model: a single end-to-end model that understands text, images, video, and audio, and answers in text, speech, or interleaved text and speech. It is built for on-device use, where one model replaces a separate ASR model, vision-language model, LLM, and TTS model. It is a non-reasoning model.
 
 <div style={{display: 'flex', gap: '0.5rem', margin: '0.5rem 0 1.5rem 0'}}>
   <a href="https://huggingface.co/LiquidAI/LFM3-Omni-2B" style={{padding: '0.35rem 0.7rem', borderRadius: '4px', fontSize: '0.85rem', fontWeight: 600, textDecoration: 'none', backgroundColor: '#fbbf24'}}><span style={{color: '#000'}}>HF</span></a>
@@ -18,8 +18,8 @@
 |----------|-------|
 | Parameters | 2.17B |
 | Context Length | 32K tokens |
 | Architecture | LFM3 backbone, LFM3 vision encoder, LFM3 causal audio encoder, and LiquidCodec audio tokenizer |
-| Input | Text, images, video (with or without its soundtrack), audio |
+| Input | Text, images, video (with or without its audio track), audio |
 | Output | Text, speech, or interleaved text and speech (24kHz) |
 | Text Languages | English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish |
 | Speech Languages | Arabic, Danish, German, Greek, British English, American English, Finnish, French, Hebrew, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish, Swedish, Turkish, Chinese; female and male voices |
@@ -34,7 +34,7 @@
 </Card>
 
 <Card title="Audio-Visual Video" icon="video">
-  Video frames and their soundtrack, interleaved in time
+  Video frames and their audio track, interleaved in time
 </Card>
 
 <Card title="Multimodal Tool Calling" icon="wrench">
@@ -132,7 +132,7 @@
     ])
     ```
 
-    **Video with Soundtrack:**
+    **Video with Audio Track:**
 
     Set `load_audio_from_video=True` and `use_audio_in_video=True` to interleave the video's audio track with its frames. Without the two flags, only the frames are used.
 
@@ -157,10 +157,18 @@
   </Tab>
 
   <Tab title="SGLang">
-    SGLang serves LFM3-Omni-2B with text output. Text, image, video (optionally with its soundtrack, via `use_audio_in_video`), and audio inputs are supported.
+    SGLang serves LFM3-Omni-2B with text output. Text, image, video (optionally with its audio track, via `use_audio_in_video`), and audio inputs are supported.
 
     ```bash
     python -m sglang.launch_server --model-path LiquidAI/LFM3-Omni-2B
     ```
   </Tab>
 </Tabs>
+
+## Video Frame Rate and Frame Limit
+
+By default, LFM3-Omni-2B samples video at 2 fps, with at least 4 and at most 384 frames per video. Both are configurable: each frame adds vision tokens, so a lower frame rate or frame limit uses less context and memory and lowers latency, while a higher one preserves more detail. Set them according to your latency and quality requirements. These settings do not change the audio track: the model always hears all of it.
+
+- **Transformers:** set the frame rate per call with `processor.apply_chat_template(..., processor_kwargs={"fps": 1})`. With the audio track, use `processor_kwargs={"use_audio_in_video": True, "fps": 1}`. Set the frame limit when you load the processor: `AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128)`.
+- **SGLang:** change `fps`, `min_frames`, and `max_frames` under `video_processor` in `processor_config.json` in a local copy of the checkpoint.
+- **llama.cpp:** always uses 2 fps and 4 to 384 frames for this model. These values are fixed in llama.cpp itself, so `--video-fps` has no effect, and editing `processor_config.json` before conversion doesn't help either: the GGUF conversion rejects other values.

lfm/models/lfm3-vl-2b.mdx

@@ -19,7 +19,7 @@
 | Property | Value |
 |----------|-------|
 | Parameters | 2B (1.73B language model + 164M vision encoder + 25M projector) |
 | Architecture | LFM3-VL — LFM3 backbone + Liquid hybrid vision encoder, 14 layers (8 gated depthwise 2D convolution + 6 bidirectional attention) |
 | Context Length | 128K tokens |
 | Languages | English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish |
 | Vocabulary Size | 128K |
@@ -30,7 +30,7 @@
 <CardGroup cols={3}>
 
 <Card title="Video Understanding" icon="video">
   Questions about recorded lectures, phone walkthroughs, and camera clips
 </Card>
 
 <Card title="Thinking" icon="lightbulb">
@@ -136,7 +136,7 @@
     ```
   </Tab>
   <Tab title="SGLang">
     Requires an SGLang build with LFM3-VL support.
 
     **Launch server:**
 
@@ -203,3 +203,12 @@
 ## Video Input
 
 Images and videos are processed at their native resolution, and every 2×2 block of patches is merged into one language token. Videos are sampled at 2 fps, consecutive frames are paired, and each frame is prefixed with its timestamp. See [Vision Capabilities](/lfm/key-concepts/vision-capabilities#video-understanding) for a video example.
+
+### Frame Rate and Frame Limit
+
+By default, LFM3-VL-2B samples video at 2 fps, with at least 4 and at most 768 frames per video. Both are configurable: each frame adds vision tokens, so a lower frame rate or frame limit uses less context and memory and lowers latency, while a higher one preserves more detail. Set them according to your latency and quality requirements.
+
+- **Transformers:** set the frame rate per call with `processor.apply_chat_template(..., processor_kwargs={"fps": 1})`. Set the frame limit when you load the processor: `AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128)`.
+- **vLLM:** set a server default in `--media-io-kwargs`, for example `'{"video": {"video_backend": "qwen3_vl", "fps": 1, "max_frames": 128}}'`. Override it per request with `extra_body={"media_io_kwargs": {"video": {"fps": 1, "max_frames": 128}}}`.
+- **SGLang:** change `fps`, `min_frames`, and `max_frames` under `video_processor` in `processor_config.json` in a local copy of the checkpoint.
+- **llama.cpp:** always uses 2 fps and 4 to 768 frames for this model. These values are fixed in llama.cpp itself, so `--video-fps` has no effect, and editing `processor_config.json` before conversion doesn't help either: the GGUF conversion rejects other values.