feat(video): expose Seedance reference media and output controls

SEEDANCE_CAPABILITIES.md

@@ -0,0 +1,76 @@
+# Seedance input and output capabilities
+
+The three gateways use the same public generation fields. Wallet authentication,
+API-key holds, signed poll URLs and settlement timing are unchanged.
+
+## Supported combinations
+
+| Model | First + last frame | Reference images | Reference video/audio combinations |
+| --- | --- | --- | --- |
+| Seedance 1.5-pro | Yes | No | No |
+| Seedance 2.0 / Fast / Mini | Yes | 1–9 | Image + video, image + audio, video + audio, or all three; 1–3 clips of each type |
+| Seedance 2.5 | Yes | 1–30 | Still held pending render/cost verification |
+
+`image_url` means a first-frame seed. For a character/style image alongside a
+reference video, use `reference_image_urls`, not `image_url`. Frame seeding and
+reference mode remain mutually exclusive. Seedance 2.0 audio references require
+at least one reference image or video. Upstream media duration, size and content
+constraints still apply; accepting a URL does not verify the remote file.
+
+```json
+{
+  "model": "bytedance/seedance-2.0-fast",
+  "prompt": "Use image 1 for the character and video 1 for the motion",
+  "duration_seconds": 5,
+  "reference_image_urls": ["https://example.com/character.png"],
+  "reference_videos": [{"url": "https://example.com/motion.mp4"}],
+  "input_type": "reference",
+  "return_last_frame": true
+}
+```
+
+POST to `/v1/videos/generations` or `/api/v1/videos/generations`. Native
+`content[]` also works on these endpoints and `/v1/videos`: `reference_image`,
+`reference_video`, `reference_audio`, `first_frame`, and `last_frame` roles map
+to the corresponding validated flat fields. A role-less single image keeps its
+existing first-frame meaning. Alternatively use `frame_images` with `frame_type`
+or typed `input_references` with role `reference`. Use one media syntax per
+request; conflicting aliases or media fields return 400 before payment.
+
+## Additional output controls
+
+| Field | Models | Values |
+| --- | --- | --- |
+| `bitrate_mode` | Seedance 2.x | `standard`, `high` |
+| `output_format` | Seedance 2.5 | `mp4`, `mov` |
+| `camera_fixed` | Seedance 1.5-pro | Boolean |
+| `safety_identifier` | Seedance family | String |
+| `return_last_frame` | Seedance family | Boolean |
+
+When the upstream returns a last frame, completed `data[0]` includes
+`last_frame_url` and `last_frame_backed_up`. The frame uses the same storage
+backup/fallback semantics as the video. Solana starts the copy without delaying
+settlement, preserving its blockhash timing. An upstream that omits the frame
+produces no invented frame URL. Failover is refused when it would drop a
+requested control or an asset reference.
+
+Python uses the snake_case fields above. TypeScript uses `referenceImageUrls`,
+`referenceVideos`, `referenceAudios`, `bitrateMode`, `outputFormat`, `cameraFixed`,
+`safetyIdentifier`, `returnLastFrame`, and `inputType`. MCP exposes snake_case
+fields and reserves the existing reference-media surcharge before payment.
+
+## Operational limits
+
+`R2V_ENABLED=false` still refuses NEW reference-video/audio jobs with 503.
+This change does not modify deployment configuration or re-enable production.
+Jobs already accepted remain pollable. Image-only references are not subject
+to that operational switch.
+
+Automatic duration (`-1`), 2.5 editing/extension task modes, 2.5 reference media,
+2.5 1080p, draft/flex service tiers, callbacks, and task-list/cancel APIs remain
+outside this change. The first group needs verified cost/output bounds; the
+lifecycle features need a separate ownership and settlement design. Known
+unsupported request controls are rejected instead of silently discarded.
+
+New behavior is covered by local request-contract and mocked payment-lifecycle
+tests. New paid upstream renders and production rollout are separate checks.

blockrun_llm/solana_client.py

@@ -2366,6 +2366,12 @@ def video(
         image_url: str | None = None,
         last_frame_url: str | None = None,
         reference_image_urls: list[str] | None = None,
+        reference_videos: list[dict[str, str]] | None = None,
+        reference_audios: list[dict[str, str]] | None = None,
+        bitrate_mode: str | None = None,
+        output_format: str | None = None,
+        camera_fixed: bool | None = None,
+        safety_identifier: str | None = None,
         real_face_asset_id: str | None = None,
         duration_seconds: int | None = None,
         aspect_ratio: str | None = None,
@@ -2399,6 +2405,12 @@ def video(
             image_url=image_url,
             last_frame_url=last_frame_url,
             reference_image_urls=reference_image_urls,
+            reference_videos=reference_videos,
+            reference_audios=reference_audios,
+            bitrate_mode=bitrate_mode,
+            output_format=output_format,
+            camera_fixed=camera_fixed,
+            safety_identifier=safety_identifier,
             real_face_asset_id=real_face_asset_id,
             duration_seconds=duration_seconds,
             aspect_ratio=aspect_ratio,
@@ -2738,6 +2750,12 @@ def _build_video_body(
         image_url: str | None,
         last_frame_url: str | None,
         reference_image_urls: list[str] | None,
+        reference_videos: list[dict[str, str]] | None = None,
+        reference_audios: list[dict[str, str]] | None = None,
+        bitrate_mode: str | None = None,
+        output_format: str | None = None,
+        camera_fixed: bool | None = None,
+        safety_identifier: str | None = None,
         real_face_asset_id: str | None,
         duration_seconds: int | None,
         aspect_ratio: str | None,
@@ -2773,8 +2791,29 @@ def _build_video_body(
                     "reference_image_urls is mutually exclusive with image_url, "
                     "last_frame_url, and real_face_asset_id."
                 )
-            if len(reference_image_urls) > 9:
-                raise ValueError("reference_image_urls accepts at most 9 images.")
+            image_limit = 30 if (model or "").removeprefix("bytedance/") == "seedance-2.5" else 9
+            if len(reference_image_urls) > image_limit:
+                raise ValueError(f"reference_image_urls accepts at most {image_limit} images.")
+        if (reference_videos or reference_audios) and (
+            image_url or last_frame_url or real_face_asset_id
+        ):
+            raise ValueError(
+                "reference media is mutually exclusive with frame-seed inputs; use reference_image_urls."
+            )
+        for clips in (reference_videos, reference_audios):
+            if clips is not None:
+                if not 1 <= len(clips) <= 3:
+                    raise ValueError("reference media accepts 1 to 3 clips per type.")
+                if any(
+                    not isinstance(clip, dict)
+                    or not isinstance(clip.get("url"), str)
+                    or not clip["url"].startswith(("https://", "http://"))
+                    or clip.get("role", "reference") != "reference"
+                    for clip in clips
+                ):
+                    raise ValueError(
+                        "reference clips require an http(s) URL and optional reference role."
+                    )
         if real_face_asset_id is not None and not real_face_asset_id.startswith("ta_"):
             raise ValueError(
                 "real_face_asset_id must start with 'ta_' "
@@ -2792,6 +2831,18 @@ def _build_video_body(
             body["last_frame_url"] = last_frame_url
         if reference_image_urls:
             body["reference_image_urls"] = reference_image_urls
+        if reference_videos is not None:
+            body["reference_videos"] = reference_videos
+        if reference_audios is not None:
+            body["reference_audios"] = reference_audios
+        if bitrate_mode is not None:
+            body["bitrate_mode"] = bitrate_mode
+        if output_format is not None:
+            body["output_format"] = output_format
+        if camera_fixed is not None:
+            body["camera_fixed"] = camera_fixed
+        if safety_identifier is not None:
+            body["safety_identifier"] = safety_identifier
         if real_face_asset_id:
             body["real_face_asset_id"] = real_face_asset_id
         if duration_seconds is not None:
@@ -4568,6 +4619,12 @@ async def video(
         image_url: str | None = None,
         last_frame_url: str | None = None,
         reference_image_urls: list[str] | None = None,
+        reference_videos: list[dict[str, str]] | None = None,
+        reference_audios: list[dict[str, str]] | None = None,
+        bitrate_mode: str | None = None,
+        output_format: str | None = None,
+        camera_fixed: bool | None = None,
+        safety_identifier: str | None = None,
         real_face_asset_id: str | None = None,
         duration_seconds: int | None = None,
         aspect_ratio: str | None = None,
@@ -4588,6 +4645,12 @@ async def video(
             image_url=image_url,
             last_frame_url=last_frame_url,
             reference_image_urls=reference_image_urls,
+            reference_videos=reference_videos,
+            reference_audios=reference_audios,
+            bitrate_mode=bitrate_mode,
+            output_format=output_format,
+            camera_fixed=camera_fixed,
+            safety_identifier=safety_identifier,
             real_face_asset_id=real_face_asset_id,
             duration_seconds=duration_seconds,
             aspect_ratio=aspect_ratio,

blockrun_llm/types.py

@@ -628,6 +628,8 @@ class VideoClip(BaseModel):
     duration_seconds: Optional[int] = None
     request_id: Optional[str] = None  # Upstream provider's request id (xAI)
     backed_up: Optional[bool] = None
+    last_frame_url: Optional[str] = None
+    last_frame_backed_up: Optional[bool] = None
 
 
 class VideoResponse(BaseModel):

blockrun_llm/video.py

@@ -165,6 +165,12 @@ def generate(
         image_url: str | None = None,
         last_frame_url: str | None = None,
         reference_image_urls: list[str] | None = None,
+        reference_videos: list[dict[str, str]] | None = None,
+        reference_audios: list[dict[str, str]] | None = None,
+        bitrate_mode: str | None = None,
+        output_format: str | None = None,
+        camera_fixed: bool | None = None,
+        safety_identifier: str | None = None,
         real_face_asset_id: str | None = None,
         duration_seconds: int | None = None,
         aspect_ratio: str | None = None,
@@ -194,11 +200,19 @@ def generate(
                 `image_url` -> `last_frame_url`. Requires `image_url` and a
                 Seedance model (bytedance/seedance-1.5-pro, seedance-2.0,
                 or seedance-2.0-fast). Priced identically to image-to-video.
-            reference_image_urls: Omni / multi-reference — up to 9 reference
-                image URLs for character/style consistency (Seedance 2.0
-                only). Cite them as "image 1", "image 2" in the prompt.
+            reference_image_urls: Omni / multi-reference — up to 9 (2.0) or 30 (2.5) reference
+                image URLs for character/style consistency (Seedance 2.0/2.5).
+                Cite them as "image 1", "image 2" in the prompt.
                 Mutually exclusive with `image_url`, `last_frame_url`, and
                 `real_face_asset_id`.
+            reference_videos: Up to 3 http(s) motion references on Seedance 2.0.
+                May be combined with reference_image_urls and reference_audios.
+            reference_audios: Up to 3 http(s) audio references on Seedance 2.0.
+                Requires at least one reference image or video.
+            bitrate_mode: Seedance 2.x output bitrate, "standard" or "high".
+            output_format: Seedance 2.5 output container, "mp4" or "mov".
+            camera_fixed: Seedance 1.5-pro fixed-camera control.
+            safety_identifier: Safety identifier forwarded with a Seedance request.
             real_face_asset_id: A `ta_xxxxxx` face/character asset for
                 identity consistency — either a Virtual Portrait (AI
                 character, via `PortraitClient`, $0.01) or a RealFace
@@ -261,8 +275,29 @@ def generate(
                     "reference_image_urls is mutually exclusive with image_url, "
                     "last_frame_url, and real_face_asset_id."
                 )
-            if len(reference_image_urls) > 9:
-                raise ValueError("reference_image_urls accepts at most 9 images.")
+            image_limit = 30 if (model or "").removeprefix("bytedance/") == "seedance-2.5" else 9
+            if len(reference_image_urls) > image_limit:
+                raise ValueError(f"reference_image_urls accepts at most {image_limit} images.")
+        if (reference_videos or reference_audios) and (
+            image_url or last_frame_url or real_face_asset_id
+        ):
+            raise ValueError(
+                "reference media is mutually exclusive with frame-seed inputs; use reference_image_urls."
+            )
+        for clips in (reference_videos, reference_audios):
+            if clips is not None:
+                if not 1 <= len(clips) <= 3:
+                    raise ValueError("reference media accepts 1 to 3 clips per type.")
+                if any(
+                    not isinstance(clip, dict)
+                    or not isinstance(clip.get("url"), str)
+                    or not clip["url"].startswith(("https://", "http://"))
+                    or clip.get("role", "reference") != "reference"
+                    for clip in clips
+                ):
+                    raise ValueError(
+                        "reference clips require an http(s) URL and optional reference role."
+                    )
         if real_face_asset_id is not None and not real_face_asset_id.startswith("ta_"):
             raise ValueError(
                 "real_face_asset_id must start with 'ta_' "
@@ -282,6 +317,18 @@ def generate(
             body["last_frame_url"] = last_frame_url
         if reference_image_urls:
             body["reference_image_urls"] = reference_image_urls
+        if reference_videos is not None:
+            body["reference_videos"] = reference_videos
+        if reference_audios is not None:
+            body["reference_audios"] = reference_audios
+        if bitrate_mode is not None:
+            body["bitrate_mode"] = bitrate_mode
+        if output_format is not None:
+            body["output_format"] = output_format
+        if camera_fixed is not None:
+            body["camera_fixed"] = camera_fixed
+        if safety_identifier is not None:
+            body["safety_identifier"] = safety_identifier
         if real_face_asset_id:
             body["real_face_asset_id"] = real_face_asset_id
         if duration_seconds is not None:

tests/unit/test_solana_media.py

@@ -724,3 +724,28 @@ async def test_async_image_rejects_unknown_quality_before_paying(self) -> None:
         with pytest.raises(ValueError, match="quality must be one of"):
             await client.image("a cat", quality="hd")
         assert calls == []
+
+
+@pytest.mark.asyncio
+async def test_async_mixed_video_references():
+    import json
+
+    calls: list[httpx.Request] = []
+    client = _make_async_client(_paid_flow(calls, _VIDEO_OK))
+    await client.video(
+        "test",
+        model="bytedance/seedance-2.0",
+        reference_image_urls=["https://example.com/person.png"],
+        reference_videos=[{"url": "https://example.com/motion.mp4"}],
+        reference_audios=[{"url": "https://example.com/music.mp3"}],
+        bitrate_mode="high",
+        safety_identifier="test",
+        return_last_frame=True,
+    )
+    body = json.loads(calls[0].content)
+    assert body["reference_videos"] == [{"url": "https://example.com/motion.mp4"}]
+    assert body["reference_audios"] == [{"url": "https://example.com/music.mp3"}]
+    assert body["reference_image_urls"] == ["https://example.com/person.png"]
+    assert body["bitrate_mode"] == "high"
+    assert body["return_last_frame"] is True
+    await client.close()

tests/unit/test_video_params.py

@@ -155,3 +155,64 @@ def test_input_type_mismatch_is_left_to_the_gateway(client, captured):
     """
     client.generate("x", input_type="image")  # no image_url — gateway's call
     assert captured["body"]["input_type"] == "image"
+
+
+def test_mixed_references_and_controls_reach_body(client, captured):
+    client.generate(
+        "follow the motion",
+        model="bytedance/seedance-2.0",
+        reference_image_urls=["https://example.com/person.png"],
+        reference_videos=[{"url": "https://example.com/motion.mp4"}],
+        reference_audios=[{"url": "https://example.com/music.mp3"}],
+        bitrate_mode="high",
+        safety_identifier="test",
+        return_last_frame=True,
+        input_type="reference",
+    )
+    body = captured["body"]
+    assert body["reference_videos"] == [{"url": "https://example.com/motion.mp4"}]
+    assert body["reference_audios"] == [{"url": "https://example.com/music.mp3"}]
+    assert body["reference_image_urls"] == ["https://example.com/person.png"]
+    assert body["bitrate_mode"] == "high"
+    assert body["safety_identifier"] == "test"
+    assert body["input_type"] == "reference"
+
+
+def test_25_reference_limit_and_output_controls(client, captured):
+    images = ["https://example.com/person.png"] * 30
+    client.generate(
+        "test", model="bytedance/seedance-2.5", reference_image_urls=images, output_format="mov"
+    )
+    assert captured["body"]["reference_image_urls"] == images
+    assert captured["body"]["output_format"] == "mov"
+    with pytest.raises(ValueError, match="at most 30"):
+        client.generate(
+            "test", model="bytedance/seedance-2.5", reference_image_urls=images + images
+        )
+    client.generate("test", model="bytedance/seedance-1.5-pro", camera_fixed=False)
+    assert captured["body"]["camera_fixed"] is False
+
+
+def test_reference_media_cannot_be_frame_seeds(client):
+    with pytest.raises(ValueError, match="mutually exclusive"):
+        client.generate(
+            "test",
+            image_url="https://example.com/frame.png",
+            reference_videos=[{"url": "https://example.com/motion.mp4"}],
+        )
+
+
+def test_last_frame_response_is_not_dropped():
+    result = VideoResponse(
+        created=1,
+        model="bytedance/seedance-2.0",
+        data=[
+            {
+                "url": "https://example.com/movie.mp4",
+                "last_frame_url": "https://example.com/last.png",
+                "last_frame_backed_up": True,
+            }
+        ],
+    )
+    assert result.data[0].last_frame_url == "https://example.com/last.png"
+    assert result.data[0].last_frame_backed_up is True