Effects, captions, words that move, zoom to clicks

README.md

@@ -106,8 +106,17 @@ make install-app   # Splice.app in /Applications
 - **Text** (⌘T, or the Text shelf): titles in any installed font, with size,
   bold, italic, alignment, colour, a box behind and a soft shadow. Type in the
   inspector, or double-click a title on the stage.
-- **Colour** shelf: full-frame colours for backgrounds and title cards, and a
-  half-black tint.
+- **Words come in** one at a time: by word, letter or line, fading,
+  rising, popping or typed; set it in the Text section.
+- **Captions** (Edit → Add Captions from Speech, or the clip's Sound
+  section): Splice hears what is said, on your Mac (whisper.cpp,
+  `brew install whisper-cpp`; a 148 MB speech model downloads once), and lays
+  captions over it, each word lighting up as it is said. Each caption is a
+  title you can edit. `.srt` and `.vtt` files load too.
+- **Colour** shelf: full-frame colours for backgrounds and title cards, a
+  half-black tint, and light leaks that wash warm or cool light over a shot.
+- **Letterbox** (the canvas inspector, with nothing selected): bars that
+  frame the whole movie to 2.39, 2:1, 1.85 or 4:3.
 - **Inspector** (⌘2), for the selected clip:
   - **Place:** drag a picture on the stage, or its corners to resize. It snaps
     to the centre lines. There's a nine-point grid, plus size, opacity and
@@ -125,6 +134,12 @@ make install-app   # Splice.app in /Applications
   - **Colour:** a look (warm, cool, vivid, muted, faded, film, teal & orange,
     black & white, noir) with exposure, contrast, saturation, warmth and tint
     over it. *Use on every clip* grades the whole movie the same.
+  - **Effects:** blur, vignette, film grain, a handheld shake and a glitch,
+    each a slider.
+  - **Background:** fill the canvas behind a picture with a blurred, darkened
+    copy of it: a phone video on a wide canvas.
+  - **Hide part:** blur, pixelate or black out a box of the picture (a
+    face, a screen, a number plate).
   - **Speed:** 0.25× to 4× (0.1× to 16× from the assistant), or backwards.
     A faster clip gets shorter and what follows on its track moves up; sound
     keeps its pitch. *Freeze frame* (⌥F) holds the frame at the playhead.
@@ -316,6 +331,9 @@ with `make install-skill`) teaches Claude Code to use it.
 | `splice ctl transition` | dissolves, dips, wipes, pushes and zooms at cuts |
 | `splice ctl speed`, `freeze` | faster, slower, backwards; hold a frame |
 | `splice ctl keyframe` | animate position, size and opacity between moments |
+| `splice ctl captions` | captions from speech (on this Mac) or a subtitle file |
+| `splice ctl zoom` | zoom a screen recording in where it was clicked |
+| `splice ctl leak` | light leaks over a shot |
 | `splice ctl score new / read / write / instruments` | compose music: a score as JSON |
 | `splice ctl scene new / read / write / help` | animations and diagrams, described as JSON |
 | `splice ctl scene sheet / frame / patch / add / remove / check` | see a scene's motion as a contact sheet; edit layers by name; validate first |

crates/splice-app/src/actions.rs

@@ -59,6 +59,7 @@ actions!(
         FreezeFrame,
         ReverseClip,
         AddKeyframe,
+        AddCaptions,
         ShowSymbols,
         ShowMedia,
         ShowTextStyles,
@@ -202,6 +203,7 @@ pub fn menus() -> Vec<Menu> {
                 MenuItem::action("Freeze Frame", FreezeFrame),
                 MenuItem::action("Play Backwards", ReverseClip),
                 MenuItem::action("Add Keyframe", AddKeyframe),
+                MenuItem::action("Add Captions from Speech", AddCaptions),
                 MenuItem::separator(),
                 MenuItem::action("Make Symbol…", MakeSymbol),
                 MenuItem::action("Break Symbol Apart", BreakApart),
@@ -313,6 +315,11 @@ pub fn commands() -> Vec<(&'static str, &'static str, Box<dyn Action>)> {
         ),
         ("Play the selection backwards", "", Box::new(ReverseClip)),
         ("Keyframe place and size at the playhead", "⌥K", Box::new(AddKeyframe)),
+        (
+            "Add captions from what is said (selected clip)",
+            "",
+            Box::new(AddCaptions),
+        ),
         ("Show symbols", "", Box::new(ShowSymbols)),
         ("Story view / edit view", "⌘3", Box::new(ToggleStory)),
         ("Add a story beat", "B", Box::new(AddBeat)),

crates/splice-app/src/cli.rs

@@ -107,6 +107,7 @@ fn new(path: &Path, media: &[String]) -> anyhow::Result<()> {
             source: std::fs::canonicalize(&file).unwrap_or(file.clone()),
             recipe: None,
             content: None,
+            clicks: Vec::new(),
         });
         project.append_clip(id);
     }

crates/splice-app/src/ctl.rs

@@ -33,7 +33,7 @@ Editing (each is one undo step; clips and media are named by id, media also by n
   duplicate --clip C                       copy clips to just after themselves
   volume --clip C --value V                1 is as recorded, 0 silent, up to 4
   track --track N [--mute|--unmute] [--hide|--show]
-  canvas [--aspect 16:9|9:16|1:1|4:5] [--size WxH] [--fps N]
+  canvas [--aspect 16:9|9:16|1:1|4:5] [--size WxH] [--fps N] [--letterbox 2.39|1.85|2|none]
   undo | redo
 
 Titles, colours and looks
@@ -47,6 +47,8 @@ Titles, colours and looks
 
   style:  --font NAME  --size PX  --color C  --box C|none  --align left|center|right
           --bold/--no-bold  --italic/--no-italic  --shadow/--no-shadow
+          --animate fade|rise|pop|type|karaoke|none [--by word|letter|line] [--over S]
+                (words come in one at a time, spread over S seconds, or as captions' words are said)
   look:   --x F --y F (centre offset, fractions of the canvas)  --scale F (1 = natural)  --opacity F
           --enter KIND [--enter-for S]  --exit KIND [--exit-for S]
               KIND: none, fade, slide-up, slide-down, slide-left, slide-right, zoom, pop
@@ -59,6 +61,10 @@ Titles, colours and looks
           --grade none|warm|cool|vivid|muted|faded|film|teal-orange|mono|noir   (a ready-made look)
           --exposure F (stops, −1 to 1) --contrast F --saturation F --warmth F --tint F (each −1 to 1)
                 (colour, for video and pictures; `look --all` gives every video and picture the same)
+          --blur F --vignette F --grain F --shake F --glitch F   (effects, 0 to 1; 0 is off)
+          --backdrop on|off [--backdrop-blur F --backdrop-dim F]   (a blurred copy filling the canvas behind)
+          --redact blur|pixelate|black|none [--redact-x F --redact-y F --redact-width F --redact-height F]
+                (hide a box of the picture: a face, a screen; fractions of the clip's frame, centre and size)
   C is #rrggbb, #rrggbbaa or a name: white, black, paper, ink, vermilion, gold, green, blue, clear.
 
 Transitions, speed and keyframes
@@ -79,6 +85,14 @@ Transitions, speed and keyframes
                                  (from this keyframe to the next): linear, in, out, in-out (default), hold
   keyframe --clip C --at T --drop [--prop x|y|scale|opacity]   take keyframes at T away
   keyframe --clip C --clear [--prop NAME]  take them all away; values stay as they are at the playhead
+  zoom --clip C [--to 2] | --none          a screen recording made in Splice: zoom in towards each burst
+                                 of clicks noted while recording (keyframes on place and size)
+  leak --kind warm|sun|cool [--at T]       a light leak washing over what is there, 4s (a scene: edit as one)
+  captions --clip C [--words N] [--plain] [--wait]
+                                 hear what is said in clip C (on this Mac, whisper.cpp) and lay captions
+                                 over it on a new track: lines of up to N words (6), each word lighting up
+                                 as it is said (--plain: whole lines). Each caption is a title to edit
+  captions --file subtitles.srt  captions from an .srt or .vtt file
 
 Music (a score: instruments and notes, played by General MIDI sounds)
   score new [--file score.json] [--name N] [--at T] [--track N] [--wait]
@@ -447,13 +461,16 @@ fn build(args: &[String]) -> Result<Value, String> {
         let mut next = |name: &str| rest.next().ok_or(format!("{name} needs a value"));
         match arg.as_str() {
             "--font" | "--color" | "--box" | "--align" | "--enter" | "--exit" | "--drift" | "--key" | "--mask"
-            | "--query" | "--crop" | "--grade" | "--ease" | "--prop" => {
+            | "--query" | "--crop" | "--grade" | "--ease" | "--prop" | "--backdrop" | "--redact" | "--letterbox"
+            | "--animate" | "--by" => {
                 request.insert(arg[2..].into(), json!(value(next(arg)?)?));
             }
             "--size" | "--x" | "--y" | "--scale" | "--opacity" | "--similarity" | "--blend" | "--feather"
             | "--mask-x" | "--mask-y" | "--mask-width" | "--mask-height" | "--enter-for" | "--exit-for"
             | "--crop-left" | "--crop-top" | "--crop-right" | "--crop-bottom" | "--exposure" | "--contrast"
-            | "--saturation" | "--warmth" | "--tint" | "--rate" => {
+            | "--saturation" | "--warmth" | "--tint" | "--rate" | "--blur" | "--vignette" | "--grain" | "--shake"
+            | "--glitch" | "--backdrop-blur" | "--backdrop-dim" | "--redact-x" | "--redact-y" | "--redact-width"
+            | "--redact-height" => {
                 let raw = next(arg)?;
                 let v = match raw.parse::<f64>() {
                     Ok(v) => json!(v),
@@ -490,6 +507,10 @@ fn build(args: &[String]) -> Result<Value, String> {
                 let w = next(arg)?;
                 request.insert("window".into(), w.parse::<u64>().map_or(json!(w), |id| json!(id)));
             }
+            "--over" => {
+                let v: f64 = next(arg)?.parse().map_err(|_| "--over expects seconds".to_string())?;
+                request.insert("over".into(), json!(v));
+            }
             "--for" => {
                 let v: f64 = next(arg)?.parse().map_err(|_| "--for expects seconds".to_string())?;
                 request.insert("for".into(), json!(v));
@@ -501,6 +522,10 @@ fn build(args: &[String]) -> Result<Value, String> {
                 let v: f64 = next(arg)?.parse().map_err(|_| "--every expects seconds".to_string())?;
                 request.insert("every".into(), json!(v));
             }
+            "--words" if command == "captions" => {
+                let n: u64 = next(arg)?.parse().map_err(|_| "--words expects a number".to_string())?;
+                request.insert("words".into(), json!(n));
+            }
             "--value" => {
                 let v: f64 = next(arg)?.parse().map_err(|_| "--value expects a number".to_string())?;
                 request.insert("value".into(), json!(v));
@@ -540,7 +565,8 @@ fn build(args: &[String]) -> Result<Value, String> {
                 let v: f64 = next(arg)?.parse().map_err(|_| format!("{arg} expects points"))?;
                 request.insert(arg[2..].into(), json!(v));
             }
-            "--ripple" | "--wait" | "--keep" | "--none" | "--all" | "--reset" | "--clear" | "--seam" | "--drop" => {
+            "--ripple" | "--wait" | "--keep" | "--none" | "--all" | "--reset" | "--clear" | "--seam" | "--drop"
+            | "--plain" => {
                 request.insert(arg[2..].into(), json!(true));
             }
             "--reverse" | "--forwards" => {
@@ -565,7 +591,8 @@ fn build(args: &[String]) -> Result<Value, String> {
     let cmd = match command.as_str() {
         "status" | "read" | "versions" | "add" | "move" | "trim" | "split" | "delete" | "duplicate" | "volume"
         | "track" | "canvas" | "undo" | "redo" | "play" | "pause" | "select" | "save" | "say" | "wait" | "text"
-        | "color" | "style" | "look" | "fonts" | "transition" | "speed" | "freeze" | "keyframe" => command.as_str(),
+        | "color" | "style" | "look" | "fonts" | "transition" | "speed" | "freeze" | "keyframe" | "leak"
+        | "captions" | "zoom" => command.as_str(),
         "restore" => {
             if let Some(id) = words.first().and_then(|w| w.parse::<u64>().ok()) {
                 request.insert("version".into(), json!(id));

crates/splice-app/src/editor.rs

@@ -200,6 +200,8 @@ pub struct Editor {
     /// The sound of clips played at another speed or backwards, by
     /// `retimed_key`; None while it is decoded.
     retimed_sound: Arc<Mutex<HashMap<String, Option<Arc<ffmpeg::Pcm>>>>>,
+    /// Clips being heard for captions.
+    captioning: std::collections::HashSet<ClipId>,
     selection: Vec<ClipId>,
     bin_selection: Option<MediaId>,
     view: View,
@@ -368,6 +370,7 @@ impl Editor {
             player: Player::new(),
             audio: AudioOut::open(),
             retimed_sound: Default::default(),
+            captioning: Default::default(),
             selection: Vec::new(),
             bin_selection: None,
             view: View::default(),
@@ -654,6 +657,8 @@ impl Editor {
                             source: path.clone(),
                             recipe: None,
                             content: None,
+                            // A screen recording's clicks, noted beside it.
+                            clicks: crate::recorder::read_clicks(&path),
                         });
                         let media = project.media(id).cloned().expect("just added");
                         if let Some(landing) = self.take_after_import.remove(&path) {