Acting Performance Transfer

An agent skill for moving a real recorded acting performance into a different character, wardrobe, location, or visual style without asking the video model to reinvent the acting.

The source performance stays in control of movement, expression, dialogue, lip sync, timing, camera, and embedded audio. A restyled first frame controls the new visual world. Optional identity images reinforce likeness without becoming pose or expression references.

This is the technique:

Reference	Authority
@Video1	Acting, facial gestures, gaze, mouth shapes, body motion, prop paths, timing, camera, dialogue, lip sync, and source audio
@Image1	Literal first output frame, composition, wardrobe, props, environment, lighting, lens character, and grade
@Image2	Optional stable full-body or wardrobe identity only
@Image3	Optional facial construction and likeness only

The still references are never allowed to overwrite the source video's changing performance.

Why this exists

Most “restyle this actor” attempts use image-to-video or a loose character reference. The model sees a look and invents a new performance: timing drifts, gestures get simplified, mouth shapes change, and the voice is regenerated.

Acting Performance Transfer approaches the job as a constrained video edit:

1. trim the real performance carefully;
2. extract the actual first frame from that final trim;
3. restyle the frame into the desired finished scene;
4. use reference-video editing with source-length timing;
5. define a strict authority hierarchy in the prompt;
6. preserve the embedded source audio as the master.

The result is a much stronger chance of retaining the original delivery while changing the scene around it.

What this skill does

The skill teaches an AI production agent how to:

• inspect and trim a performance without cutting a word, breath, gesture, or settling pose;
• select a clean opening expression instead of feeding the model dead air or a distracting false start;
• extract and restyle the literal first frame;
• decide whether full-body and face references are actually needed;
• assign narrow authority to each image and video reference;
• write a performance-locked reference-video edit prompt;
• preserve source voice, room tone, contact sounds, and lip sync;
• transform props while retaining the original hand paths and contact timing;
• add environmental spectacle without causing the actor to perform a new reaction;
• preflight provider limits, duration, settings, price, and authorization;
• preserve raw provider output and production evidence;
• diagnose identity drift, voice drift, donor-scene leakage, frozen poses, and start-frame morphs.

What it does not do

This is not:

• a face-swap tool;
• a voice-cloning workflow;
• a way to fabricate a real person's performance without permission;
• ordinary color grading;
• image-to-video character animation;
• a prompt for inventing new dialogue;
• a guarantee that a generative model will reproduce every frame perfectly.

Use only footage, voice, identity, and character references you are authorized to use.

Tested route

The production-tested path is:

• Provider: Pika developer API
• Model operation: bytedance/seedance-2.5/reference-to-video
• Mode: prompt begins edit @Video1:
• Duration: auto
• Ratio: adaptive
• Audio: source embedded audio enabled and explicitly protected
• Start frame: the restyled frame zero from the final trimmed performance

The core skill is provider-aware rather than provider-locked. It can be adapted to another model when that model supports video-reference editing, source-length timing, and audio preservation.

Provider schemas and prices change. The skill requires live documentation and price checks before paid runs. See references/pika-seedance-2.5.md.

Installation

Codex

Clone the repository directly into your Codex skills directory:

git clone https://github.com/thenpceo/acting-performance-transfer.git \
  ~/.codex/skills/acting-performance-transfer

Restart or refresh your agent session after installation. Invoke it explicitly with $acting-performance-transfer, or let Codex select it automatically for matching performance-transfer requests.

Other skill-aware agents

Clone or copy the repository into the agent's skills directory, keeping SKILL.md at the root of the installed folder. The skill is plain Markdown plus one Python helper.

Requirements

Minimum:

• a source performance video the user is authorized to transform;
• a reference-video editing model or service;
• an agent that can read SKILL.md;
• ffmpeg and ffprobe for the optional prep helper.

Check the local media tools:

ffmpeg -version
ffprobe -version

The repository stores no credentials and makes no paid API call by itself.

Quick start

Give the agent the performance clip and the intended transformation:

Use $acting-performance-transfer on ./performance.mov.

Keep my acting, facial gestures, dialogue, lip sync, timing, camera, and original
audio exactly. Move the performance into a candlelit medieval library and change
my wardrobe into dark plate armor. Start by extracting the correct first frame
from the final trimmed take and help me approve a restyled start frame before any
paid video generation.

The agent should guide the production in this order:

1. inspect the performance;
2. confirm the exact in/out points;
3. extract frame zero;
4. create or receive the target start frame;
5. approve that frame;
6. build the reference authority map;
7. show the final prompt and paid preflight;
8. generate only after approval.

Prepare a reference packet

The included helper trims and normalizes a source clip, extracts its literal first frame, and writes a manifest with hashes and reference roles.

python3 scripts/prepare_reference.py \
  --input ./performance.mov \
  --output-dir ./packet \
  --start 1.5 \
  --end 9.0833

Output:

packet/
  performance-reference.mp4
  first-frame.png
  packet.json

The script is local and deterministic. It does not upload anything or call a model.

Provider minimums

Never slow or stretch acting to reach a provider minimum. Prefer more real pre-roll or follow-through. If no additional source exists, the script can hold the final frame and append silence—but only when that tradeoff is understood:

python3 scripts/prepare_reference.py \
  --input ./short-take.mov \
  --output-dir ./packet \
  --start 0 \
  --end 2.712 \
  --minimum-seconds 4 \
  --pad-tail

The generated prompt must identify the padded tail as technical and forbid invented acting during it.

The first-frame gate

The most important visual step is not a generic style reference. It is a restyled version of the exact first frame from the final performance trim.

Preserve:

• body pose and weight;
• head angle and gaze;
• eyelids, brows, and mouth state;
• hands, objects, and contact points;
• camera height, crop, perspective, and subject scale;
• foreground occlusion and major geometry.

Change only the requested visual axes: character, wardrobe, prop design, room, light, texture, or grade.

If the style frame has the right aesthetic but the wrong mouth, hands, camera, or pose, fix the frame before generating video.

Prompt architecture

The core prompt hierarchy looks like this:

edit @Video1: Re-render @Video1 in the finished visual world of @Image1.

@Video1 is the immutable movement, expression, timing, camera, dialogue,
lip-sync, and embedded-audio clock.

@Image1 is the literal first output frame and controls the finished visual
world, composition, wardrobe, props, environment, lighting, and grade.

@Image2 and @Image3 may reinforce stable identity only. They must never
replace @Video1's changing pose, gaze, mouth shapes, gestures, or timing.

Then protect the recorded take:

Preserve every recorded beat one-to-one at natural real-time 1x speed.
Do not restage, simplify, smooth, delay, repeat, exaggerate, paraphrase,
revoice, omit, invent, compress, stretch, or time-shift any human movement,
expression, word, pause, breath, or sound.

And protect the audio:

Use @Video1's original embedded audio verbatim as the sole audio master.
Preserve every original word, vocal identity, pacing, inflection, breath,
pause, room tone, contact sound, and lip synchronization.

The full adaptable template is in references/prompt-contract.md.

Adding environmental action

New action should be spatially and semantically separated from the actor:

The performer remains the crisp, stable focal plane while the environment
carries the only new action.

Specify the event physically—where it starts, when it happens, speed, direction, impact, debris, light spill, shadows, atmosphere, and camera vibration. Then state that the actor does not react unless that reaction exists in the source take.

If acting fidelity matters more than spectacle, generate the locked-performance version first and test spectacle as a separate variant.

Best practices

Trim for performance, not convenience

Keep complete words, breaths, hand actions, anticipation, and follow-through. The cleanest visual cut is not always the correct acting cut.

Extract frame zero after the final trim

Changing the in point invalidates the previous start frame. Re-extract and restyle it.

Use fewer references

Every extra image introduces a competing pose, expression, light, and composition. Add identity sheets only when they solve a real identity problem.

Give every reference a forbidden role

Do not only say what a reference controls. Say what must be ignored: studio background, turnaround grid, pose, gaze, expression, lighting, crop, or donor scene.

Keep the camera locked by default

Invented zooms and camera moves compete with face and motion fidelity. If a new move is required, test it separately and accept the tradeoff.

Make props obey the video

The target image controls what the prop looks like. The source performance controls its grip, trajectory, contact timing, and rhythm.

Preserve embedded audio directly

Do not introduce TTS, voice cloning, a separate voice reference, cleanup, denoising, or loudness normalization unless the user explicitly requests a new audio workflow.

Gate every paid run

Show the current operation, duration, resolution, audio setting, references, request count, and maximum price. A failed request does not authorize a retry.

Preserve raw output

Keep the untouched provider file separate from trims, audio remuxes, captions, grades, and delivery exports.

Common failure modes

Failure	Fix
Generic replacement acting	Use reference-video edit mode and put @Video1 authority first
Voice or lip-sync drift	Use source-length output and the embedded audio as sole master
Opening morph	Rebuild @Image1 from the final trimmed frame zero
Frozen performer	State that @Image1 owns appearance only and must not freeze motion
Identity reference changes expression	Narrow it to stable facial construction and explicitly ignore its expression
Donor room or clothes leak through	Enumerate donor exclusions
New VFX changes the actor	Isolate new action to the environment
Camera move degrades likeness	Lock the camera or test the move separately
Short clip rejected	Use more real footage or an approved held-tail pad; never stretch the acting

See references/troubleshooting.md for the detailed diagnostic table.

Suggested production structure

shot-name/
  inputs/
    performance-reference.mp4
    first-frame.png
    identity-full-body.png
    identity-face.png
  planning/
    authority-map.md
  prompts/
    performance-transfer-v1.txt
  outputs/
    raw/provider/
    derived/
    run-state/

Production checklist

• The user is authorized to use the footage, voice, and likeness references.
• The source range includes complete words, gestures, and the final pose.
• Frame zero was extracted after the final trim.
• The restyled first frame preserves pose, mouth, gaze, hands, props, and camera.
• Every reference has a narrow authority and explicit exclusions.
• The prompt uses reference-video editing, not image-to-video.
• Source duration is preserved at natural 1x speed.
• Embedded audio is the sole voice/lip-sync master.
• New environment action is isolated from the performer.
• Current provider limits and price were checked live.
• The exact paid request was approved.
• Job ID and run state were saved immediately.
• Raw provider output is preserved separately from derivatives.

Security and privacy

This public repository contains no:

• API keys or tokens;
• private footage, faces, voices, or character sheets;
• generated videos or stills;
• personal filesystem paths;
• provider job IDs;
• model weights;
• vendored third-party libraries.

Reference packets and manifests can reveal private filenames, hashes, project details, and local paths. Keep production packets out of public repositories. Store provider credentials in a keychain, provider CLI, or uncommitted environment configuration.

For security reporting, see SECURITY.md.

Limitations

• Generative video is not deterministic.
• Dense finger motion, occluded faces, fast prop interaction, and extreme expressions remain difficult.
• More identity references can reduce rather than improve performance fidelity.
• New camera motion or spectacle can compete with the recorded take.
• Provider duration, reference, audio, and pricing constraints change.
• “Preserve exactly” is a direction to the model, not a mathematical guarantee.
• Final creative acceptance still requires a human watch-through.

Contributing

Issues and pull requests are welcome. Useful contributions include provider-specific mappings, reproducible failure cases, safer preparation checks, and authority-contract improvements. Do not submit private media, credentials, output URLs, or examples featuring people without permission.

License

MIT. See LICENSE.

Acknowledgment

This workflow was developed through repeated short-film production tests, then generalized so the public skill contains no project-specific performer, character, media, prompt, filesystem path, or provider credential.