Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions COREVIDEO_PRO_PRODUCT_SPEC.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# CoreVideo Pro Product Spec and MVP Plan

> **First-release scope, November 9, 2026:** CoreVideo Tiles replaces legacy Magic
> Scene composition. Set & Forget and legacy Magic Scene are excluded and cannot
> be enabled. Scene selection and Take remain manual. Older automation promises
> below are deferred vision, not release acceptance. See
> [first-release scene control](docs/reference/first-release-scene-control.md).

> **Implementation status (2026-06-21).** This document is the product vision and MVP
> plan. The shipping architecture is a **WinUI 3 (.NET 9) shell embedding a React/TypeScript
> operator UI over a C++20 media core**, **Windows-first** (macOS is a roadmap target, not
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

CoreVideo Pro is a **native desktop production studio for Windows and macOS** for building polished
live shows, recordings, and streams from Zoom participants, cameras, screen capture and
media: Magic Scene auto-layout, lower-thirds, graphics, audio leveling, and
media: CoreVideo Tiles composition, lower-thirds, graphics, audio leveling, and
multi-destination output, in a single operator console.

**Work order lives in one place:** [`docs/BACKLOG.md`](docs/BACKLOG.md). This README is
Expand Down Expand Up @@ -93,7 +93,7 @@ the stream, every ISO writer and the camera publisher ran clean, and Program rec
| | DeckLink / AJA | Not shipped: probe only ([#537](https://github.com/iamfatness/CoreVideoPro/issues/537)) |
| **Media** | Local video and still playback with core-owned cue/live/pause | Shipped, not live-verified; ProRes fails to start in the installed build ([#473](https://github.com/iamfatness/CoreVideoPro/issues/473)); stutter reported ([#701](https://github.com/iamfatness/CoreVideoPro/issues/701)) |
| **Compositor** | D3D11 Program/Preview/Multiview, per-source framing, borders, grade presets and curves, lower-thirds and graphics templates | Proven: Program render held every deadline in the soak (minimum slack 3.6 ms) |
| | Set & Forget and Magic Scene automation driven by the core's slot bindings | Shipped, not live-verified: bound in [#1021](https://github.com/iamfatness/CoreVideoPro/pull/1021), unit-tested to the coordinator |
| | Scene selection and Take | Manual control through Studio, keyboard and Stream Deck. Legacy Magic Scene and Set & Forget are excluded from the first release. |
| | Captions | Not shipped: the controls were removed in #1021 because no caption source exists; `push-caption-cue` remains for external cue feeds |
| **Audio** | Routing matrix, Program and ISO taps, BS.1770 meter, bus dynamics and limiter, WASAPI monitor output | Proven on live shows; recorded Program A/V skew measured near -20 ms, live monitor skew -46 ms after [#655](https://github.com/iamfatness/CoreVideoPro/pull/655) ([#652](https://github.com/iamfatness/CoreVideoPro/issues/652) open on the limit) |
| | Per-participant audio delay | Not shipped ([#627](https://github.com/iamfatness/CoreVideoPro/issues/627)) |
Expand Down Expand Up @@ -180,7 +180,7 @@ The first fully useful milestone:

1. Join Zoom.
2. See clean participant feeds and metadata.
3. Click Magic Scene.
3. Compose the show with CoreVideo Tiles and cue the desired scene on Preview.
4. Get a polished show with lower-thirds, graphics, audio leveling, and
stream / local-recording / virtual-camera controls.

Expand Down
1 change: 1 addition & 0 deletions companion-module-corevideopro/src/feedbacks.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ export interface FlagSource {

/** The boolean ControlState fields worth lighting a button. */
export const FEEDBACK_FLAGS: { id: string; name: string; on: number; off?: number }[] = [
{ id: 'programBlackout', name: 'Program faded to black', on: combineRgb(200, 40, 40) },
{ id: 'recording', name: 'Recording active', on: combineRgb(200, 40, 40) },
{ id: 'streaming', name: 'Streaming active', on: combineRgb(200, 40, 40) },
{ id: 'engineOn', name: 'Capture engine on', on: combineRgb(40, 140, 60) },
Expand Down
7 changes: 7 additions & 0 deletions companion-module-corevideopro/src/presets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ const black = combineRgb(0, 0, 0)
export function buildPresets(): CompanionPresetDefinitions {
const presets: CompanionPresetDefinitions = {}

presets['ftb'] = {
type: 'button', category: 'Transport', name: 'Fade to Black',
style: { text: 'FTB', size: '18', color: white, bgcolor: black },
steps: [{ down: [{ actionId: 'transport.ftb.toggle', options: {} }], up: [] }],
feedbacks: [{ feedbackId: 'programBlackout', options: {}, style: { text: 'BLACK', bgcolor: combineRgb(200, 40, 40) } }],
}

presets['take'] = {
type: 'button',
category: 'Transport',
Expand Down
10 changes: 10 additions & 0 deletions contracts/observation.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,16 @@
"qualification": "include",
"control": "include"
},
"programBlackout": {
"typed": "include",
"qualification": "include",
"control": "include"
},
"programBlackoutOpacity": {
"typed": "include",
"qualification": "include",
"control": "include"
},
"brandKit": {
"typed": "include",
"qualification": "include",
Expand Down
7 changes: 7 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# CoreVideo Pro — ranked backlog

**Owner release-scope decision, November 9:** defer the legacy scene director
tracked under [#668](https://github.com/iamfatness/CoreVideoPro/issues/668).
Exclude Set & Forget and old Magic Scene from the first release, including
background and remote activation. Preserve CoreVideo Tiles and manual Take.
Implementation contract: [first-release scene control](reference/first-release-scene-control.md).
This is a release exclusion, not completion of the deferred director's acceptance.

## Now — Zoom ISO recording size, November 9

| Order | Issue | Scope and acceptance |
Expand Down
86 changes: 86 additions & 0 deletions docs/reference/first-release-scene-control.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# First-release scene control

Owner decision, November 9, 2026: exclude Set & Forget and the legacy Magic Scene
workflow from the first release. CoreVideo Tiles owns composition. Scene selection
and Take remain manual through Studio, keyboard, and Stream Deck/Companion.
This supersedes the automation promises in the earlier product and director specs;
it does not change the Tiles speaker-follow or membership policies.

Windows removes the legacy footer buttons and Automation navigation. A restored
legacy page provides manual-production guidance and the separate free-input filling
setting. The production MagicSceneCoordinator constructor cannot enable scene
automation: commands, direct mode assignments and automatic-Take intent are blocked,
and snapshot/timer evaluation cannot bind Preview, start the timer, change graphics,
or Take. The internal opt-in constructor is solely for deferred-engine unit tests;
there is no user setting, environment variable or license override.

The remote action catalog no longer advertises automation.magic, automation.toggle
or automation.autoTake.set. Saved HTTP/OSC/Companion commands fail validation instead
of silently re-enabling the feature. The adapter also rejects those legacy IDs if
called directly. Existing clients should remove those buttons. Manual transport.take,
scene controls and Tiles controls remain registered. The outputs drill no longer
sends the retired auto-Take command.

macOS removes the same controls and navigation. Its immutable release gate prevents
setAutoDirect from starting a timer; the evaluator and one-shot command are guarded.
Legacy auto-Take preferences are ignored. Native macOS build/runtime verification
requires macOS and is not covered by Windows checks.

Automatic filling of free input slots and the separate native graphics automation
are not the retired scene director and remain available. No installer or saved user
profile is modified by this source change.

## Studio manual controls

The Windows Studio footer groups Graphics and Outputs on the left, Take/FTB on the right, above a separate
health/audio row. Output option flyouts remain available. Edit Tiles on a Tiles
scene cues that scene to Preview and opens its wall inspector; it never Takes.

Ctrl+Enter invokes the existing Take command in Studio. Ctrl+Shift+B invokes FTB.
Both shortcuts ignore text-entry controls. Existing graphics F-key mappings and
remote `transport.take` remain unchanged. Companion discovers `transport.ftb.toggle`
and `transport.ftb.set(enabled)` from the action manifest; its FTB preset lights
from `programBlackout` feedback. Use the idempotent set action when retrying requests.

FTB fades Program video over 300 ms; pressing again restores the picture from the
current fade level. The core appends a neutral black GPU layer above scene and
global graphics. Preview, source ISOs, audio, and running output sessions are
unchanged. Taking another scene while black does not release the blackout. The
RC button has a black fill with no visible text and a red outline when black is selected. No saved
preference automatically engages FTB; a new media-core session starts clear.
Older cores without the blackout observation reject FTB rather than implying success.

Layout checks run in an isolated offscreen window. GPU pixel tests prove the native
compositor fade and restoration; these checks are not a live multi-output soak.

The layout fidelity correction keeps Edit Tiles inside its scene card, separates
Take from the transition selector, arranges Graphics / Outputs / transport as three
measured groups, and gives telemetry its own full-width row. Zoom and Health are
direct top navigation actions. The graphics panel starts closed and the on-air/cued
strip appears with that panel; the Graphics button and function keys remain available.
Scene removal remains in the scene context menu. Background import is available in
the background picker context menu and the scene editor. Layout review builds are
separate from the running application so reviewing them does not restart a meeting.

The footer keeps the lower-third shortcut but omits the generic Preview graphic /
Show on Program buttons. Show, Cue and Hide remain attached to each graphic in
the Graphics workspace; graphics keyboard and remote actions are unchanged.

RC footer outputs use compact Record, broadcast, webcam icons and an NDI badge.
Separate option arrows preserve output settings; tooltips and accessible names
identify each action. Take, transition choice, and the black FTB key are anchored
to the far right. Keyboard and remote commands remain unchanged. Manual blend
is excluded from the RC.

Graphics and output shortcuts form a left-aligned group with a fixed gap; only
the space before Take grows with the window. Output icons use matching 26px
line artwork and neutral idle backgrounds, with record/stream colors reserved
for their active states.

## Compiled Windows layout

These captures come from the isolated compiled layout check, with no live video.

![Studio controls](images/rc-studio-controls.png)

![Footer close-up](images/rc-studio-footer.png)
Binary file added docs/reference/images/rc-studio-controls.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/reference/images/rc-studio-footer.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 9 additions & 7 deletions mac-shell/Sources/CoreVideoProShell/AppModel.swift
Original file line number Diff line number Diff line change
Expand Up @@ -368,8 +368,9 @@ final class AppModel: ObservableObject {
@Published var autoSceneId = ""
@Published var autoConfidence = 0
@Published var autoRationale = ""
@Published var autoDirectEnabled = false
@Published var autoTakeEnabled = true
static let sceneAutomationAvailable = false
@Published private(set) var autoDirectEnabled = false
@Published var autoTakeEnabled = false
@Published var autoConfidenceThreshold = 70.0
@Published var autoHoldSeconds = 4.0
@Published var autoStatus = "Manual mode"
Expand Down Expand Up @@ -491,7 +492,7 @@ final class AppModel: ObservableObject {
monitorVolume = prefs.monitorVolume
isoRecordingEnabled = prefs.isoRecordingEnabled
isoSelectedSourceIds = Set(prefs.isoSelectedSourceIds)
autoTakeEnabled = prefs.autoTakeEnabled
autoTakeEnabled = false // Ignore legacy saved auto-take intent in the first release.
autoConfidenceThreshold = prefs.autoConfidenceThreshold
autoHoldSeconds = prefs.autoHoldSeconds
// Optional key: a prefs file written before this shipped decodes to nil
Expand Down Expand Up @@ -1875,12 +1876,13 @@ final class AppModel: ObservableObject {
private var pendingAutoSince = Date.distantPast

func setAutoDirect(enabled: Bool) {
autoDirectEnabled = enabled
autoDirectEnabled = enabled && Self.sceneAutomationAvailable
autoTakeEnabled = autoTakeEnabled && Self.sceneAutomationAvailable
pendingAutoSceneId = ""
autoTimer?.invalidate()
autoTimer = nil
guard enabled else {
autoStatus = "Manual mode"
guard autoDirectEnabled else {
autoStatus = "Scene automation is unavailable in this release"
evaluateAutomationLowerThird() // retires an automation-owned key
return
}
Expand All @@ -1891,7 +1893,7 @@ final class AppModel: ObservableObject {
}

private func evaluateAutoDirect() {
guard autoDirectEnabled else { return }
guard Self.sceneAutomationAvailable && autoDirectEnabled else { return }
// Overlay policy runs BEFORE the roster/confidence guards — the
// MagicSceneCoordinator.EvaluateAutomationPolicy order. It is a no-op
// unless the resolved program source actually changed.
Expand Down
8 changes: 6 additions & 2 deletions mac-shell/Sources/CoreVideoProShell/AutomationExtras.swift
Original file line number Diff line number Diff line change
Expand Up @@ -124,12 +124,16 @@ extension AppModel {

// ── Magic Scene (MagicSceneCoordinator.RunMagicScene, :112) ───────────────

var canRunMagicScene: Bool { !autoSceneId.isEmpty && recommendedScene != nil }
var canRunMagicScene: Bool { AppModel.sceneAutomationAvailable && !autoSceneId.isEmpty && recommendedScene != nil }

/// Queues the director's recommendation onto PREVIEW — never program (the
/// Windows command sets PreviewSceneId + SchedulePreviewRoutingRefresh; the
/// mac equivalent of that refresh is the coalesced syncScenes()).
func runMagicScene() {
guard Self.sceneAutomationAvailable else {
autoStatus = "Scene automation is unavailable in this release"
return
}
guard meetingState == "in_meeting" || !roster.isEmpty else {
autoStatus = "Magic Scene requires an active Zoom meeting"
return
Expand All @@ -149,7 +153,7 @@ extension AppModel {
/// The Windows defaults, verbatim: auto-take on, prefer screen share on,
/// lower thirds on, confidence 70, hold 4s, panel at 4.
func resetAutomationDefaults() {
autoTakeEnabled = true
autoTakeEnabled = Self.sceneAutomationAvailable
autoConfidenceThreshold = 70
autoHoldSeconds = 4
autoExtras = AutomationExtrasState()
Expand Down
85 changes: 3 additions & 82 deletions mac-shell/Sources/CoreVideoProShell/ShellApp.swift
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,7 @@ struct HeaderBar: View {
TabPill(tab: .studio)
GroupLabel("Setup")
ForEach([StudioTab.zoom, .sources, .scenes, .routing], id: \.self) { TabPill(tab: $0) }
ForEach([StudioTab.overlays, .audio, .media, .automation], id: \.self) {
ForEach([StudioTab.overlays, .audio, .media], id: \.self) {
TabPill(tab: $0)
}
GroupLabel("Diagnose")
Expand Down Expand Up @@ -1110,42 +1110,6 @@ struct TransportBar: View {

var body: some View {
HStack(spacing: 10) {
// Magic Scene = auto-direct with auto-take, the one-tap AI pill.
Button {
model.autoTakeEnabled = true
model.setAutoDirect(enabled: !model.autoDirectEnabled)
} label: {
HStack(spacing: 5) {
Image(systemName: "sparkles").font(.system(size: 10)) // design-lint: allow
VStack(alignment: .leading, spacing: 0) {
Text("Magic Scene").font(.grotesk(11, .semibold))
Text("AI auto-direct").font(.grotesk(8))
}
}
.foregroundStyle(model.autoDirectEnabled ? .black : Studio.accent)
.padding(.horizontal, 10).padding(.vertical, 5)
.background(RoundedRectangle(cornerRadius: 8)
.fill(model.autoDirectEnabled ? Studio.accent
: Studio.accent.opacity(0.12)))
.overlay(RoundedRectangle(cornerRadius: 8)
.stroke(Studio.accent.opacity(0.7), lineWidth: 1))
}
.buttonStyle(.plain)
HStack(spacing: 6) {
VStack(alignment: .leading, spacing: 0) {
Text("Set & Forget").font(.grotesk(10, .medium))
Text("Automation \(model.autoDirectEnabled ? "On" : "Off")")
.font(.grotesk(8)).foregroundStyle(Studio.secondary)
}
Toggle("", isOn: Binding(
get: { model.autoDirectEnabled },
set: { model.setAutoDirect(enabled: $0) }))
.toggleStyle(.switch)
.controlSize(.mini)
.labelsHidden()
}
.padding(.horizontal, 8).padding(.vertical, 4)
.background(RoundedRectangle(cornerRadius: 8).fill(Studio.card))
HStack(spacing: 4) {
Text("LT")
.font(.grotesk(10, .bold))
Expand Down Expand Up @@ -2004,51 +1968,8 @@ struct AutomationPane: View {

var body: some View {
VStack(alignment: .leading, spacing: 8) {
Text("Automation").font(.grotesk(14, .semibold))
Toggle("Set & Forget (auto-direct)", isOn: Binding(
get: { model.autoDirectEnabled },
set: { model.setAutoDirect(enabled: $0) }))
Toggle("Auto-take (off = queue on preview)", isOn: $model.autoTakeEnabled)
.disabled(!model.autoDirectEnabled)
HStack {
Text("Confidence ≥ \(Int(model.autoConfidenceThreshold))")
.font(.grotesk(11))
Slider(value: $model.autoConfidenceThreshold, in: 0...100, step: 5)
}
HStack {
Text(String(format: "Hold %.0fs", model.autoHoldSeconds)).font(.grotesk(11))
Slider(value: $model.autoHoldSeconds, in: 0...30, step: 1)
}
Text(model.autoStatus)
.font(.plexMono(11))
.foregroundStyle(model.autoDirectEnabled ? Studio.accent : Studio.secondary)
Divider()
Text("SCENE INTELLIGENCE")
.font(.plexMono(10, .bold))
.foregroundStyle(Studio.secondary)
if model.autoSceneId.isEmpty {
Text("Director idle — recommendations appear once the meeting has participants.")
.font(.grotesk(11)).foregroundStyle(Studio.secondary)
} else {
HStack {
Text(model.autoSceneId)
.font(.grotesk(13, .semibold))
Text("\(model.autoConfidence)%")
.font(.plexMono(11))
.foregroundStyle(model.autoConfidence >= Int(model.autoConfidenceThreshold)
? Studio.accent : Studio.secondary)
Spacer()
Text(model.autoRuleId)
.font(.plexMono(9))
.foregroundStyle(Studio.secondary)
}
Text(model.autoRationale)
.font(.grotesk(11)).foregroundStyle(Studio.secondary)
}
// AutomationPage.xaml parity: Magic Scene, dynamic lower thirds,
// reset defaults, the scene-intelligence readouts, and the honest
// disabled block. Lives in AutomationExtras.swift.
AutomationExtrasSection()
Text("Manual production").font(.grotesk(14, .semibold))
Text("Scene automation is unavailable in this release. Compose your show and use manual Take.").font(.grotesk(12))
}
.modifier(StudioPanel())
}
Expand Down
Loading
Loading