Skip to content

sensing: capture boundary and backend selection ahead of Argus (draft, Argus untested) - #1427

Closed
gareth-morgan-nv wants to merge 6 commits into
gmorgan/FastLiveKitfrom
gmorgan/FastLiveKit-argus
Closed

gareth-morgan-nv wants to merge 6 commits into
gmorgan/FastLiveKitfrom
gmorgan/FastLiveKit-argus

Conversation

@gareth-morgan-nv

Copy link
Copy Markdown
Contributor

Description

Draft. Stacked on #1426 (base is gmorgan/FastLiveKit, so this diff is only the Argus preparation). There is no Argus SDK, Argus camera or Jetson available to the author, so none of the Argus path exists yet or has been tested. This PR does the part that does not need that hardware.

  • Capture boundary (core/camera_capture.hpp): a small vendor-neutral CameraCapture interface, a move-only FrameLease, and a tagged pixel type (borrowed CUDA RGBA for SIPL, leased NV12 DMA-BUF reserved for Argus). The release and DMA-BUF contracts are written down in the header.
  • SIPL behind the interface (SiplCapture): a thin adapter with no extra copy; the output still copies the borrowed frame out after its pacing check, as before. SensingCamera and LiveKitVideoOutput take the interface's frame type; other pixel types are rejected until an encoder input exists. Behaviour is intended to be unchanged.
  • Backend selection (--camera-backend=auto|sipl|argus): chosen once at startup from each backend's probe result. An explicit request must match or fail with that backend's own reason (no silent fallback); auto takes the only match and otherwise names the override. Only SIPL is built in, so argus fails with "this build does not include that backend".
  • Encoder: a stopped capture thread is now reported by poll() instead of looking like a healthy encoder; removed an unused per-frame stamp.
  • Unit tests for the selection rules and the frame lease (Catch2, no CUDA/SIPL needed).

Not done (needs an Orin or an Argus camera)

  • The Argus capture module and the encoder's NV12 DMA-BUF input.
  • A supported per-frame correlation between encoder input and output on the Jetson. Units are paired by submission order today, so an encoder-internal drop can shift frame ids; this needs target-header verification before it is fixed.
  • Distinguishing "no input buffer yet" from a terminal driver error on the encoder's output dequeue (depends on the driver's errno contract).
  • Optional dynamically loaded backend modules.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Testing

  • Selection and frame-lease tests: 12 cases pass when built standalone with g++ against a Catch2 checkout. The CMake test target itself was not configured.
  • New and edited C++ files pass clang-format-14; the SIPL adapter, selection and main.cpp pass a g++ syntax check. jetson_encoder.cpp could not be syntax-checked here (needs the Multimedia API headers).
  • Nothing has been built or run on a Jetson since these changes.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the linter and formatter with SKIP=check-copyright-year pre-commit run --all-files
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix/feature works (or explained why not)
  • I have signed off all my commits (git commit -s) per the DCO

🤖 Generated with Claude Code

gareth-morgan-nv and others added 5 commits November 5, 2026 09:00
…rough

A vendor-neutral CameraCapture interface, a move-only frame lease and a
tagged pixel representation (CUDA RGBA for SIPL, NV12 DMA-BUF for Argus).
Header only; nothing uses it yet, so behaviour is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Gareth Morgan <[email protected]>
SiplCapture adapts SiplCamera without copying: frames stay the borrowed
CUDA RGBA views, which the output still copies out before it returns, after
the pacing check as before. SensingCamera and LiveKitVideoOutput now take
the interface's frame type; other pixel types are rejected until an encoder
input exists. Behaviour is unchanged. Syntax-checked here; the Jetson build
and run are not.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Gareth Morgan <[email protected]>
…llback

select_camera_backend takes each backend's probe result: an explicit request
must match or fail with its own reason; auto takes the only match and
otherwise names the --camera-backend override. main.cpp parses the flag and
probes SIPL, the only backend built in. Syntax-checked here; Jetson build
not run.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Gareth Morgan <[email protected]>
…d encoder stamp

Release callbacks must not throw and may run on any thread; a DMA-BUF frame is
producer-complete pitch-linear NV12 on return. FrameMeta no longer carries a
stamp the FIFO pairing does not use.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Gareth Morgan <[email protected]>
…rom poll, document the flag

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Gareth Morgan <[email protected]>
@gareth-morgan-nv gareth-morgan-nv self-assigned this Nov 5, 2026
@coderabbitai

coderabbitai Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

… as a dropped frame

Only EAGAIN and EINTR are the ordinary no-buffer result; any other errno is
recorded for poll() to surface. Relies on the Multimedia API plane leaving
errno set by the failed ioctl, unverified here (no Jetson); the first Orin
run shows whether a healthy encoder trips it.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Signed-off-by: Gareth Morgan <[email protected]>
@gareth-morgan-nv

Copy link
Copy Markdown
Contributor Author

Superseded by #1428: the capture-backend rearchitecture now lives on gmorgan/RearchBackend (base gmorgan/FastLiveKit). The Argus work will be gmorgan/Argus on top of that branch, so this PR's head branch is retired.

This branch had an error being deployed

1 failed deployment
dev — 75ba3392 Deployed Nov 5, 2026 by gareth-morgan-nv via publish-wheel #6463
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants