Add the atcam extra, recorded install.sh extras and the deploy key

docs/build_notes/FEI_build/software_build.rst

@@ -88,27 +88,43 @@ Prefer running Qt GUIs on an operator workstation.
 2. Python Environment
 =====================
 
-HISPEC Software
----------------
+ImageStreamIO
+-------------
+
+The tracking camera publishes frames to shared memory through
+`ImageStreamIO <https://github.com/milk-org/ImageStreamIO>`_, which no distro
+packages:
+
+.. code-block:: bash
 
-Install the HISPEC software as described in :doc:`../../install`, with
-``--host-role feiserver``:
+   git clone https://github.com/milk-org/ImageStreamIO.git
+   cmake -S ImageStreamIO -B ImageStreamIO/build
+   cmake --build ImageStreamIO/build -j
+   sudo cmake --install ImageStreamIO/build
+   sudo ldconfig
+
+Upstream installs its CMake config straight into ``/usr/local/lib/cmake``,
+where ``find_package`` does not look. Point every build on this host at it,
+including the ones ``install.sh`` runs under ``sudo``:
 
 .. code-block:: bash
 
-   sudo /opt/hispec/app/systemd/install.sh hsfei --host-role feiserver
+   echo 'ImageStreamIO_ROOT=/usr/local/lib/cmake' | sudo tee -a /etc/environment
+
+Log out and back in for it to take effect.
+
+HISPEC Software
+---------------
 
-The tracking camera module is not a hispec dependency, since it needs CMake
-and ImageStreamIO to build. Install it into the venv from the instrument
-checkout, as ``hsfei`` (the venv belongs to ``hispec-ops``):
+Install the HISPEC software as described in :doc:`../../install`, with the
+tracking camera module:
 
 .. code-block:: bash
 
-   SKBUILD_CMAKE_DEFINE="ImageStreamIO_DIR=/usr/local/lib/cmake" \
-       uv pip install --python /opt/hispec/venv/bin/python <hispec-tracking-camera-instrument>
+   sudo /opt/hispec/app/systemd/install.sh hsfei --host-role feiserver --extra atcam
 
-``uv sync`` removes anything the project does not declare, so repeat this after
-every ``install.sh`` run.
+``--extra atcam`` is recorded in ``/etc/hispec/extras``, so later runs keep the
+module without repeating it.
 
 .. warning::
    ``PyQt5`` comes from ``apt``, not the venv.

docs/install.md

@@ -43,7 +43,18 @@ activating anything.
        https://github.com/CaltechOpticalObservatories/hispec.git /opt/hispec/app
    ```
 
-3. Run the installer, naming every account that operates this host:
+3. Install the deploy key. `uv lock` reads every extra on every host, and the
+   tracking camera repo is private, so each host needs its read-only key:
+
+   ```bash
+   sudo install -D -m 600 -o root -g root <deploy-key> /etc/hispec/deploy-key
+   ```
+
+   The key is generated once and registered as a read-only deploy key on
+   `hispec-tracking-camera-instrument`; every host gets a copy of the same
+   private half.
+
+4. Run the installer, naming every account that operates this host:
 
    ```bash
    sudo /opt/hispec/app/systemd/install.sh hsdev alice bob --host-role <role>
@@ -53,9 +64,13 @@ activating anything.
    (`hispecserver`, `feiserver`, ...). Leave it out to use the short hostname.
    It is written once, to `/etc/hispec/host`; edit that file to change it.
 
-4. Log out and back in, so the new groups and `PATH` take effect.
+   A host that runs the tracking camera adds `--extra atcam`, which builds the
+   camera module into the venv. Extras are recorded in `/etc/hispec/extras`,
+   so later runs keep them; `hispec doctor` reports one that has gone missing.
+
+5. Log out and back in, so the new groups and `PATH` take effect.
 
-5. Add the credentials this host's daemons need:
+6. Add the credentials this host's daemons need:
 
    ```bash
    $EDITOR /etc/hispec/secrets.env
@@ -64,13 +79,13 @@ activating anything.
    [Secrets](operations/systemd.md#secrets) lists the variables.
    `hispec doctor` says which are still missing.
 
-6. Check the host:
+7. Check the host:
 
    ```bash
    hispec doctor
    ```
 
-7. Deploy and start daemons, as yourself:
+8. Deploy and start daemons, as yourself:
    {doc}`operations/systemd` from here on.
 
 ## What install.sh does

pyproject.toml

@@ -37,6 +37,14 @@ dependencies = [
     "camera_daemon@git+https://github.com/COO-Utilities/camera-daemon.git"
 ]
 
+[project.optional-dependencies]
+# The tracking camera module builds against ImageStreamIO, so only the host
+# running the camera installs it: install.sh --extra atcam. The repo is private;
+# hosts fetch it with /etc/hispec/deploy-key
+atcam = [
+    "camera-interface-tracking@git+ssh://[email protected]/CaltechOpticalObservatories/hispec-tracking-camera-instrument.git",
+]
+
 [project.scripts]
 hispec = "hispec.cli:main"
 

systemd/install.sh

@@ -5,6 +5,10 @@
 #   sudo ./systemd/install.sh                    # set the host up
 #   sudo ./systemd/install.sh alice bob          # ... and enrol two operators
 #   sudo HISPEC_OPS_USERS="alice bob" ./systemd/install.sh   # same thing
+#   sudo ./systemd/install.sh --extra atcam      # ... with the tracking camera
+#
+# Extras are recorded in /etc/hispec/extras, and every later run syncs them
+# again, so a plain re-run keeps a camera host's camera module.
 #
 # Re-run it to enrol operators, after a dependency change, or to repair a host.
 # It is NOT part of adding a daemon: once a host is set up, operators do that
@@ -22,7 +26,8 @@ set -euo pipefail
 
 REPO_DIR="${HISPEC_REPO_DIR:-/opt/hispec/app}"
 VENV_DIR="${HISPEC_VENV_DIR:-/opt/hispec/venv}"
-OPS_USERS="${HISPEC_OPS_USERS:-} $*"
+EXTRAS_FILE="${HISPEC_ETC_DIR:-/etc/hispec}/extras"
+DEPLOY_KEY="${HISPEC_ETC_DIR:-/etc/hispec}/deploy-key"
 UV_VERSION="${HISPEC_UV_VERSION:-0.12.23}"
 # Its own copy rather than whatever uv is on PATH, since a snap refreshes
 # itself and would unpin the resolver
@@ -49,6 +54,31 @@ if [[ ! -d "$REPO_DIR" ]]; then
     exit 1
 fi
 
+# --extra is ours; everything else goes to `hispec install`
+read -ra INSTALL_ARGS <<<"${HISPEC_OPS_USERS:-}"
+EXTRAS=()
+if [[ -f "$EXTRAS_FILE" ]]; then
+    mapfile -t EXTRAS <"$EXTRAS_FILE"
+fi
+while (($#)); do
+    case $1 in
+        --extra) EXTRAS+=("$2"); shift 2 ;;
+        *) INSTALL_ARGS+=("$1"); shift ;;
+    esac
+done
+if ((${#EXTRAS[@]})); then
+    mapfile -t EXTRAS < <(printf '%s\n' "${EXTRAS[@]}" | sort -u)
+fi
+EXTRA_FLAGS=()
+for extra in "${EXTRAS[@]}"; do
+    EXTRA_FLAGS+=(--extra "$extra")
+done
+
+# uv lock resolves every extra on every host, including the private camera repo
+if [[ -f "$DEPLOY_KEY" ]]; then
+    export GIT_SSH_COMMAND="ssh -i $DEPLOY_KEY -o IdentitiesOnly=yes -o StrictHostKeyChecking=accept-new"
+fi
+
 # Pinned, so re-running the installer cannot resolve with a different uv
 if [[ "$("$UV_BIN" --version 2>/dev/null)" != "uv $UV_VERSION"* ]]; then
     curl -LsSf "https://astral.sh/uv/$UV_VERSION/install.sh" |
@@ -64,8 +94,13 @@ fi
     "$UV_BIN" lock --upgrade
     # --all-packages installs the vendored drivers too; plain sync covers only
     # the root project and leaves their dependencies out
-    "$UV_BIN" sync --all-packages
+    "$UV_BIN" sync --all-packages "${EXTRA_FLAGS[@]}"
 )
 
-# shellcheck disable=SC2086  # OPS_USERS is a deliberately word-split list
-exec "$VENV_DIR/bin/hispec" install $OPS_USERS
+# Only once the sync accepted them, so a mistyped extra is not kept
+if ((${#EXTRAS[@]})); then
+    mkdir -p "$(dirname "$EXTRAS_FILE")"
+    printf '%s\n' "${EXTRAS[@]}" >"$EXTRAS_FILE"
+fi
+
+exec "$VENV_DIR/bin/hispec" install "${INSTALL_ARGS[@]}"