Pre-flight gates so a provincial rerun cannot silently skip its work (#246 Phases 1-2)

CLAUDE.md

@@ -9,7 +9,7 @@ Experimental package — breaking all the time and loving the learning curve. St
 **Repository:** NewGraphEnvironment/link
 **Primary Language:** R
 **Prefix:** `lnk_`
-**Branch:** `main` (v0.45.1 as of 2026-08-06)
+**Branch:** `main` (v0.46.0 as of 2026-08-28)
 
 ## Status (2026-08-06) — v0.45.1 shipped (WSG drainage closure rebuilt; #227 re-scoped)
 

DESCRIPTION

@@ -1,7 +1,7 @@
 Package: link
 Title: Stream Network Habitat Interpretation (Experimental)
-Version: 0.46.0
-Date: 2026-08-28
+Version: 0.47.0
+Date: 2026-08-30
 Authors@R: c(
     person("Allan", "Irvine", , "[email protected]",
            role = c("aut", "cre"),
@@ -21,6 +21,7 @@ Imports:
     crate (>= 0.0.2),
     DBI,
     digest,
+    fresh (>= 0.33.0),
     httr,
     jsonlite,
     RPostgres,
@@ -37,7 +38,6 @@ Suggests:
     bcdata,
     bookdown,
     dplyr,
-    fresh (>= 0.33.0),
     gq,
     knitr,
     lintr,

NAMESPACE

@@ -42,6 +42,10 @@ export(lnk_pipeline_run)
 export(lnk_pipeline_setup)
 export(lnk_pipeline_species)
 export(lnk_points_snap)
+export(lnk_preflight_fresh)
+export(lnk_preflight_parity)
+export(lnk_preflight_stamp)
+export(lnk_preflight_vintage)
 export(lnk_presence)
 export(lnk_rollup_wsg)
 export(lnk_rules_build)

NEWS.md

@@ -1,3 +1,23 @@
+# link 0.47.0
+
+Closes the three gaps that would have made a provincial rerun silently skip most of its work ([#246](https://github.com/NewGraphEnvironment/link/issues/246)), and absorbs the sibling-host parity hook ([#183](https://github.com/NewGraphEnvironment/link/issues/183)).
+
+**The root cause was narrower than it looked, and it was a declaration.** The cypher image bakes `fresh` 0.31.0, which exports neither `frs_wsg_drainage` nor `frs_wsg_outlets` — `frs_wsg_drainage` arrived in 0.32.0 and `frs_wsg_outlets` only in 0.33.0, so the floor is v0.33.0 rather than "newer than 0.31.0". `DESCRIPTION` already pinned `NewGraphEnvironment/[email protected]`, but the pin was never resolved: `fresh` sat in **Suggests**, and `pak::local_install()` defaults to `dependencies = NA`, which is hard dependencies only. Meanwhile 20 files call `fresh::` with no `requireNamespace()` guard, including a default argument on an exported function — so `fresh` is a hard runtime dependency that was declared optional, and `library(link)` loaded cleanly on a host whose fresh could not run the pipeline. Moving it to **Imports** makes the existing pin load-bearing with no change to the `pak` call at all: `upgrade = FALSE` suppresses gratuitous upgrades, not required ones, and 0.31.0 does not satisfy the floor. The failure it prevents was silent in the worst way — `wsg_run_one.R` catches the missing-symbol error and `quit(status = 1)`s, the bucket loop logs `[WARN]` and continues, and the host exits 0 with none of its watershed groups persisted.
+
+New exported **`lnk_preflight_fresh()`** asserts symbols rather than a version string, because a version is a proxy that fails in both directions: it reads `0.33.0` on a partial install or against a shadowing library path, and reads "wrong" while every needed symbol is present. A drift guard walks link's own parsed **namespace** rather than the `R/` directory — a source scan finds nothing in an installed package and would report a clean check — so a new `fresh::` call site cannot ship undeclared. Two independent methods agree the real call-site count is 12; a naive grep would have listed 21 and included `frs_point_snap_knn`, which fresh 0.33.0 does not export and link never calls.
+
+**`study_area_run.sh` gains pre-flight gates in two blocks, because there are two questions.** A cypher's software is predictable from the dispatcher before the cypher exists — its link from `git reset --hard origin/$BRANCH`, its fresh from `DESCRIPTION` — so `preflight_local()` validates what the workers are going to get, free and before the spin, and `preflight_hosts()` confirms after prep that they got it. Predict before spend; verify before write. The second cannot run earlier and does not need to: it precedes any write to the persist, and a failure trips the existing EXIT trap, bounding the loss at prep time.
+
+The credential probe is the substantive fix. `doctl` and the tofu provider use **different** tokens; both were minted 2026-05-18 and both expired on the same day, and only the first was ever checked. `tofu plan` cannot serve as the second probe — against a workspace with no resources it returns `Plan: N to add` without contacting DigitalOcean at all. The tfvars token now goes to `/v2/account` directly, with 401, an unreachable API and any other status reported as distinct outcomes rather than collapsed into "not success".
+
+New exported **`lnk_preflight_vintage()`** and **`lnk_preflight_parity()`** / **`lnk_preflight_stamp()`** put the load-bearing predicates in R where testthat can prove them. Vintage: absent is not fresh, a NULL timestamp is not fresh, and an empty result set is not a pass. `last_analyze` alone turned out to be unusable — measured NULL on all ten input primitives — so the query takes `GREATEST` of it and `last_autoanalyze`. Parity keys on **`repo_sha`**, not `link_sha`: the latter is a real SHA on the `load_all` dispatcher and `NA` on every pak-installed cypher, so comparing it would fail every legitimate run, while `fresh_sha` is `NA` on both and would pass vacuously. `repo_sha` is read on each host from the checkout it installed from — evidence, rather than a restatement of what the dispatcher believes. For the same reason `cypher_prep.sh` writes each cypher's **own** observed SHA into its `~/.Renviron`, while only `FWAPG_GIT_SHA` crosses the ssh boundary, since cyphers have no fwapg checkout and would otherwise log `NULL`.
+
+**Two post-conditions, and a sentinel that was wrong in both directions.** `schema_consolidate` DELETEs the destination bucket and *then* COPYs, so a host that produced nothing does not merely fail to add rows — it removes the rows already there and returns `ok = TRUE`. Under #246 the run was deleting data and reporting success. Every host must now account for its whole bucket before consolidate, and every run WSG must have rows in the persist afterwards. Separately, the prep gate grepped for `snapshot_bcfp.sh: complete`, which the snapshot emits *before* `lnk_persist_init` runs: a persist_init FATAL exited 1 with the sentinel already in the log, so the gate passed and watershed groups ran against a half-prepped cypher. The snapshot's legitimate skip-if-current path, meanwhile, never emits it, so a skipped load read as FATAL. An anchored `=== READY` is correct in all four measured cases.
+
+New **`data-raw/study_area_buckets.R`** derives the host buckets instead of asserting them: union-find over per-WSG `frs_wsg_drainage()` closures, then greedy LPT over the resulting components, writing `research/study_areas.md`. Components rather than study areas because overlapping closures make consolidate last-writer-wins on the shared groups — whichever host finished last would silently decide the answer — so the script **asserts** disjointness rather than assuming it. It reproduces the issue's numbers from first principles: 96 focal groups, 125 in closure, 119 modelable, 22 drainage-independent components, 39 on the dispatcher, and the same six groups dropped by species presence.
+
+There is deliberately **no global pre-flight bypass**. `--preflight-note="<why>"` downgrades only vintage and parity, and only with a written justification, mirroring `lnk_wsg_downstream_check(override=)`; `--auto-install` is remediation rather than a skip. New `--preflight-only` runs every local gate and exits, so the whole block is testable with one command and no spend. Every gate was exercised against **both** known answers on m1 before shipping — tables in the PR and in `RUNBOOK.md` §8d.
+
 # link 0.46.0
 
 Enforces the DS-first precondition that `data-raw/wsg_run_one.R` has stated in its own header since link#175 and that nothing checked ([#227](https://github.com/NewGraphEnvironment/link/issues/227)). Accessibility is not computed from the focal watershed group alone — link reads the **already-persisted** barriers of the groups downstream — so modelling a WSG out of order writes `streams_access` / `streams_mapping_code` marking segments accessible that are in fact dammed off, prints `done in N min`, and exits 0. A wrong answer indistinguishable from a right one. New exported `lnk_wsg_downstream_check()` verifies it instead: find the blocking dams on the focal WSG's downstream flow path, confirm each is already persisted as a barrier, and fail loud naming them when it is not.

R/lnk_preflight_fresh.R

@@ -0,0 +1,189 @@
+#' Does the installed 'fresh' provide what the pipeline calls?
+#'
+#' `link` calls `fresh::` in a dozen places with no `requireNamespace()`
+#' guard, including a default argument on an exported function
+#' ([lnk_wsg_downstream_check()], whose `outlets` defaults to
+#' `fresh::frs_wsg_outlets()`). A `fresh` that is merely *present* is
+#' therefore not enough — it has to export the symbols.
+#'
+#' On the cypher droplets the installed version is whatever was baked into
+#' the machine image, and the failure mode is silent: `wsg_run_one.R`
+#' catches the missing-symbol error and `quit(status = 1)`s, the bucket
+#' loop logs `[WARN]` and continues, the host exits 0, and the watershed
+#' groups are simply absent from the persist. Nothing downstream can tell
+#' "not modelled" from "modelled empty". That is the 2026-08 failure this
+#' exists to stop (link#246).
+#'
+#' Symbols are checked rather than a version string because a version is a
+#' proxy that fails in both directions: it can read `0.33.0` on a partial
+#' install or against a shadowing library path, and it can read "wrong"
+#' while every needed symbol is present. Loading the namespace also
+#' exercises `fresh`'s own `Imports` resolution, which reading a
+#' `DESCRIPTION` off disk does not.
+#'
+#' @param required Character vector of `fresh` exports the pipeline cannot
+#'   run without. Defaults to the curated list in `.lnk_fresh_required()`.
+#' @param required_internal Character vector of non-exported `fresh`
+#'   objects reached via [utils::getFromNamespace()].
+#' @param min_version Minimum acceptable `fresh` version. Defaults to the
+#'   floor declared in link's own `DESCRIPTION`, so the pin lives in one
+#'   place.
+#' @param quiet Suppress the human-readable report. The report is the
+#'   point on a cypher, where the log is all the operator gets.
+#'
+#' @return Invisibly, a list with `ok`, `version`, `version_ok`,
+#'   `missing`, `missing_internal` and `message`.
+#'
+#' @family preflight
+#'
+#' @export
+#'
+#' @examples
+#' res <- lnk_preflight_fresh(quiet = TRUE)
+#' res$ok
+#' res$version
+#'
+#' # A symbol fresh does not export fails, and is named in the report:
+#' bad <- lnk_preflight_fresh(required = "frs_not_a_real_export", quiet = TRUE)
+#' bad$missing
+lnk_preflight_fresh <- function(required = .lnk_fresh_required(),
+                                required_internal = .lnk_fresh_required_internal(),
+                                min_version = .lnk_fresh_floor(),
+                                quiet = FALSE) {
+  stopifnot(
+    is.character(required), length(required) >= 1L, all(nzchar(required)),
+    is.character(required_internal), all(nzchar(required_internal)),
+    is.character(min_version), length(min_version) == 1L, nzchar(min_version),
+    is.logical(quiet), length(quiet) == 1L, !is.na(quiet))
+
+  version <- .lnk_pkg_version_or_na("fresh")
+  ns <- .lnk_fresh_ns()
+
+  if (is.null(ns)) {
+    out <- list(ok = FALSE, version = NA_character_, version_ok = FALSE,
+                missing = required, missing_internal = required_internal,
+                message = "fresh is not installed or its namespace will not load")
+  } else {
+    missing <- setdiff(required, getNamespaceExports(ns))
+    missing_internal <- required_internal[
+      !vapply(required_internal, exists, logical(1),
+              envir = ns, inherits = FALSE)]
+    version_ok <- !is.na(version) &&
+      utils::compareVersion(version, min_version) >= 0L
+    out <- list(
+      ok = length(missing) == 0L && length(missing_internal) == 0L && version_ok,
+      version = version, version_ok = version_ok,
+      missing = missing, missing_internal = missing_internal,
+      message = .lnk_fresh_message(version, min_version, missing,
+                                   missing_internal, version_ok))
+  }
+
+  if (!quiet) message(out$message)
+  invisible(out)
+}
+
+
+# Namespace lookup behind its own function so the "fresh is absent" branch
+# stays testable. Mocking `base::asNamespace` instead would break every
+# other namespace lookup in the same test file, including testthat's own.
+.lnk_fresh_ns <- function() {
+  tryCatch(asNamespace("fresh"), error = function(e) NULL)
+}
+
+# The `fresh` exports link actually calls, curated rather than derived at
+# runtime. Auto-deriving by grepping for `fresh::` over-fires: most matches
+# in R/ are roxygen cross-references and ordinary comments, not call sites
+# (measured 2026-08-30: 21 distinct symbols mentioned, 12 genuinely called).
+# `.lnk_fresh_callsites()` below is the drift guard that keeps this honest.
+.lnk_fresh_required <- function() {
+  c("frs_break_apply", "frs_break_find", "frs_candidates_pick",
+    "frs_col_generate", "frs_col_join", "frs_habitat_classify",
+    "frs_habitat_overlay", "frs_network_features", "frs_order_child",
+    "frs_params", "frs_wsg_drainage", "frs_wsg_outlets")
+}
+
+# Non-exported fresh objects link reaches via getFromNamespace().
+# R/lnk_pipeline_connect.R:101.
+.lnk_fresh_required_internal <- function() {
+  ".frs_run_connectivity"
+}
+
+# Every `fresh::sym` / `fresh:::sym` reached from link's own namespace,
+# discovered by walking parsed function bodies rather than reading R/.
+#
+# Walking the namespace works for an INSTALLED package, where R/ does not
+# exist. A source-directory scan would silently find nothing there and
+# report a clean drift check — the "guard that matches a container rather
+# than the artifact" failure. This measures the code that will actually run.
+.lnk_fresh_callsites <- function(pkg = "link") {
+  ns <- asNamespace(pkg)
+  found <- character(0)
+
+  # A formal with no default is the empty symbol. It is a perfectly good
+  # object to hold in a list, but passing it as an argument raises
+  # "argument is missing, with no default" — so filter on identity before
+  # recursing, never by trying to evaluate it.
+  drop_empty <- function(xs) {
+    xs[!vapply(xs, identical, logical(1), quote(expr = ))]
+  }
+
+  walk <- function(x) {
+    if (is.call(x)) {
+      fn <- x[[1L]]
+      if (is.call(fn) && length(fn) == 3L && is.name(fn[[1L]]) &&
+          as.character(fn[[1L]]) %in% c("::", ":::") &&
+          identical(as.character(fn[[2L]]), "fresh")) {
+        found <<- c(found, as.character(fn[[3L]]))
+      }
+    }
+    if (is.call(x) || is.pairlist(x) || is.expression(x) || is.list(x)) {
+      for (el in drop_empty(as.list(x))) walk(el)
+    }
+    invisible(NULL)
+  }
+
+  for (nm in ls(ns, all.names = TRUE)) {
+    obj <- tryCatch(get(nm, envir = ns), error = function(e) NULL)
+    if (!is.function(obj)) next
+    walk(body(obj))
+    walk(formals(obj))            # default args count: frs_wsg_outlets()
+  }
+  sort(unique(found))
+}
+
+# Single-source the version floor from link's own DESCRIPTION, so the pin
+# cannot rot in a second place. Returns "0.0.0" when no floor is declared —
+# a permissive default is correct here because the symbol check, not the
+# version, is what this function is actually for.
+.lnk_fresh_floor <- function(desc = utils::packageDescription("link")) {
+  dep <- paste(c(desc$Imports, desc$Depends, desc$Suggests), collapse = ", ")
+  if (!nzchar(dep)) return("0.0.0")
+  m <- regmatches(dep, regexpr("fresh[[:space:]]*\\([[:space:]]*>=[^)]*\\)", dep))
+  if (!length(m) || !nzchar(m)) return("0.0.0")
+  v <- gsub("[^0-9.]", "", sub(".*>=", "", m))
+  if (!nzchar(v)) "0.0.0" else v
+}
+
+.lnk_fresh_message <- function(version, min_version, missing,
+                               missing_internal, version_ok) {
+  head <- sprintf("fresh %s (floor %s)",
+                  if (is.na(version)) "NOT INSTALLED" else version, min_version)
+  if (length(missing) == 0L && length(missing_internal) == 0L && version_ok) {
+    return(paste0("[preflight] ", head, " - OK, all required symbols present"))
+  }
+  parts <- character(0)
+  if (!version_ok) {
+    parts <- c(parts, sprintf("  version below the floor declared in link's DESCRIPTION"))
+  }
+  if (length(missing)) {
+    parts <- c(parts, sprintf("  missing exports: %s",
+                              paste(missing, collapse = ", ")))
+  }
+  if (length(missing_internal)) {
+    parts <- c(parts, sprintf("  missing internals: %s",
+                              paste(missing_internal, collapse = ", ")))
+  }
+  parts <- c(parts,
+    "  fix: pak::pkg_install(\"NewGraphEnvironment/fresh@<ref>\") at or above the floor")
+  paste(c(paste0("[preflight] ", head, " - FAILED"), parts), collapse = "\n")
+}