diff --git a/DESCRIPTION b/DESCRIPTION index 1c59862..46b4edc 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -1,6 +1,6 @@ Package: link Title: Stream Network Habitat Interpretation (Experimental) -Version: 0.47.1 +Version: 0.47.2 Date: 2026-08-31 Authors@R: c( person("Allan", "Irvine", , "airvine@newgraphenvironment.com", diff --git a/NEWS.md b/NEWS.md index ef98d05..560244e 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,3 +1,11 @@ +# link 0.47.2 + +Stops shipping `comms/` and `research/` in the built package ([#235](https://github.com/NewGraphEnvironment/link/pull/235)). `R CMD build` includes every top-level directory not named in `.Rbuildignore`, so `pak::pak("NewGraphEnvironment/link")` was installing 15 files of cross-repo coordination notes and 22 of research working files into the user's library. `R CMD check` reports this only as a NOTE and `.gitignore` does not cover it. Verified against the built tarball rather than the config, because the regex is easy to get subtly wrong: both directories go 15/22 → 0. The gap opens over time rather than at scaffold — `planning`, `dev`, `.claude` and `CLAUDE.md` were already excluded, and these two were added to the repo later. + +Also packs distributed work by **finish time rather than segment count** ([#253](https://github.com/NewGraphEnvironment/link/pull/253)). The hosts are not interchangeable: measured on identical work, the dispatcher runs at 0.0391 min per 1000 persisted segments and a cypher at 0.0872 — **2.23× slower**. Balancing raw segments therefore balances *work* and unbalances *time*, and the makespan ends up set by a host that was given less to do; on the provincial set the dispatcher finished at 95 min while every cypher ran to ~190. Assigning each component to the host that would complete it earliest gives 149/149/150/150, cutting the provincial modelling phase by 41 minutes (21%) and taking the end-to-end estimate from 5.0 h to 4.3 h. `--host-speeds=` overrides the measured default. The relabelling step is gone rather than adapted — it packed into anonymous equal-speed bins and renamed them by load, which is meaningless once host 1 *is* the dispatcher and cannot be swapped for a cypher. Drainage-closure and downstream-first assertions were re-verified against the new assignment by restoring the bug, not inherited from the old one. + +Adds `research/run_record_2026_08_31_cypher_pilots.md` and `data-raw/logs/README.md` — the first record under [soul#129](https://github.com/NewGraphEnvironment/soul/issues/129), covering the four cypher pilots that produced the rates above and the four defects they exposed, including the ones where a correct diagnosis was retracted on a bad inference and where two gates passed tests that could not fail. + # link 0.47.1 Fixes two defects in the v0.47.0 pre-flight gates, both found by piloting them against a real cypher for the first time ([#246](https://github.com/NewGraphEnvironment/link/issues/246)). Both would have blocked the provincial run, and neither could have been caught by the tests as written. diff --git a/data-raw/logs/README.md b/data-raw/logs/README.md index 4521a35..041482c 100644 --- a/data-raw/logs/README.md +++ b/data-raw/logs/README.md @@ -1,55 +1,45 @@ -# `data-raw/logs/` - -Run artifacts from pipeline drivers (`compare_bcfishpass_wsg.R`, `run_provincial_parity.R`, the trifecta scripts) plus operational outputs (pg_dumps, methodology-delta queries, regression logs). - -## Subdirectories - -Per-run output is keyed by topic. Each subdir holds `.rds` per-WSG rollup tibbles plus `_per_wsg_times.csv` host-tagged timing rows. - -| Subdir | Source script | Contents | -|--------|---------------|----------| -| `provincial_parity/` | `run_provincial_parity.R --config=bcfishpass` | bcfishpass-bundle rollups (link vs bcfp tunnel) | -| `provincial_default/` | `run_provincial_parity.R --config=default` | default-bundle rollups | -| `provincial_default_extrabreaks/` | `run_provincial_parity.R --config=default_extrabreaks` | orphan-class break-source experiment (v0.28.0) | -| `methodology_delta/` | `query_schema_delta.R` | schema-vs-schema delta RDS snapshots | -| `dumps_/` | `consolidate_schema.R` (manual) | pg_dump custom-format files for cross-host consolidation | -| `baseline_pre_*/` | hand-archived | Pre-change baselines kept for regression diffs | - -## Top-level files - -### `bcfp_baselines.csv` — bcfp build inventory per run - -Records which `bcfishpass.*` schema rebuild each provincial run was compared against. Critical for paper trail because: - -- The tunnel's `bcfishpass.*` schema rebuilds **weekly Tuesdays ~20:00 PDT** via `smnorris/db_newgraph`'s scheduled GHA workflow. -- Today's rollups in `provincial_*/` carry `bcfishpass_value` columns sourced from whichever build was live at the moment of comparison. -- Without recording the build, tomorrow's same-config rerun produces shifts that look like methodology change but are actually upstream-rebuild change (`bcfishpass.streams_habitat_*` repopulated from new code / new input data). - -Columns: - -- `run_started_pdt` — local time the provincial dispatch fired -- `run_label` — directory name where rollup RDS files landed -- `link_schema` — persistent target schema for `lnk_pipeline_persist` -- `bcfp_model_run_id` — primary key from `bcfishpass.log` -- `bcfp_model_version` — `--g` string -- `bcfp_date_completed` — when Simon's rebuild finished -- `notes` — anything else (orphan-branch experiments, partial reruns, etc.) - -How to query the current bcfp baseline (run before any provincial dispatch): - -```sql --- localhost:63333 / dbname=bcfishpass / user=newgraph / password=PG_PASS_SHARE -SELECT model_run_id, date_completed, model_version -FROM bcfishpass.log -ORDER BY model_run_id DESC LIMIT 1; +# Run logs + +**These are retained deliberately as contemporaneous evidence of measurement +runs, not accumulated by accident.** They are tracked in git, not gitignored, +and should stay that way — see NewGraphEnvironment/soul#129. + +Every number quoted in a `research/` run record, a NEWS entry or a PR body +should be traceable to a file here. + +## What is here + +| pattern | produced by | holds | +|---|---|---| +| `study_area_run/_up_.log` | `cypher_up.sh` via `study_area_run.sh` | droplet spin, tofu apply, cloud-init wait | +| `study_area_run/_prep_.log` | `cypher_prep.sh` | git reset, package install, snapshot, persist_init | +| `study_area_run/_stamps.tsv` | `host_stamp.R` | one provenance line per host — the parity gate's input | +| `study_area_run/_vintage.log` | `host_vintage.R` | primitive freshness per host | +| `study_area_run/_run_{local,}.log` | `wsg_run_one.R` | per-WSG modelling, `done in N min` | +| `study_area_run/_consolidate.log` | `schema_consolidate.R` | cross-host COPY | +| `study_area_run/_recompute.log` | `wsg_recompute_one.R` | post-consolidate access rebuild | +| `study_area_run/_compare.{log,csv}` | `study_area_compare.R` | bcfishpass parity | +| `study_area_run/_burn_.log` | `cypher_down.sh` | teardown + verification | +| `bcfp_baselines.csv` | `snapshot_bcfp.sh` | which upstream bcfp build each host loaded | +| `provincial_*/`, `methodology_delta/` | earlier orchestrators | historical runs | + +`` is UTC `YYYYMMDD_HHMMSS` and is shared by every file from one run, so a +single run's artifacts sort together. + +## Reading a run + +Phase durations are not logged as such — reconstruct them from file mtimes, +which is how the timings in `research/` were derived: + +```bash +python3 -c " +import glob,os,datetime +fs=sorted(glob.glob('data-raw/logs/study_area_run/_*'),key=os.path.getmtime) +t0=os.path.getmtime(fs[0]) +for f in fs: print('%6.1f min %s' % ((os.path.getmtime(f)-t0)/60, os.path.basename(f)))" ``` -### Future automation - -The csv-sync rewrite ([link#117](https://github.com/NewGraphEnvironment/link/issues/117)) will append to this CSV at sync time, recording which bcfp build the bundle CSVs are now SHA-pinned to. That closes the loop: every comparison rollup has both the bcfp build AND the matching bundle CSV state on file. - -Until then, manually append a row at the start of each provincial run. - -## Naming convention for log files - -Run logs follow `__.txt` where `` is `YYYYMMDDHHMM`. See `data-raw/README.md` (parent) for the broader conventions. +Per-WSG runtimes are better read from the run log itself +(`grep 'done in' _run_*.log`) or, for any run after v0.45.0, from +`.log` in Postgres, which records `date_start` / `date_end` per WSG +alongside the software SHAs. diff --git a/data-raw/logs/study_area_run/20260831_190558_burn_job1.log b/data-raw/logs/study_area_run/20260831_190558_burn_job1.log new file mode 100644 index 0000000..3c1187e --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_burn_job1.log @@ -0,0 +1,140 @@ +[cypher_down] destroying workspace 'job1' (no reserved IP) +module.droplet.digitalocean_droplet.this: Refreshing state... [id=596689967] +module.droplet.digitalocean_firewall.this[0]: Refreshing state... [id=2b99f569-a20d-49bf-a516-a39d795c2779] + +OpenTofu used the selected providers to generate the following execution +plan. Resource actions are indicated with the following symbols: + - destroy + +OpenTofu will perform the following actions: + + # module.droplet.digitalocean_droplet.this will be destroyed + - resource "digitalocean_droplet" "this" { + - backups = false -> null + - created_at = "2026-08-31T19:06:08Z" -> null + - disk = 400 -> null + - graceful_shutdown = false -> null + - id = "596689967" -> null + - image = "228350154" -> null + - ipv4_address = "147.182.228.14" -> null + - ipv4_address_private = "10.124.0.2" -> null + - ipv6 = false -> null + - locked = false -> null + - memory = 32768 -> null + - monitoring = false -> null + - name = "cypher-job1" -> null + - price_hourly = 0.25 -> null + - price_monthly = 168 -> null + - private_networking = true -> null + - public_networking = true -> null + - region = "sfo3" -> null + - resize_disk = true -> null + - size = "s-8vcpu-32gb-amd" -> null + - ssh_keys = [ + - "53795424", + - "56433424", + ] -> null + - status = "active" -> null + - tags = [ + - "compute", + - "cypher", + - "dev", + - "job1", + ] -> null + - urn = "do:droplet:596689967" -> null + - user_data = (sensitive value) -> null + - vcpus = 8 -> null + - volume_ids = [] -> null + - vpc_uuid = "0503d836-77ab-45e3-a50f-52542155755f" -> null + } + + # module.droplet.digitalocean_firewall.this[0] will be destroyed + - resource "digitalocean_firewall" "this" { + - created_at = "2026-08-31T19:08:27Z" -> null + - droplet_ids = [ + - 596689967, + ] -> null + - id = "2b99f569-a20d-49bf-a516-a39d795c2779" -> null + - name = "cypher-job1-fw" -> null + - pending_changes = [] -> null + - status = "succeeded" -> null + - tags = [] -> null + + - inbound_rule { + - port_range = "22" -> null + - protocol = "tcp" -> null + - source_addresses = [ + - "0.0.0.0/0", + ] -> null + - source_droplet_ids = [] -> null + - source_kubernetes_ids = [] -> null + - source_load_balancer_uids = [] -> null + - source_tags = [] -> null + } + - inbound_rule { + - port_range = "41641" -> null + - protocol = "udp" -> null + - source_addresses = [ + - "0.0.0.0/0", + - "::/0", + ] -> null + - source_droplet_ids = [] -> null + - source_kubernetes_ids = [] -> null + - source_load_balancer_uids = [] -> null + - source_tags = [] -> null + } + + - outbound_rule { + - destination_addresses = [ + - "0.0.0.0/0", + - "::/0", + ] -> null + - destination_droplet_ids = [] -> null + - destination_kubernetes_ids = [] -> null + - destination_load_balancer_uids = [] -> null + - destination_tags = [] -> null + - protocol = "icmp" -> null + } + - outbound_rule { + - destination_addresses = [ + - "0.0.0.0/0", + - "::/0", + ] -> null + - destination_droplet_ids = [] -> null + - destination_kubernetes_ids = [] -> null + - destination_load_balancer_uids = [] -> null + - destination_tags = [] -> null + - port_range = "1-65535" -> null + - protocol = "tcp" -> null + } + - outbound_rule { + - destination_addresses = [ + - "0.0.0.0/0", + - "::/0", + ] -> null + - destination_droplet_ids = [] -> null + - destination_kubernetes_ids = [] -> null + - destination_load_balancer_uids = [] -> null + - destination_tags = [] -> null + - port_range = "1-65535" -> null + - protocol = "udp" -> null + } + } + +Plan: 0 to add, 0 to change, 2 to destroy. + +Changes to Outputs: + - droplet_id = "596689967" -> null + - droplet_ip = "147.182.228.14" -> null + - tailnet_name = "cypher-job1" -> null +module.droplet.digitalocean_firewall.this[0]: Destroying... [id=2b99f569-a20d-49bf-a516-a39d795c2779] +module.droplet.digitalocean_firewall.this[0]: Destruction complete after 0s +module.droplet.digitalocean_droplet.this: Destroying... [id=596689967] +module.droplet.digitalocean_droplet.this: Still destroying... [id=596689967, 10s elapsed] +module.droplet.digitalocean_droplet.this: Still destroying... [id=596689967, 20s elapsed] +module.droplet.digitalocean_droplet.this: Destruction complete after 22s + +Destroy complete! Resources: 2 destroyed. + +[cypher_down] complete. Workspace 'job1' state empty (still listed). +Delete the workspace with: cd /Users/airvine/Projects/repo/rtj/env/do/dev/cypher && tofu workspace select default && tofu workspace delete job1 diff --git a/data-raw/logs/study_area_run/20260831_190558_compare.csv b/data-raw/logs/study_area_run/20260831_190558_compare.csv new file mode 100644 index 0000000..e096744 --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_compare.csv @@ -0,0 +1,12 @@ +"wsg","species","total_segs","match_pct","n_diffs","top_pattern","top_pattern_count" +"COWN","BT",19564,99.25,146,"REAR;MODELLED | ACCESS;MODELLED",32 +"COWN","CH",19564,99.74,51,"ACCESS;NONE | SPAWN;NONE",38 +"COWN","CM",19564,99.06,183,"ACCESS;NONE | SPAWN;NONE",169 +"COWN","CO",19564,99.34,129,"ACCESS;NONE | SPAWN;NONE",40 +"COWN","PK",19564,99.95,9,"ACCESS;MODELLED;INTERMITTENT | ",3 +"COWN","ST",19564,99.83,33,"ACCESS;NONE | SPAWN;NONE",10 +"TSIT","CH",15898,99.94,9,"ACCESS;MODELLED | ACCESS;REMEDIATED",2 +"TSIT","CM",15898,99.96,6,"SPAWN | ",3 +"TSIT","CO",15898,99.95,8,"ACCESS;MODELLED | ACCESS;REMEDIATED",2 +"TSIT","PK",15898,99.94,9,"ACCESS;NONE | SPAWN;NONE",3 +"TSIT","ST",15898,99.76,38,"REAR;MODELLED | ACCESS;MODELLED",7 diff --git a/data-raw/logs/study_area_run/20260831_190558_compare.log b/data-raw/logs/study_area_run/20260831_190558_compare.log new file mode 100644 index 0000000..a1c968f --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_compare.log @@ -0,0 +1,15 @@ +[study_area_compare] 11 rows across 2/2 WSGs -> /Users/airvine/Projects/repo/link/data-raw/logs/study_area_run/20260831_190558_compare.csv +# A tibble: 11 × 7 + wsg species total_segs match_pct n_diffs top_pattern top_pattern_count + * + 1 COWN BT 19564 99.2 146 "REAR;MODELLED … 32 + 2 COWN CH 19564 99.7 51 "ACCESS;NONE | … 38 + 3 COWN CM 19564 99.1 183 "ACCESS;NONE | … 169 + 4 COWN CO 19564 99.3 129 "ACCESS;NONE | … 40 + 5 COWN PK 19564 100.0 9 "ACCESS;MODELLE… 3 + 6 COWN ST 19564 99.8 33 "ACCESS;NONE | … 10 + 7 TSIT CH 15898 99.9 9 "ACCESS;MODELLE… 2 + 8 TSIT CM 15898 100.0 6 "SPAWN | " 3 + 9 TSIT CO 15898 100.0 8 "ACCESS;MODELLE… 2 +10 TSIT PK 15898 99.9 9 "ACCESS;NONE | … 3 +11 TSIT ST 15898 99.8 38 "REAR;MODELLED … 7 diff --git a/data-raw/logs/study_area_run/20260831_190558_consolidate.log b/data-raw/logs/study_area_run/20260831_190558_consolidate.log new file mode 100644 index 0000000..5e915eb --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_consolidate.log @@ -0,0 +1,93 @@ +[12:17:55] backup local fresh_smoke -> /tmp/fresh_smoke_pre_consolidate_202608311217.dump +[12:17:58] cypher@147.182.228.14 -> DELETE bucket (1 WSGs) from 15 tables +[12:17:59] cypher@147.182.228.14 -> COPY (bucket-filtered) for 15 tables +COPY 44 +COPY 31515 +COPY 1 +COPY 11 +COPY 38574 +COPY 38574 +COPY 0 +COPY 38574 +COPY 38574 +COPY 38574 +COPY 38574 +COPY 0 +COPY 38574 +COPY 0 +COPY 38574 +[12:18:18] cypher@147.182.228.14 -> DELETE bucket from source (post-COPY cleanup) +DELETE 44 +DELETE 31515 +DELETE 1 +DELETE 11 +DELETE 38574 +DELETE 38574 +DELETE 0 +DELETE 38574 +DELETE 38574 +DELETE 38574 +DELETE 38574 +DELETE 0 +DELETE 38574 +DELETE 0 +DELETE 38574 +[12:18:19] verify destination tables in schema 'fresh_smoke' + tbl rows +1 barrier_overrides 528 +2 barriers 56527 +3 log 2 +4 log_dimensions 8 +5 log_input 22 +6 log_parameters_fresh 11 +7 streams 71544 +8 streams_access 71544 +9 streams_habitat_bt 32970 +10 streams_habitat_ch 71544 +11 streams_habitat_cm 71544 +12 streams_habitat_co 71544 +13 streams_habitat_pk 71544 +14 streams_habitat_sk 0 +15 streams_habitat_st 71544 +16 streams_habitat_wct 0 +17 streams_mapping_code 71544 +$sources +$sources$`cypher@147.182.228.14` +$sources$`cypher@147.182.228.14`$ok +[1] TRUE + +$sources$`cypher@147.182.228.14`$stage +[1] "complete" + +$sources$`cypher@147.182.228.14`$pre_rows +[1] 322238 + +$sources$`cypher@147.182.228.14`$post_rows +[1] 662401 + +$sources$`cypher@147.182.228.14`$copied + [1] "barrier_overrides" "barriers" "log" + [4] "log_input" "streams" "streams_access" + [7] "streams_habitat_bt" "streams_habitat_ch" "streams_habitat_cm" +[10] "streams_habitat_co" "streams_habitat_pk" "streams_habitat_sk" +[13] "streams_habitat_st" "streams_habitat_wct" "streams_mapping_code" + +$sources$`cypher@147.182.228.14`$errored +character(0) + +$sources$`cypher@147.182.228.14`$skipped_source_only +character(0) + +$sources$`cypher@147.182.228.14`$skipped_dest_only +character(0) + + + +$tables + [1] "barrier_overrides" "barriers" "log" + [4] "log_dimensions" "log_input" "log_parameters_fresh" + [7] "streams" "streams_access" "streams_habitat_bt" +[10] "streams_habitat_ch" "streams_habitat_cm" "streams_habitat_co" +[13] "streams_habitat_pk" "streams_habitat_sk" "streams_habitat_st" +[16] "streams_habitat_wct" "streams_mapping_code" + diff --git a/data-raw/logs/study_area_run/20260831_190558_prep_job1.log b/data-raw/logs/study_area_run/20260831_190558_prep_job1.log new file mode 100644 index 0000000..01f3d09 --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_prep_job1.log @@ -0,0 +1,75 @@ +From https://github.com/NewGraphEnvironment/link + 11e9c22..0fa2c27 main -> origin/main + * [new branch] 138-lnk-pipeline-crossings-build-slim-fresh- -> origin/138-lnk-pipeline-crossings-build-slim-fresh- + * [new branch] 152-unified-persist-schema-barriers -> origin/152-unified-persist-schema-barriers + * [new branch] 162-lnk-compare-wsg-annotated-csv -> origin/162-lnk-compare-wsg-annotated-csv + * [new branch] 221-per-wsg-habitat-access-km-rollup -> origin/221-per-wsg-habitat-access-km-rollup + ff85787..f150a4c gh-pages -> origin/gh-pages + * [new branch] rbuildignore-internal-dirs -> origin/rbuildignore-internal-dirs + * [new tag] v0.47.1 -> v0.47.1 + * [new tag] v0.30.0 -> v0.30.0 + * [new tag] v0.30.1 -> v0.30.1 + * [new tag] v0.30.2 -> v0.30.2 + * [new tag] v0.31.0 -> v0.31.0 + * [new tag] v0.31.1 -> v0.31.1 + * [new tag] v0.32.0 -> v0.32.0 + * [new tag] v0.32.1 -> v0.32.1 + * [new tag] v0.33.0 -> v0.33.0 + * [new tag] v0.34.0 -> v0.34.0 + * [new tag] v0.35.0 -> v0.35.0 + * [new tag] v0.35.1 -> v0.35.1 + * [new tag] v0.36.0 -> v0.36.0 + * [new tag] v0.36.1 -> v0.36.1 + * [new tag] v0.37.0 -> v0.37.0 + * [new tag] v0.38.0 -> v0.38.0 + * [new tag] v0.38.1 -> v0.38.1 + * [new tag] v0.39.0 -> v0.39.0 + * [new tag] v0.39.1 -> v0.39.1 + * [new tag] v0.40.0 -> v0.40.0 + * [new tag] v0.40.1 -> v0.40.1 + * [new tag] v0.40.2 -> v0.40.2 + * [new tag] v0.40.3 -> v0.40.3 + * [new tag] v0.40.4 -> v0.40.4 + * [new tag] v0.40.5 -> v0.40.5 + * [new tag] v0.41.0 -> v0.41.0 + * [new tag] v0.41.1 -> v0.41.1 + * [new tag] v0.41.2 -> v0.41.2 + * [new tag] v0.41.3 -> v0.41.3 + * [new tag] v0.41.4 -> v0.41.4 + * [new tag] v0.42.0 -> v0.42.0 + * [new tag] v0.43.0 -> v0.43.0 + * [new tag] v0.44.0 -> v0.44.0 + * [new tag] v0.44.1 -> v0.44.1 + * [new tag] v0.44.2 -> v0.44.2 + * [new tag] v0.44.3 -> v0.44.3 + * [new tag] v0.45.0 -> v0.45.0 + * [new tag] v0.45.1 -> v0.45.1 + * [new tag] v0.45.2 -> v0.45.2 + * [new tag] v0.45.3 -> v0.45.3 + * [new tag] v0.46.0 -> v0.46.0 + * [new tag] v0.47.0 -> v0.47.0 +Your branch is behind 'origin/main' by 181 commits, and can be fast-forwarded. + (use "git pull" to update your local branch) +HEAD is now at 0fa2c27 Merge pull request #252 from NewGraphEnvironment/246-pilot-gate-fixes +✔ Built link 0.47.1 (10.1s) +✔ Installed link 0.47.1 (local) (210ms) +✔ 1 pkg + 55 deps: kept 42, upd 3, added 1, dld 4 (NA B) [52.6s] +=== provenance: link_sha=0fa2c27d486f dirty=false fresh_sha=7f12d99115b7 +[preflight] fresh 0.33.0 (floor 0.33.0) - OK, all required symbols present +=== link: 0.47.1 fresh: 0.33.0 +2026-08-31 19:13:54,870:INFO:bcdata.cli: Load of whse_fish.pscis_remediation_svw to whse_fish.pscis_remediation_svw in postgresql://postgres:postgres@localhost:5432/fwapg complete +DROP TABLE +CREATE TABLE +Stamped data-raw/logs/bcfp_baselines.csv with bcfp build v0.7.15-47-ga702229 (head_sha=a702229) +snapshot_bcfp.sh: complete. + +NOTICE: column "rear_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "notes" of relation "log_dimensions" already exists, skipping + +=== lnk_persist_init done +=== READY diff --git a/data-raw/logs/study_area_run/20260831_190558_recompute.log b/data-raw/logs/study_area_run/20260831_190558_recompute.log new file mode 100644 index 0000000..4cdc21e --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_recompute.log @@ -0,0 +1,114 @@ +[1] 0 +[1] 0 +Warning message: +In read.table(file = file, header = header, sep = sep, quote = quote, : + incomplete final line found by readTableHeader on '/Users/airvine/Projects/repo/link/inst/extdata/configs/bcfishpass/overrides/cabd_blkey_xref.csv' +NOTICE: view "barriers_bt_unified" does not exist, skipping + +NOTICE: view "barriers_bt_access" does not exist, skipping + +NOTICE: view "barriers_ch_unified" does not exist, skipping + +NOTICE: view "barriers_ch_access" does not exist, skipping + +NOTICE: view "barriers_cm_unified" does not exist, skipping + +NOTICE: view "barriers_cm_access" does not exist, skipping + +NOTICE: view "barriers_co_unified" does not exist, skipping + +NOTICE: view "barriers_co_access" does not exist, skipping + +NOTICE: view "barriers_pk_unified" does not exist, skipping + +NOTICE: view "barriers_pk_access" does not exist, skipping + +NOTICE: view "barriers_st_unified" does not exist, skipping + +NOTICE: view "barriers_st_access" does not exist, skipping + +NOTICE: view "barriers_anthropogenic_unified" does not exist, skipping + +NOTICE: view "barriers_pscis_unified" does not exist, skipping + +NOTICE: view "barriers_dams_unified" does not exist, skipping + +NOTICE: table "zz_lnk_streams_cown" does not exist, skipping + +Warning messages: +1: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +2: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +3: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +4: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +5: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +6: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +7: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +8: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +9: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +# A tibble: 32,970 × 7 + id_segment mapping_code_bt mapping_code_ch mapping_code_cm mapping_code_co + + 1 12905 "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" + 2 25055 "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" + 3 13380 "" "" "" "" + 4 13381 "" "" "" "" + 5 18907 "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" + 6 27698 "ACCESS;MODELLED;… "" "" "" + 7 10923 "ACCESS;MODELLED;… "" "" "" + 8 8292 "" "" "" "" + 9 11530 "ACCESS;MODELLED" "" "" "" +10 15589 "" "" "" "" +# ℹ 32,960 more rows +# ℹ 2 more variables: mapping_code_pk , mapping_code_st +[1] 32970 +[wsg_recompute_one] COWN recomputed in 0.26 min (persist=fresh_smoke) +[1] 0 +[1] 0 +Warning message: +In read.table(file = file, header = header, sep = sep, quote = quote, : + incomplete final line found by readTableHeader on '/Users/airvine/Projects/repo/link/inst/extdata/configs/bcfishpass/overrides/cabd_blkey_xref.csv' +NOTICE: table "zz_lnk_streams_tsit" does not exist, skipping + +Warning messages: +1: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +2: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +3: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +4: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +5: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +6: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +7: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +8: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +# A tibble: 38,574 × 6 + id_segment mapping_code_ch mapping_code_cm mapping_code_co mapping_code_pk + + 1 17140 "" "" "" "" + 2 8529 "" "" "" "" + 3 14699 "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" + 4 14726 "" "" "" "" + 5 59 "" "" "" "" + 6 29979 "" "" "" "" + 7 31267 "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" "ACCESS;NONE" + 8 6612 "" "" "" "" + 9 6902 "" "" "" "" +10 3584 "" "" "" "" +# ℹ 38,564 more rows +# ℹ 1 more variable: mapping_code_st +[1] 38574 +[wsg_recompute_one] TSIT recomputed in 0.41 min (persist=fresh_smoke) diff --git a/data-raw/logs/study_area_run/20260831_190558_run_job1.log b/data-raw/logs/study_area_run/20260831_190558_run_job1.log new file mode 100644 index 0000000..5f171e3 --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_run_job1.log @@ -0,0 +1,533 @@ +Warning message: +In read.table(file = file, header = header, sep = sep, quote = quote, : + incomplete final line found by readTableHeader on '/home/cypher/R/x86_64-pc-linux-gnu-library/4.6/link/extdata/configs/bcfishpass/overrides/cabd_blkey_xref.csv' +NOTICE: column "run_id" of relation "log" already exists, skipping + +NOTICE: column "watershed_group_code" of relation "log" already exists, skipping + +NOTICE: column "date_start" of relation "log" already exists, skipping + +NOTICE: column "date_end" of relation "log" already exists, skipping + +NOTICE: column "run_label" of relation "log" already exists, skipping + +NOTICE: column "host" of relation "log" already exists, skipping + +NOTICE: column "config_name" of relation "log" already exists, skipping + +NOTICE: column "config_hash" of relation "log" already exists, skipping + +NOTICE: column "config_drift" of relation "log" already exists, skipping + +NOTICE: column "link_version" of relation "log" already exists, skipping + +NOTICE: column "link_sha" of relation "log" already exists, skipping + +NOTICE: column "link_dirty" of relation "log" already exists, skipping + +NOTICE: column "fresh_version" of relation "log" already exists, skipping + +NOTICE: column "fresh_sha" of relation "log" already exists, skipping + +NOTICE: column "fresh_dirty" of relation "log" already exists, skipping + +NOTICE: column "crate_version" of relation "log" already exists, skipping + +NOTICE: column "fwapg_sha" of relation "log" already exists, skipping + +NOTICE: column "arg_dams" of relation "log" already exists, skipping + +NOTICE: column "arg_mapping_code" of relation "log" already exists, skipping + +NOTICE: column "arg_cleanup_working" of relation "log" already exists, skipping + +NOTICE: column "schema_persist" of relation "log" already exists, skipping + +NOTICE: column "species" of relation "log" already exists, skipping + +NOTICE: column "wsg_upstream" of relation "log" already exists, skipping + +NOTICE: column "bcfp_model_run_id" of relation "log" already exists, skipping + +NOTICE: column "bcfp_model_version" of relation "log" already exists, skipping + +NOTICE: column "notes" of relation "log" already exists, skipping + +NOTICE: column "run_id" of relation "log_input" already exists, skipping + +NOTICE: column "watershed_group_code" of relation "log_input" already exists, skipping + +NOTICE: column "table_name" of relation "log_input" already exists, skipping + +NOTICE: column "row_count" of relation "log_input" already exists, skipping + +NOTICE: column "row_count_estimated" of relation "log_input" already exists, skipping + +NOTICE: column "size_bytes" of relation "log_input" already exists, skipping + +NOTICE: column "last_analyze" of relation "log_input" already exists, skipping + +NOTICE: column "source" of relation "log_input" already exists, skipping + +NOTICE: column "source_at" of relation "log_input" already exists, skipping + +NOTICE: column "config_hash" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "species_code" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "access_gradient_max" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "spawn_gradient_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "rear_gradient_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_rearing" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_direction" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_bridge_gradient" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_bridge_distance" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_confluence_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawning" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_direction" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_bridge_gradient" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_bridge_distance" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_confluence_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_threshold" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_date_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_buffer_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_species" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_control_apply" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "config_hash" of relation "log_dimensions" already exists, skipping + +NOTICE: column "species" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_lake" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_stream" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_stream_in_waterbody" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_area_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_no_fw" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_in_waterbody" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_polygon" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_area_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_all_edges" of relation "log_dimensions" already exists, skipping + +NOTICE: column "river_skip_cw_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_bypass" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_parent_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_child_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_child_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_requires_connected" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_direction" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_gradient_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_cw_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_edge_types" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_lake_adjacent" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_requires_connected" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "notes" of relation "log_dimensions" already exists, skipping + +NOTICE: schema "working_tsit" does not exist, skipping + +NOTICE: schema "working_tsit" does not exist, skipping + +NOTICE: schema "working_tsit" does not exist, skipping + +NOTICE: schema "working_tsit" does not exist, skipping + +NOTICE: schema "fresh" already exists, skipping + +NOTICE: table "barriers_definite" does not exist, skipping + +NOTICE: table "barriers_definite_control" does not exist, skipping + +NOTICE: table "observations" does not exist, skipping + +NOTICE: table "streams_blk" does not exist, skipping + +NOTICE: table "gradient_barriers_raw" does not exist, skipping + +NOTICE: table "natural_barriers" does not exist, skipping + +NOTICE: table "barriers_subsurfaceflow" does not exist, skipping + +NOTICE: table "barrier_overrides" does not exist, skipping + +NOTICE: table "barriers_bt" does not exist, skipping + +NOTICE: table "barriers_ch" does not exist, skipping + +NOTICE: table "barriers_cm" does not exist, skipping + +NOTICE: table "barriers_co" does not exist, skipping + +NOTICE: table "barriers_pk" does not exist, skipping + +NOTICE: table "barriers_sk" does not exist, skipping + +NOTICE: table "barriers_st" does not exist, skipping + +NOTICE: table "barriers_wct" does not exist, skipping + +NOTICE: table "gradient_barriers_minimal" does not exist, skipping + +NOTICE: table "streams" does not exist, skipping + +NOTICE: table "streams_habitat" does not exist, skipping + +NOTICE: table "cabd_exclusions" does not exist, skipping + +NOTICE: table "cabd_blkey_xref" does not exist, skipping + +NOTICE: table "cabd_passability_status_updates" does not exist, skipping + +NOTICE: table "cabd_additions" does not exist, skipping + +NOTICE: table "cabd_dams_raw" does not exist, skipping + +NOTICE: table "dams" does not exist, skipping + +NOTICE: table "pscis_modelledcrossings_streams_xref" does not exist, skipping + +NOTICE: table "pscis_stream_candidates" does not exist, skipping + +NOTICE: table "pscis_streams_150m" does not exist, skipping + +NOTICE: table "pscis_picked" does not exist, skipping + +NOTICE: table "pscis" does not exist, skipping + +NOTICE: table "crossings_lookup" does not exist, skipping + +NOTICE: table "barriers_anthropogenic" does not exist, skipping + +NOTICE: table "barriers_pscis" does not exist, skipping + +NOTICE: table "barriers_dams" does not exist, skipping + +NOTICE: table "barriers_remediations" does not exist, skipping + +NOTICE: table "observations_breaks" does not exist, skipping + +NOTICE: table "habitat_endpoints" does not exist, skipping + +NOTICE: table "crossings_breaks" does not exist, skipping + +NOTICE: table "streams_breaks" does not exist, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_02" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_cm" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_pk" does not exist, skipping + +NOTICE: table "streams_acc_02_ovr_st" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_st" does not exist, skipping + +NOTICE: schema "fresh_smoke" already exists, skipping + +NOTICE: schema "fresh_smoke" already exists, skipping + +NOTICE: relation "log" already exists, skipping + +NOTICE: column "run_id" of relation "log" already exists, skipping + +NOTICE: column "watershed_group_code" of relation "log" already exists, skipping + +NOTICE: column "date_start" of relation "log" already exists, skipping + +NOTICE: column "date_end" of relation "log" already exists, skipping + +NOTICE: column "run_label" of relation "log" already exists, skipping + +NOTICE: column "host" of relation "log" already exists, skipping + +NOTICE: column "config_name" of relation "log" already exists, skipping + +NOTICE: column "config_hash" of relation "log" already exists, skipping + +NOTICE: column "config_drift" of relation "log" already exists, skipping + +NOTICE: column "link_version" of relation "log" already exists, skipping + +NOTICE: column "link_sha" of relation "log" already exists, skipping + +NOTICE: column "link_dirty" of relation "log" already exists, skipping + +NOTICE: column "fresh_version" of relation "log" already exists, skipping + +NOTICE: column "fresh_sha" of relation "log" already exists, skipping + +NOTICE: column "fresh_dirty" of relation "log" already exists, skipping + +NOTICE: column "crate_version" of relation "log" already exists, skipping + +NOTICE: column "fwapg_sha" of relation "log" already exists, skipping + +NOTICE: column "arg_dams" of relation "log" already exists, skipping + +NOTICE: column "arg_mapping_code" of relation "log" already exists, skipping + +NOTICE: column "arg_cleanup_working" of relation "log" already exists, skipping + +NOTICE: column "schema_persist" of relation "log" already exists, skipping + +NOTICE: column "species" of relation "log" already exists, skipping + +NOTICE: column "wsg_upstream" of relation "log" already exists, skipping + +NOTICE: column "bcfp_model_run_id" of relation "log" already exists, skipping + +NOTICE: column "bcfp_model_version" of relation "log" already exists, skipping + +NOTICE: column "notes" of relation "log" already exists, skipping + +NOTICE: relation "log_input" already exists, skipping + +NOTICE: column "run_id" of relation "log_input" already exists, skipping + +NOTICE: column "watershed_group_code" of relation "log_input" already exists, skipping + +NOTICE: column "table_name" of relation "log_input" already exists, skipping + +NOTICE: column "row_count" of relation "log_input" already exists, skipping + +NOTICE: column "row_count_estimated" of relation "log_input" already exists, skipping + +NOTICE: column "size_bytes" of relation "log_input" already exists, skipping + +NOTICE: column "last_analyze" of relation "log_input" already exists, skipping + +NOTICE: column "source" of relation "log_input" already exists, skipping + +NOTICE: column "source_at" of relation "log_input" already exists, skipping + +NOTICE: relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "config_hash" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "species_code" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "access_gradient_max" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "spawn_gradient_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "rear_gradient_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_rearing" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_direction" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_bridge_gradient" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_bridge_distance" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_confluence_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawning" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_direction" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_bridge_gradient" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_bridge_distance" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_confluence_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_threshold" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_date_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_buffer_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_species" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_control_apply" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: relation "log_dimensions" already exists, skipping + +NOTICE: column "config_hash" of relation "log_dimensions" already exists, skipping + +NOTICE: column "species" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_lake" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_stream" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_stream_in_waterbody" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_area_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_no_fw" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_in_waterbody" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_polygon" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_area_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_all_edges" of relation "log_dimensions" already exists, skipping + +NOTICE: column "river_skip_cw_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_bypass" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_parent_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_child_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_child_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_requires_connected" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_direction" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_gradient_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_cw_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_edge_types" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_lake_adjacent" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_requires_connected" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "notes" of relation "log_dimensions" already exists, skipping + +NOTICE: relation "log_wsg_date_idx" already exists, skipping + +NOTICE: relation "log_config_idx" already exists, skipping + +NOTICE: relation "log_input_rn_idx" already exists, skipping + +NOTICE: table "barriers" does not exist, skipping + +NOTICE: view "barriers_ch_unified" does not exist, skipping + +NOTICE: view "barriers_ch_access" does not exist, skipping + +NOTICE: view "barriers_cm_unified" does not exist, skipping + +NOTICE: view "barriers_cm_access" does not exist, skipping + +NOTICE: view "barriers_co_unified" does not exist, skipping + +NOTICE: view "barriers_co_access" does not exist, skipping + +NOTICE: view "barriers_pk_unified" does not exist, skipping + +NOTICE: view "barriers_pk_access" does not exist, skipping + +NOTICE: view "barriers_st_unified" does not exist, skipping + +NOTICE: view "barriers_st_access" does not exist, skipping + +NOTICE: view "barriers_anthropogenic_unified" does not exist, skipping + +NOTICE: view "barriers_pscis_unified" does not exist, skipping + +NOTICE: view "barriers_dams_unified" does not exist, skipping + +[wsg_run_one] TSIT done in 3.4 min (persist=fresh_smoke) +Warning messages: +1: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +2: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +3: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +4: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +5: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +6: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +7: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +8: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +9: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. +10: In st_read.DBIObject(conn, query = query) : + Could not find a simple features geometry column. Will return a `data.frame`. diff --git a/data-raw/logs/study_area_run/20260831_190558_run_local.log b/data-raw/logs/study_area_run/20260831_190558_run_local.log new file mode 100644 index 0000000..6bbb718 --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_run_local.log @@ -0,0 +1,521 @@ +Warning message: +In read.table(file = file, header = header, sep = sep, quote = quote, : + incomplete final line found by readTableHeader on '/Users/airvine/Projects/repo/link/inst/extdata/configs/bcfishpass/overrides/cabd_blkey_xref.csv' +NOTICE: column "run_id" of relation "log" already exists, skipping + +NOTICE: column "watershed_group_code" of relation "log" already exists, skipping + +NOTICE: column "date_start" of relation "log" already exists, skipping + +NOTICE: column "date_end" of relation "log" already exists, skipping + +NOTICE: column "run_label" of relation "log" already exists, skipping + +NOTICE: column "host" of relation "log" already exists, skipping + +NOTICE: column "config_name" of relation "log" already exists, skipping + +NOTICE: column "config_hash" of relation "log" already exists, skipping + +NOTICE: column "config_drift" of relation "log" already exists, skipping + +NOTICE: column "link_version" of relation "log" already exists, skipping + +NOTICE: column "link_sha" of relation "log" already exists, skipping + +NOTICE: column "link_dirty" of relation "log" already exists, skipping + +NOTICE: column "fresh_version" of relation "log" already exists, skipping + +NOTICE: column "fresh_sha" of relation "log" already exists, skipping + +NOTICE: column "fresh_dirty" of relation "log" already exists, skipping + +NOTICE: column "crate_version" of relation "log" already exists, skipping + +NOTICE: column "fwapg_sha" of relation "log" already exists, skipping + +NOTICE: column "arg_dams" of relation "log" already exists, skipping + +NOTICE: column "arg_mapping_code" of relation "log" already exists, skipping + +NOTICE: column "arg_cleanup_working" of relation "log" already exists, skipping + +NOTICE: column "schema_persist" of relation "log" already exists, skipping + +NOTICE: column "species" of relation "log" already exists, skipping + +NOTICE: column "wsg_upstream" of relation "log" already exists, skipping + +NOTICE: column "bcfp_model_run_id" of relation "log" already exists, skipping + +NOTICE: column "bcfp_model_version" of relation "log" already exists, skipping + +NOTICE: column "notes" of relation "log" already exists, skipping + +NOTICE: column "run_id" of relation "log_input" already exists, skipping + +NOTICE: column "watershed_group_code" of relation "log_input" already exists, skipping + +NOTICE: column "table_name" of relation "log_input" already exists, skipping + +NOTICE: column "row_count" of relation "log_input" already exists, skipping + +NOTICE: column "row_count_estimated" of relation "log_input" already exists, skipping + +NOTICE: column "size_bytes" of relation "log_input" already exists, skipping + +NOTICE: column "last_analyze" of relation "log_input" already exists, skipping + +NOTICE: column "source" of relation "log_input" already exists, skipping + +NOTICE: column "source_at" of relation "log_input" already exists, skipping + +NOTICE: column "config_hash" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "species_code" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "access_gradient_max" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "spawn_gradient_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "rear_gradient_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_rearing" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_direction" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_bridge_gradient" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_bridge_distance" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_confluence_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawning" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_direction" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_bridge_gradient" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_bridge_distance" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_confluence_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_threshold" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_date_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_buffer_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_species" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_control_apply" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "config_hash" of relation "log_dimensions" already exists, skipping + +NOTICE: column "species" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_lake" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_stream" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_stream_in_waterbody" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_area_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_no_fw" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_in_waterbody" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_polygon" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_area_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_all_edges" of relation "log_dimensions" already exists, skipping + +NOTICE: column "river_skip_cw_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_bypass" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_parent_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_child_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_child_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_requires_connected" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_direction" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_gradient_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_cw_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_edge_types" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_lake_adjacent" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_requires_connected" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "notes" of relation "log_dimensions" already exists, skipping + +NOTICE: schema "working_cown" does not exist, skipping + +NOTICE: schema "working_cown" does not exist, skipping + +NOTICE: schema "working_cown" does not exist, skipping + +NOTICE: schema "working_cown" does not exist, skipping + +NOTICE: schema "fresh" already exists, skipping + +NOTICE: table "barriers_definite" does not exist, skipping + +NOTICE: table "observations" does not exist, skipping + +NOTICE: table "streams_blk" does not exist, skipping + +NOTICE: table "gradient_barriers_raw" does not exist, skipping + +NOTICE: table "natural_barriers" does not exist, skipping + +NOTICE: table "barriers_subsurfaceflow" does not exist, skipping + +NOTICE: table "barrier_overrides" does not exist, skipping + +NOTICE: table "barriers_bt" does not exist, skipping + +NOTICE: table "barriers_ch" does not exist, skipping + +NOTICE: table "barriers_cm" does not exist, skipping + +NOTICE: table "barriers_co" does not exist, skipping + +NOTICE: table "barriers_pk" does not exist, skipping + +NOTICE: table "barriers_sk" does not exist, skipping + +NOTICE: table "barriers_st" does not exist, skipping + +NOTICE: table "barriers_wct" does not exist, skipping + +NOTICE: table "gradient_barriers_minimal" does not exist, skipping + +NOTICE: table "streams" does not exist, skipping + +NOTICE: table "streams_habitat" does not exist, skipping + +NOTICE: table "cabd_exclusions" does not exist, skipping + +NOTICE: table "cabd_blkey_xref" does not exist, skipping + +NOTICE: table "cabd_passability_status_updates" does not exist, skipping + +NOTICE: table "cabd_additions" does not exist, skipping + +NOTICE: table "cabd_dams_raw" does not exist, skipping + +NOTICE: table "dams" does not exist, skipping + +NOTICE: table "pscis_modelledcrossings_streams_xref" does not exist, skipping + +NOTICE: table "pscis_stream_candidates" does not exist, skipping + +NOTICE: table "pscis_streams_150m" does not exist, skipping + +NOTICE: table "pscis_picked" does not exist, skipping + +NOTICE: table "pscis" does not exist, skipping + +NOTICE: table "crossings_lookup" does not exist, skipping + +NOTICE: table "barriers_anthropogenic" does not exist, skipping + +NOTICE: table "barriers_pscis" does not exist, skipping + +NOTICE: table "barriers_dams" does not exist, skipping + +NOTICE: table "barriers_remediations" does not exist, skipping + +NOTICE: table "observations_breaks" does not exist, skipping + +NOTICE: table "habitat_endpoints" does not exist, skipping + +NOTICE: table "crossings_breaks" does not exist, skipping + +NOTICE: table "streams_breaks" does not exist, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_02" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_cm" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_pk" does not exist, skipping + +NOTICE: table "streams_acc_02_ovr_st" does not exist, skipping + +NOTICE: table "frs_clusters_bt" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_st" does not exist, skipping + +NOTICE: schema "fresh_smoke" already exists, skipping + +NOTICE: schema "fresh_smoke" already exists, skipping + +NOTICE: relation "log" already exists, skipping + +NOTICE: column "run_id" of relation "log" already exists, skipping + +NOTICE: column "watershed_group_code" of relation "log" already exists, skipping + +NOTICE: column "date_start" of relation "log" already exists, skipping + +NOTICE: column "date_end" of relation "log" already exists, skipping + +NOTICE: column "run_label" of relation "log" already exists, skipping + +NOTICE: column "host" of relation "log" already exists, skipping + +NOTICE: column "config_name" of relation "log" already exists, skipping + +NOTICE: column "config_hash" of relation "log" already exists, skipping + +NOTICE: column "config_drift" of relation "log" already exists, skipping + +NOTICE: column "link_version" of relation "log" already exists, skipping + +NOTICE: column "link_sha" of relation "log" already exists, skipping + +NOTICE: column "link_dirty" of relation "log" already exists, skipping + +NOTICE: column "fresh_version" of relation "log" already exists, skipping + +NOTICE: column "fresh_sha" of relation "log" already exists, skipping + +NOTICE: column "fresh_dirty" of relation "log" already exists, skipping + +NOTICE: column "crate_version" of relation "log" already exists, skipping + +NOTICE: column "fwapg_sha" of relation "log" already exists, skipping + +NOTICE: column "arg_dams" of relation "log" already exists, skipping + +NOTICE: column "arg_mapping_code" of relation "log" already exists, skipping + +NOTICE: column "arg_cleanup_working" of relation "log" already exists, skipping + +NOTICE: column "schema_persist" of relation "log" already exists, skipping + +NOTICE: column "species" of relation "log" already exists, skipping + +NOTICE: column "wsg_upstream" of relation "log" already exists, skipping + +NOTICE: column "bcfp_model_run_id" of relation "log" already exists, skipping + +NOTICE: column "bcfp_model_version" of relation "log" already exists, skipping + +NOTICE: column "notes" of relation "log" already exists, skipping + +NOTICE: relation "log_input" already exists, skipping + +NOTICE: column "run_id" of relation "log_input" already exists, skipping + +NOTICE: column "watershed_group_code" of relation "log_input" already exists, skipping + +NOTICE: column "table_name" of relation "log_input" already exists, skipping + +NOTICE: column "row_count" of relation "log_input" already exists, skipping + +NOTICE: column "row_count_estimated" of relation "log_input" already exists, skipping + +NOTICE: column "size_bytes" of relation "log_input" already exists, skipping + +NOTICE: column "last_analyze" of relation "log_input" already exists, skipping + +NOTICE: column "source" of relation "log_input" already exists, skipping + +NOTICE: column "source_at" of relation "log_input" already exists, skipping + +NOTICE: relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "config_hash" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "species_code" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "access_gradient_max" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "spawn_gradient_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "rear_gradient_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_rearing" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_direction" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_bridge_gradient" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_bridge_distance" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_confluence_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawning" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_direction" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_bridge_gradient" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_bridge_distance" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "cluster_spawn_confluence_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_threshold" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_date_min" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_buffer_m" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_species" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: column "observation_control_apply" of relation "log_parameters_fresh" already exists, skipping + +NOTICE: relation "log_dimensions" already exists, skipping + +NOTICE: column "config_hash" of relation "log_dimensions" already exists, skipping + +NOTICE: column "species" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_lake" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_stream" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_stream_in_waterbody" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_area_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_no_fw" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_in_waterbody" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_polygon" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_area_only" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_all_edges" of relation "log_dimensions" already exists, skipping + +NOTICE: column "river_skip_cw_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_bypass" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_parent_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_child_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_child_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_stream_order_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_requires_connected" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_direction" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_gradient_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_cw_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_edge_types" of relation "log_dimensions" already exists, skipping + +NOTICE: column "spawn_connected_lake_adjacent" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_requires_connected" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_connected_distance_max" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_lake_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "rear_wetland_ha_min" of relation "log_dimensions" already exists, skipping + +NOTICE: column "notes" of relation "log_dimensions" already exists, skipping + +NOTICE: relation "log_wsg_date_idx" already exists, skipping + +NOTICE: relation "log_config_idx" already exists, skipping + +NOTICE: relation "log_input_rn_idx" already exists, skipping + +NOTICE: table "barriers" does not exist, skipping + +NOTICE: view "barriers_bt_unified" does not exist, skipping + +NOTICE: view "barriers_bt_access" does not exist, skipping + +NOTICE: view "barriers_ch_unified" does not exist, skipping + +NOTICE: view "barriers_ch_access" does not exist, skipping + +NOTICE: view "barriers_cm_unified" does not exist, skipping + +NOTICE: view "barriers_cm_access" does not exist, skipping + +NOTICE: view "barriers_co_unified" does not exist, skipping + +NOTICE: view "barriers_co_access" does not exist, skipping + +NOTICE: view "barriers_pk_unified" does not exist, skipping + +NOTICE: view "barriers_pk_access" does not exist, skipping + +NOTICE: view "barriers_st_unified" does not exist, skipping + +NOTICE: view "barriers_st_access" does not exist, skipping + +NOTICE: view "barriers_anthropogenic_unified" does not exist, skipping + +NOTICE: view "barriers_pscis_unified" does not exist, skipping + +NOTICE: view "barriers_dams_unified" does not exist, skipping + +There were 11 warnings (use warnings() to see them) +[wsg_run_one] COWN done in 1.3 min (persist=fresh_smoke) diff --git a/data-raw/logs/study_area_run/20260831_190558_stamps.tsv b/data-raw/logs/study_area_run/20260831_190558_stamps.tsv new file mode 100644 index 0000000..6c7a135 --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_stamps.tsv @@ -0,0 +1,2 @@ +m1 0.47.1 0fa2c27d486f 0.33.0 NA 0fa2c27d486f TRUE sha256:19e3a e6e1eb0f4718 4.5.2 +cypher-job1 0.47.1 0fa2c27d486f 0.33.0 7f12d99115b7 0fa2c27d486f TRUE sha256:19e3a e6e1eb0f4718 4.6.1 diff --git a/data-raw/logs/study_area_run/20260831_190558_up_job1.log b/data-raw/logs/study_area_run/20260831_190558_up_job1.log new file mode 100644 index 0000000..37244a7 --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_up_job1.log @@ -0,0 +1,161 @@ +[cypher_up] workspace: job1 (logs: /Users/airvine/Projects/repo/rtj/scripts/cypher/logs/202608311206_cypher_up_job1.txt) + +OpenTofu used the selected providers to generate the following execution +plan. Resource actions are indicated with the following symbols: + + create + +OpenTofu will perform the following actions: + + # module.droplet.digitalocean_droplet.this will be created + + resource "digitalocean_droplet" "this" { + + backups = false + + created_at = (known after apply) + + disk = (known after apply) + + graceful_shutdown = false + + id = (known after apply) + + image = "228350154" + + ipv4_address = (known after apply) + + ipv4_address_private = (known after apply) + + ipv6 = false + + ipv6_address = (known after apply) + + locked = (known after apply) + + memory = (known after apply) + + monitoring = false + + name = "cypher-job1" + + price_hourly = (known after apply) + + price_monthly = (known after apply) + + private_networking = (known after apply) + + public_networking = (known after apply) + + region = "sfo3" + + resize_disk = true + + size = "s-8vcpu-32gb-amd" + + ssh_keys = [ + + "53795424", + + "56433424", + ] + + status = (known after apply) + + tags = [ + + "compute", + + "cypher", + + "dev", + + "job1", + ] + + urn = (known after apply) + + user_data = (sensitive value) + + vcpus = (known after apply) + + volume_ids = (known after apply) + + vpc_uuid = (known after apply) + } + + # module.droplet.digitalocean_firewall.this[0] will be created + + resource "digitalocean_firewall" "this" { + + created_at = (known after apply) + + droplet_ids = (known after apply) + + id = (known after apply) + + name = "cypher-job1-fw" + + pending_changes = (known after apply) + + status = (known after apply) + + + inbound_rule { + + port_range = "22" + + protocol = "tcp" + + source_addresses = [ + + "0.0.0.0/0", + ] + + source_droplet_ids = [] + + source_kubernetes_ids = [] + + source_load_balancer_uids = [] + + source_tags = [] + } + + inbound_rule { + + port_range = "41641" + + protocol = "udp" + + source_addresses = [ + + "0.0.0.0/0", + + "::/0", + ] + + source_droplet_ids = [] + + source_kubernetes_ids = [] + + source_load_balancer_uids = [] + + source_tags = [] + } + + + outbound_rule { + + destination_addresses = [ + + "0.0.0.0/0", + + "::/0", + ] + + destination_droplet_ids = [] + + destination_kubernetes_ids = [] + + destination_load_balancer_uids = [] + + destination_tags = [] + + protocol = "icmp" + } + + outbound_rule { + + destination_addresses = [ + + "0.0.0.0/0", + + "::/0", + ] + + destination_droplet_ids = [] + + destination_kubernetes_ids = [] + + destination_load_balancer_uids = [] + + destination_tags = [] + + port_range = "1-65535" + + protocol = "tcp" + } + + outbound_rule { + + destination_addresses = [ + + "0.0.0.0/0", + + "::/0", + ] + + destination_droplet_ids = [] + + destination_kubernetes_ids = [] + + destination_load_balancer_uids = [] + + destination_tags = [] + + port_range = "1-65535" + + protocol = "udp" + } + } + +Plan: 2 to add, 0 to change, 0 to destroy. + +Changes to Outputs: + + droplet_id = (known after apply) + + droplet_ip = (known after apply) + + tailnet_name = "cypher-job1" +module.droplet.digitalocean_droplet.this: Creating... +module.droplet.digitalocean_droplet.this: Still creating... [10s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [20s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [30s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [40s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [50s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [1m0s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [1m10s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [1m20s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [1m30s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [1m40s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [1m50s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [2m0s elapsed] +module.droplet.digitalocean_droplet.this: Still creating... [2m10s elapsed] +module.droplet.digitalocean_droplet.this: Creation complete after 2m20s [id=596689967] +module.droplet.digitalocean_firewall.this[0]: Creating... +module.droplet.digitalocean_firewall.this[0]: Creation complete after 1s [id=2b99f569-a20d-49bf-a516-a39d795c2779] + +Apply complete! Resources: 2 added, 0 changed, 0 destroyed. + +Outputs: + +droplet_id = "596689967" +droplet_ip = "147.182.228.14" +tailnet_name = "cypher-job1" + +[cypher_up] droplet IP: 147.182.228.14 (id: 596689967) +[cypher_up] waiting for cloud-init to finish (up to 20 min for cold first build)... +[cypher_up] cloud-init complete + +[cypher_up] ready + ssh cypher@147.182.228.14 # direct droplet IP (no reserved IP for named workspaces) + +Next: + scripts/cypher/cypher_run.sh --workspace job1 + scripts/cypher/cypher_down.sh --workspace job1 # full destroy diff --git a/data-raw/logs/study_area_run/20260831_190558_vintage.log b/data-raw/logs/study_area_run/20260831_190558_vintage.log new file mode 100644 index 0000000..dd434bf --- /dev/null +++ b/data-raw/logs/study_area_run/20260831_190558_vintage.log @@ -0,0 +1 @@ +[preflight] primitive vintage (window 200 d) - OK, oldest 0.0 d (whse_fish.pscis_assessment_svw) diff --git a/research/run_record_2026_08_31_cypher_pilots.md b/research/run_record_2026_08_31_cypher_pilots.md new file mode 100644 index 0000000..738af28 --- /dev/null +++ b/research/run_record_2026_08_31_cypher_pilots.md @@ -0,0 +1,100 @@ +# Does the v0.47.0 orchestration work on a real cypher? — 2026-08-31 + +First run record under the shape proposed in +[soul#129](https://github.com/NewGraphEnvironment/soul/issues/129). +Logs: `data-raw/logs/study_area_run/20260831_*`. + +## Question + +link v0.47.0 (#246) added pre-flight gates, a fresh-install path for cyphers, +and post-conditions around consolidate. **None of it had ever run against a +real droplet.** The provincial run it was built for is ~4 h and several hundred +WSG-hours; committing to that on unexercised code was the risk we were trying +to retire. + +Two unknowns, and we could not answer either by reading: + +1. Does the orchestration survive first contact with a cypher? +2. How long does a run actually take post-#223? Every timing on record + predated the 2–3.5× segmentation change, so the estimate spanned 5–9 h with + nothing to narrow it. + +## Method + +Four droplet spins, ~$1.00 total. Two tiny drainage-closed components +(`COWN`, `TSIT`) into a scratch schema `fresh_smoke`, so `fresh` was never +touched. Plus one read-only diagnostic spin that ran nothing but probes. + +## Measurement + +| spin | result | log | +|---|---|---| +| 1 | prep died. Whole diagnostic: `scp: Connection closed` | `20260831_164236_*` | +| diagnostic | `cypher_up` reports ready at **t+156 s**; `cypher@` first works at **t+383 s**. Marker mtime **2026-05-12** while `cloud-init status: running`, VM up 14 s | — | +| 3 | prep **succeeded** (fresh installed from GitHub on a droplet, first time). Two gates then fired wrongly | `20260831_174503_*` | +| 4 | **clean end-to-end, 7.0 min** | `20260831_190558_*` | + +Per-segment modelling rate, measured on identical work: + +| host | min / 1000 persisted segments | +|---|---| +| dispatcher (m1) | **0.0391** | +| cypher | **0.0872** — 2.23× slower | + +Recompute: 0.0112 min/1k segments. Persisted/source segment ratio ≈ 3.5 +(median of six WSGs). + +Provenance, `fresh_smoke.log`, cypher row — the #246 Phase 5 acceptance test: + +``` +TSIT | cypher-job1 | link_sha 0fa2c27d486f | fresh_sha 7f12d99115b7 | fwapg_sha e6e1eb0f4718 +``` + +All four columns non-null on a worker. They were all NULL before this work. + +## Conclusion + +Four defects found, all fixed: + +| defect | where | +|---|---| +| readiness loop exhausted then ran `scp` anyway — no failure branch | link#251 | +| `cypher_up`'s marker is baked into the snapshot, so it never waited | rtj#248 → rtj#250 | +| parity gate's dirty check fires on every run (the run dirties its own repo) | link#252 | +| vintage gate conflated "table absent" with "statistics not yet collected" | link#252 | + +The 2.23× ratio then rewrote the packer (link#253): balancing raw segments +balanced *work* and unbalanced *time*, leaving the dispatcher idle 96 minutes +on a provincial run. Packing by finish time cut the provincial modelling phase +**191 → 150 min** and moved the end-to-end estimate **5.0 h → 4.3 h**. + +Field-scope run (34 WSGs) is now **~1.7 h**, not the 2.6 h projected from May +data and not the 9 h feared at the outset. + +## Dead ends — kept, because they are the evidence + +**The root cause was diagnosed correctly, retracted on a bad inference, then +confirmed by measurement.** The retraction is the instructive part: +`cypher_snapshot.sh:169` scrubs the marker and that line predates the image by +eleven days, so I concluded the scrub had *run*. It had not. Presence taken as +provenance — one level above the bug itself, which was the same error. One +`ls -l --time-style=full-iso` settled it, and should be the first command +against this class of question. + +**Two of the four defects were in gates written to catch exactly that class**, +and both passed tests that could not fail: the dirty fixture set +`dirty = "FALSE"`, and the vintage fixtures never modelled a freshly-restored +database. The rule "a fixture set that cannot reach the failure mode is not +validation" had been written into this branch's own `findings.md` two days +earlier. + +**A cheap probe reported success against a garbage input.** `tofu output -raw` +exits 0 with a "No outputs found" warning while a droplet is still creating, so +a non-empty check passed on multi-line prose and the probe ssh'd to nonsense +for a minute. + +## What it cost + +~$1.00 and about an hour. Each spin found something that would otherwise have +surfaced partway through a 4-hour run — and the first two would have surfaced +as `scp: Connection closed` with no further detail.