Navigation: Docs Hub | Analysis | Setup | Shared Scripts | Reproduce | Pipeline Map | Find Data | Data Products | Dashboard
Compiled and cleaned forest disturbance, climate, inventory, species-niche, and thermophilization datasets for analysis. The repository contains these active production paths:
IDS + climate: clean USDA Forest Service Insect and Disease Survey data, then extract TerraClimate, PRISM, or WorldClim climate values at IDS locations.FIA: compile Forest Inventory and Analysis plot data into documented forest structure, species composition, disturbance, treatment, and site-climate outputs.Species niches: build BIEN range-map climate niches for FIA/P2VEG species.Thermophilization: join species niches to FIA recruitment composition and build community-weighted climate-affinity summaries.Analysis: build stable-condition cumulative mortality, three climate-niche responses, cumulative site CWD, and the current preliminary models. A distinct live+dead severity definition is pending.
Module-level data/ directories keep .gitkeep placeholders where useful, but large raw, intermediate, and generated outputs are kept out of git.
If you are reviewing the repo, start with these pages:
- Searchable Data Catalog to find a product or variable, its row scale, path, and producer.
- Docs Hub for the full navigation map.
- Reproduce for exact run order.
- Pipeline Map for visual orientation.
- Data Products for storage conventions and workflow context.
If you are working locally and want the easiest visual overview, run streamlit run docs/dashboard/app.py. Use Catalog to search products and variables, or Architecture for the workflow map.
If you want a specific workstream right away:
- IDS overview
- TerraClimate overview
- PRISM overview
- WorldClim overview
- FIA overview
- Species niche overview
- Thermophilization overview
| Workstream | Purpose | Start here |
|---|---|---|
01_ids/ |
Download, inspect, clean, and spatially organize IDS damage and survey layers | 01_ids/README.md |
02_terraclimate/ |
Extract monthly TerraClimate values at IDS locations using Google Earth Engine | 02_terraclimate/README.md |
03_prism/ |
Extract monthly PRISM climate values for CONUS IDS observations | 03_prism/README.md |
04_worldclim/ |
Extract monthly WorldClim values from locally downloaded GeoTIFFs | 04_worldclim/README.md |
05_fia/ |
Build FIA plot/condition summaries, species composition tables, disturbance/treatment history, and optional site climate | 05_fia/README.md |
06_species_niches/ |
Build species-level climate niche indicators from BIEN range maps and TerraClimate | 06_species_niches/README.md |
07_thermophilization/ |
Join FIA communities to species niches, calculate plot-year community climate affinity, and estimate repeated-survey change rates | 07_thermophilization/README.md |
08_disturbance_linkage/ |
Prepare reusable FIA, MTBS, IDS, and spatial disturbance evidence; not required by the current mortality models | 08_disturbance_linkage/README.md |
09_analysis/ |
Stable-condition mortality and temperature/precipitation/CWD analytical candidates | 09_analysis/README.md |
Simple pipelines keep one flat numeric sequence. Modules with independent product families use named subdirectories and restart numbering inside each family. QA code always belongs under qa/scripts/; generated diagnostics belong under qa/outputs/ and are not committed.
flowchart LR
A[External data sources] --> B[IDS foundation]
A --> C[FIA pipeline]
B --> D[Climate workflows]
D --> E[Climate outputs]
C --> F[FIA outputs]
F --> G[Species niches]
G --> H[Thermophilization]
E --> I[Dashboard and review outputs]
H --> I
- Run the IDS foundation pipeline.
- Choose one or more climate datasets:
- Build final summaries with the shared script
scripts/build_climate_summaries.R. - Use docs/REPRODUCE.md for the exact command order.
- Run the FIA overview and quick-start.
- Use 05_fia/WORKFLOW.md for per-script technical detail.
- Use docs/DATA_PRODUCTS.md to see which outputs are tracked in git, which are local-only, and which directories are placeholders.
- Build the species universe and BIEN range-map climate niches with 06_species_niches/README.md.
- Build FIA community-weighted climate metrics and repeated-survey change tables with 07_thermophilization/README.md.
- Review the QA summaries in each module before modeling.
| Page | What it is for |
|---|---|
| docs/README.md | Central documentation hub and navigation page |
| docs/REPRODUCE.md | Exact run order for all active production pipelines |
| docs/PIPELINE_MAP.md | GitHub-renderable pipeline diagrams and links |
| docs/DATA_PRODUCTS.md | Output inventory, storage locations, server-aligned skeleton, and producer scripts |
| docs/ARCHITECTURE.md | Shared climate extraction architecture |
| docs/TESTING.md | QC, validation, and coverage gaps |
| docs/fia-explorer.html | Static FIA visual explainer for plot design, sampling grain, and FIADB tables |
| scripts/SETUP.md | Environment setup, dependencies, and dashboard launch |
| scripts/README.md | Shared root scripts, utilities, demos, and tests |
| docs/dashboard/app.py | Local dashboard entrypoint; start on the Architecture page in the sidebar |
Shared helpers live under scripts/. This includes setup, test running, reusable utilities, optional demos, and the shared IDS climate summary builder used by TerraClimate, PRISM, and WorldClim. The Streamlit review app lives under docs/dashboard/.
| Output family | Status | Notes |
|---|---|---|
| IDS cleaned layers | Complete | Produced by 01_ids/; raw regional downloads stay under 01_ids/data/raw/ |
| TerraClimate summaries | Complete | Final per-variable parquets live under processed/climate/terraclimate/ |
| PRISM summaries | Complete | CONUS only |
| WorldClim summaries | Complete | Local GeoTIFF-based workflow |
| FIA plot summaries | Complete | Generated locally by the FIA workflow; large Parquets are gitignored |
| FIA site climate | Partial | The FIA-wide input template is tracked, but its pixel map and climate output are not built; a separate point-climate extraction exists locally |
| Species niches | Active | BIEN range-map niche workflow with QA summaries and documented missing-data handling |
| Thermophilization | Active | Plot-year CWM and repeated-survey change products consume the species niche table |