Ferrybox R scripts

The following scripts extract ferrybox measurements (https://thredds.niva.no/thredds/dodsC/datasets/nrt/color_fantasy.nc) and river logger measurements (https://thredds.niva.no/thredds/dodsC/datasets/loggers/glomma/baterod.nc) from NIVA thredds for marine and freshwater parameters.

All available scripts are located in the "all_functions" folder. First run the script "netcdf_extract_fb_data.R" to exctract ferrybox measurements. Available paramaters are [salinity, chlorophyll, turbidity, fdom, temperature, oxygen_sat]. The ferrybox measurements are collected in a transect from Oslo to Kiel (lat 60N to 54N), and the function can extract all data in this transect or filtered for specific latitudes. Data is collected from 2023 and forward, specific dates can also be extracted.

Second run the script "netcdf_logger_extract.R" to extract river logger data from Baterod river located inside Glomma in the south-eastern part of Oslofjorden. Available parameters are [temp_water_avg, phvalue_avg, condvalue_avg, turbidity_avg, cdomdigitalfinal]. Data is collected from 2023 and forward, specific dates can also be extracted.

From here you can run the "netcdf_assessment_area.R", where you can provide an assessment waterbody area that you want to work with. Example from https://karteksport.miljodirektoratet.no/ you can extract "Vannforekomster". You must specify a layer input if the file is geojson and contains multiple geometry layers. The ferrybox and river files extracted previously should also be added to see if the data fits. Similar dataset could be replaced and the assessment area would still be shown. NOTICE: Bounding box is fixed around first input data (i.e. ferrybox data).

The "netcdf_join_dataframes.R" joins the ferrybox and logger csv data frame (or other if provided) by a specific parameter from the joined dataframes and calculates daily mean values.

"netcdf_scatter_datax_vs_datay.R" creates a scatterplot from the joined dataframes from the parameters specfied and "netcdf_scatter_station_plot.R" creates scatterplot from a single dataframes from two specified parameters (could be ferrybox salinity vs chlorophyll). "netcdf_tile_plot.R" creates a Hovmöller style plot of ferrybox measurements. The plot illustrates the latitude and y-axis, data at x-axis and plot fill is the measurement value of specified parameter.

GALAXY

The workflow is published on the Galaxy platform and a data-to-knowledge package explaining how the workflow runs is available at https://zenodo.org/records/22868890.

Docker

The environment for the R scripts can also be created using docker.

This repo is intended for:

• local execution via Docker
• browser-based interactive use via MyBinder
• workflow use via Galaxy

today=$(date '+%Y%m%d')
docker build . -t ferry-rscripts:${today}

# Better:
# This includes the git commit hash, so please
# make sure all your changes are committed/stashed:
githash=$(git rev-parse --short HEAD)
docker build \
  --build-arg GIT_COMMIT=${githash} \
  -t ferry-rscripts:${today}-${githash} .

To run an interactive session to execute several scripts:

docker run -it --entrypoint /bin/bash ferry-rscripts${today}

To run a single script, with input parameters:

(When removing the trailing comments, make sure to remove all trailing whitespace, so that the backslash is the last character on the line. Otherwise subsequent lines will not be passed on to the docker command.)

# Example: netcdf_extract_fb_data.R
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_extract_fb_data.R' \
  ferry-rscripts:${today} \
  'https://thredds.niva.no/thredds/dodsC/datasets/nrt/color_fantasy.nc' \
  '/out/myferryboxtest.csv' \
  'temperature,salinity,chlorophyll,turbidity' \
  '2023-01-01' \
  '2023-12-31' \
  'null' 'null' 'null' 'null'

For example commands for all contained scripts, and an explanation of their input parameters, please see below.

netcdf_extract_fb_data.R

# netcdf_extract_fb_data.R
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_extract_fb_data.R' \
  ferry-rscripts:${today} \
  'https://thredds.niva.no/thredds/dodsC/datasets/nrt/color_fantasy.nc' \
  '/out/myferryboxtest.csv' \
  'temperature,salinity,chlorophyll,turbidity' \
  '2023-01-01' \
  '2023-12-31' \
  'null' 'null' 'null' 'null'

# Explanation of the parameters:
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_extract_fb_data.R' \
  ferry-rscripts:${today} \

  # Thredds link to FerryBox data
  'https://thredds.niva.no/thredds/dodsC/datasets/nrt/color_fantasy.nc' \

  # Output CSV (if only a directory is given, defaults to ferrybox.csv)
  '/out/myferryboxtest.csv' \

  # Parameters (NULL = ALL)
  'temperature,salinity,chlorophyll,turbidity' \

  # Start date
  '2023-01-01' \

  # End date
  '2023-12-31' \

  # Bounding box (minLon maxLon minLat maxLat) or 'null' 'null' 'null' 'null'
  'null' 'null' 'null' 'null'

netcdf_logger_extract.R

# netcdf_logger_extract.R
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_logger_extract.R' \
  ferry-rscripts:${today} \
  'https://thredds.niva.no/thredds/dodsC/datasets/loggers/glomma/baterod.nc' \
  '/out/data/myloggertest.csv' \
  'NULL' \
  '2023-01-01' \
  '2023-12-31'

# Explanation of the parameters:
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_logger_extract.R' \
  ferry-rscripts:${today} \

  #
  'https://thredds.niva.no/thredds/dodsC/datasets/loggers/glomma/baterod.nc' \

  # Output CSV (if only a directory is given, defaults to logger.csv)
  '/out/data/myloggertest.csv' \

  # Parameters (NULL = ALL)
  'NULL' \

  # Start date
  '2023-01-01' \

  # End date
  '2023-12-31'

netcdf_assessment_area.R

# netcdf_assessment_area.R
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_assessment_area.R' \
  ferry-rscripts:${today} \
  '/out/data/myferryboxtest.csv' \
  '/out/plots/mypositionplottest.png' \
  '/out/data/myloggertest.csv' \
  'NULL'

# Explanation of the parameters:
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_assessment_area.R' \
  ferry-rscripts:${today} \

  # Input FerryBox CSV
  '/out/data/myferryboxtest.csv' \

  # Output plot (if only a directory is given, defaults to assessment_area.png)
  '/out/plots/mypositionplottest.png' \

  # Input river/logger CSV
  '/out/data/myloggertest.csv' \

  # Waterbodies shapefile (NULL if none)
  'NULL'

netcdf_scatter_station_plot.R

# netcdf_scatter_station_plot.R
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_scatter_station_plot.R' \
  ferry-rscripts:${today} \
  '/out/data/myferryboxtest.csv' \
  '/out/plots/myscatterplottest.png' \
  'chlorophyll' \
  'salinity'

# Explanation of the parameters:
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_scatter_station_plot.R' \
  ferry-rscripts:${today} \

  # Input FerryBox CSV
  '/out/data/myferryboxtest.csv' \

  # Output plot (if only a directory is given, defaults to ferrybox_scatter.png)
  '/out/plots/myscatterplottest.png' \

  # Parameter X
  'chlorophyll' \

  # Parameter Y
  'salinity'

netcdf_join_dataframes.R

# netcdf_join_dataframes.R
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_join_dataframes.R' \
  ferry-rscripts:${today} \
  '/out/data/myferryboxtest.csv' \
  '/out/data/myloggertest.csv' \
  'turbidity' \
  'turbidity_avg' \
  'station_name' \
  'Baterod' \
  'datetime' \
  'datetime' \
  '/out/data/myjoinedtest.csv'

# Explanation of the parameters:
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_join_dataframes.R' \
  ferry-rscripts:${today} \

  # FerryBox dataframe CSV
  '/out/data/myferryboxtest.csv' \

  # River/logger dataframe CSV
  '/out/data/myloggertest.csv' \

  # Parameter from first dataframe
  'turbidity' \

  # Parameter from second dataframe
  'turbidity_avg' \

  # Station column name in second dataframe
  'station_name' \

  # Station ID/name to filter in second dataframe
  'Baterod' \

  # Time column in first dataframe
  'datetime' \

  # Time column in second dataframe
  'datetime' \

  # Output joined CSV (if only a directory is given, defaults to joined.csv)
  '/out/data/myjoinedtest.csv'

netcdf_scatter_datax_vs_datay.R

# Case 1: Filter by latitude range (simplest test)
docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_scatter_datax_vs_datay.R' \
  ferry-rscripts:local \
  'https://aquainfra.ogc.igb-berlin.de/exampledata/niva/netcdf_join_dataframes/joined.csv' \
  '/out/scatter_lat.png' \
  'null' \
  'null' \
  'null' \
  '59.1' \
  '59.3' \
  'null'

# Case 2: Filter by waterbody
docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_scatter_datax_vs_datay.R' \
  ferry-rscripts:local \
  'https://aquainfra.ogc.igb-berlin.de/exampledata/niva/netcdf_join_dataframes/joined.csv' \
  '/out/scatter_waterbody.png' \
  'https://github.com/NIVANorge/niva-aquainfra/raw/refs/heads/main/Ferrybox%20scripts/test_data/Vannforekomster_202604091250.zip' \
  'Torbjørnskjær,Færder' \
  'navn' \
  'null' \
  'null' \
  'VannforekomstKyst'

netcdf_tile_plot.R

# netcdf_tile_plot.R
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_tile_plot.R' \
  ferry-rscripts:${today} \
  '/out/data/myferryboxtest.csv' \
  '/out/plots/mytileplottest.png' \
  '2023-01-01' \
  '2023-12-31' \
  'salinity,chlorophyll' \
  'null' 'null' \
  '2023-08-08'

# Explanation of the parameters:
date; docker run \
  -v './testresults:/out:rw' \
  -e 'SCRIPT=netcdf_tile_plot.R' \
  ferry-rscripts:${today} \

  # Input FerryBox CSV
  '/out/data/myferryboxtest.csv' \

  # Output plot (if only a directory is given, defaults to ferrybox_tile.png)
  '/out/plots/mytileplottest.png' \

  # Start date
  '2023-01-01' \

  # End date
  '2023-12-31' \

  # Parameters (comma-separated)
  'salinity,chlorophyll' \

  # Latitude filter (minLat maxLat) or 'null' 'null'
  'null' 'null' \

  # Storm date (or 'null')
  '2023-08-08'

MyBinder / Virtual lab

This repository can also be launched as a browser-based virtual lab with MyBinder. This is useful when you want to explore the R scripts interactively without installing R and the required dependencies locally.

To use it:

1. Open the Binder URL for this repository: https://mybinder.org/v2/gh/NIVANorge/Ferrybox-galaxy-scripts/main
2. Wait for the environment to build.
3. Start RStudio or the relevant interactive tooling from the launched environment.

The MyBinder setup is intended for interactive analysis and exploration, while the Docker instructions above are intended for users who want to run the scripts locally in a container.

Pygeoapi / OGC HTTP API

The Pygeoapi / OGC API service layer is maintained in the companion repository for the deployed web service setup:

https://github.com/NIVANorge/niva-aquainfra

This repository focuses on the Ferrybox R scripts, local execution, and interactive access via Docker and MyBinder. The web-service deployment and JSON metadata files used for OGC API process endpoints are kept in the companion repository where the service is actively deployed and maintained.