Skip to content

feat: implement automated reporting and visualization layer - #4040

Open
ayushman1210 wants to merge 18 commits into
PecanProject:developfrom
ayushman1210:automated-reporting-and-visualization
Open

feat: implement automated reporting and visualization layer#4040
ayushman1210 wants to merge 18 commits into
PecanProject:developfrom
ayushman1210:automated-reporting-and-visualization

Conversation

@ayushman1210

Copy link
Copy Markdown
Contributor

Description

Here's a breakdown of what I worked on:

  • Updating the plotting functions: I refactored the old metric_timeseries_plot.R, metric_scatter_plot.R, and metric_residual_plot.R functions so they can easily consume the data.frame or data.table outputs straight from the run_benchmark() pipeline. I also took the opportunity to clean up the ggplot2 aesthetics (better themes, removing hardcoded date limitations, adding reference lines, etc.).
  • Quarto report template: I created a new Validation_report.qmd template in inst/reports/. This generates a nice HTML scorecard (ILAMB-style) that compiles all our computed metrics alongside the plots.
  • Reporting wrapper: I added generate_validation_report.R, which acts as a wrapper to render the Quarto template automatically at the end of a benchmark run.
  • Testing: Added some basic tests in test-visualization.R to make sure the plots render correctly and the wrapper function handles missing templates gracefully.

Types of changes

  • New feature (non-breaking change which adds functionality)

Fixes #4037

@ayushman1210
ayushman1210 requested a review from infotroph June 30, 2026 15:54

@divine7022 divine7022 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

solid work! thanks for putting this together
the bigger picture, ryt now this is plotting layer. and 4037 is validation report, and the job of a validation report is to answer two questions "is the model good enough ?" and "where does it fail ?". a stack of pretty panels doesn't answer either. the upgrades I am trying to asking are what turn this from "here are some plots" into "here's whether the model passes, and where it doesn't" none of it is hard and you have already got the pieces in.

and consolidate one off plot_time_series into these, it is not touched here. either fold it in or spin quick follow up

Comment thread modules/benchmark/R/metric_timeseries_plot.R Outdated
Comment thread modules/benchmark/tests/testthat/test-visualization.R
Comment thread modules/benchmark/R/metric_timeseries_plot.R Outdated
Comment thread modules/benchmark/inst/reports/Validation_report.qmd
Comment thread modules/benchmark/R/metric_residual_plot.R
Comment thread modules/benchmark/R/metric_timeseries_plot.R
Comment thread modules/benchmark/R/generate_validation_report.R Outdated
Comment thread modules/benchmark/R/generate_validation_report.R Outdated
Comment thread modules/benchmark/R/generate_validation_report.R Outdated
Comment thread modules/benchmark/R/metric_residual_plot.R Outdated
ayushman1210 and others added 4 commits July 14, 2026 17:16
- Add per-site faceting and scorecards to timeseries and residual plots
- Standardize plot styles (lowercase axes, dynamic units, 12pt base_size)
- Implement linear trendlines for residual plots
- Incorporate obvs_sd into overlap/coverage logic
- Remove redundant stop() calls and ensure terminating newlines
@ayushman1210
ayushman1210 requested a review from divine7022 July 21, 2026 13:02

@divine7022 divine7022 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the updates!,
please check all the ci failing, same as I dropped in #4041
update the changes suggested in slack discussion here or may be as followup

Comment thread modules/benchmark/R/metric_timeseries_plot.R Outdated
Comment thread modules/benchmark/R/metric_residual_plot.R
… lm fit in residual plot, and standardize 90% interval
@ayushman1210
ayushman1210 requested a review from divine7022 July 24, 2026 04:40
@dlebauer

Copy link
Copy Markdown
Member

this may not be caused by this PR, but according to CI error, may need to add to .github/workflows/ci.yml

- name: Install system dependencies
  run: |
    sudo apt-get update
    sudo apt-get install -y libsodium-dev pkg-config

but please confirm that this is the correct solution

@divine7022 divine7022 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for solid work on this, pretty close.
droped couple of inline
then it's good to merge

Comment thread modules/benchmark/R/metric_Coverage.R Outdated
Comment thread modules/benchmark/R/metric_timeseries_plot.R Outdated
Comment thread modules/benchmark/R/run_benchmark.R Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Review

Development

Successfully merging this pull request may close these issues.

Phase 3: Automated reporting & visualization

3 participants