Skip to content

Brush's logfile option now writes structured, valid CSV logs that cover the whole run, without any third-party library. This PR also adds a guide notebook showing how to load and plot them. - #72

Open
gAldeia wants to merge 8 commits into
masterfrom
feature/logger

Conversation

@gAldeia

@gAldeia gAldeia commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Better run logs

Every call to fit / partial_fit appends to these files. Each row carries a run_id, so several runs can share one logfile.

File Content
<logfile> One row per generation: train/validation loss of the best individual, population median/max of loss, size and complexity, plus new run_id, random_state, best_model (the best expression), best_size, best_complexity, stall_count, archive_size, n_evaluations (cumulative)
<logfile>_islands.csv New. The same statistics for each island, one row per island per generation
<logfile>_simplifications.csv New. Every replacement the simplifiers actually made: generation, individual id, simplifier (constants / inexact), return type, original → replacement, and distance (MSE between the program's predictions before and after; inexact only)
<logfile>_simplification_table The dump of the inexact simplifier's stored expressions, now valid CSV and with a run_id column
<logfile>_runs.jsonl New. One JSON line per run with the seed and the full Parameters

Fixes and improvements

  • The simplification table was not valid CSV. Expressions such as Pow(x0,x1) were written without quotes, so pd.read_csv failed on them. All fields are now quoted when needed.
  • Headers were repeated. Appending another run wrote a new header line into the middle of the file. The header is now written only when the file is new or empty.
  • Rows are flushed after every generation, so the logs are usable during a run and after a crash.
  • Individual ids and generation numbers were stale in the variator. It keeps its own copy of Parameters, and its current_gen stayed at the default (1). Ids repeated across generations as a result. It is now synced every generation. Ids are only used for parent tracking and serialization, so evolution results are unchanged.
  • Logger::verbosity was uninitialized; it now starts at 0. using namespace std is removed from logger.h.

Implementation

  • src/util/csv.h: a small header-only CsvWriter. It quotes fields when needed, writes the header once, flushes every row and writes floats with enough digits to read back exactly. It holds its stream in a shared_ptr, so Engine stays copyable.
  • Engine: the statistics loop from calculate_stats() moved into a new summarize(), which is reused for per-island stats. New open_logs() / log_stats() / close_logs(). run_id is a timestamp plus a random suffix from std::random_device, so logging never touches Brush's random generator.
  • The simplifiers take an optional SimplificationRecords*. Replacements are only recorded when logfile is set.
  • Evaluation::n_evaluations counts calls to assign_fit.

Docs

  • New guide notebook docs/guide/logging.ipynb (added to the guide's table of contents). It generates logs for 3 seeds and shows run metadata, convergence, stall count, archive size, evaluations, how the best expression changed, per-island curves, the final Pareto fronts, and replacements per generation with examples.

Tests

  • tests/cpp/test_logging.cpp (new): quoting, float formatting, header written once, row width checked, error on different columns.
  • tests/cpp/test_brush.cpp: the engine test starts from clean log files and checks the header, one row per generation, and that the other files were created.
  • tests/python/test_sklearn_interface.py: fits twice into the same logfile, parses all five files and checks columns, run_ids, per-island rows and that n_evaluations only increases. A new test checks that two runs with the same random_state log identical best_model columns.
  • Seeded runs give the same best model with and without logfile.

@gAldeia gAldeia mentioned this pull request Sep 23, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant