Skip to content

fix(compose): bound the bundled MySQL binary log - #1196

Open
linus-liu-web wants to merge 1 commit into
Tencent:mainfrom
linus-liu-web:fix/compose-mysql-binlog-bounds
Open

linus-liu-web wants to merge 1 commit into
Tencent:mainfrom
linus-liu-web:fix/compose-mysql-binlog-bounds

Conversation

@linus-liu-web

Copy link
Copy Markdown

What changed

The default Compose stack starts mysql:8.4 with binary logging on and no retention. Every checkpoint, telemetry and log row the platform writes is also written to the binary log with no upper bound, so a write storm fills loopforge_mysql-data with binlogs while the database itself stays small. #1138 reports 298 GB of binlogs in four days with the largest table at 0.16 GB; a Compose deployment here accumulated 1.18 GB in about a day.

Bound the log instead of leaving it unbounded:

  • --binlog-expire-logs-seconds=86400 keeps one day of binary logs;
  • --max-binlog-size=67108864 rotates every 64 MiB.

Deployments that need point-in-time recovery or replication can raise those values or remove the flags; single-node evaluation installations that need no recovery can pass --skip-log-bin. The deployment guide documents both choices and the PURGE BINARY LOGS / SET PERSIST binlog_expire_logs_seconds commands for installations that already accumulated logs.

Related: #1138 (binary log growth from the workspace checkpoint loop; the engine-side retry/lease fix is #1148 and explicitly leaves this MySQL/Compose configuration aside). This PR only bounds the default; it does not claim to fix the loop itself.

Scope

  • Public adapter / deployment
  • Documentation only
  • Platform core / API / runner
  • Classic
  • Portable
  • Shared behavior contract
  • Installer / CLI

Compatibility

No application behavior change: the database schema, DSN and data are untouched. Existing volumes keep working; MySQL applies the new retention/size limits on restart. Operators who rely on binary logs for point-in-time recovery or replication must raise the values or remove the flags, as documented. MySQL 8.4 has binary logging enabled by default, so the flags only bound it.

Validation

  • docker compose --env-file <existing .env> -f compose.yaml --profile platform config renders the new MySQL command unchanged.
  • Started a throwaway mysql:8.4 with the exact new flags and confirmed SHOW VARIABLES: binlog_expire_logs_seconds=86400, max_binlog_size=67108864, log_bin=ON. Container removed afterwards.
  • Not run: an upgrade of a live stack with an existing mysql-data volume (the flags are read at startup and do not require reinitialization).

Safety

  • I did not include credentials, internal endpoints, production data, or organization-specific infrastructure.
  • I preserved user files and unrelated changes.
  • I updated generated Classic host bundles when applicable. (n/a — no Classic change)
  • I documented third-party content and licensing changes when applicable. (n/a)

The default Compose stack starts MySQL with binary logging on and no
retention, so every checkpoint, telemetry and log row is also written to the
binlog without an upper bound. A write storm then fills loopforge_mysql-data
with binary logs while the database itself stays small: Tencent#1138 reported 298 GB
of binlogs in four days while the largest table was 0.16 GB, and a Compose
deployment here accumulated 1.18 GB of binlogs in about a day.

Bound the log instead of leaving it unbounded:
- --binlog-expire-logs-seconds=86400 keeps one day of binary logs;
- --max-binlog-size=67108864 rotates every 64 MiB.

Deployments that need point-in-time recovery or replication can raise or drop
the two flags; single-node evaluation stacks can pass --skip-log-bin. Document
both, plus the PURGE BINARY LOGS command for installations that already
accumulated logs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant