Skip to content

docs: migration 008 rebuild time on a cold host #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
christabone merged 1 commit into from
Oct 12, 2026
Merged

docs: migration 008 rebuild time on a cold host #5

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions docs/deployment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,10 @@ unprivileged `flybase` user (uid 10001). The image exposes 7200 and its `HEALTHC
`pg_trgm` and trigram indexes, and run `007_grants`, which gives `flybase_mcp` `SELECT` and `EXECUTE` on what
the server reads. `008_located_feature` then builds the window-query table and grants it itself. Each one is
recorded in `mcp.migration_log` with the crate version and release.
Rebuilding the derived tables takes a few minutes on a full load.
Rebuilding the derived tables takes a few minutes on a full load when the data is in cache; on a host that has
just restored the dump (cold page cache, Postgres memory-capped) `008_located_feature` alone scanned
`featureloc` and `feature` for about 40 minutes. Budget up to an hour for `migrate` right after a restore; the
previous server keeps serving while it runs.

Download `precomputed_files/orthologs/dmel_human_orthologs_disease_fb_<release>.tsv.gz` from the release
FTP and run `flybase-mcp migrate --diopt <path>`; without it `get_orthologs` still works (scores from
Expand Down Expand Up @@ -164,7 +167,8 @@ Window queries (`map_features`, `cytosearch`) read `mcp.located_feature`, a deri
`008_located_feature` that holds every non-obsolete feature on a golden-path arm except computed alignments
(type `match`), with its species, arm, type, and coordinates. The migration rebuilds it on every release load,
so `migrate` must run after each restore. On a full release restore it has about 1.7 million rows and takes
about 290 MB with its two indexes; the rebuild takes about 1.5 minutes.
about 290 MB with its two indexes; the rebuild takes about 1.5 minutes with the source tables cached and up to
about 40 minutes cold (it reads the whole of `featureloc` and `feature`).

## Logs

Expand Down
Loading