Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,47 @@ All notable changes to GenAIRR are documented here.

## Unreleased

### Changed

- **`DETERMINISM_VERSION` is now 3.** Clonal lineage population regulation is
Beverton-Holt, `lambda_base / (1 + (lambda_base - 1) * P / n_max)`, in place
of the discrete logistic map `lambda_base * max(0, 1 - P / n_max)` and the
hard cap that truncated any generation exceeding `n_max`. A seed does not
carry across the two: every `clonal_lineage()` and duplex family run moves.
Datasets generated before this are datasets from the old law and carry
`determinism_version` 2 to say so.

The old law was wrong in three ways. Reaching capacity was fatal: the cap
truncated a generation to exactly `n_max`, where `1 - P/n_max` is exactly 0,
so the whole next generation drew zero offspring and the lineage died with
certainty. Measured across 180 runs, 67 reached capacity and all 67 went
extinct in the following generation; over 40 generations at `lambda_base=4.0`
every cap hit was fatal at every capacity tested. With the default
`allow_extinction=False` such a family is silently discarded and re-grown
under a fresh seed, so the delivered data quietly excluded the clones that
expanded most; when every attempt hit the cap the run raised an error whose
advice -- raise `lambda_base` -- made the failure more likely, because a
faster-growing population reaches capacity sooner.

`n_max` was also not the capacity it is named for. The logistic law settles
at `n_max * (1 - 1/lambda_base)`, so a requested 1000 delivered 333 cells at
the default `lambda_base=1.5`. And above `lambda_base=3` the map is past its
period-doubling threshold, so there was no settled population at all: it
cycled between 383 and 875 at 3.5 and moved chaotically between 218 and 942
at 4.0, while the documentation promised a plateau.

Beverton-Holt is monotone in `P`, so it settles at `n_max` for every
`lambda_base` and cannot oscillate. It is exactly 1 at `P = n_max`, so a
population at capacity replaces itself. The hard cap is gone rather than
repaired: a generation may overshoot capacity -- about 6% in practice -- and
the next one is regulated back below replacement. Truncating has to choose
which cells die, and the previous choice let vector position decide, keeping
the broods of early-indexed parents and dropping every later parent's brood
entire. **`n_max` now means what it says, so the same `n_max` yields more
cells than before** -- about 3x at the default `lambda_base=1.5`, with
proportional memory and runtime. Lower `n_max` to recover the old
population: the old plateau was `n_max * (1 - 1/lambda_base)`.

### Added

- `result.to_genotype_tsv(path)` and `CohortResult.to_genotype_tsv(path)`:
Expand Down
3 changes: 2 additions & 1 deletion engine_rs/src/determinism.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,8 @@
//! |---------|-------|
//! | (absent) / 1 | one random stream shared by the whole plan; a substitution could redraw the base already present |
//! | 2 | one random substream per pass, and one per clonal-family role (growth, sampling, target) instead of a seed salted by XOR; a clone, a survival retry and a read take keyed substreams of the run seed instead of offsets added to it; a cohort subject and a genotype viability attempt take keyed substreams instead of words from a `random.Random` seeded with the run seed; a substitution draws a base the site does not hold |
//! | 3 | clonal lineage population regulation is Beverton-Holt, `lambda/(1 + (lambda-1)N/n_max)`, in place of the discrete logistic map `lambda(1 - N/n_max)`; `n_max` is the population a lineage settles at rather than the pivot of a law settling at `n_max(1 - 1/lambda)`, the law is monotone so it no longer oscillates above `lambda = 3`, and a generation may overshoot `n_max` instead of being truncated at it -- the truncation landed the population on `n_max` exactly, where the old law returned a rate of exactly zero, so reaching capacity extinguished the lineage |

/// The engine's sampling-model version. Stamped on every exported
/// record and readable as `GenAIRR.DETERMINISM_VERSION`.
pub const DETERMINISM_VERSION: u32 = 2;
pub const DETERMINISM_VERSION: u32 = 3;
145 changes: 124 additions & 21 deletions engine_rs/src/lineage/branching.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,11 @@ pub struct BranchingParams {
pub lambda_mut: f64,
/// Maximum number of generations to grow.
pub max_generations: u32,
/// Carrying capacity: the live population per generation is capped at this,
/// and it is the pivot for logistic offspring-rate damping (the effective
/// rate falls to zero as the live set approaches `n_max`).
/// Carrying capacity: the live population a lineage settles at. It is the
/// pivot for Beverton-Holt offspring-rate regulation -- the effective rate
/// is `lambda_base` in an empty environment and exactly 1 at `n_max`, so
/// the population levels off there rather than being truncated. A single
/// generation may overshoot it; the next one is regulated back.
pub n_max: u32,
/// Number of cells to sample at the end (used in a later task).
pub n_sample: u32,
Expand Down Expand Up @@ -272,13 +274,31 @@ fn grow_core(
if live.is_empty() {
break; // lineage went extinct (every cell drew 0 offspring)
}
// Logistic carrying-capacity damping: as the live population approaches
// n_max, the effective offspring rate falls toward zero (plateau).
let live_frac = (live.len() as f64) / (params.n_max as f64);
let eff_lambda = params.lambda_base * (1.0 - live_frac).max(0.0);
// Beverton-Holt regulation. The effective offspring rate falls from
// `lambda_base` toward 1 as the live population approaches `n_max`, and
// below 1 above it, so `n_max` is the population a lineage settles at.
//
// The discrete logistic law this replaces, `lambda * (1 - N/n_max)`,
// settled at `n_max * (1 - 1/lambda)` rather than at `n_max`; above
// `lambda = 3` it had no stable population at all and oscillated; and
// paired with a hard cap that landed the population on `n_max` exactly
// it returned an effective rate of exactly zero there, so reaching
// capacity extinguished the lineage with certainty. Beverton-Holt is
// monotone in N, so it cannot oscillate at any lambda, and it is
// exactly 1 at N = n_max: a population at capacity replaces itself.
//
// Sub-replacement growth is excluded rather than damped. It is already
// shrinking, so there is nothing to regulate, and the denominator would
// turn negative past `n_max / (1 - lambda)`.
let eff_lambda = if params.lambda_base <= 1.0 {
params.lambda_base
} else {
let live_frac = (live.len() as f64) / (params.n_max as f64);
params.lambda_base / (1.0 + (params.lambda_base - 1.0) * live_frac)
};

let mut next_live: Vec<u32> = Vec::new();
'generation: for &parent_id in &live {
for &parent_id in &live {
let cell_lambda = match affinity {
Some(m) => eff_lambda * m.fitness_from_affinity(nodes[parent_id as usize].affinity),
None => eff_lambda,
Expand All @@ -289,10 +309,13 @@ fn grow_core(
poisson_sample(&mut rng, cell_lambda)
};
for _ in 0..k {
// Hard cap: a Poisson draw can still overshoot near saturation.
if next_live.len() >= params.n_max as usize {
break 'generation;
}
// No truncation. A generation may overshoot `n_max`; regulation
// returns an effective rate below 1 there and the next
// generation shrinks. Discarding the overshoot instead would
// have to choose which cells die, and the cheap choice -- stop
// at the first parent that fills the quota -- kept the broods of
// early-indexed parents and dropped every later parent's brood
// entire, letting vector position decide survival.
let parent_mut_count = sims[parent_id as usize].mutation_count;
let (child_sim, edge) = match mutator {
Some(m) => {
Expand Down Expand Up @@ -768,7 +791,7 @@ mod tests {
}

#[test]
fn carrying_capacity_bounds_live_population() {
fn regulation_settles_the_population_at_capacity() {
let params = BranchingParams {
lambda_base: 4.0, // would explode unbounded
lambda_mut: 0.0,
Expand All @@ -777,22 +800,102 @@ mod tests {
n_sample: 10,
seed: 7,
};
let (_tree, peak_live, _sims, _live, _edges) =
let (_tree, peak_live, _sims, live, _edges) =
grow_core(&founder(), &params, None, None, None, None).unwrap();
// hard cap: live population never exceeds n_max
// This is the assertion that was missing. For 40 generations the test
// asserted a bound on a lineage that had been extinct since generation
// 11 and called the result a plateau: the hard cap landed the
// population on n_max exactly, where the old logistic law returned an
// effective rate of exactly zero.
assert!(
!live.is_empty(),
"the population was regulated and then died: peak {peak_live}, \
nothing alive after {} generations",
params.max_generations
);
// Beverton-Holt's fixed point is n_max itself. The logistic law it
// replaces settled at n_max * (1 - 1/lambda) = 150 here, so this bound
// also distinguishes the two laws rather than merely bounding growth.
let n = params.n_max as f64;
assert!(
peak_live <= params.n_max as usize,
"peak live {peak_live} exceeded n_max {}",
(live.len() as f64) > 0.8 * n && (live.len() as f64) < 1.2 * n,
"final live {} is not at capacity {}",
live.len(),
params.n_max
);
// damping plateau (equilibrium ~ (1 - 1/lambda_base) * n_max = 150) is
// comfortably above half capacity — confirms it grew, not died early
// A generation may overshoot capacity, by design -- truncating it would
// have to pick which cells die. Regulation returns a rate below 1 above
// n_max, so the overshoot stays small and self-correcting.
assert!(
peak_live > (params.n_max as usize) / 2,
"peak live {peak_live} did not approach capacity"
peak_live < 2 * params.n_max as usize,
"peak live {peak_live} ran away from capacity {}",
params.n_max
);
}

/// The plateau is `n_max` whatever `lambda_base` is. Under the logistic law
/// this replaces it was `n_max * (1 - 1/lambda)` -- 100 at lambda 2, 150 at
/// lambda 4 -- so `n_max` named a capacity the population never reached.
#[test]
fn the_plateau_does_not_depend_on_lambda_base() {
let settled = |lambda_base: f64| {
let params = BranchingParams {
lambda_base,
lambda_mut: 0.0,
max_generations: 40,
n_max: 200,
n_sample: 1,
seed: 7,
};
let (_t, _peak, _s, live, _e) =
grow_core(&founder(), &params, None, None, None, None).unwrap();
live.len()
};
let slow = settled(2.0);
let fast = settled(4.0);
// Guard the premise: two dead lineages would agree at zero.
assert!(slow > 0 && fast > 0, "a lineage died: slow {slow}, fast {fast}");
for (name, n) in [("lambda 2", slow), ("lambda 4", fast)] {
assert!(
n > 160 && n < 240,
"{name} settled at {n}, not at the capacity 200"
);
}
}

/// Reaching the carrying capacity must not be fatal. The parameter's name
/// and the damping comment both promised a population that levels off at
/// `n_max`; the logistic law plus a hard cap delivered a population that
/// landed on `n_max` exactly, drew an effective rate of exactly zero, and
/// died in one generation with certainty -- 67 cap hits, 67 extinctions.
#[test]
fn no_lineage_that_reaches_capacity_dies_from_reaching_it() {
let mut reached = 0;
let mut died = 0;
for seed in 1..40u64 {
let params = BranchingParams {
lambda_base: 4.0,
lambda_mut: 0.0,
max_generations: 40,
n_max: 50,
n_sample: 1,
seed,
};
let (_t, peak, _s, live, _e) =
grow_core(&founder(), &params, None, None, None, None).unwrap();
if peak >= params.n_max as usize {
reached += 1;
if live.is_empty() {
died += 1;
}
}
}
// Guard the premise: seeds that never reach capacity say nothing about
// what reaching it costs.
assert!(reached > 30, "only {reached} of 39 seeds reached capacity");
assert_eq!(died, 0, "{died} of {reached} lineages died after reaching capacity");
}

#[test]
fn final_live_is_empty_when_founder_never_divides() {
// lambda_base = 0 => the founder draws 0 offspring => after generation 1
Expand Down
51 changes: 35 additions & 16 deletions engine_rs/src/lineage/replay_receipt/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,26 @@ fn literal_capacity_receipt(
let driver = FamilyReplayDriver::new(config.clone(), founder, None, &mut budget).unwrap();
// Build static commitments through the real zero-mutation pass. The
// topology and sample arithmetic below are literal independent controls.
//
// The arithmetic is Beverton-Holt at lambda_base 3 and n_max 4:
// generation 1 draws against 3/(1 + 2*(1/4)) = 2.0 from one live cell, and
// generation 2 against 3/(1 + 2*(3/4)) = 1.2 from three. Generation 2's
// nine proposed offspring overshoot the capacity of 4 and every one of them
// is created: regulation answers an overshoot by lowering the next
// generation's rate, not by discarding cells. Under the logistic law this
// replaces the rates were 2.25 and 0.75, and a hard cap discarded parent
// 3's child -- node 8 here -- once the generation had filled n_max.
//
// Node 8 is the cell that cap once suppressed.
let parent_of = |id: u32| -> u32 {
if id <= 3 {
0
} else if id <= 7 {
1
} else {
3
}
};
let context = LineageMutationContext {
refdata: reference,
contracts: None,
Expand All @@ -232,32 +252,32 @@ fn literal_capacity_receipt(
ParentProposal {
generation: 1,
parent_id: 0,
rate: F64Bits::new(2.25).unwrap(),
rate: F64Bits::new(2.0).unwrap(),
proposed_offspring: 3,
},
ParentProposal {
generation: 2,
parent_id: 1,
rate: F64Bits::new(0.75).unwrap(),
rate: F64Bits::new(1.2).unwrap(),
proposed_offspring: 4,
},
ParentProposal {
generation: 2,
parent_id: 2,
rate: F64Bits::new(0.75).unwrap(),
rate: F64Bits::new(1.2).unwrap(),
proposed_offspring: 0,
},
ParentProposal {
generation: 2,
parent_id: 3,
rate: F64Bits::new(0.75).unwrap(),
rate: F64Bits::new(1.2).unwrap(),
proposed_offspring: 1,
},
];
result.edges = (1..=7)
result.edges = (1..=8)
.map(|id| DivisionReceipt {
node_id: id,
parent_id: if id <= 3 { 0 } else { 1 },
parent_id: parent_of(id),
generation: if id <= 3 { 1 } else { 2 },
seed_label: id.to_string(),
trace: trace.clone(),
Expand All @@ -266,7 +286,7 @@ fn literal_capacity_receipt(
events_digest: events.clone(),
})
.collect();
result.final_live = vec![4, 5, 6, 7];
result.final_live = vec![4, 5, 6, 7, 8];
result.samples = vec![
SampleChoice {
live_index: 0,
Expand All @@ -282,16 +302,12 @@ fn literal_capacity_receipt(
},
];
result.completed_generations = 2;
result.peak_live = 4;
result.peak_live = 5;
result.termination = "max_generations".into();
result.nodes = (0..=7)
result.nodes = (0..=8)
.map(|id| NodeSummary {
node_id: id,
parent_id: if id == 0 {
None
} else {
Some(if id <= 3 { 0 } else { 1 })
},
parent_id: if id == 0 { None } else { Some(parent_of(id)) },
generation: if id == 0 {
0
} else if id <= 3 {
Expand All @@ -314,11 +330,14 @@ fn literal_capacity_receipt(
result
}
#[test]
fn literal_capacity_replay_consumes_suppressed_proposal_without_inventing_child() {
fn literal_overshoot_replay_creates_every_proposed_child() {
let (reference, founder, config) = setup();
let receipt = literal_capacity_receipt(&reference, &founder, config);
let (tree, _, _) = run(&reference, &founder, receipt.config.clone(), Some(&receipt)).unwrap();
assert_eq!(tree.nodes.len(), 8);
assert_eq!(tree.nodes.len(), 9);
// Node 8 exists because nothing truncates a generation that overshoots
// capacity. Under the hard cap this proposal was consumed and dropped.
assert_eq!(tree.nodes[8].parent_id, Some(3));
assert_eq!(tree.nodes[4].parent_id, Some(1));
assert_eq!(tree.nodes[4].abundance, 2);
assert_eq!(tree.nodes[7].abundance, 1);
Expand Down
Loading
Loading