Keep the time shape of a column placed after another

CHANGELOG.md

@@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+### Changed
+- **The determinism promise is per version.** The same model, seed, `--as-of` and options still
+  give the same files, byte for byte, on any machine and any day, with the same model2data
+  version. A new release may change what a seed produces, and when it does this changelog says
+  which models are affected and why. Pin the version to keep fixtures the same across upgrades.
+  (The README promised the same bytes across every 1.x release.)
+
+### Fixed
+- **A column placed after another keeps its time shape.** A date or timestamp column with an
+  `after`, or one a created/updated/closed name pair places after another, was set to that
+  column plus a random gap, which threw away the draw its `business_hours` had shaped:
+  appointments `after` a booking with `business_hours: true` came out at 3am as often as at
+  10am. The gap still decides roughly when; `business_hours` (the column's own, or the run's)
+  now decides the weekday and hour. The share of such a column in weekday working hours goes
+  from about 29%, the rate of a uniform draw, to about 72%, the same as a column drawn on its
+  own. The column's growth and seasonality still come from the column it follows.
+- **No pile-up at `--as-of` midnight.** A gap that ran past the end of the window set the value
+  to exactly `--as-of` midnight, so about 1% of a dependent column's rows shared that one
+  timestamp. Such a gap is now drawn again, and after 20 tries the value is drawn evenly
+  between the column it follows and the window's end. A `when` column filled in on day 0 is
+  placed the same way.
+- **Output changes for the same seed** in a model with a column placed after another, wherever
+  either fix applies: a dependent column with business hours (its own or the run's), and every
+  table where a row's gap ran past `--as-of` (about 1 row in 100, so most such tables beyond a
+  few hundred rows). The values drawn after the first such row in a table change too. A table
+  with no column placed after another writes the same bytes as in 1.11.0.
+
 ## [1.11.0] - 2026-10-02
 
 ### Added

README.md

@@ -57,8 +57,9 @@ access required.
 - **Relationship-preserving.** Foreign keys resolve to real parent rows; tables are generated in
   dependency order.
 - **Deterministic, byte for byte.** The same model, `--seed` and `--as-of` (and the same options)
-  give the same files, byte for byte, on any machine and in every 1.x release — safe to commit
-  fixtures, safe to diff across CI runs and upgrades. See [The determinism promise](#the-determinism-promise).
+  give the same files, byte for byte, on any machine and any day with the same model2data
+  version — safe to commit fixtures, safe to diff across CI runs. See
+  [The determinism promise](#the-determinism-promise).
 - **Re-rollable one table at a time.** `--table-seed orders=7` regenerates a single table and
   leaves every other table byte-identical, so you can keep the four tables that look right.
 - **A real dbt project, not just CSVs.** Seeds, staging models that `ref()` them, schema tests,
@@ -323,13 +324,15 @@ The same model, seed and `--as-of` — with the same options (`--rows`, `--rows-
 
 - on any machine and operating system, and on every supported Python version;
 - with any Faker or pandas version model2data accepts;
-- in every 1.x release. A release may make generation faster or add options, but never changes
-  what an existing model, seed and set of options produce. A fix that cannot keep that waits for
-  a new major version, and the changelog says so.
+- with the same model2data version, whenever you run it.
+
+A new release may change what a seed produces, to fix a bug or make the data more realistic.
+When it does, the changelog says which models are affected and why. Pin the version
+(`model2data==1.11.0`) when fixtures must stay the same across upgrades.
 
 `tests/test_determinism.py` holds the file hashes of a spread of models, presets, multi-day runs
 and locales, and runs on every pull request — against the newest Faker and pandas too — so a
-change to a single output byte fails the build rather than reaching a release.
+change to a single output byte fails the build unless it is deliberate and in the changelog.
 
 If one table comes out wrong and the rest looks right, `--table-seed` re-rolls just that table.
 Every other table's seed CSV stays byte-identical, and children of the re-rolled table still

model2data/generate/core.py

@@ -248,7 +248,7 @@ def generate_data_from_dbml(
         # composite-key retry regenerates a temporal column's value into a
         # frame that already respects created/updated/closed ordering, rather
         # than one where only the untouched columns do.
-        df = order_row_times(df, table_def, as_of=as_of)
+        df = order_row_times(df, table_def, as_of=as_of, time_profile=profile)
         df = _resolve_self_referencing_fks(
             df,
             table_def,

model2data/generate/timeline.py

@@ -197,6 +197,60 @@ def weighted_timestamps(row_count: int, profile: TimeProfile, anchor: date) -> l
 _TIMESTAMP_GAP_SCALE_SECONDS = 3 * 86400
 _DATE_GAP_SCALE_DAYS = 3
 
+# How many gaps `place_after` draws for a moment inside the window, in the
+# column's shape, before it settles for one drawn evenly between the bounds.
+_PLACE_TRIES = 20
+
+# A weekend day's weight against a weekday's under `business_hours`, as
+# `_day_weight` weighs it.
+_WEEKEND_WEIGHT = 0.3
+
+
+def gap(kind: str) -> timedelta:
+    """How far after what it follows a dependent column lands: 3 days on average."""
+    if kind == "timestamp":
+        return timedelta(seconds=round(random.expovariate(1 / _TIMESTAMP_GAP_SCALE_SECONDS)))
+    return timedelta(days=max(0, round(random.expovariate(1 / _DATE_GAP_SCALE_DAYS))))
+
+
+def place_after(lower: datetime, kind: str, upper: datetime, business_hours: bool) -> datetime:
+    """A moment a random gap after `lower`, in the column's shape, before `upper`.
+
+    The gap decides roughly when; `business_hours` then decides the weekday
+    and the hour, as it does for a column drawn on its own: a weekend day is
+    kept 3 times in 10, and a timestamp's time of day is redrawn from
+    `HOUR_WEIGHTS` on the gap's day (a time before `lower` is drawn again).
+    A timestamp lands before `upper`, a date on or before it. A draw past
+    `upper` is drawn again, so a short window does not pile rows up on its
+    last moment; after `_PLACE_TRIES` the moment is drawn evenly between the
+    bounds, and a window with no room at all returns `lower`: the column
+    never lands before what it follows.
+    """
+
+    def inside(moment: datetime) -> bool:
+        return moment < upper if kind == "timestamp" else moment.date() <= upper.date()
+
+    for _ in range(_PLACE_TRIES):
+        moment = lower + gap(kind)
+        if business_hours:
+            if moment.weekday() >= 5 and random.random() >= _WEEKEND_WEIGHT:
+                continue
+            if kind == "timestamp":
+                hour = random.choices(range(24), weights=HOUR_WEIGHTS)[0]
+                moment = datetime(moment.year, moment.month, moment.day, hour) + timedelta(
+                    seconds=random.randrange(3600)
+                )
+                if moment < lower:
+                    continue
+        if inside(moment):
+            return moment
+    if kind == "timestamp":
+        span = int((upper - lower).total_seconds())
+        return lower + timedelta(seconds=random.randrange(span)) if span > 0 else lower
+    span = (upper.date() - lower.date()).days
+    return lower + timedelta(days=random.randint(0, span)) if span > 0 else lower
+
+
 # Column-name stems that place a column at a stage in the created -> updated ->
 # closed chain. A column matches a stage by tokenizing its name on non-
 # alphanumeric characters and looking for the stem as a contiguous run of
@@ -330,6 +384,14 @@ def _format_value(moment: datetime, kind: str) -> Union[str, date]:
     return moment.date()
 
 
+def column_business_hours(column: ColumnDef, time_profile: Optional[TimeProfile]) -> bool:
+    """Whether `column` is drawn in business hours: its own hint, else the run's."""
+    note = column.note or {}
+    if "business_hours" in note:
+        return bool(note["business_hours"])
+    return bool(time_profile and time_profile.business_hours)
+
+
 def _build_dependencies(
     table_def: TableDef, columns_by_name: dict, temporal_columns: list[ColumnDef]
 ) -> dict[str, set[str]]:
@@ -413,15 +475,20 @@ def _topological_order(table_def: TableDef, deps: dict[str, set[str]]) -> list[s
     return order
 
 
-def order_row_times(df, table_def: TableDef, as_of: AsOf = None):
+def order_row_times(
+    df, table_def: TableDef, as_of: AsOf = None, time_profile: Optional[TimeProfile] = None
+):
     """Fix up temporal columns so an earlier-stage column never lands later.
 
     Runs unconditionally (every profile, not just non-uniform ones): a row
     updated before it was created is wrong regardless of how the timestamps
     were drawn. A column with dependencies is set, per row, to the latest of
-    those dependencies plus a random gap (capped at the window's end), unless
-    either side is null for that row -- a nullable column that happened to
-    come back null keeps its null, rather than this pass forcing a value in.
+    those dependencies plus a random gap, inside the window and in the
+    column's business-hours shape (see `place_after`; `time_profile` is the
+    run's, a column's own `business_hours` hint replacing it), unless either
+    side is null for that row -- a nullable column that happened to come back
+    null keeps its null, rather than this pass forcing a value in. The run's
+    growth and seasonality reach the column through the one it follows.
 
     Leaves `df` untouched when the table holds no `after` hint and no
     recognisable created/updated/closed-style name pair, and never modifies a
@@ -456,7 +523,7 @@ def order_row_times(df, table_def: TableDef, as_of: AsOf = None):
             other_kind = _column_kind(columns_by_name[name].data_type)
             assert other_kind is not None
             required_kinds[name] = other_kind
-        scale = _TIMESTAMP_GAP_SCALE_SECONDS if kind == "timestamp" else _DATE_GAP_SCALE_DAYS
+        business_hours = column_business_hours(columns_by_name[column_name], time_profile)
 
         for idx in df.index:
             if _parse_value(df.at[idx, column_name], kind) is None:
@@ -475,14 +542,7 @@ def order_row_times(df, table_def: TableDef, as_of: AsOf = None):
                 continue
             latest_required = max(earlier)
 
-            gap = random.expovariate(1 / scale)
-            if kind == "timestamp":
-                moment = latest_required + timedelta(seconds=round(gap))
-            else:
-                moment = latest_required + timedelta(days=max(0, round(gap)))
-            if moment > cap:
-                moment = cap
-
+            moment = place_after(latest_required, kind, cap, business_hours)
             df.at[idx, column_name] = _format_value(moment, kind)
 
     return df

model2data/generate/when.py

@@ -45,12 +45,12 @@
 from model2data.generate.faker import AsOf, generate_column_values
 from model2data.generate.options import TimeProfile
 from model2data.generate.timeline import (
-    _DATE_GAP_SCALE_DAYS,
-    _TIMESTAMP_GAP_SCALE_SECONDS,
     _build_dependencies,
     _format_value,
     _parse_value,
     _topological_order,
+    column_business_hours,
+    place_after,
 )
 from model2data.parse.dbml import ColumnDef, TableDef
 
@@ -123,7 +123,13 @@ def _temporal_kind(column: ColumnDef) -> Optional[str]:
 class _Order:
     """What a filled date or timestamp of one table must sit between, row by row."""
 
-    def __init__(self, table_def: TableDef, frame: pd.DataFrame):
+    def __init__(
+        self,
+        table_def: TableDef,
+        frame: pd.DataFrame,
+        time_profile: Optional[TimeProfile] = None,
+    ):
+        self.time_profile = time_profile
         self.by_name = {column.name: column for column in table_def.columns}
         temporal = [
             column
@@ -154,6 +160,11 @@ def before(self, name: str, row: int) -> Optional[datetime]:
         found = self._moments(followers, row)
         return min(found) if found else None
 
+    def place(self, name: str, lower: datetime, kind: str, upper: datetime) -> datetime:
+        """A moment for `name` a random gap after `lower`, in the column's shape."""
+        business_hours = column_business_hours(self.by_name[name], self.time_profile)
+        return place_after(lower, kind, upper, business_hours)
+
     def settle(self, name: str, row: int, cap: datetime) -> None:
         """Move what follows `name` in this row to no earlier than what it follows, in order.
 
@@ -172,17 +183,10 @@ def settle(self, name: str, row: int, cap: datetime) -> None:
             lower = self.after(other, row)
             if value is None or lower is None or value >= lower:
                 continue
-            self.frame.iat[row, column] = _format_value(min(lower + _gap(kind), cap), kind)
+            self.frame.iat[row, column] = _format_value(self.place(other, lower, kind, cap), kind)
             moved.add(other)
 
 
-def _gap(kind: str) -> timedelta:
-    """How far after what it follows a filled value lands: the ordering pass's own gap."""
-    if kind == "timestamp":
-        return timedelta(seconds=round(random.expovariate(1 / _TIMESTAMP_GAP_SCALE_SECONDS)))
-    return timedelta(days=max(0, round(random.expovariate(1 / _DATE_GAP_SCALE_DAYS))))
-
-
 def _draw_like(
     column: ColumnDef,
     held: list,
@@ -223,7 +227,7 @@ def apply_when(
     columns = when_columns(table_def)
     if not columns:
         return
-    order = _Order(table_def, frame)
+    order = _Order(table_def, frame, time_profile)
     for column in columns:
         name = column.name
         match = matching(frame, order.by_name, conditions(column) or {})
@@ -274,7 +278,7 @@ def _moment(
         span = max(0, int((upper - start).total_seconds()))
         moment = start + timedelta(seconds=random.randint(0, span))
     elif lower is not None:
-        moment = lower + _gap(kind)
+        moment = order.place(name, lower, kind, upper)
     else:
         held = [
             parsed

tests/fixtures/determinism.json

@@ -729,7 +729,7 @@
   "dbt_det/seeds/raw/feature_limits.csv": "c991751064bf749aa1277c53eca6ba4e5941566f6ab1b083d3e49d14eb4ffab2",
   "dbt_det/seeds/raw/invoices.csv": "19489a185cd13149b16f76e62e6b508de437d67648c30f7e6d753bd305ec6268",
   "dbt_det/seeds/raw/organizations.csv": "bb9019c41083da011f0512d4a5ea4225fa766a8c9d3feaabdfb6713d19a412e0",
-  "dbt_det/seeds/raw/subscriptions.csv": "de43c0cacdd8589a8c26c5a65b8b3efbc6e12874ad7ed19bf4d4bbcf0880dcf3",
+  "dbt_det/seeds/raw/subscriptions.csv": "02bb173269e98ca870de3cfc875f23b54630cc11a6162d35d0640ae4193d0296",
   "dbt_det/seeds/raw/users.csv": "d55263c0ea25908f11d2a300106749af4022e2f05b0a7e452ef171f2e707ea46"
  },
  "table-seed": {

tests/test_determinism.py

@@ -1,15 +1,16 @@
-"""The same model, seed and `--as-of` write byte-identical files in every 1.x release.
+"""The same model, seed and `--as-of` write byte-identical files on the same version.
 
 That is the product's promise: a committed dataset, a diffed CI run and a teaching
 exercise all lean on it. This module enforces it. Each case runs the real CLI
 in-process, hashes every file it writes, and compares against the hashes in
-`fixtures/determinism.json`, which were recorded from a published release.
+`fixtures/determinism.json`.
 
-If a case fails, output changed for the same seed. That is a breaking change and
-needs a major version; it is not a snapshot to refresh. Faker or pandas releases
-that change the data show up here too (CI runs this file against the newest of both).
+If a case fails, output changed for the same seed. That is not a snapshot to
+refresh: either the change is unintended (find it), or it is deliberate and the
+changelog says which models it affects and why. Faker or pandas releases that
+change the data show up here too (CI runs this file against the newest of both).
 
-Re-recording is for adding or deliberately versioned changes only:
+Re-recording is for adding cases or deliberate, changelogged changes only:
 
     MODEL2DATA_RECORD_DETERMINISM=1 uv run pytest tests/test_determinism.py
 
@@ -123,9 +124,9 @@ def test_same_seed_same_bytes(case_id, tmp_path, recorded):
         f"DETERMINISM BROKEN in case {case_id!r}: the same model, seed and --as-of no "
         "longer write the same files.\n"
         f"  changed files: {changed}\n  new files: {added}\n  missing files: {missing}\n"
-        "Changing output for the same seed breaks model2data's promise of byte-identical "
-        "output across 1.x releases and needs a MAJOR version. Do not just re-record the "
-        "hashes. If the change comes from a Faker or pandas release, pin or work around it."
+        "Do not just re-record the hashes: re-record only for a deliberate change the "
+        "CHANGELOG describes (which models, and why). If the change comes from a Faker or "
+        "pandas release, pin or work around it."
     )