Cross-table after (spec 0.5.0), a warning that suggests it, and integer keys 1..N, and release 1.16.0

CHANGELOG.md

@@ -7,6 +7,89 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+## [1.16.0] - 2026-10-13
+
+The v1.15.0 GitHub release never reached PyPI (its package still said 1.14.0), so its fixes
+ship here, together with the changes below.
+
+### Added
+- **`after` can name a parent's column.** `after: customers.created_at` on `orders.order_date`
+  keeps every order on or after the `created_at` of the customer its foreign key points at (in
+  `examples/ecommerce.model2data.yml`, half the orders used to predate their customer). `after`
+  takes a column of the same row as before, a parent's column as `<table>.<column>` (the last
+  `.` separates the column, so `raw.customers.created_at` works), or a list of either, all of
+  which the value is not before: `after: [orders.ordered_at, products.launched_at]`, or
+  `after: [placed_at, customers.created_at]`. A date compared with a timestamp compares by day.
+  - **Reaching the parent:** through exactly one one-column foreign key onto the parent's
+    primary key or a unique column. A null foreign key constrains nothing on its row. Errors,
+    in the style of the others, for an unknown table or column, a column that is no date, a
+    table the row has no foreign key to (or only a composite one, or one onto a column that is
+    no key), the column's own table (a column of the row is named without its table; a
+    self-reference is not followed), more than one foreign key to the parent (`messages`
+    reaching `users` through `sender_id` and `receiver_id` is an error naming both; picking one
+    of several foreign keys to a parent is not built yet), two
+    tables whose `after`s name each other's columns, and a parent generated after the child
+    (a cycle of foreign keys broken at the very key followed).
+  - **The child keeps its own shape.** A row whose date falls before its parent's takes another
+    parent, one created by then, drawn from the parents the column already points at; its date
+    keeps its draw, so growth, seasonality, business hours and the `as_of` window hold exactly.
+    Only a row with no parent created early enough (or a one-to-one, which cannot change
+    parent) moves its date instead, drawn again from the column's own shape cut to the part of
+    the window on or after the parent's; a parent created on the window's last day gives a child
+    on that day, never past `as_of`. Parents created early collect more children, so a skewed
+    foreign key is steeper: on a coffee-webshop model with `skew: 0.8`, customers growing over
+    the year and `after: customers.created_at` on the orders, the top fifth of customers hold
+    about 80% of the orders rather than 65%, while the orders' gift-season peak and growth are
+    unchanged. A composite-key repair or a `when` fill that dates a row before its parent
+    afterwards moves only that row's date, and its later dates follow it again.
+  - **Days after the first:** a new row whose parent was inserted the same day lands later that
+    day than its parent.
+  - **Spec 0.5.0.** A list or a parent's column in `after` is new vocabulary: a document using
+    either says `model2data: 0.5.0`, and a 0.2, 0.3 or 0.4 document using it gets an error
+    naming the version to write (and otherwise reads exactly as before). DBML with such a note
+    converts to 0.5.0. The schema's `$id` moves to `/spec/0.5.0/`.
+  - **dbt: `model2data_not_before_parent`.** With hint tests on (`--hint-tests`), each parent
+    column an `after` names writes a test that joins the child to the parent model through the
+    foreign key and returns the rows dated before their parent row (by day for a date against a
+    timestamp; a key several parent rows share counts its earliest, leaving duplicates to the
+    `unique` test). The macro ships in `macros/model2data_parent_tests.sql`, written only when a
+    project has such a test. `hint_tests_for(tables, refs=...)` takes the references for it when
+    given bare tables. The defects report checks it like every other test, so a defect that
+    dates a row before its parent lists it in `EXPECTED_FAILURES.md`; `late_arriving` takes
+    rows that stay clear of their parents first, so it still breaks no test where it can.
+- **A warning where a child's date can fall before its parent's.** When a table reaches a parent
+  through one foreign key, the parent has a date that reads as its creation (`created_at`,
+  `signup_date`, or its first date naming no later stage), and the table's own first date has no
+  `after` naming that parent, validation (and so every run) warns, on
+  `tables.orders.columns.order_date`: "can fall before customers.created_at (the customers row
+  it points at through customer_id); add `after: customers.created_at` to keep it after". The
+  warning's `Issue` carries a `suggestion` (`model2data.model.Suggestion`: the `path` to set,
+  the `value`, and the `spec` version an older document must then say) for a tool to offer as a
+  fix. No warning for a parent without such a date, a self-reference, two foreign keys to one
+  parent, or a table whose `after` already names that parent. Generation itself does not change.
+- The bundled examples use it where it is plainly right: `ecommerce`, `ecommerce_daily`,
+  `ecommerce_training` (orders after their customer), `advanced_features` (time entries after
+  the employee's hire date), `saas_platform` (subscriptions, invoices and audit events after
+  their organisation), and the spec's coffee-webshop example; those documents now say 0.5.0.
+
+### Changed
+- **Integer primary keys number the rows 1..N, and the rows come out in key order.** A table's
+  single-column integer primary key was a shuffled sample starting at 0 (the first customers
+  were 4251, 4557, 1941, ...); it is now 1, 2, 3, ... N, like an identity column, and every
+  output (seeds, the days' batches and change log, the defects' files) lists the rows in that
+  order. A key with a `min`/`max` or a `distribution`, and an integer key that is also a foreign
+  key (a one-to-one child keyed by its parent), is drawn as before and listed in ascending
+  order. Text, uuid and composite keys keep their values and order. Foreign keys, one-to-one
+  foreign keys and `--table-seed` re-rolls work as before (a re-rolled table keeps its ids 1..N
+  with new rows behind them), and a table with `incremental` continues from the largest key on
+  later days, as before (from 1 when it starts empty).
+- **Output changes for the same seed (the determinism promise is per version).** Every model
+  with an integer primary key generates different bytes than 1.14.0: the key no longer draws
+  from the table's random stream, so every column after it draws from a different point of it.
+  Dates change only where a model uses a parent's column in `after` (the examples listed
+  above now do). The same model, seed, `--as-of` and options still give the same bytes on any
+  machine and any day with 1.16.0. Pin `model2data==1.14.0` to keep fixtures of the old shape.
+
 ### Fixed
 - **A `binary` column holds a short value.** `binary` is also the name of a Faker provider, and
   the generic fallback called it, which returns a whole MiB of random bytes per value: about 3 MB
@@ -24,8 +107,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
   names it, its row count and the reference that set it, the way `generate --days` already stops
   adding rows once a one-to-one reference runs out of parents.
 
-A model with no byte column, and no key referencing a column short of distinct values, writes
-byte for byte what 1.14.0 wrote.
+These two fixes change nothing for a model with no byte column and no key referencing a column
+short of distinct values (the integer keys above change its bytes regardless).
 
 ## [1.14.0] - 2026-10-07
 

LLMS.md

@@ -22,7 +22,7 @@ install. If the person you're helping wants to *look at* or share the model rath
 its generation, point them there; the DBML guidance below applies to both.
 
 > **Two input formats.** Since 1.8, model2data reads a model as one YAML document,
-> `<name>.model2data.yml` ([spec 0.4.0](model2data/spec/README.md), with a JSON Schema at
+> `<name>.model2data.yml` ([spec 0.5.0](model2data/spec/README.md), with a JSON Schema at
 > `model2data/spec/model.schema.json` and a complete example at
 > `model2data/spec/examples/coffee_webshop.model2data.yml`), and DBML as supported input that it
 > converts to the same model. The DBML guidance below still holds; if you write the YAML form
@@ -134,6 +134,7 @@ is_paid boolean [note: '{"true_rate": 0.9}']               ' fraction of non-nul
 shipping_city varchar [note: '{"distinct": 12}']           ' draw from a pool this size (not fk/pk/unique/enum)
 customer_id int [note: '{"skew": 0.9}']                    ' per-column override of --skew (fk columns only)
 updated_at timestamp [note: '{"after": "created_at"}']     ' must fall after another date/timestamp column
+order_date timestamp [note: '{"after": "customers.created_at"}']  ' ... or after a parent's column, or a list of both
 completed_at timestamp [note: '{"when": {"status": ["done"]}}']  ' set only where status is done, null elsewhere
 created_at timestamp [note: '{"business_hours": true}']    ' per-column override of --business-hours
 created_at timestamp [note: '{"growth": 0.4}']             ' per-column override of --growth (date/timestamp only)
@@ -161,6 +162,41 @@ shipped_at timestamp [note: '{"after": "ordered_at"}']
 A birth-date-style column (`birth_date`, `date_of_birth`, `dob`) is never folded into this chain —
 it describes the person, not the record.
 
+**`after` can name a parent's column.** `<table>.<column>` reaches the parent row through the
+row's foreign key: `after: customers.created_at` keeps every order on or after its customer's
+sign-up. A list follows every entry, and may mix the row's own columns:
+```dbml
+order_date timestamp [note: '{"after": "customers.created_at"}']
+shipped_at timestamp [note: '{"after": ["order_date", "customers.created_at"]}']
+```
+The table must reach the parent through exactly one one-column foreign key onto its pk or a unique
+column (two, like `sender_id`/`receiver_id` to `users`, is an error naming them). Rows dated too
+early take an older parent, so the column's growth/seasonality/business hours hold; a row with
+none (or a one-to-one) moves its date. A null foreign key constrains nothing; a self-reference is
+not a parent (name the column without a table). In YAML it is spec 0.5.0 (`model2data: 0.5.0`).
+Validation warns where a child's first date can fall before a parent's `created_at`-like date,
+with the `after` to add. With `--hint-tests` each parent column writes the dbt test
+`model2data_not_before_parent`. Integer primary keys are 1..N and every output lists rows in key
+order.
+
+*When to use it:* whenever a child row cannot exist before its parent: an order before its
+customer signed up, a subscription before its organisation, a time entry before the employee's
+hire date. Skip it when the child's date has no such tie (an event log that may predate the
+account it is later linked to).
+
+*Acting on the warning:* it reads `tables.orders.columns.order_date: can fall before
+customers.created_at (...); add `after: customers.created_at` to keep it after`. Add exactly
+that `after` to `generate` on the column the path names (an existing `after` there becomes a
+list: `after: [placed_at, customers.created_at]`), set `model2data: 0.5.0`, and run `validate`
+again; the warning is gone and the file conforms either way. Over the Python API the warning's
+`Issue.suggestion` (a `model2data.model.Suggestion`) carries `path`, `value` and `spec` for the
+same edit. If the child may legitimately predate the parent, leave the warning.
+
+*Not built yet:* a table with two foreign keys to the same parent (`sender_id` and `receiver_id`
+onto `users`) cannot use `after: users.created_at`: model2data cannot tell which row is meant,
+so validation refuses it with an error naming both keys. There is no way yet to pick one; drop
+the hint (the warning is not raised for such a table either).
+
 **Tie lifecycle columns to the status they belong to with `when`.** Without it, every nullable
 column is filled or nulled independently of the others, so a `todo` task gets a `completed_at`
 and a `cancelled` subscription has no `cancelled_at`. `when` maps another column of the same
@@ -287,7 +323,7 @@ model2data --file SCHEMA.dbml [OPTIONS]
 --adapter, -a    TEXT      duckdb (default) or postgres
 --unit-tests               Also generate dbt unit test fixtures
 --hint-tests     TEXT      error, warn (default) or off: severity of the dbt tests written from
-                           min/max, after, null_rate, distinct, when and grain hints
+                           min/max, after (also a parent's column), null_rate, distinct, when and grain hints
 --test-tolerance FLOAT     Absolute slack of the null_rate test (default 0.1)
 --defects        TEXT      clean (default), messy or training: break the data on purpose, so dbt
                            tests are seen to fail; writes defects_report.json and

README.md

@@ -42,9 +42,12 @@ emails, not `Lorem ipsum`. Type a column with any Faker provider (`billing_count
 come from one country. → [The model file](docs/model-file.md)
 
 **Foreign keys that resolve.** Tables are generated in dependency order and every foreign key
-points at a parent row that exists — onto a primary key or a unique one. Composite primary and
-unique keys hold, and a column that repeats a parent's value through a foreign key
-(`orders.customer_email`) holds the email of the order's own customer.
+points at a parent row that exists — onto a primary key or a unique one — and
+`after: customers.created_at` keeps an order on or after the sign-up of the customer it points
+at (`validate` suggests it where a date could fall before its parent's). Integer primary keys
+number the rows 1..N, in order. Composite primary and unique keys hold, and a column that repeats a
+parent's value through a foreign key (`orders.customer_email`) holds the email of the order's
+own customer.
 → [The model file](docs/model-file.md)
 
 **A dbt project that runs.** Seeds, staging models that `ref()` them, `not_null`, `unique`,
@@ -68,8 +71,7 @@ byte-identical. → [The determinism promise](docs/generating.md#the-determinism
 
 **When things happen.** `--business-hours`, `--growth` and `--seasonality` shape every timestamp
 (weekdays and working hours, a trend, a Q4 peak), or one column at a time. `updated_at` never
-lands before `created_at`, `after` orders any two columns, and `when` gives `shipped_at` a value
-on shipped and delivered orders only. → [When things happen](docs/time-shapes.md)
+lands before `created_at`, `after` orders any two columns, or a column after a parent's, and `when` gives `shipped_at` a value on shipped and delivered orders only. → [When things happen](docs/time-shapes.md)
 
 **How the data is spread.** `--skew` lets a few customers place most of the orders; `weights`,
 `true_rate`, `null_rate` and `distinct` shape a column; `distribution` draws a number from a

README_PYPI.md

@@ -42,9 +42,12 @@ emails, not `Lorem ipsum`. Type a column with any Faker provider (`billing_count
 come from one country. → [The model file](https://github.com/JB-Analytica/model2data/blob/main/docs/model-file.md)
 
 **Foreign keys that resolve.** Tables are generated in dependency order and every foreign key
-points at a parent row that exists — onto a primary key or a unique one. Composite primary and
-unique keys hold, and a column that repeats a parent's value through a foreign key
-(`orders.customer_email`) holds the email of the order's own customer.
+points at a parent row that exists — onto a primary key or a unique one — and
+`after: customers.created_at` keeps an order on or after the sign-up of the customer it points
+at (`validate` suggests it where a date could fall before its parent's). Integer primary keys
+number the rows 1..N, in order. Composite primary and unique keys hold, and a column that repeats a
+parent's value through a foreign key (`orders.customer_email`) holds the email of the order's
+own customer.
 → [The model file](https://github.com/JB-Analytica/model2data/blob/main/docs/model-file.md)
 
 **A dbt project that runs.** Seeds, staging models that `ref()` them, `not_null`, `unique`,
@@ -68,8 +71,7 @@ byte-identical. → [The determinism promise](https://github.com/JB-Analytica/mo
 
 **When things happen.** `--business-hours`, `--growth` and `--seasonality` shape every timestamp
 (weekdays and working hours, a trend, a Q4 peak), or one column at a time. `updated_at` never
-lands before `created_at`, `after` orders any two columns, and `when` gives `shipped_at` a value
-on shipped and delivered orders only. → [When things happen](https://github.com/JB-Analytica/model2data/blob/main/docs/time-shapes.md)
+lands before `created_at`, `after` orders any two columns, or a column after a parent's, and `when` gives `shipped_at` a value on shipped and delivered orders only. → [When things happen](https://github.com/JB-Analytica/model2data/blob/main/docs/time-shapes.md)
 
 **How the data is spread.** `--skew` lets a few customers place most of the orders; `weights`,
 `true_rate`, `null_rate` and `distinct` shape a column; `distribution` draws a number from a

docs/dbt-project.md

@@ -52,6 +52,7 @@ dbt package, so `dbt build` still works offline):
 | `generate: {distinct: n}` | `model2data_max_distinct`: at most `n` distinct values | none |
 | `generate: {when}` | `model2data_when`: null exactly where the condition does not hold (with `null_rate`, only there; its null share is then tested among the matching rows) | none |
 | `grain` on a table | `model2data_unique_combination` | none |
+| `generate: {after: table.column}` (a parent's column, see [When things happen](time-shapes.md)) | `model2data_not_before_parent`, one per parent column, in `macros/model2data_parent_tests.sql`: no row dated before the parent row it joins to | none |
 | an enum-typed column | `accepted_values` (always written) | none |
 
 `true_rate`, `weights`, `skew`, `distribution` and the temporal shape hints (`business_hours`,

docs/distributions.md

@@ -34,7 +34,11 @@ orders:
       generate: {distinct: 12}
 ```
 
-`skew` on a foreign key overrides `--skew` for just that column. `weights` biases an enum column
+`skew` on a foreign key overrides `--skew` for just that column. When the child's date has an
+`after` naming its parent's (see [When things happen](time-shapes.md)), the parents created
+early also pick up the children dated before the later ones existed, so the spread is steeper than `skew`
+alone: on the coffee-webshop example at `skew: 0.8`, the top fifth of customers hold about 80%
+of the orders rather than 65%. `weights` biases an enum column
 toward the values named (unnamed values still appear, at weight 1). `true_rate` is the fraction of
 non-null rows a boolean column comes back `true`. `null_rate` replaces the column's default null
 fraction outright. `distinct` draws the column's values from a fixed-size pool instead of a fresh

docs/generating.md

@@ -38,8 +38,10 @@ The same model, seed and `--as-of` — with the same options (`--rows`, `--rows-
 - with the same model2data version, whenever you run it.
 
 A new release may change what a seed produces, to fix a bug or make the data more realistic.
-When it does, the changelog says which models are affected and why. Pin the version
-(`model2data==1.11.0`) when fixtures must stay the same across upgrades.
+When it does, the changelog says which models are affected and why (1.16.0, for one, numbers
+integer primary keys 1..N, which changes the bytes of every model that has one, and moves dates
+only where a model uses a parent's column in `after`). Pin the version (`model2data==1.14.0`)
+when fixtures must stay the same across upgrades.
 
 `tests/test_determinism.py` holds the file hashes of a spread of models, presets, multi-day runs
 and locales, and runs on every pull request — against the newest Faker and pandas too — so a