fix(core-ts,mcp-server): RFC-0045 step 7a, held rows for old-shape values and whole-list re-translation

.changeset/rfc45-step5-mcp.md

@@ -12,7 +12,7 @@ RFC-0045 rollout step 5: reference and repeater fields over MCP. The two types s
 - **Approval prompts** name targets readably, from the dry run's planned entry (canonical ids): `venue → "Main hall" (entry_…, Places)`, `speakers: 2 items → 3 items (added "Ada Lovelace", Speaker; changed item 1: person → "Grace Hopper" (entry_…, People))`, quoted and truncated.
 - **New read-only tool `list_references`** (`content:read`) with `direction: "to"` (referrers, published first, with the members that show them) or `"from"` (the groups an entry names, its original's inherited ones included). 110 → 111 tools. The API's `GET …/entries/:entryId/references` and the tool share core-ts's new `listReferrers` (and `listReferenceTargets`, `referenceCard`).
 - **`list_entries`** returns each entry's `groupId`, the value a reference stores.
-- **Translation tools**: `get_translation_source` lists a localized repeater's text and rich-text sub-values by item path (`speakers/0/talk`) with the source's `sourceRev`; `save_translation` takes them back with that `sourceRev` (`conflict` when the source changed) and folds them into a copy of the source's list, keeping each item's other sub-values (core-ts `translatedItemValues`). Once the translation holds its own list for that repeater, item paths are refused and the list is sent whole; a whole list of another length than the source's is refused too (`translation_shape_mismatch`, new code, 409 in the API). A path naming no such value is refused. Keys whose first part is not a repeater (`og/image`) stay plain keys.
+- **Translation tools**: `get_translation_source` lists a localized repeater's text and rich-text sub-values by item path (`speakers/0/talk`) with the source's `sourceRev`; `save_translation` takes them back with that `sourceRev` (`conflict` when the source changed) and folds them into a copy of the source's list, keeping each item's other sub-values (core-ts `translatedItemValues`). Once the translation holds its own list for that repeater, item paths are refused and the list is sent whole; a whole list given together with item paths must have the source's length (`translation_shape_mismatch`, new code, 409 in the API). A path naming no such value is refused. Keys whose first part is not a repeater (`og/image`) stay plain keys.
 - **`normalizeEntryFields`** is exported from `@cloudhal/core-ts`: MCP's approval prompts read stored values through the markers exactly as the store does.
 - **`continueFieldOperation`** with `onInvalid: "clear"` reports the values it drops as cleared, without the stored request's `field_values_unconvertible` refusal.
 - **Removals during `rename_field`** over MCP: the old and the new key are one field; removing either removes both, so the writer's move cannot bring the value back, and setting one while removing the other is `invalid`.

.changeset/rfc45-step7a-fixes.md

@@ -0,0 +1,10 @@
+---
+"@cloudhal/core-ts": patch
+"@cloudhal/mcp-server": patch
+---
+
+RFC-0045 rollout step 7a: two fixes ahead of enabling reference and repeater fields. Both types stay refused on every production write path until step 7b; nothing changes for the nine existing types.
+
+- **core-ts (#215).** While a top-level conversion from references runs without a text ↔ reference path (reference (multiple) ↔ repeater, reference single ↔ multiple, a repeater converting away), a stored value still in the old shape that does not convert keeps its `entry_refs` rows through a person's unrelated write, such as a title edit. Before, the write dropped them, and the target's delete guard under-blocked. The writer holds the rows at the old type's paths, the same ones the import rebuild keeps (one helper, `oldShapePairs`), so the rows no longer depend on which touched the entry last.
+- **core-ts: held rows follow their holder's status.** A row a write holds (a value a running conversion did not accept, here or during a text ↔ reference conversion) kept the `source_status` of its last insert, so publishing the holder left the target deletable and unpublishing left it locked. The write now updates the held rows' `source_status` in the same batch, anchored on the entry's new hash.
+- **mcp-server, core-ts (#218).** Once a translation holds its own list for a localized repeater, `get_translation_source` returns the source's whole list for that key under `fields` (in the source's order, with every sub-value) instead of item paths, and names it in the new `wholeLists` output: send it back whole, translated. `save_translation` already refused item paths there (`translation_shape_mismatch`); its hint and the tool descriptions now say what to send. `save_translation` also checks a given `sourceRev` against whole repeater lists (`conflict` when the source moved on); without one they save as before. New core-ts export: `translationSourceValues`.

docs/rfcs/0045-collections-stage-4-references-and-repeaters.md

@@ -1381,7 +1381,7 @@ Round 1 dropped the "fits" shortcut for text ↔ reference, but then the table r
   - A key is an item path only when the part before its first `/` names a repeater of the collection (`itemPathRepeater`); any other key, such as `og/image` in a collection without definitions, is a plain key, as before.
   - Item paths write into a copy of the source's list (or into a list given whole in the same call), so each item keeps the source's other sub-values: targets, images, selects.
   - **Once the same-locale translation holds its own list for that repeater, item paths are refused** (`translation_shape_mismatch`, 409, "send the whole list"): re-translate by sending the whole list, in the source's order. Either side may have been reordered or edited since the first translation, and no revision check can tell whether an index still names the same item, so the rule does not try (review round 3 replaced a `translation.source.rev` check that had holes).
-  - A list given whole must have the source's length (`translation_shape_mismatch`).
+  - A list given whole must have the source's length (`translation_shape_mismatch`). *Amended in 7a:* only when item paths come in the same call; a whole list sent alone is the translation's own and is stored as sent.
   - Paths name items by index, so `get_translation_source` returns `sourceRev`, and `save_translation` requires it with item paths (`invalid` without, `conflict` when the source moved on). The handler refuses paths that no longer fit, as the precheck does; nothing is dropped silently.
 - **`set_field_localized`** takes no `sub` over MCP, since the store refuses it (§7 "Amended in 4").
 - **The dry run's reach.** It returns after the asset references, the release hold (`scheduled_locked`), time gates and the term invariants (`required_term`), with the planned entry. It does not plan the path change (`path_conflict`, redirects), a collection's first entry, list values or source-forward, and slug uniqueness is decided by the write. The prechecks build the same entry the handler writes (terms, cover, title and body included).
@@ -1446,3 +1446,11 @@ Round 1 dropped the "fits" shortcut for text ↔ reference, but then the table r
 - **A type change's target settings lock** (a disabled `fieldset`) whenever its inputs do: running, stuck or finished.
 - **"Other entries left" is decided per referring field**, as the store decides: referrers whose field has no usable other member are listed as showing nothing, the others name the language that loses the card; both can show.
 - **The blocked count is the store's** (`total`, never capped at the list's 100), the "more references" note shows whenever rows remain unlisted, and a late refusal reads the referrers again.
+
+### Amended in 7a (2026-11-13, rollout step 7, first PR: #215 and #218)
+
+- **Old-shape values under a reshape from references** ([#215](https://github.com/cloudhal-dev/cloudhal/issues/215)). A running top-level conversion from references that no text ↔ reference rule reads (reference (multiple) ↔ repeater, reference single ↔ multiple, a repeater converting away) has no row path (`convertingPath`). A stored value still in the old shape that does not convert, such as a malformed list for reference → repeater, now keeps its rows through a person's unrelated write: `normalizeWrite` holds them at the old type's paths, each id-shaped string at `<field>` from a reference and each item's id-shaped sub-value at `<field>.<sub>` from a repeater. The rebuild (`reshapedRows`) and the writer share one helper (`references/old-shape.ts`, `oldShapePairs`), so the rows are the same whichever touched the node last, and the target's delete guard holds.
+- **Re-translating a repeater the translation already holds its own list of** ([#218](https://github.com/cloudhal-dev/cloudhal/issues/218)). `save_translation` refuses item paths for it (Amended in 5), so `get_translation_source` no longer offers them. For each such repeater it returns the source's whole list under the repeater's key in `fields`, in the source's order and with every sub-value, and names it in a new `wholeLists` output (core-ts `translationSourceValues`). The agent translates its text and rich-text sub-values and sends the whole list back. The `translation_shape_mismatch` hint and the tool descriptions say so.
+- **Held rows follow their holder's status** (PR #221 review). A row a person's write holds kept the `source_status` of its last insert, since the writer leaves it out of its `DELETE` and `INSERT`, while the delete guard reads that column. The write now sets the held pairs' `source_status` to the entry's new status in the same batch, anchored on its new hash (`references/held-status.ts`), for reshapes and text ↔ reference conversions alike; the rebuild writes the same value.
+- **`sourceRev` with whole lists.** A given `sourceRev` is checked for a whole repeater list too (`conflict` when the source moved on), so a list read before the source changed is not linked to the new revision. It stays optional there.
+- **The length rule, qualified.** Amended in 5 said a whole list of another length than the source's is refused. That holds only when item paths come in the same call, where the paths index into the given list (`baseList`). A whole list sent alone is the translation's own, like any other localized value, and is stored as sent. §3's "only into a list of the original's length" describes item paths.

packages/core-ts/src/fields/index.ts

@@ -142,8 +142,10 @@ export {
   sharedFieldIssues,
   sharedFieldKeys,
   type TranslatedItemsInput,
+  type TranslationSourceValues,
   translatableFieldValues,
   translatedItemValues,
+  translationSourceValues,
 } from "./translation.ts";
 export {
   ALL_FIELD_TYPES,

packages/core-ts/src/fields/translation.ts

@@ -5,8 +5,10 @@
  * the localized `text` and `richText` values to translate: top-level by key, and inside a
  * localized repeater by item path, `speakers/0/talk` (RFC-0045 §3). Item paths write into a copy
  * of the source's list, so an item's other sub-values (references, selects, images) come along.
- * Once the translation holds its own list, that repeater is sent whole; other lengths are
- * `translation_shape_mismatch`. Worker-safe.
+ * Once the translation holds its own list, that repeater is sent whole: the source hands it over
+ * whole (`translationSourceValues`), and item paths for it are `translation_shape_mismatch`. A
+ * list given whole together with item paths must have the source's length; a whole list sent
+ * alone is the translation's own and is stored as sent (RFC-0045 "Amended in 7a"). Worker-safe.
  */
 import type { FieldDefinition } from "@cloudhal/core-contracts";
 import { type FieldIssue, isRecord } from "./types.ts";
@@ -62,6 +64,40 @@ export function translatableFieldValues(
   return out;
 }
 
+/** What a translation source hands over to translate (`translationSourceValues`). */
+export interface TranslationSourceValues {
+  /** `translatableFieldValues`, with a list given whole for each repeater in `whole`. */
+  readonly fields: Record<string, unknown>;
+  /** The repeaters sent whole: the existing translation holds its own list of them. */
+  readonly whole: string[];
+}
+
+/**
+ * RFC-0045 §3 (#218): the values to translate for a source and the existing same-locale
+ * translation, if any. As `translatableFieldValues`, except a localized repeater the translation
+ * already holds its own list of: item paths would be refused for it (`baseList`), so the source's
+ * whole list is handed over under its key instead, in the source's order and with every
+ * sub-value, to be translated and sent back whole.
+ */
+export function translationSourceValues(
+  defs: readonly FieldDefinition[],
+  source: Values | undefined,
+  existing: Values | undefined,
+): TranslationSourceValues {
+  const fields: Record<string, unknown> = translatableFieldValues(defs, source);
+  const whole: string[] = [];
+  for (const def of defs) {
+    const list = source?.[def.key];
+    if (translatableSubs(def).size === 0 || !Array.isArray(list)) continue;
+    if (!Array.isArray(existing?.[def.key])) continue;
+    for (const key of Object.keys(fields))
+      if (itemPathRepeater(defs, key) === def.key) delete fields[key];
+    fields[def.key] = list;
+    whole.push(def.key);
+  }
+  return { fields, whole };
+}
+
 /**
  * The repeater an item path (`speakers/0/talk`) names: the part before the key's first `/`, when
  * it is a repeater of `defs`. Any other key is a plain key, `/` included (a collection without
@@ -133,6 +169,8 @@ const shapeIssue = (key: string, message: string): FieldIssue => ({
  * source's length), else a copy of the source's. When the translation already holds its own list,
  * a path cannot know which of its items matches the source's by index (either side may have been
  * reordered or edited since), so that repeater must be sent whole (`translation_shape_mismatch`).
+ * Only paths reach here: a whole list sent with no path for it is the translation's own and is
+ * stored as sent, whatever its length.
  */
 function baseList(
   key: string,

packages/core-ts/src/references/held-status.ts

@@ -0,0 +1,21 @@
+/**
+ * RFC-0045 §7: a held row follows its holder's status. A person's write leaves the rows of values
+ * a running conversion did not accept as they are (`HeldRef`): `entryRefStatements` keeps them out
+ * of its `DELETE` and its `INSERT`. Their `source_status` is what the delete guard reads (`SHOWN`,
+ * `references/sql.ts`), so the write sets it to the entry's new status, in the same batch and
+ * anchored on the entry's new hash like the other rows statements (rule 4). Worker-safe.
+ */
+import type { SqlStatement } from "@cloudhal/storage";
+
+/** The `UPDATE` of the held pairs' `source_status` (`kept`: `"<field> <target>"` keys). */
+export function heldStatusStatement(
+  at: { siteId: string; nodeId: string; status: string; hash: string },
+  kept: ReadonlySet<string>,
+): SqlStatement {
+  return {
+    sql: `UPDATE entry_refs SET source_status = ?
+      WHERE site_id = ? AND source_id = ? AND field || ' ' || target_id IN (SELECT value FROM json_each(?))
+        AND EXISTS (SELECT 1 FROM nodes WHERE id = ? AND hash = ?)`,
+    params: [at.status, at.siteId, at.nodeId, JSON.stringify([...kept]), at.nodeId, at.hash],
+  };
+}

packages/core-ts/src/references/old-shape.ts

@@ -0,0 +1,48 @@
+/**
+ * RFC-0045 §7: the rows a stored value keeps under a running top-level conversion from references
+ * that no text ↔ reference rule reads (`convertingPath` is undefined): reference (multiple) ↔
+ * repeater, reference single ↔ multiple, a repeater converting away. A value still in the old
+ * shape is read through the old type, so its rows stand until phase B rewrites it.
+ *
+ * One rule for both sides that write rows (#215): the rebuild (`rebuildRows`) keeps these rows
+ * for every stored value, and a person's write holds them for a stored value that did not
+ * convert (`normalizeWrite`), so whichever touched the node last leaves the same rows. Pure.
+ */
+import type { FieldDefinition } from "@cloudhal/core-contracts";
+import { GROUP_ID_PATTERN } from "../fields/props.ts";
+import { isRecord } from "../fields/types.ts";
+import { convertingPath } from "./conversion-rows.ts";
+
+const GROUP_ID = new RegExp(GROUP_ID_PATTERN);
+const isId = (v: unknown): v is string => typeof v === "string" && GROUP_ID.test(v);
+
+/** One old-type (field path, group id) pair. */
+export interface OldShapePair {
+  readonly f: string;
+  readonly t: string;
+}
+
+/** Whether `def` carries a top-level marker converting from references with no row path. */
+export function readsOldShape(def: FieldDefinition): boolean {
+  const m = def.migration;
+  if (m === undefined || m.sub !== undefined) return false;
+  if (m.fromType !== "reference" && m.fromType !== "repeater") return false;
+  return convertingPath(def) === undefined;
+}
+
+/**
+ * The pairs `value`, stored under `key`, holds under `def`'s old type, in order, duplicates
+ * included: from a reference, each id-shaped string at `key`; from a repeater, each item's
+ * id-shaped sub-value at `<key>.<sub>`. Empty when `def` carries no such marker.
+ */
+export function oldShapePairs(def: FieldDefinition, key: string, value: unknown): OldShapePair[] {
+  if (!readsOldShape(def)) return [];
+  const from = def.migration?.fromType;
+  const out: OldShapePair[] = [];
+  for (const v of Array.isArray(value) ? value : [value]) {
+    if (from === "reference" && isId(v)) out.push({ f: key, t: v });
+    if (from === "repeater" && isRecord(v))
+      for (const [sub, t] of Object.entries(v)) if (isId(t)) out.push({ f: `${key}.${sub}`, t });
+  }
+  return out;
+}

packages/core-ts/src/references/rebuild.ts

@@ -17,6 +17,7 @@ import { GROUP_ID_PATTERN } from "../fields/props.ts";
 import { fieldDefinitions, isRecord } from "../fields/types.ts";
 import { chunks, REFS_STATEMENT_BYTES } from "../media/refs-rebuild.ts";
 import { convertingPath } from "./conversion-rows.ts";
+import { oldShapePairs } from "./old-shape.ts";
 import { type EntryRefRow, entryRefRows, sharedOf, storedKey } from "./rows.ts";
 
 const GROUP_ID = new RegExp(GROUP_ID_PATTERN);
@@ -64,30 +65,20 @@ export function rebuildRows(site: Pick<Site, "collections">, entry: ContentEntry
 }
 
 /**
- * A running reference (multiple) ↔ repeater conversion (top level): a stored value still in the
- * old shape is read through the old type, so its rows stand until phase B rewrites it. From a
- * reference, each id-shaped string of the list gets a row at the field path; from a repeater,
- * each item's id-shaped sub-value at `<field>.<sub>`.
+ * A running top-level conversion from references with no text ↔ reference path (reference
+ * (multiple) ↔ repeater, among others): a stored value still in the old shape is read through the
+ * old type (`oldShapePairs`, the same pairs a person's write holds for a value that does not
+ * convert), so its rows stand until phase B rewrites it.
  */
 function reshapedRows(
   def: FieldDefinition,
   values: Readonly<Record<string, unknown>>,
   have: readonly EntryRefRow[],
 ): EntryRefRow[] {
-  const from = def.migration?.fromType;
-  if (def.migration?.sub !== undefined || (from !== "reference" && from !== "repeater")) return [];
   const k = storedKey(def, values) ?? def.key;
-  const stored = values[k];
-  const found: { f: string; t: string }[] = [];
-  const id = (v: unknown): v is string => typeof v === "string" && GROUP_ID.test(v);
-  for (const v of Array.isArray(stored) ? stored : [stored]) {
-    if (from === "reference" && id(v)) found.push({ f: k, t: v });
-    if (from === "repeater" && isRecord(v))
-      for (const [sub, t] of Object.entries(v)) if (id(t)) found.push({ f: `${k}.${sub}`, t });
-  }
   const seen = new Set(have.map((r) => `${r.f} ${r.t}`));
   const out: EntryRefRow[] = [];
-  for (const { f, t } of found) {
+  for (const { f, t } of oldShapePairs(def, k, values[k])) {
     if (seen.has(`${f} ${t}`)) continue;
     seen.add(`${f} ${t}`);
     out.push({ t, f, k, shared: sharedOf(def) });