Freshness ideas from the ocabox Staleness Contract: state publisher with keepalive, contact clock, FreshnessPolicy, SharedReader

[majkelx]

Discussion issue. Source: ocabox "Staleness Contract" closing report (2026-09-04), Part VII — a code survey of serverish and ocabox-tcs against seven freshness ideas that the ocabox family adopted.

What serverish lacks today (with evidence)

• Freshness on read — none. MsgReader.read_next never returns without data (msg_reader.py:191-197); single_read(deliver_policy='last') returns a week-old message as current; MsgDocumentReader swallows its initial_wait timeout and continues with an empty document (msg_document_read.py:120-125). health_status.last_message_ago measures delivery cadence, not contact with the source.
• Publisher keepalive / suppression — none. Status is published on change only, heartbeat every 10 s carries just service_id/timestamp/status (monitored_nats.py:110-114; tcs doc/nats.md says 30 s). The guider had to add its own 10 s republish loop calling private _send_status_report (ocabox-tcs manager.py:149-193). Only tombstone concept: KV DEL surfaced as (None, meta) (msg_kv.py:84-94).
• Fan-out — none. One MsgReader = one JetStream consumer = one deque; two awaiters compete. MsgSingleReader.read acks in pop_msg before returning inside wait_for → ack-then-cancel loses a message.
• Liveness of work — not detected. tcs heartbeat proves the event loop lives, not that the service works; hung-service detection lives only in tcsctl heuristics (30 s/120 s). Guider implements an ad hoc T2 (last_cycle_age_s > 30 → DEGRADED).

Candidates, in priority order

1. MsgStatePublisher (get_statepublisher(subject, keepalive=N)): publish on change, republish unchanged every N s with tag 'keepalive', publish_gap(reason) tombstone with meta['gap']={reason, last_good_ts}. Switch _send_status_report to it (removes the guider loop). Silence > N on a state subject then means dead.
2. Contact clock: MsgReader._last_contact_time bumped on every successful exchange (message, 404/408, consumer_info), health_status['last_contact_ago']; ReportingMonitoredObject.touch() + last_activity_ts in the heartbeat so tcs/tcsctl can tell "process alive, work stalled" from "dead" without per-tool thresholds. Meaningless without (1); introduce together.
3. FreshnessPolicy(max_age, refresh_after, on_stale=RAISE|NONE|LAST_GOOD) in policies.py (frozen, None = driver default, default UNLIMITED = no behaviour change), enforced in read_next.pop_msg, single_read, MsgDocumentReader.open from meta['nats']['timestamp']; stale delivered as (None, meta) with meta['stale']={reason, last_good_seq, last_good_ts} and tag 'stale' (KV precedent), opt-in only.
4. Presets Preset.INTERACTIVE/SERVICE/DISPLAY/FAIL_FAST = (ErrorPolicy, FreshnessPolicy, BatchPolicy) unpacked in Messenger.get_reader, keeping the existing PolicyConflict rule.
5. SharedReader broadcast: one reader task, bounded queue per subscriber, meta['nats']['seq'] as delivery number, shielded futures; LiveDocument rebuilt on it; fixes the MsgSingleReader ack/cancel loss.
6. Heartbeat credibility: reject a heartbeat whose data.timestamp lags meta['nats']['timestamp'] beyond a threshold (starved publisher, cf. Health monitoring: detect and report asyncio event-loop starvation #36); RTT as a connection metric.

Blockers found on the way

ServiceController.stop_service sets Status.OK "Service stopped" (a stopped service looks healthy); MetaValidator.meta_schema vs schema/meta.schema.json disagree on required; tags is list[str], so rich fields need a new meta key; NATSConfigSource is a placeholder, so per-installation T1/T2 have no config channel yet.

Related: #17 (readers on connection issues), #32 (cancellation lost in fetch/wait), #33 (ephemeral consumer leak — a SharedReader reduces consumer count), #36 (event-loop starvation health).

[majkelx]

Report v4 (2026-09-05): the serverish chapter (Part VII) is rewritten as client use cases UC1–UC8 (freshness on read, dead vs stable via keepalive, alive vs working, many widgets one subject, no loss on cancel, presets vs envelope conventions, per-installation config, many metrics per message) with concrete mechanisms, what is not transferable from ocabox, and an envelope spec for a JS TOI-like frontend. Vault: Architecture/Staleness Contract — raport zamykający (2026-09-04).md.