Displays stay in sync: the server says what changed (a change counter per domain, a cheap check, one client data layer) #325

Problem

A household often runs several displays against one server: a kitchen tablet, a hallway screen, a parent's phone. When someone ticks a chore on one, the others don't show it until they happen to reload.

How the Chores widget picks up changes on main today:

• Its refresh timer (WidgetContainer's countdown, refreshNonce), which is off by default.
• A tab switch, which remounts it and refetches everything.
• A plugin on the same display posting homeglow:data-changed (utils/pluginDataBridge.js).
• The event stream (/api/plugin/v1/events/stream, via utils/pluginEventBridge.js), but only for prize.redeemed (balances and the store) and chore.allCompleted (the celebration). A chore.completed or chore.uncompleted from another display does nothing.
• On top of that, the event stream doesn't reliably reach the browser on every deployment, behind some reverse proxies and tunnels. Anything that depends on it alone stays out of step there.

Each widget also fetches its own copy of the same data (users, chores, history), so two widgets on one display can disagree, and every tab switch re-fetches what was already there.

#324 makes a tap feel instant on the display where it happens. It's a good step, but it doesn't change what the other displays see, and it shouldn't try to.

Goal

A change on one display shows on every other display within seconds, on any deployment, whether the event stream gets through or not. No widget should need to know which server data a write touches.

Design

1. The server says what changed: a counter per domain

The server keeps a revision counter for each kind of household data:

• Bumped in one place, not in every route. An onResponse hook bumps a domain when a non-GET request under its route prefixes succeeds (/api/chores, /api/chore-schedules, /api/users/:id/clams, /api/routines, /api/prizes, /api/settings, …).
• Background work bumps explicitly:
  • the nightly processing (missed chores, bonus chores reset);
  • calendar sync and the routine calendar sync;
  • anything else that writes without a request.
• Time-based visibility isn't a write and needs no bump: a snooze ending, a follow-up's delay passing, midnight, an expiry. Displays already work those out from their own clock.
• Counters live in memory and start at a random base on boot, so a restart reads as "everything changed" and nothing stale survives it.

2. Every display learns the counters, two ways

• GET /api/changes returns { chores: 1042, users: 77, prizes: 12, settings: 31, calendar: 9 } with an ETag, and answers 304 when nothing changed.
  • A display checks it every 15–30 seconds while it's visible (document.visibilityState), and right away when it becomes visible again.
  • It's a handful of bytes, and on any deployment it caps how long a display can lag.
• The event stream carries the same counters with each event. Where it gets through, displays catch up in under a second. Where it doesn't, the check above still does it.

3. One shared data layer in the client

• Widgets read household data from one per-page store, keyed by domain, instead of each fetching its own copy. The store could be a small context of our own or a query cache.
• When a counter moves, the store reloads that domain once, and every widget showing it re-renders from the same data. A tab switch reads the cache instead of re-fetching.
• Writes stay as they are:
  • the tapping display updates instantly, as perf(client): instant chore toggles and fewer widget re-renders #324 does;
  • it reloads after the server confirms;
  • its own write moves the counter, so every other display reloads too.
• Plugins are told when a domain changes, as homeglow:data-changed is today. A plugin showing chore data (Chore Metrics) stays in step without its own polling.

What it replaces

• The hand-written refetches after an action: each widget's "refetch these four things after a transfer" goes away.
• The refresh timer's role for household data. The timer stays for external data: weather, photos, the calendar's upstream.

Phases

1. Server: the counters, the onResponse hook, the explicit bumps, GET /api/changes, and the counters on the event stream. Tests: each kind of write moves its domain, a GET moves nothing, and 304 when nothing changed.
2. Client: the store for chores and users, with the Chores widget moved onto it. Two Edge windows check that a tap in one shows in the other within the check interval, with the event stream blocked.
3. The rest: prizes, settings, calendar, the other widgets, and plugin notifications.

Open questions

• Is 15 seconds right for the visible-display check, or should it be a setting? A wall display with nobody near it could check less often.
• Do we keep one counter per domain, or let a widget ask finer questions, like "only today's history"? Coarse is simpler, and a full chores reload is cheap now that perf(client): instant chore toggles and fewer widget re-renders #324 batches it.
• Should the phone layout get the same treatment, or check only when it becomes visible?

Related

• perf(client): instant chore toggles and fewer widget re-renders #324: instant chore taps, the client half of a write.
• The event stream: the fast path here, and its unreliability is why the check exists.