Skip to content

Fixes: stop sync, restore and undo losing data, and fix dates, publishing, offline maps and the e2e suite - #31

Merged
davd-gzl merged 113 commits into
mainfrom
audit-fixes
Sep 30, 2026
Merged

davd-gzl merged 113 commits into
mainfrom
audit-fixes

Conversation

@davd-gzl

@davd-gzl davd-gzl commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

A device sync or a CSV import drops every photo, an edit made while a sync runs is undone, and a restored backup is deleted again by the next sync. Those three lose data on paths users take every day. The rest of this branch fixes defects of the same kind found in one pass over the app: dates shown a month early west of UTC, a published site that leaks photos from outside its date range, offline maps that forget what was downloaded, and e2e tests that pass without checking anything.

Every fix carries a test that failed on main before it and passes now. The unit suite passes 654 of 654 on this head and the e2e suite 83 of 83, after one rerun of a spec that timed out on a loaded machine. The branch also merges main at 85ee6e3, and each fix was carried onto main's new journal composer and per-stop trip dates where the defect still existed there.

Ten areas follow, each readable alone.


Sync and backup: records and photos lost or brought back

Rewriting the visits table cleared the photo store, then skipped writing any photo it had already seen, so every photo vanished after a sync or a CSV import. Photos are now written back whenever the table is rewritten. A sync now merges in whatever changed on the device while it ran instead of saving its starting snapshot. A restore clears the deletions it overrides, so the next sync keeps the restored records, and undoing a trip or story deletion now survives a sync too. Sync reads a synced file over 1 MB, which GitHub's contents API returns empty, waits for the stores to load, keeps one visit per place across devices, and compares stamps as times. It also stops committing an unchanged file on every run. Two open tabs no longer write over each other.


Dates: a month early west of UTC

Weekday headers and month-precision trip dates were built at midnight UTC and printed in local time, so in New York the journal calendar labelled Monday as Sunday and a trip dated August 2024 read July 2024. They are now printed in the zone they were built in. A date given as a year or a month is shown at that precision instead of gaining a first day. The file format accepts only real calendar dates, and moving a trip stop keeps its date.


CSV and imports

A CSV row with an out-of-range coordinate or a long name was imported unchecked, and then every export and sync failed on it. Rows are now held to the file format, and the ones that fail are skipped and counted. Empty coordinates stay empty instead of becoming 0,0. Formula characters are neutralized in the CSV export only, so a note starting with a dash keeps it through a backup. Zero-width joiners stay in text, so emoji and Persian or Indic spelling survive. An import refuses a photo whose base64 does not decode, while an export leaves such a photo out, so a device already holding one still backs up.


Publish: what reaches the public site

A site published for March 2024 included photos from a 2019 story. It now includes only what the chosen range covers, every stop of a multi-stop trip, and trips dated by month or year. A travel named without Latin letters no longer maps to the same folder as every other, so it stops overwriting the last one. A passphrase-locked travel gets a random folder and a generic commit message, the site's root page is rewritten only if Postcards wrote it, and publishing into the sync repository is refused.


Map, places and stats

The filter panel no longer pulls focus away on every tap, and the Visited list shows every city logged, not only the 2,000 largest. Stats tiles open lists that match their counts, count cities under the chosen country scope, and open the right city when two share a name. The guide shows the place on screen instead of the previous one. A map change made before the map finished loading is now drawn. Picking a place no longer sends its name to Wikipedia unless guides are on. Several strings were translated, and French counts zero in the singular.


Offline maps and the installed app

The service worker's cache expiry deleted downloaded regions and the full city list while Settings still showed them as saved. It no longer expires them by age. The worker now takes control on the first visit, so a download started then is kept. A region download stops when Offline mode turns on or Settings closes. Guides moved out of localStorage, which they filled until settings and the sync token silently stopped saving. A pack reinstalled keeps its place ids, and a dismissed update banner comes back on the next check.


The move to the new address

The new address now shows what it received and asks before saving it, even on an empty device. Once the move is done, the old address clears its sync settings and places. The sync docs recommend a token limited to the one sync repository. They also explain that browser storage is shared by every app served from offware-apps.github.io, and they state what the page's CSP does and does not stop.


Android: install the next build over the last one

Each CI build was signed with a fresh debug key and kept versionCode 1, so a new APK could not install over the previous one, and uninstalling first deleted the user's data. The workflow now signs a release build with one key from four repository secrets, numbers each build, and republishes it at a URL that never changes, linked from the README and from Settings on the web. Without the secrets it builds the debug APK as before. The launcher shows the name Postcards and the app's own icon. docs/NATIVE-BUILDS.md has the commands that create the key and set the secrets.


E2E suite

The flakes blamed on CPU load were chunk downloads aborted with net::ERR_NETWORK_CHANGED, which a host with churning network interfaces triggers. The app now shows a reload screen instead of a blank page when its code fails to download. Specs wait for the app through one helper, seed places before the app opens where marking is not the subject, and block the service worker except where it is. The perf, intro, privacy and trip tests now fail when the behaviour they name breaks. A backup round trip through both export formats is covered. Traces of a failed CI run are uploaded.


Cleanup

42 translation keys, 21 CSS classes and 15 helpers that nothing but their own tests used are gone, along with three copies of the coordinate lookup and a second data-URL decoder. The shared decoder now keeps a percent-encoded photo's real bytes, where the old one stored one wrong byte per character.

Every filter tap moved focus back to the dialog itself. The hosts pass an
inline onClose, so each tap re-rendered them with a new callback, and the
focus effect listed it as a dependency: it re-ran, restored focus to the
opener and focused the panel again. The effect now reads onClose through a
ref and runs on open alone.
Hovering or focusing a toast pauses its timer, and closing it from there
unmounted it with no mouseleave or blur, so the pause stayed set and every
later toast stayed on screen until closed by hand. The pause now clears
when the toast goes.
The Counter component was declared inside StatStrip, so each render made
a new component type and React replaced every counter button, dropping
keyboard focus whenever a visit changed. Counter now lives at module
scope.
Moving a city or country page to another place kept the previous place's
overview and full guide. GuideContent reads its saved guides in useState
initialisers, which run once, and the page stays mounted across the move.
GuideSection now keys GuideContent by the guide's country and title.
The Cities browse with a personal status scanned only the 2000 most
populous cities, so a visited or wished-for small town never showed. Those
views now start from the cities you hold a record for, ordered by
population like the pool.
The installed app showed as Place'Been under its icon and in the task
switcher: the rename to Postcards changed capacitor.config.ts but not the
Android string resources the launcher reads.
West of UTC the trip composer listed "December" for month 01, a month-dated
trip read as the month before, and the Monday-first journal calendar was headed
Sun…Sat. Each label came from a date built at UTC midnight and formatted in local
time; the composer now reuses the local month-name helper and the other two
format in UTC.
Each CI run generated a fresh debug keystore, so every APK carried a
different certificate and versionCode 1: Android refused it as an update,
and the only way forward was to uninstall, which deletes the app's data.
With the keystore in repository secrets CI now builds a release APK signed
with one stable key; versionCode is the run number and versionName the app
version plus the short sha. Without the secrets it builds the debug APK as
before. The rolling release moves to android-latest with a postcards.apk
asset, install notes, and a URL that never changes.
A category picked under Monuments persists and keeps narrowing the saved
places list, but no chip named it, so it could be neither seen nor cleared
from any other view. The list also ignored a category change until another
filter moved, since its memo did not depend on it. The category now gets a
chip on Places (the map, which ignores it, hides it), and both memos
depend on it.
formatDate read every date as a day, so a trip dated 2024-03 showed as
"Mar 1, 2024" and 2024 as "Jan 1, 2024" in the publish trip picker, the
Markdown export and the publish summary, and an impossible 2024-02-31 rolled into
March. It now labels a month or a year as such, formats in UTC, and passes an
impossible day through as typed; parseTripDate rejects a day its month lacks.
The file schema checked a date's shape only, so 2024-13-45 validated on a visit,
a trip or a story. A story date must now be a real day; a visit or trip date that
is not one loads as undated rather than failing the file, since the CSV import
once stored such dates and a file the app wrote must still load.
The Cities, Monuments and Airports browse list the gazetteer with your
status laid over it, and ignored the date, folder, sort and favourite,
photo and note filters the panel still offered there, which the badge and
the chips counted as active. Those dimensions describe a saved record, so
applying them would turn the world browse into your own list and leave
Not visited always empty. The panel now leaves them out on a browse kind,
and neither the badge nor the chips count them; your saved places keep
every dimension.
The composer offers a year and a month only, and rebuilt the date from them on
every save, so opening a multi-stop trip dated 2024-03-15 and saving it stored
2024-03. A full date now survives while its year and month are left as they were.
With the scope set to UN members, a city in a territory still counted
toward the cities visited while its country dropped out of both the
country count and the city total, so the city share mixed two scopes.
Cities now honour the scope like their denominator.
julianToDate took "this year" from UTC, so a pass scanned on the morning of
1 January east of UTC, while UTC was still on 31 December, was dated a year back.
A device sync, a CSV import and the undo of a place removal all lost the
place photos on the next launch. replaceAllVisits and replaceAllPortable
cleared the photos store, and dehydrateVisit skipped the write for any photo
whose blob id it had cached, so the rewritten refs pointed at nothing;
deleteVisit likewise removed the blobs an undo still referenced.

The rewrite now keeps the blobs and deletes, in the same transaction, only
those no visit references any more, so the store stays atomic and a sync
does not decode every photo again. dehydrateVisit checks that a cached id
still has its blob with a key lookup, which reads no image, and stores it
again from memory only when it is missing.
A place added, edited or deleted while a sync was pulling or pushing was
undone when the run finished: runDeviceSync took its snapshot of the stores
before the pull, and persist rewrote IndexedDB and memory from the merge of
that snapshot, which also wiped the tombstone of a deletion made meanwhile.

persist now folds every record changed since the snapshot back onto the
merged result, newest still winning, and tombstones the records deleted
since. It does so once for the disk write and again for memory, since an
edit made during that write lands in its own later transaction. The next
run, which auto-sync schedules for any edit, pushes them.
The offline region's "saved" date was the UTC day, so a region saved in the
morning east of UTC read as saved the day before. It now uses the same local
today as a new visit.
The published site formatted every step date as a day, so a step from a trip
dated 2024-03 read "Mar 1, 2024" and one dated 2024 read "Jan 1, 2024". The
reader now shows a month or a year as such, in UTC, like the app.
The cities, big cities and megacities tiles, the Cities bar and the
strip's been counter count visited cities, but opened your saved places of
every kind, so a monument, an airport or a country padded the list past
the tile's number. The monuments tile opened every monument in the world.
They now open the Cities or Monuments browse with the Visited status, and
the unused kind-less visited view goes.
Restoring a backup brought back a place, trip or story deleted since, and
the next sync deleted it again: restore kept the local tombstones, and the
restored record, stamped at its old updatedAt, lost to them in the merge
(the remote holds the same deletion once it was synced).

A restored record with a local tombstone is now stamped now, as an undo
does, so it wins over the deletion, and its tombstone is dropped in the same
transaction as the records.
sanitizeText dropped U+200D and U+200C with the other zero-width characters, so
every export, import and sync split emoji such as 👩‍💻 and 🏳️‍🌈 into their parts
and changed Persian and Indic spelling. The joiners are not spoofing characters;
only U+200B, U+2060, U+FEFF and the bidi controls are still stripped.
The northernmost, southernmost and biggest city records looked their
city up by name within its country, so where two cities share a name the
map opened the first one. The southernmost record was also hidden
whenever it shared the northernmost one's name. Records now carry the
city's id, and both the lookup and the comparison use it.
Nothing in the web app or at the top of the README pointed to the Android
build, which was reachable only through the Releases page. Settings now has
an Android app section, on the web only, linking to the permanent APK URL,
and the README names the same link under the web address.
Undoing the removal of a trip or story brought it back until the next sync
deleted it again, and undoing an add or an edit already synced was undone
by the next sync too. Every Travel and Journal undo called setAll with the
whole list captured before the action, which neither moved updatedAt nor
removed the tombstone, and dropped an added record without one.

The undo of a removal or an edit now calls restoreTrip or restoreStory,
modelled on useVisits.restoreVisit: one record put back, stamped now, its
tombstone cleared. The undo of an add, a boarding-pass connection included,
removes the added records, tombstoned like any removal. The setAll and
replaceAll functions of trips and stories have no caller left and go.
French reads "0 lieu", but the plural helper chose the singular for a
count of exactly 1 in every locale but Korean, so French showed "0 lieux".
It now picks the form from Intl.PluralRules for the active locale.
When IndexedDB failed to open (a private window, blocked storage), load
rejected unhandled, the stores stayed loaded=false for good, so auto-sync
and the moved-address screen waited forever, and every later write
rejected against the cached failed open.

A failed startup read now reads as empty, settles the store loaded and
switches the session to memory, as a browser without IndexedDB already
runs: every store write becomes a no-op instead of a rejection. The device
store is not retried within the session, since a sync or an import would
then rewrite whole tables from stores that never loaded.
sanitizeText stripped a leading - + = @ from all user text on every parse, so a
journal entry "- packed: boots" lost its dash on each export, import and sync,
while the places CSV export, the one file a spreadsheet opens, wrote names such as
=HYPERLINK(...) as they were. The CSV writer now prefixes such a field with an
apostrophe and the CSV importer drops it again; stored text keeps what was typed.
Text already stripped cannot be recovered.
A CSV row skipped PlaceRefSchema, so one latitude of 999 or a 300-character name
was stored and then made every export, archive and sync fail validation; an
empty lat/lon cell read as 0,0 and put the place at Null Island, and a date such
as 2024-13-45 was kept. Each row is now checked against the file schema and
skipped and counted in the import result when it fails, an empty coordinate
means none, and only a real calendar day is kept as the date.
In French and Korean the Stats travel mode tooltip read "1 en flight"
since it was handed the raw mode id, and the Stats dataset note, the
keyboard shortcuts dialog and the Settings data sources label were
hard-coded English. The tooltip now takes the mode's label, and the rest
read new keys in all three catalogs.
The feed filters by folder through matchesFolder directly.
Every caller (restore, erase all, the moved-origin reset and the tests)
passes stories, so the branch that left the stories store untouched
when they were omitted could no longer run.
Every test opens a fresh context, so the worker installed and precached the
whole app in each one. It is blocked by default now; offline.spec and
privacy.spec, which exercise the installed app, opt back in. Three
alternating full runs each: 47.4, 49.0, 42.7s before, 40.8, 41.1, 50.6s
after, on a shared machine at load 37 to 49.
StoryMap and myPlaces each carried a private copy of distance.ts's
coordsOf, and the passport's fallback anchors inlined the same four
branches. They now call coordsOf; the three agreed with it on every one
of 200,000 generated places.
The zip archive carried its own copy of the base64 encoder that
photoBlobs uses for blobToDataUrl. photoBlobs now exports it as
bytesToDataUrl and the archive imports it; both produced the same
string for 1,000 generated byte arrays, chunk edges included. The two
decoders stay apart: on a non-base64 payload one keeps UTF-8 bytes and
the bare mime, the other one byte per char and the mime's parameters.
renderPoster fetched and parsed countries-50m.json on every export,
beside the once-per-session getLand the route and coverage maps use.
It now awaits getLand and still throws when the geometry is missing.
The main map keeps its own copy, which also tags each feature with its
numeric id.
The country screen's sites and cities lists and the journal feed each
inlined the pager markup ListPager already renders. They now use it,
with the same Showing and Show N more text; a label prop carries the
cities list's most-populous wording. Their Show more button gains the
hold-to-repeat every other paged list already has.
…der test

openAppWithVisits writes the cities, looked up in the bundled gazetteer, into
a first-version database from an init script, and the app's own upgrade
carries them. Ten tests that marked two or three cities through search to
set up their subject use it; the tests about marking still go through
search. Over three alternating full runs those ten tests took 44 to 51s of
worker time instead of 76 to 81s, and the suite 47.5 to 53.7s of wall time
instead of 49.4 to 71.0s.
A photo whose data URL carries a percent-encoded payload rather than
base64 (the schema admits one, e.g. data:image/png;charset=utf-8,...)
was stored as one byte per character, so a ✓ became 0x13, under a Blob
type that kept ;charset=utf-8. An escape that was not UTF-8, such as
%89, threw URIError and failed the write. The payload is now
percent-decoded as the URL standard does, to UTF-8 bytes with each %XX
as its byte, under the bare mime.
The archive's own data-URL decoder percent-decoded a non-base64 photo
with decodeURIComponent, which throws URIError on an escape that is not
UTF-8 such as %89, aborting the whole Save everything export although
its comment said such a photo could not throw. The archive now uses the
photo store's decoder, which percent-decodes byte by byte and never
throws on that path. On 100,000 generated data URLs the two agreed on
bytes and mime wherever the old one returned; the other 57,151 threw
URIError before and decode now.
…sage

When the entry or a module it imports fails to download, the module graph
never runs and LoadFailure cannot mount, so the page stayed blank.
public/boot-guard.js, a classic script from the app's own origin loaded
before the entry, shows the same translated message on the failed script's
error event, or after 30s with nothing run, into the empty #root. main.tsx
cancels it as soon as it runs, which removes the message if it is up, so a
boot that is only slow does not keep it. The CSP is unchanged. Lazy chunks
need no listener: Vite rethrows a failed preload unless vite:preloadError
is cancelled, and LoadBoundary catches it.
…ke cause

Traces of the failures show Chromium aborting in-flight requests with
net::ERR_NETWORK_CHANGED on a host whose docker interfaces churn, while boot
itself took 1 to 4s.
…ot one

"Postcards has moved" waits for the move screen to load, and "Done" for the
whole app to boot in the new tab and take the file; both ran past the 5s
default under full-suite load, with no network error in the trace.
The photo schema checked only the data:image/...; prefix, so a file
could bring a photo like data:image/png;base64,@@@. On a place it
failed the restore's save; on a story it was stored as is, and then
Save everything threw InvalidCharacterError. Such a photo now fails
validation with the normal import error, sync pulls included. A device
that already holds one keeps backing up: buildFile, which every
export, the archive and each sync push go through, leaves out any photo
that does not decode before it validates. A sync test fixture whose
second image was itself undecodable (ggA=) now uses ggA==.
The handoff's new confirm prompt takes the same counted phrases as the other
restore messages, so it reads "1 place" rather than "1 places".
French takes the singular for 0 (CLDR's "one" covers 0 and 1), which the
plural helper now follows, so the move message reads "0 voyage".
Brings in railway stations, the journal redesign (a place-optional postcard
composer), per-stop trip dates and the GeoNames region names.

- Schema: the story date keeps its real-day check, which now also holds for
  endDate, and an impossible per-stop trip date loads as that stop undated,
  as the trip date does.
- Publish: photos still come only from records inside the date range, and a
  place-less postcard adds none.
- Journal: the inline composer moved to StoryComposer. Removing a story keeps
  its undo through restoreStory. StoryComposer never mirrors an edit to the
  draft, so the untouched-Edit draft fix has nothing left to guard there, and
  its tests now check that composer; it offers no undo after a save, so the
  add and edit undo tests go.
- Trip composer: per-stop date inputs replace the year and month picker, and
  they keep a full day on their own, so the month-picker fix goes; its tests
  now check the day, a month-dated trip and per-stop dates. Moving a stop
  keeps focus on its arrow and now carries the stop dates with it.
- tripStops: legModeAt stays deleted, since only its test called it;
  setStopDate stays, since the composer calls it.
- myPlaces and StoryMap keep the shared coordsOf, which has the station branch.
- Places: stations count as a world browse for the filter panel. They are
  browsed in full, so the recorded-cities list needs no station twin.
- Stat strip: the cities counter keeps opening your visited cities, next to
  the new stations counter.
- Map: the load handler keeps main's resize guard and reads the theme from
  darkRef.
- i18n: the deleted keys stay deleted in all three catalogs, and the new
  shortcuts-help row reads new keys like the rest of that dialog.
- Tests: the backup round trip writes its story through the postcard
  composer, and main's new route-map and a11y tests use the shared helpers.
The merge of main gave the new W row of the shortcuts help keys in all three
catalogs; the French check now names that row's English too, so it fails if
the row goes back to hard-coded English.
Moving a stop with its arrows goes through the composer's move helper, which
kept focus on the arrow but built the chain from the stops and leg modes
alone, so once per-stop dates arrived every move would have wiped them. The
merge of main passes the whole chain; this test fails without that.
The postcard-trip, postcard-multi and trip-stopdates tests from main marked
their cities through search with a first-match click, the race that lets the
open results cover the next click: on the merged branch two of them timed out
on Reconstruct a journey behind the result list. None of them is about
marking, so they seed the cities with openAppWithVisits like the rest of the
suite.
The cache-age test waited for the caches to fill, then aged every entry the
expiration database held and expected four. The worker writes an entry's
timestamp after its cache write settles, so on a loaded host one had not
landed yet and the full run counted three. The test now ages again until it
finds all four.
An export keeps both filters: photos that do not decode and tags that clean
away. The poster reads the shared land loader, which already unwraps shapes
crossing the 180th meridian.
The sanitizer here keeps a leading dash, equals or at sign, so "-" is a
valid tag and a formula-leading title keeps its first character.
@davd-gzl
davd-gzl merged commit 402e5e0 into main Sep 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant