Add persistent covers and backgrounds for PS5 and Switch

deployment/README.md

@@ -68,6 +68,12 @@ require manual attention. Original download archives remain available for seedin
 
 ## Optional cover resolver
 
+SteamGridDB supplies portraits and hero backgrounds for PS5 and Switch, with
+official Steam library assets as a fallback for missing games/images and verified
+PlayStation Store products for configured console-only editions. Configure
+its separate provider key in `/opt/botty-artwork/steamgriddb-api-key` (root:botty-artwork,
+0640); see [key setup and cache behavior](botty-artwork/README.md).
+
 The resolver listens on `127.0.0.1:9697`; Nginx exposes only its authenticated
 artwork route. It uses the **same API key** as the console's Prowlarr configuration.
 
@@ -78,10 +84,13 @@ sudo apt-get install python3-pil
 sudo useradd --system --no-create-home --shell /usr/sbin/nologin botty-artwork
 sudo install -d -o root -g botty-artwork -m 0750 /opt/botty-artwork
 sudo install -m 0644 deployment/botty-artwork/server.py /opt/botty-artwork/server.py
-sudo install -m 0644 deployment/botty-artwork/sources.json /opt/botty-artwork/sources.json
+sudo install -m 0644 deployment/botty-artwork/steamgriddb-games.json /opt/botty-artwork/steamgriddb-games.json
+sudo install -m 0644 deployment/botty-artwork/playstation-games.json /opt/botty-artwork/playstation-games.json
 sudo install -m 0644 deployment/botty-artwork/aliases.json /opt/botty-artwork/aliases.json
 sudo install -o root -g botty-artwork -m 0640 /dev/null /opt/botty-artwork/api-key
 sudoedit /opt/botty-artwork/api-key
+sudo install -o root -g botty-artwork -m 0640 /dev/null /opt/botty-artwork/steamgriddb-api-key
+sudoedit /opt/botty-artwork/steamgriddb-api-key
 sudo install -m 0644 deployment/botty-artwork/botty-artwork.service /etc/systemd/system/
 sudo systemctl daemon-reload
 sudo systemctl enable --now botty-artwork
@@ -90,9 +99,10 @@ sudo systemctl status botty-artwork
 
 Skip user creation if the account already exists. Enter only the API key into the
 key file; do not put it in shell arguments or source control. On updates, preserve
-`api-key`, `aliases.json`, `sources.json` and `/var/cache/botty-artwork`; do not
-repeat the command that creates an empty key file. Source mappings may have
-local additions. See [resolver behavior](botty-artwork/README.md).
+both key files, `aliases.json`, `steamgriddb-games.json`, `playstation-games.json`
+and `/var/cache/botty-artwork`; merge new default mappings with operator additions.
+Do not repeat the command that creates an empty key file. Legacy `sources.json`
+mappings are left untouched. See [resolver behavior](botty-artwork/README.md).
 
 ## Updates and rollback
 

deployment/botty-artwork/README.md

@@ -1,78 +1,138 @@
-# Cover resolver
-
-Run `server.py` as the dedicated `botty-artwork` user with Python 3 and Pillow,
-using the systemd service and authenticated TLS nginx route shipped alongside it.
-The API key remains in `/opt/botty-artwork/api-key`, never in this repository.
-
-Scene suffixes such as PROPER and INTERNAL are removed only when the input has
-a PS5, CUSA/PPSA title-ID, [FPKG], UPDATE, `v<number>` (for example `v1.02`
-or `v10.02`), MULTI, or incl/including marker. The literal word `version` is
-not a marker. Release detection and metadata truncation share the same pattern;
-PS5, CUSA/PPSA IDs, and [FPKG] are also detected before bracket removal.
-Platform markers accept word boundaries, including bracketed IDs and hyphen,
-dot, or colon separators (for example `Proper-PPSA12345`). Scene suffixes
-also accept these separators when a release marker is present, so
-`Some Game-PROPER-PPSA12345` resolves to `Some Game`. UPDATE, `v<number>`, MULTI,
-and incl/including require preceding whitespace; `[FPKG]` does not.
-Metadata and edition suffixes are
-trimmed repeatedly in either order; unmarked canonical scene words are preserved.
-
-Resolution uses normalized game names (scene tags, editions, punctuation, accents,
-Roman numerals, trademark symbols), Steam game search/library artwork, Wikipedia
-exact article lookup followed by search, then a Steam store banner if no portrait
-is available. Matches must preserve the game identity and sequel number. Provider
-IDs, source URLs, canonical titles and outcomes are saved next to RGB images.
-Positive results last thirty days. Confirmed misses last one hour; network/provider
-errors are not negative-cached. Different titles can resolve concurrently; duplicate
-lookups coalesce and network concurrency is bounded to four.
-
-Near-uniform provider images are rejected, including grey CDN placeholders returned
-with HTTP 200. Existing cached placeholders are re-resolved rather than retained for
-thirty days. Battlefield 6 and God of War Sons of Sparta have verified PlayStation
-product mappings in the seed file.
-
-For regional names or verified exceptions, `/opt/botty-artwork/aliases.json` can map
-a normalized title to a canonical game name. It is read on each request and survives
-updates. Example: `{"regionalname":"Canonical Game"}`. No arbitrary image URL is
-accepted from the client. HTTPS destinations and redirects are allowlisted, response
-sizes and image dimensions are bounded. Images retain their complete composition.
-
-Merge the seed `aliases.json` into the installed alias file when deploying, preserving
-operator entries. It covers the eFootball 2027 seasonal name, Sonic Unleashed's
-unofficial recompilation port, and Resident Evil 4's explicit remake release name.
-The latter uses the verified 2023 PlayStation product in `sources.json`.
-It also restores the missing ampersands in The Bearer & The Last Flame and
-Snoopy & The Great Mystery Club, and maps WILD HEARTS Karakuri Edition to its game.
-
-Verified PlayStation product pages can be registered in `/opt/botty-artwork/sources.json`
-(mapping normalized game names to official product URLs; seed file provided). The
-resolver reads Product JSON-LD, verifies the title and sequel, and downloads only
-from allowlisted PlayStation origins. Publisher prefixes such as EA SPORTS and
-trademark marks do not change identity. This mapping survives service upgrades.
-
-Release 1.5.2 includes verified PS5 product mappings for God of War Sons of Sparta,
-Resident Evil 4 (2023), Infliction: Extended Cut, Snoopy & The Great Mystery Club,
-and Crazy Chicken Shooter Edition. Keep `Shooter Edition` in Crazy Chicken's name:
-it identifies the product and is not a generic edition suffix.
-
-To update an existing installation, retain rollback copies of `server.py`,
-`aliases.json` and `sources.json`, replace the server code, and merge both seed
-JSON objects into their installed counterparts. Preserve existing operator entries
-outside the updated keys. Keep the API key, cache ownership and service user.
-Restart only `botty-artwork` if the server code changed; mappings are read on each
-request. An alias changes the canonical cache key automatically. For a newly
-mapped product with the same canonical name, let its one-hour negative cache expire
-or back up and remove only that title's empty RGB cache and matching JSON metadata.
-On the console, failed lookups retry after one minute; change Explore pages and
-return to request the artwork again. No console binary update is required.
-
-Coverage cannot be guaranteed for every release: games missing from both sources
-retain an explicit title placeholder. Do not silently substitute another sequel.
-The audit and cached metadata identify unresolved names for later corrections.
-
-References:
-- https://partner.steamgames.com/doc/store/assets/libraryassets
-- https://www.mediawiki.org/wiki/API:Pageimages
-
-Run `python3 test_server.py` to check matching, cache reuse, fallback, outage retry,
-origin restrictions and persistent aliases without public network requests.
+# Game artwork resolver
+
+SteamGridDB is the primary image provider for PS5 and Switch. If the game or
+requested image is unavailable there, the resolver uses official Steam library
+assets as a fallback. Operator-verified PlayStation Store products provide a
+final fallback for the configured console-only releases/editions. The authenticated
+`/artwork?title=...&kind=cover|hero` route returns RGB bytes to the console service:
+
+- `cover` prefers static **600×900** grids, also accepting **660×930** and
+  **342×482** portraits when needed. Every image is fitted into **160×240**;
+  landscape banners and square icons are excluded.
+- `hero` selects static **1920×620**, **3840×1240** or **1600×650** heroes and fits
+  them into a **640×360** canvas while preserving the complete composition, with
+  blurred fill above/below wide artwork. Up to three top-ranked heroes are checked
+  to prefer illustration visible on the right, away from the UI text scrim.
+
+Steam fallback matches the complete game name, fetches typed game metadata from
+`IStoreBrowseService/GetItems`, and uses `library_capsule[_2x]` or
+`library_hero[_2x]`. Modern per-image hash paths are read from the metadata; old
+hard-coded CDN paths can return 404 even for available artwork. Steam requires
+no additional API key. Store banners, demos, DLC and unrelated annual editions
+are never substituted for the game portrait. All providers share the same
+160×240 / 640×360 output dimensions and persistent caches.
+
+For PlayStation Store, `/opt/botty-artwork/playstation-games.json` maps normalized
+names to verified HTTPS product URLs (override with `BOTTY_PLAYSTATION_GAMES`).
+NHL 27 and Dragon’s Dogma 2: Dark Arisen are included. The resolver checks the
+JSON-LD product name/SKU and reads only that exact product’s media from the
+embedded store data. `PORTRAIT_BANNER` supplies the cover; `GAMEHUB_COVER_ART`,
+`EDITION_KEY_ART` and `BACKGROUND` supply the background. Concept/base-game,
+upsell, screenshot, logo and square master images are excluded. This preserves
+the actual Dark Arisen edition rather than selecting the original game’s art.
+No PlayStation account, session or API key is required. Product media metadata
+and resulting images persist in the existing caches.
+
+The app uses a blurred cover behind its text if neither provider has a hero. It keeps a
+uniform title placeholder if there is no portrait; square Switch icons, Steam
+banners and Wikipedia images are no longer substituted.
+
+## API key
+
+On the artwork host (the current installation uses SSH alias `test`), put only the
+SteamGridDB API key in **`/opt/botty-artwork/steamgriddb-api-key`**. The service reads
+it on each uncached API lookup; replacing the key needs no restart.
+
+```sh
+sudo install -o root -g botty-artwork -m 0640 /dev/null /opt/botty-artwork/steamgriddb-api-key
+sudoedit /opt/botty-artwork/steamgriddb-api-key
+```
+
+Run the first command only when creating the file; it would erase an existing key.
+Generate a key at https://www.steamgriddb.com/profile/preferences/api . To use another
+path, set `BOTTY_STEAMGRIDDB_KEY_FILE` in the systemd unit's environment. Keep this
+key on the server, outside source control and console packages. The existing
+`/opt/botty-artwork/api-key` is a separate key: it still authenticates the console's
+Prowlarr/artwork proxy requests.
+
+## Matching and caches
+
+Release metadata, regions, editions, accents, punctuation and Roman numerals are normalized.
+Switch scene suffixes (`NSW-GROUP`, `NSW Switch`) and PS5 VR markers are removed.
+Only a parenthesized year disambiguates releases; numbers inside a game name,
+such as Cyberpunk 2077, remain part of its identity. Canonical SteamGridDB names
+are compared without release cleanup so editions, DLC and fan demakes stay distinct,
+including qualifiers in parentheses. Only explicit release-year parentheses are
+ignored when comparing names.
+A match must preserve title/sequel identity. Multiple matching games (for example
+a remake) require an explicit year in the canonical alias or a game-ID override.
+Missing conjunctions and articles can be normalized and a distinctive subtitle
+can be queried again, while the complete game identity still has to match.
+No first-result fuzzy match is silently accepted.
+
+`/opt/botty-artwork/aliases.json` maps normalized names to canonical titles. A
+`ps5:` prefix limits an alias to explicitly marked PS5 releases, allowing the
+correct Resident Evil 4 / Demon’s Souls remake without affecting Switch names.
+Optional `/opt/botty-artwork/steamgriddb-games.json` maps normalized titles to
+positive numeric SteamGridDB game IDs, for example `{"astrobot": 5454475}`. These
+operator files are read on lookup and survive upgrades. `sources.json` from the
+previous resolver is preserved but no longer used. When changing an override,
+back up/remove that title's `sgdb-v1` cache entries to apply it immediately.
+
+Successful Steam game metadata is cached alongside SteamGridDB matches. Removed Steam image URLs
+trigger fresh asset metadata rather than pinning a stale hash directory.
+Server matches, portraits and heroes persist in `/var/cache/botty-artwork`.
+Positive images and game matches last thirty days; confirmed misses last an hour.
+Authentication, rate limits and network errors are retried rather than saved as
+missing artwork. A canonical-title lock coalesces concurrent cover/hero lookups and unique
+temporary files make metadata/image publication atomic across processes; network
+concurrency is bounded to four. Credentials never enter saved metadata. Images,
+response sizes and HTTPS origins/redirects are checked, including rejecting
+landscape grids, animated images and blank CDN tiles.
+
+The console service uses separate `cache/sgdb-covers-v1` and
+`cache/sgdb-heroes-v2` directories under its root, bounded to 512 portraits and 128
+heroes. Full release names reach the resolver so scene markers and DLC Unlocker
+suffixes can be cleaned correctly. The native app has its own persistent `/download0/botty-sgdb-covers-v1.cache`
+and `/download0/botty-sgdb-backgrounds-v2.cache` files with the same capacities
+(about 56 and 84 MiB). They work after closing/reopening the app, without a service
+request for a cached image. Least recently used native entries are replaced;
+checksums reject damaged or interrupted writes. Delete these two files with the
+app closed to reset the native caches. The new namespaces automatically bypass
+older mixed-format caches; those old files can be removed later with the app
+closed. They are not deleted by this update.
+
+## Deployment and checks
+
+Use Python 3, Pillow, the dedicated `botty-artwork` user and the supplied systemd
+unit. The listener stays on `127.0.0.1:9697`; nginx exposes only its authenticated
+HTTPS route. Preserve both key files, aliases, game-ID overrides, ownership and
+rollback copies when updating. Deploy the matching console service/native app to
+enable Switch portraits and heroes throughout the UI. Do not interrupt active
+extraction/compression or replace an open native title.
+
+Run `python3 deployment/botty-artwork/test_server.py` for deterministic provider,
+matching, format, cache, secret-handling and outage checks, without public requests.
+See https://www.steamgriddb.com/api/v2 for the provider API.
+
+
+## Audit real Prowlarr results
+
+Keep raw Prowlarr responses and credentials in ignored private backups. Export a
+sanitized JSON array containing only title strings or objects with a `title` field.
+On the artwork host, run under the artwork service user:
+
+```sh
+sudo -u botty-artwork python3 audit.py --resolver /opt/botty-artwork/server.py \
+  --corpus /private/prowlarr-titles.json --images --output /private/artwork-audit.json
+```
+
+The audit checks every release, downloads/verifies the final portrait with
+`--images`, and records matched game IDs, missing portraits, unresolved candidates
+and request failures separately, including the actual image provider. It never downloads torrents or changes console
+files. At most two workers run. Known multi-game library/collection packs are reported as `not-single-game`.
+Reports contain no credentials or download URLs;
+existing output files are not overwritten. Game-library bundles and releases for
+other consoles can appear in category 1000; keep these distinct when interpreting
+coverage of individual PS5/Switch games. Do not substitute a previous sequel or
+a different game just to eliminate a missing-artwork result.

deployment/botty-artwork/aliases.json

@@ -5,5 +5,39 @@
   "residentevil4remake": "Resident Evil 4 (2023 video game)",
   "thebearerthelastflame": "The Bearer & The Last Flame",
   "snoopythegreatmysteryclub": "Snoopy & The Great Mystery Club",
-  "wildheartskarakuriedition": "WILD HEARTS"
+  "wildheartskarakuriedition": "WILD HEARTS",
+  "ps5:residentevil4": "Resident Evil 4 (2023)",
+  "ps5:demonssouls": "Demon's Souls (2020)",
+  "ps5:kaothekangaroo": "Kao the Kangaroo (2022)",
+  "ps5:monopoly": "Monopoly 2024",
+  "dispatch": "Dispatch (2025)",
+  "constance": "Constance (2025)",
+  "blueysquestforthegoldpen": "Bluey's Quest for the Gold Pen (2025)",
+  "marioplusrabbidskingdombattle": "Mario + Rabbids Kingdom Battle",
+  "rayman30th": "Rayman: 30th Anniversary Edtion",
+  "akibalost": "AKIBA LOST - アキバロスト",
+  "viractalwillyoutrustyourparty": "VIractal",
+  "subwaysimulatorundergroundtrainridestationultimatedrivinggames": "SUBWAY SIMULATOR: UNDERGROUND TRAIN RIDE",
+  "nordicashesthecompletesaga": "Nordic Ashes",
+  "tombraideri3remasteredstarringlaracroft": "Tomb Raider I-III Remastered",
+  "metalgearsolidmastercollectionvolume1": "Metal Gear Solid: Master Collection Vol. 1",
+  "horizoncallofthemountain": "Horizon VR: Call of the Mountain",
+  "bloodraynerevamped": "BloodRayne",
+  "bloodrayne2revamped": "BloodRayne 2",
+  "crimsondesertenhanced": "Crimson Desert",
+  "legomarvelsuperheroes2": "LEGO Marvel Super Heroes 2",
+  "nintendo64nintendoswitchonlinemature17plus": "Nintendo 64: Nintendo Switch Online - Mature 17+",
+  "nintendo64nintendoswitchonline18plus": "Nintendo 64: Nintendo Switch Online 18+",
+  "thelastoricrufinalcut": "The Last Oricru",
+  "talesofeterniaremastered": "Tales of Eternia",
+  "fishingvalleyremastered": "Fishing Valley",
+  "theprinceoftennissweetschoolfestivallove40andmore": "The Prince of Tennis Sweet School Festival 0-40 and more...",
+  "guiltygearstrivenintendoswitcheditionggstadditionalcharactercolorpack": "Guilty Gear: Strive",
+  "guiltygearstrivenintendoswitcheditioncht": "Guilty Gear: Strive",
+  "castlevaniaanniversarycollectioneshop": "Castlevania Anniversary Collection",
+  "supernintendoentertainmentsystemnintendoswitchonlineinternaleshop": "Super Nintendo Entertainment System: Nintendo Switch Online",
+  "nintendoentertainmentsystemnintendoswitchonlineinternaleshop": "Nintendo Entertainment System: Nintendo Switch Online",
+  "minecraftnintendoswitcheditioneshop": "Minecraft: Nintendo Switch Edition",
+  "fc2027": "EA SPORTS FC 27",
+  "bluereflectionquartlet": "BLUE REFLECTION Quartet"
 }