https://nathanneurotic.github.io/PS2SP/
PS2SP is a community-oriented index of PlayStation 2 software and the wider PS2 ecosystem.
- keep software project records in plain, reviewable Markdown;
- keep GitHub repository/release metadata synchronized automatically;
- discover current and historical PS2 software without brute-force rescanning GitHub;
- index non-repository PS2 resources such as communities, documentation, archives, hardware, mods, online services, and media;
- deploy as a static Astro site on Cloudflare Pages;
- keep Git as the durable source of truth;
- require no application database or always-on backend.
software discovery + known upstreams PS2 ecosystem directory
| |
v v
content/projects/*.md data/resources/directory.json
\ /
\ /
+------------ Astro ---------------+
|
v
static site + JSON API
|
v
Cloudflare Pages
Human-written project metadata stays separate from synchronized repository fields. Automated jobs can refresh source activity, releases, repository statistics, and cache validators without overwriting curated descriptions.
Requires Node.js 22.12 or newer.
npm install
npm run devProduction build:
npm run buildThe generated site is written to dist/.
Every npm run build also checks all generated assets against Cloudflare Pages'
25 MiB per-file limit, including builds in scheduled catalog workflows.
The resource API is paginated: start at /api/resources.json, consume its
groups, and follow next until it is absent. Subsequent URLs have the form
/api/resources/1.json. Pages target at most 4 MiB of UTF-8 JSON; categories
can span pages, so concatenate links for matching categories. Consumers that
previously assumed a single response must follow next to retrieve the complete
catalog. The resource page does this automatically and only enables the complete
index after every page loads successfully.
Run npm test for offline discovery regression tests. These use simulated hosts
and temporary catalogs, so they do not modify the live catalog or depend on
external services. CI runs them on every pull request and before external Git
discovery.
External Git discovery (.github/workflows/external-git-discovery.yml) runs
once a week. Nearly all of its sources are on git.techwritescode.dev, a small
personal server, so each run attempts to check the current queue within its
repository and request budgets instead of the job running hourly. It retries timeouts, network failures, HTTP 408/429, and
server errors once. If the retry fails, that host is deferred for the remainder
of the run while other hosts continue. Each attempt counts against the request
budget, and the 30-second deadline covers both headers and the JSON body.
Warnings identify the failed endpoint; discovery/external-git-state.json records
lastRun.failedHosts. A successful job with deferred hosts means partial discovery,
not that every host was reachable. Existing links and unfinished branch scans
are preserved for later scheduled runs. Authentication errors and invalid JSON
still fail the job and require investigation.
Use:
- Production branch:
main - Build command:
npm run build - Build output directory:
dist
PS2SP separates known-project synchronization from new-project discovery.
.github/workflows/catalog-sync.yml runs hourly. The registry is split into 24 deterministic shards, so each GitHub-backed project is normally checked about once per day.
GitHub ETags are stored and reused through If-None-Match. Unchanged resources return 304 Not Modified, avoiding unnecessary processing and catalog commits.
When synchronization code itself changes on upstream main, the workflow performs one bounded forced full refresh. This is used to apply metadata migrations and cleanup without waiting for every shard to cycle.
.github/workflows/catalog-discovery.yml runs every two hours with a hard request budget.
Discovery combines:
- trusted PS2 organizations and developers;
- incremental recent searches using a persisted watermark;
- a resumable historical bootstrap scanning backward in bounded monthly windows.
Forks are intentionally stricter than normal repositories. Copied upstream README text is not enough to auto-publish a fork; a fork needs PS2-specific identity in its own name/topics/release metadata, or trusted/manual curation.
Age and inactivity are not exclusion criteria. Archived, dormant, discontinued, unreleased, and historical PS2 software can still be indexed when relevance is established.
The workflows support the repository-scoped GITHUB_TOKEN. For additional headroom, add:
PS2SP_GITHUB_TOKEN
The code enforces request budgets and reserve thresholds independently of the token's maximum quota.
Local commands:
GITHUB_TOKEN=... npm run catalog:sync
GITHUB_TOKEN=... npm run catalog:discover
npm run catalog:promote-pendingPending discovery records are reviewed automatically by the promotion step. Released projects can be promoted at lower confidence; recent unreleased projects require much stronger PS2-specific evidence. Explicit WIP, archived, dormant/discontinued, profile, documentation-only, and list/database repositories remain in discovery/pending/ for manual review.
Force a local complete refresh:
GITHUB_TOKEN=... SYNC_ALL=true FORCE_REFRESH=true npm run catalog:syncAutomatically discovered entries retain their discovery method and confidence. Low-confidence candidates remain outside the public catalog until evidence improves or a maintainer reviews them.
Maintained forks that later prove unrelated to PS2 can be hidden automatically without destroying their record, while curated or verified projects are preserved.
Unknown dates are shown as unknown rather than fake epoch dates.
General ecosystem resources live in:
data/resources/directory.json
This is PS2SP-owned catalog data and can include websites, communities, archives, documentation, hardware resources, media, downloads, and other useful PS2 material regardless of where it is hosted.
The build exposes:
/api/projects.json/api/categories.json/api/releases.json/api/resources.json
Site source and catalog metadata can be licensed separately from third-party software. PS2SP does not redistribute third-party binaries by default; software download links point to upstream project infrastructure.
The fork syncs upstream daily at 04:00 UTC and can be synced manually with the
Sync Fork & Deploy to GitHub Pages workflow. Sync merges upstream into the fork
without rebasing or resetting fork history. A conflict aborts the merge and fails
the job; resolve it in a pull request, preserving the mirror configuration.
Tests, UI checks, and the complete mirror build run before a sync is pushed.
A concurrent change to main causes the normal push to fail safely; rerun the job.
For local mirror validation, set BASE_PATH=/PS2SP and run
npm run build:gh-pages. This runs the normal asset audit, prepares Pages output,
checks final asset sizes again, and loads every resource API page under the mirror
base path. Pull request CI runs this same build before changes can be reviewed.