VGMdb

VGMdb.net is an excellent resource, containing gigabytes of information about video game music. It documents the relationships between games, albums that represent games, and all the people that contributed to an album.

However, VGMdb does not provide an API to access its information, which provides an obstacle to programs that want to use its information. This site is a way to programmatically access the information on VGMdb.net.

VGMdb.net stores the information of each item at a friendly URL, such as album/79 or artist/137. This friendly url will work on this site. Once there, an overview of the item's information can be found, along with a few extra features.

Self-hosted setup

Needs docker and the compose plugin.

curl -fsSL https://raw.githubusercontent.com/salarx/vgmdb/selfhosted/install.sh | sh

Clones to ~/vgmdb (VGMDB_DIR to change) and asks how to set it up. Pass setup.sh options through to skip the questions:

curl -fsSL .../install.sh | sh -s - --with-byparr --non-interactive

From a checkout

./setup.sh                                        # interactive
./setup.sh --with-byparr                          # run Byparr alongside
./setup.sh --solver-url http://10.0.0.5:8191      # use an existing solver
./setup.sh --no-solver --cookie "cf_clearance=…"  # manual cookie

--help lists every option. Serves on port 8085, settings at /settings.

Configuration

Environment variables. The first two can also be set on /settings, taking effect immediately without a restart.

Variable	Purpose
USER_COOKIE	Cookie header sent to vgmdb.net, normally just cf_clearance=…
USER_AGENT	User-Agent paired with that cookie
BYPARR_URL	FlareSolverr/Byparr endpoint. Unset disables automatic refresh
BYPARR_TIMEOUT	Seconds to allow for a solve (default 120)
RUNTIME_CONFIG_PATH	Where /settings persists overrides; needs a writable volume

About the Cloudflare cookie

cf_clearance is bound to the IP, User-Agent and TLS fingerprint of whatever solved the challenge, and is rotated as that browser keeps using the site.

• Cookie and User-Agent only work as a pair; a mismatch is refused exactly like sending nothing.
• A token copied from your desktop browser is usually refused on a server, and is short-lived even when it works.
• Only cf_clearance is needed, not your login cookies.

A solver sidesteps all of it, and re-solves when vgmdb.net starts refusing requests — once per expiry, not per lookup.

Usage

In the right sidebar, there are links to view the page's information in different formats. The raw information that has been parsed out of the original VGMdb.net item is available in JSON format by adding ?format=json to any URL. The YAML format is also available for this information, available by adding ?format=yaml.

Additionally, an effort has been made to produce RDF tuples representing some of the information in VGMdb. The default HTML view of the data has RDFa encoded into it, allowing any RDFa library to parse the page. Additionally, RDF/XML and Turtle serialization formats are available.

The page will produce the output format that most-closely matches the request's HTTP Accept header. A particular format can be forced by adding ?format=html, ?format=xml, or ?format=turtle arguments to the URL

There are also links to validation services, to verify that the page's data is in fact valid and parseable in the respective formats. The main page doesn't produce any other formats, but every other page does.

Authentication

Requests need a cf_clearance token. Set BYPARR_URL and it is obtained and refreshed for you — see Self-hosted setup.

To supply one by hand, paste it on /settings or set USER_COOKIE:

• Login to https://vgmdb.net/
• Open Web Developer Tools by right clicking and selecting Inspect
• Switch to the Network tab in the dev tools subwindow and reload the page
• Select a request to a document/HTML and within the Request Headers you should find the Cookie header and the value should be a pretty large string looking something like: cf_clearance=...; vgmsessionhash=...;
• Copy the cf_clearance=... portion and set it as USER_COOKIE

Set USER_AGENT to the same browser's User-Agent, or the pair will be refused. Expect to repeat this every few hours.

Development and Installation

Unit tests are employed, both in the parsing and in the output stages, to validate that the information is being correctly parsed and encoded. Run the tests.sh script to run the tests.

Most of any runtime requirements should be documented in requirements.txt. Use pip install -r requirements.txt to install them.

The run.py example script will start up an HTTP server process for the site. An example Apache2 configuration, using mod_wsgi, is also provided. It should work just as well under any other WSGI server.

The docker/docker-build.sh script should produce a vgmdb_reqs base image with the dependencies and a vgmdb image with the current version of the code installed. This image exposes port 80, and vgmdb/autoload_settings.py implements loading optional dependencies and settings based on environment variables.

Dockerfile.slim builds an alternative image on top of vgmdb_reqs that runs gunicorn alone — no nginx, runit or celery workers — and serves on port 9990. It is what setup.sh deploys. The upstream layout splits the web interface and the background workers so a public deployment can scale them separately; a single-user instance does not need that, and nginx's bot blocklist rejects Python clients, which is usually the point of running this at all.