Background
The CLI command table on docs.prolific.com (docs/public_api/fern/docs/pages/tooling/cli.mdx in prolific-oss/prolific) is hand-maintained and drifts silently. It just took over a year to notice: it still listed the requirements command a year after removal, and was separately missing 9 other commands (aitaskbuilder, bonus, collection, feedback, filters, eligibility-count, invitation, researcher, reward-recommendations, survey, template) that had shipped since. Fixed manually in prolific-oss/prolific#16055, but nothing stops it drifting again.
Idea
Mirror the existing schema-drift-check.yml pattern in this repo (which diffs the CLI against the live OpenAPI spec and files a schema-drift issue on divergence), but for the command tree vs. the docs table instead of the API spec vs. the client.
Rough shape:
- This repo (public) exposes the current command list + descriptions in a machine-readable form — e.g. walk
rootCmd.Commands() and emit JSON, or reuse Cobra's own doc generation.
- A scheduled check (could live here or in
prolific-oss/prolific, since this repo is public either side can read it) fetches that manifest and diffs it against the docs table, filing an issue (or opening an auto-PR) on drift.
Open questions
- Which repo owns the check? The manifest's source of truth is here; the table it's checked against lives in the private
prolific-oss/prolific repo, so this needs a decision either way, plus whatever CI wiring the docs repo's checks/hooks require.
- Auto-PR vs. issue-only, matching or diverging from the existing
schema-drift-check.yml behavior.
Backlog item — not scoped/estimated yet.
Background
The CLI command table on docs.prolific.com (
docs/public_api/fern/docs/pages/tooling/cli.mdxinprolific-oss/prolific) is hand-maintained and drifts silently. It just took over a year to notice: it still listed therequirementscommand a year after removal, and was separately missing 9 other commands (aitaskbuilder,bonus,collection,feedback,filters,eligibility-count,invitation,researcher,reward-recommendations,survey,template) that had shipped since. Fixed manually in prolific-oss/prolific#16055, but nothing stops it drifting again.Idea
Mirror the existing
schema-drift-check.ymlpattern in this repo (which diffs the CLI against the live OpenAPI spec and files aschema-driftissue on divergence), but for the command tree vs. the docs table instead of the API spec vs. the client.Rough shape:
rootCmd.Commands()and emit JSON, or reuse Cobra's own doc generation.prolific-oss/prolific, since this repo is public either side can read it) fetches that manifest and diffs it against the docs table, filing an issue (or opening an auto-PR) on drift.Open questions
prolific-oss/prolificrepo, so this needs a decision either way, plus whatever CI wiring the docs repo's checks/hooks require.schema-drift-check.ymlbehavior.Backlog item — not scoped/estimated yet.