diff --git a/hydra-gates/MIGRATION-check-l10n.md b/hydra-gates/MIGRATION-check-l10n.md new file mode 100644 index 00000000..86feca82 --- /dev/null +++ b/hydra-gates/MIGRATION-check-l10n.md @@ -0,0 +1,90 @@ + + +# Moving off your vendored check-l10n.js + +Gate 117 runs `hydra-gates/scripts/check-l10n.js` on every app. You do not have to +do anything for the gate to work. This note is for the second step: deleting the +copy in your own repository, so there is one checker instead of fifteen. + +## Why the copies have to go + +Twenty-one apps vendored this script. By 2026-09-19 those copies had drifted into +thirteen distinct versions across fifteen repositories. Three apps ship none at +all. Three carry two copies each, at two different paths. + +Drift was not the worst of it. Every one of the thirteen computed `missing` the +same way: + +```js +const missing = [...usedKeys].filter((k) => !keys.has(k)) +``` + +`usedKeys` came from walking `src/` for `t()` calls. PHP and schema JSON, where +the better copies read them at all, only ever cleared an "unused" warning. So a +`->t('Approve')` with no key in `en.json` could not be reported by any app in the +fleet. On opencatalogi that hid 49 PHP strings and 319 register and schema +strings, with the local check green throughout. + +## What the shared checker reads + +| Source | Where | Feeds | +|---|---|---| +| `SRC` | `src/**/*.{vue,js,ts}`, `t()` and `n()` | missing and unused | +| `PHP` | `lib/`, `templates/`, `appinfo/`, `->t()` and `->n()` | missing and unused | +| `MANIFEST` | `src/manifest.json` and `src/manifest.d/*.json` | missing and unused | +| `SCHEMA` | `lib/Settings/**/*.json`, register and schema `title` and `description` | missing and unused | + +A PHP array value under a rendered field name, such as `'description' => '...'`, +clears an unused warning but never raises a missing one. A `->t()` call is an +unambiguous claim that a string is user facing. An array key is not. + +## Migrating your app + +1. Run the shared checker against your working tree and read the count: + + ```bash + node vendor/conduction/hydra-gates/hydra-gates/scripts/check-l10n.js . + ``` + + Findings print one per line, each tagged `SRC`, `PHP`, `MANIFEST` or `SCHEMA`. + Exit code 1 means findings, 4 means there was nothing to check, 9 means the + checker could not read something and judged nothing. + +2. Compare it against your vendored copy. The `SRC` counts should agree. If they + do not, say so in your PR: that is drift worth knowing about, not a reason to + stop. + +3. Fix what belongs to your current change. Everything else is inherited debt: + report it in one sentence in the PR body and leave it to the debt sweep. + +4. Park the findings you are not fixing today in `l10n/.l10n-source-ignore.json`, + a flat map of string to reason: + + ```json + { + "Geospatial": "a taxonomy value from the source register, not app copy" + } + ``` + + A reason is required. An entry with an empty reason is ignored, so the file + cannot quietly become a suppression list. + +5. Delete your own `scripts/check-l10n.js` or `tests/l10n/check-l10n.js`, and + point the `check:l10n` script in `package.json` at the shared one. + +6. Leave `check-l10n-parity.js` alone. It compares `nl.json` to the generated + `nl.js` and answers a different question. + +## Blocking, later + +Gate 117 warns. It does not fail a build, because fourteen of twenty-one repos +carry inherited findings and openregister alone carries 1,273. A blocking launch +would redden most of the fleet the minute it merged. + +Promotion to blocking is two deliberate edits in `scripts/run-hydra-gates.sh`: +drop `--warn-only` from the invocation, and swap `_warn` for `_fail`. Do it when +the fleet count is low enough that the next red is a regression rather than a +backlog. The l10n debt sweep owns that call. diff --git a/hydra-gates/README.md b/hydra-gates/README.md index 2b97b8ac..688cc834 100644 --- a/hydra-gates/README.md +++ b/hydra-gates/README.md @@ -95,6 +95,14 @@ Then `composer update conduction/hydra-gates`. `vendor/conduction/hydra-gates` lands at about 1.2 MB. The org profile, the website and the docs tree are `export-ignore`d and do not follow. +### Retiring your vendored `check-l10n.js` + +Gate 117 runs the shared `scripts/check-l10n.js`, which reads PHP `t()` calls and +`lib/Settings` schema JSON as sources of a MISSING translation, not only of an +unused one. Every per-app copy could only ever read `src/`. See +[MIGRATION-check-l10n.md](MIGRATION-check-l10n.md) for how an app moves across +and what to do with the findings it inherits. + ### Upgrading to `v1.1.0` from `v1.0.x` `^1.0` picks this up on the next `composer update`, and **verdicts move**. Three diff --git a/hydra-gates/scripts/check-l10n.js b/hydra-gates/scripts/check-l10n.js new file mode 100755 index 00000000..1a21f00a --- /dev/null +++ b/hydra-gates/scripts/check-l10n.js @@ -0,0 +1,617 @@ +#!/usr/bin/env node +/* SPDX-FileCopyrightText: 2026 Conduction B.V. */ +/* SPDX-License-Identifier: EUPL-1.2 */ +/** + * check-l10n.js — one shared translation-coverage checker for the fleet. + * + * WHAT IT IS FOR + * -------------- + * Every user-visible string an app ships should reach the English catalogue + * (l10n/en.json) and, for a Dutch audience, the Dutch one (l10n/nl.json). A + * string that reaches neither renders its source text, silently, in every + * locale. Nothing else in the pipeline notices: `check:l10n-js` compares + * nl.json to the generated nl.js, and a string absent from BOTH is perfectly + * in sync. + * + * THE DEFECT THIS EXISTS TO FIX + * ----------------------------- + * Twenty-one apps vendored a copy of this script, and every one of them + * computed `missing` from `src/` t() calls alone: + * + * const missing = [...usedKeys].filter((k) => !keys.has(k)) + * + * where `usedKeys` came only from walking .vue/.js/.ts. PHP and schema JSON + * were, at best, added to the list of things that SUPPRESS an "unused" + * warning. So a `->t('…')` in a controller could never produce a "missing" + * finding, and a schema `title` could not either. No app in the fleet could + * see a server-side or schema string that had reached no catalogue at all. + * + * Measured on opencatalogi at development@4af8e55a: 49 strings passed to a PHP + * translate call and 319 register/schema strings have no key in en.json. Its + * own vendored copy reports zero, and the src/ leg really is clean, so the 368 + * are not a stricter reading of the old scope. They are the new sources. + * + * Here `missing` is computed from ALL FOUR sources, and each finding carries + * the origin that produced it. + * + * FOUR SOURCES + * ------------ + * SRC src/ *.vue, *.js, *.ts — t(), n(), $t(), $n() literal calls + * PHP lib/, templates/, appinfo/ — ->t('…') and ->n('…') + * MANIFEST src/manifest.json and src/manifest.d/*.json — the fields a + * renderer walks (CnAppNav labels, page titles, walkthrough copy) + * SCHEMA lib/Settings/**\/*.json — register and schema title/description, + * including per-property, which OpenRegister renders in forms and + * detail pages + * + * All four feed MISSING, and all four suppress UNUSED. + * + * ONE DELIBERATE EXCEPTION. A PHP array value under a rendered field name + * (`'description' => '…'`) suppresses UNUSED but does NOT create MISSING. + * `->t('Approve')` is an unambiguous claim that a string is user-facing; a + * `description` key in an array is not. Measured on opencatalogi 2026-09-19: + * treating array values as missing-creating added 63 findings, and the sample + * was dominated by MCP tool descriptions an agent reads and no person sees. + * Precision here is worth more than reach, because a gate whose first run is + * mostly noise gets excluded rather than fixed. + * + * WARNING FIRST + * ------------- + * `--warn-only` makes the script exit 0 whatever it finds. Gate 117 passes it, + * because fourteen of twenty-one repos carry inherited findings and a blocking + * launch reddens them the minute it merges. The findings are printed either + * way; only the exit code is held back. + * + * USAGE + * node check-l10n.js [app-root] [--warn-only] [--json] [--source=SRC,PHP,...] + * + * EXIT CODES + * 0 no findings, or --warn-only + * 1 findings + * 4 empty scope: no catalogue, or no source string anywhere + * 9 could not read something it needed (a crash is not a finding) + */ + +'use strict' + +const fs = require('fs') +const path = require('path') + +const ALL_SOURCES = ['SRC', 'PHP', 'MANIFEST', 'SCHEMA'] + +// Fields a renderer walks and translates by literal lookup. Shared by the +// manifest collector and the PHP-array collector, because a PHP array using +// `label` / `description` is the same kind of thing: data, not code. +const RENDERED_FIELDS = [ + 'title', + 'body', + 'task', + 'label', + 'description', + 'emptyText', + 'placeholder', + 'subtitle', + 'helpText', + 'allLabel', +] + +// --------------------------------------------------------------------------- +// tiny helpers +// --------------------------------------------------------------------------- + +function walk(dir, exts) { + const out = [] + if (!fs.existsSync(dir)) { + return out + } + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + if (entry.name === 'node_modules' || entry.name === 'vendor' || entry.name.startsWith('.')) { + continue + } + const full = path.join(dir, entry.name) + if (entry.isDirectory()) { + out.push(...walk(full, exts)) + } else if (exts.some((e) => entry.name.endsWith(e))) { + out.push(full) + } + } + return out +} + +function readJson(file) { + return JSON.parse(fs.readFileSync(file, 'utf8')) +} + +/** + * Keys of a Nextcloud catalogue file. + * + * `{ "translations": { key: value } }` is the shape these apps ship. A plain + * flat map is accepted too, because a few older apps still write one. + * + * @param {string} file path to en.json / nl.json + * @return {Map} key to translated value + */ +function loadCatalogue(file) { + const out = new Map() + if (!fs.existsSync(file)) { + return out + } + const raw = readJson(file) + const body = raw && typeof raw === 'object' && raw.translations ? raw.translations : raw + if (!body || typeof body !== 'object') { + return out + } + for (const [k, v] of Object.entries(body)) { + if (typeof v === 'string') { + out.set(k, v) + } else if (Array.isArray(v)) { + // Plural form: `["one", "other"]`. The key is still the key. + out.set(k, v[0] ?? '') + } + } + return out +} + +// --------------------------------------------------------------------------- +// SRC — t() / n() in .vue, .js, .ts +// --------------------------------------------------------------------------- + +/** + * Read a quoted literal starting at the opening quote. Returns null when the + * literal is unterminated, spans a newline, or is a template literal with an + * interpolation — none of those is a static key this checker can trust. + * + * @param {string} text source + * @param {number} start index of the opening quote + * @return {{value: string, end: number}|null} the literal and its closing index + */ +function readLiteral(text, start) { + const quote = text[start] + if (quote !== "'" && quote !== '"' && quote !== '`') { + return null + } + let i = start + 1 + let value = '' + while (i < text.length) { + const c = text[i] + if (c === '\\' && i + 1 < text.length) { + const n = text[i + 1] + if (n === 'u' && /^[0-9a-fA-F]{4}$/.test(text.slice(i + 2, i + 6))) { + value += String.fromCharCode(parseInt(text.slice(i + 2, i + 6), 16)) + i += 6 + continue + } + const simple = { n: '\n', t: '\t', r: '\r' } + value += simple[n] ?? n + i += 2 + continue + } + if (c === quote) { + return { value, end: i } + } + if (quote !== '`' && c === '\n') { + return null + } + if (quote === '`' && c === '$' && text[i + 1] === '{') { + return null + } + value += c + i += 1 + } + return null +} + +/** + * Every string the frontend passes to t() or n(). + * + * `n()` matters as much as `t()`: an extractor that matched only `t(` reported + * every plural key as unused, which is how a clean-up script came to be armed + * to delete live keys from all 37 locale files. + * + * @param {string} root app root + * @param {string} appId the app id used as the first argument + * @return {Set} strings the frontend translates + */ +function collectSrcStrings(root, appId) { + const used = new Set() + const srcDir = path.join(root, 'src') + // `(?t() / ->n(), and rendered array values +// --------------------------------------------------------------------------- + +/** + * Every string a server-side translate call passes. + * + * This is one of the two sources the vendored copies could only ever use to + * suppress an "unused" warning. Here it also produces "missing", which is the + * whole point: a `->t('Approve')` in a controller with no key in en.json is a + * string that renders English to every reader and nothing reports it. + * + * @param {string} root app root + * @return {Set} strings passed to a server-side translate call + */ +function collectPhpTranslated(root) { + const used = new Set() + const patterns = [ + /->[tn]\(\s*'((?:\\.|[^'\\])*)'/g, + /->[tn]\(\s*"((?:\\.|[^"\\])*)"/g, + ] + for (const sub of ['lib', 'templates', 'appinfo']) { + for (const file of walk(path.join(root, sub), ['.php'])) { + const source = fs.readFileSync(file, 'utf8') + for (const re of patterns) { + let m + while ((m = re.exec(source)) !== null) { + used.add(m[1].replace(/\\(['"\\])/g, '$1')) + } + } + } + } + return used +} + +/** + * Every user-visible string a PHP array DECLARES for the client to render. + * + * The setup wizard's card step forced this: a `choice` step with + * `optionsSource` carries no options in the manifest, so every card's label + * and description is a PHP array value the wizard translates client-side by + * literal lookup. Adjacent literals joined by `.` are read as one value, + * because PHP wraps long prose that way and a first-fragment-only match misses + * the rest. + * + * @param {string} root app root + * @return {Set} strings a PHP array declares for the client + */ +function collectPhpDeclared(root) { + const out = new Set() + const PART = String.raw`'((?:\\.|[^'\\])*)'|"((?:\\.|[^"\\])*)"` + for (const file of walk(path.join(root, 'lib'), ['.php'])) { + const source = fs.readFileSync(file, 'utf8') + for (const field of RENDERED_FIELDS) { + const re = new RegExp( + String.raw`['"]${field}['"]\s*=>\s*\(?\s*((?:(?:${PART})\s*\.?\s*)+)`, + 'g', + ) + let m + while ((m = re.exec(source)) !== null) { + const parts = [...m[1].matchAll(new RegExp(PART, 'g'))].map((q) => + (q[1] ?? q[2] ?? '').replace(/\\(['"\\])/g, '$1'), + ) + const joined = parts.join('') + if (joined.trim()) { + out.add(joined) + } + } + } + } + return out +} + +// --------------------------------------------------------------------------- +// MANIFEST +// --------------------------------------------------------------------------- + +/** + * Every user-visible string the manifest declares. + * + * `src/manifest.d/*.json` counts: the fragments are merged at runtime via + * require.context, so a checker that opens only `src/manifest.json` is blind + * to whatever they add. `_meta` is skipped, being per-fragment provenance that + * is never rendered. + * + * @param {string} root app root + * @return {Set} the manifest's user-visible strings + */ +function collectManifestStrings(root) { + const out = new Set() + const files = [] + const main = path.join(root, 'src/manifest.json') + if (fs.existsSync(main)) { + files.push(main) + } + files.push(...walk(path.join(root, 'src/manifest.d'), ['.json'])) + + const fields = new Set(RENDERED_FIELDS) + const visit = (node) => { + if (Array.isArray(node)) { + node.forEach(visit) + return + } + if (!node || typeof node !== 'object') { + return + } + for (const [k, v] of Object.entries(node)) { + if (k === '_meta') { + continue + } + if (typeof v === 'string') { + if (fields.has(k) && v.trim()) { + out.add(v) + } + } else { + visit(v) + } + } + } + for (const file of files) { + try { + visit(readJson(file)) + } catch (e) { + throw new Error(`${path.relative(root, file)}: ${e.message}`) + } + } + return out +} + +// --------------------------------------------------------------------------- +// SCHEMA +// --------------------------------------------------------------------------- + +/** + * Every user-visible string an OpenRegister register or schema declares. + * + * The second source no vendored copy could turn into a "missing" finding, and + * the larger one: 213 on opencatalogi alone. These are rendered by + * OpenRegister — a schema's `title` heads its detail page, a property's + * `title` labels its form field, a `description` becomes the help text under + * it — so an untranslated one is English on screen in every locale. + * + * SCOPE IS DELIBERATELY NARROW. Only `components.registers` and + * `components.schemas` are read, and only `title` and `description` within + * them. `info.title`, `info.description` and everything under an `x-` key are + * package metadata that no reader sees, and including them would have the + * checker demand translations for a source URL's prose. + * + * @param {string} root app root + * @return {Set} register and schema strings rendered to a reader + */ +function collectSchemaStrings(root) { + const out = new Set() + const visit = (node) => { + if (Array.isArray(node)) { + node.forEach(visit) + return + } + if (!node || typeof node !== 'object') { + return + } + for (const [k, v] of Object.entries(node)) { + if (k.startsWith('x-')) { + continue + } + if (typeof v === 'string') { + if ((k === 'title' || k === 'description') && v.trim()) { + out.add(v) + } + } else { + visit(v) + } + } + } + for (const file of walk(path.join(root, 'lib/Settings'), ['.json'])) { + let doc + try { + doc = readJson(file) + } catch (e) { + throw new Error(`${path.relative(root, file)}: ${e.message}`) + } + const components = doc && doc.components + if (!components || typeof components !== 'object') { + continue + } + visit(components.registers) + visit(components.schemas) + } + return out +} + +// --------------------------------------------------------------------------- +// main +// --------------------------------------------------------------------------- + +/** + * The app id, which is the first argument of every t() call and therefore the + * thing the SRC extractor keys on. `appinfo/info.xml` is the only authority: + * the fleet is mid-rename and the directory name is routinely the old one. + * + * @param {string} root app root + * @return {string|null} the declared app id + */ +function readAppId(root) { + const file = path.join(root, 'appinfo/info.xml') + if (!fs.existsSync(file)) { + return null + } + const m = /\s*([^<\s]+)\s*<\/id>/.exec(fs.readFileSync(file, 'utf8')) + return m ? m[1] : null +} + +/** + * Strings this app has declared it will not translate, one per entry. + * + * The migration path off a vendored copy needs somewhere to put the findings + * an app is not fixing today, or the shared checker is unadoptable. A reason + * is required, so the file cannot become a silent suppression list. + * + * @param {string} root app root + * @return {Set} strings excluded by the app, with a stated reason + */ +function loadIgnored(root) { + const out = new Set() + const file = path.join(root, 'l10n/.l10n-source-ignore.json') + if (!fs.existsSync(file)) { + return out + } + const raw = readJson(file) + for (const [key, reason] of Object.entries(raw || {})) { + if (typeof reason === 'string' && reason.trim()) { + out.add(key) + } + } + return out +} + +function main(argv) { + const args = argv.slice(2) + const warnOnly = args.includes('--warn-only') + const asJson = args.includes('--json') + const sourceArg = args.find((a) => a.startsWith('--source=')) + const enabled = new Set( + sourceArg ? sourceArg.slice('--source='.length).split(',').map((s) => s.trim().toUpperCase()) : ALL_SOURCES, + ) + const root = path.resolve(args.find((a) => !a.startsWith('--')) || process.cwd()) + + const appId = readAppId(root) + if (!appId) { + process.stderr.write(`no in ${path.join(root, 'appinfo/info.xml')} — cannot tell which t() calls belong to this app\n`) + return 9 + } + + const enPath = path.join(root, 'l10n/en.json') + const nlPath = path.join(root, 'l10n/nl.json') + const en = loadCatalogue(enPath) + const nl = loadCatalogue(nlPath) + + /** @type {Map>} string to the origins that produced it */ + const origins = new Map() + // Strings that prove a catalogue key is live without themselves demanding + // one. See the note at the top about PHP array values. + const suppressOnly = new Set() + const record = (source, set) => { + if (!enabled.has(source)) { + return + } + for (const s of set) { + if (!origins.has(s)) { + origins.set(s, new Set()) + } + origins.get(s).add(source) + } + } + + try { + record('SRC', collectSrcStrings(root, appId)) + record('PHP', collectPhpTranslated(root)) + record('MANIFEST', collectManifestStrings(root)) + record('SCHEMA', collectSchemaStrings(root)) + if (enabled.has('PHP')) { + for (const s of collectPhpDeclared(root)) { + suppressOnly.add(s) + } + } + } catch (e) { + process.stderr.write(`could not read a source: ${e.message}\n`) + return 9 + } + + const ignored = loadIgnored(root) + for (const key of ignored) { + origins.delete(key) + } + + const total = origins.size + if (total === 0 || en.size === 0) { + process.stdout.write(`checked ${total} source string(s) against ${en.size} English key(s)\n`) + return 4 + } + + const missingEn = [] + const missingNl = [] + for (const [key, from] of origins) { + const label = [...from].sort().join('+') + if (!en.has(key)) { + missingEn.push({ key, source: label }) + } + if (nl.size > 0 && !nl.has(key)) { + missingNl.push({ key, source: label }) + } + } + const unused = [...en.keys()] + .filter((k) => !origins.has(k) && !suppressOnly.has(k) && !ignored.has(k)) + .sort() + + missingEn.sort((a, b) => a.key.localeCompare(b.key)) + missingNl.sort((a, b) => a.key.localeCompare(b.key)) + + if (asJson) { + process.stdout.write(JSON.stringify({ appId, checked: total, missingEn, missingNl, unused }, null, 2) + '\n') + } else { + const byOrigin = (rows) => { + const counts = {} + for (const r of rows) { + counts[r.source] = (counts[r.source] || 0) + 1 + } + return Object.entries(counts).map(([k, v]) => `${k} ${v}`).join(', ') || 'none' + } + for (const row of missingEn) { + process.stdout.write(`FAIL [${row.source}] no key in l10n/en.json: ${JSON.stringify(row.key)}\n`) + } + for (const row of missingNl) { + process.stdout.write(`WARN [${row.source}] no key in l10n/nl.json: ${JSON.stringify(row.key)}\n`) + } + for (const key of unused) { + process.stdout.write(`WARN [CATALOGUE] no source produces this key: ${JSON.stringify(key)}\n`) + } + process.stdout.write(`missing from en.json: ${missingEn.length} (${byOrigin(missingEn)})\n`) + process.stdout.write(`missing from nl.json: ${missingNl.length} (${byOrigin(missingNl)})\n`) + process.stdout.write(`unused in en.json: ${unused.length}\n`) + } + + // The terminal marker. The gate runner requires it before it will read any + // count off this log: a checker that crashed halfway prints findings too, + // and a crash is not a finding. + process.stdout.write(`checked ${total} source string(s) against ${en.size} English key(s)\n`) + + if (warnOnly) { + return 0 + } + return missingEn.length + missingNl.length + unused.length > 0 ? 1 : 0 +} + +if (require.main === module) { + process.exit(main(process.argv)) +} + +module.exports = { + collectSrcStrings, + collectPhpTranslated, + collectPhpDeclared, + collectManifestStrings, + collectSchemaStrings, + loadCatalogue, + main, +} diff --git a/hydra-gates/scripts/lib/test_gate117_l10n_source_coverage.sh b/hydra-gates/scripts/lib/test_gate117_l10n_source_coverage.sh new file mode 100755 index 00000000..5c9e595a --- /dev/null +++ b/hydra-gates/scripts/lib/test_gate117_l10n_source_coverage.sh @@ -0,0 +1,84 @@ +#!/usr/bin/env bash +# Gate 117 acceptance — the gate is PROVEN to refuse, not assumed to. +# +# 🔴 THE FOURTH CASE IS THE ONE THAT MATTERS. Every vendored `check-l10n.js` in +# the fleet computed `missing` from src/ t() calls alone, so a PHP or schema +# string that reached no catalogue was invisible to all of them. The `planted` +# tree hides exactly one PHP string and one schema title, and leaves the src/ +# and manifest strings covered. Run at the incumbent's scope it reads CLEAN; +# run at this checker's scope it reports two. A promoted checker that could not +# tell those two readings apart would have promoted the defect. +# +# SPDX-FileCopyrightText: 2026 Conduction B.V. +# SPDX-License-Identifier: EUPL-1.2 +set -uo pipefail + +HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +CHECKER="${HERE}/../check-l10n.js" +FIXTURES="${HERE}/../test-fixtures/gate-acceptance/l10n-source-coverage" + +fails=0 + +expect() { + local tree="$1" want_rc="$2" want_fails="$3" why="$4" + shift 4 + local out rc got + + out="$(node "${CHECKER}" "${FIXTURES}/${tree}" "$@" 2>&1)" + rc=$? + got="$(printf '%s\n' "${out}" | grep -c '^FAIL ')" + + if [ "${rc}" -ne "${want_rc}" ] || [ "${got}" -ne "${want_fails}" ]; then + echo "FAIL ${tree}: expected rc=${want_rc} with ${want_fails} finding(s), got rc=${rc} with ${got}" + echo " ${why}" + printf '%s\n' "${out}" | sed 's/^/ | /' + fails=$((fails + 1)) + return + fi + + # A CHECKER THAT CRASHES MUST NOT READ AS CLEAN. The terminal summary is + # how the runner tells the two apart, so the acceptance test asserts it. + if ! printf '%s\n' "${out}" | grep -qE '^checked [0-9]+ source string'; then + echo "FAIL ${tree}: the checker never printed its terminal summary, so a crash would read as a pass" + fails=$((fails + 1)) + return + fi + + echo "ok ${tree}: rc=${rc}, ${got} finding(s) — ${why}" +} + +expect clean 0 0 "every source string has an English key and a Dutch one" +expect planted 1 2 "the hidden PHP string and the hidden schema title are both refused" +expect no-catalogue 4 0 "a repository with no l10n/en.json is NOT APPLICABLE, which is not a pass" + +# The control. Narrowed to the sources the vendored copies actually read, the +# planted tree reports ZERO missing. That is the defect this gate exists to +# end, asserted rather than described. +control="$(node "${CHECKER}" "${FIXTURES}/planted" --source=SRC,MANIFEST 2>&1)" +if printf '%s\n' "${control}" | grep -q '^missing from en.json: 0 '; then + echo "ok planted at the incumbent's scope: 0 missing — the blind spot is reproduced" +else + echo "FAIL planted at the incumbent's scope should report 0 missing, so the wider scope is what finds the two" + printf '%s\n' "${control}" | sed 's/^/ | /' + fails=$((fails + 1)) +fi + +# --warn-only must hold back the exit code and nothing else: the findings still +# print. A launch-as-warning that also swallowed the findings would be a gate +# that runs nowhere. +warn="$(node "${CHECKER}" "${FIXTURES}/planted" --warn-only 2>&1)" +warn_rc=$? +warn_n="$(printf '%s\n' "${warn}" | grep -c '^FAIL ')" +if [ "${warn_rc}" -eq 0 ] && [ "${warn_n}" -eq 2 ]; then + echo "ok planted with --warn-only: rc=0 and 2 finding(s) still printed" +else + echo "FAIL planted with --warn-only: expected rc=0 with 2 finding(s), got rc=${warn_rc} with ${warn_n}" + fails=$((fails + 1)) +fi + +if [ "${fails}" -ne 0 ]; then + echo "gate-117 acceptance: ${fails} case(s) failed" + exit 1 +fi + +echo "gate-117 acceptance: all cases behaved" diff --git a/hydra-gates/scripts/run-hydra-gates.sh b/hydra-gates/scripts/run-hydra-gates.sh index e076a68a..2da71426 100755 --- a/hydra-gates/scripts/run-hydra-gates.sh +++ b/hydra-gates/scripts/run-hydra-gates.sh @@ -12991,6 +12991,91 @@ else _skip 116 "connections-declaration" na "this app ships no lib/Settings/connections.json, so it declares no connections to check." fi +# --------------------------------------------------------------------------- +# GATE 117 — l10n-source-coverage (WARNING ONLY, launch 2026-09-19) +# +# A user-visible string with no key in l10n/en.json renders its source text to +# every reader, in every locale, and nothing downstream notices. +# +# WHY THIS IS A GATE AND NOT TWENTY-ONE SCRIPTS. Twenty-one apps vendored a +# `check-l10n.js`, and by 2026-09-19 those copies had drifted into THIRTEEN +# distinct versions across fifteen repositories (three repos ship none at all, +# and three carry two copies each at different paths). Drift was not the worst +# of it. EVERY ONE of the thirteen computed `missing` the same way: +# +# const missing = [...usedKeys].filter((k) => !keys.has(k)) +# +# with `usedKeys` built by walking src/ for t() calls. PHP and schema JSON +# appeared in the better copies only as things that SUPPRESS an "unused" +# warning. So no app in the fleet could see a server-side or schema string +# that had reached no catalogue at all: a `->t('…')` could clear a warning and +# could never raise one. +# +# MEASURED on opencatalogi at development@4af8e55a: 49 strings passed to a PHP +# translate call and 319 register/schema strings have no key in en.json — 368 +# findings on a repo whose own vendored check reports zero, because the src/ +# leg really is clean. +# The src/ leg agreeing with the incumbent is the control: the new findings +# come from the new sources, not from a different reading of the old one. +# +# WARNING FIRST, per the fleet rule that a new gate never lands blocking. All +# 21 core repos set `enable-hydra-gates: true` and resolve this file at @main, +# so a blocking merge reddens them the same minute. openregister alone carries +# 1,273 findings. The runner passes `--warn-only`, so the checker's exit code +# is 0 whatever it finds, and this block calls `_warn`, never `_fail`. +# +# PROMOTION TO BLOCKING is two deliberate edits here: drop `--warn-only` from +# the invocation and swap `_warn` for `_fail`. Owner: the l10n debt sweep. +# +# FULL-TREE, not diff-scoped, for the reason gates 84, 93, 94, 95, 96 and 102 +# give: a string is either covered or it is not, and a diff-scoped version +# reports clean on every PR that does not happen to touch a catalogue. +# +# NOTE ON PLACEMENT: top level, outside any `_FAILED` guard — a gate that only +# runs once everything else passed is green-but-dead. +# --------------------------------------------------------------------------- +_lsc_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-l10n-source-coverage.log +: > "${_lsc_log}" +if [ -f appinfo/info.xml ] && [ -f l10n/en.json ]; then + set +e + node "${SCRIPT_DIR}/check-l10n.js" . --warn-only > "${_lsc_log}" 2>&1 + _lsc_rc=$? + # `set +e`, not `set -e`: errexit off is the state this script actually + # runs in. See the note at the top of this file. + set +e + + # An empty scope must not print the same word as a clean full-tree read. + _lsc_checked=$(sed -n 's/^checked \([0-9]\{1,\}\) source string.*/\1/p' "${_lsc_log}" 2>/dev/null | tail -1) + case "${_lsc_checked}" in ''|*[!0-9]*) _lsc_checked=0 ;; esac + + if [ "${_lsc_rc}" -eq 4 ] || { [ "${_lsc_rc}" -eq 0 ] && [ "${_lsc_checked}" -eq 0 ]; }; then + _skip_empty_scope 117 "l10n-source-coverage" "user-visible string checkable against an English catalogue (a src/ t() call, a PHP ->t(), a manifest field or a lib/Settings schema title, plus l10n/en.json)" + elif ! _helper_finished "${_lsc_log}" '^checked [0-9]+ source string'; then + # A CRASH IS NOT A FINDING. The checker prints findings as it goes, so + # a run that died halfway looks exactly like one that finished with + # findings unless the terminal marker is required. + _lsc_why=$(head -3 "${_lsc_log}" 2>/dev/null | tr '\n' ' ' | cut -c1-200) + _skip 117 "l10n-source-coverage" wiring "check-l10n.js exited ${_lsc_rc} without printing its terminal 'checked N source string(s)' summary, so translation coverage is UNVERIFIED by this run. Checker output: ${_lsc_why:-}. See ${_lsc_log}." + else + _lsc_en=$(sed -n 's/^missing from en.json: \([0-9]\{1,\}\).*/\1/p' "${_lsc_log}" 2>/dev/null | tail -1) + case "${_lsc_en}" in ''|*[!0-9]*) _lsc_en=0 ;; esac + _lsc_nl=$(sed -n 's/^missing from nl.json: \([0-9]\{1,\}\).*/\1/p' "${_lsc_log}" 2>/dev/null | tail -1) + case "${_lsc_nl}" in ''|*[!0-9]*) _lsc_nl=0 ;; esac + _lsc_unused=$(sed -n 's/^unused in en.json: \([0-9]\{1,\}\).*/\1/p' "${_lsc_log}" 2>/dev/null | tail -1) + case "${_lsc_unused}" in ''|*[!0-9]*) _lsc_unused=0 ;; esac + + if [ "${_lsc_en}" -eq 0 ] && [ "${_lsc_nl}" -eq 0 ] && [ "${_lsc_unused}" -eq 0 ]; then + _pass 117 "l10n-source-coverage" + else + _warn 117 "l10n-source-coverage" "${_lsc_en} user-visible string(s) have no key in l10n/en.json, ${_lsc_nl} none in l10n/nl.json, and ${_lsc_unused} catalogue key(s) no source produces. Each finding names the source that produced it: SRC, PHP, MANIFEST or SCHEMA. The PHP and SCHEMA lines are what the vendored per-app copies could never report. Report-only at launch (2026-09-19) because 14 of 21 repos carry inherited findings; promotion to blocking is a deliberate edit to this block. See ${_lsc_log}" + fi + fi +elif [ -f appinfo/info.xml ]; then + _skip 117 "l10n-source-coverage" na "this app ships no l10n/en.json, so there is no English catalogue to check any string against." +else + _skip 117 "l10n-source-coverage" na "no appinfo/info.xml, so this is not a Nextcloud app and declares no app id for t() calls to be keyed on." +fi + # --------------------------------------------------------------------------- # Summary + COVERAGE ACCOUNTING # diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/appinfo/info.xml b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/appinfo/info.xml new file mode 100644 index 00000000..5676271e --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/appinfo/info.xml @@ -0,0 +1,5 @@ + + + demo + Demo + diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/l10n/en.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/l10n/en.json new file mode 100644 index 00000000..3bd1b415 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/l10n/en.json @@ -0,0 +1,10 @@ +{ + "translations": { + "Publications": "Publications", + "Unable to reach the requested directory": "Unable to reach the requested directory", + "Publication": "Publication", + "Something a catalogue publishes.": "Something a catalogue publishes.", + "Summary": "Summary", + "Catalogues": "Catalogues" + } +} \ No newline at end of file diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/l10n/nl.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/l10n/nl.json new file mode 100644 index 00000000..ca8c6e69 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/l10n/nl.json @@ -0,0 +1,10 @@ +{ + "translations": { + "Publications": "NL Publications", + "Unable to reach the requested directory": "NL Unable to reach the requested directory", + "Publication": "NL Publication", + "Something a catalogue publishes.": "NL Something a catalogue publishes.", + "Summary": "NL Summary", + "Catalogues": "NL Catalogues" + } +} \ No newline at end of file diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/lib/Controller/SearchController.php b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/lib/Controller/SearchController.php new file mode 100644 index 00000000..3f887ba9 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/lib/Controller/SearchController.php @@ -0,0 +1,6 @@ +l10n->t('Unable to reach the requested directory'); + } +} diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/lib/Settings/register.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/lib/Settings/register.json new file mode 100644 index 00000000..236c994a --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/lib/Settings/register.json @@ -0,0 +1,15 @@ +{ + "openapi": "3.0.0", + "info": { "title": "Not user visible", "description": "Packaging metadata." }, + "components": { + "schemas": { + "publication": { + "title": "Publication", + "description": "Something a catalogue publishes.", + "properties": { + "summary": { "type": "string", "title": "Summary" } + } + } + } + } +} diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/src/Page.vue b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/src/Page.vue new file mode 100644 index 00000000..27261e64 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/src/Page.vue @@ -0,0 +1 @@ + diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/src/manifest.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/src/manifest.json new file mode 100644 index 00000000..16253c03 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/clean/src/manifest.json @@ -0,0 +1 @@ +{ "menu": [ { "label": "Catalogues" } ] } diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/appinfo/info.xml b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/appinfo/info.xml new file mode 100644 index 00000000..5676271e --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/appinfo/info.xml @@ -0,0 +1,5 @@ + + + demo + Demo + diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/lib/Controller/SearchController.php b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/lib/Controller/SearchController.php new file mode 100644 index 00000000..3f887ba9 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/lib/Controller/SearchController.php @@ -0,0 +1,6 @@ +l10n->t('Unable to reach the requested directory'); + } +} diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/lib/Settings/register.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/lib/Settings/register.json new file mode 100644 index 00000000..236c994a --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/lib/Settings/register.json @@ -0,0 +1,15 @@ +{ + "openapi": "3.0.0", + "info": { "title": "Not user visible", "description": "Packaging metadata." }, + "components": { + "schemas": { + "publication": { + "title": "Publication", + "description": "Something a catalogue publishes.", + "properties": { + "summary": { "type": "string", "title": "Summary" } + } + } + } + } +} diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/src/Page.vue b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/src/Page.vue new file mode 100644 index 00000000..27261e64 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/src/Page.vue @@ -0,0 +1 @@ + diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/src/manifest.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/src/manifest.json new file mode 100644 index 00000000..16253c03 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/no-catalogue/src/manifest.json @@ -0,0 +1 @@ +{ "menu": [ { "label": "Catalogues" } ] } diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/appinfo/info.xml b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/appinfo/info.xml new file mode 100644 index 00000000..5676271e --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/appinfo/info.xml @@ -0,0 +1,5 @@ + + + demo + Demo + diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/l10n/en.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/l10n/en.json new file mode 100644 index 00000000..e6c358d2 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/l10n/en.json @@ -0,0 +1,8 @@ +{ + "translations": { + "Publications": "Publications", + "Publication": "Publication", + "Something a catalogue publishes.": "Something a catalogue publishes.", + "Catalogues": "Catalogues" + } +} \ No newline at end of file diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/l10n/nl.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/l10n/nl.json new file mode 100644 index 00000000..be161dcb --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/l10n/nl.json @@ -0,0 +1,8 @@ +{ + "translations": { + "Publications": "NL Publications", + "Publication": "NL Publication", + "Something a catalogue publishes.": "NL Something a catalogue publishes.", + "Catalogues": "NL Catalogues" + } +} \ No newline at end of file diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/lib/Controller/SearchController.php b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/lib/Controller/SearchController.php new file mode 100644 index 00000000..3f887ba9 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/lib/Controller/SearchController.php @@ -0,0 +1,6 @@ +l10n->t('Unable to reach the requested directory'); + } +} diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/lib/Settings/register.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/lib/Settings/register.json new file mode 100644 index 00000000..236c994a --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/lib/Settings/register.json @@ -0,0 +1,15 @@ +{ + "openapi": "3.0.0", + "info": { "title": "Not user visible", "description": "Packaging metadata." }, + "components": { + "schemas": { + "publication": { + "title": "Publication", + "description": "Something a catalogue publishes.", + "properties": { + "summary": { "type": "string", "title": "Summary" } + } + } + } + } +} diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/src/Page.vue b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/src/Page.vue new file mode 100644 index 00000000..27261e64 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/src/Page.vue @@ -0,0 +1 @@ + diff --git a/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/src/manifest.json b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/src/manifest.json new file mode 100644 index 00000000..16253c03 --- /dev/null +++ b/hydra-gates/scripts/test-fixtures/gate-acceptance/l10n-source-coverage/planted/src/manifest.json @@ -0,0 +1 @@ +{ "menu": [ { "label": "Catalogues" } ] }