--profile is ignored by the exit code for problem kinds without resolutionKind (InternalResolutionError, FalseCJS, …) #284

attw --profile esm-only on an ESM-only package prints green cells for node16 (from ESM) and bundler, labels the node10 problems "(ignored)", and exits 1 anyway. CI fails while the output shows nothing wrong in the resolutions I asked it to check.

The exit code honours --profile only for problems that carry a resolutionKind field. Four of the twelve problem kinds carry it; the other eight (InternalResolutionError, FalseESM, FalseCJS, NamedExports, FalseExportDefault, MissingExportEquals, UnexpectedModuleSyntax, CJSOnlyExportsDefault) always count toward exit 1, even when every resolution they appear in is ignored. The table gets this right, so the table and the exit code disagree. The README says "All are evaluated but failures outside of those required are ignored."

Reproduction (@arethetypeswrong/cli 0.18.5)

Files, packed with npm pack:

// package.json
{
  "name": "attw-repro-internal",
  "version": "1.0.0",
  "type": "module",
  "main": "./dist/index.js",
  "types": "./dist/index.d.ts",
  "exports": { ".": { "types": "./dist/index.d.ts", "default": "./dist/index.js" } },
  "imports": { "#lib/*": "./dist/lib/*" }
}

// dist/index.d.ts and dist/index.js
export { x } from "#lib/util.js";
// dist/lib/util.d.ts
export declare const x: number;
// dist/lib/util.js
export const x = 1;

$ npx --yes --package @arethetypeswrong/[email protected] attw attw-repro-internal-1.0.0.tgz --profile esm-only

attw-repro-internal v1.0.0

 (ignoring resolutions: 'node10', 'node16-cjs')

(ignored per resolution) 🥴 Import found in a type declaration file failed to resolve. Either this indicates that runtime resolution errors will occur, or (more likely) the types misrepresent the contents of the JavaScript files. Use -f json to see the imports that failed to resolve. https://github.com/arethetypeswrong/arethetypeswrong.github.io/blob/main/docs/problems/InternalResolutionError.md

(ignored per resolution) ⚠️ A require call resolved to an ESM JavaScript file, which is an error in Node and some bundlers. CommonJS consumers will need to use a dynamic import. https://github.com/arethetypeswrong/arethetypeswrong.github.io/blob/main/docs/problems/CJSResolvesToESM.md


┌───────────────────┬────────────────────────────────────────┐
│                   │ "attw-repro-internal"                  │
├───────────────────┼────────────────────────────────────────┤
│ node16 (from ESM) │ 🟢 (ESM)                               │
├───────────────────┼────────────────────────────────────────┤
│ bundler           │ 🟢                                     │
├───────────────────┼────────────────────────────────────────┤
│ node10            │ (ignored) 🥴 Internal resolution error │
├───────────────────┼────────────────────────────────────────┤
│ node16 (from CJS) │ (ignored) ⚠️ ESM (dynamic import only) │
└───────────────────┴────────────────────────────────────────┘
exit=1

With -f json, visibleProblems is empty for node16-esm and bundler; the only problem without resolutionKind is InternalResolutionError, resolutionOption: "node10".

Controls: --profile strict exits 1, correctly; --ignore-rules internal-resolution-error exits 0.

Same for FalseCJS under node16 (from CJS): a require condition pairing CommonJS types with an ES module is labelled "(ignored)" and exits 1 under esm-only.

// package.json
{
  "name": "attw-repro-falsecjs",
  "version": "1.0.0",
  "exports": {
    ".": {
      "import": { "types": "./index.d.mts", "default": "./index.mjs" },
      "require": { "types": "./index.d.ts", "default": "./index.mjs" }
    }
  }
}

// index.d.mts and index.d.ts
export declare const x: number;
// index.mjs
export const x = 1;

$ npx --yes --package @arethetypeswrong/[email protected] attw attw-repro-falsecjs-1.0.0.tgz --profile esm-only

attw-repro-falsecjs v1.0.0

 (ignoring resolutions: 'node10', 'node16-cjs')

(ignored per resolution) ⚠️ A require call resolved to an ESM JavaScript file, which is an error in Node and some bundlers. CommonJS consumers will need to use a dynamic import. https://github.com/arethetypeswrong/arethetypeswrong.github.io/blob/main/docs/problems/CJSResolvesToESM.md

(ignored per resolution) 🎭 Import resolved to a CommonJS type declaration file, but an ESM JavaScript file. https://github.com/arethetypeswrong/arethetypeswrong.github.io/blob/main/docs/problems/FalseCJS.md


┌───────────────────┬────────────────────────────────────────┐
│                   │ "attw-repro-falsecjs"                  │
├───────────────────┼────────────────────────────────────────┤
│ node16 (from ESM) │ 🟢 (ESM)                               │
├───────────────────┼────────────────────────────────────────┤
│ bundler           │ 🟢                                     │
├───────────────────┼────────────────────────────────────────┤
│ node10            │ (ignored) 🟢                           │
├───────────────────┼────────────────────────────────────────┤
│ node16 (from CJS) │ (ignored) ⚠️ ESM (dynamic import only) │
│                   │ (ignored) 🎭 Masquerading as CJS       │
└───────────────────┴────────────────────────────────────────┘
exit=1

Cause

packages/cli/src/getExitCode.ts (0.18.5):

const notResolutionIgnored =
  "resolutionKind" in problem ? !ignoreResolutions.includes(problem.resolutionKind) : true;

Core already knows where each problem is visible: checkPackage.ts records it in visibleProblems of every entrypoint resolution that produced it, and render/typed.ts uses problemAffectsResolutionKind to print "(ignored per resolution)". getExitCode ignores that mapping.

#248 reported the same symptom in 2025 and was closed when a later commit happened to pass. #278 adds resolutionKind to InternalResolutionError, which would fix that one kind as a side effect and leave the other seven.

Proposed fix

Count a problem only when it is not rule-ignored and problemAffectsResolutionKind is true for a resolution kind not in ignoreResolutions, the predicate the table already applies. One CLI file, no change to problem shapes, so it composes with #278. A pull request with a test that fails on main and passes with the change follows.

Why this matters to us

We run attw --profile esm-only as a release check on ESM-only packages whose declaration files use # subpath imports. Those resolve under node16-esm and bundler and cannot under node10, which is what the profile is for. The check fails a correct package, so we read the JSON and derive our own verdict from visibleProblems. We would rather the exit code said what the table says.

Investigated and drafted with AI assistance; the reproduction above and the fix in the follow-up pull request are there for you to check.