[Monkey7539]

For a release after 4.1.0. Two commits: the feature, rebased onto main 0441d23 after 4.1.0 (migration 0072; the scheduled-send routes reviewed for tokens, none allowlisted), and the changes from your review round 1 on top, so the delta is easy to see. The review's four needed changes and the optional archive check are all in; the description below reflects them.

Summary

This is the backend of #617: personal API tokens, as settled in that thread. With a token, a script or tool can use MailFlow's API without the account password, and keeps working with two-factor sign-in on. My own case is an MCP server for Claude, running on another machine.

• Tokens stay off until an admin turns them on.
• Each token reaches only an explicit list of routes for its scopes (read, organize, draft).
• Sending, deleting, settings, admin and auth stay out of reach.

This is the first of the two PRs #617 describes. The Settings UI (the token card, the admin switch and 11 locales) is the second, and I'll open it once this one merges. The MCP server itself stays outside MailFlow, at https://github.com/Monkey7539/mailflow-mcp.

Changes

Storage. Migration 0072 adds api_tokens. For each token it holds:

• only the SHA-256 of the token, as trusted_devices and reset links do;
• its scopes, as JSONB;
• its expiry, which every token has;
• what confirmed its creation (password, totp or oidc), and for single sign-on, the provider.

api_tokens_enabled in system_settings defaults to off. It is read through a 30 s cache, which the admin settings PATCH clears.

Which requests are token requests. Only Authorization: Bearer mfat_…, as agreed on #617:

• an mfat_ value that doesn't check out gets a 401, and any cookie on the request is ignored;
• any other Authorization value goes to the cookie flow untouched.

Allowlist (services/apiTokenAllowlist.js). Each entry is an exact method and path, compared with req.originalUrl:

• matching is case-sensitive, with no trailing slash, and parameters must be UUIDs;
• HEAD is not on it.

Scope	Routes
read	accounts (a narrower response), folders, message lists, a message and its body, resolve-message, the query form of thread, unread and category counts, search, contact search
organize	bulk read, bulk star, bulk archive, bulk move, snooze, reschedule and unsnooze, category
draft	save a draft, the reply-draft indicators
any valid token	/api/health, /api/version, /api/api-tokens/self

Every other route answers 403, the scheduled-send routes included.

Guards after parsing (middleware/apiTokenGuards.js):

• Bulk-move refuses Trash, Junk, Drafts, Snoozed, no-select folders and folders MailFlow does not sync. The refused set comes from two sources, so it cannot drift from the code that uses them:
  • the existing folder SQL in mailUtils, through a new folderRoles helper;
  • the provider skip list.
• Drafts never move. Bulk-move, bulk-archive and snooze refuse any request that would carry a draft along.
• The archive destination. Bulk-archive resolves each account's archive folder as the route does and holds it to a move's rules, so an archive mapping that points at Trash, Junk, Drafts, Snoozed or an unsynced folder is refused. On Gmail without an archive label or mapping the destination is All Mail, where the mail would drop out of MailFlow; that error says to create a Gmail label named Archive or map an archive folder. The archive button is unchanged.
• Token drafts can only be created, never replaced, and must be plain text:
  • existing* fields, fromScheduled (which deletes the scheduled message a draft was made from), any truthy bodyIsHtml, and quotedBodyHtml are refused;
  • recipients, ids and the signature are validated.

Session plumbing:

• A token request gets a request-only identity on req.session: no cookie and no Redis write.
• The /api chain is one function, mountApiAuth, so a test can check its order: token auth, body parsers, session, CSRF, screen lock, guards.
• The CSRF check and the lock gate moved into modules, with the same logic and messages. Only a token that authenticated skips the CSRF check, never a request that merely carries an Authorization header.
• reloadSession re-checks the token after a large upload.
• requireAuth answers a token whose user is gone with 401, and doesn't update the user's last-seen time for token requests.
• requireAdmin refuses tokens outright.

Management API (routes/apiTokens.js):

• list, which also returns reauth, expiryChoices and maxTokens;
• create, up to 20 per user;
• revoke;
• /self, so a token can see its own name and scopes.

Confirming creation (services/apiTokenReauth.js), as settled on #617:

• TOTP: the current two-factor code, when the user has two-factor on and can sign in with a password. It has the same replay guard and totp:<uid> limit as the sign-in challenge.
• Password: otherwise the password, counted against the sign-in budget auth:<ip>.
• SSO: for an account that cannot sign in with a password (password login off, or no password set), a sign-in made through its SSO provider in the last 10 minutes from the token form's Confirm button, /auth/oidc/:slug/start?action=reauth:
  • the authorize request carries prompt=login and max_age=0, so the provider asks for credentials again and must put auth_time in the id_token;
  • the flow records when it began, and the callback refuses an id_token whose auth_time is missing or earlier than that, less a minute of clock skew, since not every provider honours prompt;
  • the session must belong to the user who started the flow, and the identity must be linked to that user; nothing else about the session changes;
  • an ordinary SSO sign-in records no confirmation: the provider may have kept its own session, so it proves no more than the MailFlow session does.
• Required two-factor: a user without an authenticator is asked to set one up first.

Expiries are 7, 30, 90 or 365 days, and 7 or 30 for SSO-confirmed tokens. A wrong code or password is logged as totp_fail or login_fail. The confirmation is never answered with a 401, which the web client would take for an expired session.

Revocation.

• A password reset deletes the user's tokens; it now runs as one transaction. So does an admin setting a password.
• Deleting a user cascades.
• A backup restore deletes every token, because api_tokens is not restored.
• A user turning their own two-factor off deletes the tokens it confirmed; those confirmed with the password, just given again, stay.
• A user unlinking their last SSO identity at a provider deletes the tokens confirmed through that provider.

A change that tightens sign-in deletes the tokens it no longer allows, in the transaction that makes the change. The [admin] log line says how many went:

• password login turned off: every token not confirmed through single sign-on;
• two-factor required: tokens confirmed with only a password, while TOTP- and SSO-confirmed tokens stay;
• an admin turning off a user's two-factor: all of that user's tokens;
• an SSO provider turned off, deleted, or pointed at another issuer or client: the tokens confirmed through it, the delete by the foreign key.

The policy and a creation cannot race. createToken reads mfa_enforcement and internal_auth_disabled with SELECT … FOR SHARE inside its transaction, after the confirmation, and refuses (409) a confirmation the policy no longer allows. The settings PATCH updates those rows in its own transaction before deleting tokens, so whichever runs first, the other waits and then sees it: a token confirmed with a password can neither slip in behind a change to required two-factor nor survive it.

Limits (numbers as agreed on #617):

• failed attempts: 20 per 15 minutes per IP, in their own bucket apitoken-fail:<ip>, not the sign-in budget. Only malformed or unknown tokens spend it, so a stale token in a tool can't lock its owner out of web sign-in. An expired token always gets 401 and spends nothing;
• valid tokens: 600 requests per token and 1,200 per user, per 5 minutes;
• 60 token drafts per 10 minutes;
• 1 MB per token request;
• token searches share one per-user bucket, separate from the browser's.

A refused body is drained up to 1 MB, then the connection is closed.

Logging. There are three auth events: api_token_created, api_token_revoked and api_token_auth_fail. Neither the token nor its hash is ever logged, and the diagnostics scrubber redacts mfat_.

Where this differs from #617's original text

The thread settled the Bearer rule, the move and archive guards, re-authentication and expiry, the policy changes and the limits. These details were not in it:

1. Revoking deletes the row instead of setting revoked_at, so no lookup can forget to check a flag. The auth events keep the history.
2. Token drafts are new and plain text only. Replacing a draft expunges the old copy, and quotedBodyHtml is stored unsanitized today.
3. GET /api/accounts returns a narrower shape to tokens: no hosts, ports, TLS settings, login names, signatures, folder mappings or plugin settings.
4. Not done:
   • Refusing tokens over plain HTTP (open question 1 in Scoped, revocable API tokens, so scripts and tools don't need the account password #617). On the port-80 server, req.secure reflects whatever the client claims, so the check would mean little.
   • Per-account limits, which Scoped, revocable API tokens, so scripts and tools don't need the account password #617 leaves for version 2.

Testing

• Suite: npm test passes 4,053 tests; the 209 skipped are the PostgreSQL-only ones, which your CI now runs with MAILFLOW_TEST_DATABASE_URL. npm run lint and npm run lint:plugins are clean.

• What the new tests cover:

  • the matcher, against every bypass shape: trailing slash, case, HEAD, encoded or doubled slashes, dot segments, extra segments and non-UUID parameters;
  • route coverage: the test reads index.js's mounts from its source and walks every reviewed router in both directions, the scheduled router included;
  • the whole mountApiAuth chain on a real express-session, Basic credentials included;
  • every guard, the routes, each way of confirming, the OIDC reauth flow (prompt, max_age, auth_time, a plain sign-in confirming nothing), each revocation path, the policy re-check and its FOR SHARE, and the limits.

  I proved each security check by breaking the code and watching its test fail.

• PostgreSQL 16 and Redis, in a temporary workflow on my fork: https://github.com/Monkey7539/mailflow/actions/runs/41069390623 (the review-round commit)

  • Database tests: all 196 PostgreSQL tests pass, including the token, guard and policy SQL tests built on migration 0072. The whole suite also passes with the database (4,260; the 2 skipped are in an upstream test, not this PR's).
  • End to end, against the real backend, all 79 checks pass:
    • First sign-up, with tokens off by default. The admin turns them on, and tokens are created with password confirmation.
    • The choices are 7, 30, 90 and 365 days, and a token that never expires is refused.
    • Allowlisted reads and writes succeed.
    • 16 out-of-scope requests get 403: admin, auth, send, scheduled send, delete, raw .eml, token management, trailing slash, upper case, HEAD and others.
    • A non-mfat Bearer header, and a sign-in behind a proxy that adds one, pass untouched.
    • Revoking a token and the switch both take effect at once.
    • After 20 junk tokens from one address, the next one gets 429, while a valid token from the same address still works.
    • Requiring two-factor deletes the password-confirmed tokens, and turning it off again brings none back.
    • Only hashes are stored, and every token records its confirmation.
  • mailflow-mcp ran over stdio and over HTTP against that live backend.

• Not checked: multi-replica deployments. The search limiter is per process, as it is today.

---

Contributor License Agreement

By submitting this pull request I confirm that:

• I have read and agree to the Contributor License Agreement.
• My contribution is my own original work (or I have identified any
  third-party material and confirmed it is compatible with the CLA).
• I have the right to submit this contribution under the terms of the CLA.

🤖 Generated with Claude Code

[salmonumbrella]

@Monkey7539 yessss!

[Monkey7539]

Rebased onto main 0441d23 (after 4.1.0), still one commit, and the description is updated to match. What the rebase changed:

• Migration: 0072_api_tokens.sql, after 4.1's three. Nothing in it changed.
• Scheduled send: the new routes under /api/mail are in the token review now. None is allowlisted, so a token gets 403 there, as a token may draft but never send. The route-walking test covers the router, and the end-to-end run below checks three of its routes.
• bodyParsers.js: 4.1's /api/mail/scheduled parser sits beside the token-aware draft parser; that was the only conflict.
• Tests: the accounts fixture gained 4.1's two new SAFE_FIELDS columns (oauth_public_client, sync_error_code); the narrowed token view of an account is an explicit field list, so it never included them.

Results on the rebased commit: npm test passes 4,028 (207 PostgreSQL-only skipped locally, which your CI now runs), both lints clean, and the fork's end-to-end run against PostgreSQL 16 and Redis is green: https://github.com/Monkey7539/mailflow/actions/runs/40783550696.

[maathimself]

I reviewed the whole diff, ran the full suite against PostgreSQL 16 (4,233 pass; lint and lint:plugins clean), and had an independent second review done. The allowlist, the session isolation and the CSRF change hold up: neither review found a way for a token to reach anything outside its scopes. What needs changing is how a token's creation is confirmed, plus a few revocation gaps.

Needed before merge

1. SSO confirmation does not make the user sign in again. action=reauth sends the normal authorize request, with no prompt or max_age, and the callback does not look at auth_time, so with a live session at the provider the round trip completes without any input. rememberOidcSession also records a confirmation at every SSO login. So at an unlocked browser, or with a stolen MailFlow session cookie in the 10 minutes after a login, an SSO-only account can get a 30-day token, while an account with a password always has to enter it or a two-factor code. Please:
   • send prompt=login and max_age=0 on the reauth authorize request;
   • store when the reauth flow started in oidcPending, and in the callback refuse an id_token whose auth_time is missing or earlier than that (allowing a minute of clock skew), since not every provider honors prompt;
   • record a confirmation only from the reauth flow, not from ordinary logins. That drops "an SSO sign-in counts" from the design; the token form's Confirm button covers it.
2. fromScheduled on token drafts. routes/draft.js deletes the scheduled message named by fromScheduled (from 4.1). A token cannot learn an edit token today, since only routes/scheduled.js returns one, but a draft-scope request should not carry a delete at all. Please refuse fromScheduled in the draft guard, with a test.
3. Revocation gaps. Please delete tokens in the same transaction as each of these:
   • POST /api/totp/disable: the user's tokens confirmed with totp.
   • DELETE /api/auth/oidc/identities/:id: the user's tokens confirmed through that identity's provider, when they have no other identity left at it.
   • PATCH /api/admin/oidc/:id changing issuer_url or client_id: the provider's tokens, as for enabled: false.
4. Policy race. reauthFor reads the sign-in policy before bcrypt or TOTP runs, and nothing checks it again at insert. If an admin requires two-factor or turns off password sign-in in between, the revoking DELETE cannot see the uncommitted row, and a password-confirmed token survives the change. Re-checking inside the transaction is not enough on its own at READ COMMITTED. In createToken's transaction, read mfa_enforcement and internal_auth_disabled with SELECT … FOR SHARE (both rows are seeded by migrations), and refuse if confirmedWith no longer qualifies. The settings PATCH updates those rows, so the two then wait for each other in either order.

Optional

• The bulk-archive guard refuses only All Mail. Running the resolved archive destination through isProtectedDestination, as bulk-move does, would also cover an archive mapping that points at Trash or Junk.

Separately, linking a new identity (action=link) needs only a signed-in session, so whoever holds a session can link an identity of their own and then use it to confirm. That predates this PR and I'll track it in its own issue; it doesn't block this one.

[Monkey7539]

Thanks for the thorough review, and for the second one. All four are done, plus the optional archive check, as a second commit (8539946) on top of the rebased feature, so the delta is easy to see. The description is updated to match.

1. SSO confirmation is a sign-in just made. Agreed on every part, and the point about rememberOidcSession was the bigger hole: a stolen cookie in the ten minutes after an SSO login got a 30-day token with nothing typed. Now:

• the reauth authorize request carries prompt=login and max_age=0;
• oidcPending records when the flow began, and the callback refuses an id_token whose auth_time is missing or earlier than that, less 60 s of clock skew, with "Sign in again at your provider to confirm";
• only the reauth callback records a confirmation. An ordinary login or a link records nothing, and the tests pin that neither sends prompt.

One consequence to be aware of: a provider that never emits auth_time cannot confirm tokens at all. Entra ID, for one, sends it only with the auth_time optional claim configured. I think refusing is the right default, as you said, since the alternative is trusting that prompt was honoured; it is a line for the SSO docs.

2. fromScheduled. The draft guard refuses it (403, "API tokens cannot remove a scheduled message"), any non-null value, with a test. A token may draft, never take back a message due to go out.

3. Revocation gaps, each in the same transaction as the change:

• POST /api/totp/disable: the user's tokens confirmed with totp. The password-confirmed ones stay, since the password was just given again.
• DELETE /api/auth/oidc/identities/:id: the tokens confirmed through that identity's provider, when the user has no other identity left there. The route now runs on one client inside BEGIN/COMMIT; the last-login-method refusal still happens before anything is written.
• PATCH /api/admin/oidc/:id: a changed issuer_url or client_id deletes the provider's tokens as enabled: false does. Sending the same values back, or whitespace, deletes nothing.

4. Policy race. createToken now reads mfa_enforcement and internal_auth_disabled with SELECT … FOR SHARE after the count and before the insert, and returns a refusal when confirmedWith no longer qualifies, by the same rule revokeTokensForPolicy applies. The route answers 409 (not 401, which signs the client out) with "The server's sign-in policy changed while this token was being created. Confirm again." The settings PATCH's upsert takes the row lock, so the two wait for each other in either order. Both rows are seeded (0004, 0020), as you noted. The unit tests pin the FOR SHARE statement and its place in the transaction; the PostgreSQL test pins the refusal against the real rows.

Optional: archive. Done. The resolved archive folder goes through isProtectedDestination, so a mapping at Trash, Junk, Drafts or Snoozed is refused as well as All Mail, which keeps its own message about an Archive label.

Each new check was proved by breaking it and watching its test fail (11 mutations, all caught). Results: npm test 4,053 pass locally, both lints clean, and the fork's PostgreSQL and end-to-end run on this commit is green: https://github.com/Monkey7539/mailflow/actions/runs/41069390623.