[Monkey7539]

For a release after 4.1.2. Three commits on main dd7944f: the feature (migration 0074, after main's 0072 and 0073; the scheduled-send routes reviewed for tokens, none allowlisted), your review round 1, and round 2 on top, so each delta is easy to see. Round 2: what confirmed a token is checked again under the locks when it is stored, a single sign-on confirmation makes one token, and the rebase onto #714's first part puts the unlink revocation in services/userIdentities.js, which the new admin unlink shares. The description below reflects all of it.

Summary

This is the backend of #617: personal API tokens, as settled in that thread. With a token, a script or tool can use MailFlow's API without the account password, and keeps working with two-factor sign-in on. My own case is an MCP server for Claude, running on another machine.

• Tokens stay off until an admin turns them on.
• Each token reaches only an explicit list of routes for its scopes (read, organize, draft).
• Sending, deleting, settings, admin and auth stay out of reach.

This is the first of the two PRs #617 describes. The Settings UI (the token card, the admin switch and 11 locales) is the second, and I'll open it once this one merges. The MCP server itself stays outside MailFlow, at https://github.com/Monkey7539/mailflow-mcp.

Changes

Storage. Migration 0074 adds api_tokens. For each token it holds:

• only the SHA-256 of the token, as trusted_devices and reset links do;
• its scopes, as JSONB;
• its expiry, which every token has;
• what confirmed its creation (password, totp or oidc), and for single sign-on, the provider.

api_tokens_enabled in system_settings defaults to off. It is read through a 30 s cache, which the admin settings PATCH clears.

Which requests are token requests. Only Authorization: Bearer mfat_…, as agreed on #617:

• an mfat_ value that doesn't check out gets a 401, and any cookie on the request is ignored;
• any other Authorization value goes to the cookie flow untouched.

Allowlist (services/apiTokenAllowlist.js). Each entry is an exact method and path, compared with req.originalUrl:

• matching is case-sensitive, with no trailing slash, and parameters must be UUIDs;
• HEAD is not on it.

Scope	Routes
read	accounts (a narrower response), folders, message lists, a message and its body, resolve-message, the query form of thread, unread and category counts, search, contact search
organize	bulk read, bulk star, bulk archive, bulk move, snooze, reschedule and unsnooze, category
draft	save a draft, the reply-draft indicators
any valid token	/api/health, /api/version, /api/api-tokens/self

Every other route answers 403, the scheduled-send routes included.

Guards after parsing (middleware/apiTokenGuards.js):

• Bulk-move refuses Trash, Junk, Drafts, Snoozed, no-select folders and folders MailFlow does not sync. The refused set comes from two sources, so it cannot drift from the code that uses them:
  • the existing folder SQL in mailUtils, through a new folderRoles helper;
  • the provider skip list.
• Drafts never move. Bulk-move, bulk-archive and snooze refuse any request that would carry a draft along.
• The archive destination. Bulk-archive resolves each account's archive folder as the route does and holds it to a move's rules, so an archive mapping that points at Trash, Junk, Drafts, Snoozed or an unsynced folder is refused. On Gmail without an archive label or mapping the destination is All Mail, where the mail would drop out of MailFlow; that error says to create a Gmail label named Archive or map an archive folder. The archive button is unchanged.
• Token drafts can only be created, never replaced, and must be plain text:
  • existing* fields, fromScheduled (which deletes the scheduled message a draft was made from), any truthy bodyIsHtml, and quotedBodyHtml are refused;
  • recipients, ids and the signature are validated.

Session plumbing:

• A token request gets a request-only identity on req.session: no cookie and no Redis write.
• The /api chain is one function, mountApiAuth, so a test can check its order: token auth, body parsers, session, CSRF, screen lock, guards.
• The CSRF check and the lock gate moved into modules, with the same logic and messages. Only a token that authenticated skips the CSRF check, never a request that merely carries an Authorization header.
• reloadSession re-checks the token after a large upload.
• requireAuth answers a token whose user is gone with 401, and doesn't update the user's last-seen time for token requests.
• requireAdmin refuses tokens outright.

Management API (routes/apiTokens.js):

• list, which also returns reauth, expiryChoices and maxTokens;
• create, up to 20 per user;
• revoke;
• /self, so a token can see its own name and scopes.

Confirming creation (services/apiTokenReauth.js), as settled on #617:

• TOTP: the current two-factor code, when the user has two-factor on and can sign in with a password. It has the same replay guard and totp:<uid> limit as the sign-in challenge.
• Password: otherwise the password, counted against the sign-in budget auth:<ip>.
• SSO: for an account that cannot sign in with a password (password login off, or no password set), a sign-in made through its SSO provider in the last 10 minutes from the token form's Confirm button, /auth/oidc/:slug/start?action=reauth:
  • the authorize request carries prompt=login and max_age=0, so the provider asks for credentials again and must put auth_time in the id_token;
  • the flow records when it began, and the callback refuses an id_token whose auth_time is missing or earlier than that, less a minute of clock skew, since not every provider honours prompt;
  • the session must belong to the user who started the flow, and the identity must be linked to that user; nothing else about the session changes;
  • an ordinary SSO sign-in records no confirmation: the provider may have kept its own session, so it proves no more than the MailFlow session does;
  • each confirmation makes one token. Creating it removes the record from the session and marks its nonce spent in Redis (SET NX, as the two-factor replay guard does), so two requests carrying the same confirmation cannot both pass;
  • a provider that sends no auth_time cannot confirm tokens. Entra ID sends it only with the auth_time optional claim configured; the Settings UI PR says so beside the Confirm button.
• Required two-factor: a user without an authenticator is asked to set one up first.

Expiries are 7, 30, 90 or 365 days, and 7 or 30 for SSO-confirmed tokens. A wrong code or password is logged as totp_fail or login_fail. The confirmation is never answered with a 401, which the web client would take for an expired session.

Revocation.

• A password reset deletes the user's tokens; it now runs as one transaction. So does an admin setting a password.
• Deleting a user cascades.
• A backup restore deletes every token, because api_tokens is not restored.
• A user turning their own two-factor off deletes the tokens it confirmed; those confirmed with the password, just given again, stay.
• A user, or an admin (Linking an SSO identity needs only a signed-in session #714), unlinking the user's last SSO identity at a provider deletes the tokens confirmed through that provider. The rule is unlinkUserIdentity in services/userIdentities.js, which both routes call.

A change that tightens sign-in deletes the tokens it no longer allows, in the transaction that makes the change. The [admin] log line says how many went:

• password login turned off: every token not confirmed through single sign-on;
• two-factor required: tokens confirmed with only a password, while TOTP- and SSO-confirmed tokens stay;
• an admin turning off a user's two-factor: all of that user's tokens;
• an SSO provider turned off, deleted, or pointed at another issuer or client: the tokens confirmed through it, the delete by the foreign key.

Neither the policy nor a revocation can race a creation. createToken takes its locks in the order the revocations take theirs, and checks under each what the confirmation relied on:

• the users row, FOR UPDATE, which a password reset or set, two-factor turned off and an unlink lock first: the password hash, two-factor flag and secret must still be what the confirmation was checked against;
• for a single sign-on confirmation, the provider row FOR SHARE, which must still be on and point at the issuer and client the id_token was checked against (recorded with the confirmation), and the user's identities there FOR KEY SHARE, of which one must remain. FOR KEY SHARE blocks the unlink's delete, not a sign-in updating last_used_at;
• mfa_enforcement and internal_auth_disabled, FOR SHARE.

A change that committed first shows in the rows and the token is refused with 409 ("Confirm again"); one still to come waits for the commit, then sees the token and deletes it with the rest. Deleting a provider waits the same way and cascades.

Limits (numbers as agreed on #617):

• failed attempts: 20 per 15 minutes per IP, in their own bucket apitoken-fail:<ip>, not the sign-in budget. Only malformed or unknown tokens spend it, so a stale token in a tool can't lock its owner out of web sign-in. An expired token always gets 401 and spends nothing;
• valid tokens: 600 requests per token and 1,200 per user, per 5 minutes;
• 60 token drafts per 10 minutes;
• 1 MB per token request;
• token searches share one per-user bucket, separate from the browser's.

A refused body is drained up to 1 MB, then the connection is closed.

Logging. There are three auth events: api_token_created, api_token_revoked and api_token_auth_fail. Neither the token nor its hash is ever logged, and the diagnostics scrubber redacts mfat_.

Where this differs from #617's original text

The thread settled the Bearer rule, the move and archive guards, re-authentication and expiry, the policy changes and the limits. These details were not in it:

1. Revoking deletes the row instead of setting revoked_at, so no lookup can forget to check a flag. The auth events keep the history.
2. Token drafts are new and plain text only. Replacing a draft expunges the old copy, and quotedBodyHtml is stored unsanitized today.
3. GET /api/accounts returns a narrower shape to tokens: no hosts, ports, TLS settings, login names, signatures, folder mappings or plugin settings.
4. Not done:
   • Refusing tokens over plain HTTP (open question 1 in Scoped, revocable API tokens, so scripts and tools don't need the account password #617). On the port-80 server, req.secure reflects whatever the client claims, so the check would mean little.
   • Per-account limits, which Scoped, revocable API tokens, so scripts and tools don't need the account password #617 leaves for version 2.

Testing

• Suite: npm test passes 4,099 tests; the 251 skipped are the PostgreSQL-only ones, which your CI now runs with MAILFLOW_TEST_DATABASE_URL. npm run lint and npm run lint:plugins are clean.

• What the new tests cover:

  • the matcher, against every bypass shape: trailing slash, case, HEAD, encoded or doubled slashes, dot segments, extra segments and non-UUID parameters;
  • route coverage: the test reads index.js's mounts from its source and walks every reviewed router in both directions, the scheduled router included;
  • the whole mountApiAuth chain on a real express-session, Basic credentials included;
  • every guard, the routes, each way of confirming, the OIDC reauth flow (prompt, max_age, auth_time, a plain sign-in confirming nothing), each revocation path (the admin's unlink included), one token per SSO confirmation (two requests at once included), the re-check of what confirmed a token under its locks (each row and lock clause, in order), the policy re-check, and the limits.

  I proved each security check by breaking the code and watching its test fail.

• PostgreSQL 16 and Redis, in a temporary workflow on my fork: https://github.com/Monkey7539/mailflow/actions/runs/41511033774 (the round-2 commit)

  • Database tests: all 238 PostgreSQL tests pass, including the token, guard, policy and re-check SQL tests built on migration 0074, and the identities service with its token revocation. The whole suite also passes with the database (4,348; the 2 skipped are in an upstream test, not this PR's).
  • End to end, against the real backend, all 79 checks pass:
    • First sign-up, with tokens off by default. The admin turns them on, and tokens are created with password confirmation.
    • The choices are 7, 30, 90 and 365 days, and a token that never expires is refused.
    • Allowlisted reads and writes succeed.
    • 16 out-of-scope requests get 403: admin, auth, send, scheduled send, delete, raw .eml, token management, trailing slash, upper case, HEAD and others.
    • A non-mfat Bearer header, and a sign-in behind a proxy that adds one, pass untouched.
    • Revoking a token and the switch both take effect at once.
    • After 20 junk tokens from one address, the next one gets 429, while a valid token from the same address still works.
    • Requiring two-factor deletes the password-confirmed tokens, and turning it off again brings none back.
    • Only hashes are stored, and every token records its confirmation.
  • mailflow-mcp ran over stdio and over HTTP against that live backend.

• Not checked: multi-replica deployments. The search limiter is per process, as it is today.

---

Contributor License Agreement

By submitting this pull request I confirm that:

• I have read and agree to the Contributor License Agreement.
• My contribution is my own original work (or I have identified any
  third-party material and confirmed it is compatible with the CLA).
• I have the right to submit this contribution under the terms of the CLA.

🤖 Generated with Claude Code

[salmonumbrella]

@Monkey7539 yessss!

[Monkey7539]

Rebased onto main 0441d23 (after 4.1.0), still one commit, and the description is updated to match. What the rebase changed:

• Migration: 0072_api_tokens.sql, after 4.1's three. Nothing in it changed.
• Scheduled send: the new routes under /api/mail are in the token review now. None is allowlisted, so a token gets 403 there, as a token may draft but never send. The route-walking test covers the router, and the end-to-end run below checks three of its routes.
• bodyParsers.js: 4.1's /api/mail/scheduled parser sits beside the token-aware draft parser; that was the only conflict.
• Tests: the accounts fixture gained 4.1's two new SAFE_FIELDS columns (oauth_public_client, sync_error_code); the narrowed token view of an account is an explicit field list, so it never included them.

Results on the rebased commit: npm test passes 4,028 (207 PostgreSQL-only skipped locally, which your CI now runs), both lints clean, and the fork's end-to-end run against PostgreSQL 16 and Redis is green: https://github.com/Monkey7539/mailflow/actions/runs/40783550696.

[maathimself]

I reviewed the whole diff, ran the full suite against PostgreSQL 16 (4,233 pass; lint and lint:plugins clean), and had an independent second review done. The allowlist, the session isolation and the CSRF change hold up: neither review found a way for a token to reach anything outside its scopes. What needs changing is how a token's creation is confirmed, plus a few revocation gaps.

Needed before merge

1. SSO confirmation does not make the user sign in again. action=reauth sends the normal authorize request, with no prompt or max_age, and the callback does not look at auth_time, so with a live session at the provider the round trip completes without any input. rememberOidcSession also records a confirmation at every SSO login. So at an unlocked browser, or with a stolen MailFlow session cookie in the 10 minutes after a login, an SSO-only account can get a 30-day token, while an account with a password always has to enter it or a two-factor code. Please:
   • send prompt=login and max_age=0 on the reauth authorize request;
   • store when the reauth flow started in oidcPending, and in the callback refuse an id_token whose auth_time is missing or earlier than that (allowing a minute of clock skew), since not every provider honors prompt;
   • record a confirmation only from the reauth flow, not from ordinary logins. That drops "an SSO sign-in counts" from the design; the token form's Confirm button covers it.
2. fromScheduled on token drafts. routes/draft.js deletes the scheduled message named by fromScheduled (from 4.1). A token cannot learn an edit token today, since only routes/scheduled.js returns one, but a draft-scope request should not carry a delete at all. Please refuse fromScheduled in the draft guard, with a test.
3. Revocation gaps. Please delete tokens in the same transaction as each of these:
   • POST /api/totp/disable: the user's tokens confirmed with totp.
   • DELETE /api/auth/oidc/identities/:id: the user's tokens confirmed through that identity's provider, when they have no other identity left at it.
   • PATCH /api/admin/oidc/:id changing issuer_url or client_id: the provider's tokens, as for enabled: false.
4. Policy race. reauthFor reads the sign-in policy before bcrypt or TOTP runs, and nothing checks it again at insert. If an admin requires two-factor or turns off password sign-in in between, the revoking DELETE cannot see the uncommitted row, and a password-confirmed token survives the change. Re-checking inside the transaction is not enough on its own at READ COMMITTED. In createToken's transaction, read mfa_enforcement and internal_auth_disabled with SELECT … FOR SHARE (both rows are seeded by migrations), and refuse if confirmedWith no longer qualifies. The settings PATCH updates those rows, so the two then wait for each other in either order.

Optional

• The bulk-archive guard refuses only All Mail. Running the resolved archive destination through isProtectedDestination, as bulk-move does, would also cover an archive mapping that points at Trash or Junk.

Separately, linking a new identity (action=link) needs only a signed-in session, so whoever holds a session can link an identity of their own and then use it to confirm. That predates this PR and I'll track it in its own issue; it doesn't block this one.

[Monkey7539]

Thanks for the thorough review, and for the second one. All four are done, plus the optional archive check, as a second commit (8539946) on top of the rebased feature, so the delta is easy to see. The description is updated to match.

1. SSO confirmation is a sign-in just made. Agreed on every part, and the point about rememberOidcSession was the bigger hole: a stolen cookie in the ten minutes after an SSO login got a 30-day token with nothing typed. Now:

• the reauth authorize request carries prompt=login and max_age=0;
• oidcPending records when the flow began, and the callback refuses an id_token whose auth_time is missing or earlier than that, less 60 s of clock skew, with "Sign in again at your provider to confirm";
• only the reauth callback records a confirmation. An ordinary login or a link records nothing, and the tests pin that neither sends prompt.

One consequence to be aware of: a provider that never emits auth_time cannot confirm tokens at all. Entra ID, for one, sends it only with the auth_time optional claim configured. I think refusing is the right default, as you said, since the alternative is trusting that prompt was honoured; it is a line for the SSO docs.

2. fromScheduled. The draft guard refuses it (403, "API tokens cannot remove a scheduled message"), any non-null value, with a test. A token may draft, never take back a message due to go out.

3. Revocation gaps, each in the same transaction as the change:

• POST /api/totp/disable: the user's tokens confirmed with totp. The password-confirmed ones stay, since the password was just given again.
• DELETE /api/auth/oidc/identities/:id: the tokens confirmed through that identity's provider, when the user has no other identity left there. The route now runs on one client inside BEGIN/COMMIT; the last-login-method refusal still happens before anything is written.
• PATCH /api/admin/oidc/:id: a changed issuer_url or client_id deletes the provider's tokens as enabled: false does. Sending the same values back, or whitespace, deletes nothing.

4. Policy race. createToken now reads mfa_enforcement and internal_auth_disabled with SELECT … FOR SHARE after the count and before the insert, and returns a refusal when confirmedWith no longer qualifies, by the same rule revokeTokensForPolicy applies. The route answers 409 (not 401, which signs the client out) with "The server's sign-in policy changed while this token was being created. Confirm again." The settings PATCH's upsert takes the row lock, so the two wait for each other in either order. Both rows are seeded (0004, 0020), as you noted. The unit tests pin the FOR SHARE statement and its place in the transaction; the PostgreSQL test pins the refusal against the real rows.

Optional: archive. Done. The resolved archive folder goes through isProtectedDestination, so a mapping at Trash, Junk, Drafts or Snoozed is refused as well as All Mail, which keeps its own message about an Archive label.

Each new check was proved by breaking it and watching its test fail (11 mutations, all caught). Results: npm test 4,053 pass locally, both lints clean, and the fork's PostgreSQL and end-to-end run on this commit is green: https://github.com/Monkey7539/mailflow/actions/runs/41069390623.

[maathimself]

All four from the last review are in, and they hold up: the fresh SSO sign-in with prompt=login, max_age=0 and the auth_time check, fromScheduled refused, revocation on 2FA off, unlink and provider changes, and FOR SHARE on the policy rows. A few things before merge:

1. Migration number. Main now has 0072_messages_has_attachments_heal.sql (4.1.2) and 0073_email_accounts_sort_order_renumber.sql. Please rebase and rename yours to 0074_api_tokens.sql.
2. A revocation can miss a token created at the same moment. routes/apiTokens.js checks the password, code or SSO confirmation before createToken opens its transaction, and createToken locks the users row but doesn't look at what was confirmed again. A password reset, /totp/disable, an identity unlink or a provider change that commits in between deletes the user's tokens before the new row exists, so the new token survives it. Under the FOR UPDATE on users, please re-read password_hash and totp_enabled and refuse if either changed since the confirmation, and for an SSO confirmation take FOR SHARE on the user_identities row and the provider row. A test per revocation path would be good.
3. An SSO confirmation can be used more than once. oidcConfirmedAt stays in the session for its ten minutes, so one fresh sign-in can create up to the per-user limit of tokens. Please clear it when a token is created.
4. Rebase onto Linking an SSO identity needs only a signed-in session #714's first part. d464f6e on main logs identity_linked/identity_unlinked, removes linked identities on a password reset while password sign-in is on, and lets admins unlink, and it touches oidc.js, auth.js (reset-password) and admin.js (set password), so expect conflicts there. The remaining part of Linking an SSO identity needs only a signed-in session #714, confirming before a link, should reuse your reauth flow: the link start takes the same confirmation as creating a token. I'm happy for that to go in this PR or right after it, your call.
5. Docs. A provider that doesn't send auth_time (Entra, unless the optional claim is configured) can't confirm tokens. Worth a line in the description and the settings hint.

[Monkey7539]

Thanks for round 2. All five are in, as a third commit (7142ce9) on top of the rebase, so the delta is easy to see; the description is updated to match.

1. Migration. Rebased onto dd7944f; the migration is 0074_api_tokens.sql, after main's 0072 and 0073.

2. A revocation at the same moment. createToken now takes the users row the confirmation was checked against and, under its FOR UPDATE, re-reads password_hash, totp_enabled and totp_secret, refusing if any differs. The secret too: two-factor turned off and set up again in between would leave the flag as it was while the tokens it confirmed were deleted. For an SSO confirmation it then takes FOR SHARE on the provider row and refuses if the provider is off or its issuer_url or client_id differ from what the id_token was checked against, which the confirmation now records; then the user's identities at the provider, refusing with none left. I used FOR KEY SHARE for the identities rather than FOR SHARE: it still blocks the unlink's DELETE, but not the ON CONFLICT DO UPDATE SET last_used_at a sign-in does on the same row, so a sign-in never waits on a creation and the two cannot deadlock (the login callback updates the identity before it touches the users row). The lock order, users → provider → identities → policy, is the one the revocations take: each user-level change takes the users row first, a provider change or delete the provider row first, so whichever committed first shows in the rows and refuses the token, and whichever comes second waits for the commit and then deletes it with the rest. The route answers 409 "Your password, two-factor or single sign-on setup changed while this token was being created. Confirm again."

Tests: a unit test per changed column and per provider and identity case, each pinning the statement, its lock clause and the order; PostgreSQL tests for each path against the real rows; route tests for the 409; and a two-connection PostgreSQL test (apiTokens.locks.postgres.test.js) that holds a creation at each of its locks and watches an unlink, a provider change and a policy change wait, then delete the token once it commits, and a creation wait behind a password change, then get refused.

3. One token per SSO confirmation. Creating a token deletes the record from the session. Two requests racing on one session would both still carry it, so confirmOidc also marks the confirmation's nonce spent in Redis with SET NX EX 600, the guard confirmTotp already uses against a replayed code. The route test fires three creations at once on one confirmation and gets 201, 400, 400; the unit test checks the session forgets it and a copy of the record is refused.

4. Rebase onto #714's first part. Done; the conflicts were where you said. The unlink revocation now lives in unlinkUserIdentity in services/userIdentities.js, so the admin's unlink revokes as the user's does, with the count in its log line, and both routes and the service have tests for it. The reset-password route is one transaction that spends the link, locks the users row, sets the password and deletes devices, identities and tokens.

Confirming before a link: I'd do it as the next PR, right after this one, so this diff stays about tokens. The pieces are ready for it: confirmReauth already takes a password, a code or a fresh SSO sign-in and answers with what confirmed or a refusal, so the link start would ask it the same way the token form does and refuse without one.

Also, from a second pass over the round:

• The two password-reset paths remove the linked identities before deleting tokens. Removing them reads the sign-in policy row shared, and the settings PATCH updates that row before it deletes tokens, so with the old order the two could have waited on each other.
• PATCH /api/admin/oidc/:id takes enabled only as a boolean. The column would have read the string "false" as off, past the lockout check and the revocation.
• The SSO confirmation record names the user it vouches for, and confirmOidc checks it, as a cheap second line behind the session's own user id.

5. Docs. The auth_time note is in the description's SSO section and in the module comment. The hint beside the Confirm button is in the UI PR, which I'll word to say the provider has to send auth_time.

Each new check was proved by breaking it and watching its test fail (24 mutations, all caught). Results: npm test 4,099 pass locally, both lints clean, and the fork's PostgreSQL and end-to-end run on this commit is green: https://github.com/Monkey7539/mailflow/actions/runs/41511033774.