Skip to content

fix(cli): let the TUI wizard create custom relay connections - #3467

Open
UncertaintyDeterminesYou4ndMe wants to merge 1 commit into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:fix/tui-onboarding-custom-relays
Open

fix(cli): let the TUI wizard create custom relay connections#3467
UncertaintyDeterminesYou4ndMe wants to merge 1 commit into
apache:mainfrom
UncertaintyDeterminesYou4ndMe:fix/tui-onboarding-custom-relays

Conversation

@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor

Summary

The TUI setup wizard (/setup and first-run) could not create any of the three custom relay providers — Desktop could. Two gaps, as diagnosed in the issue: the catalog filtered out every provider without a built-in base URL, and the onboarding protocol had no field to carry an endpoint even if they were listed. Per the archaeology in the issue thread, the filter was an explicit phase-1 scope cut in #1254 whose phase-2 base-URL step never landed.

This implements the issue's proposed boundary:

  1. listApiKeyOnboardableProviders() now lists the category: 'custom' relays; requiresBaseUrl tells the wizard to collect an endpoint. Providers whose endpoint is derived rather than user-supplied stay excluded — cloudflare-workers-ai also has an empty registry baseUrl, but it interpolates an account id into a URL template, and Desktop deliberately keeps it out of the base-URL field too; a naive !baseUrl unfilter would have offered it a relay prompt it cannot use.
  2. The wizard gains a base-URL step between provider pick and API key for requiresBaseUrl providers (relays show a 4-step flow, everything else keeps its 3 steps). Input is validated in place with the same rules the Host enforces (http/https, no credentials/query/fragment, 2048-byte cap), so mistakes fail with a readable message instead of a protocol decode error. Blank input is allowed only when a connection already exists — it reuses the persisted endpoint, mirroring the blank-key-reuses-stored-secret pattern.
  3. connection.onboarding.verify/save carry an always-present baseUrl: string | null (exact-record wire style, like apiKey). A non-null value goes through the shared catalog normalizer (normalizeCatalogConnectionBaseUrl; a provider-default value collapses to null). A relay with no endpoint from input, existing connection, or registry is rejected with the new base_url_not_configured reason before any network probe. Since the input shapes and result unions are closed wire schemas, RUNTIME_HOST_COMPATIBILITY_EPOCH moves to 37 so a mixed pair fails the handshake instead of tearing down the session on the first /setup.
  4. Discovery runs against the supplied endpoint, and commitConnectionOnboarding persists it: the journaled onboarding intent gains the field (allowed-but-not-required on read, so an intent journaled by an older build still replays as "no override"), and prepareOnboardingUpsert resolves input ?? previous ?? registry default. Two consistency rules ride along: the no-change early-return also compares the base URL so a URL-only change still commits, and a swapped endpoint drops relayModelProfiles and the last test result — the same endpoint-keyed contract applyConnectionUpdate already enforces, so a new relay does not inherit capability declarations or a "verified" badge from the relay it replaced.

Saving a relay updates the same derived-slug connection Desktop manages, so both surfaces stay in sync.

Fixes #3405

Verification

  • npm --workspace @maka/runtime-host run test — 1064 pass / 0 fail
  • npm --workspace @maka/storage run test — 864 pass / 0 fail (14 pre-existing skips)
  • npm --workspace maka-agent run test (CLI) — 358 pass / 0 fail
  • npm --workspace @maka/desktop run typecheck + tests — clean, 1038 pass (desktop consumes the changed protocol types; it has no onboarding-operation callers)
  • biome check on all changed files — clean; knip output byte-identical to main

New tests (kept deliberately light — five additions, each pinning one contract):

  • protocol: verify/save round-trip with the new field, plus two invalid-URL negatives; the epoch-37 pin alongside the existing epoch ladder
  • coordinator, end to end through real stores: a relay with no endpoint is rejected base_url_not_configured before any probe; a supplied endpoint reaches discovery and persists; a blank re-verify reuses the persisted endpoint; a re-onboarding that swaps the URL persists it and drops the old relay's profile table
  • storage: the onboarding-intent journal round-trips the endpoint, and a journal written by a build that predates the field still replays (crash-recovery compatibility)
  • TUI runner, end to end: filtering to a relay inserts the base-URL step (2/4), a malformed endpoint is rejected in place, and the endpoint threads through both verify and save

An independent adversarial review pass probed the change with running experiments before submission; it surfaced five defects — the missing compatibility-epoch bump, the cloudflare-workers-ai scope leak, stale endpoint-keyed state surviving a URL swap, a missing local byte cap, and two formatter misses — all fixed and re-verified above, with its blank-reuse/crash-replay/normalizer/state-machine probes coming back clean. Details in the review-record comment on this PR.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code implemented the fix across the protocol/coordinator/storage/TUI layers, wrote the tests, and ran the verification; an independent adversarial review pass (also Claude) probed the change with fault-injection experiments and its findings were fixed before submission. I reviewed and verified the result.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

The setup wizard filtered out every provider without a built-in base
URL — an explicit phase-1 scope cut (apache#1254) whose phase-2 base-URL
prompt never landed — and the onboarding protocol had no field to carry
an endpoint anyway, so the three custom relays were creatable from
Desktop but unreachable from the TUI.

List the category:'custom' relays (cloudflare-workers-ai stays out:
its endpoint is an account-id template, not a user-supplied URL), add
a base-URL step to the wizard between provider pick and API key with
host-mirroring local validation, and thread an always-present
'baseUrl: string | null' through connection.onboarding.verify/save —
exact-record wire style like apiKey, normalized by the shared catalog
rules, rejected as base_url_not_configured when a relay has no endpoint
from input, existing connection, or registry. Blank input on an
existing relay reuses its persisted endpoint, mirroring the blank-key
contract.

Discovery runs against the supplied endpoint and commit persists it:
the intent journal gains the field (legacy journals still replay), the
upsert resolves input ?? previous ?? registry default, a URL-only
change still commits, and a swapped endpoint drops relayModelProfiles
and lastTest — the endpoint-keyed contract the update path already
enforces. The onboarding wire shapes are closed schemas, so the
compatibility epoch moves to 37.

Fixes apache#3405

Generated-by: Claude Code
@UncertaintyDeterminesYou4ndMe

Copy link
Copy Markdown
Contributor Author

Adversarial review record (pre-submission)

An independent review pass probed the change with running experiments (driven wizard sessions against the built TUI, storage-level probes through the real document stack, protocol decode drills, dist fault injections). It found five defects, all fixed in the submitted head, plus confirmations:

Defects found and fixed

  1. Compatibility epoch not bumped (blocker). baseUrl is required on two closed request shapes and the result unions gained a rejection reason. It demonstrated both mixed-pair directions dying: the frame decode throws inside the session pump and tears the whole Host session down on the first /setup. Fixed: RUNTIME_HOST_COMPATIBILITY_EPOCH = 37, pinned by a protocol test alongside the existing epoch ladder.
  2. cloudflare-workers-ai leaked into the wizard. Four providers have an empty registry baseUrl, not three — Cloudflare's endpoint is an account-id template that Desktop deliberately keeps out of its base-URL field. Verified by driving the real wizard: it landed on a relay prompt it cannot use. Fixed: the catalog lists empty-baseUrl providers only when category === 'custom'; verified the offered set is now exactly the three relays.
  3. A swapped endpoint kept endpoint-keyed state. Re-onboarding relay-a → relay-b retained relayModelProfiles and a "verified" lastTest — state the applyConnectionUpdate path explicitly drops on endpoint change (its comment: the old table must not outlive the relay it described). Demonstrated side by side through the real storage stack. Fixed: prepareOnboardingUpsert derives endpointChanged and feeds it into both the profile branch and the test-basis reset; pinned by an end-to-end coordinator test.
  4. Local URL validation omitted the 2048-byte cap, so an oversized endpoint sailed past the wizard and hit the host's frame decode (fatal per defect 1). Fixed in the wizard's validator.
  5. Two formatter misses the author's earlier check pipeline had swallowed. Fixed.

It also flagged the storage half as untested — two dist fault injections (removing the URL-change commit condition; making the intent field required) survived every existing suite. The submitted head adds a storage intent-journal test (round-trip + legacy replay) and the endpoint-swap coordinator assertions to close the reachable half of that gap.

Confirmed safe (with evidence)

  • Blank-reuse and crash-recovery replay: a blank save preserves the persisted URL; a hand-written legacy journal (no baseUrl key) replays and applies its model change without touching the URL.
  • URL replacement probes the new endpoint while reusing the stored secret, and persists at the next revision.
  • Normalizer boundary: empty/whitespace → null; query/fragment/credentials/ftp/file/javascript/non-string/oversize all rejected; provider-default collapses to null; decode idempotent; the OAuth-override throw is unreachable (OAuth providers never pass providerAuthSupportsApiKey).
  • Wizard state machine: 2/4→3/4→4/4 labels; Esc walks exactly one level (models→key→baseUrl→search); a re-entered URL is what reaches verify and save; errors clear on typing; a late verify result after Esc is dropped; a non-relay picked after abandoning a relay carries no leaked URL.
  • Rejection ordering: a blank key wins over a missing endpoint, and no discovery probe is issued on any rejection.
  • Bypassing the coordinator guard cannot poison the catalog (an endpoint-less relay entry is unusable but loadable).
  • Intent schemaVersion staying at 1 is correct: strict-equality versioning would break the crash-then-upgrade replay this change is designed to support, while a downgrade already fails closed on the unknown field.

Two observations left as-is, for the record: the protocol accepts an endpoint override for any api-key provider (Desktop's settings already allow the same, and the TUI only offers the step for relays); hasConnection is slug-derived, so a Desktop-created relay under a custom slug reads as new in the wizard — pre-existing design, noted since relays are now listed.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for extending the existing Runtime Host onboarding seam instead of adding a CLI-only connection path. The protocol, validation, journal replay, endpoint-change cleanup, and TUI lifecycle all line up well on this head. One existing-connection identity gap becomes user-visible for custom relays, noted inline.

AI-assisted review disclosure: OpenAI Codex coordinated two independent exact-head review passes. I verified the retained connection-identity path, current CI state, reviews, and mergeability, and I made the final review decision.

providerType: input.providerType,
apiKey: normalizedSecret(input.apiKey),
apiKey: trimmedOrNull(input.apiKey),
baseUrl: trimmedOrNull(input.baseUrl),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] The new relay URL is saved by provider type, but projectProviders() recognizes an existing connection only at the derived canonical slug. A relay created in Desktop with a valid custom slug is therefore shown as unconfigured in /setup; saving here creates a second canonical-slug connection instead of editing/reusing the existing one, leaving the old credential and default target behind. Could onboarding carry the actual existing connectionId/slug (or preserve the sole connection for this provider type) through verify/save, with a Desktop custom-slug → TUI re-onboard test? That keeps connection identity authoritative instead of deriving a second one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(cli): TUI onboarding cannot create custom relay connections (OpenAI Chat / OpenAI Responses / Anthropic) — Desktop can

2 participants