Skip to content

feat(serve,proxy): the caller's prompt and tools through the client; rotate exits on the free-tier refusal - #3

Merged
Rethinger merged 1 commit into
masterfrom
feat/proxy-and-client-bridge
Oct 14, 2026
Merged

Rethinger merged 1 commit into
masterfrom
feat/proxy-and-client-bridge

Conversation

@Rethinger

Copy link
Copy Markdown
Owner

what this is

one session's work on both halves of the same problem: getting free-tier models
out of a running opencode client, and keeping the api faithful while it does it.

serve: the caller's prompt, history and tools now cross the boundary

The free tier leases its session to the running client, so serve asks the
client to make the call — but the client's http api takes nothing but text, so
the caller's system prompt, history and tool schemas used to be dropped.

  • each request is parked at <dataDir>/bridge/<sessionId>.json, next to the
    session serve just opened, and the model's tool calls come back through
    <sessionId>.calls.jsonl;
  • examples/lanvello-bridge.ts (install: ~/.config/opencode/plugins/) reads
    that spool in session.hook("context") and rebuilds the model call from it —
    the caller's system prompt replaces the client's agent prompt, the caller's
    history replaces the session's, and the caller's tools replace the client's;
  • the caller's tool calls are blocked inside the client (tool.hook( "execute.before")) and recorded instead, and serve answers with an openai
    tool_calls turn (finish_reason: "tool_calls"), streamed or not. the caller
    runs the tool and sends the result back as a role: "tool" message;
  • a tool-less api request replaces the client's tools with nothing, so it cannot
    run bash in the client's directory;
  • without the plugin the bridge still answers — with the client's own agent and
    its own tools, which is what serve did before this PR.

proxy: exits rotate on the free tier's own signals

  • internal/proxy + lanvello proxy: a local https proxy that terminates the
    client's TLS with a CA minted in the data dir, so a 429/403 is a status
    instead of an opaque stream, and the same request can be replayed on another
    lane before a byte reaches the client;
  • the free tier's 403 FreeTierError body arrives brotli-encoded, so the
    peek is decoded (br/gzip/plain) before matching — a raw-byte match sees
    nothing, which is how a refusal rule silently never fires;
  • a refusal is per exit, like a 429: park the exit, rotate the lane, replay
    the request. only that refusal rotates — any other 403 is an answer the
    caller asked for, and its body reaches the client byte for byte;
  • rotating leaves the lane briefly unusable, so the retry loop now waits
    (laneWait, 30s) for the fresh exit instead of handing back the refusal;
  • lanes.NoteLimited honours the far end's whole retry-after (capped at 6h)
    instead of clamping it to 10 minutes, which used to hand the same spent exit
    straight to the next request.

docs

README: what crosses the boundary now, the client-daemon env (the daemon needs
HTTP(S)_PROXY, NODE_EXTRA_CA_CERTS and NO_PROXY — without NO_PROXY
the client proxies its own loopback api through a lane and dies), and a
reverse-engineering note on the refusal itself.

verification

  • go test ./... green. new tests:

    • TestFreeTierRefusalIsRotatedAndReplayed — a refusing exit is parked, the
      request is replayed, the client sees the replayed answer (not the refusal);
    • TestFreeTierRefusedOnlyOnTheRefusal — br, gzip and plain refusals are
      detected; every other 403 is passed through untouched, byte for byte;
    • TestParseChatRequest* — the caller's system/history/tools reach the spool
      in the shape the plugin rebuilds the model call from.
  • bridge, end to end, without depending on a healthy provider: a request with a
    system prompt and one tool through serve made the client's own outgoing
    request carry exactly, and only, what the caller sent:

    system=[{"type":"text","text":"ZZTEST pirate: answer every question with exactly one word: arr."}]
    messages=[{"role":"user","content":[{"type":"text","text":"ZZTEST what is the capital of France?"}],"metadata":{}}]
    tools=["get_weather_zz"]
    

notes for review

  • live answers are intermittent right now, and not because of this code: the
    free tier hands 403 FreeTierError to almost every egress we have (tor exits
    in us/de/nl/fr/ro were all refused within a minute of traffic; a handful of
    requests still pass). that is what the rotation in this PR exists to exploit,
    and it needs a pool of exits to exploit it with.
  • the tool-call round trip (tool_calls → result → answer) is implemented and
    unit-tested at both ends, but a live model call could not be observed to
    completion for the reason above.
  • AGENTS.md in the working tree is the operator's own file and is deliberately
    not part of this PR.

…nt; rotate exits on the free-tier refusal

serve: park each request (system, history, tool schemas) at <dataDir>/bridge/<sessionId>.json next to the session it opened, rebuild the model call from it in the client-side plugin (examples/lanvello-bridge.ts, installed in ~/.config/opencode/plugins/), block the caller's tool calls inside the client and answer with an openai tool_calls turn instead of executing them there. a tool-less api request replaces the client's tools with nothing.

proxy: the free tier's 403 FreeTierError arrives brotli-encoded, so decode the peek (br/gzip/plain) before matching it; a refusal is per exit, like a 429, so park it, rotate the lane and replay the request (and rotate only on that refusal, not on every 403), wait for the fresh exit instead of answering with the refusal, and honour the far end's whole retry-after in lanes.NoteLimited (capped at 6h) instead of clamping it to 10 minutes.

README: what crosses the boundary now, the client daemon's env (NO_PROXY is not optional), and the refusal in the reverse-engineering notes.
@Rethinger
Rethinger requested a balanced review from Copilot October 14, 2026 19:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Rethinger
Rethinger merged commit 7079f07 into master Oct 14, 2026
11 checks passed
@Rethinger
Rethinger deleted the feat/proxy-and-client-bridge branch October 14, 2026 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants