Repository navigation
feat(serve,proxy): the caller's prompt and tools through the client; rotate exits on the free-tier refusal - #3
Merged
Conversation
…nt; rotate exits on the free-tier refusal serve: park each request (system, history, tool schemas) at <dataDir>/bridge/<sessionId>.json next to the session it opened, rebuild the model call from it in the client-side plugin (examples/lanvello-bridge.ts, installed in ~/.config/opencode/plugins/), block the caller's tool calls inside the client and answer with an openai tool_calls turn instead of executing them there. a tool-less api request replaces the client's tools with nothing. proxy: the free tier's 403 FreeTierError arrives brotli-encoded, so decode the peek (br/gzip/plain) before matching it; a refusal is per exit, like a 429, so park it, rotate the lane and replay the request (and rotate only on that refusal, not on every 403), wait for the fresh exit instead of answering with the refusal, and honour the far end's whole retry-after in lanes.NoteLimited (capped at 6h) instead of clamping it to 10 minutes. README: what crosses the boundary now, the client daemon's env (NO_PROXY is not optional), and the refusal in the reverse-engineering notes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
what this is
one session's work on both halves of the same problem: getting free-tier models
out of a running opencode client, and keeping the api faithful while it does it.
serve: the caller's prompt, history and tools now cross the boundaryThe free tier leases its session to the running client, so
serveasks theclient to make the call — but the client's http api takes nothing but text, so
the caller's system prompt, history and tool schemas used to be dropped.
<dataDir>/bridge/<sessionId>.json, next to thesession
servejust opened, and the model's tool calls come back through<sessionId>.calls.jsonl;examples/lanvello-bridge.ts(install:~/.config/opencode/plugins/) readsthat spool in
session.hook("context")and rebuilds the model call from it —the caller's system prompt replaces the client's agent prompt, the caller's
history replaces the session's, and the caller's tools replace the client's;
tool.hook( "execute.before")) and recorded instead, andserveanswers with an openaitool_callsturn (finish_reason: "tool_calls"), streamed or not. the callerruns the tool and sends the result back as a
role: "tool"message;run bash in the client's directory;
its own tools, which is what
servedid before this PR.proxy: exits rotate on the free tier's own signalsinternal/proxy+lanvello proxy: a local https proxy that terminates theclient's TLS with a CA minted in the data dir, so a
429/403is a statusinstead of an opaque stream, and the same request can be replayed on another
lane before a byte reaches the client;
403 FreeTierErrorbody arrives brotli-encoded, so thepeek is decoded (br/gzip/plain) before matching — a raw-byte match sees
nothing, which is how a refusal rule silently never fires;
429: park the exit, rotate the lane, replaythe request. only that refusal rotates — any other
403is an answer thecaller asked for, and its body reaches the client byte for byte;
(
laneWait, 30s) for the fresh exit instead of handing back the refusal;lanes.NoteLimitedhonours the far end's wholeretry-after(capped at 6h)instead of clamping it to 10 minutes, which used to hand the same spent exit
straight to the next request.
docs
README: what crosses the boundary now, the client-daemon env (the daemon needs
HTTP(S)_PROXY,NODE_EXTRA_CA_CERTSandNO_PROXY— withoutNO_PROXYthe client proxies its own loopback api through a lane and dies), and a
reverse-engineering note on the refusal itself.
verification
go test ./...green. new tests:TestFreeTierRefusalIsRotatedAndReplayed— a refusing exit is parked, therequest is replayed, the client sees the replayed answer (not the refusal);
TestFreeTierRefusedOnlyOnTheRefusal— br, gzip and plain refusals aredetected; every other
403is passed through untouched, byte for byte;TestParseChatRequest*— the caller's system/history/tools reach the spoolin the shape the plugin rebuilds the model call from.
bridge, end to end, without depending on a healthy provider: a request with a
system prompt and one tool through
servemade the client's own outgoingrequest carry exactly, and only, what the caller sent:
notes for review
free tier hands
403 FreeTierErrorto almost every egress we have (tor exitsin us/de/nl/fr/ro were all refused within a minute of traffic; a handful of
requests still pass). that is what the rotation in this PR exists to exploit,
and it needs a pool of exits to exploit it with.
tool_calls→ result → answer) is implemented andunit-tested at both ends, but a live model call could not be observed to
completion for the reason above.
AGENTS.mdin the working tree is the operator's own file and is deliberatelynot part of this PR.