diff --git a/free/README.md b/free/README.md index c01b469..ab82c78 100644 --- a/free/README.md +++ b/free/README.md @@ -6,20 +6,21 @@ It covers **Claude, ChatGPT (Codex) and Kimi subscriptions** and **API keys** fo - CPA's dashboard, downloaded at build time from the latest panel release (pin one with `PANEL_VERSION`, and its checksum with `PANEL_SHA256`). - Claude and ChatGPT (Codex) OAuth logins from the dashboard's **OAuth Login** page (the same PKCE flows as CPA, Claude Code and the Codex CLI), with tokens refreshed automatically. -- Kimi logins (kimi.com or kimi.ai) through Kimi's device sign-in, as in CPA. Claude Code and both OpenAI formats reach Kimi untranslated: ask for a Kimi model such as `kimi-k3` or `kimi-k2.8`. Requests name this gateway, as CPA's name CLIProxyAPI. +- Kimi logins (kimi.com or kimi.ai) through Kimi's device sign-in, as in CPA. Claude Code and both OpenAI formats reach Kimi untranslated, and Gemini clients through Anthropic Messages: ask for a Kimi model such as `kimi-k3` or `kimi-k2.8`. Requests name this gateway, as CPA's name CLIProxyAPI. - Several logins per provider, used in turn, with a failing or rate-limited Claude or ChatGPT login skipped for the next. Upload, download, disable, edit and delete them under **Auth Files**, in CPA's auth-file format, so files move between the two installs. - CPA's YAML config, edited in the **Config Panel**. Client API keys live in `access.api-keys`, as in CPA. - `POST /v1/messages` and `POST /v1/messages/count_tokens`, streamed straight through to Anthropic. - API-key providers added under **AI Providers** (CPA's `api-keys` config): Claude, Gemini, Vertex AI, Codex, xAI and OpenAI-compatible endpoints such as OpenRouter. Model aliases, `prefix/model` routing, priorities, key weights (with the `weighted-round-robin` strategy) and excluded models work as in CPA, and a request moves on to the next key when one is rate limited or failing. - Codex CLI through a ChatGPT login, on `/backend-api/codex/responses` or `/v1/responses`. Other OpenAI Responses clients work too: their requests are adapted to what the ChatGPT endpoint takes, and a non-streaming request gets the final response as JSON. - Claude Code on an OpenAI-compatible key (OpenRouter and the like): ask for one of the provider's model aliases and the request goes out as Chat Completions, with the answer streamed back as Anthropic events, following CPA's Claude-to-OpenAI translator. -- Vertex AI service accounts imported under **OAuth Login** (a Google Cloud key file and a region, as in CPA). The Durable Object signs in to Google with the key and refreshes the token hourly. Gemini clients use them as they are, and Claude Code through the Gemini translator below. +- Vertex AI service accounts imported under **OAuth Login** (a Google Cloud key file and a region, as in CPA). The Durable Object signs in to Google with the key and refreshes the token hourly. Gemini clients use them as they are, and other clients through the translators below. - xAI keys take OpenAI Responses and Chat clients as they are, and Claude Code through the Claude-to-Responses translator below (for example `ANTHROPIC_MODEL=grok-4.5`). - Claude Code on a ChatGPT login or a Codex API key: ask for a ChatGPT model (for example `ANTHROPIC_MODEL=gpt-5.6-sol`) and the request is translated to OpenAI Responses and the answer streamed back as Anthropic events, with thinking, tool calls and images, following CPA's Claude-to-Codex translator. - Claude Code on a Gemini API key: ask for a Gemini model (for example `ANTHROPIC_MODEL=gemini-2.5-flash`) and the request goes out as Gemini `generateContent`, with thinking, tool calls and images, following CPA's Claude-to-Gemini translator. Unlike CPA, Google's default safety filters stay on; a blocked answer ends with a `refusal` stop reason. - OpenAI Chat clients (Cursor, Cline, the OpenAI SDKs and the like) on a Claude API key: ask `/v1/chat/completions` for a Claude model and the request goes out as Anthropic Messages, with reasoning effort, tool calls, images, files and prompt caching, and the answer comes back as Chat Completions, following CPA's OpenAI-to-Claude translator. Claude logins aren't offered to these clients, since without CPA's cloaking Anthropic accepts a subscription token from Claude Code only. - OpenAI Responses clients on a Claude API key, the same way, following CPA's Responses-to-Claude translator: reasoning, function and custom tool calls, images and files, streamed as Responses events or returned whole. -- OpenAI Chat and Responses clients on any other provider too (a Gemini key, an OpenAI-compatible key, a Codex key or a ChatGPT login): the request is translated to Anthropic Messages and on to the provider's format, and the answer back the same way. +- Gemini clients (the Gemini SDKs, Gemini CLI with an API key) on a Claude API key, following CPA's Gemini-to-Claude translator: function calls and results, images and files, thinking settings, streamed with or without `?alt=sse` or returned whole. +- OpenAI Chat, Responses and Gemini clients on any other provider too (a Gemini key, an OpenAI-compatible key, a Codex key or a ChatGPT login): the request is translated to Anthropic Messages and on to the provider's format, and the answer back the same way. - Each client talks in its own format: Anthropic (`/v1/messages`), OpenAI Chat (`/v1/chat/completions`), OpenAI Responses (`/v1/responses`) and Gemini (`/v1beta/models/{model}:generateContent`). A request goes to a provider that takes the same format, apart from the translations above. - Model lists as in CPA: `GET /v1/models` (OpenAI's shape, or Anthropic's for Claude Code and the Anthropic SDKs), `GET /v1beta/models` and `GET /v1beta/models/{model}` list every model across the logins and API keys, and so does the dashboard's **Available Models**. A login or key with no model list of its own lists CPA's catalog for its provider, and a ChatGPT login the catalog for its plan. The catalog is refreshed from CPA's model catalog every three hours, with a built-in copy as fallback; it only decides what is listed, never where a request goes. - Usage and logs as in CPA: each login and API key shows its success and failure counts and the last 200 minutes in 10-minute buckets, and with **Logging to file** on, the **Logs** page shows a line per API request (the last 2,000). Lines hold the time, status, latency, client IP, path, provider and model, never request bodies or keys. Records are sent to the Durable Object in batches a few seconds apart. @@ -28,7 +29,7 @@ It covers **Claude, ChatGPT (Codex) and Kimi subscriptions** and **API keys** fo Claude Code and Codex CLI request bodies are never parsed, so a Worker spends well under 1 ms of its 10 ms CPU budget even on a 600 KB Claude Code request (see the CPU test in `test/app.test.ts`). Requests that need a model rewritten or a body adapted are parsed once. Translated requests cost the most, about 4 to 6 ms for a 600 KB conversation; very long conversations translated can run past the free plan's 10 ms, which the Container install doesn't have. -Not included yet: Gemini clients on other providers, Gemini and other OAuth logins, the Codex CLI's own model list (`/v1/models?client_version=`), and CPA's per-request detail logs. Deliberately left out: CPA's request cloaking and TLS fingerprint spoofing. Clients other than Claude Code may be refused by Anthropic when they use a subscription token. +Not included yet: Gemini and other OAuth logins, the Codex CLI's own model list (`/v1/models?client_version=`), and CPA's per-request detail logs. Deliberately left out: CPA's request cloaking and TLS fingerprint spoofing. Clients other than Claude Code may be refused by Anthropic when they use a subscription token. ## Try it diff --git a/free/src/app.ts b/free/src/app.ts index 5494be2..cb77ab1 100644 --- a/free/src/app.ts +++ b/free/src/app.ts @@ -4,6 +4,7 @@ import { claudeToGemini, geminiClaudeMessage, GeminiClaudeStream } from "./claud import { chatClaudeMessage, ChatClaudeStream, claudeToChat } from "./claude-openai.ts"; import { claudeEventStream, claudeMessage, ClaudeStream, claudeToResponses, type EventTranslator, type Json } from "./claude-responses.ts"; import { chatCompletion, ChatStream, chatToClaude } from "./openai-claude.ts"; +import { geminiResponse, GeminiStream, geminiToClaude } from "./gemini-claude.ts"; import { kimiUpstreamModel } from "./kimi-oauth.ts"; import { responsesResponse, ResponsesStream, responsesToClaude } from "./responses-claude.ts"; import { MANAGEMENT_PREFIX } from "./management.ts"; @@ -241,12 +242,6 @@ async function viaCodexLogin(request: Request, url: URL, config: AppConfig, text return collected(upstream); } -// Formats a provider takes with translation: Claude, OpenAI Chat and Responses clients reach -// every provider through Anthropic Messages. -function servable(format: Format, kind: Candidate["credential"]["kind"]): boolean { - return takesNatively(kind, format) || format !== "gemini"; -} - function parsedObject(text: string): Record | null { try { const value: unknown = JSON.parse(text); @@ -320,14 +315,20 @@ async function translatedClaude(request: Request, route: Route, config: AppConfi return Response.json(claudeMessage((await final.json()) as Record, requested, translated.toolNames), { headers: { "cache-control": "no-store" } }); } -// An OpenAI Chat or Responses request: the body is translated to Anthropic Messages and sent +// An OpenAI Chat, Responses or Gemini request: the body is translated to Anthropic Messages and sent // to a Claude API key, or translated on to the candidate's own format; the answer comes back // the same way, streamed or whole. async function viaClaudeMessages(request: Request, url: URL, route: Route & { format: Format }, config: AppConfig, candidate: Candidate, text: string): Promise { const client = parsedObject(text); if (client === null) return formatError(route.format, 400, "invalid_request_error", "The request body must be a JSON object."); - const requested = typeof client.model === "string" ? client.model : candidate.model; - const translated = route.format === "openai-response" ? responsesToClaude(client, candidate.model) : { ...chatToClaude(client, candidate.model), customTools: new Set() }; + const requested = route.geminiModel ?? (typeof client.model === "string" ? client.model : candidate.model); + if (route.geminiAction === "countTokens") return Response.json({ totalTokens: Math.ceil(text.length / 4) }, { headers: { "cache-control": "no-store" } }); + const translated = + route.format === "openai-response" + ? responsesToClaude(client, candidate.model) + : route.format === "gemini" + ? { ...geminiToClaude(client, candidate.model, route.geminiAction === "streamGenerateContent"), customTools: new Set() } + : { ...chatToClaude(client, candidate.model), customTools: new Set() }; let upstream: Response | null; if (NATIVE_FORMAT[candidate.credential.kind] === "claude") { // The client's own headers describe an OpenAI request, so none of them go upstream. @@ -340,12 +341,24 @@ async function viaClaudeMessages(request: Request, url: URL, route: Route & { fo if (upstream === null) return formatError(route.format, 502, "api_error", "The upstream couldn't be reached."); if (!upstream.ok) return formatError(route.format, upstream.status, upstream.status === 429 ? "rate_limit_error" : "api_error", await upstreamMessage(upstream)); if (translated.stream && upstream.body !== null) { - const translator: EventTranslator = route.format === "openai-response" ? new ResponsesStream(requested, translated.toolNames, translated.customTools) : new ChatStream(requested, translated.toolNames); - return new Response(claudeEventStream(upstream.body, translator), { headers: { "content-type": "text/event-stream; charset=utf-8", "cache-control": "no-cache" } }); + // Gemini streams SSE only with ?alt=sse, and a JSON array otherwise. + const sse = route.format !== "gemini" || url.searchParams.get("alt") === "sse"; + const translator: EventTranslator = + route.format === "openai-response" + ? new ResponsesStream(requested, translated.toolNames, translated.customTools) + : route.format === "gemini" + ? new GeminiStream(requested, translated.toolNames, sse) + : new ChatStream(requested, translated.toolNames); + return new Response(claudeEventStream(upstream.body, translator), { headers: { "content-type": sse ? "text/event-stream; charset=utf-8" : "application/json; charset=utf-8", "cache-control": "no-cache" } }); } const final = parsedObject(await upstream.text()); if (final === null) return formatError(route.format, 502, "api_error", "The upstream sent a response that isn't JSON."); - const answer = route.format === "openai-response" ? responsesResponse(final, requested, translated.toolNames, translated.customTools) : chatCompletion(final, requested, translated.toolNames); + const answer = + route.format === "openai-response" + ? responsesResponse(final, requested, translated.toolNames, translated.customTools) + : route.format === "gemini" + ? geminiResponse(final, requested, translated.toolNames) + : chatCompletion(final, requested, translated.toolNames); return Response.json(answer, { headers: { "cache-control": "no-store" } }); } @@ -477,7 +490,7 @@ async function proxied(request: Request, url: URL, route: Route & { format: Form const claudeLogins = logins.filter((login) => login.provider === "claude"); const codexLogins = logins.filter((login) => login.provider === "codex"); // Claude clients with one Claude login and nothing else: stream the body through unread. - if (route.format === "claude" && claudeLogins.length === 1 && codexLogins.length === 0 && !table.vertexOAuth && !table.kimiOAuth && !table.credentials.some((credential) => servable("claude", credential.kind))) { + if (route.format === "claude" && claudeLogins.length === 1 && codexLogins.length === 0 && !table.vertexOAuth && !table.kimiOAuth && table.credentials.length === 0) { const only = loginCandidate(claudeLogins[0] as LoginEntry, ""); const response = await viaClaudeLogin(request, url, config, undefined, loginName(only) ?? ""); noteAttempt(trace, config, only, response.status); @@ -491,7 +504,9 @@ async function proxied(request: Request, url: URL, route: Route & { format: Form if (model === null) return formatError(route.format, 400, "invalid_request_error", "The request needs a model."); trace.model = model; const found = candidates(table, model); - const usable = found.filter((candidate) => servable(route.format, candidate.credential.kind)); + // Every client format reaches every provider: Claude requests are translated to the + // provider's format, and the others through Anthropic Messages. + const usable = [...found]; // "prefix/model" picks the logins with that prefix; other names may use any login. const { prefix, model: bare } = splitModel(table, model); const loginsFor = (provider: LoginEntry["provider"]) => (provider === "claude" ? claudeLogins : codexLogins).filter((login) => prefix === "" || login.prefix === prefix).map((login) => loginCandidate(login, bare)); @@ -500,10 +515,11 @@ async function proxied(request: Request, url: URL, route: Route & { format: Form if (route.format === "openai-response" && (inDefaultFamily("codex", bare) || usable.length === 0)) usable.push(...loginsFor("codex")); // Claude Code or a Chat client asking for a ChatGPT model by name, e.g. ANTHROPIC_MODEL=gpt-5.6-sol. if ((route.format === "claude" || route.format === "openai") && inDefaultFamily("codex", bare)) usable.push(...loginsFor("codex")); - // Vertex service accounts take Gemini models, from Gemini clients and translated Claude ones. - if ((route.format === "gemini" || route.format === "claude") && table.vertexOAuth === true && inDefaultFamily("vertex", model)) usable.push(servedCandidate("vertex", model)); - // Kimi logins take Kimi's models in Anthropic and both OpenAI formats, untranslated. - if (route.format !== "gemini" && table.kimiOAuth === true && inDefaultFamily("kimi", model)) usable.push(servedCandidate("kimi", kimiUpstreamModel(model))); + // Vertex service accounts take Gemini models, from any client, translated where needed. + if (table.vertexOAuth === true && inDefaultFamily("vertex", model)) usable.push(servedCandidate("vertex", model)); + // Kimi logins take Kimi's models in Anthropic and both OpenAI formats untranslated, and + // Gemini clients through Anthropic Messages. + if (table.kimiOAuth === true && inDefaultFamily("kimi", model)) usable.push(servedCandidate("kimi", kimiUpstreamModel(model))); if (usable.length === 0) { if (route.format === "claude" && table.claudeOAuth) { const response = await viaClaudeLogin(request, url, config, text); @@ -514,8 +530,6 @@ async function proxied(request: Request, url: URL, route: Route & { format: Form const reason = route.format !== "claude" && table.claudeOAuth && /^claude-/i.test(model) ? `Claude logins serve Claude Code only. To use ${model} from another client, add a Claude API key under AI Providers in the dashboard.` - : found.length > 0 - ? `Model ${model} is served by a ${found[0]?.credential.kind} provider, which doesn't take this request format yet.` : `No provider is configured for model ${model}. Add one under AI Providers in the dashboard.`; return formatError(route.format, 400, "invalid_request_error", reason); } diff --git a/free/src/gemini-claude.ts b/free/src/gemini-claude.ts new file mode 100644 index 0000000..cd5a559 --- /dev/null +++ b/free/src/gemini-claude.ts @@ -0,0 +1,292 @@ +import { isObject, text, type EventTranslator, type Json } from "./claude-responses.ts"; +import { claudeName, DEFAULT_MAX_TOKENS, FABLE_MAX_TOKENS, formatInstruction, inputSchema, mergeUsage, thinking, toolId, type ClaudeRequest, type Usage } from "./openai-claude.ts"; + +// Translation between Gemini generateContent clients and Anthropic Messages, ported from CPA's +// translator/claude/gemini. Other providers are reached by translating the Claude request on. + +function inline(part: Json): Json | null { + const data = isObject(part.inlineData) ? part.inlineData : isObject(part.inline_data) ? part.inline_data : null; + if (data === null) return null; + const mediaType = text(data.mimeType) || text(data.mime_type); + const payload = text(data.data); + if (mediaType === "" || payload === "") return null; + const source = { type: "base64", media_type: mediaType, data: payload }; + return mediaType.startsWith("image/") ? { type: "image", source } : { type: "document", source }; +} + +function fileReference(part: Json): Json | null { + const file = isObject(part.fileData) ? part.fileData : isObject(part.file_data) ? part.file_data : null; + const url = file === null ? "" : text(file.fileUri) || text(file.file_uri); + if (!/^https:\/\//i.test(url)) return null; + const mediaType = file === null ? "" : text(file.mimeType) || text(file.mime_type); + return mediaType.startsWith("image/") ? { type: "image", source: { type: "url", url } } : { type: "document", source: { type: "url", url } }; +} + +function responseContent(response: unknown): string { + if (typeof response === "string") return response; + if (isObject(response) && Object.keys(response).length === 1) { + const only = response.output ?? response.result ?? response.content; + if (typeof only === "string") return only; + } + return JSON.stringify(response ?? {}); +} + +function systemText(value: unknown): Json[] { + if (typeof value === "string") return value.trim() === "" ? [] : [{ type: "text", text: value }]; + if (!isObject(value)) return []; + return (Array.isArray(value.parts) ? value.parts.filter(isObject) : []).flatMap((part) => (text(part.text).trim() === "" ? [] : [{ type: "text", text: text(part.text) }])); +} + +// Gemini's thinking settings as an effort: a level as is, a budget by its size. +function effortOf(config: Json): string { + const level = text(config.thinkingLevel ?? config.thinking_level).toLowerCase(); + if (level !== "") return level; + const budget = config.thinkingBudget ?? config.thinking_budget; + if (typeof budget !== "number") return config.includeThoughts === true || config.include_thoughts === true ? "auto" : ""; + if (budget < 0) return "auto"; + if (budget === 0) return "none"; + return budget <= 1024 ? "low" : budget <= 8192 ? "medium" : budget <= 24576 ? "high" : "xhigh"; +} + +export function geminiToClaude(request: Json, model: string, stream: boolean): ClaudeRequest { + const toolNames = new Map(); + const nameOf = (name: string) => { + const short = claudeName(name); + toolNames.set(short, name); + return short; + }; + const system = systemText(request.systemInstruction ?? request.system_instruction); + const messages: { role: string; content: Json[] }[] = []; + // Gemini calls carry no ID unless the client set one, so results pair with calls by name, in order. + const pending = new Map(); + let synthesized = 0; + for (const content of Array.isArray(request.contents) ? request.contents.filter(isObject) : []) { + const role = content.role === "model" ? "assistant" : "user"; + const blocks: Json[] = []; + const results: Json[] = []; + for (const part of Array.isArray(content.parts) ? content.parts.filter(isObject) : []) { + // Gemini's own thoughts and signatures can't be replayed to Claude. + if (part.thought === true) continue; + if (typeof part.text === "string") { + if (part.text !== "") blocks.push({ type: "text", text: part.text }); + continue; + } + const call = isObject(part.functionCall) ? part.functionCall : isObject(part.function_call) ? part.function_call : null; + if (call !== null && role === "assistant") { + const name = text(call.name); + const id = text(call.id) === "" ? `toolu_gemini_${synthesized++}` : toolId(text(call.id)); + pending.set(name, [...(pending.get(name) ?? []), id]); + blocks.push({ type: "tool_use", id, name: nameOf(name), input: isObject(call.args) ? call.args : {} }); + continue; + } + const answer = isObject(part.functionResponse) ? part.functionResponse : isObject(part.function_response) ? part.function_response : null; + if (answer !== null) { + const name = text(answer.name); + const queue = pending.get(name) ?? []; + const wanted = text(answer.id) === "" ? "" : toolId(text(answer.id)); + const at = wanted === "" ? 0 : queue.indexOf(wanted); + const id = at >= 0 ? queue.splice(at, 1)[0] : undefined; + const content = responseContent(answer.response); + results.push(id === undefined ? { type: "text", text: `Tool result ${name}: ${content}` } : { type: "tool_result", tool_use_id: id, content }); + continue; + } + const media = inline(part) ?? fileReference(part); + if (media !== null) blocks.push(media); + } + const ordered = [...results, ...blocks]; + if (ordered.length === 0) continue; + const last = messages.at(-1); + if (last !== undefined && last.role === role) last.content.push(...ordered); + else messages.push({ role, content: ordered }); + } + // Calls left unanswered get an error result, so Claude always sees each call answered. + for (let index = 0; index < messages.length; index += 1) { + const message = messages[index] as { role: string; content: Json[] }; + if (message.role !== "assistant") continue; + const ids = message.content.filter((block) => block.type === "tool_use").map((block) => text(block.id)); + if (ids.length === 0) continue; + let next = messages[index + 1]; + if (next?.role !== "user") { + next = { role: "user", content: [] }; + messages.splice(index + 1, 0, next); + } + const answered = new Set(next.content.filter((block) => block.type === "tool_result").map((block) => text(block.tool_use_id))); + const missing = ids.filter((id) => !answered.has(id)).map((id) => ({ type: "tool_result", tool_use_id: id, content: "Tool call was interrupted before any output was recorded.", is_error: true })); + next.content.unshift(...missing); + } + if (messages.length === 0) messages.push({ role: "user", content: [{ type: "text", text: "" }] }); + + const config = isObject(request.generationConfig) ? request.generationConfig : isObject(request.generation_config) ? request.generation_config : {}; + const requestedMax = config.maxOutputTokens ?? config.max_output_tokens; + const maxTokens = typeof requestedMax === "number" ? requestedMax : /fable/i.test(model) ? FABLE_MAX_TOKENS : DEFAULT_MAX_TOKENS; + const body: Json = { model, max_tokens: maxTokens, messages, stream }; + const mime = text(config.responseMimeType ?? config.response_mime_type); + if (mime === "application/json") { + const schema = config.responseJsonSchema ?? config.response_json_schema ?? config.responseSchema ?? config.response_schema; + const instruction = formatInstruction(schema === undefined ? { type: "json_object" } : { type: "json_schema", schema }); + system.push({ type: "text", text: instruction }); + } + if (system.length > 0) body.system = system; + const topP = config.topP ?? config.top_p; + if (typeof topP === "number") body.top_p = topP; + const stops = config.stopSequences ?? config.stop_sequences; + if (Array.isArray(stops) && stops.length > 0) body.stop_sequences = stops.map(String); + const thinkingConfig = config.thinkingConfig ?? config.thinking_config; + Object.assign(body, thinking(isObject(thinkingConfig) ? effortOf(thinkingConfig) : "", model, maxTokens)); + + const tools: Json[] = []; + for (const group of Array.isArray(request.tools) ? request.tools.filter(isObject) : []) { + const declarations = group.functionDeclarations ?? group.function_declarations; + for (const declaration of Array.isArray(declarations) ? declarations.filter(isObject) : []) { + if (text(declaration.name) === "") continue; + tools.push({ name: nameOf(text(declaration.name)), description: text(declaration.description), input_schema: inputSchema(lowerTypes(declaration.parametersJsonSchema ?? declaration.parameters)) }); + } + } + if (tools.length > 0) body.tools = tools; + const toolConfig = isObject(request.toolConfig) ? request.toolConfig : isObject(request.tool_config) ? request.tool_config : {}; + const calling = isObject(toolConfig.functionCallingConfig) ? toolConfig.functionCallingConfig : isObject(toolConfig.function_calling_config) ? toolConfig.function_calling_config : {}; + const mode = text(calling.mode).toUpperCase(); + const allowed = calling.allowedFunctionNames ?? calling.allowed_function_names; + if (mode === "NONE") body.tool_choice = { type: "none" }; + else if ((mode === "ANY" || mode === "VALIDATED") && tools.length > 0) { + const names = Array.isArray(allowed) ? allowed.map(String) : []; + body.tool_choice = names.length === 1 ? { type: "tool", name: nameOf(names[0] ?? "") } : { type: "any" }; + } + return { body, stream, toolNames }; +} + +// Gemini's OpenAPI schemas spell types in capitals ("OBJECT"); JSON Schema wants them lower. +function lowerTypes(value: unknown): unknown { + if (Array.isArray(value)) return value.map(lowerTypes); + if (!isObject(value)) return value; + return Object.fromEntries(Object.entries(value).map(([key, child]) => [key, key === "type" && typeof child === "string" ? child.toLowerCase() : lowerTypes(child)])); +} + +function finishReason(stop: string): string { + if (stop === "max_tokens" || stop === "model_context_window_exceeded") return "MAX_TOKENS"; + if (stop === "refusal") return "SAFETY"; + return "STOP"; +} + +function geminiUsage(usage: Usage): Json { + const prompt = usage.input + usage.cacheWrite + usage.cacheRead; + const result: Json = { promptTokenCount: prompt, candidatesTokenCount: usage.output, totalTokenCount: prompt + usage.output }; + if (usage.cacheRead > 0) result.cachedContentTokenCount = usage.cacheRead; + if (usage.thinking !== null) result.thoughtsTokenCount = usage.thinking; + return result; +} + +function functionCall(name: string, args: Json, id: string, toolNames: Map): Json { + return { functionCall: { name: toolNames.get(name) ?? name, args, id } }; +} + +// Turns Anthropic events into Gemini stream chunks: SSE with ?alt=sse, otherwise one JSON +// array written as it goes, as Google's API does. +export class GeminiStream implements EventTranslator { + private id = ""; + private readonly usage: Usage = { input: 0, output: 0, cacheWrite: 0, cacheRead: 0, thinking: null }; + private readonly tools = new Map(); + private sent = 0; + private finished = false; + + constructor( + private readonly model: string, + private readonly toolNames: Map, + private readonly sse: boolean, + ) {} + + private chunk(parts: Json[], finish?: string): string { + const candidate: Json = { content: { role: "model", parts }, index: 0 }; + const data: Json = { candidates: [candidate], modelVersion: this.model, responseId: this.id }; + if (finish !== undefined) { + candidate.finishReason = finish; + data.usageMetadata = geminiUsage(this.usage); + } + return this.write(data); + } + + private write(data: Json): string { + const json = JSON.stringify(data); + if (this.sse) return `data: ${json}\r\n\r\n`; + return `${this.sent++ === 0 ? "[" : ",\r\n"}${json}`; + } + + private close(): string { + this.finished = true; + return this.sse ? "" : this.sent === 0 ? "[]" : "]"; + } + + event(event: Json): string { + if (this.finished) return ""; + const index = typeof event.index === "number" ? event.index : 0; + const delta = isObject(event.delta) ? event.delta : {}; + switch (event.type) { + case "message_start": { + const message = isObject(event.message) ? event.message : {}; + this.id = text(message.id); + mergeUsage(this.usage, message.usage); + return ""; + } + case "content_block_start": { + const block = isObject(event.content_block) ? event.content_block : {}; + if (block.type === "tool_use") this.tools.set(index, { id: text(block.id), name: text(block.name), args: "" }); + return ""; + } + case "content_block_delta": + if (delta.type === "text_delta" && text(delta.text) !== "") return this.chunk([{ text: text(delta.text) }]); + if (delta.type === "thinking_delta" && text(delta.thinking) !== "") return this.chunk([{ text: text(delta.thinking), thought: true }]); + if (delta.type === "input_json_delta") { + const tool = this.tools.get(index); + if (tool !== undefined) tool.args += text(delta.partial_json); + } + return ""; + case "content_block_stop": { + const tool = this.tools.get(index); + if (tool === undefined) return ""; + this.tools.delete(index); + let args: unknown; + try { + args = JSON.parse(tool.args || "{}"); + } catch { + args = {}; + } + return this.chunk([functionCall(tool.name, isObject(args) ? args : {}, tool.id, this.toolNames)]); + } + case "message_delta": + mergeUsage(this.usage, event.usage); + if (typeof delta.stop_reason !== "string") return ""; + return this.chunk([], finishReason(delta.stop_reason)); + case "message_stop": + return this.close(); + case "error": { + const error = isObject(event.error) ? event.error : {}; + return this.write({ error: { code: 500, message: text(error.message) || "The response failed upstream.", status: "INTERNAL" } }) + this.close(); + } + default: + return ""; + } + } + + end(): string { + if (this.finished) return ""; + return this.write({ error: { code: 502, message: "The upstream stream ended early.", status: "UNAVAILABLE" } }) + this.close(); + } +} + +// A whole Anthropic message as one generateContent response. +export function geminiResponse(message: Json, model: string, toolNames: Map): Json { + const parts: Json[] = []; + for (const block of Array.isArray(message.content) ? message.content.filter(isObject) : []) { + if (block.type === "text" && text(block.text) !== "") parts.push({ text: text(block.text) }); + else if (block.type === "thinking" && text(block.thinking) !== "") parts.push({ text: text(block.thinking), thought: true }); + else if (block.type === "tool_use") parts.push(functionCall(text(block.name), isObject(block.input) ? block.input : {}, text(block.id), toolNames)); + } + const usage: Usage = { input: 0, output: 0, cacheWrite: 0, cacheRead: 0, thinking: null }; + mergeUsage(usage, message.usage); + return { + candidates: [{ content: { role: "model", parts }, finishReason: finishReason(text(message.stop_reason)), index: 0 }], + usageMetadata: geminiUsage(usage), + modelVersion: model, + responseId: text(message.id), + }; +} diff --git a/free/src/proxy.ts b/free/src/proxy.ts index 3db71e4..ce12bcd 100644 --- a/free/src/proxy.ts +++ b/free/src/proxy.ts @@ -210,6 +210,7 @@ export function formatError(format: Format | "models", status: number, type: str function geminiStatus(status: number): string { if (status === 400) return "INVALID_ARGUMENT"; if (status === 401) return "UNAUTHENTICATED"; + if (status === 403) return "PERMISSION_DENIED"; if (status === 404) return "NOT_FOUND"; if (status === 405) return "METHOD_NOT_ALLOWED"; if (status === 429) return "RESOURCE_EXHAUSTED"; diff --git a/free/test/app.test.ts b/free/test/app.test.ts index 9c7c0cc..fc7a613 100644 --- a/free/test/app.test.ts +++ b/free/test/app.test.ts @@ -333,8 +333,6 @@ describe("API-key providers", () => { const unknown = await handle(post("/v1/chat/completions", { model: "nothing" }), config); expect(unknown.status).toBe(400); expect(await unknown.json()).toMatchObject({ error: { type: "invalid_request_error", message: expect.stringMatching(/No provider is configured for model nothing/) } }); - const wrongShape = await handle(at(`/v1beta/models/kimi:generateContent?key=${KEY}`, { method: "POST", body: "{}" }), config); - expect(await wrongShape.json()).toMatchObject({ error: { message: expect.stringMatching(/openai-compatibility provider, which doesn.t take this request format yet/) } }); backend.table = { ...backend.table, claudeOAuth: false }; config.cache.forgetConfig(); expect((await handle(post("/v1/messages", { model: "x" }), config)).status).toBe(400); @@ -462,6 +460,10 @@ describe("Vertex", () => { expect(seen[1]?.url).toBe(`${ENDPOINT}/gemini-2.5-pro:streamGenerateContent?alt=sse`); expect(seen[1]?.headers.get("authorization")).toBe("Bearer ya29.vertex"); expect(await seen[1]?.json()).toMatchObject({ contents: [{ role: "user" }] }); + // OpenAI clients too, through Anthropic Messages. + await handle(post("/v1/chat/completions", { model: "gemini-2.5-pro", messages: [{ role: "user", content: "hi" }] }), config); + expect(seen[2]?.url).toBe(`${ENDPOINT}/gemini-2.5-pro:generateContent`); + expect(seen[2]?.headers.get("authorization")).toBe("Bearer ya29.vertex"); expect(backend.calls.filter((call) => call === "loginToken:vertex")).toHaveLength(1); await new Promise((resolve) => setTimeout(resolve, 0)); }); @@ -504,6 +506,11 @@ describe("Kimi logins", () => { // Claude models still go to the Claude login. await handle(post("/v1/messages", { model: "claude-sonnet-5-5", messages: [] }), config); expect(seen[3]?.url).toBe("https://api.anthropic.com/v1/messages"); + // Gemini clients reach Kimi through Anthropic Messages. + await handle(at(`/v1beta/models/kimi-k3:generateContent?key=${KEY}`, { method: "POST", body: JSON.stringify({ contents: [{ role: "user", parts: [{ text: "hi" }] }] }) }), config); + expect(seen[4]?.url).toBe("https://api.kimi.com/coding/v1/messages"); + expect(seen[4]?.headers.get("authorization")).toBe("Bearer kimi-at"); + expect(await seen[4]?.json()).toMatchObject({ model: "k3", messages: [{ role: "user", content: [{ type: "text", text: "hi" }] }] }); expect(backend.calls.filter((call) => call === "loginToken:kimi")).toHaveLength(1); }); @@ -1154,3 +1161,58 @@ describe("OpenAI Responses clients on other providers", () => { expect(await response.json()).toEqual({ error: { message: "Overloaded", type: "api_error", code: null } }); }); }); + +describe("Gemini clients on other providers", () => { + const claudeAnswer = () => + new Response( + [ + { type: "message_start", message: { id: "msg_1", usage: { input_tokens: 4 } } }, + { type: "content_block_start", index: 0, content_block: { type: "text", text: "" } }, + { type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "Hi" } }, + { type: "content_block_stop", index: 0 }, + { type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 1 } }, + { type: "message_stop" }, + ] + .map((event) => `event: x\ndata: ${JSON.stringify(event)}\n\n`) + .join(""), + { headers: { "content-type": "text/event-stream" } }, + ); + const body = JSON.stringify({ contents: [{ role: "user", parts: [{ text: "hi" }] }] }); + + it("serves a Gemini client from a Claude API key: SSE, a JSON array, whole, and token counts", async () => { + const { config, seen } = await providersSetup(async (request) => + ((await request.clone().json()) as { stream?: boolean }).stream ? claudeAnswer() : Response.json({ id: "msg_2", content: [{ type: "text", text: "Hello" }], stop_reason: "end_turn", usage: { input_tokens: 4, output_tokens: 1 } }), + ); + const model = encodeURIComponent("work/claude-sonnet-5-5"); + const streamed = await handle(at(`/v1beta/models/${model}:streamGenerateContent?alt=sse&key=${KEY}`, { method: "POST", body }), config); + expect(streamed.headers.get("content-type")).toMatch(/^text\/event-stream/); + const text = await streamed.text(); + expect(text).toContain('data: {"candidates":[{"content":{"role":"model","parts":[{"text":"Hi"}]}'); + expect(text).toContain('"finishReason":"STOP"'); + const [sent] = seen; + expect(sent?.url).toBe("https://api.anthropic.com/v1/messages"); + expect(sent?.headers.get("x-api-key")).toBe("sk-ant-1"); + expect(sent?.headers.get("x-goog-api-key")).toBeNull(); + expect(await sent?.json()).toEqual({ model: "claude-sonnet-5-5", max_tokens: 32000, stream: true, messages: [{ role: "user", content: [{ type: "text", text: "hi" }] }] }); + const array = await handle(at(`/v1beta/models/${model}:streamGenerateContent?key=${KEY}`, { method: "POST", body }), config); + expect(array.headers.get("content-type")).toMatch(/^application\/json/); + expect(await array.json()).toHaveLength(2); + const whole = await handle(at(`/v1beta/models/${model}:generateContent?key=${KEY}`, { method: "POST", body }), config); + expect(await whole.json()).toMatchObject({ candidates: [{ content: { parts: [{ text: "Hello" }] }, finishReason: "STOP" }], modelVersion: "work/claude-sonnet-5-5" }); + const count = await handle(at(`/v1beta/models/${model}:countTokens?key=${KEY}`, { method: "POST", body }), config); + expect(await count.json()).toEqual({ totalTokens: Math.ceil(body.length / 4) }); + expect(seen).toHaveLength(3); + }); + + it("serves a Gemini client from an OpenAI-compatible key, and errors in Gemini's shape", async () => { + const chat = () => Response.json({ id: "chatcmpl-1", choices: [{ message: { content: "Hi" }, finish_reason: "stop" }] }); + const { config, seen } = await providersSetup(async () => chat()); + const response = await handle(at(`/v1beta/models/kimi:generateContent?key=${KEY}`, { method: "POST", body }), config); + expect(await response.json()).toMatchObject({ candidates: [{ content: { parts: [{ text: "Hi" }] } }] }); + expect(seen[0]?.url).toBe("https://router.example/api/v1/chat/completions"); + const failing = await providersSetup(async () => Response.json({ error: { message: "Bad key" } }, { status: 403 })); + const refused = await handle(at(`/v1beta/models/kimi:generateContent?key=${KEY}`, { method: "POST", body }), failing.config); + expect(refused.status).toBe(403); + expect(await refused.json()).toEqual({ error: { code: 403, message: expect.any(String), status: "PERMISSION_DENIED" } }); + }); +}); diff --git a/free/test/gemini-claude.test.ts b/free/test/gemini-claude.test.ts new file mode 100644 index 0000000..1d58ca9 --- /dev/null +++ b/free/test/gemini-claude.test.ts @@ -0,0 +1,146 @@ +import { describe, expect, it } from "vitest"; +import { geminiResponse, GeminiStream, geminiToClaude } from "../src/gemini-claude.ts"; + +const INTERRUPTED = "Tool call was interrupted before any output was recorded."; + +describe("Gemini to Claude", () => { + it("translates contents, calls and results, media, tools and settings as CPA does", () => { + const { body, stream, toolNames } = geminiToClaude( + { + systemInstruction: { parts: [{ text: "Be brief." }] }, + contents: [ + { role: "user", parts: [{ text: "hi" }, { inlineData: { mimeType: "image/png", data: "iV" } }, { inline_data: { mime_type: "application/pdf", data: "JVBER" } }, { fileData: { mimeType: "image/jpeg", fileUri: "https://example.com/a.jpg" } }] }, + { role: "model", parts: [{ text: "plan", thought: true, thoughtSignature: "s" }, { text: "Running." }, { functionCall: { name: "fs.read", args: { path: "a" } } }, { functionCall: { name: "fs.read", args: { path: "b" } } }, { functionCall: { name: "grep", args: {}, id: "call.9" } }] }, + { role: "user", parts: [{ functionResponse: { name: "fs.read", response: { output: "A" } } }, { functionResponse: { name: "fs.read", response: { lines: 2 } } }, { functionResponse: { name: "nobody", response: "x" } }] }, + { role: "user", parts: [{ text: "next" }] }, + ], + tools: [{ functionDeclarations: [{ name: "fs.read", description: "Read", parameters: { type: "OBJECT", properties: { path: { type: "STRING" } } } }, { name: "grep", parametersJsonSchema: { type: "object" } }] }, { googleSearch: {} }], + toolConfig: { functionCallingConfig: { mode: "ANY", allowedFunctionNames: ["grep"] } }, + generationConfig: { maxOutputTokens: 4000, topP: 0.9, stopSequences: ["END"], thinkingConfig: { thinkingBudget: 2048 }, responseMimeType: "application/json" }, + }, + "claude-sonnet-4-5", + true, + ); + expect(stream).toBe(true); + expect(toolNames.get("fs_read")).toBe("fs.read"); + expect(body).toEqual({ + model: "claude-sonnet-4-5", + max_tokens: 4000, + stream: true, + top_p: 0.9, + stop_sequences: ["END"], + thinking: { type: "enabled", budget_tokens: 3999 }, + system: [ + { type: "text", text: "Be brief." }, + { type: "text", text: expect.stringContaining("valid JSON object") }, + ], + messages: [ + { + role: "user", + content: [ + { type: "text", text: "hi" }, + { type: "image", source: { type: "base64", media_type: "image/png", data: "iV" } }, + { type: "document", source: { type: "base64", media_type: "application/pdf", data: "JVBER" } }, + { type: "image", source: { type: "url", url: "https://example.com/a.jpg" } }, + ], + }, + { + role: "assistant", + content: [ + { type: "text", text: "Running." }, + { type: "tool_use", id: "toolu_gemini_0", name: "fs_read", input: { path: "a" } }, + { type: "tool_use", id: "toolu_gemini_1", name: "fs_read", input: { path: "b" } }, + { type: "tool_use", id: "call_9", name: "grep", input: {} }, + ], + }, + { + role: "user", + content: [ + { type: "tool_result", tool_use_id: "call_9", content: INTERRUPTED, is_error: true }, + { type: "tool_result", tool_use_id: "toolu_gemini_0", content: "A" }, + { type: "tool_result", tool_use_id: "toolu_gemini_1", content: '{"lines":2}' }, + { type: "text", text: "Tool result nobody: x" }, + { type: "text", text: "next" }, + ], + }, + ], + tools: [ + { name: "fs_read", description: "Read", input_schema: { type: "object", properties: { path: { type: "string" } } } }, + { name: "grep", description: "", input_schema: { type: "object", properties: {} } }, + ], + tool_choice: { type: "tool", name: "grep" }, + }); + }); + + it("maps thinking levels on adaptive models, and fills in defaults", () => { + expect(geminiToClaude({ contents: [], generationConfig: { thinkingConfig: { thinkingLevel: "HIGH" } } }, "claude-opus-5-5", false).body).toEqual({ + model: "claude-opus-5-5", + max_tokens: 32000, + stream: false, + messages: [{ role: "user", content: [{ type: "text", text: "" }] }], + thinking: { type: "adaptive" }, + output_config: { effort: "high" }, + }); + expect(geminiToClaude({ contents: [], generationConfig: { thinkingConfig: { thinkingBudget: 0 } }, toolConfig: { functionCallingConfig: { mode: "NONE" } } }, "claude-fable-5-1", false).body).toMatchObject({ + max_tokens: 64000, + thinking: { type: "disabled" }, + tool_choice: { type: "none" }, + }); + // A file reference that isn't https is dropped. + expect(geminiToClaude({ contents: [{ role: "user", parts: [{ fileData: { fileUri: "gs://bucket/a" } }, { text: "hi" }] }] }, "claude-opus-5-5", false).body.messages).toEqual([{ role: "user", content: [{ type: "text", text: "hi" }] }]); + }); +}); + +const EVENTS = [ + { type: "message_start", message: { id: "msg_1", usage: { input_tokens: 10, cache_read_input_tokens: 4 } } }, + { type: "content_block_start", index: 0, content_block: { type: "thinking", thinking: "" } }, + { type: "content_block_delta", index: 0, delta: { type: "thinking_delta", thinking: "Hmm" } }, + { type: "content_block_stop", index: 0 }, + { type: "content_block_start", index: 1, content_block: { type: "text", text: "" } }, + { type: "content_block_delta", index: 1, delta: { type: "text_delta", text: "Hi" } }, + { type: "content_block_stop", index: 1 }, + { type: "content_block_start", index: 2, content_block: { type: "tool_use", id: "toolu_1", name: "fs_read", input: {} } }, + { type: "content_block_delta", index: 2, delta: { type: "input_json_delta", partial_json: '{"path":"a"}' } }, + { type: "content_block_stop", index: 2 }, + { type: "message_delta", delta: { stop_reason: "tool_use" }, usage: { output_tokens: 5 } }, + { type: "message_stop" }, +]; + +describe("Claude events to Gemini", () => { + it("streams SSE chunks with thoughts, text, calls and a final usage chunk", () => { + const stream = new GeminiStream("claude-sonnet-5-5", new Map([["fs_read", "fs.read"]]), true); + const out = EVENTS.map((event) => stream.event(event)).join("") + stream.end(); + const chunks = out + .split("\r\n\r\n") + .filter((chunk) => chunk !== "") + .map((chunk) => JSON.parse(chunk.slice(6)) as { candidates: { content: { parts: unknown[] }; finishReason?: string }[]; usageMetadata?: unknown }); + expect(chunks.map((chunk) => chunk.candidates[0]?.content.parts)).toEqual([[{ text: "Hmm", thought: true }], [{ text: "Hi" }], [{ functionCall: { name: "fs.read", args: { path: "a" }, id: "toolu_1" } }], []]); + expect(chunks.at(-1)).toMatchObject({ candidates: [{ finishReason: "STOP" }], usageMetadata: { promptTokenCount: 14, candidatesTokenCount: 5, totalTokenCount: 19, cachedContentTokenCount: 4 }, responseId: "msg_1" }); + }); + + it("writes one JSON array without ?alt=sse, and closes it on a cut-off stream", () => { + const stream = new GeminiStream("m", new Map(), false); + const out = EVENTS.map((event) => stream.event(event)).join("") + stream.end(); + expect((JSON.parse(out) as unknown[]).length).toBe(4); + const cut = new GeminiStream("m", new Map(), false); + const partial = cut.event(EVENTS[0] ?? {}) + cut.event(EVENTS[4] ?? {}) + cut.event(EVENTS[5] ?? {}) + cut.end(); + expect(JSON.parse(partial)).toEqual([expect.objectContaining({ candidates: expect.any(Array) }), { error: { code: 502, message: "The upstream stream ended early.", status: "UNAVAILABLE" } }]); + const empty = new GeminiStream("m", new Map(), false); + expect(JSON.parse(empty.event({ type: "message_stop" }) || "null")).toEqual([]); + }); + + it("converts a whole message", () => { + expect( + geminiResponse( + { id: "msg_2", content: [{ type: "text", text: "Hi" }, { type: "tool_use", id: "toolu_1", name: "fs_read", input: { path: "a" } }], stop_reason: "max_tokens", usage: { input_tokens: 3, output_tokens: 2 } }, + "claude-sonnet-5-5", + new Map([["fs_read", "fs.read"]]), + ), + ).toEqual({ + candidates: [{ content: { role: "model", parts: [{ text: "Hi" }, { functionCall: { name: "fs.read", args: { path: "a" }, id: "toolu_1" } }] }, finishReason: "MAX_TOKENS", index: 0 }], + usageMetadata: { promptTokenCount: 3, candidatesTokenCount: 2, totalTokenCount: 5 }, + modelVersion: "claude-sonnet-5-5", + responseId: "msg_2", + }); + }); +});