Skip to content

About

GemRouter is an OpenAI-compatible backend router for Gemini API, designed to maximize free-tier through multi-key routing, local quota tracking, fallback, and error handling. It also exposes DeepSeek and Ollama-compatible surfaces, with local Ollama routes for embeddings and vision plus an operator admin dashboard for live account, quota, telemetry

Topics

Resources

Stars

4 stars

Watchers

0 watching

Forks

Repository files navigation

GemRouter

GemRouter is a self-hosted HTTP gateway for applications that share access to AI providers. Each application gets its own API key and model permissions. The gateway chooses an eligible provider account, tracks local quota usage and can move a text request to another permitted model when the first one is busy. Operators manage accounts, app keys, routing and usage from the built-in dashboard.

An OpenAI-compatible client can use /v1/chat/completions or /v1/responses. The root chat endpoints and the Ollama-compatible routes serve other clients. These interfaces implement a subset of the upstream APIs. For example, use plain text for Gemini chat, use a dedicated endpoint for vision, and expect Gemini and Codex chat streaming to deliver buffered text. Client API and examples

Run locally

Use Node.js 24.18.0 and pnpm 10.26.1, the versions specified in .nvmrc and package.json. From a fresh checkout, install dependencies and create a private configuration file.

pnpm install --frozen-lockfile
touch .env && chmod 600 .env

For a first Gemini text request, set these entries in .env. Generate two different random local secrets and replace the Google key placeholder with a credential for your own project. The default data directory is ./data and the default port is 4024.

HOST=127.0.0.1
GEMROUTER_ADMIN_TOKEN=<random-operator-secret>
GEMROUTER_BOOTSTRAP_API_KEY=<different-random-client-secret>
GEMROUTER_GEMINI_API_KEYS=<google-project-api-key>
GEMROUTER_GEMINI_API_ACCOUNTS_PATH=data/gemini-api-accounts.json
pnpm build
pnpm start

In a second shell, set GEMROUTER_APP_KEY to the bootstrap client key or an app key created in the dashboard. The model list shows what that key can discover. Omitting model in the text example exercises the default selection. Include model when your application needs a particular starting model.

curl -fsS http://127.0.0.1:4024/health
curl -fsS http://127.0.0.1:4024/v1/models \
  -H "Authorization: Bearer $GEMROUTER_APP_KEY"

curl -sS -i http://127.0.0.1:4024/v1/chat/completions \
  -H "Authorization: Bearer $GEMROUTER_APP_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Reply with OK."}],"max_tokens":128}'

Check the HTTP status and x-gemrouter-backend-model to see which model served the reply. For clients that take an OpenAI base URL, use http://127.0.0.1:4024/v1. The operator dashboard is at /admin. Installation and initial configuration

Models and routing

Workload Entry point Routing behavior
Text chat POST /v1/chat/completions, POST /v1/responses The built-in default is gemini-4-hydrogen. With both models enabled and permitted, busy Hydrogen can fall back to gemini-3.8-flash. Requests for a particular model start with that model when eligible.
Classification and scoring POST /v1beta/decisions Requires the gemini-decision-1 grant. Returns typed answers and usage. Accepts text and inline media.
Optional text providers Chat endpoints NVIDIA NIM, configured Ollama servers and Codex each have their own routing rules. Codex requires provider enablement, a signed-in account and an app grant.
Images, video, vision, embeddings Dedicated endpoints Availability depends on the configured provider. Vision and embeddings use local Ollama. Agnes can serve image and video generation.

Gemini admission is tracked per quota group and model. Put keys from the same Google project in the same group. A catalog entry confirms that the provider lists a model. A successful inference also verifies that the project could use it at the time of the test. For billing and current project limits, consult the provider. Local counters and dashboard token estimates serve operational monitoring. Routing and quota · Google rate limits

Documentation

Task Guide
Integrate an application, interpret fallback headers or handle errors Client API
Set up a new installation or choose configuration values Setup · Configuration
Understand model selection, account discovery and quota accounting Routing · Architecture and workflow, in Italian
Deploy the service, operate the dashboard or recover configuration Operations
Connect a Codex account or manage its runtime Codex provider · Runtime updates
Check which Gemini models individual accounts can actually use Account census

Development

pnpm check
pnpm test
pnpm build

The test suite includes local HTTP fixtures for authentication, routing, quota accounting and fallback. pnpm smoke targets a running service and can send real inference requests. Choose the target and credentials before running it. Operations

The repository's non-commercial license covers eligible original material. Third-party components retain their respective terms. Licensing scope · License · Third-party notices

About

GemRouter is an OpenAI-compatible backend router for Gemini API, designed to maximize free-tier through multi-key routing, local quota tracking, fallback, and error handling. It also exposes DeepSeek and Ollama-compatible surfaces, with local Ollama routes for embeddings and vision plus an operator admin dashboard for live account, quota, telemetry

Topics

Resources

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages