Turn a spare Android phone into an always-on AI appliance — your own relay station for local inference.
Relais runs an LLM directly on the phone's GPU with Google's
LiteRT-LM runtime and serves it
over your LAN as a standard OpenAI-compatible API — /v1/chat/completions,
streaming SSE, API-key auth, mDNS discovery. Point any OpenAI client at the
phone's IP and it answers. Ollama, llama.cpp-server, and LM Studio all need a
desktop and can't reach a phone's neural silicon; Relais is headless by
design — no chat UI to babysit. It auto-starts, survives Doze, draws a few
watts, and sits on a shelf as private, cloud-free inference infrastructure you
operate rather than rent.
Spare flagship in, OpenAI endpoint out.
Relais is a fork of Google's AI Edge Gallery, rebuilt as a network service rather than an app. Licensed under AGPL-3.0 (see LICENSE and NOTICE).
- Is: a headless LAN inference node — one resident model on the phone GPU,
exposed as an OpenAI-compatible HTTP API and hardened for always-on operation:
foreground service, wake lock, Doze survival, crash/OOM auto-recovery,
thermal backpressure,
/metrics. - Isn't: the fastest way to run a model. A desktop GPU crushes a phone on tokens/sec. Relais's value is always-on, low-power, private, zero-marginal-cost — small-model and agent/automation workloads, not 70B frontier serving.
Prior art, honestly: OlliteRT does the same thing on the same runtime. Relais's bet is execution — Doze reliability, Pixel/Tensor tuning, a real security posture (TLS on the LAN, encrypted key at rest), and the control/observability surface.
- A device LiteRT-LM supports on GPU. Developed and validated on a Pixel 9 Pro
Fold (Tensor G4), Android 12+ (
minSdk 31). - To build: JDK 17+ and Android Studio with SDK 35. No NDK required — the inference engine ships as a prebuilt AAR.
From fresh clone to serving node:
# 1. Clone
git clone https://github.com/entrevoix/relais.git
cd relais/Android/src
# 2. Point the build at your SDK (or set sdk.dir in Android Studio)
echo "sdk.dir=$ANDROID_HOME" > local.properties
# 3. Build + install the debug APK on a connected device
./gradlew :app:assembleDebug
adb install -r app/build/outputs/apk/debug/app-debug.apkThen on the phone:
- Open the Relais Node launcher icon.
- Tap START. On first run it downloads the default open model
(
litert-community/gemma-4-E4B-it-litert-lm, ~3.7 GB — no HuggingFace token needed) and initializes the resident engine. - The panel shows status (
LIVE), the LAN endpoint (https://<phone-ip>:8443), and your access key (tap to copy). The HTTPS node listens on both IPv4 and IPv6; use brackets around an IPv6 literal, for examplehttps://[2001:db8::1]:8443.
Reach it from any machine on the LAN:
curl -k https://<phone-ip>:8443/v1/chat/completions \
-H "Authorization: Bearer <access-key>" \
-H "Content-Type: application/json" \
-d '{"model":"gemma-4-e4b-it","messages":[{"role":"user","content":"reply one word: ping"}]}'The LAN endpoint is HTTPS with a self-signed cert (hence
-kfor now). The plaintexthttp://127.0.0.1:8080endpoint is loopback-only by design — see SECURITY.md. On untrusted networks, put the node behind a WireGuard/Tailscale overlay.
IPv6 link-local addresses are intentionally not advertised: they require an interface scope which cannot be represented in a certificate SAN. Every IPv4 or IPv6 address shown in the node's certificate is served on HTTPS
:8443.
If DHCP finishes after the node starts, it observes the new LAN address set and refreshes its certificate and HTTPS listeners after that set has remained stable for 15 seconds. The imported CA and the node's SPKI pin stay the same; see SECURITY.md for verification notes.
Discovery: the node advertises _relais._tcp over mDNS, so clients can find it
by name even after its IP changes.
The control panel's MODEL row opens a selector with three ways to pick what the node serves:
- Curated — the node-runnable LiteRT-LM chat models from the built-in allowlist, with sizes. Tap one to select it.
- HuggingFace search — type to search the Hub, or paste a full
org/repoid to resolve it directly. Relais resolves the repo's.litertlmfile, commit, and size and provisions it on the next start — any compatible repo works, not just the curated list. Repos with no.litertlmfile are flagged as incompatible. - Manual id — enter an allowlist repo id by hand (resolved via the allowlist on start).
Selection takes effect on the next START (the panel shows "Restart to
apply"). Gated repos (e.g. official google/gemma-*) are marked token —
for HuggingFace search results this reflects the Hub's real gated flag. Paste
a HuggingFace access token into HF TOKEN and SAVE HF TOKEN first, or the
resolve/download will fail with a gated error. Open repos like the default need
no token.
Any OpenAI-compatible client — point its base URL at https://<phone-ip>:8443/v1
and set the API key:
- Open WebUI — add an OpenAI connection.
- Editors / agents — anything that speaks the OpenAI API.
- curl / scripts — see above.
These are "works with," not dependencies. Relais never calls the cloud and is not coupled to any gateway.
| Method | Path | Auth | Notes |
|---|---|---|---|
| GET | /health |
none | {status, ready, thermal_state, state} — state is one of OFF/STARTING/LIVE/HOT/ERROR/IDLE; IDLE = engine released by idle-TTL, warms on the next request |
| GET | /metrics |
key | Prometheus text (or JSON via Accept: application/json) |
| POST | /generate |
key | {text, image_b64?, audio_b64?} — native multimodal |
| POST | /v1/chat/completions |
key | OpenAI-compatible, stream supported |
- Auto-start on boot: opt-in in the control panel (off by default).
- Idle unload: the resident engine releases itself after an idle period (default 15 min,
operator-configurable in CONFIGURE, or off entirely) to free memory;
/healththen reportsstate: IDLE— a healthy node, not a dead one — and warms again on the next request. - Unattended / force-stop resilience: the control panel is excluded from the recents switcher, so a "clear all recent apps" sweep won't force-stop the node. For Doze survival, the panel shows a POWER readout — tap ALLOW UNRESTRICTED to exempt the node from battery optimization. One caveat: a deliberate Settings → Force stop (or a cleaner that force-stops every package) can't be auto-recovered — Android disables a force-stopped app's alarms and boot-receiver until it's launched again — so relaunch the node (or reboot with auto-start on) to recover.
- Observability: scrape
https://<phone-ip>:8443/metricswith Prometheus (bearer-token auth). Tokens/sec, latency histogram, thermal state, RSS, restart count, queue depth. - Thermal: under sustained load the node sheds with
503 + Retry-Afterrather than melting; seerelais_thermal_status/relais_shed_total. - Endurance: see docs/soak for the soak-test harness.
See SECURITY.md. Short version: trusted-LAN-safe by default — HTTPS on the LAN, encrypted key at rest, rate limiting. For untrusted networks, use an overlay or cert pinning.
AGPL-3.0. Relais is a network service, and AGPL's network-use clause keeps modified, network-served versions open. Files derived from Google AI Edge Gallery retain their Apache-2.0 headers; the work as a whole is AGPL-3.0. See LICENSE and NOTICE.