Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
ab5f3ba
feat: 建立 LiteLLM P1 运行基线
Aug 8, 2026
fe08b87
fix: 修复 LiteLLM Prisma 运行时缓存
Aug 8, 2026
85b81a3
fix: 强化 LiteLLM smoke 密钥与撤销验收
Aug 8, 2026
eacd309
fix: 完善 LiteLLM HA 撤销与 DeepSeek smoke
Aug 8, 2026
f45b760
fix: 缩短 LiteLLM Redis 恢复窗口
Aug 8, 2026
cf441ba
fix: 加固 LiteLLM P1 日志与 Redis 密码安全
Aug 8, 2026
762ba1e
fix: 分离 LiteLLM 迁移与副本启动
Aug 8, 2026
33a58c6
test: 输出 LiteLLM P1 脱敏验证摘要
Aug 8, 2026
24a7a5f
test: 验证 LiteLLM 双副本共享限流
Aug 8, 2026
1b8f50f
test: 验证 LiteLLM Redis 中断恢复
Aug 8, 2026
5a41397
test: 验证 LiteLLM 多副本模型与用量一致性
Aug 8, 2026
d07b467
test: 恢复 Redis 网络中断清理保障
Aug 8, 2026
0cd389b
test: 记录 LiteLLM smoke 失败阶段
Aug 8, 2026
fc0e3dc
test: 拆分 Redis 恢复验收脚本
Aug 8, 2026
dffb3b2
docs: 汇总 LiteLLM P1 验证证据
Aug 9, 2026
a62d095
fix: 修复 LiteLLM P1 可复现验收报告
Aug 9, 2026
e64eefc
fix: 恢复 LiteLLM 验收脚本可执行权限
Aug 9, 2026
f0bc8ba
fix: 规范化 LiteLLM 上游 provider
Aug 9, 2026
6a720cd
fix: 兼容 DeepSeek 本地 provider 标识
Aug 9, 2026
ff23998
fix: 扩展 DeepSeek P1 provider 映射
Aug 9, 2026
26cc536
fix: 规范化 LiteLLM provider 输入
Aug 9, 2026
0a588e6
fix: 让 LiteLLM 迁移失败时终止
Aug 9, 2026
8c109c2
fix: 归一化 DeepSeek provider 变体
Aug 9, 2026
09bc3c8
fix: 支持 LiteLLM smoke provider 覆盖
Aug 9, 2026
8973251
fix: 使用 Compose 有效环境执行 LiteLLM smoke
Aug 9, 2026
c224b36
fix: 从 Compose 有效环境读取 smoke 端口
Aug 9, 2026
34d0202
fix: 安全恢复 LiteLLM 虚拟密钥创建响应
Aug 9, 2026
9b2c166
fix: 适配 Redis 恢复验证的 Compose 端口
Aug 9, 2026
6c0f0ba
fix: 强化 LiteLLM 内容日志验收
Aug 9, 2026
a9500b6
fix: 扫描 P1 SpendLog 数据库正文
Aug 9, 2026
2dfb89e
fix: 正确验证 LiteLLM 跨副本预算限制
Aug 9, 2026
e7eac4e
fix: 验证 LiteLLM 跨副本 TPM 门限
Aug 9, 2026
84108ba
fix: 强化 LiteLLM P1 验证门禁
Aug 9, 2026
00c7899
fix: 强制 P1 清理与报告失败语义
Aug 9, 2026
676e5ad
fix: 按验证运行隔离 LiteLLM 测试资源
Aug 9, 2026
02d7f23
fix: 原子写入 LiteLLM smoke 报告
Aug 9, 2026
c634188
fix: 防止 P1 验证退出 trap 重入
Aug 9, 2026
3893f55
fix: 加固 P1 验收报告与清理门禁
Aug 9, 2026
497ab1d
test: 复用 P1 负向验证运行标识
Aug 9, 2026
7b12b94
fix: 保留 smoke 失败报告并隔离资源名
Aug 9, 2026
6b7f186
fix: 强制 cleanup 失败返回非零
Aug 9, 2026
5f6ee45
fix: 以 LiteLLM limiter 证据验证共享限流
Aug 9, 2026
29848d5
fix: 保护 cleanup 失败报告写入
Aug 9, 2026
50e39ec
fix: 原子创建 smoke 报告临时文件
Aug 9, 2026
16d4557
fix: 保留 cleanup 失败的原子报告
Aug 9, 2026
efaf6e5
fix: 同步 cleanup 失败报告落盘
Aug 9, 2026
893f1bc
fix: 生成完整 smoke 失败摘要
Aug 9, 2026
f0e0a8f
fix: 强化 P1 运行门禁与并发迁移验证
Aug 9, 2026
6f94364
fix: 绑定 LiteLLM 限流响应头证据
Aug 9, 2026
bb0f645
fix: 完善 P1 报告门禁证据
Aug 9, 2026
cf500f6
fix: 校验 LiteLLM 限流类型响应证据
Aug 9, 2026
3913cf2
docs: 明确 P1 Redis 与报告门禁语义
Aug 9, 2026
f9ec5ea
fix: 强化 P1 总控验收门禁
Aug 9, 2026
0959fae
fix: 隔离 cleanup 负向测试退出
Aug 9, 2026
9403255
test: 覆盖 LiteLLM 限流来源负向判定
Aug 9, 2026
6fdb637
feat: 新增 P6 本地黄金链路编排门禁
Aug 10, 2026
1b45628
fix: 扩展 P6 运行态密钥扫描范围
Aug 10, 2026
9d78780
feat: 完成 P6 OpenClaw 端到端黄金联调
Aug 10, 2026
45c3858
docs: 记录 P6 黄金链验收回执
Aug 10, 2026
bc3875d
feat(hermes): 固定 P7 可复现构建与验证入口
Aug 10, 2026
90328f6
fix(hermes): 实现 P7 参数化黄金链
Aug 10, 2026
fcbcec7
fix(hermes): 修正 P7 报告默认载荷
Aug 10, 2026
74e4464
fix(hermes): 闭合 P7 参数化黄金链预检
Aug 10, 2026
822e36a
fix(hermes): 固定毫秒租约兼容制品
Aug 10, 2026
bd7aba4
fix(hermes): 对齐固定 CLI 黄金调用参数
Aug 10, 2026
48cd79d
test(hermes): 固定 P7 黄金链证据
Aug 10, 2026
323208a
test(hermes): 固定产品仓证据回执
Aug 10, 2026
9f1e88d
fix(hermes): 兼容中文证据路径预检
Aug 10, 2026
6cab6df
fix: 补齐 Hermes Chat TUI Node 运行时
Aug 12, 2026
db6954c
chore: 清理 LLM Hub 阶段证据文档
Aug 16, 2026
fdbbab2
chore: 改用本地忽略项目文档
Aug 16, 2026
82c6df1
fix(litellm): 收紧 Compose 凭据边界
Aug 19, 2026
87182eb
test(litellm): 覆盖 Compose 凭据边界回归
Aug 19, 2026
2dac102
fix(litellm): 脱敏 migration 连接串日志
Aug 19, 2026
aa9d3b6
chore(openclaw): 归档 P6 一次性证据编排
Aug 19, 2026
ca3a2d6
chore(hermes): 归档 P7 一次性证据编排
Aug 20, 2026
6e3a55d
test(openclaw): 修正 P6 Compose 端口门禁匹配
Aug 20, 2026
173b12e
fix(litellm): 参数化 Compose 实例资源命名
Aug 20, 2026
05e4c9e
refactor(litellm): 提取可移植权限检查 helper
Aug 20, 2026
d9f1391
fix(litellm): 原子写入 standalone 配置
Aug 20, 2026
f7a0916
fix(litellm): 禁止 Compose 隐式拉取镜像
Aug 20, 2026
9b15852
Merge branch 'dev/hardening-ph4-compose-script-hygiene' into integrat…
Aug 20, 2026
f2c474a
chore: 测试移出产品分支(本地保留)
Aug 20, 2026
73f58c7
refactor: 服务模块 demo/ 目录统一更名为 compose/
Aug 21, 2026
999e188
chore: 清理开发过程记录与阶段过程标记
Aug 21, 2026
f87a544
chore: 清理 Dockerfile/config 注释中的阶段标记
Aug 21, 2026
aaddd13
chore: AI 协作约定文件移出仓库(本地保留)
Aug 21, 2026
a7c8a64
chore: AGENTS.md 移出仓库(本地保留,补前一提交)
Aug 21, 2026
3ad8c8f
Merge remote-tracking branch 'origin/main' into integration/llm-hub-v1
Aug 23, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .claude/skills

This file was deleted.

8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,14 @@ celerybeat.pid

# Environments
.env
docker_litellm/compose/artifacts/
docker_openclaw/p6/artifacts/
docker_openclaw/p6/p6-inputs.json
docker_openclaw/p6/.p6-work/
docker_hermes/p7/artifacts/
docker_hermes/p7/.p7-work/
docker_hermes/p7/p7-inputs.json
docker_hermes/p7/source/
.venv
env/
venv/
Expand Down
Empty file removed AGENTS.md
Empty file.
1 change: 0 additions & 1 deletion CLAUDE.md

This file was deleted.

33 changes: 29 additions & 4 deletions docker_hermes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

`hermes` is a containerized agentic assistant platform based on the [Hermes Agent](https://github.com/nousresearch/hermes-agent) project, built using Node.js and Python runtime stacks.

Dockerfile 以固定的 Hermes repository 与 40 位 commit 作为制品输入,不以移动的
`main` 构建。默认 standalone Compose 服务于本地开发,只接受本机已存在的明确镜像
引用,不会静默 pull `latest`。

---

## 1. Port Configuration
Expand Down Expand Up @@ -48,21 +52,42 @@ source ./tool.sh
build_image_no_tag hermes local docker_hermes/hermes.Dockerfile
```

### 可复现构建(固定源码提交)

需要可复现构建时,使用固定 tag 与明确的 Hermes source identity:

```bash
build_image_no_tag hermes src-<12hex> docker_hermes/hermes.Dockerfile \
--build-arg HERMES_SOURCE_REPOSITORY=<observed-source-remote> \
--build-arg HERMES_SOURCE_COMMIT=<40-hex-commit>
```

镜像会记录 `org.opencontainers.image.source` 与
`org.opencontainers.image.revision`,并在 `/opt/hermes/.labnow-source-*` 保存
相同的非敏感 provenance。只在本地命名为 `quay.io/labnow/hermes:src-<12hex>`,不 push。

### Dashboard Chat TUI runtime

Hermes 的 Dashboard 在 `/api/pty` 中执行已经构建的
`/opt/hermes/ui-tui/dist/entry.js`。运行基础镜像不是 Node 镜像,因此 Dockerfile 会从
同一目标架构的 builder 复制固定的 `/opt/node` runtime,并将其放入 `PATH`。这避免用户
第一次打开 Chat 时触发 Node 下载/解压;不改变 Hermes source、TUI build 或模型配置。

### Start with Docker Compose

1. Copy the sample environment file:
```bash
cp docker_hermes/.env.example docker_hermes/demo/.env
cp docker_hermes/compose/.env.example docker_hermes/compose/.env
```

2. Specify the built image in `docker_hermes/demo/.env`:
2. Specify the built image in `docker_hermes/compose/.env`:
```env
HERMES_IMAGE=quay.io/labnow/hermes:local
```

3. Launch the container:
```bash
docker compose --env-file docker_hermes/demo/.env -f docker_hermes/demo/docker-compose.yml up -d
docker compose --env-file docker_hermes/compose/.env -f docker_hermes/compose/docker-compose.yml up -d
```

### Execution Modes
Expand All @@ -89,7 +114,7 @@ python -c "from plugins.dashboard_auth.basic import hash_password; print(hash_pa

### Model Provider Setup

Hermes requires an LLM inference provider. Configure credentials in `docker_hermes/demo/.env`:
Hermes requires an LLM inference provider. Configure credentials in `docker_hermes/compose/.env`:

```env
OPENAI_API_KEY=your-key
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# Hermes local runtime configuration.
#
# Copy this file to docker_hermes/demo/.env or pass it with:
# docker compose --env-file docker_hermes/demo/.env.example -f docker_hermes/demo/docker-compose.yml up -d
# Copy this file to docker_hermes/compose/.env or pass it with:
# docker compose --env-file docker_hermes/compose/.env.example -f docker_hermes/compose/docker-compose.yml up -d

TZ=Asia/Shanghai

# Image to run. Build with REGISTRY_DST=quay.io via tool.sh before starting Compose.
# Default local output: quay.io/labnow/hermes:local (no automatic push).
HERMES_IMAGE=quay.io/labnow/hermes
HERMES_IMAGE=quay.io/labnow/hermes:local

# Host-side persistent data directory.
HERMES_DATA_DIR=../../.data/hermes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@ services:
hermes:
container_name: ${HERMES_CONTAINER_NAME:-svc-hermes}
hostname: svc-hermes
image: "${HERMES_IMAGE:-quay.io/labnow/hermes:latest}"
pull_policy: always
image: "${HERMES_IMAGE:?set a fixed local Hermes image}"
# Local development must not silently pull a mutable image. Use an
# explicit Compose action when an operator intentionally needs a pull.
pull_policy: never
restart: unless-stopped
env_file: [".env.example"]
environment:
Expand Down
49 changes: 45 additions & 4 deletions docker_hermes/hermes.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,19 @@
ARG BASE_NAMESPACE
ARG BASE_IMG_BUILD="node"
ARG BASE_IMG="base"
ARG HERMES_BUILD_BASE_IMAGE
ARG HERMES_RUNTIME_BASE_IMAGE
# The upstream source is a release input, not a moving branch. Keep the
# repository and commit overridable only so the local runner can bind both to
# its protected input and record the exact provenance.
ARG HERMES_SOURCE_REPOSITORY="https://github.com/nousresearch/hermes-agent.git"
ARG HERMES_SOURCE_COMMIT="1388cd1c0c1800078bfcc92aebd144fbf145fdb4"

# --- Building Stage ---
FROM ${BASE_NAMESPACE:+$BASE_NAMESPACE/}${BASE_IMG_BUILD} AS builder
FROM ${HERMES_BUILD_BASE_IMAGE:-${BASE_NAMESPACE:+$BASE_NAMESPACE/}${BASE_IMG_BUILD}} AS builder

ARG HERMES_SOURCE_REPOSITORY
ARG HERMES_SOURCE_COMMIT

# Build-time environment
ENV NODE_ENV=development
Expand All @@ -21,17 +31,26 @@ COPY work /opt/utils/
# Install build-time system dependencies (compilers + native libs needed for Python extensions).
# Without these, `uv sync` fails when compiling packages like `matrix-*-crypto`, `cryptography`, or `ffi`-based wheels on cold builds.
RUN set -eux \
&& printf 'Acquire::Retries "5";\nAcquire::http::Timeout "30";\n' > /etc/apt/apt.conf.d/80-labnow-retries \
&& . /opt/utils/script-utils.sh && install_apt /opt/utils/install_list_hermes.apt \
&& rm -f /etc/apt/apt.conf.d/80-labnow-retries \
## Clone source (full clone for reproducibility; depth 1 for speed)
&& git clone --depth 1 --branch main https://github.com/nousresearch/hermes-agent.git . \
&& test "$(printf '%s' "$HERMES_SOURCE_COMMIT" | wc -c | tr -d ' ')" = 40 \
&& git init . \
&& git remote add origin "$HERMES_SOURCE_REPOSITORY" \
&& git fetch --depth 1 origin "$HERMES_SOURCE_COMMIT" \
&& git checkout --detach FETCH_HEAD \
&& test "$(git rev-parse HEAD)" = "$HERMES_SOURCE_COMMIT" \
&& printf '%s\n' "$HERMES_SOURCE_REPOSITORY" > /opt/hermes/.labnow-source-repository \
&& printf '%s\n' "$HERMES_SOURCE_COMMIT" > /opt/hermes/.labnow-source-commit \
&& chmod +x /opt/utils/*.sh && mv /opt/utils/*hermes*.sh /opt/utils/install_list_hermes.apt /opt/utils/supervisord.conf /opt/hermes/ \
## ---------- hack python-olm for building compatible wheels ----------
&& mkdir -pv /opt/hermes/vendor \
&& mkdir -pv /tmp/olm && cd /tmp/olm \
&& curl -s https://pypi.org/pypi/python-olm/3.2.16/json \
| jq -r '.urls[] | select(.packagetype=="sdist").url' \
| xargs curl -L -o python-olm-3.2.16.tar.gz \
&& tar xf python-olm-3.2.16.tar.gz && cd python-olm-3.2.16 \
&& python -c 'import tarfile; tarfile.open("python-olm-3.2.16.tar.gz").extractall(path=".", filter="data")' && cd python-olm-3.2.16 \
&& sed -i 's/cmake_minimum_required(VERSION [0-9.]*)/cmake_minimum_required(VERSION 3.5)/' libolm/CMakeLists.txt \
&& pip wheel . --no-build-isolation -w /tmp/olm/wheels \
&& mv /tmp/olm/wheels/*olm*.whl /opt/hermes/vendor/ \
Expand All @@ -58,29 +77,51 @@ RUN set -eux \
&& printf 'docker\n' > /opt/hermes/.install_method

### --- Runtime Stage ---
FROM ${BASE_NAMESPACE:+$BASE_NAMESPACE/}${BASE_IMG}
FROM ${HERMES_RUNTIME_BASE_IMAGE:-${BASE_NAMESPACE:+$BASE_NAMESPACE/}${BASE_IMG}}

ARG HERMES_SOURCE_REPOSITORY
ARG HERMES_SOURCE_COMMIT
ARG HERMES_BUILD_BASE_IMAGE
ARG HERMES_RUNTIME_BASE_IMAGE

LABEL maintainer="[email protected]"
LABEL org.opencontainers.image.source="${HERMES_SOURCE_REPOSITORY}"
LABEL org.opencontainers.image.revision="${HERMES_SOURCE_COMMIT}"
LABEL io.labnow.hermes.build-base="${HERMES_BUILD_BASE_IMAGE}"
LABEL io.labnow.hermes.runtime-base="${HERMES_RUNTIME_BASE_IMAGE}"

# Production environment
ENV NODE_ENV=production
ENV PLAYWRIGHT_BROWSERS_PATH=/opt/hermes/.playwright
ENV PYTHONPATH="/opt/hermes:${PYTHONPATH:-}"
ENV HERMES_HOME=/root/.hermes
ENV HERMES_ALLOW_ROOT_GATEWAY=1
# The Dashboard PTY starts the already-built ui-tui bundle with `node`. The
# runtime base is intentionally Python-only, so copy the architecture-matched
# Node runtime produced by the builder instead of lazily downloading one after
# an operator opens Chat.
ENV PATH="/opt/node/bin:${PATH}"
# Copy the full hermes install tree from the builder (source + browsers + built frontends)
COPY --from=builder /opt/hermes /opt/hermes
# `/opt/node` contains node, npm and the Node runtime's bundled execution
# material. Both stages use the same Docker target platform.
COPY --from=builder /opt/node /opt/node

# Discover the real python site-packages so legacy env-var fallbacks point at the right tree.
# Keep explicit versioned fallbacks around in case detection runs before the first pip install.
RUN set -eux && cd /opt/hermes \
&& printf 'Acquire::Retries "5";\nAcquire::http::Timeout "30";\n' > /etc/apt/apt.conf.d/80-labnow-retries \
&& . /opt/utils/script-utils.sh && install_apt /opt/hermes/install_list_hermes.apt \
&& rm -f /etc/apt/apt.conf.d/80-labnow-retries \
&& uv pip install ./vendor/*.whl && rm -rf ./vendor \
&& uv pip install -e ".[all,messaging,anthropic,bedrock,azure-identity,hindsight,matrix]" \
&& rm -rf /opt/hermes/bin \
&& ln -sf /opt/hermes/start-hermes.sh /opt/conda/bin/hermes /usr/local/bin/ \
&& . /opt/utils/script-setup-sys.sh && setup_supervisord \
&& mkdir -pv /etc/supervisord/ && mv /opt/hermes/supervisord.conf /etc/supervisord/supervisord.conf \
&& node --version \
&& test -s /opt/hermes/ui-tui/dist/entry.js \
&& node --check /opt/hermes/ui-tui/dist/entry.js \
&& install__clean

# Data persistence is owned by the runtime orchestrator.
Expand Down
94 changes: 73 additions & 21 deletions docker_litellm/README.md
Original file line number Diff line number Diff line change
@@ -1,37 +1,89 @@
# LiteLLM Proxy

`litellm` is a lightweight proxy server to call 100+ LLM APIs using the OpenAI format, with a built-in UI dashboard.
本目录维护 LiteLLM 的部署适配,不 fork 或修改 LiteLLM 上游业务逻辑。提供可复现的本地基线:共享 PostgreSQL、共享 Redis、单副本与双副本 LiteLLM;凭据不写入仓库。

---
默认不构建 LiteLLM Dashboard 静态资源:固定源码在当前构建基础镜像上导出 `/_not-found` 时失败,而代理 API 与管理面不依赖该资源。需要 Dashboard 时可显式传入 `--build-arg BUILD_DASHBOARD=true` 单独处理该上游前端兼容性。

## 1. Port Configuration
## 固定版本与镜像

- **`4000` (HTTP)**: Serves the OpenAI-compatible REST API endpoints and the admin control panel dashboard interface.
| 项目 | 固定值 | 用途 |
| --- | --- | --- |
| LiteLLM 源码 | `v1.97.0-dev.1` / `ead62528e607b9d8e61273def638799c9c3a69ba` | Dockerfile 精确 fetch 并校验 HEAD |
| FastAPI | `0.136.3` | 固定到该 LiteLLM commit 仍使用 `get_flat_dependant` 的兼容版本 |
| Prisma Python client | `0.15.0` | LiteLLM 连接 PostgreSQL 所需客户端,兼容基础镜像的 Python 3.13 |
| 本地产物镜像 | `quay.io/labnow/litellm:1.97.0-ead62528e607` | Compose 的默认 LiteLLM 镜像 |
| PostgreSQL | `postgres:17-alpine@sha256:742f40ea20b9ff2ff31db5458d127452988a2164df9e17441e191f3b72252193` | 用户、凭证、模型、虚拟 key 与 spend 持久化 |
| Redis | `redis:7.4-alpine@sha256:e7723ff73d963f5cc6d9c4643ea3d989527a402a319239054e9472a7fb9219a2` | 副本共享认证缓存、RPM/TPM limiter 与协调缓存;SpendLog 的事实源是 PostgreSQL |

---
镜像构建会对 wheel 自带的 LiteLLM Prisma schema 运行 `prisma generate`,并把生成的查询引擎固定在 `/opt/litellm/.cache`;没有该步骤,或将该缓存随 `/root/.cache` 清理,代理会在 PostgreSQL startup 时报缺少 Prisma binaries 或无法连接查询引擎。

## 2. Data Persistence & Configurations
必须通过根目录 `tool.sh` 构建,避免基础镜像退回 Docker Hub:

LiteLLM looks for `config.yaml` in its home directory at startup:
```bash
export REGISTRY_SRC=quay.io
export REGISTRY_DST=quay.io
export CI_PROJECT_NAME=LabNow/lab-dev
source ./tool.sh
build_image_no_tag litellm 1.97.0-ead62528e607 docker_litellm/litellm.Dockerfile
```

- **`/opt/litellm`**: Sourced workspace directory (configured via `HOME_LITELLM`). This is where `config.yaml` is written and read.
- **`/root/workspace`**: Additional shared data directories volume.
构建完成后记录本地 digest:

### Custom Home Directory
You can override the home location using the environment variable:
- `HOME_LITELLM`: Paths to store the active configs (e.g. `/root/workspace`).
```bash
docker image inspect quay.io/labnow/litellm:1.97.0-ead62528e607 \
--format 'image_id={{.Id}} created={{.Created}}'
```

---
## 本地启动

## 3. Quickstart Example
准备不会被 Git 跟踪的配置。不要把 `.env` 发送到聊天、日志或提交中。

Run LiteLLM Proxy with mapped configuration folder:
```bash
docker run -d \
--name svc-litellm \
-p 4000:4000 \
-v /path/to/your/config:/opt/litellm \
labnow/litellm:latest
cd docker_litellm/compose
cp .env.example .env
# 在 .env 中生成并填写 LITELLM_MASTER_KEY、POSTGRES_PASSWORD、REDIS_PASSWORD。
# 真实上游调用另行填写 UPSTREAM_PROVIDER、UPSTREAM_API_KEY、UPSTREAM_BASE_URL、UPSTREAM_MODEL。
docker compose --env-file .env -f docker-compose.litellm.yml --profile single up -d
```

By default, it will look for a `config.yaml` in the directory. If not found, a basic template targeting `gpt-3.5-turbo` is auto-generated.
Compose 的项目、显式容器和外部网络均以仓库既有的 `PROFILE_ENV` 推导,默认实例为
`litellm-baseline`:Compose project 为 `litellm-baseline-svc-litellm`,网络为
`litellm-baseline-svc-litellm-net`。若需与另一套本地 LiteLLM 基线并行运行,在同一条命令前
设置不同实例名;不要混用 `-p` 或 `COMPOSE_PROJECT_NAME`,以免项目名与显式容器/网络命名源分离。

```bash
PROFILE_ENV=litellm-dev-a docker compose --env-file .env -f docker-compose.litellm.yml --profile single up -d
PROFILE_ENV=litellm-dev-b docker compose --env-file .env -f docker-compose.litellm.yml --profile single up -d
```

迁移与代理启动刻意分离:先用 `./scripts/run-migration.sh` 执行数据库迁移(migration profile),再启动代理 profile;不要把 migration profile 与代理 profile 放入同一条 `up` 命令。

默认端口只发布在 `127.0.0.1`:副本 1 为 `4000`,副本 2 为 `4001`。PostgreSQL 与 Redis 不发布宿主机端口。停止服务不会删除卷;如需删除本地数据,先人工确认后使用 `docker compose ... down -v`。

## 配置与安全边界

`config.yaml` 从运行时环境读取管理面 `LITELLM_MASTER_KEY`、`DATABASE_URL` 与 Redis 凭据。Compose 不把管理密钥、数据库密码或含密码的连接串写入服务 `environment`:它将 `LITELLM_MASTER_KEY`、`POSTGRES_PASSWORD` 和 `REDIS_PASSWORD` 交给 Docker Secret;PostgreSQL 使用官方 `POSTGRES_PASSWORD_FILE`,LiteLLM 的 `start-litellm.sh` 在最终 `exec` 前读取 Secret 文件、构造 `DATABASE_URL` 并立即转交 LiteLLM。管理面 key 仅用于 `/user/new`、`/credentials`、`/model/new`、`/key/generate`、`/key/block` 和 `/key/delete` 等管理接口;数据面虚拟 key 应为短期、模型白名单、TTL、预算、RPM、TPM 与 `llm_api` 路由限制的独立 key。双副本基线启用 `enable_redis_auth_cache`,并将 `user_api_key_cache_ttl` 设为 1 秒,以使撤销在 30 秒内经共享 Redis 重新校验。

| 变量 | 是否必填 | 作用 | 风险说明 |
| --- | ---: | --- | --- |
| `LITELLM_MASTER_KEY` | 是 | 管理面认证 | 仅放在忽略的 `.env` 或部署 Secret |
| `POSTGRES_PASSWORD` | 是 | PostgreSQL 密码 | 仅限本地测试或部署 Secret |
| `REDIS_PASSWORD` | 是 | Redis 认证 | 仅限本地测试或部署 Secret |
| `UPSTREAM_API_KEY` | 真实调用时是 | 上游模型凭据 | 仅由本地验证客户端读取;不会注入 LiteLLM 容器、不提交、不打印 |
| `UPSTREAM_PROVIDER` | 真实调用时是 | 上游 provider 选择 | 当前明确支持 `deepseek` |
| `UPSTREAM_BASE_URL` | 真实调用时是 | OpenAI 兼容上游地址 | 由环境决定 |
| `UPSTREAM_MODEL` | 真实调用时是 | 上游模型名 | 用于创建测试模型 |
| `REDIS_CIRCUIT_BREAKER_RECOVERY_TIMEOUT` | `5` | Redis 断连后的 LiteLLM 缓存恢复探测窗口(秒) | 恢复期间管理面可能暂时返回 500 |

Compose 凭据边界的残余风险:LiteLLM 上游配置接口仍要求 `LITELLM_MASTER_KEY` 与 `DATABASE_URL` 在其最终进程环境中可见;本基线已接受这一点。凭据不出现在 Compose 渲染、容器 `docker inspect` metadata、命令行参数、容器日志或运行时临时文件中。使用具有 Docker daemon 访问权限或容器内同等调试权限的主体仍应视为高权限主体,不应以该边界替代主机与容器访问控制。

## Readiness 与 Redis 说明

LiteLLM `v1.97.0-dev.1` 的公开 `/health/readiness` 仅返回服务与数据库连通性,不将 Redis 纳入公开 readiness。因此 Compose 健康检查只能确认 LiteLLM + PostgreSQL;可额外从每个 LiteLLM 容器执行 Redis `PING` 确认。若 Redis 不可用,多副本认证缓存、RPM/TPM limiter 与协调结论无效,不能宣称为高可用。Redis 恢复后,LiteLLM 的认证缓存 circuit breaker 需要经过 `REDIS_CIRCUIT_BREAKER_RECOVERY_TIMEOUT` 后才会重新探测,默认 5 秒。跨副本 SpendLog 只证明 PostgreSQL 可见性,不是 Redis Spend counter 或预算准入控制证据。

## 常见问题

- `LITELLM_MASTER_KEY` 或数据库密码缺失:先检查被忽略的 `compose/.env`,不要将其内容贴出。
- readiness 未连接数据库:查看 `docker compose ... logs postgres litellm-1`,并保留卷以便排查迁移。
- Redis 探针失败:先确认 `redis` health 与密码一致,再做双副本撤销验证。
- 上游调用:仅在 `.env` 中提供专用、低权限、可轮换的测试 key。
27 changes: 27 additions & 0 deletions docker_litellm/compose/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Copy this file to docker_litellm/compose/.env. It is intentionally ignored.
# Do not commit real API keys, management keys, passwords, or virtual keys.

TZ=Asia/Hong_Kong

# Build this exact source baseline through ../../tool.sh before running Compose.
LITELLM_IMAGE=quay.io/labnow/litellm:1.97.0-ead62528e607
LITELLM_1_CONTAINER_NAME=svc-litellm-1
LITELLM_2_CONTAINER_NAME=svc-litellm-2
LITELLM_PUBLISH_HOST=127.0.0.1
LITELLM_1_PORT=4000
LITELLM_2_PORT=4001

# Local-only secrets. Generate unique values; these examples are placeholders.
LITELLM_MASTER_KEY=<local-only-management-key>
POSTGRES_DB=litellm
POSTGRES_USER=litellm
POSTGRES_PASSWORD=<local-only-postgres-password>
REDIS_PASSWORD=<local-only-redis-password>

# Optional upstream used only by local verification clients; it is never injected
# into the LiteLLM containers. Supported provider: deepseek.
# Keep UPSTREAM_API_KEY empty to validate infrastructure paths only.
UPSTREAM_PROVIDER=deepseek
UPSTREAM_API_KEY=
UPSTREAM_BASE_URL=https://api.deepseek.com/v1
UPSTREAM_MODEL=deepseek-v4-flash
20 changes: 20 additions & 0 deletions docker_litellm/compose/config.migrate.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Dedicated one-shot migration configuration. Keep this in sync with
# config.yaml; only this job is permitted to apply Prisma migrations.
model_list: []

general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
database_url: os.environ/DATABASE_URL
store_model_in_db: true
disable_spend_logs: false
disable_prisma_schema_update: false

litellm_settings:
turn_off_message_logging: true
cache: true
enable_redis_auth_cache: true
cache_params:
type: redis
host: os.environ/REDIS_HOST
port: os.environ/REDIS_PORT
password: os.environ/REDIS_PASSWORD
Loading
Loading