Dev

Dockerfile

@@ -1,4 +1,4 @@
-# EGo — 通用信息转发平台 v1.2.4
+# EGo — 通用信息转发平台
 FROM python:3.11-alpine3.18
 
 LABEL maintainer="EGo Team"
@@ -13,7 +13,7 @@ RUN set -eux && \
     apk -U --no-cache add tzdata openssl && \
     cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \
     echo "Asia/Shanghai" > /etc/timezone && \
-    mkdir -p /app/data /app/config /app/parsers
+    mkdir -p /app/data /app/config /app/parsers /app/channels
 
 WORKDIR /app
 
@@ -22,7 +22,10 @@ RUN python3 -m pip install --no-cache-dir -r requirements.txt -q
 
 COPY . .
 
-VOLUME ["/app/data", "/app/config"]
+# 持久化：运行时产生的数据 + **用户上传的插件**。
+# 内置插件在 parsers_builtin/ 与 channels_builtin/（随镜像更新），故意**不**打卷——
+# named volume 首次创建会把镜像内容拷进去，之后以卷为准，内置插件就永远升不上去了。
+VOLUME ["/app/data", "/app/config", "/app/parsers", "/app/channels"]
 
 ENV WEB_PORT=5000
 ENV WEB_SSL_PORT=5001

README.en.md

@@ -1,4 +1,4 @@
-# EverywhereYouGo (EGo) v1.3.0
+# EverywhereYouGo (EGo) v1.3.2
 
 [中文](README.md) | English
 
@@ -35,7 +35,7 @@ HTTP POST → Data Source → Parser → Route Match → Template Render → Pus
 | **Parser** | Python script, extracts fields and defines variable names |
 | **Route** | Condition expression matches channel-template pairs |
 | **Template** | Simple / Jinja2 renders title and content |
-| **Channel** | WeChat Work, DingTalk, Feishu, Telegram, Bark |
+| **Channel** | WeChat Work, DingTalk, Feishu, Telegram, Bark, Email (SMTP) |
 
 ## Authentication
 
@@ -80,7 +80,75 @@ Hand-editing `config/*.json` is only read on first import when the database is e
 it is not the normal path for applying changes.
 
 System settings (DND, log level, etc.), the message log and the queue are also stored in SQLite.
-For backup/restore use **Settings → Backup**, which packages `config/*.json` + `parsers/*.py`.
+For backup/restore use **Settings → Backup**, which packages `config/*.json` plus **your uploaded**
+`parsers/*.py` and `channels/*.py`.
+
+**Restore is not the same path as startup loading**: on restore, config from the backup is
+written to the database (`config/*.json` → SQLite), then plugins are reloaded and every
+data-source listener is restarted.
+
+**Restore is a partial restore**: only config files *present* in the backup are written to the
+database; anything not included is **left unchanged** (not wiped), and a notice listing the missing
+files is returned. A hand-made or truncated ZIP therefore cannot destroy data it never contained.
+To restore a table to empty, ship a file containing `[]` rather than omitting it
+(**file present and `[]` → table cleared; file absent → table untouched**).
+
+> ⚠️ **Back up before upgrading**: export a ZIP via *Settings → Backup* before any version/image
+> upgrade. Backup files contain **full push credentials** (SMTP password / auth code / tokens) —
+> keep them safe.
+
+## Plugin Directories
+
+Built-in plugins and user-uploaded plugins live in **separate directories**:
+
+| Directory | Content | Docker |
+|-----------|---------|--------|
+| `parsers_builtin/` | Built-in parsers | Shipped in the image, **no volume** |
+| `parsers/` | Your uploaded parsers | `ego_parsers` volume — survives container recreation |
+| `channels_builtin/` | Built-in channel plugins | Shipped in the image, **no volume** |
+| `channels/` | Your uploaded channel plugins | `ego_channels` volume — survives container recreation |
+
+Rules:
+
+- **Resolution order: user directory first**, then built-in
+- **Uploading a name that collides with a built-in is rejected** (with a clear error) —
+  built-ins are updated with the image, so a same-named file would never take effect
+- **Built-in plugins are read-only**: viewable in the WebUI, but not editable or deletable
+- To actually change a built-in's behaviour, put your version under a **different filename**
+  in the user directory, or edit the source and rebuild the image
+
+> **Why the split is required**: mounting a volume over the directory that holds the built-ins
+> makes Docker copy the image's contents into the volume on first creation, and the volume
+> becomes authoritative from then on — image upgrades would never reach the built-ins.
+> With one shared directory there is no way out: you either lose user files, or the built-ins
+> can never be upgraded.
+
+**Upgrading from v1.3.0 to v1.3.1+**
+Older versions wrote user-uploaded plugins directly into the **built-in** directory,
+which is **not persisted** — recreating the container would lose them. Before
+upgrading to v1.3.1+, **back up your user plugins first**:
+
+1. **Export** (pick one):
+   - Export a ZIP via the old container's *Settings → Backup* (if the old version supports it);
+   - Or `docker cp` the plugin directory from the old container (in the old version,
+     user plugins share one unvolume-mounted directory with the built-ins — the exact
+     path depends on your old deployment):
+     ```bash
+     docker cp <ego-container>:/app/parsers_builtin  /tmp/old-parsers
+     docker cp <ego-container>:/app/channels_builtin /tmp/old-channels
+     ```
+2. **Upgrade the image**: from v1.3.1+ onward, user plugins live in the **user**
+   directory backed by named volumes (`ego_parsers`/`ego_channels`). A regular image
+   upgrade (pull the new image, recreate the container) **does not require re-uploading
+   plugins** — they survive via the volumes.
+3. **Restore** (only when migrating from the old version): restore writes into the
+   new user directory and automatically skips entries that collide with built-ins
+   (to prevent stale copies from shadowing the new built-in plugins).
+
+> **Volume operation semantics** (`ego_parsers` / `ego_channels`):
+> - `docker compose up` (or recreating the container) → volume is kept, plugins survive
+> - `docker compose down` → named volumes are kept by default, plugins survive
+> - `docker compose down -v` → **volumes are deleted, all user plugins are lost** — confirm first
 
 ## Parsers
 
@@ -135,10 +203,15 @@ By default **only the channels that failed are retried** — already-succeeded c
 not pushed a second time. Use `scope=all` to force a full re-push.
 
 ### Import & Export
-- **Backup**: Download ZIP package (`config/*.json` + `parsers/*.py`)
-- **Restore**: Upload ZIP package, automatically takes effect after overwriting configuration
+- **Backup**: Download ZIP package (`config/*.json` + `parsers/*.py` + `channels/*.py`)
+- **Restore**: Upload ZIP package; config from the backup is written to the database (JSON → DB), plugin files are restored and hot-reloaded. Config not included in the backup is left unchanged, with a notice listing the missing files
+- **Preview**: runs the **same** validation as a real restore (size / completeness / JSON shape / plugin loadability); the "confirm restore" button only appears when it passes
 - **JSON Import**: Supports dry_run preview, insert/overwrite two modes, dependency check
 
+> **Security note**
+> - The backup ZIP contains **full push credentials** (SMTP password / auth code / token, etc.) — keep it secure and never share it.
+> - JSON export masks sensitive fields (`password` / `token` / `secret` / `webhook` / `device_key`, etc.) as `***` for display and archiving only. Full credentials are preserved only in the backup ZIP and restored from it.
+
 ### Channel Circuit Breaker
 Automatically isolates a channel that keeps failing, so one broken third party cannot
 drag down the whole send path:
@@ -157,6 +230,12 @@ Per-channel rate limit (messages per minute) to avoid getting blocked by the rem
 Retries cannot fix a 429 — limiting has to happen **before** sending. A message that
 cannot get a token waits in the queue instead of being dropped.
 
+**Token bucket**: each channel has an independent token bucket whose capacity (burst)
+equals the configured per-minute quota; it refills at `quota ÷ 60` tokens per second.
+Sending one message takes 1 token; when the bucket is empty the message is queued —
+not dropped — until a token is available (up to `EGO_RATE_MAX_WAIT` seconds, after which
+it is deferred instead).
+
 ### Resilience UI
 | Where | What you can do |
 |-------|-----------------|
@@ -194,6 +273,7 @@ Built-in Chinese and English bilingual support, switch languages anytime via lan
 | Feishu | Webhook | `feishu` |
 | Telegram | Bot API | `telegram_bot` |
 | Bark | API | `bark` |
+| Email (SMTP) | SMTP | `smtp_email` |
 
 ## Environment Variables
 

README.md

@@ -1,4 +1,4 @@
-# EverywhereYouGo (EGo) v1.3.0
+# EverywhereYouGo (EGo) v1.3.2
 
 [English](README.en.md) | 中文
 
@@ -35,7 +35,7 @@ HTTP POST → 数据源 → 解析器 → 路由匹配 → 模板渲染 → 推
 | **解析器** | Python 脚本，提取字段并定义变量名 |
 | **路由** | 条件表达式匹配渠道-模板对 |
 | **模板** | Simple / Jinja2 渲染标题和内容 |
-| **渠道** | 企业微信、钉钉、飞书、Telegram、Bark |
+| **渠道** | 企业微信、钉钉、飞书、Telegram、Bark、邮件 (SMTP) |
 
 ## 认证
 
@@ -78,7 +78,65 @@ EGO_AUTH_TOKEN=your-secret-token python3 main.py
 「数据库为空」的首次导入场景才会被读取，不是常规生效路径。
 
 系统设置（DND、日志级别等）、消息日志与队列同样存储在 SQLite。
-配置备份 / 恢复请用「系统设置 → 备份」，会打包 `config/*.json` + `parsers/*.py`。
+配置备份 / 恢复请用「系统设置 → 备份」，会打包 `config/*.json` + **用户上传的** `parsers/*.py` 与 `channels/*.py`。
+
+**恢复（Restore）与启动加载不是同一条路径**：恢复时把备份里的配置写入数据库
+（`config/*.json` → SQLite），随后重载插件并重启各数据源监听。
+
+**恢复是"部分恢复"**：只把备份里**存在**的配置文件写入数据库，未包含的配置**保持原样**
+（不会被清空），并在结果里给出提示。所以手搓/截断的 ZIP 不会把没带上的数据弄丢。
+需要把某张表恢复成空，请在 ZIP 里放一个内容为 `[]` 的文件，而不是删掉它
+（**文件存在且为 `[]` → 清空该表；文件不存在 → 该表不动**）。
+
+> ⚠️ **升级前请先备份**：任何版本 / 镜像升级前，请先在「系统设置 → 备份」导出 ZIP。
+> 备份文件含**完整推送凭据**（SMTP 密码 / 授权码 / token 等），请妥善保管。
+
+## 插件目录
+
+内置插件与用户上传的插件**分开放**：
+
+| 目录 | 内容 | Docker |
+|------|------|--------|
+| `parsers_builtin/` | 内置解析器 | 随镜像发布，**不打卷** |
+| `parsers/` | 你上传的解析器 | 挂 `ego_parsers` 卷，容器重建不丢 |
+| `channels_builtin/` | 内置通道插件 | 随镜像发布，**不打卷** |
+| `channels/` | 你上传的通道插件 | 挂 `ego_channels` 卷，容器重建不丢 |
+
+规则：
+
+- **解析顺序：用户目录优先**，其次内置目录
+- **与内置插件同名不允许上传**（返回明确错误）—— 内置插件随镜像更新，
+  同名文件不会生效，不如一开始就说清楚
+- **内置插件只读**：WebUI 里可以查看，但不能改、不能删
+- 确实要改内置插件的行为：把改好的文件**换个名字**放进用户目录，
+  或直接改源码重建镜像
+
+> **为什么必须分开**：如果把卷挂在放内置插件的目录上，named volume 首次创建会把
+> 镜像里该目录的内容拷进卷里，之后就以卷为准 —— 镜像升级再也更新不到内置插件。
+> 两者同目录无解：要么丢用户文件，要么内置永远升不上去。
+
+**从旧版本升级（v1.3.0 → v1.3.1+）**
+旧版本把用户上传的插件直接写在**内置目录**里，而该目录**没有持久化**——容器重建
+就会丢失。升级到 v1.3.1+ 前，请**先**把用户插件备份出来：
+
+1. **导出**（二选一）：
+   - 旧容器里「系统设置 → 备份」导出 ZIP（若旧版支持）；
+   - 或直接 `docker cp` 导出旧容器里的插件目录（旧版用户插件与内置混在同一
+     未挂载卷的目录里，具体路径看你的旧版部署）：
+     ```bash
+     docker cp <ego-container>:/app/parsers_builtin  /tmp/old-parsers
+     docker cp <ego-container>:/app/channels_builtin /tmp/old-channels
+     ```
+2. **升级镜像**：v1.3.1+ 起用户插件落在**用户目录**并挂 named volume
+   （`ego_parsers`/`ego_channels`），普通镜像升级（拉新镜像、重建容器）**无需
+   重传插件**——插件随卷保留。
+3. **恢复**（仅从旧版迁移时需要）：恢复会写进新的用户目录，并自动跳过与内置
+   同名的条目（避免用旧副本遮蔽新版内置插件）。
+
+> **卷操作语义**（`ego_parsers` / `ego_channels`）：
+> - `docker compose up`（或重建容器）→ 保留卷，插件不丢
+> - `docker compose down` → 默认保留 named volume，插件不丢
+> - `docker compose down -v` → **删除卷，用户插件全部丢失**，操作前先确认
 
 ## 解析器
 
@@ -132,10 +190,17 @@ def parse(raw_body: bytes, headers: dict, query_params: dict) -> dict:
 需要整体重推时可用 `scope=all`。
 
 ### 导入导出
-- **备份**：下载 ZIP 包（`config/*.json` + `parsers/*.py`）
-- **恢复**：上传 ZIP 包，覆盖配置后自动生效
+- **备份**：下载 ZIP 包（`config/*.json` + `parsers/*.py` + `channels/*.py`）
+- **恢复**：上传 ZIP 包，**备份中的配置写入数据库**（JSON → DB），插件文件同步恢复并热重载。
+  备份未包含的配置保持原样，并在结果里提示缺了哪些文件
+- **预览**：点「预览」会先跑一遍与真实恢复**相同**的校验（体积 / 完整性 / JSON 结构 / 插件可加载），
+  校验不通过（如 JSON 结构非法）时不会给出"确认恢复"按钮
 - **JSON 导入**：支持 dry_run 预览、insert/overwrite 两种模式、依赖检查
 
+> **安全提示**
+> - 备份 ZIP 内含**完整推送凭据**（SMTP 密码 / 授权码 / token 等），需妥善保管，勿外泄。
+> - JSON 导出已对敏感字段（`password` / `token` / `secret` / `webhook` / `device_key` 等）脱敏为 `***`，仅用于展示与归档；完整凭据仅经备份 ZIP 保留并恢复。
+
 ### 通道熔断
 第三方渠道持续故障时自动隔离，避免拖垮整条发送链路：
 
@@ -149,6 +214,10 @@ def parse(raw_body: bytes, headers: dict, query_params: dict) -> dict:
 按通道独立限流（条/分钟），防止发送过快被对方封禁。重试解决不了 429——
 限流必须发生在发送**之前**。拿不到令牌的消息会排队等待，而不是被丢弃。
 
+**令牌桶（Token Bucket）**：每个通道独立令牌桶，桶容量（burst）等于配置的分钟
+额度，按 `额度 ÷ 60` 每秒补充令牌；发一条消息需取 1 个令牌，桶空时不立即
+丢弃而是排队直到有令牌（最长等 `EGO_RATE_MAX_WAIT` 秒，超时则延迟重排）。
+
 ### 韧性界面
 | 位置 | 能做什么 |
 |------|---------|
@@ -185,6 +254,7 @@ def parse(raw_body: bytes, headers: dict, query_params: dict) -> dict:
 | 飞书 | Webhook | `feishu` |
 | Telegram | Bot API | `telegram_bot` |
 | Bark | API | `bark` |
+| 邮件 (SMTP) | SMTP | `smtp_email` |
 
 ## 环境变量
 

api/__init__.py

@@ -33,6 +33,13 @@ def create_app(source_mgr=None):
     app.secret_key = secret
     app.config["PERMANENT_SESSION_LIFETIME"] = timedelta(hours=24)
 
+    # ── 上传体积上限（v1.3.2 review #11）──
+    # 第一层防护：超过上限的请求由 Flask 直接以 413 拒绝，**不进入视图函数**，
+    # 因此超大 body 不会被 file.read() 整个读进内存。
+    # 备份上传另有 MAX_BACKUP_UPLOAD_SIZE（api/backup.py）做友好报错。
+    _max_upload_mb = int(os.getenv("EGO_MAX_UPLOAD_MB", "32"))
+    app.config["MAX_CONTENT_LENGTH"] = _max_upload_mb * 1024 * 1024
+
     # ── CSRF 缓解（最小化，#24）──
     # 内网自管理场景无需引入完整 Flask-WTF CSRF：
     #   SameSite=Lax 阻断跨站顶层表单提交（主要 CSRF 向量），
@@ -144,4 +151,13 @@ def _auth_middleware():
     def _handle_validation_error(e):
         return jsonify({"status": "error", "error": str(e)}), 400
 
+    # ── 413：请求体超过 MAX_CONTENT_LENGTH（review #11）──
+    @app.errorhandler(413)
+    def _handle_request_too_large(e):
+        msg = i18n._("err.upload_too_large").replace(
+            "{size}", str(app.config.get("MAX_CONTENT_LENGTH", 0) // (1024 * 1024)))
+        if request.path.startswith("/api/"):
+            return jsonify({"error": msg}), 413
+        return msg, 413
+
     return app