chenghuaWang · Sep 8, 2026 · Sep 8, 2026 · Sep 8, 2026 · coderabbitai · Sep 8, 2026
diff --git a/README-ZH.md b/README-ZH.md
@@ -17,6 +17,7 @@ mllm
 
 ## 最新动态
 
+- [2026 年 9 月 8 日] MLLM 现已支持 **[MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)**！在手机上通过 ARM CPU 本地运行。[快速上手](./examples/minicpm5/README.md)。
 - [2026 年 6 月 8 日] `pymllm` 已覆盖 Qwen3、Qwen3-VL 与 Qwen3.5 在 Jetson Orin 上的 W4A16 / W8A8 serving；Qwen3-VL-2B W8A8 在 AGX Orin 32GB 上最高达到 3.12x prefill 加速比，decode 吞吐整体与 llama.cpp 接近。
 - [2026 年 4 月 30 日] `pymllm` 新增面向 Jetson 的 Qwen3 / Qwen3-VL BF16、W4A16 和 W8A8 serving 支持，覆盖 compressed-tensors AWQ 与 W8A8 INT8 路径。
 - [2026 年 3 月 18 日] 🔥🔥🔥 `pymllm` 已支持在 Jetson Orin 和 Jetson Thor 设备上使用 CUDA（实验特性，仍在持续开发中）。
@@ -105,6 +106,8 @@ mllm 框架可以与主流社区框架的模型检查点无缝集成。通过 ml
 | [Qwen3-0.6B](https://github.com/QwenLM/Qwen3)                     | [✔️ w4a8](https://www.modelscope.cn/models/mllmTeam/Qwen3-0.6B-w4a32kai)  |  |  |
 | [Qwen3-1.7B](https://github.com/QwenLM/Qwen3)                     | [✔️ w4a8](https://www.modelscope.cn/models/mllmTeam/Qwen3-1.7B-w4a8-i8mm-kai)  |  | [W4A16-SM8650](https://modelscope.cn/models/mllmTeam/Qwen3-1.7B-Qnn-AOT-SM8650/summary) |
 | [Qwen3-4B](https://github.com/QwenLM/Qwen3)                      | [✔️ w4a8](https://www.modelscope.cn/models/mllmTeam/Qwen3-4B-w4a8-i8mm-kai)  |  |  |
+| [MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)          | [✔️ w4a8](./examples/minicpm5/README.md) | | |
+| [MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)          | [✔️ w4a8](./examples/minicpm5/README.md) | | |
 | Qwen3.5-2B                                                       |  | ✔️ W4A16 / W8A8 |  |
 | Qwen3.5-4B                                                       |  | ✔️ W4A16 / W8A8 |  |
 | Qwen3-VL-2B-Instruct                                            |  | ✔️ W4A16 / W8A8 |  |

diff --git a/README.md b/README.md
@@ -17,6 +17,7 @@ mllm
 
 ## Latest News
 
+- [2026 Sep 08] MLLM now supports **[MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)**! Run it locally on your phone with ARM CPU inference. [Get started](./examples/minicpm5/README.md).
 - [2026 Jun 08] 🔥🔥🔥`pymllm` now covers Qwen3, Qwen3-VL, and Qwen3.5 on Jetson Orin with W4A16 / W8A8 serving; Qwen3-VL-2B W8A8 reaches up to 3.12x prefill speedup on AGX Orin 32GB, while decode throughput stays broadly close to llama.cpp.
 - [2026 May 02] 🔥🔥🔥 MLLM now supports the Ascend NPU backend, with ATB graph execution and Qwen3 W8A8 inference on Ascend devices.
 - [2026 Apr 30] 🔥🔥🔥`pymllm` adds Jetson-oriented Qwen3 / Qwen3-VL BF16, W4A16, and W8A8 serving support, including compressed-tensors AWQ and W8A8 INT8 paths.
@@ -107,6 +108,7 @@ The mllm framework integrates seamlessly with popular community frameworks' chec
 | [Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B)           | [✔️ w4a8](./examples/qwen3_5/README.md) | | |
 | [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)               | [✔️ w4a8](./examples/qwen3_5/README.md) | | |
 | [MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)          | [✔️ w4a8](./examples/minicpm5/README.md) | | |
+| [MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)          | [✔️ w4a8](./examples/minicpm5/README.md) | | |
 | [LFM2.5-2.6B](https://huggingface.co/LiquidAI/LFM2.5-2.6B)         | [✔️ w4a8](./examples/lfm2/README.md) | | |
 | [DeepSeek-OCR](https://github.com/deepseek-ai/DeepSeek-OCR)       | [✔️ w4a8](https://www.modelscope.cn/models/mllmTeam/DeepSeek-OCR-w4a8-i8mm-kai)  |  | |
 | [SmolLM3](https://huggingface.co/blog/smollm3)| [✔️ w4a8](https://www.modelscope.cn/models/mllmTeam/SmolLM3-3B-w4a8-i8mm-kai)  |  | |

diff --git a/examples/minicpm5/README.md b/examples/minicpm5/README.md
@@ -1,13 +1,26 @@
-# MiniCPM5-1B on ARM CPU
+# MiniCPM5 on ARM CPU
 
-This example supports the text-only [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) checkpoint.
+This example supports the text-only [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) and
+[`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B/tree/0e9c66dce9fedde5ba8663bbcdd54b6810bb929a) checkpoints.
 The runtime contract is batch 1, a 2048-token mobile cache, 16 query heads, 2 native KV heads, and explicit
 `head_dim=128`. The implementation keeps KV history at KV-head count and uses a correctness-first eager GQA path;
 it does not materialize a persistent 16-head KV cache.
 
 The runner implements the official no-tool chat-template branch, including optional system text and
 `enable_thinking=true|false`. Tool schemas, tool calls, and multi-turn history are outside this first product surface.
 
+| Variant | Layers | Hidden / FFN width | Runtime config | Quantization config |
+| --- | ---: | --- | --- | --- |
+| 1B | 24 | 1536 / 4608 | `config_1B_w4a32_kai.json` | `quant_cfg_1B_w4a32_kai.json` |
+| 2B | 42 | 2048 / 6144 | `config_2B_w4a32_kai.json` | `quant_cfg_2B_w4a32_kai.json` |
+
+Both variants reuse the same model graph, native KV cache and GQA kernels. At the supplied 2048-token
+capacity, FP32 K/V payloads occupy 96 MiB for 1B and 168 MiB for 2B. The official 131072-position
+RoPE limit does not enlarge the runner's supported mobile cache.
+Use the matching checkpoint, runtime config and quantization config together; mixed 1B/2B geometry is rejected.
+The 2B tokenizer vocabulary is unchanged. Its updated history template does not change the supported
+single-user branch; multi-turn history and tool calling remain unsupported.
+
 ## Convert
 
 Keep embeddings and norms in FP32; the supplied configuration packs transformer Linear weights and the independent
@@ -26,6 +39,10 @@ python -m pymllm.mobile.utils.mllm_convertor \
 
 ## Run
 
+For 2B, use `MiniCPM5-2B` and `minicpm5-2b-w4a32-kai.mllm` in these commands and select
+`quant_cfg_2B_w4a32_kai.json` during conversion and `config_2B_w4a32_kai.json` during inference.
+The same `demo_prompt_200.txt` is used for both sizes.
+
 ```bash
 mllm-minicpm5-runner \
   --model_path /path/to/minicpm5-1b-w4a32-kai.mllm \

diff --git a/examples/minicpm5/config_2B_w4a32_kai.json b/examples/minicpm5/config_2B_w4a32_kai.json
@@ -0,0 +1,28 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "model_type": "llama",
+  "bos_token_id": 0,
+  "eos_token_id": [
+    1,
+    130073
+  ],
+  "pad_token_id": 1,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "intermediate_size": 6144,
+  "max_position_embeddings": 131072,
+  "num_attention_heads": 16,
+  "num_hidden_layers": 42,
+  "num_key_value_heads": 2,
+  "head_dim": 128,
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 5000000,
+  "rope_scaling": null,
+  "tie_word_embeddings": false,
+  "use_cache": true,
+  "vocab_size": 130560,
+  "max_cache_length": 2048,
+  "linear_impl_type": "KaiLinear_f32_qai8dxp_qsi4c32p_mxk_nxk_qai8dxp1x8_qsi4c32p8x8_1x8x32"
+}
diff --git a/examples/minicpm5/main.cpp b/examples/minicpm5/main.cpp
@@ -216,7 +216,7 @@ MLLM_MAIN({
       }
       if (exit_code == 0) { fmt::print("Benchmark records: {}\n", benchmark_jsonl.get()); }
     } else {
-      fmt::print("\n{:*^60}\n", prompt.isSet() ? " MiniCPM5-1B One-shot CLI " : " MiniCPM5-1B Interactive CLI ");
+      fmt::print("\n{:*^60}\n", prompt.isSet() ? " MiniCPM5 One-shot CLI " : " MiniCPM5 Interactive CLI ");
       if (!prompt.isSet()) fmt::print("Enter 'exit' or 'quit' to end the session\n\n");
 
       while (true) {

diff --git a/examples/minicpm5/quant_cfg_2B_w4a32_kai.json b/examples/minicpm5/quant_cfg_2B_w4a32_kai.json
@@ -0,0 +1,62 @@
+{
+  "^model\\.layers\\.\\d+\\.self_attn\\.q_proj\\.(bias|weight)": {
+    "hints": {
+      "quant_method": "kai",
+      "kai_matmul_triplet": "f32_qai8dxp_qsi4c32p",
+      "kai_matmul_layout": "mxk_nxk",
+      "kai_matmul_tile_cfg": "qai8dxp1x8_qsi4c32p8x8_1x8x32",
+      "shape": [2048, 2048],
+      "replace": true
+    }
+  },
+  "^model\\.layers\\.\\d+\\.self_attn\\.(k_proj|v_proj)\\.(bias|weight)": {
+    "hints": {
+      "quant_method": "kai",
+      "kai_matmul_triplet": "f32_qai8dxp_qsi4c32p",
+      "kai_matmul_layout": "mxk_nxk",
+      "kai_matmul_tile_cfg": "qai8dxp1x8_qsi4c32p8x8_1x8x32",
+      "shape": [256, 2048],
+      "replace": true
+    }
+  },
+  "^model\\.layers\\.\\d+\\.self_attn\\.o_proj\\.(bias|weight)": {
+    "hints": {
+      "quant_method": "kai",
+      "kai_matmul_triplet": "f32_qai8dxp_qsi4c32p",
+      "kai_matmul_layout": "mxk_nxk",
+      "kai_matmul_tile_cfg": "qai8dxp1x8_qsi4c32p8x8_1x8x32",
+      "shape": [2048, 2048],
+      "replace": true
+    }
+  },
+  "^model\\.layers\\.\\d+\\.mlp\\.(gate_proj|up_proj)\\.(bias|weight)": {
+    "hints": {
+      "quant_method": "kai",
+      "kai_matmul_triplet": "f32_qai8dxp_qsi4c32p",
+      "kai_matmul_layout": "mxk_nxk",
+      "kai_matmul_tile_cfg": "qai8dxp1x8_qsi4c32p8x8_1x8x32",
+      "shape": [6144, 2048],
+      "replace": true
+    }
+  },
+  "^model\\.layers\\.\\d+\\.mlp\\.down_proj\\.(bias|weight)": {
+    "hints": {
+      "quant_method": "kai",
+      "kai_matmul_triplet": "f32_qai8dxp_qsi4c32p",
+      "kai_matmul_layout": "mxk_nxk",
+      "kai_matmul_tile_cfg": "qai8dxp1x8_qsi4c32p8x8_1x8x32",
+      "shape": [2048, 6144],
+      "replace": true
+    }
+  },
+  "^lm_head\\.(bias|weight)": {
+    "hints": {
+      "quant_method": "kai",
+      "kai_matmul_triplet": "f32_qai8dxp_qsi4c32p",
+      "kai_matmul_layout": "mxk_nxk",
+      "kai_matmul_tile_cfg": "qai8dxp1x8_qsi4c32p8x8_1x8x32",
+      "shape": [130560, 2048],
+      "replace": true
+    }
+  }
+}
diff --git a/mllm/models/minicpm5/configuration_minicpm5.hpp b/mllm/models/minicpm5/configuration_minicpm5.hpp
@@ -95,23 +95,29 @@ struct MiniCPM5Config : protected ConfigFile {
   aops::LinearImplTypes linear_impl_type = aops::LinearImplTypes::kDefault;
 };
 
-inline auto matchesOfficialMiniCPM5_1BRuntimeContract(const MiniCPM5Config& config) -> bool {
+inline auto matchesOfficialMiniCPM5RuntimeContract(const MiniCPM5Config& config) -> bool {
   constexpr auto kKaiLinearImpl = aops::LinearImplTypes::kKaiLinear_f32_qai8dxp_qsi4c32p_mxk_nxk_qai8dxp1x8_qsi4c32p8x8_1x8x32;
-  return config.vocab_size == 130560 && config.hidden_size == 1536 && config.intermediate_size == 4608
-         && config.num_hidden_layers == 24 && config.num_attention_heads == 16 && config.num_key_value_heads == 2
-         && config.head_dim == 128 && config.hidden_act == "silu" && config.max_position_embeddings == 131072
-         && config.rms_norm_eps == 1.0e-6F && config.rope_theta == 5000000.0F && !config.attention_bias
-         && !config.tie_word_embeddings && config.bos_token_id == 0 && config.pad_token_id == 1
+  const bool supported_dimensions =
+      (config.hidden_size == 1536 && config.intermediate_size == 4608 && config.num_hidden_layers == 24)
+      || (config.hidden_size == 2048 && config.intermediate_size == 6144 && config.num_hidden_layers == 42);
+  return supported_dimensions && config.vocab_size == 130560 && config.num_attention_heads == 16
+         && config.num_key_value_heads == 2 && config.head_dim == 128 && config.hidden_act == "silu"
+         && config.max_position_embeddings == 131072 && config.rms_norm_eps == 1.0e-6F && config.rope_theta == 5000000.0F
+         && !config.attention_bias && !config.tie_word_embeddings && config.bos_token_id == 0 && config.pad_token_id == 1
          && config.eos_token_ids == std::vector<int64_t>({1, 130073}) && config.max_cache_length == 2048
          && config.linear_impl_type == kKaiLinearImpl;
 }
 
+inline auto matchesOfficialMiniCPM5_1BRuntimeContract(const MiniCPM5Config& config) -> bool {
+  return matchesOfficialMiniCPM5RuntimeContract(config) && config.hidden_size == 1536;
+}
+
 inline void validateModelConfigMatch(const MiniCPM5Config& config, const ParameterFile::ptr_t& parameter_file) {
   constexpr auto kEmbeddingWeight = "model.embed_tokens.weight";
   constexpr auto kLmHeadWeight = "lm_head.weight";
-  if (!matchesOfficialMiniCPM5_1BRuntimeContract(config)) {
+  if (!matchesOfficialMiniCPM5RuntimeContract(config)) {
     throw std::invalid_argument(
-        "MiniCPM5 model/config mismatch: CPU runner supports only the official MiniCPM5-1B runtime contract");
+        "MiniCPM5 model/config mismatch: CPU runner supports only the official MiniCPM5-1B and MiniCPM5-2B runtime contracts");
   }
   if (parameter_file == nullptr || !parameter_file->has(kEmbeddingWeight) || !parameter_file->has(kLmHeadWeight)) {
     throw std::invalid_argument("MiniCPM5 model/config mismatch: checkpoint requires model.embed_tokens.weight and "
@@ -132,7 +138,7 @@ inline void validateModelConfigMatch(const MiniCPM5Config& config, const Paramet
 
   const auto shape = embedding.shape();
   if (shape.size() != 2 || shape[0] != config.vocab_size || shape[1] != config.hidden_size) {
-    throw std::invalid_argument("MiniCPM5 model/config mismatch: embedding shape must be [130560, 1536]");
+    throw std::invalid_argument("MiniCPM5 model/config mismatch: embedding shape must match [vocab_size, hidden_size]");
   }
 }
 

diff --git a/tests/models/minicpm5/MiniCPM5ConfigTest.cpp b/tests/models/minicpm5/MiniCPM5ConfigTest.cpp
@@ -16,8 +16,8 @@ auto exampleDir() -> std::string {
   return example_dir_override == nullptr ? std::string(MINICPM5_EXAMPLE_DIR) : std::string(example_dir_override);
 }
 
-auto loadConfig() -> mllm::models::minicpm5::MiniCPM5Config {
-  return mllm::models::minicpm5::MiniCPM5Config(exampleDir() + "/config_1B_w4a32_kai.json");
+auto loadConfig(const std::string& variant = "1B") -> mllm::models::minicpm5::MiniCPM5Config {
+  return mllm::models::minicpm5::MiniCPM5Config(exampleDir() + "/config_" + variant + "_w4a32_kai.json");
 }
 
 auto parameterFile(mllm::ModelFileVersion version, const std::vector<int32_t>& embedding_shape, bool include_lm_head = true)
@@ -55,6 +55,42 @@ TEST(MiniCPM5ConfigTest, RejectsRuntimeContractDrift) {
   EXPECT_FALSE(mllm::models::minicpm5::matchesOfficialMiniCPM5_1BRuntimeContract(config));
 }
 
+TEST(MiniCPM5ConfigTest, SupportsTwoBillionParameterContractAndRejectsMixedVariants) {
+  const auto config = loadConfig("2B");
+  EXPECT_TRUE(mllm::models::minicpm5::matchesOfficialMiniCPM5RuntimeContract(config));
+  EXPECT_FALSE(mllm::models::minicpm5::matchesOfficialMiniCPM5_1BRuntimeContract(config));
+  EXPECT_EQ(config.hidden_size, 2048);
+  EXPECT_EQ(config.intermediate_size, 6144);
+  EXPECT_EQ(config.num_hidden_layers, 42);
+  EXPECT_EQ(config.num_attention_heads * config.head_dim, 2048);
+  EXPECT_EQ(config.num_key_value_heads * config.head_dim, 256);
+  for (int field = 0; field < 3; ++field) {
+    auto mixed = config;
+    if (field == 0) mixed.hidden_size = 1536;
+    if (field == 1) mixed.intermediate_size = 4608;
+    if (field == 2) mixed.num_hidden_layers = 24;
+    EXPECT_FALSE(mllm::models::minicpm5::matchesOfficialMiniCPM5RuntimeContract(mixed));
+  }
+  auto drift = config;
+  drift.tie_word_embeddings = true;
+  EXPECT_FALSE(mllm::models::minicpm5::matchesOfficialMiniCPM5RuntimeContract(drift));
+  drift = config;
+  drift.rope_theta = 10000;
+  EXPECT_FALSE(mllm::models::minicpm5::matchesOfficialMiniCPM5RuntimeContract(drift));
+}
+
+TEST(MiniCPM5ConfigTest, RejectsCrossVariantModelPairing) {
+  const auto one_b = loadConfig();
+  const auto two_b = loadConfig("2B");
+  for (const auto version : {mllm::ModelFileVersion::kV1, mllm::ModelFileVersion::kV2}) {
+    const auto parameters = parameterFile(version, {two_b.vocab_size, two_b.hidden_size});
+    EXPECT_NO_THROW(mllm::models::minicpm5::validateModelConfigMatch(two_b, parameters));
+    EXPECT_THROW(mllm::models::minicpm5::validateModelConfigMatch(one_b, parameters), std::invalid_argument);
+    const auto one_parameters = parameterFile(version, {one_b.vocab_size, one_b.hidden_size});
+    EXPECT_THROW(mllm::models::minicpm5::validateModelConfigMatch(two_b, one_parameters), std::invalid_argument);
+  }
+}
+
 TEST(MiniCPM5ConfigTest, ValidatesEmbeddingAndIndependentLmHead) {
   const auto config = loadConfig();
   const auto matching_v2 = parameterFile(mllm::ModelFileVersion::kV2, {config.vocab_size, config.hidden_size});

diff --git a/tests/models/minicpm5/MiniCPM5ModelTest.cpp b/tests/models/minicpm5/MiniCPM5ModelTest.cpp
@@ -18,8 +18,12 @@ auto exampleDir() -> std::string {
 
 }  // namespace
 
-TEST(MiniCPM5ModelTest, BuildsNativeKVHeadLogicalSlotsAndResetsThem) {
-  mllm::initializeContext();
+class MiniCPM5ModelTest : public ::testing::Test {
+ protected:
+  static void SetUpTestSuite() { mllm::initializeContext(); }
+};
+
+TEST_F(MiniCPM5ModelTest, BuildsNativeKVHeadLogicalSlotsAndResetsThem) {
   const auto config = mllm::models::minicpm5::MiniCPM5Config(exampleDir() + "/config_1B_w4a32_kai.json");
   auto model = mllm::models::minicpm5::MiniCPM5ForCausalLM(config);
   auto& cache = model.kvCache();
@@ -38,3 +42,17 @@ TEST(MiniCPM5ModelTest, BuildsNativeKVHeadLogicalSlotsAndResetsThem) {
   model.resetState();
   EXPECT_EQ(cache.getCurrentSeqCnt(7), 0);
 }
+
+TEST_F(MiniCPM5ModelTest, TwoBillionParameterModelResetsLastLogicalSlot) {
+  const auto config = mllm::models::minicpm5::MiniCPM5Config(exampleDir() + "/config_2B_w4a32_kai.json");
+  auto model = mllm::models::minicpm5::MiniCPM5ForCausalLM(config);
+  auto& cache = model.kvCache();
+  EXPECT_EQ(cache.getLayerNums(), 42);
+  EXPECT_EQ(cache.getKCacheBuffer(41).shape(), (mllm::Tensor::shape_t{1, 2, 2048, 128}));
+  auto key = mllm::Tensor::zeros({1, 2, 1, 128}, mllm::kFloat32, mllm::kCPU);
+  cache.updateKVCache(41, key, key);
+  EXPECT_EQ(cache.getCurrentSeqCnt(41), 1);
+  EXPECT_EQ(cache.getCurrentSeqCnt(0), 0);
+  model.resetState();
+  EXPECT_EQ(cache.getCurrentSeqCnt(41), 0);
+}