MiniSeek-V1:单卡 RTX 3090 从零训练的 DeepSeek-V4.1-Flash 全架构超小语言模型(187M 参数 / 每 token 激活 33M)。A tiny DeepSeek-V4.1-Flash with the full layer stack, trained from scratch on one RTX 3090.
pytorch mhc language-model from-scratch mixture-of-experts single-gpu engram sparse-attention llm speculative-decoding deepseek tiny-llm deepseek-v4 causal-encoder-decoder
-
Updated
Nov 5, 2026 - Python