All pull requests

Pull requests

• Tile query rows in quantized_scaled_dot_product_attention to bound long-context scores memory

  #1929·badvision opened Sep 27, 2026·

  ·

• MiMo-V2: fused prefill and decode attention for the 192/128 global layers

  #1927·aidiffuser opened Sep 27, 2026Contributor·

  ·

• Honour the left-padding mask in MiniCPM3 and Phixtral batches

  #1925·bhaskargurram-ai opened Sep 26, 2026·

  ·

• Extend _rewind_cache() for Non-trimmable KV Caches

  #1923·austin362667 opened Sep 25, 2026·

  ·

• Fused 4-bit matmul for batched decode

  #1922·pythongiant opened Sep 24, 2026·

  ·

• Fix GraniteMoeHybrid padded batch logprobs (#1908)

  #1921·aakashkardam opened Sep 24, 2026·

  ·
  1

• server: add --kv-preallocate-size for stable long-context serving on memory-constrained Apple Silicon

  #1920·Py528 opened Sep 24, 2026·

  ·

• Add Nemotron-Labs-Diffusion architecture support

  #1918·12ohan opened Sep 23, 2026·

  ·

• server: add tokenizer endpoint

  #1917·ykhrustalev opened Sep 23, 2026Contributor·

  ·
  1

• Fix MiniCPM3 attention masking for padded batches

  #1913·provoke210 opened Sep 22, 2026·

  ·
  1

• Fix unbounded live-buffer growth in BatchGenerator decode loop

  #1911·bizapen opened Sep 21, 2026·

  ·

• Fix Qwen3 Coder argument parsing when the top-level type is missing

  #1910·zmylol opened Sep 21, 2026·

  ·

• Context sharding

  #1905·danxn opened Sep 20, 2026·

  ·

• Fix incorrect assumption when parsing tool_calls[].function.arguments

  #1904·LxYuan0420 opened Sep 20, 2026Contributor·

  ·

• Fix Gemma2 attention bug

  #1903·ursk opened Sep 20, 2026·

  ·

• Add xing4_0 model support

  #1901·TokenAI-zer opened Sep 18, 2026·

  ·

• Add tool call parser for MiniCPM5

  #1899·dasashreeya opened Sep 17, 2026Contributor·

  ·
  1

• Add DeepSeek-V4.1 model

  new_model
  #1895·Goekdeniz-Guelmez opened Sep 16, 2026Contributor·

  ·
  1

• server: --allowed-models to restrict which model paths a request may load

  #1893·mrcushen-arch opened Sep 15, 2026·

  ·
  1

• Swallow missing-stream errors when BatchGenerator is torn down off its GPU thread

  #1890·gyanu2507 opened Sep 15, 2026Contributor·

  ·
  1

• Tolerate a null rope_scaling config when constructing PhiMoE models

  #1889·gyanu2507 opened Sep 15, 2026Contributor·

  ·
  1

• server: make exact prompt-cache hits generation-safe

  #1887·deyi2026 opened Sep 14, 2026·

  ·

• Chunkwise-parallel gated delta rule for training

  #1870·adityak74 opened Sep 10, 2026Contributor·

  ·
  3

• Report the matched stop sequence on GenerationBatch.Response

  #1869·mloiterman opened Sep 9, 2026Contributor·

  ·

• Parse gpt-oss's harmony channel format in non-streaming completions

  #1867·eejd opened Sep 9, 2026·

  ·
  1