Skip to content

Bug: AssertionError: Not enough memory for KV cache when serving Qwen3.8-27B-NVFP4 on Windows #561

Open
Open
Bug: AssertionError: Not enough memory for KV cache when serving Qwen3.8-27B-NVFP4 on Windows#561
Labels
bugSomething isn't workingwindows

Description

@hulonghe
hulonghe
opened on Sep 28, 2026

Before you start

  • I have read the FAQ and my problem is not answered there.
  • I have read the Roadmap and this is not already planned there.
  • I have searched existing issues and found no duplicate.
  • I am on the latest release, or on a freshly rebuilt main when building from source.

What happened

Image When starting the FreeToken server (ft serve) with the Qwen3.8-27B-NVFP4 model on Windows, the server crashes during initialization with the following error: AssertionError: Not enough memory for KV cache, try reducing --num-pages The server fails to start the scheduler process and exits with code 15. This happens consistently across multiple restart attempts with different --memory-ratio values (0.85, 0.92, 0.93). Interestingly, the server was able to run successfully for ~1.5 hours on the first launch after a fresh boot, but crashed immediately after restart without any changes to configuration.

How did you install FreeToken

pip / uv wheel

FreeToken version

v0.2.0-beta.22

OS

Windows 11

OS details

Windows 11 专业版

GPU and driver

RTX 5090 Dv2 24GB RAM

CPU and system RAM

32Gb RAM,Intel Core i7-14700K

Checkpoint

Qwen/Qwen3.8-27B-NVFP4

Command

Launch the GUI after downloading

Full log

[16:44:40] stdout/INFO  [2026-09-24|16:44:40|core|rank=0] INFO     Prefill batch, #new-seq: 1, #new-token: 6538, #cached-token: 69184, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, #running-req: 1, #queue-req: 0, input throughput (token/s): 40.20
[16:44:40] stdout/INFO  [2026-09-24|16:44:40|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 75739, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 13.03, #queue-req: 0
[16:44:41] stdout/INFO  [2026-09-24|16:44:41|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 75779, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 42.70, #queue-req: 0
[16:44:42] stdout/INFO  [2026-09-24|16:44:42|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 75819, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 41.25, #queue-req: 0
[16:44:43] stdout/INFO  [2026-09-24|16:44:43|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 75859, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 42.17, #queue-req: 0
[16:44:44] stdout/INFO  [2026-09-24|16:44:44|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 75899, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 42.29, #queue-req: 0
[16:44:45] stdout/INFO  [2026-09-24|16:44:45|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 75939, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 42.33, #queue-req: 0
[16:44:46] stdout/INFO  [2026-09-24|16:44:46|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 75979, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 42.67, #queue-req: 0
[16:44:47] stdout/INFO  [2026-09-24|16:44:47|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 76019, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 43.15, #queue-req: 0
[16:44:48] stdout/INFO  [2026-09-24|16:44:48|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 76059, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 43.68, #queue-req: 0
[16:44:49] stdout/INFO  [2026-09-24|16:44:49|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 76099, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 39.72, #queue-req: 0
[16:44:50] stdout/INFO  [2026-09-24|16:44:50|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 76139, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 42.39, #queue-req: 0
[16:44:51] stdout/INFO  [2026-09-24|16:44:51|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 76179, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 40.43, #queue-req: 0
[16:44:52] stdout/INFO  [2026-09-24|16:44:52|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 76219, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 41.35, #queue-req: 0
[16:44:52] stdout/INFO  INFO:     127.0.0.1:55288 - "POST /v1/chat/completions HTTP/1.1" 200 OK
[16:44:52] stdout/INFO  [2026-09-24|16:44:52|core|rank=0] INFO     Scheduler is idle, waiting for new reqs...
[16:44:53] stdout/INFO  [2026-09-24|16:44:53|core|rank=0] INFO     Prefill batch, #new-seq: 1, #new-token: 675, #cached-token: 75712, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, #running-req: 1, #queue-req: 0, input throughput (token/s): 50.22
[16:44:54] stdout/INFO  [2026-09-24|16:44:53|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 76401, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 20.60, #queue-req: 0
[16:44:55] stdout/INFO  [2026-09-24|16:44:55|core|rank=0] INFO     Decode batch, #running-req: 1, #token: 76441, token usage: 0.13, #mamba-slot: 4/24, mamba usage: 0.17, gen throughput (token/s): 33.78, #queue-req: 0
[16:44:55] stdout/INFO  [2026-09-24|16:44:55|core|rank=0] INFO     Scheduler is idle, waiting for new reqs...
[22:26:02] stdout/INFO  INFO:     127.0.0.1:54821 - "POST /v1/admin/prepare-stop HTTP/1.1" 200 OK
[22:26:02] stdout/INFO  serve stopped (pid=13048)
[22:26:08] cmd/INFO  ft serve --model D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 --port 1919 --max-running-requests 4 --memory-ratio 0.85 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420 --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[22:26:09] stdout/INFO  serve started (pid=23076 model=D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 port=1919)
[22:26:34] stdout/INFO  [2026-09-24|22:26:34] INFO     Parsed arguments:
[22:26:34] stdout/INFO  ServerArgs(model_path='D:\\resources\\.freetoken\\models\\Qwen3.8-27B-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.85, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=2896', _ipc_base_port=60381, server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Qwen3.8-27B-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=('GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576',), gpu_assigned=None)
[22:26:34] stdout/INFO  [2026-09-24|22:26:34|initializer] INFO     --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576 -> GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[22:26:34] stdout/INFO  [2026-09-24|22:26:34|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[22:26:34] stdout/INFO  INFO:     Started server process [2896]
[22:26:34] stdout/INFO  INFO:     Waiting for application startup.
[22:26:34] stdout/INFO  INFO:     Application startup complete.
[22:26:34] stdout/INFO  INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
[22:26:37] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py:81: FutureWarning: torch.cuda._set_allocator_settings is deprecated. Use torch._C._accelerator_setAllocatorSettings instead.
[22:26:37] stdout/INFO    scheduler = Scheduler(args)
[22:26:37] stdout/INFO  [2026-09-24|22:26:37|core|rank=0] INFO     Enabled expandable_segments (override via PYTORCH_ALLOC_CONF)
[22:26:37] stdout/INFO  [2026-09-24|22:26:37|core|rank=0] INFO     Auto-selected attention backend: triton
[22:26:37] stdout/INFO  [2026-09-24|22:26:37|core|rank=0] INFO     Resolved config: attention_backend='triton', cache_type='hybrid_radix', page_size=1
[22:26:37] stdout/INFO  [2026-09-24|22:26:37|core|rank=0] INFO     Free memory before loading model: 22.29 GiB
[22:26:38] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\torch\utils\_device.py:116: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:39.)
[22:26:38] stdout/INFO    return func(*args, **kwargs)
[22:34:11] stdout/INFO  Loading weights: 100%|██████████| 3/3 [07:24<00:00, 148.02s/it]
[22:34:16] stdout/INFO  [2026-09-24|22:34:16|core|rank=0] INFO     Multimodal enabled: QwenVLMMProcessor, encoders ['vision'] on host, serving ['image']
[22:34:18] stdout/ERROR  [2026-09-24|22:34:18|FrontendAPI] ERROR    Backend supervisor: AssertionError: Not enough memory for KV cache, try reducing --num-pages
[22:34:18] stdout/INFO  Process freetoken-TP0-scheduler:
[22:34:18] stdout/INFO  Traceback (most recent call last):
[22:34:18] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 314, in _bootstrap
[22:34:18] stdout/INFO      self.run()
[22:34:18] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 108, in run
[22:34:18] stdout/INFO      self._target(*self._args, **self._kwargs)
[22:34:18] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 81, in _run_scheduler
[22:34:18] stdout/INFO      scheduler = Scheduler(args)
[22:34:18] stdout/INFO                  ^^^^^^^^^^^^^^^
[22:34:18] stdout/INFO    File "python/freetoken/scheduler/scheduler.py", line 68, in freetoken.scheduler.scheduler.Scheduler.__init__
[22:34:18] stdout/INFO    File "python/freetoken/engine/engine.py", line 479, in freetoken.engine.engine.Engine.__init__
[22:34:18] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\kvcache\base.py", line 73, in solve_num_pages
[22:34:18] stdout/INFO      assert num_pages > 1, "Not enough memory for KV cache, try reducing --num-pages"
[22:34:18] stdout/INFO             ^^^^^^^^^^^^^
[22:34:18] stdout/INFO  AssertionError: Not enough memory for KV cache, try reducing --num-pages
[22:34:27] health/ERROR  AssertionError: Not enough memory for KV cache, try reducing --num-pages  (×5)
[22:34:28] stdout/ERROR  [2026-09-24|22:34:28|FrontendAPI] ERROR    Backend worker is gone and cannot be restarted; stopping the API server
[22:34:28] stdout/INFO  serve exited with code 15 (pid=23076)
[09:32:51] cmd/INFO  ft serve --model D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 --port 1919 --max-running-requests 4 --memory-ratio 0.85 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420 --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[09:32:51] stdout/INFO  serve started (pid=22004 model=D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 port=1919)
[09:33:15] stdout/INFO  [2026-09-28|09:33:15] INFO     Parsed arguments:
[09:33:15] stdout/INFO  ServerArgs(model_path='D:\\resources\\.freetoken\\models\\Qwen3.8-27B-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.85, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=17536', _ipc_base_port=58912, server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Qwen3.8-27B-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=('GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576',), gpu_assigned=None)
[09:33:15] stdout/INFO  [2026-09-28|09:33:15|initializer] INFO     --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576 -> GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[09:33:15] stdout/INFO  [2026-09-28|09:33:15|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[09:33:15] stdout/INFO  INFO:     Started server process [17536]
[09:33:15] stdout/INFO  INFO:     Waiting for application startup.
[09:33:15] stdout/INFO  INFO:     Application startup complete.
[09:33:15] stdout/INFO  INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
[09:33:18] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py:81: FutureWarning: torch.cuda._set_allocator_settings is deprecated. Use torch._C._accelerator_setAllocatorSettings instead.
[09:33:18] stdout/INFO    scheduler = Scheduler(args)
[09:33:18] stdout/INFO  [2026-09-28|09:33:18|core|rank=0] INFO     Enabled expandable_segments (override via PYTORCH_ALLOC_CONF)
[09:33:18] stdout/INFO  [2026-09-28|09:33:18|core|rank=0] INFO     Auto-selected attention backend: triton
[09:33:18] stdout/INFO  [2026-09-28|09:33:18|core|rank=0] INFO     Resolved config: attention_backend='triton', cache_type='hybrid_radix', page_size=1
[09:33:19] stdout/INFO  [2026-09-28|09:33:19|core|rank=0] INFO     Free memory before loading model: 22.29 GiB
[09:33:19] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\torch\utils\_device.py:116: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:39.)
[09:33:19] stdout/INFO    return func(*args, **kwargs)
[09:33:53] stdout/INFO  Loading weights: 100%|██████████| 3/3 [00:33<00:00, 11.26s/it]
[09:33:53] stdout/INFO  [2026-09-28|09:33:53|core|rank=0] INFO     Multimodal enabled: QwenVLMMProcessor, encoders ['vision'] on host, serving ['image']
[09:34:23] stdout/ERROR  [2026-09-28|09:34:23|FrontendAPI] ERROR    Backend supervisor: AssertionError: Not enough memory for KV cache, try reducing --num-pages
[09:34:23] stdout/INFO  Process freetoken-TP0-scheduler:
[09:34:23] stdout/INFO  Traceback (most recent call last):
[09:34:23] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 314, in _bootstrap
[09:34:23] stdout/INFO      self.run()
[09:34:23] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 108, in run
[09:34:23] stdout/INFO      self._target(*self._args, **self._kwargs)
[09:34:23] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 81, in _run_scheduler
[09:34:23] stdout/INFO      scheduler = Scheduler(args)
[09:34:23] stdout/INFO                  ^^^^^^^^^^^^^^^
[09:34:23] stdout/INFO    File "python/freetoken/scheduler/scheduler.py", line 68, in freetoken.scheduler.scheduler.Scheduler.__init__
[09:34:23] stdout/INFO    File "python/freetoken/engine/engine.py", line 479, in freetoken.engine.engine.Engine.__init__
[09:34:23] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\kvcache\base.py", line 73, in solve_num_pages
[09:34:23] stdout/INFO      assert num_pages > 1, "Not enough memory for KV cache, try reducing --num-pages"
[09:34:23] stdout/INFO             ^^^^^^^^^^^^^
[09:34:23] stdout/INFO  AssertionError: Not enough memory for KV cache, try reducing --num-pages
[09:34:33] health/ERROR  AssertionError: Not enough memory for KV cache, try reducing --num-pages  (×5)
[09:34:33] stdout/ERROR  [2026-09-28|09:34:33|FrontendAPI] ERROR    Backend worker is gone and cannot be restarted; stopping the API server
[09:34:33] stdout/INFO  serve exited with code 15 (pid=22004)
[09:47:56] cmd/INFO  ft serve --model D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 --port 1919 --max-running-requests 4 --memory-ratio 0.85 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420 --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[09:47:56] stdout/INFO  serve started (pid=1156 model=D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 port=1919)
[09:48:00] stdout/INFO  [2026-09-28|09:48:00] INFO     Parsed arguments:
[09:48:00] stdout/INFO  ServerArgs(model_path='D:\\resources\\.freetoken\\models\\Qwen3.8-27B-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.85, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=10876', _ipc_base_port=58612, server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Qwen3.8-27B-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=('GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576',), gpu_assigned=None)
[09:48:00] stdout/INFO  [2026-09-28|09:48:00|initializer] INFO     --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576 -> GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[09:48:00] stdout/INFO  [2026-09-28|09:48:00|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[09:48:00] stdout/INFO  INFO:     Started server process [10876]
[09:48:00] stdout/INFO  INFO:     Waiting for application startup.
[09:48:00] stdout/INFO  INFO:     Application startup complete.
[09:48:00] stdout/INFO  INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
[09:48:03] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py:81: FutureWarning: torch.cuda._set_allocator_settings is deprecated. Use torch._C._accelerator_setAllocatorSettings instead.
[09:48:03] stdout/INFO    scheduler = Scheduler(args)
[09:48:03] stdout/INFO  [2026-09-28|09:48:03|core|rank=0] INFO     Enabled expandable_segments (override via PYTORCH_ALLOC_CONF)
[09:48:03] stdout/INFO  [2026-09-28|09:48:03|core|rank=0] INFO     Auto-selected attention backend: triton
[09:48:03] stdout/INFO  [2026-09-28|09:48:03|core|rank=0] INFO     Resolved config: attention_backend='triton', cache_type='hybrid_radix', page_size=1
[09:48:03] stdout/INFO  [2026-09-28|09:48:03|core|rank=0] INFO     Free memory before loading model: 22.29 GiB
[09:48:03] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\torch\utils\_device.py:116: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:39.)
[09:48:03] stdout/INFO    return func(*args, **kwargs)
[09:48:35] stdout/INFO  Loading weights: 100%|██████████| 3/3 [00:31<00:00, 10.40s/it]
[09:48:35] stdout/INFO  [2026-09-28|09:48:35|core|rank=0] INFO     Multimodal enabled: QwenVLMMProcessor, encoders ['vision'] on host, serving ['image']
[09:48:53] stdout/INFO  Process freetoken-TP0-scheduler:
[09:48:53] stdout/ERROR  [2026-09-28|09:48:53|FrontendAPI] ERROR    Backend supervisor: AssertionError: Not enough memory for KV cache, try reducing --num-pages
[09:48:53] stdout/INFO  Traceback (most recent call last):
[09:48:53] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 314, in _bootstrap
[09:48:53] stdout/INFO      self.run()
[09:48:53] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 108, in run
[09:48:53] stdout/INFO      self._target(*self._args, **self._kwargs)
[09:48:53] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 81, in _run_scheduler
[09:48:53] stdout/INFO      scheduler = Scheduler(args)
[09:48:53] stdout/INFO                  ^^^^^^^^^^^^^^^
[09:48:53] stdout/INFO    File "python/freetoken/scheduler/scheduler.py", line 68, in freetoken.scheduler.scheduler.Scheduler.__init__
[09:48:53] stdout/INFO    File "python/freetoken/engine/engine.py", line 479, in freetoken.engine.engine.Engine.__init__
[09:48:53] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\kvcache\base.py", line 73, in solve_num_pages
[09:48:53] stdout/INFO      assert num_pages > 1, "Not enough memory for KV cache, try reducing --num-pages"
[09:48:53] stdout/INFO             ^^^^^^^^^^^^^
[09:48:53] stdout/INFO  AssertionError: Not enough memory for KV cache, try reducing --num-pages
[09:49:03] health/ERROR  AssertionError: Not enough memory for KV cache, try reducing --num-pages  (×5)
[09:49:03] stdout/ERROR  [2026-09-28|09:49:03|FrontendAPI] ERROR    Backend worker is gone and cannot be restarted; stopping the API server
[09:49:03] stdout/INFO  serve exited with code 15 (pid=1156)
[10:37:21] cmd/INFO  ft serve --model D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 --port 1919 --max-running-requests 4 --memory-ratio 0.85 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420 --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[10:37:21] stdout/INFO  serve started (pid=9544 model=D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 port=1919)
[10:37:46] stdout/INFO  [2026-09-28|10:37:46] INFO     Parsed arguments:
[10:37:46] stdout/INFO  ServerArgs(model_path='D:\\resources\\.freetoken\\models\\Qwen3.8-27B-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.85, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=2712', _ipc_base_port=49636, server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Qwen3.8-27B-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=('GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576',), gpu_assigned=None)
[10:37:46] stdout/INFO  [2026-09-28|10:37:46|initializer] INFO     --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576 -> GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[10:37:46] stdout/INFO  [2026-09-28|10:37:46|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[10:37:46] stdout/INFO  INFO:     Started server process [2712]
[10:37:46] stdout/INFO  INFO:     Waiting for application startup.
[10:37:46] stdout/INFO  INFO:     Application startup complete.
[10:37:46] stdout/INFO  INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
[10:37:46] cmd/INFO  ft serve --model D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 --port 1919 --max-running-requests 4 --memory-ratio 0.92 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420 --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[10:37:46] stdout/INFO  serve started (pid=4080 model=D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 port=1919)
[10:37:47] stdout/INFO  serve stopped (pid=9544)
[10:37:50] stdout/INFO  [2026-09-28|10:37:50] INFO     Parsed arguments:
[10:37:50] stdout/INFO  ServerArgs(model_path='D:\\resources\\.freetoken\\models\\Qwen3.8-27B-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.92, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=22696', _ipc_base_port=64672, server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Qwen3.8-27B-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=('GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576',), gpu_assigned=None)
[10:37:50] stdout/INFO  [2026-09-28|10:37:50|initializer] INFO     --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576 -> GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[10:37:50] stdout/INFO  [2026-09-28|10:37:50|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[10:37:51] stdout/INFO  INFO:     Started server process [22696]
[10:37:51] stdout/INFO  INFO:     Waiting for application startup.
[10:37:51] stdout/INFO  INFO:     Application startup complete.
[10:37:51] stdout/INFO  INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
[10:37:51] stdout/INFO  INFO:     127.0.0.1:65501 - "POST /v1/admin/prepare-stop HTTP/1.1" 200 OK
[10:37:51] stdout/INFO  serve stopped (pid=4080)
[10:38:13] cmd/INFO  ft serve --model D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 --port 1919 --max-running-requests 4 --memory-ratio 0.93 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420 --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[10:38:13] stdout/INFO  serve started (pid=22236 model=D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 port=1919)
[10:38:17] stdout/INFO  [2026-09-28|10:38:17] INFO     Parsed arguments:
[10:38:17] stdout/INFO  ServerArgs(model_path='D:\\resources\\.freetoken\\models\\Qwen3.8-27B-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.93, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=616', _ipc_base_port=54258, server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Qwen3.8-27B-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=('GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576',), gpu_assigned=None)
[10:38:17] stdout/INFO  [2026-09-28|10:38:17|initializer] INFO     --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576 -> GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[10:38:17] stdout/INFO  [2026-09-28|10:38:17|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[10:38:17] stdout/INFO  INFO:     Started server process [616]
[10:38:17] stdout/INFO  INFO:     Waiting for application startup.
[10:38:17] stdout/INFO  INFO:     Application startup complete.
[10:38:17] stdout/INFO  INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
[10:38:20] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py:81: FutureWarning: torch.cuda._set_allocator_settings is deprecated. Use torch._C._accelerator_setAllocatorSettings instead.
[10:38:20] stdout/INFO    scheduler = Scheduler(args)
[10:38:20] stdout/INFO  [2026-09-28|10:38:20|core|rank=0] INFO     Enabled expandable_segments (override via PYTORCH_ALLOC_CONF)
[10:38:20] stdout/INFO  [2026-09-28|10:38:20|core|rank=0] INFO     Auto-selected attention backend: triton
[10:38:20] stdout/INFO  [2026-09-28|10:38:20|core|rank=0] INFO     Resolved config: attention_backend='triton', cache_type='hybrid_radix', page_size=1
[10:38:20] stdout/INFO  [2026-09-28|10:38:20|core|rank=0] INFO     Free memory before loading model: 22.29 GiB
[10:38:20] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\torch\utils\_device.py:116: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:39.)
[10:38:20] stdout/INFO    return func(*args, **kwargs)
[10:38:56] stdout/INFO  Loading weights: 100%|██████████| 3/3 [00:35<00:00, 11.92s/it]
[10:38:57] stdout/INFO  [2026-09-28|10:38:57|core|rank=0] INFO     Multimodal enabled: QwenVLMMProcessor, encoders ['vision'] on host, serving ['image']
[10:39:23] stdout/ERROR  [2026-09-28|10:39:23|FrontendAPI] ERROR    Backend supervisor: AssertionError: Not enough memory for KV cache, try reducing --num-pages
[10:39:23] stdout/INFO  Process freetoken-TP0-scheduler:
[10:39:23] stdout/INFO  Traceback (most recent call last):
[10:39:23] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 314, in _bootstrap
[10:39:23] stdout/INFO      self.run()
[10:39:23] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 108, in run
[10:39:23] stdout/INFO      self._target(*self._args, **self._kwargs)
[10:39:23] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 81, in _run_scheduler
[10:39:23] stdout/INFO      scheduler = Scheduler(args)
[10:39:23] stdout/INFO                  ^^^^^^^^^^^^^^^
[10:39:23] stdout/INFO    File "python/freetoken/scheduler/scheduler.py", line 68, in freetoken.scheduler.scheduler.Scheduler.__init__
[10:39:23] stdout/INFO    File "python/freetoken/engine/engine.py", line 479, in freetoken.engine.engine.Engine.__init__
[10:39:23] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\kvcache\base.py", line 73, in solve_num_pages
[10:39:23] stdout/INFO      assert num_pages > 1, "Not enough memory for KV cache, try reducing --num-pages"
[10:39:23] stdout/INFO             ^^^^^^^^^^^^^
[10:39:23] stdout/INFO  AssertionError: Not enough memory for KV cache, try reducing --num-pages
[10:39:33] health/ERROR  AssertionError: Not enough memory for KV cache, try reducing --num-pages  (×5)
[10:39:33] stdout/ERROR  [2026-09-28|10:39:33|FrontendAPI] ERROR    Backend worker is gone and cannot be restarted; stopping the API server
[10:39:33] stdout/INFO  serve exited with code 15 (pid=22236)
[10:58:51] cmd/INFO  ft serve --model D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 --port 1919 --max-running-requests 4 --memory-ratio 0.93 --host 127.0.0.1 --cors-origins tauri://localhost,http://tauri.localhost,http://localhost:1420 --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[10:58:51] stdout/INFO  serve started (pid=13552 model=D:\resources\.freetoken\models\Qwen3.8-27B-NVFP4 port=1919)
[10:58:55] stdout/INFO  [2026-09-28|10:58:55] INFO     Parsed arguments:
[10:58:55] stdout/INFO  ServerArgs(model_path='D:\\resources\\.freetoken\\models\\Qwen3.8-27B-NVFP4', tp_info=DistributedInfo(rank=0, size=1), dtype=torch.bfloat16, max_running_req=4, attention_backend='auto', moe_strategy='auto', quant_backend=None, ple_backend='disk', expert_load='auto', moe_cache_size=0, moe_cache_rate=None, moe_cache_auto=False, kv_reserve_tokens=8192, moe_cache_policy='lru', moe_prefill_overlap=True, moe_prefill_hit_d2d=False, moe_collect_stats=False, moe_cpu_threads=0, moe_cpu_layers=None, moe_hybrid_max_fetch=-1, cuda_graph_bs=None, cuda_graph_max_bs=None, page_size=1, memory_ratio=0.93, linear_state_cache_ratio=2.0, swa_full_tokens_ratio=0.2, swa_num_pages_override=None, distributed_timeout=60.0, use_dummy_weight=False, use_pynccl=True, max_seq_len_override=None, num_page_override=None, num_token_override=None, mm=MultimodalConfig(disabled_encoders=frozenset(), embed_cache_device='cpu', encoder_weights='host', image_min_tokens=None, image_max_tokens=None, processor_kwargs={}), max_extend_tokens=8192, cache_type='radix', offline_mode=False, decode_log_interval=40, special_token_ckpt=False, _unique_suffix='.pid=16176', _ipc_base_port=55281, server_host='127.0.0.1', server_port=1919, num_tokenizer=0, silent_output=False, shell_mode=False, served_model_name='Qwen3.8-27B-NVFP4', tool_call_parser='qwen3_coder', reasoning_parser='qwen3', sampling_defaults='model', max_output_tokens=None, enable_cache_report=False, allowed_media_domains='', allowed_local_media_path='', cors_origins='tauri://localhost,http://tauri.localhost,http://localhost:1420', gpu=('GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576',), gpu_assigned=None)
[10:58:55] stdout/INFO  [2026-09-28|10:58:55|initializer] INFO     --gpu GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576 -> GPU-d16d9a22-7b09-5a7a-87a0-b66af2ea4576
[10:58:55] stdout/INFO  [2026-09-28|10:58:55|FrontendAPI] INFO     Default sampling config (source=model): temperature=1.0, top_k=20, top_p=0.95
[10:58:55] stdout/INFO  INFO:     Started server process [16176]
[10:58:55] stdout/INFO  INFO:     Waiting for application startup.
[10:58:55] stdout/INFO  INFO:     Application startup complete.
[10:58:55] stdout/INFO  INFO:     Uvicorn running on http://127.0.0.1:1919 (Press CTRL+C to quit)
[10:58:58] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py:81: FutureWarning: torch.cuda._set_allocator_settings is deprecated. Use torch._C._accelerator_setAllocatorSettings instead.
[10:58:58] stdout/INFO    scheduler = Scheduler(args)
[10:58:58] stdout/INFO  [2026-09-28|10:58:58|core|rank=0] INFO     Enabled expandable_segments (override via PYTORCH_ALLOC_CONF)
[10:58:58] stdout/INFO  [2026-09-28|10:58:58|core|rank=0] INFO     Auto-selected attention backend: triton
[10:58:58] stdout/INFO  [2026-09-28|10:58:58|core|rank=0] INFO     Resolved config: attention_backend='triton', cache_type='hybrid_radix', page_size=1
[10:58:58] stdout/INFO  [2026-09-28|10:58:58|core|rank=0] INFO     Free memory before loading model: 22.29 GiB
[10:58:58] stdout/INFO  C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\torch\utils\_device.py:116: UserWarning: expandable_segments not supported on this platform (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\pytorch\c10/cuda/CUDAAllocatorConfig.h:39.)
[10:58:58] stdout/INFO    return func(*args, **kwargs)
[10:59:38] stdout/INFO  Loading weights: 100%|██████████| 3/3 [00:39<00:00, 13.33s/it]
[10:59:39] stdout/INFO  [2026-09-28|10:59:38|core|rank=0] INFO     Multimodal enabled: QwenVLMMProcessor, encoders ['vision'] on host, serving ['image']
[10:59:56] stdout/ERROR  [2026-09-28|10:59:55|FrontendAPI] ERROR    Backend supervisor: AssertionError: Not enough memory for KV cache, try reducing --num-pages
[10:59:56] stdout/INFO  Process freetoken-TP0-scheduler:
[10:59:56] stdout/INFO  Traceback (most recent call last):
[10:59:56] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 314, in _bootstrap
[10:59:56] stdout/INFO      self.run()
[10:59:56] stdout/INFO    File "C:\Users\hello\AppData\Roaming\uv\python\cpython-3.12-windows-x86_64-none\Lib\multiprocessing\process.py", line 108, in run
[10:59:56] stdout/INFO      self._target(*self._args, **self._kwargs)
[10:59:56] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\server\launch.py", line 81, in _run_scheduler
[10:59:56] stdout/INFO      scheduler = Scheduler(args)
[10:59:56] stdout/INFO                  ^^^^^^^^^^^^^^^
[10:59:56] stdout/INFO    File "python/freetoken/scheduler/scheduler.py", line 68, in freetoken.scheduler.scheduler.Scheduler.__init__
[10:59:56] stdout/INFO    File "python/freetoken/engine/engine.py", line 479, in freetoken.engine.engine.Engine.__init__
[10:59:56] stdout/INFO    File "C:\Users\hello\AppData\Local\FreeToken\venv\Lib\site-packages\freetoken\kvcache\base.py", line 73, in solve_num_pages
[10:59:56] stdout/INFO      assert num_pages > 1, "Not enough memory for KV cache, try reducing --num-pages"
[10:59:56] stdout/INFO             ^^^^^^^^^^^^^
[10:59:56] stdout/INFO  AssertionError: Not enough memory for KV cache, try reducing --num-pages
[10:59:57] health/ERROR  AssertionError: Not enough memory for KV cache, try reducing --num-pages

Anything else

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingwindows

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions