The latest optimization docs snapshot says V1 enables chunked prefill by default whenever possible and uses max_num_batched_tokens as the token budget for pending prefills. | main-2026-07-21 | Behavior | chunked prefill decode-priority scheduling | Aug 20 | raw.githubusercontent.com |
The latest optimization docs snapshot recommends max_num_batched_tokens greater than 8192 for optimal throughput, especially for smaller models on large GPUs. | main-2026-07-21 | Setting | max_num_batched_tokens | Aug 20 | raw.githubusercontent.com |
The latest optimization docs snapshot warns that when enable_chunked_prefill=False, max_num_batched_tokens must be greater than max_model_len or vLLM may crash at startup. | main-2026-07-21 | Setting | max_num_batched_tokens | Aug 20 | raw.githubusercontent.com |
The latest optimization docs snapshot says that decreasing max_num_seqs can reduce concurrent requests in a batch and require less KV cache space when preemptions are frequent. | main-2026-07-21 | Setting | max_num_seqs | Aug 20 | raw.githubusercontent.com |
Schema.ai· vLLM