All versions · Version range: v0.4.2

Scheduler token budget

seed slicecovered at legacy evidence: v0.4.2legacy evidence: main-2026-07-21evidence: source code and documentation

Coverage for how vLLM represents and constrains max_num_batched_tokens in scheduler iteration token budgets.

Strong for the seeded v0.4.2 defaults and constraints; partial for latest-docs behavior and broader tuning.

This topic is intentionally scoped to promoted max_num_batched_tokens evidence, not all scheduler token-budget behavior.

Every declared topic entity has applicable claims, availability, or admitted history at this version.

7 claims0 history facts2/2 entities

Claim	Verified
The vLLM v0.4.2 performance docs state that 512 is the default `max_num_batched_tokens` when `enable_chunked_prefill=True`.	Sep 29
In vLLM v0.4.2 `SchedulerConfig`, when `max_num_batched_tokens` is unset and `enable_chunked_prefill=True`, code sets `max_num_batched_tokens` to 512.	Sep 29
In vLLM v0.4.2 `SchedulerConfig`, when `max_num_batched_tokens` is unset and `enable_chunked_prefill=False`, code sets `max_num_batched_tokens` to max(max_model_len, 2048).	Sep 29
In vLLM v0.4.2, `SchedulerConfig` raises an error when `max_num_batched_tokens` is smaller than `max_model_len` and `enable_chunked_prefill=False`.	Sep 29
The latest optimization docs snapshot recommends `max_num_batched_tokens` greater than 8192 for optimal throughput, especially for smaller models on large GPUs.	Aug 20
The latest optimization docs snapshot says V1 enables chunked prefill by default whenever possible and uses `max_num_batched_tokens` as the token budget for pending prefills.	Aug 20
The latest optimization docs snapshot warns that when `enable_chunked_prefill=False`, `max_num_batched_tokens` must be greater than `max_model_len` or vLLM may crash at startup.	Aug 20

Runtime observations for real workloads are not represented in this seed corpus.

Schema.ai can cite docs and code defaults, but should not claim measured throughput or latency outcomes.

Check: Add runtime benchmark or incident evidence before serving workload-specific performance conclusions.

Coverage is limited to v0.4.2 source/docs and a latest-docs snapshot.

Schema.ai should not infer behavior for unobserved intermediate or future releases.

Check: Promote additional tagged source snapshots before broadening version applicability.