| max_num_batched_tokens | Maximum number of tokens to process in a scheduler iteration. | v0.23.0 | Jul 9 | github.com |
| max_num_seqs | Maximum number of sequences to process in a scheduler iteration. | v0.23.0 | Jul 9 | github.com |
| max_model_len | Maximum sequence length, including prompt and output tokens, used by vLLM configuration and scheduler validation. | v0.23.0 | Jul 9 | github.com |
| is_encoder_decoder | SchedulerConfig InitVar marking encoder-decoder models; stored in ModelConfig and used to disable chunked prefill and prefix caching. | v0.23.0 | Jul 9 | github.com |
| runner_type | SchedulerConfig field selecting the runner type to launch for the model: generate, pooling, or draft. | v0.23.0 | Jul 9 | github.com |
| max_num_scheduled_tokens | SchedulerConfig field capping the tokens the scheduler may issue in a single iteration; defaults to max_num_batched_tokens. | v0.23.0 | Jul 9 | github.com |
| max_num_partial_prefills | SchedulerConfig field for the maximum number of sequences that can be partially prefilled concurrently under chunked prefill. | v0.23.0 | Jul 9 | github.com |
| max_long_partial_prefills | SchedulerConfig field for the maximum number of prompts longer than long_prefill_token_threshold prefilled concurrently under chunked prefill. | v0.23.0 | Jul 9 | github.com |
| long_prefill_token_threshold | SchedulerConfig field defining when a chunked-prefill request counts as long, in prompt tokens. | v0.23.0 | Jul 9 | github.com |
| enable_chunked_prefill | SchedulerConfig flag enabling chunking of prefill requests based on the remaining max_num_batched_tokens budget. | v0.23.0 | Jul 9 | github.com |
| is_multimodal_model | SchedulerConfig flag marking the model as multimodal. | v0.23.0 | Jul 9 | github.com |
| max_num_encoder_input_tokens | SchedulerConfig derived field for the multimodal encoder compute budget; init=False and overridden from max_num_batched_tokens. | v0.23.0 | Jul 9 | github.com |
| encoder_cache_size | SchedulerConfig derived field for the multimodal encoder cache size; init=False and overridden from max_num_batched_tokens. | v0.23.0 | Jul 9 | github.com |
| policy | SchedulerConfig field selecting the scheduling policy: fcfs or priority. | v0.23.0 | Jul 9 | github.com |
| disable_chunked_mm_input | SchedulerConfig flag preventing partial scheduling of a multimodal item under chunked prefill. | v0.23.0 | Jul 9 | github.com |
| scheduler_cls | SchedulerConfig field selecting the scheduler class, as a class or an import path string. | v0.23.0 | Jul 9 | github.com |
| disable_hybrid_kv_cache_manager | SchedulerConfig flag forcing the KV cache manager to allocate the same KV cache size for all attention layer types. | v0.23.0 | Jul 9 | github.com |
| scheduler_reserve_full_isl | SchedulerConfig flag making the scheduler check that the full input sequence length fits in the KV cache before admitting a request. | v0.23.0 | Jul 9 | github.com |
| async_scheduling | SchedulerConfig flag controlling async scheduling, which helps avoid gaps in GPU utilization. | v0.23.0 | Jul 9 | github.com |
| stream_interval | SchedulerConfig field for the streaming interval or buffer size in token lengths. | v0.23.0 | Jul 9 | github.com |
Schema.ai· vLLM