Settings

20 entities matching "source.vllm.code.config_scheduler.v0230"

Name	Description	Version	Verified	Sources
async_scheduling	SchedulerConfig flag controlling async scheduling, which helps avoid gaps in GPU utilization.	v0.23.0	Jul 9	github.com
disable_chunked_mm_input	SchedulerConfig flag preventing partial scheduling of a multimodal item under chunked prefill.	v0.23.0	Jul 9	github.com
disable_hybrid_kv_cache_manager	SchedulerConfig flag forcing the KV cache manager to allocate the same KV cache size for all attention layer types.	v0.23.0	Jul 9	github.com
enable_chunked_prefill	SchedulerConfig flag enabling chunking of prefill requests based on the remaining max_num_batched_tokens budget.	v0.23.0	Jul 9	github.com
encoder_cache_size	SchedulerConfig derived field for the multimodal encoder cache size; init=False and overridden from max_num_batched_tokens.	v0.23.0	Jul 9	github.com
is_encoder_decoder	SchedulerConfig InitVar marking encoder-decoder models; stored in ModelConfig and used to disable chunked prefill and prefix caching.	v0.23.0	Jul 9	github.com
is_multimodal_model	SchedulerConfig flag marking the model as multimodal.	v0.23.0	Jul 9	github.com
long_prefill_token_threshold	SchedulerConfig field defining when a chunked-prefill request counts as long, in prompt tokens.	v0.23.0	Jul 9	github.com
max_long_partial_prefills	SchedulerConfig field for the maximum number of prompts longer than long_prefill_token_threshold prefilled concurrently under chunked prefill.	v0.23.0	Jul 9	github.com
max_model_len	Maximum sequence length, including prompt and output tokens, used by vLLM configuration and scheduler validation.	v0.23.0	Jul 9	github.com
max_num_batched_tokens	Maximum number of tokens to process in a scheduler iteration.	v0.23.0	Jul 9	github.com
max_num_encoder_input_tokens	SchedulerConfig derived field for the multimodal encoder compute budget; init=False and overridden from max_num_batched_tokens.	v0.23.0	Jul 9	github.com
max_num_partial_prefills	SchedulerConfig field for the maximum number of sequences that can be partially prefilled concurrently under chunked prefill.	v0.23.0	Jul 9	github.com
max_num_scheduled_tokens	SchedulerConfig field capping the tokens the scheduler may issue in a single iteration; defaults to max_num_batched_tokens.	v0.23.0	Jul 9	github.com
max_num_seqs	Maximum number of sequences to process in a scheduler iteration.	v0.23.0	Jul 9	github.com
policy	SchedulerConfig field selecting the scheduling policy: fcfs or priority.	v0.23.0	Jul 9	github.com
runner_type	SchedulerConfig field selecting the runner type to launch for the model: generate, pooling, or draft.	v0.23.0	Jul 9	github.com
scheduler_cls	SchedulerConfig field selecting the scheduler class, as a class or an import path string.	v0.23.0	Jul 9	github.com
scheduler_reserve_full_isl	SchedulerConfig flag making the scheduler check that the full input sequence length fits in the KV cache before admitting a request.	v0.23.0	Jul 9	github.com
stream_interval	SchedulerConfig field for the streaming interval or buffer size in token lengths.	v0.23.0	Jul 9	github.com