Schema.ai· vLLM
vLLM official site Log in
vLLM v0.23.0 · Version scoped

Settings

20 entities matching "source.vllm.code.config_scheduler.v0230"

NameDescriptionVersionVerifiedSources
max_num_batched_tokensMaximum number of tokens to process in a scheduler iteration.v0.23.0Jul 9config/scheduler.py
max_num_seqsMaximum number of sequences to process in a scheduler iteration.v0.23.0Jul 9config/scheduler.py
max_model_lenMaximum sequence length, including prompt and output tokens, used by vLLM configuration and scheduler validation.v0.23.0Jul 9config/scheduler.py
is_encoder_decoderSchedulerConfig InitVar marking encoder-decoder models; stored in ModelConfig and used to disable chunked prefill and prefix caching.v0.23.0Jul 9config/scheduler.py
runner_typeSchedulerConfig field selecting the runner type to launch for the model: generate, pooling, or draft.v0.23.0Jul 9config/scheduler.py
max_num_scheduled_tokensSchedulerConfig field capping the tokens the scheduler may issue in a single iteration; defaults to max_num_batched_tokens.v0.23.0Jul 9config/scheduler.py
max_num_partial_prefillsSchedulerConfig field for the maximum number of sequences that can be partially prefilled concurrently under chunked prefill.v0.23.0Jul 9config/scheduler.py
max_long_partial_prefillsSchedulerConfig field for the maximum number of prompts longer than long_prefill_token_threshold prefilled concurrently under chunked prefill.v0.23.0Jul 9config/scheduler.py
long_prefill_token_thresholdSchedulerConfig field defining when a chunked-prefill request counts as long, in prompt tokens.v0.23.0Jul 9config/scheduler.py
enable_chunked_prefillSchedulerConfig flag enabling chunking of prefill requests based on the remaining max_num_batched_tokens budget.v0.23.0Jul 9config/scheduler.py
is_multimodal_modelSchedulerConfig flag marking the model as multimodal.v0.23.0Jul 9config/scheduler.py
max_num_encoder_input_tokensSchedulerConfig derived field for the multimodal encoder compute budget; init=False and overridden from max_num_batched_tokens.v0.23.0Jul 9config/scheduler.py
encoder_cache_sizeSchedulerConfig derived field for the multimodal encoder cache size; init=False and overridden from max_num_batched_tokens.v0.23.0Jul 9config/scheduler.py
policySchedulerConfig field selecting the scheduling policy: fcfs or priority.v0.23.0Jul 9config/scheduler.py
disable_chunked_mm_inputSchedulerConfig flag preventing partial scheduling of a multimodal item under chunked prefill.v0.23.0Jul 9config/scheduler.py
scheduler_clsSchedulerConfig field selecting the scheduler class, as a class or an import path string.v0.23.0Jul 9config/scheduler.py
disable_hybrid_kv_cache_managerSchedulerConfig flag forcing the KV cache manager to allocate the same KV cache size for all attention layer types.v0.23.0Jul 9config/scheduler.py
scheduler_reserve_full_islSchedulerConfig flag making the scheduler check that the full input sequence length fits in the KV cache before admitting a request.v0.23.0Jul 9config/scheduler.py
async_schedulingSchedulerConfig flag controlling async scheduling, which helps avoid gaps in GPU utilization.v0.23.0Jul 9config/scheduler.py
stream_intervalSchedulerConfig field for the streaming interval or buffer size in token lengths.v0.23.0Jul 9config/scheduler.py