Prometheus /metrics endpoint

complete inventoryvLLM v0.23.045 inventoried members

Surface

Prometheus-compatible metrics surface exposed by vLLM.

The inventory is complete at this version: a member name not listed below does not exist on this surface — a truthful absence, not a coverage gap.

Inventoried members (45)

All members require: PrometheusStatLogger metrics publisher

Member	Description	Registration gates	Source
cache_config_info	`vllm:cache_config_info` is registered as a Prometheus gauge in vLLM v0.23.0.		github.com
corrupted_requests	`vllm:corrupted_requests` is registered as a Prometheus counter in vLLM v0.23.0. Registration requires `VLLM_COMPUTE_NANS_IN_LOGITS=1`.	`VLLM_COMPUTE_NANS_IN_LOGITS=1`	github.com
e2e_request_latency_seconds	`vllm:e2e_request_latency_seconds` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
engine_sleep_state	`vllm:engine_sleep_state` is registered as a Prometheus gauge in vLLM v0.23.0.		github.com
estimated_flops_per_gpu_total	`vllm:estimated_flops_per_gpu_total` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
estimated_read_bytes_per_gpu_total	`vllm:estimated_read_bytes_per_gpu_total` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
estimated_write_bytes_per_gpu_total	`vllm:estimated_write_bytes_per_gpu_total` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
external_prefix_cache_hits	`vllm:external_prefix_cache_hits` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
external_prefix_cache_queries	`vllm:external_prefix_cache_queries` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
generation_tokens	`vllm:generation_tokens` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
inter_token_latency_seconds	`vllm:inter_token_latency_seconds` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
iteration_tokens_total	`vllm:iteration_tokens_total` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
kv_block_idle_before_evict_seconds	`vllm:kv_block_idle_before_evict_seconds` is registered as a Prometheus histogram in vLLM v0.23.0. Registration requires observability_config.kv_cache_metrics=True.	observability_config.kv_cache_metrics=True	github.com
kv_block_lifetime_seconds	`vllm:kv_block_lifetime_seconds` is registered as a Prometheus histogram in vLLM v0.23.0. Registration requires observability_config.kv_cache_metrics=True.	observability_config.kv_cache_metrics=True	github.com
kv_block_reuse_gap_seconds	`vllm:kv_block_reuse_gap_seconds` is registered as a Prometheus histogram in vLLM v0.23.0. Registration requires observability_config.kv_cache_metrics=True.	observability_config.kv_cache_metrics=True	github.com
kv_cache_usage_perc	`vllm:kv_cache_usage_perc` is registered as a Prometheus gauge in vLLM v0.23.0.		github.com
lora_requests_info	`vllm:lora_requests_info` is registered as a Prometheus gauge in vLLM v0.23.0. Registration requires `lora_config` is not None.	`lora_config` is not None	github.com
mm_cache_hits	`vllm:mm_cache_hits` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
mm_cache_queries	`vllm:mm_cache_queries` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
num_preemptions	`vllm:num_preemptions` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
num_requests_running	`vllm:num_requests_running` is registered as a Prometheus gauge in vLLM v0.23.0.		github.com
num_requests_waiting	`vllm:num_requests_waiting` is registered as a Prometheus gauge in vLLM v0.23.0.		github.com
num_requests_waiting_by_reason	`vllm:num_requests_waiting_by_reason` is registered as a Prometheus gauge with a reason label in vLLM v0.23.0.		github.com
prefix_cache_hits	`vllm:prefix_cache_hits` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
prefix_cache_queries	`vllm:prefix_cache_queries` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
prompt_tokens	`vllm:prompt_tokens` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
prompt_tokens_by_source	`vllm:prompt_tokens_by_source` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
prompt_tokens_cached	`vllm:prompt_tokens_cached` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
request_decode_time_seconds	`vllm:request_decode_time_seconds` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_generation_tokens	`vllm:request_generation_tokens` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_inference_time_seconds	`vllm:request_inference_time_seconds` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_max_num_generation_tokens	`vllm:request_max_num_generation_tokens` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_params_max_tokens	`vllm:request_params_max_tokens` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_params_n	`vllm:request_params_n` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_prefill_kv_computed_tokens	`vllm:request_prefill_kv_computed_tokens` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_prefill_time_seconds	`vllm:request_prefill_time_seconds` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_prompt_tokens	`vllm:request_prompt_tokens` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_queue_time_seconds	`vllm:request_queue_time_seconds` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
request_success	`vllm:request_success` is registered as a Prometheus counter in vLLM v0.23.0.		github.com
request_time_per_output_token_seconds	`vllm:request_time_per_output_token_seconds` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com
spec_decode_num_accepted_tokens	`vllm:spec_decode_num_accepted_tokens` is registered as a Prometheus counter in vLLM v0.23.0. Registration requires `speculative_config` is not None.	`speculative_config` is not None	github.com
spec_decode_num_accepted_tokens_per_pos	`vllm:spec_decode_num_accepted_tokens_per_pos` is registered as a Prometheus counter in vLLM v0.23.0. Registration requires `speculative_config` is not None.	`speculative_config` is not None	github.com
spec_decode_num_draft_tokens	`vllm:spec_decode_num_draft_tokens` is registered as a Prometheus counter in vLLM v0.23.0. Registration requires `speculative_config` is not None.	`speculative_config` is not None	github.com
spec_decode_num_drafts	`vllm:spec_decode_num_drafts` is registered as a Prometheus counter in vLLM v0.23.0. Registration requires `speculative_config` is not None.	`speculative_config` is not None	github.com
time_to_first_token_seconds	`vllm:time_to_first_token_seconds` is registered as a Prometheus histogram in vLLM v0.23.0.		github.com

Covered by topics

Scheduler and KV pressure metrics