| num_requests_running | Gauge for the number of requests in model execution batches. | Prometheus | v0.23.0 | 4 hr ago | github.com |
| num_requests_waiting | Gauge for the number of requests waiting to be processed. | Prometheus | v0.23.0 | 4 hr ago | github.com |
| num_requests_waiting_by_reason | Gauge that breaks waiting requests into capacity and deferred reasons. | Prometheus | v0.23.0 | 4 hr ago | github.com |
| kv_cache_usage_perc | Gauge for KV-cache usage where 1 means 100 percent usage. | Prometheus | v0.23.0 | 4 hr ago | github.com |
| num_preemptions | Counter for cumulative preemptions from the engine. | Prometheus | v0.23.0 | 4 hr ago | github.com |
| engine_sleep_state | Gauge for engine sleep state; sleep_state label values awake, weights_offloaded, and discard_all encode sleep levels. | Prometheus | v0.23.0 | Jun 25 | github.com |
| corrupted_requests | Counter for corrupted requests, measured as requests with NaNs in logits. | Prometheus | v0.23.0 | Jun 25 | github.com |
| prefix_cache_queries | Counter for prefix cache queries, in queried tokens. | Prometheus | v0.23.0 | 4 hr ago | github.com |
| prefix_cache_hits | Counter for prefix cache hits, in cached tokens. | Prometheus | v0.23.0 | 4 hr ago | github.com |
| external_prefix_cache_queries | Counter for external prefix cache queries from KV connector cross-instance cache sharing, in queried tokens. | Prometheus | v0.23.0 | Jun 25 | github.com |
| external_prefix_cache_hits | Counter for external prefix cache hits from KV connector cross-instance cache sharing, in cached tokens. | Prometheus | v0.23.0 | Jun 25 | github.com |
| mm_cache_queries | Counter for multi-modal cache queries, in queried items. | Prometheus | v0.23.0 | Jun 25 | github.com |
| mm_cache_hits | Counter for multi-modal cache hits, in cached items. | Prometheus | v0.23.0 | Jun 25 | github.com |
| prompt_tokens | Counter for the number of prefill tokens processed. | Prometheus | v0.23.0 | Jun 25 | github.com |
| prompt_tokens_by_source | Counter for prompt tokens broken down by a source label. | Prometheus | v0.23.0 | Jun 25 | github.com |
| prompt_tokens_cached | Counter for cached prompt tokens, local plus external. | Prometheus | v0.23.0 | Jun 25 | github.com |
| generation_tokens | Counter for the number of generation tokens processed. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_success | Counter for successfully processed requests, labeled by finished_reason. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_prompt_tokens | Histogram of prefill token counts per request. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_generation_tokens | Histogram of generation token counts per request. | Prometheus | v0.23.0 | Jun 25 | github.com |
| iteration_tokens_total | Histogram of the number of tokens per engine step. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_max_num_generation_tokens | Histogram of the maximum requested generation tokens per request. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_params_n | Histogram of the n request parameter. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_params_max_tokens | Histogram of the max_tokens request parameter. | Prometheus | v0.23.0 | Jun 25 | github.com |
| time_to_first_token_seconds | Histogram of time to first token in seconds. | Prometheus | v0.23.0 | Jun 25 | github.com |
| inter_token_latency_seconds | Histogram of inter-token latency in seconds. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_time_per_output_token_seconds | Histogram of mean time per output token per request. | Prometheus | v0.23.0 | Jun 25 | github.com |
| e2e_request_latency_seconds | Histogram of end-to-end request latency in seconds. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_queue_time_seconds | Histogram of time spent in the WAITING phase per request. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_inference_time_seconds | Histogram of time spent in the RUNNING phase per request. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_prefill_time_seconds | Histogram of time spent in the PREFILL phase per request. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_decode_time_seconds | Histogram of time spent in the DECODE phase per request. | Prometheus | v0.23.0 | Jun 25 | github.com |
| request_prefill_kv_computed_tokens | Histogram of new KV tokens computed during prefill, excluding cached tokens. | Prometheus | v0.23.0 | Jun 25 | github.com |
| kv_block_lifetime_seconds | Histogram of KV cache block lifetime from allocation to eviction; sampled metric. | Prometheus | v0.23.0 | Jun 25 | github.com |
| kv_block_idle_before_evict_seconds | Histogram of idle time before KV cache block eviction; sampled metric. | Prometheus | v0.23.0 | Jun 25 | github.com |
| kv_block_reuse_gap_seconds | Histogram of time gaps between consecutive KV cache block accesses; sampled metric. | Prometheus | v0.23.0 | Jun 25 | github.com |
| lora_requests_info | Gauge with running stats on LoRA requests, labeled by max_lora and running/waiting adapter lists. | Prometheus | v0.23.0 | Jun 25 | github.com |
| cache_config_info | Info-style gauge permanently set to 1 that carries LLMEngine CacheConfig fields as labels; registered at engine initialization. | Prometheus | v0.23.0 | Jun 25 | github.com |
| spec_decode_num_drafts | Counter for the number of speculative decoding drafts. | Prometheus | v0.23.0 | Jul 9 | github.com |
| spec_decode_num_draft_tokens | Counter for the number of speculative draft tokens. | Prometheus | v0.23.0 | Jul 9 | github.com |
| spec_decode_num_accepted_tokens | Counter for the number of accepted speculative tokens. | Prometheus | v0.23.0 | Jul 9 | github.com |
| spec_decode_num_accepted_tokens_per_pos | Counter for accepted speculative tokens per draft position, labeled by position. | Prometheus | v0.23.0 | Jul 9 | github.com |
| estimated_flops_per_gpu_total | Counter of estimated floating point operations per GPU, for Model FLOPs Utilization calculations. | Prometheus | v0.23.0 | Jul 9 | github.com |
| estimated_read_bytes_per_gpu_total | Counter of estimated bytes read from memory per GPU, for Model FLOPs Utilization calculations. | Prometheus | v0.23.0 | Jul 9 | github.com |
| estimated_write_bytes_per_gpu_total | Counter of estimated bytes written to memory per GPU, for Model FLOPs Utilization calculations. | Prometheus | v0.23.0 | Jul 9 | github.com |
Schema.ai· vLLM