Schema.ai· vLLM
vLLM official site Log in

Metrics

NameDescriptionTopicsUse casesProviderSupported versionsVerifiedSources
num_requests_runningGauge for the number of requests in model execution batches.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.pymetrics/stats.py
kv_cache_usage_percGauge for KV-cache usage where 1 means 100 percent usage.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.pymetrics/stats.py
prefix_cache_queriesCounter for prefix cache queries, in queried tokens.Prefix cache effectiveness metrics, Scheduler and KV pressure metrics +13Prometheusv0.7.3 – v0.29.012 hr agometrics/loggers.pymetrics/stats.py
prefix_cache_hitsCounter for prefix cache hits, in cached tokens.Prefix cache effectiveness metrics, Scheduler and KV pressure metrics +13Prometheusv0.7.3 – v0.29.012 hr agometrics/loggers.pymetrics/stats.pydesign/metrics.md
prompt_tokensCounter for the number of prefill tokens processed.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.py
prompt_tokens_cachedCounter for cached prompt tokens, local plus external.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.py
generation_tokensCounter for the number of generation tokens processed.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.py
request_successCounter for successfully processed requests, labeled by finished_reason.Scheduler and KV pressure metrics, vLLM metric identity2Prometheusv0.23.012 hr agometrics/loggers.py
iteration_tokens_totalHistogram of the number of tokens per engine step.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.py
time_to_first_token_secondsHistogram of time to first token in seconds.Scheduler and KV pressure metrics, vLLM metric identity2Prometheusv0.23.012 hr agometrics/loggers.pymetrics/stats.py
inter_token_latency_secondsHistogram of inter-token latency in seconds.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.pymetrics/stats.py
request_time_per_output_token_secondsHistogram of mean time per output token per request.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.pymetrics/stats.py
e2e_request_latency_secondsHistogram of end-to-end request latency in seconds.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.pymetrics/stats.py
request_queue_time_secondsHistogram of time spent in the WAITING phase per request.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.pymetrics/stats.py
request_prefill_kv_computed_tokensHistogram of new KV tokens computed during prefill, excluding cached tokens.Scheduler and KV pressure metrics, vLLM metric identity1Prometheusv0.23.012 hr agometrics/loggers.py