Schema.ai· vLLM
vLLM v0.23.0API up

vLLM v0.23.0 v1 metrics loggers.py

Type
source_file
Publisher
vllm-project
Relationship
first_party
Version
vLLM v0.23.0
Retrieved
Jun 25
Local snapshot
corpus/domains/vllm/core-v0/evidence/snapshots/vllm-code-v0230-v1-metrics-loggers.py
SHA-256
ce2a0535d2b2b69e…

Claims citing this source (12)

ClaimLocatorVerified
In vLLM v0.23.0, vllm:prefix_cache_queries is a Prometheus counter measured in queried tokens — it counts tokens, not requests or cache blocks.vllm/v1/metrics/loggers.py:542-551, vllm/v1/metrics/loggers.py:1083-10887 hr ago
In vLLM v0.23.0, vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — it counts tokens, not requests or cache blocks.vllm/v1/metrics/loggers.py:553-561, vllm/v1/metrics/loggers.py:1083-10887 hr ago
In vLLM v0.23.0, vllm:prefix_cache_queries is incremented from SchedulerStats.prefix_cache_stats.queries on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters).vllm/v1/metrics/loggers.py:1083-1088, vllm/v1/metrics/loggers.py:562-5887 hr ago
In vLLM v0.23.0, vllm:prefix_cache_hits is incremented from SchedulerStats.prefix_cache_stats.hits on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters).vllm/v1/metrics/loggers.py:1083-1088, vllm/v1/metrics/loggers.py:562-5887 hr ago
In vLLM v0.23.0, vllm:num_requests_running is a Prometheus gauge for the number of requests in model execution batches.vllm/v1/metrics/loggers.py:451-4597 hr ago
In vLLM v0.23.0, vllm:num_requests_running is set from SchedulerStats.num_running_reqs and observes scheduler running-request state.vllm/v1/metrics/loggers.py:1066-10697 hr ago
In vLLM v0.23.0, vllm:num_requests_waiting is a Prometheus gauge for the number of requests waiting to be processed.vllm/v1/metrics/loggers.py:461-4697 hr ago
In vLLM v0.23.0, vllm:num_requests_waiting_by_reason is a Prometheus gauge with reason labels capacity and deferred, and its reasons sum to vllm:num_requests_waiting.vllm/v1/metrics/loggers.py:471-4917 hr ago
In vLLM v0.23.0, vllm:kv_cache_usage_perc is a Prometheus gauge for KV-cache usage where 1 means 100 percent usage.vllm/v1/metrics/loggers.py:519-5277 hr ago
In vLLM v0.23.0, vllm:kv_cache_usage_perc is set from SchedulerStats.kv_cache_usage and observes KV-cache usage pressure.vllm/v1/metrics/loggers.py:1081-10817 hr ago
In vLLM v0.23.0, vllm:num_preemptions is a Prometheus counter for cumulative preemptions from the engine.vllm/v1/metrics/loggers.py:619-6267 hr ago
In vLLM v0.23.0, vllm:num_preemptions increments from IterationStats.num_preempted_reqs, which is increased when engine-core events mark requests as preempted.vllm/v1/metrics/loggers.py:1149-11507 hr ago

Metrics grounded in this source (38)

NameDescriptionLocator
num_requests_runningGauge for the number of requests in model execution batches.vllm/v1/metrics/loggers.py:451-459
num_requests_waitingGauge for the number of requests waiting to be processed.vllm/v1/metrics/loggers.py:461-469
num_requests_waiting_by_reasonGauge that breaks waiting requests into capacity and deferred reasons.vllm/v1/metrics/loggers.py:471-491
kv_cache_usage_percGauge for KV-cache usage where 1 means 100 percent usage.vllm/v1/metrics/loggers.py:519-527
num_preemptionsCounter for cumulative preemptions from the engine.vllm/v1/metrics/loggers.py:619-626
engine_sleep_stateGauge for engine sleep state; sleep_state label values awake, weights_offloaded, and discard_all encode sleep levels.vllm/v1/metrics/loggers.py:493-514
corrupted_requestsCounter for corrupted requests, measured as requests with NaNs in logits.vllm/v1/metrics/loggers.py:529-540

View all 38 in Metrics →