In vLLM v0.23.0, vllm:prefix_cache_queries is a Prometheus counter measured in queried tokens — it counts tokens, not requests or cache blocks. | v0.23.0 | Metric | prefix_cache_queries | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — it counts tokens, not requests or cache blocks. | v0.23.0 | Metric | prefix_cache_hits | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_queries is incremented from SchedulerStats.prefix_cache_stats.queries on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters). | v0.23.0 | Metric | prefix_cache_queries | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits is incremented from SchedulerStats.prefix_cache_stats.hits on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters). | v0.23.0 | Metric | prefix_cache_hits | 4 hr ago | github.com |
The vLLM v0.23.0 metrics design docs state the metric of interest is the prefix cache hit rate (hits per query): the counters are exposed raw so operators compute the rate over an interval of their choosing with PromQL (rate(hits)/rate(queries)); vLLM's own logging equivalent aggregates hit_rate over the most recent queries. | v0.23.0 | Metric | prefix_cache_hits | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running is a Prometheus gauge for the number of requests in model execution batches. | v0.23.0 | Metric | num_requests_running | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running is set from SchedulerStats.num_running_reqs and observes scheduler running-request state. | v0.23.0 | Metric | num_requests_running | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_waiting is a Prometheus gauge for the number of requests waiting to be processed. | v0.23.0 | Metric | num_requests_waiting | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_waiting_by_reason is a Prometheus gauge with reason labels capacity and deferred, and its reasons sum to vllm:num_requests_waiting. | v0.23.0 | Metric | num_requests_waiting_by_reason | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc is a Prometheus gauge for KV-cache usage where 1 means 100 percent usage. | v0.23.0 | Metric | kv_cache_usage_perc | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc is set from SchedulerStats.kv_cache_usage and observes KV-cache usage pressure. | v0.23.0 | Metric | kv_cache_usage_perc | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:num_preemptions is a Prometheus counter for cumulative preemptions from the engine. | v0.23.0 | Metric | num_preemptions | 4 hr ago | github.com |
In vLLM v0.23.0, vllm:num_preemptions increments from IterationStats.num_preempted_reqs, which is increased when engine-core events mark requests as preempted. | v0.23.0 | Metric | num_preemptions | 4 hr ago | github.com |
Schema.ai· vLLM