vllm:prefix_cache_hits

Counter for prefix cache hits, in cached tokens.

MetricPrometheusvLLM v0.23.0verified 7 hr ago

Claim	Verified
In vLLM v0.23.0, `vllm:prefix_cache_hits` is a Prometheus counter measured in cached (hit) tokens — it counts tokens, not requests or cache blocks.	7 hr ago
In vLLM v0.23.0, `vllm:prefix_cache_hits` is incremented from `SchedulerStats.prefix_cache_stats.hits` on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate `vllm:external_prefix_cache_*` counters).	7 hr ago
The vLLM v0.23.0 metrics design docs state the metric of interest is the prefix cache hit rate (hits per query): the counters are exposed raw so operators compute the rate over an interval of their choosing with PromQL (`rate(hits)/rate(queries)`); vLLM's own logging equivalent aggregates `hit_rate` over the most recent queries.	7 hr ago

vllm:prefix_cache_hits is registered as a Prometheus counter in vLLM v0.23.0.

vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:553-560