vllm:prefix_cache_hits

Summary

In vLLM v0.23.0, with the V1 metrics published by PrometheusStatLogger, vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — it counts tokens, not requests or cache blocks. In that same version and publisher, the counter is registered with the label names model_name and engine. In vLLM v0.23.0 (PrometheusStatLogger, V1 metrics), it counts queried tokens that were found in the first-party prefix cache, accumulated as a counter. In vLLM v0.23.0, the counter is incremented from SchedulerStats.prefix_cache_stats.hits on each logging update and observes prefix-cache token reuse for the first-party prefix cache, while KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters. The vLLM v0.23.0 metrics design docs state that the quantity of interest is the prefix cache hit rate (hits per query), and that the counters are exposed raw so operators can compute the rate over an interval of their choosing with PromQL as rate(hits)/rate(queries), whereas vLLM's own logging equivalent aggregates hit_rate over the most recent queries. Accordingly, in vLLM v0.23.0 this counter serves as the token numerator of a prefix-cache hit ratio computed over the same labels and the same window. For guidance in vLLM v0.23.0, divide the hit and query rates taken over the same window; no universal healthy percentage follows, because prefix reuse is workload-dependent.

Version history

• The only version in scope here is v0.23.0, where vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — counting tokens rather than requests or cache blocks — accumulating the queried tokens found in the first-party prefix cache.
• In that same v0.23.0 release, the counter is registered under the V1 PrometheusStatLogger with the label names model_name and engine.
• As of v0.23.0 its runtime behavior is to increment from SchedulerStats.prefix_cache_stats.hits on each scheduler-stats logging update, observing first-party prefix-cache token reuse while KV-connector reuse is reported by the separate vllm:external_prefix_cache_* counters.
• The v0.23.0 metrics design docs frame the quantity of interest as the prefix cache hit rate (hits per query), exposing the counters raw so operators compute the rate over an interval of their choosing with PromQL (rate(hits)/rate(queries)), whereas vLLM's own logging equivalent aggregates hit_rate over the most recent queries.
• Interpreting the v0.23.0 metric, it serves as the token numerator of a same-label, same-window prefix-cache hit ratio formed by dividing same-window hit and query rates, and no universal healthy percentage follows because reuse is workload-dependent.

Identity

Prometheus type

counter

Labels

model_nameengine

Unit

Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

At vLLM v0.23.0

        counter_prefix_cache_hits = self._counter_cls(
            name="vllm:prefix_cache_hits",
            documentation=("Prefix cache hits, in terms of number of cached tokens."),
            labelnames=labelnames,
        )

- `queries`: Refers to the number of tokens that were queried.

Lifecycle

Semantic coverage

Metric knowledge (8)

Related Use Cases (3)

• #12077: Active KV use is not reusable cache residency
• #38194: Prefix-cache percentages require comparable definitions
• #40696: Full-block eligibility creates prefix-cache cliffs

Availability