vllm:prefix_cache_queries

Summary

In vLLM v0.23.0, under V1 metrics published by PrometheusStatLogger, vllm:prefix_cache_queries counts tokens presented to the first-party prefix-cache lookup, accumulated as a counter. For the kv_cache runtime component in that version, it is a Prometheus counter measured in queried tokens — it counts tokens, not requests or cache blocks. As registered by the PrometheusStatLogger metrics publisher under V1 metrics in v0.23.0, the metric carries the label names model_name and engine. In v0.23.0 it is incremented from SchedulerStats.prefix_cache_stats.queries on each logging update and observes prefix-cache token reuse for the first-party prefix cache, while KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters. Interpretively, in v0.23.0 it serves as the token denominator for a prefix-cache hit ratio computed over the same labels and the same window. Accordingly, in v0.23.0 you should apply rate or increase over a chosen window alongside prefix_cache_hits, and should not interpret the cumulative total as a rate.

Version history

• In vLLM v0.23.0, vllm:prefix_cache_queries is exposed by the V1 PrometheusStatLogger as a Prometheus counter measured in queried tokens, accumulating the tokens presented to the first-party prefix-cache lookup rather than requests or cache blocks.
• In that same version it is registered with the label names model_name and engine.
• Also in v0.23.0, the counter increments from SchedulerStats.prefix_cache_stats.queries on each scheduler-stats logging update and observes first-party prefix-cache token reuse, while KV-connector reuse is reported separately by the vllm:external_prefix_cache_* counters.
• For interpretation in v0.23.0, it serves as the token denominator of a same-label, same-window prefix-cache hit ratio, so it should be combined with prefix_cache_hits using rate or increase over a chosen window instead of reading the cumulative total as a rate.

Identity

Prometheus type

counter

Labels

model_nameengine

Unit

Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

At vLLM v0.23.0

        counter_prefix_cache_queries = self._counter_cls(
            name="vllm:prefix_cache_queries",
            documentation=(
                "Prefix cache queries, in terms of number of queried tokens."
            ),
            labelnames=labelnames,
        )

- `queries`: Refers to the number of tokens that were queried.

Lifecycle

Semantic coverage

Metric knowledge (7)

Related Use Cases (3)

• #12077: Active KV use is not reusable cache residency
• #38194: Prefix-cache percentages require comparable definitions
• #40696: Full-block eligibility creates prefix-cache cliffs

Availability