vllm:prefix_cache_queries

Version history

• In vLLM v0.23.0, vllm:prefix_cache_queries is exposed by the V1 PrometheusStatLogger as a Prometheus counter measured in queried tokens, accumulating the tokens presented to the first-party prefix-cache lookup rather than requests or cache blocks.
• In that same version it is registered with the label names model_name and engine.
• Also in v0.23.0, the counter increments from SchedulerStats.prefix_cache_stats.queries on each scheduler-stats logging update and observes first-party prefix-cache token reuse, while KV-connector reuse is reported separately by the vllm:external_prefix_cache_* counters.
• For interpretation in v0.23.0, it serves as the token denominator of a same-label, same-window prefix-cache hit ratio, so it should be combined with prefix_cache_hits using rate or increase over a chosen window instead of reading the cumulative total as a rate.

At vLLM v0.28.0

        counter_prefix_cache_queries = self._counter_cls(
            name="vllm:prefix_cache_queries",
            documentation=(
                "Prefix cache queries, in terms of number of queried tokens."
            ),
            labelnames=labelnames,
        )

- `queries`: Refers to the number of tokens that were queried.

Lifecycle

Semantic coverage

Metric knowledge (0)

No claims about this entity yet.