vllm:prefix_cache_hits

Version history

• The only version in scope here is v0.23.0, where vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — counting tokens rather than requests or cache blocks — accumulating the queried tokens found in the first-party prefix cache.
• In that same v0.23.0 release, the counter is registered under the V1 PrometheusStatLogger with the label names model_name and engine.
• As of v0.23.0 its runtime behavior is to increment from SchedulerStats.prefix_cache_stats.hits on each scheduler-stats logging update, observing first-party prefix-cache token reuse while KV-connector reuse is reported by the separate vllm:external_prefix_cache_* counters.
• The v0.23.0 metrics design docs frame the quantity of interest as the prefix cache hit rate (hits per query), exposing the counters raw so operators compute the rate over an interval of their choosing with PromQL (rate(hits)/rate(queries)), whereas vLLM's own logging equivalent aggregates hit_rate over the most recent queries.
• Interpreting the v0.23.0 metric, it serves as the token numerator of a same-label, same-window prefix-cache hit ratio formed by dividing same-window hit and query rates, and no universal healthy percentage follows because reuse is workload-dependent.

At vLLM v0.29.0

        counter_prefix_cache_hits = self._counter_cls(
            name="vllm:prefix_cache_hits",
            documentation=("Prefix cache hits, in terms of number of cached tokens."),
            labelnames=labelnames,
        )

- `queries`: Refers to the number of tokens that were queried.

Lifecycle

Semantic coverage

Metric knowledge (0)

No claims about this entity yet.