seed slicecovered at v0.23.0legacy evidence: v0.23.0evidence: source code and documentation
Coverage
Coverage for the vLLM v0.23.0 first-party prefix-cache counters and how to compute and interpret the prefix cache hit rate.
Strong for what vllm:prefix_cache_queries and vllm:prefix_cache_hits count (tokens), where they are set from, and how to compute hit rate; none for healthy hit-rate thresholds or workload-specific tuning.
Scoped to the two first-party prefix-cache counters; external/KV-connector prefix-cache counters are inventoried but not semantically covered.
Coverage at vLLM v0.23.0
Every declared topic entity has applicable claims, availability, or admitted history at this version.
13 claims2 history facts2/2 entities
History-backed metrics: prefix_cache_hits, prefix_cache_queries
Claims in this topic (13)
| Claim | Verified |
|---|
In vLLM v0.23.0, vllm:prefix_cache_queries is a Prometheus counter measured in queried tokens — it counts tokens, not requests or cache blocks. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — it counts tokens, not requests or cache blocks. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_queries is incremented from SchedulerStats.prefix_cache_stats.queries on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters). | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_hits is incremented from SchedulerStats.prefix_cache_stats.hits on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters). | 10 hr ago |
The vLLM v0.23.0 metrics design docs state the metric of interest is the prefix cache hit rate (hits per query): the counters are exposed raw so operators compute the rate over an interval of their choosing with PromQL (rate(hits)/rate(queries)); vLLM's own logging equivalent aggregates hit_rate over the most recent queries. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_queries counts tokens presented to the first-party prefix-cache lookup, accumulated as a counter. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_queries increments from SchedulerStats.prefix_cache_stats.queries on each scheduler-stats logging update. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_queries is the token denominator for a same-label, same-window prefix-cache hit ratio. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_queries use rate or increase over a chosen window with prefix_cache_hits; do not interpret the cumulative total as a rate. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_hits counts queried tokens found in the first-party prefix cache, accumulated as a counter. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_hits increments from SchedulerStats.prefix_cache_stats.hits on each scheduler-stats logging update. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_hits is the token numerator for a same-label, same-window prefix-cache hit ratio. | 10 hr ago |
In vLLM v0.23.0, vllm:prefix_cache_hits divide same-window hit and query rates; no universal healthy percentage follows because reuse is workload-dependent. | 10 hr ago |
Metrics in this topic (2)
Known gaps
The corpus has no benchmark or operator evidence for healthy prefix-cache hit-rate ranges; hit rate is workload-dependent (prompt-prefix sharing).
Schema.ai can explain how to compute the hit rate but must not grade an observed value as good or bad.
Check: Add benchmark or design-partner telemetry evidence before serving hit-rate thresholds.
vllm:external_prefix_cache_queries/hits (KV-connector prefix cache) are availability-inventoried but carry no semantic claims.
Existence answers are available for the external counters; interpretation claims are not.
Check: Promote KV-connector prefix-cache semantics from the connector metrics source when connector deployments become an eval target.
Schema.ai