Schema.ai· vLLM· Schema 0.2.1
vLLM official site vLLM v0.23.0Log in

vllm:prefix_cache_hits

Counter for prefix cache hits, in cached tokens.

MetricPrometheusvLLM v0.23.0verified 10 hr ago

Identity

Prometheus type
counter
Labels
model_nameengine
Unit
Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

At vLLM v0.23.0

Presentcounter

Names: vllm:prefix_cache_hits

Counts prefix-cache activity in tokens.

Changed in v0.7.3: has_prometheus_metric_type claim set changed

Changed in v0.9.0: has_metric_semantics claim set changed

Changed in v0.9.2: has_name claim set changed

vLLM vllm/v1/metrics/loggers.py · v0.23.0
        counter_prefix_cache_hits = self._counter_cls(
            name="vllm:prefix_cache_hits",
            documentation=("Prefix cache hits, in terms of number of cached tokens."),
            labelnames=labelnames,
        )
vLLM vllm/v1/metrics/stats.py · v0.23.0
- `queries`: Refers to the number of tokens that were queried.

Lifecycle

vLLM versionStateMetric name
v0.27.1presentvllm:prefix_cache_hits
v0.27.0presentvllm:prefix_cache_hits
v0.26.0presentvllm:prefix_cache_hits
v0.25.1presentvllm:prefix_cache_hits
v0.25.0presentvllm:prefix_cache_hits
v0.24.0presentvllm:prefix_cache_hits
v0.23.0 · selectedpresentvllm:prefix_cache_hits
v0.22.1presentvllm:prefix_cache_hits
v0.22.0presentvllm:prefix_cache_hits
v0.21.0presentvllm:prefix_cache_hits
v0.20.2presentvllm:prefix_cache_hits
v0.20.1presentvllm:prefix_cache_hits
v0.20.0presentvllm:prefix_cache_hits
v0.19.1presentvllm:prefix_cache_hits
v0.19.0presentvllm:prefix_cache_hits
v0.18.1presentvllm:prefix_cache_hits
v0.18.0presentvllm:prefix_cache_hits
v0.17.1presentvllm:prefix_cache_hits
v0.17.0presentvllm:prefix_cache_hits
v0.16.0presentvllm:prefix_cache_hits
v0.15.1presentvllm:prefix_cache_hits
v0.15.0presentvllm:prefix_cache_hits
v0.14.1presentvllm:prefix_cache_hits
v0.14.0presentvllm:prefix_cache_hits
v0.13.0presentvllm:prefix_cache_hits
v0.12.0presentvllm:prefix_cache_hits
v0.11.2presentvllm:prefix_cache_hits
v0.11.1presentvllm:prefix_cache_hits
v0.11.0presentvllm:prefix_cache_hits
v0.10.2presentvllm:prefix_cache_hits
v0.10.1.1presentvllm:prefix_cache_hits
v0.10.1presentvllm:prefix_cache_hits
v0.10.0presentvllm:prefix_cache_hits
v0.9.2presentvllm:prefix_cache_hits
v0.9.1presentvllm:gpu_prefix_cache_hits
v0.9.0.1presentvllm:gpu_prefix_cache_hits
v0.9.0presentvllm:gpu_prefix_cache_hits
v0.8.5.post1presentvllm:gpu_prefix_cache_hits
v0.8.5presentvllm:gpu_prefix_cache_hits
v0.8.4presentvllm:gpu_prefix_cache_hits
v0.8.3presentvllm:gpu_prefix_cache_hits
v0.8.2presentvllm:gpu_prefix_cache_hits
v0.8.1presentvllm:gpu_prefix_cache_hits
v0.8.0presentvllm:gpu_prefix_cache_hits
v0.7.3presentvllm:gpu_prefix_cache_hits
v0.7.2absentvllm:prefix_cache_hits
v0.7.1absentvllm:prefix_cache_hits
v0.7.0absentvllm:prefix_cache_hits

Semantic coverage

covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance

Metric knowledge (8)

ClaimSemantic layerVerified
In vLLM v0.23.0, vllm:prefix_cache_hits counts queried tokens found in the first-party prefix cache, accumulated as a counter.Measurement semantics10 hr ago
In vLLM v0.23.0, vllm:prefix_cache_hits increments from SchedulerStats.prefix_cache_stats.hits on each scheduler-stats logging update.Causal/runtime semantics10 hr ago
In vLLM v0.23.0, vllm:prefix_cache_hits is the token numerator for a same-label, same-window prefix-cache hit ratio.Interpretation & composition10 hr ago
In vLLM v0.23.0, vllm:prefix_cache_hits divide same-window hit and query rates; no universal healthy percentage follows because reuse is workload-dependent.Guarded guidance10 hr ago
In vLLM v0.23.0, vllm:prefix_cache_hits is registered with the label names model_name, engine.Structural identity10 hr ago
In vLLM v0.23.0, vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — it counts tokens, not requests or cache blocks.Structural identity10 hr ago
In vLLM v0.23.0, vllm:prefix_cache_hits is incremented from SchedulerStats.prefix_cache_stats.hits on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters).Interpretation & composition10 hr ago
The vLLM v0.23.0 metrics design docs state the metric of interest is the prefix cache hit rate (hits per query): the counters are exposed raw so operators compute the rate over an interval of their choosing with PromQL (rate(hits)/rate(queries)); vLLM's own logging equivalent aggregates hit_rate over the most recent queries.Interpretation & composition10 hr ago

Related Use Cases (3)

Availability

vllm:prefix_cache_hits is registered as a Prometheus counter in vLLM v0.23.0.
vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:553-560