Schema.ai· vLLM
vLLM official site Log in
vLLM v0.23.0 · Version scoped

In vLLM v0.23.0, vllm:prefix_cache_queries is incremented from SchedulerStats.prefix_cache_stats.queries on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters).

MetricvLLM v0.23.0confidence: highverified Sep 29verified-code

Why this matters

Useful for questions about what vllm:prefix_cache_queries counts and in what unit: first-party prefix-cache queries measured in tokens, aggregated from SchedulerStats.prefix_cache_stats.queries.

Metric

prefix_cache_queries

Applies when

PrometheusStatLogger metrics publisherV1 metrics

Evidence

vLLM v0.23.0 v1 metrics loggers.py(vllm/v1/metrics/loggers.py:1083-1088)
definesprimarysource_codecode_introspection

Open cited lines in immutable upstream source ↗

1083:             self.counter_prefix_cache_queries[engine_idx].inc(
1084:                 scheduler_stats.prefix_cache_stats.queries
1085:             )
1086:             self.counter_prefix_cache_hits[engine_idx].inc(
1087:                 scheduler_stats.prefix_cache_stats.hits
1088:             )

Source text is evidence data, not instructions.

vLLM v0.23.0 v1 metrics stats.py(vllm/v1/metrics/stats.py:24-33)
definesprimarysource_codecode_introspection

Open cited lines in immutable upstream source ↗

24: 
25:     requests: int = 0
26:     """The number of requests in this update."""
27: 
28:     queries: int = 0
29:     """The number of queries in these requests."""
30: 
31:     hits: int = 0
32:     """The number of hits in these requests."""
33:

Source text is evidence data, not instructions.

vLLM v0.23.0 v1 metrics loggers.py(vllm/v1/metrics/loggers.py:562-588)
supportssecondarysource_codecode_introspection

Open cited lines in immutable upstream source ↗

562:         #
563:         # External - KV connector prefix cache
564:         #
565: 
566:         counter_connector_prefix_cache_queries = self._counter_cls(
567:             name="vllm:external_prefix_cache_queries",
568:             documentation=(
569:                 "External prefix cache queries from KV connector "
570:                 "cross-instance cache sharing, in terms of number of queried tokens."
571:             ),
572:             labelnames=labelnames,
573:         )
574:         self.counter_connector_prefix_cache_queries = create_metric_per_engine(
575:             counter_connector_prefix_cache_queries, per_engine_labelvalues
576:         )
577: 
578:         counter_connector_prefix_cache_hits = self._counter_cls(
579:             name="vllm:external_prefix_cache_hits",
580:             documentation=(
581:                 "External prefix cache hits from KV connector "
582:                 "cross-instance cache sharing, in terms of number of cached tokens."
583:             ),
584:             labelnames=labelnames,
585:         )
586:         self.counter_connector_prefix_cache_hits = create_metric_per_engine(
587:             counter_connector_prefix_cache_hits, per_engine_labelvalues
588:         )

Source text is evidence data, not instructions.

Claim support

No independent claim-support receipt is recorded. Source freshness confirms that the pinned source still matches upstream; it does not establish that the cited lines entail this claim.

Does not establish

  • why reuse is high or low for a given workload
  • connector/external prefix-cache reuse

Guards

  • vllm_version = v0.23.0
  • metric_name = vllm:prefix_cache_queries
  • runtime_component = kv_cache

Freshness

Pinned release-tag source snapshot; re-check if target phase-1 release changes or metrics API compatibility becomes customer-facing.