vllm:prompt_tokens_cached

Summary

In vLLM v0.23.0, under V1 metrics published by the PrometheusStatLogger, vllm:prompt_tokens_cached is registered as a Prometheus counter. In that same v0.23.0 PrometheusStatLogger/V1 metrics context, the metric is registered with the label names model_name and engine. For v0.23.0 with the PrometheusStatLogger under V1 metrics, vllm:prompt_tokens_cached counts cached prompt tokens reused during processing, combining both local and external sources. Mechanically, in v0.23.0 (PrometheusStatLogger, V1 metrics) the counter increments from the per-iteration PromptTokenStats.cached_tokens field. Interpretively, for v0.23.0 under the PrometheusStatLogger with V1 metrics, this metric represents aggregate saved prompt-token work and is not by itself a per-request cache-hit percentage. As guidance in v0.23.0 (PrometheusStatLogger, V1 metrics), compare the metric's rate against prompt-token demand using matching labels and time windows, and separate local from external attribution when that distinction is needed.

Version history

• The only version covered here is v0.23.0, where vllm:prompt_tokens_cached is registered as a Prometheus counter with the label names model_name and engine under the PrometheusStatLogger V1 metrics publisher.
• In v0.23.0 it counts cached prompt tokens reused during processing, combining local and external sources.
• In v0.23.0 the counter increments from the per-iteration PromptTokenStats.cached_tokens field.
• In v0.23.0 it represents aggregate saved prompt-token work and is not by itself a per-request cache-hit percentage.
• For v0.23.0, the guidance is to compare its rate with prompt-token demand under matching labels and windows, separating local and external attribution when required.

Identity

Prometheus type

counter

Labels

model_nameengine

Unit

Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

Metric knowledge (6)

Related Use Cases (1)

• #40696: Full-block eligibility creates prefix-cache cliffs

Availability