vllm:request_prefill_kv_computed_tokens

Summary

In vLLM v0.23.0, under V1 metrics published by the PrometheusStatLogger, vllm:request_prefill_kv_computed_tokens is registered as a Prometheus histogram. It is registered with the label names model_name and engine. It observes one histogram sample per finished request, equal to prompt tokens minus non-negative cached tokens. That value is computed in finished-request logging after a request completes, from num_prompt_tokens and num_cached_tokens. Interpretively, it captures the newly computed prefill KV work per completed request rather than total prompt length whenever caching contributes. For guidance, compare its distributions only under matched workloads, pair it with cached-token counters as companion signals, and do not read a single bucket count as token throughput.

Version history

• In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens is known to be registered by the PrometheusStatLogger (V1 metrics) as a Prometheus histogram carrying the label names model_name and engine.
• In that same v0.23.0 scope, it records one histogram sample per finished request equal to prompt tokens minus non-negative cached tokens.
• Also in v0.23.0, the value is computed during finished-request logging after the request completes, from num_prompt_tokens and num_cached_tokens.
• Interpretively for v0.23.0, the metric reflects newly computed prefill KV work per completed request rather than total prompt length whenever caching contributes.
• Guidance recorded for v0.23.0 is to compare its distributions under matched workloads alongside cached-token counters and to avoid reading a single bucket count as token throughput.

Identity

Prometheus type

histogram

Labels

model_nameengine

Buckets

Not recorded — bucket boundaries are computed at runtime from max_model_len, so no fixed values are true

Unit

Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

Metric knowledge (6)

Related Use Cases (1)

• #40696: Full-block eligibility creates prefix-cache cliffs

Availability