vLLM v0.23.0 · Version scoped

vllm:kv_cache_usage_perc

Gauge for KV-cache usage where 1 means 100 percent usage.

MetricPrometheusverified Sep 29

Summary

In vLLM v0.23.0, under V1 metrics published by the PrometheusStatLogger, vllm:kv_cache_usage_perc is a Prometheus gauge for KV-cache usage (scheduler runtime component) in which a value of 1 means 100 percent usage. As a measurement, it is a point-in-time gauge in which 1 represents fully used KV block capacity, again for v0.23.0 with V1 metrics and the PrometheusStatLogger. In that same v0.23.0 PrometheusStatLogger/V1 metrics setting, the metric is registered with the label names model_name and engine. Its value comes from SchedulerStats.kv_cache_usage in the V1 scheduler stats, so in v0.23.0 it observes KV-cache usage pressure. Mechanically, in v0.23.0 the metric is set from SchedulerStats.kv_cache_usage rather than being incremented from completed requests. Interpretively, for v0.23.0 it describes active used capacity and is not a counter of reusable cached prefixes or of completed work. Accordingly, in v0.23.0 use it as a pressure signal alongside waiting and progress metrics, and do not infer cache effectiveness from it alone.

Version history

In v0.23.0, the only version covered here, vllm:kv_cache_usage_perc is exposed by the PrometheusStatLogger under V1 metrics as a point-in-time Prometheus gauge of KV-cache usage in which a value of 1 means fully used KV block capacity (100 percent).
In that same version its value is set directly from SchedulerStats.kv_cache_usage — observing scheduler-side KV-cache usage pressure — rather than being incremented from completed requests.
In v0.23.0 the gauge is registered with the label names model_name and engine.
In v0.23.0 it describes active used capacity and is not a counter of reusable cached prefixes or of completed work.
Accordingly, in v0.23.0 it should be read as a pressure signal alongside waiting and progress metrics, and cache effectiveness should not be inferred from it alone.

Identity

Prometheus type: gauge
Labels: model_nameengine
Unit: Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance

Metric knowledge (7)

Claim	Semantic layer	Verified
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` is a point-in-time gauge where 1 represents fully used KV block capacity.	Measurement semantics	Sep 29
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` is set from `SchedulerStats.kv_cache_usage` rather than incremented from completed requests.	Causal/runtime semantics	Sep 29
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` describes active used capacity and is not a counter of reusable cached prefixes or completed work.	Interpretation & composition	Sep 29
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` use it as a pressure signal alongside waiting and progress metrics; do not infer cache effectiveness from it alone.	Guarded guidance	Sep 29
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` is registered with the label names `model_name`, engine.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` is a Prometheus gauge for KV-cache usage where 1 means 100 percent usage.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` is set from `SchedulerStats.kv_cache_usage` and observes KV-cache usage pressure.	Interpretation & composition	Sep 29

Related Use Cases (1)

#12077: Active KV use is not reusable cache residency

Availability

vllm:kv_cache_usage_perc is registered as a Prometheus gauge in vLLM v0.23.0.

vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:519-527