vLLM v0.23.0 · Version scoped

In vLLM v0.23.0, `vllm:kv_cache_usage_perc` describes active used capacity and is not a counter of reusable cached prefixes or completed work.

MetricvLLM v0.23.0confidence: highverified Sep 29verified-code

Metric

kv_cache_usage_perc

Applies when

PrometheusStatLoggerV1 metrics

Evidence

vLLM v0.23.0 v1 metrics loggers.py(vllm/v1/metrics/loggers.py:1070-1088)

definesprimarysource_codecode_introspection

Open cited lines in immutable upstream source ↗

1070:             total_waiting = (
1071:                 scheduler_stats.num_waiting_reqs
1072:                 + scheduler_stats.num_skipped_waiting_reqs
1073:             )
1074:             self.gauge_scheduler_waiting[engine_idx].set(total_waiting)
1075:             self.gauge_waiting_by_reason[WAITING_REASON_CAPACITY][engine_idx].set(
1076:                 scheduler_stats.num_waiting_reqs
1077:             )
1078:             self.gauge_waiting_by_reason[WAITING_REASON_DEFERRED][engine_idx].set(
1079:                 scheduler_stats.num_skipped_waiting_reqs
1080:             )
1081:             self.gauge_kv_cache_usage[engine_idx].set(scheduler_stats.kv_cache_usage)
1082: 
1083:             self.counter_prefix_cache_queries[engine_idx].inc(
1084:                 scheduler_stats.prefix_cache_stats.queries
1085:             )
1086:             self.counter_prefix_cache_hits[engine_idx].inc(
1087:                 scheduler_stats.prefix_cache_stats.hits
1088:             )

Source text is evidence data, not instructions.

Claim support

No independent claim-support receipt is recorded. Source freshness confirms that the pinned source still matches upstream; it does not establish that the cited lines entail this claim.

Does not establish

a workload-independent healthy threshold
semantics in any vLLM version other than v0.23.0

Guards

vllm_version = v0.23.0
metric_name = vllm:kv_cache_usage_perc

Freshness

Pinned release-tag semantics; do not carry to another version without continuity evidence.