Scheduler and KV pressure metrics

seed slicecovered at v0.23.0legacy evidence: v0.23.0evidence: source code and documentation

Coverage

Coverage for a small vLLM v0.23.0 metric slice that exposes scheduler queue state, KV cache usage, and preemption signals.

Strong for whether the selected v0.23.0 metrics exist and what source fields they observe; weak for alert thresholds and workload-specific interpretation.

This topic is intentionally scoped to selected Prometheus metrics that connect to the existing scheduler/KV pressure graph; it is not a full vLLM observability catalog.

Coverage at vLLM v0.23.0

Every declared topic entity has applicable claims, availability, or admitted history at this version.

13 claims0 history facts9/9 entities

Claims in this topic (13)

Claim	Verified
In vLLM v0.23.0, `vllm:num_requests_running` is a Prometheus gauge for the number of requests in model execution batches.	10 hr ago
In vLLM v0.23.0, `vllm:num_requests_running` is set from `SchedulerStats.num_running_reqs` and observes scheduler running-request state.	10 hr ago
In vLLM v0.23.0, `vllm:num_requests_waiting` is a Prometheus gauge for the number of requests waiting to be processed.	10 hr ago
In vLLM v0.23.0, `vllm:num_requests_waiting_by_reason` is a Prometheus gauge with reason labels capacity and deferred, and its reasons sum to `vllm:num_requests_waiting`.	10 hr ago
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` is a Prometheus gauge for KV-cache usage where 1 means 100 percent usage.	10 hr ago
In vLLM v0.23.0, `vllm:kv_cache_usage_perc` is set from `SchedulerStats.kv_cache_usage` and observes KV-cache usage pressure.	10 hr ago
In vLLM v0.23.0, `vllm:num_preemptions` is a Prometheus counter for cumulative preemptions from the engine.	10 hr ago
In vLLM v0.23.0, `vllm:num_preemptions` increments from `IterationStats.num_preempted_reqs`, which is increased when engine-core events mark requests as preempted.	10 hr ago
In vLLM v0.23.0, `decode_time` is computed as `req_stats.last_token_ts` - `req_stats.first_token_ts`; the source comment describes the decode interval as from first `NEW_TOKEN` to last `NEW_TOKEN` and says any preemptions during decode are included.	10 hr ago
In vLLM v0.23.0, `queued_time` is computed as `req_stats.scheduled_ts` - `req_stats.queued_ts`; the source comment describes the queued interval as from first `QUEUED` event to first `SCHEDULED`.	10 hr ago
In vLLM v0.23.0, `prefill_time` is computed as `req_stats.first_token_ts` - `req_stats.scheduled_ts`; the source comment describes the prefill interval as from first `SCHEDULED` to first `NEW_TOKEN` and says any preemptions during prefill are included in the interval.	10 hr ago
In vLLM v0.23.0, `inference_time` is computed as `req_stats.last_token_ts` - `req_stats.scheduled_ts`; the source comment describes the inference interval as from first `SCHEDULED` to last `NEW_TOKEN` and says any preemptions during prefill or decode are included.	10 hr ago
For each `EngineCoreEventType.QUEUED` event processed by `IterationStats.update_from_events`, `req_stats.queued_ts` is set to event.timestamp.	10 hr ago

Metrics in this topic (9)

kv_cache_usage_perc — 2 claims
num_preemptions — 2 claims
num_requests_running — 2 claims
num_requests_waiting — 1 claim
num_requests_waiting_by_reason — 1 claim
request_decode_time_seconds — 1 claim
request_inference_time_seconds — 1 claim
request_prefill_time_seconds — 1 claim
request_queue_time_seconds — 2 claims

Surfaces

Prometheus /metrics endpoint — 45 inventoried members

Known gaps

The corpus does not include alert thresholds, benchmark baselines, or workload-specific healthy ranges for these metrics.

Schema.ai can identify and explain the metrics but should not yet recommend numeric alert thresholds.

Check: Add runtime evals, design-partner telemetry, or benchmark evidence before serving threshold guidance.

Availability records now inventory the complete fixed-name v0.23.0 PrometheusStatLogger metric surface, but semantic claims cover only the selected scheduler/KV pressure metrics.

Schema.ai can answer whether a fixed-name metric exists at v0.23.0 from the complete inventory, but interpretation claims remain limited to the seed slice.

Check: Promote semantic claims for additional inventoried metrics as they become eval-covered.

KV connector per-connector Prometheus metrics and vllm.stat_loggers plugin metrics are open extension points that register dynamic metric names not covered by the fixed-name inventory.

Absence answers must be scoped to first-party fixed-name registrations; a deployment with connector or plugin metrics can expose additional names.

Check: Record deployment-specific connector/plugin context as agent-supplied bindings, not corpus facts.