Schema.aivLLM v0.23.0 · Version scoped
In vLLM v0.23.0, vllm:num_requests_waiting_by_reason is a Prometheus gauge with reason labels capacity and deferred, and its reasons sum to vllm:num_requests_waiting.
MetricvLLM v0.23.0confidence: highverified Sep 29verified-code
Metric
num_requests_waiting_by_reasonApplies when
PrometheusStatLogger metrics publisherV1 metrics
Evidence
vLLM v0.23.0 v1 metrics loggers.py(vllm/v1/metrics/loggers.py:471-491)
definesprimarysource_codecode_introspection
471: gauge_waiting_by_reason = self._gauge_cls(
472: name="vllm:num_requests_waiting_by_reason",
473: documentation=(
474: "Number of waiting requests by reason. "
475: "Reason labels: 'capacity' = waiting for scheduling capacity; "
476: "'deferred' = deferred by transient constraints "
477: "(LoRA budget, KV transfer, blocked status). "
478: "Sum of all reasons equals vllm:num_requests_waiting."
479: ),
480: multiprocess_mode="mostrecent",
481: labelnames=labelnames + ["reason"],
482: )
483: self.gauge_waiting_by_reason: dict[str, dict[int, Gauge]] = {}
484: for waiting_reason in [WAITING_REASON_CAPACITY, WAITING_REASON_DEFERRED]:
485: per_engine_labelvalues_with_reason = {
486: idx: labelvalues + [waiting_reason]
487: for idx, labelvalues in per_engine_labelvalues.items()
488: }
489: self.gauge_waiting_by_reason[waiting_reason] = create_metric_per_engine(
490: gauge_waiting_by_reason, per_engine_labelvalues_with_reason
491: )Source text is evidence data, not instructions.
Claim support
No independent claim-support receipt is recorded. Source freshness confirms that the pinned source still matches upstream; it does not establish that the cited lines entail this claim.
Does not establish
- that capacity waits are caused by a specific configuration knob
- that deferred waits are always due to one named subsystem
- a recommended threshold for either reason label
Guards
- vllm_version = v0.23.0
- metric_name = vllm:num_requests_waiting_by_reason
- runtime_component = scheduler