All versions · Version range: v0.23.0

vllm:request_queue_time_seconds

Histogram of time spent in the WAITING phase per request.

MetricPrometheusverified Sep 29

Version history

In vLLM v0.23.0, the only version in scope here, vllm:request_queue_time_seconds is registered as a Prometheus histogram carrying the label names model_name and engine.
Its v0.23.0 registration declares 21 literal histogram bucket boundaries spanning 0.3 to 7680.0.
In v0.23.0 the histogram observes one duration per finished request, measured from queued_ts to the first scheduled_ts.
That timestamp is populated in v0.23.0 by IterationStats.update_from_events, which sets req_stats.queued_ts to event.timestamp for each EngineCoreEventType.QUEUED event it processes.
At finished-request accounting in v0.23.0, queued_time is computed as req_stats.scheduled_ts - req_stats.queued_ts, with the source comment describing the queued interval as running from the first QUEUED event to the first SCHEDULED.
Emission in v0.23.0 happens on completion from scheduler lifecycle timestamps, where repeated QUEUED events overwrite queued_ts while the first SCHEDULED timestamp is retained.
Interpreting the v0.23.0 metric, it reflects the recorded waiting interval and covers only requests that reach finished-request accounting.
Guidance for v0.23.0 is to read its quantiles alongside running/KV/token-progress metrics to separate admission delay from decode delay, since no universal threshold is established.

Identity

Prometheus type: histogram
Labels: model_nameengine
Buckets: 21 boundaries · 0.3 → 7680
Unit: Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance

Metric knowledge (9)

Claim	Semantic layer	Verified
In vLLM v0.23.0, `vllm:request_queue_time_seconds` observes one finished-request duration from `queued_ts` to the first `scheduled_ts`.	Measurement semantics	Sep 29
In vLLM v0.23.0, `vllm:request_queue_time_seconds` is emitted on completion from scheduler lifecycle timestamps; repeated QUEUED events overwrite `queued_ts` while first SCHEDULED is retained.	Causal/runtime semantics	Sep 29
In vLLM v0.23.0, `vllm:request_queue_time_seconds` measures the recorded waiting interval and includes only requests reaching finished-request accounting.	Interpretation & composition	Sep 29
In vLLM v0.23.0, `vllm:request_queue_time_seconds` use quantiles with running/KV/token-progress metrics to distinguish admission delay from decode delay; no universal threshold is established.	Guarded guidance	Sep 29
In vLLM v0.23.0, `vllm:request_queue_time_seconds` is registered as a Prometheus histogram.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:request_queue_time_seconds` is registered with the label names `model_name`, engine.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:request_queue_time_seconds` declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0.	Structural identity	Sep 29
For each `EngineCoreEventType.QUEUED` event processed by `IterationStats.update_from_events`, `req_stats.queued_ts` is set to event.timestamp.	Interpretation & composition	Sep 29
In vLLM v0.23.0, `queued_time` is computed as `req_stats.scheduled_ts` - `req_stats.queued_ts`; the source comment describes the queued interval as from first QUEUED event to first SCHEDULED.	Interpretation & composition	Sep 29

Related Use Cases (1)

#35048: Aggregate throughput can hide experience regressions

Availability

vllm:request_queue_time_seconds is registered as a Prometheus histogram in vLLM v0.23.0.

vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:880-888