Histogram of time spent in the WAITING phase per request.
MetricPrometheusvLLM v0.23.0verified 10 hr ago
Identity
- Prometheus type
- histogram
- Labels
- model_nameengine
- Buckets
- 21 boundaries · 0.3 → 7680
- Unit
- Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes
Semantic coverage
covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance
Metric knowledge (9)
| Claim | Semantic layer | Verified |
|---|
In vLLM v0.23.0, vllm:request_queue_time_seconds observes one finished-request duration from queued_ts to the first scheduled_ts. | Measurement semantics | 10 hr ago |
In vLLM v0.23.0, vllm:request_queue_time_seconds is emitted on completion from scheduler lifecycle timestamps; repeated QUEUED events overwrite queued_ts while first SCHEDULED is retained. | Causal/runtime semantics | 10 hr ago |
In vLLM v0.23.0, vllm:request_queue_time_seconds measures the recorded waiting interval and includes only requests reaching finished-request accounting. | Interpretation & composition | 10 hr ago |
In vLLM v0.23.0, vllm:request_queue_time_seconds use quantiles with running/KV/token-progress metrics to distinguish admission delay from decode delay; no universal threshold is established. | Guarded guidance | 10 hr ago |
In vLLM v0.23.0, vllm:request_queue_time_seconds is registered as a Prometheus histogram. | Structural identity | 10 hr ago |
In vLLM v0.23.0, vllm:request_queue_time_seconds is registered with the label names model_name, engine. | Structural identity | 10 hr ago |
In vLLM v0.23.0, vllm:request_queue_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0. | Structural identity | 10 hr ago |
For each EngineCoreEventType.QUEUED event processed by IterationStats.update_from_events, req_stats.queued_ts is set to event.timestamp. | Interpretation & composition | 10 hr ago |
In vLLM v0.23.0, queued_time is computed as req_stats.scheduled_ts - req_stats.queued_ts; the source comment describes the queued interval as from first QUEUED event to first SCHEDULED. | Interpretation & composition | 10 hr ago |
Availability
vllm:request_queue_time_seconds is registered as a Prometheus histogram in vLLM v0.23.0.
vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher
vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:880-888
Schema.ai