vLLM v0.23.0 · Version scoped

vllm:e2e_request_latency_seconds

Histogram of end-to-end request latency in seconds.

MetricPrometheusverified Sep 29

Summary

In vLLM v0.23.0, under the PrometheusStatLogger metrics publisher with V1 metrics, vllm:e2e_request_latency_seconds is registered as a Prometheus histogram. Its registration declares the label names model_name and engine. The registration also declares 21 literal histogram bucket boundaries, ranging from 0.3 to 7680.0. Each observation records the duration of one finished request, in seconds, from arrival to logging-time completion. At runtime the value is computed when a request enters finished-request accounting, as the current time minus arrival_time. Interpretively, the metric combines queue, prefill, decode, and other request-lifecycle time rather than isolating any single phase. Accordingly, use the phase histograms as companions before assigning cause, and compare like request populations and workload distributions.

Version history

In vLLM v0.23.0, under the PrometheusStatLogger publisher with V1 metrics, vllm:e2e_request_latency_seconds is registered as a Prometheus histogram with the label names model_name and engine.
In that same v0.23.0 registration, the histogram declares 21 literal bucket boundaries spanning 0.3 to 7680.0.
In v0.23.0, each observation records one finished-request duration in seconds from arrival to logging-time completion.
In v0.23.0, that value is computed when a request enters finished-request accounting, as the current time minus arrival_time.
In v0.23.0, the resulting figure combines queue, prefill, decode, and other request-lifecycle time rather than isolating any single phase.
For v0.23.0, the guidance is to read phase histograms as companions before assigning cause and to compare only like request populations and workload distributions.

Identity

Prometheus type: histogram
Labels: model_nameengine
Buckets: 21 boundaries · 0.3 → 7680
Unit: Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance

Metric knowledge (7)

Claim	Semantic layer	Verified
In vLLM v0.23.0, `vllm:e2e_request_latency_seconds` observes one finished-request duration in seconds from arrival to logging-time completion.	Measurement semantics	Sep 29
In vLLM v0.23.0, `vllm:e2e_request_latency_seconds` is computed when a request enters finished-request accounting using current time minus `arrival_time`.	Causal/runtime semantics	Sep 29
In vLLM v0.23.0, `vllm:e2e_request_latency_seconds` combines queue, prefill, decode, and other request-lifecycle time rather than isolating one phase.	Interpretation & composition	Sep 29
In vLLM v0.23.0, `vllm:e2e_request_latency_seconds` use phase histograms as companions before assigning cause; compare like request populations and workload distributions.	Guarded guidance	Sep 29
In vLLM v0.23.0, `vllm:e2e_request_latency_seconds` is registered as a Prometheus histogram.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:e2e_request_latency_seconds` is registered with the label names `model_name`, engine.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:e2e_request_latency_seconds` declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0.	Structural identity	Sep 29

Related Use Cases (1)

#35048: Aggregate throughput can hide experience regressions

Availability

vllm:e2e_request_latency_seconds is registered as a Prometheus histogram in vLLM v0.23.0.

vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:870-878