All versions · Version range: v0.23.0

vllm:request_time_per_output_token_seconds

Histogram of mean time per output token per request.

MetricPrometheusverified Sep 29

Version history

The only version documented here is v0.23.0, where vllm:request_time_per_output_token_seconds is registered as a Prometheus histogram under the PrometheusStatLogger V1 metrics path with the label names model_name and engine, and with 19 literal bucket boundaries spanning 0.01 to 80.0.
In that same version it records one observation per finished request, namely the mean decode interval computed as first-to-last-token decode time divided by the output-token intervals after the first token (num_generation_tokens minus one), yielding zero when no such interval exists.
Still in v0.23.0, the resulting distribution is request-weighted TPOT and therefore is not the same population as the token-weighted inter_token_latency_seconds.
Accordingly, v0.23.0 guidance is to use it for per-request decode experience and to compare distributions under matched output lengths rather than substituting aggregate tokens per second.

Identity

Prometheus type: histogram
Labels: model_nameengine
Buckets: 19 boundaries · 0.01 → 80
Unit: Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance

Metric knowledge (7)

Claim	Semantic layer	Verified
In vLLM v0.23.0, `vllm:request_time_per_output_token_seconds` observes one per-finished-request mean decode interval divided by output-token intervals after the first token.	Measurement semantics	Sep 29
In vLLM v0.23.0, `vllm:request_time_per_output_token_seconds` is computed from first-to-last-token decode time and `num_generation_tokens` minus one, with zero when no interval exists.	Causal/runtime semantics	Sep 29
In vLLM v0.23.0, `vllm:request_time_per_output_token_seconds` is a request-weighted TPOT distribution and is not the same population as token-weighted `inter_token_latency_seconds`.	Interpretation & composition	Sep 29
In vLLM v0.23.0, `vllm:request_time_per_output_token_seconds` use it for per-request decode experience; compare distributions under matched output lengths rather than substituting aggregate tokens per second.	Guarded guidance	Sep 29
In vLLM v0.23.0, `vllm:request_time_per_output_token_seconds` is registered as a Prometheus histogram.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:request_time_per_output_token_seconds` is registered with the label names `model_name`, engine.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:request_time_per_output_token_seconds` declares 19 literal histogram bucket boundaries in its registration, from 0.01 to 80.0.	Structural identity	Sep 29

Related Use Cases (1)

#35048: Aggregate throughput can hide experience regressions

Availability

vllm:request_time_per_output_token_seconds is registered as a Prometheus histogram in vLLM v0.23.0.

vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:817-845