vLLM v0.23.0 · Version scoped

vllm:time_to_first_token_seconds

Histogram of time to first token in seconds.

MetricPrometheusverified Sep 29

Summary

In vLLM v0.23.0, under the PrometheusStatLogger metrics publisher with V1 metrics, vllm:time_to_first_token_seconds is registered as a Prometheus histogram. Its registration declares the label names model_name and engine. Its registration also declares 22 literal histogram bucket boundaries, ranging from 0.001 to 2560.0. The metric observes the elapsed seconds from request arrival to the first produced token. At runtime, an observation is added when iteration processing identifies the request as prefilling or producing its first token. Because its histogram count follows first-token events, that count need not equal a finished-request counter across abort and error paths. For use, take histogram quantiles with matched labels and windows, and reconcile lifecycle population before comparing its count with request_success.

Version history

In vLLM v0.23.0, vllm:time_to_first_token_seconds is registered as a Prometheus histogram with the label names model_name and engine under the PrometheusStatLogger V1 metrics publisher.
Its registration in v0.23.0 declares 22 literal histogram bucket boundaries spanning 0.001 to 2560.0.
In v0.23.0 it observes the elapsed seconds from request arrival to the first produced token, recording an observation when iteration processing identifies the request as prefilling/producing that first token.
Because its count in v0.23.0 follows first-token events, it need not equal a finished-request counter across abort and error paths, so quantiles should use matched labels and windows and lifecycle population should be reconciled before comparing the count with request_success.

Identity

Prometheus type: histogram
Labels: model_nameengine
Buckets: 22 boundaries · 0.001 → 2560
Unit: Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance

Metric knowledge (7)

Claim	Semantic layer	Verified
In vLLM v0.23.0, `vllm:time_to_first_token_seconds` observes elapsed seconds from request arrival to the first produced token.	Measurement semantics	Sep 29
In vLLM v0.23.0, `vllm:time_to_first_token_seconds` adds an observation when iteration processing identifies the request as prefilling/producing its first token.	Causal/runtime semantics	Sep 29
In vLLM v0.23.0, `vllm:time_to_first_token_seconds` its histogram count follows first-token events and need not equal a finished-request counter across abort and error paths.	Interpretation & composition	Sep 29
In vLLM v0.23.0, `vllm:time_to_first_token_seconds` use histogram quantiles with matched labels and windows; reconcile lifecycle population before comparing its count with `request_success`.	Guarded guidance	Sep 29
In vLLM v0.23.0, `vllm:time_to_first_token_seconds` is registered as a Prometheus histogram.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:time_to_first_token_seconds` is registered with the label names `model_name`, engine.	Structural identity	Sep 29
In vLLM v0.23.0, `vllm:time_to_first_token_seconds` declares 22 literal histogram bucket boundaries in its registration, from 0.001 to 2560.0.	Structural identity	Sep 29

Related Use Cases (2)

Availability

vllm:time_to_first_token_seconds is registered as a Prometheus histogram in vLLM v0.23.0.

vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:754-785