Schema.ai· vLLM
vLLM official site Log in
vLLM v0.23.0 · Version scoped

vllm:generation_tokens

Counter for the number of generation tokens processed.

MetricPrometheusverified Sep 29

Summary

In vLLM v0.23.0, with V1 metrics published by PrometheusStatLogger, vllm:generation_tokens is registered as a Prometheus counter. It is registered with the label names model_name and engine. The metric counts generation tokens processed across iterations. Under the same V1 PrometheusStatLogger path, it increments from IterationStats.num_generation_tokens during iteration logging. Interpretively, it represents aggregate token work rather than a per-request latency, fairness, or completion measure. For guidance in this version, use its rate for aggregate decode throughput and pair it with ITL/TPOT/TTFT distributions before judging user experience.

Version history

  • In vLLM v0.23.0 — the only version covered here — vllm:generation_tokens is registered as a Prometheus counter with the label names model_name and engine under the PrometheusStatLogger V1 metrics publisher.
  • In that same v0.23.0 release, the counter tracks generation tokens processed across iterations, incrementing from IterationStats.num_generation_tokens during iteration logging.
  • In v0.23.0 it represents aggregate token work rather than a per-request latency, fairness, or completion measure.
  • Accordingly, v0.23.0 guidance is to apply rate for aggregate decode throughput and to pair it with ITL/TPOT/TTFT distributions before judging user experience.

Identity

Prometheus type
counter
Labels
model_nameengine
Unit
Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance

Metric knowledge (6)

ClaimSemantic layerVerified
In vLLM v0.23.0, vllm:generation_tokens counts generation tokens processed across iterations.Measurement semanticsSep 29
In vLLM v0.23.0, vllm:generation_tokens increments from IterationStats.num_generation_tokens during iteration logging.Causal/runtime semanticsSep 29
In vLLM v0.23.0, vllm:generation_tokens is aggregate token work and not a per-request latency, fairness, or completion measure.Interpretation & compositionSep 29
In vLLM v0.23.0, vllm:generation_tokens use rate for aggregate decode throughput and pair it with ITL/TPOT/TTFT distributions before judging user experience.Guarded guidanceSep 29
In vLLM v0.23.0, vllm:generation_tokens is registered as a Prometheus counter.Structural identitySep 29
In vLLM v0.23.0, vllm:generation_tokens is registered with the label names model_name, engine.Structural identitySep 29

Related Use Cases (1)

Availability

vllm:generation_tokens is registered as a Prometheus counter in vLLM v0.23.0.
vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:662-669