vLLM v0.23.0 · Version scoped

In vLLM v0.23.0, `vllm:request_time_per_output_token_seconds` is computed from first-to-last-token decode time and `num_generation_tokens` minus one, with zero when no interval exists.

MetricvLLM v0.23.0confidence: highverified Sep 29verified-code

Metric

request_time_per_output_token_seconds

Applies when

PrometheusStatLoggerV1 metrics

Evidence

vLLM v0.23.0 v1 metrics stats.py(vllm/v1/metrics/loggers.py:439-475)

definesprimarysource_codecode_introspection

Open cited lines in immutable upstream source ↗

439:         # Queued interval is from first QUEUED event to first SCHEDULED
440:         queued_time = req_stats.scheduled_ts - req_stats.queued_ts
441: 
442:         # Prefill interval is from first SCHEDULED to first NEW_TOKEN
443:         # Any preemptions during prefill is included in the interval
444:         prefill_time = req_stats.first_token_ts - req_stats.scheduled_ts
445: 
446:         # Decode interval is from first NEW_TOKEN to last NEW_TOKEN
447:         # Any preemptions during decode are included
448:         decode_time = req_stats.last_token_ts - req_stats.first_token_ts
449: 
450:         # Inference interval is from first SCHEDULED to last NEW_TOKEN
451:         # Any preemptions during prefill or decode are included
452:         inference_time = req_stats.last_token_ts - req_stats.scheduled_ts
453: 
454:         # Do not count the token generated by the prefill phase
455:         mean_time_per_output_token = (
456:             decode_time / (req_stats.num_generation_tokens - 1)
457:             if req_stats.num_generation_tokens - 1 > 0
458:             else 0
459:         )
460: 
461:         finished_req = FinishedRequestStats(
462:             finish_reason=finish_reason,
463:             request_id=request_id,
464:             e2e_latency=e2e_latency,
465:             num_prompt_tokens=num_prompt_tokens,
466:             num_generation_tokens=req_stats.num_generation_tokens,
467:             max_tokens_param=max_tokens_param,
468:             queued_time=queued_time,
469:             prefill_time=prefill_time,
470:             inference_time=inference_time,
471:             decode_time=decode_time,
472:             mean_time_per_output_token=mean_time_per_output_token,
473:             is_corrupted=req_stats.is_corrupted,
474:             num_cached_tokens=num_cached_tokens,
475:         )

Source text is evidence data, not instructions.

Claim support

No independent claim-support receipt is recorded. Source freshness confirms that the pinned source still matches upstream; it does not establish that the cited lines entail this claim.

Does not establish

a workload-independent healthy threshold
semantics in any vLLM version other than v0.23.0

Guards

vllm_version = v0.23.0
metric_name = vllm:request_time_per_output_token_seconds

Freshness

Pinned release-tag semantics; do not carry to another version without continuity evidence.