Schema.ai· vLLM· Schema 0.2.1
vLLM official site vLLM v0.23.0Log in

#8115: TTFT observations and completed requests count different events

Histogram count is not automatically a request counter

GitHub #8115retrievalhonestyvLLM v0.23.0

Reported environment

A scrape where TTFT histogram count exceeded the sum of request-success series.

The nuance

TTFT is observed on first-token production; request_success increments from finished requests, so lifecycle and failure paths can separate them.

Expected behavior

Trace both update events and require reconciliation of lifecycle, labels, process scope, and abnormal outcomes.

What Schema establishes

Schema proves different update events; a particular delta needs deployment observations.

Runtime evidence still needed

  • Counts by finish reason process and restart window
  • Lifecycle test including abort error streaming and fan-out

Try the questions (2)

Each question runs live against the product.

Why is time_to_first_token_seconds_count greater than request_success_totalrun →Can TTFT count stand for completed requestsrun →

Grounding claims

Topics, entities, and gaps