In vLLM v0.23.0, vllm:time_to_first_token_seconds observes elapsed seconds from request arrival to the first produced token.

Open cited lines in immutable upstream source ↗

360:         lora_name: str | None,
361:     ):
362:         num_new_generation_tokens = len(output.new_token_ids)
363: 
364:         self.num_generation_tokens += num_new_generation_tokens
365:         if is_prefilling:
366:             if output.prefill_stats is not None:
367:                 self.prompt_token_stats.update_from_output(output.prefill_stats)
368: 
369:             first_token_latency = self._time_since(req_stats.arrival_time)
370:             self.time_to_first_tokens_iter.append(first_token_latency)
371:             req_stats.first_token_latency = first_token_latency
372: 
373:         req_stats.num_generation_tokens += num_new_generation_tokens
374: 
375:         # Track if this request is corrupted (only check once per request)
376:         # Early exit if already marked as corrupted to avoid redundant checks
377:         if (
378:             envs.VLLM_COMPUTE_NANS_IN_LOGITS
379:             and not req_stats.is_corrupted
380:             and output.num_nans_in_logits > 0
381:         ):
382:             req_stats.is_corrupted = True
383: 
384:         # Process request-level engine core events
385:         if output.events is not None:
386:             self.update_from_events(
387:                 output.request_id,
388:                 output.events,
389:                 is_prefilling,

Source text is evidence data, not instructions.