In vLLM v0.23.0, vllm:request_queue_time_seconds observes one finished-request duration from queued_ts to the first scheduled_ts.

Open cited lines in immutable upstream source ↗

406:         req_id: str,
407:         events: list["EngineCoreEvent"],
408:         is_prefilling: bool,
409:         req_stats: RequestStateStats,
410:         lora_states: "LoRARequestStates",
411:         lora_name: str | None,
412:     ):
413:         # Avoid circular dependency
414:         from vllm.v1.engine import EngineCoreEventType
415: 
416:         for event in events:
417:             if event.type == EngineCoreEventType.QUEUED:
418:                 req_stats.queued_ts = event.timestamp
419:                 lora_states.request_waiting(req_id, lora_name)
420:             elif event.type == EngineCoreEventType.SCHEDULED:
421:                 if req_stats.scheduled_ts == 0.0:  # ignore preemptions
422:                     req_stats.scheduled_ts = event.timestamp
423:                 lora_states.request_running(req_id, lora_name)
424:             elif event.type == EngineCoreEventType.PREEMPTED:
425:                 self.num_preempted_reqs += 1
426:                 lora_states.request_waiting(req_id, lora_name)
427: 
428:     def update_from_finished_request(
429:         self,
430:         finish_reason: "FinishReason",
431:         request_id: str,
432:         num_prompt_tokens: int,
433:         max_tokens_param: int | None,
434:         req_stats: RequestStateStats,
435:         num_cached_tokens: int = 0,
436:     ):
437:         e2e_latency = self._time_since(req_stats.arrival_time)
438: 
439:         # Queued interval is from first QUEUED event to first SCHEDULED
440:         queued_time = req_stats.scheduled_ts - req_stats.queued_ts
441: 
442:         # Prefill interval is from first SCHEDULED to first NEW_TOKEN
443:         # Any preemptions during prefill is included in the interval
444:         prefill_time = req_stats.first_token_ts - req_stats.scheduled_ts
445: 
446:         # Decode interval is from first NEW_TOKEN to last NEW_TOKEN
447:         # Any preemptions during decode are included
448:         decode_time = req_stats.last_token_ts - req_stats.first_token_ts
449: 
450:         # Inference interval is from first SCHEDULED to last NEW_TOKEN
451:         # Any preemptions during prefill or decode are included
452:         inference_time = req_stats.last_token_ts - req_stats.scheduled_ts
453: 
454:         # Do not count the token generated by the prefill phase
455:         mean_time_per_output_token = (

Source text is evidence data, not instructions.