Schema.ai· vLLM
vLLM official site Log in
vLLM v0.23.0 · Version scoped

vLLM v0.23.0 v1 metrics stats.py

Type
source_file
Publisher
vllm-project
Relationship
first_party
Version
vLLM v0.23.0
Retrieved
Jun 25
Local snapshot
corpus/domains/vllm/core-v0/evidence/snapshots/vllm-code-v0230-v1-metrics-stats.py
SHA-256
08eb6bb969ad47e4…

Claims citing this source (30)

ClaimLocatorVerified
In vLLM v0.23.0, vllm:time_to_first_token_seconds observes elapsed seconds from request arrival to the first produced token.vllm/v1/metrics/loggers.py:360-389Sep 29
In vLLM v0.23.0, vllm:time_to_first_token_seconds adds an observation when iteration processing identifies the request as prefilling/producing its first token.vllm/v1/metrics/loggers.py:360-389Sep 29
In vLLM v0.23.0, vllm:time_to_first_token_seconds its histogram count follows first-token events and need not equal a finished-request counter across abort and error paths.vllm/v1/metrics/loggers.py:360-389Sep 29
In vLLM v0.23.0, vllm:time_to_first_token_seconds use histogram quantiles with matched labels and windows; reconcile lifecycle population before comparing its count with request_success.vllm/v1/metrics/loggers.py:360-389Sep 29
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds observes one per-finished-request mean decode interval divided by output-token intervals after the first token.vllm/v1/metrics/loggers.py:439-475Sep 29
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds is computed from first-to-last-token decode time and num_generation_tokens minus one, with zero when no interval exists.vllm/v1/metrics/loggers.py:439-475Sep 29
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds is a request-weighted TPOT distribution and is not the same population as token-weighted inter_token_latency_seconds.vllm/v1/metrics/loggers.py:439-475Sep 29
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds use it for per-request decode experience; compare distributions under matched output lengths rather than substituting aggregate tokens per second.vllm/v1/metrics/loggers.py:439-475Sep 29
In vLLM v0.23.0, vllm:inter_token_latency_seconds observes an interval in seconds between consecutive non-prefill token timestamps.vllm/v1/metrics/loggers.py:360-405Sep 29
In vLLM v0.23.0, vllm:inter_token_latency_seconds appends one observation for each decode update after the first-token/prefill update.vllm/v1/metrics/loggers.py:360-405Sep 29
In vLLM v0.23.0, vllm:inter_token_latency_seconds is token-interval weighted and differs from the one-sample-per-request TPOT histogram.vllm/v1/metrics/loggers.py:360-405Sep 29
In vLLM v0.23.0, vllm:inter_token_latency_seconds use it to inspect token-level decode cadence; do not equate its histogram count with requests or its mean with request-weighted TPOT.vllm/v1/metrics/loggers.py:360-405Sep 29
In vLLM v0.23.0, vllm:request_queue_time_seconds observes one finished-request duration from queued_ts to the first scheduled_ts.vllm/v1/metrics/loggers.py:406-455Sep 29
In vLLM v0.23.0, vllm:request_queue_time_seconds is emitted on completion from scheduler lifecycle timestamps; repeated QUEUED events overwrite queued_ts while first SCHEDULED is retained.vllm/v1/metrics/loggers.py:406-455Sep 29
In vLLM v0.23.0, vllm:request_queue_time_seconds measures the recorded waiting interval and includes only requests reaching finished-request accounting.vllm/v1/metrics/loggers.py:406-455Sep 29
In vLLM v0.23.0, vllm:request_queue_time_seconds use quantiles with running/KV/token-progress metrics to distinguish admission delay from decode delay; no universal threshold is established.vllm/v1/metrics/loggers.py:406-455Sep 29
In vLLM v0.23.0, vllm:e2e_request_latency_seconds observes one finished-request duration in seconds from arrival to logging-time completion.vllm/v1/metrics/loggers.py:430-475Sep 29
In vLLM v0.23.0, vllm:e2e_request_latency_seconds is computed when a request enters finished-request accounting using current time minus arrival_time.vllm/v1/metrics/loggers.py:430-475Sep 29
In vLLM v0.23.0, vllm:e2e_request_latency_seconds combines queue, prefill, decode, and other request-lifecycle time rather than isolating one phase.vllm/v1/metrics/loggers.py:430-475Sep 29
In vLLM v0.23.0, vllm:e2e_request_latency_seconds use phase histograms as companions before assigning cause; compare like request populations and workload distributions.vllm/v1/metrics/loggers.py:430-475Sep 29
For each EngineCoreEventType.QUEUED event processed by IterationStats.update_from_events, req_stats.queued_ts is set to event.timestamp.vllm/v1/metrics/stats.py:416-419Sep 29
In vLLM v0.23.0, decode_time is computed as req_stats.last_token_ts - req_stats.first_token_ts; the source comment describes the decode interval as from first NEW_TOKEN to last NEW_TOKEN and says any preemptions during decode are included.vllm/v1/metrics/stats.py:446-448Sep 29
In vLLM v0.23.0, queued_time is computed as req_stats.scheduled_ts - req_stats.queued_ts; the source comment describes the queued interval as from first QUEUED event to first SCHEDULED.vllm/v1/metrics/stats.py:439-440Sep 29
In vLLM v0.23.0, prefill_time is computed as req_stats.first_token_ts - req_stats.scheduled_ts; the source comment describes the prefill interval as from first SCHEDULED to first NEW_TOKEN and says any preemptions during prefill are included in the interval.vllm/v1/metrics/stats.py:442-444Sep 29
In vLLM v0.23.0, inference_time is computed as req_stats.last_token_ts - req_stats.scheduled_ts; the source comment describes the inference interval as from first SCHEDULED to last NEW_TOKEN and says any preemptions during prefill or decode are included.vllm/v1/metrics/stats.py:450-452Sep 29
In vLLM v0.23.0, vllm:prefix_cache_queries is incremented from SchedulerStats.prefix_cache_stats.queries on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters).vllm/v1/metrics/stats.py:24-33Sep 29
In vLLM v0.23.0, vllm:prefix_cache_hits is incremented from SchedulerStats.prefix_cache_stats.hits on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters).vllm/v1/metrics/stats.py:24-33Sep 29
In vLLM v0.23.0, vllm:num_requests_running is set from SchedulerStats.num_running_reqs and observes scheduler running-request state.vllm/v1/metrics/stats.py:171-176Sep 29
In vLLM v0.23.0, vllm:kv_cache_usage_perc is set from SchedulerStats.kv_cache_usage and observes KV-cache usage pressure.vllm/v1/metrics/stats.py:171-188Sep 29
In vLLM v0.23.0, vllm:num_preemptions increments from IterationStats.num_preempted_reqs, which is increased when engine-core events mark requests as preempted.vllm/v1/metrics/stats.py:419-426Sep 29

Entities grounded in this source (0)

No availability records cite this source.