In vLLM v0.23.0, vllm:prefix_cache_queries counts tokens presented to the first-party prefix-cache lookup, accumulated as a counter. | v0.23.0 | Metric | prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_queries increments from SchedulerStats.prefix_cache_stats.queries on each scheduler-stats logging update. | v0.23.0 | Metric | prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_queries is the token denominator for a same-label, same-window prefix-cache hit ratio. | v0.23.0 | Metric | prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_queries use rate or increase over a chosen window with prefix_cache_hits; do not interpret the cumulative total as a rate. | v0.23.0 | Metric | prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits counts queried tokens found in the first-party prefix cache, accumulated as a counter. | v0.23.0 | Metric | prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits increments from SchedulerStats.prefix_cache_stats.hits on each scheduler-stats logging update. | v0.23.0 | Metric | prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits is the token numerator for a same-label, same-window prefix-cache hit ratio. | v0.23.0 | Metric | prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits divide same-window hit and query rates; no universal healthy percentage follows because reuse is workload-dependent. | v0.23.0 | Metric | prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc is a point-in-time gauge where 1 represents fully used KV block capacity. | v0.23.0 | Metric | kv_cache_usage_perc | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc is set from SchedulerStats.kv_cache_usage rather than incremented from completed requests. | v0.23.0 | Metric | kv_cache_usage_perc | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc describes active used capacity and is not a counter of reusable cached prefixes or completed work. | v0.23.0 | Metric | kv_cache_usage_perc | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc use it as a pressure signal alongside waiting and progress metrics; do not infer cache effectiveness from it alone. | v0.23.0 | Metric | kv_cache_usage_perc | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens_cached counts cached prompt tokens reused during processing, combining local and external sources. | v0.23.0 | Metric | prompt_tokens_cached | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens_cached increments from the iteration PromptTokenStats.cached_tokens field. | v0.23.0 | Metric | prompt_tokens_cached | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens_cached is aggregate saved prompt-token work and is not by itself a per-request cache-hit percentage. | v0.23.0 | Metric | prompt_tokens_cached | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens_cached compare its rate with prompt-token demand under matching labels and windows; separate local/external attribution when required. | v0.23.0 | Metric | prompt_tokens_cached | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens observes one histogram sample per finished request: prompt tokens minus non-negative cached tokens. | v0.23.0 | Metric | request_prefill_kv_computed_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens is computed in finished-request logging after request completion from num_prompt_tokens and num_cached_tokens. | v0.23.0 | Metric | request_prefill_kv_computed_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens measures newly computed prefill KV work per completed request, not total prompt length when caching contributes. | v0.23.0 | Metric | request_prefill_kv_computed_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens compare distributions under matched workloads; use cached-token counters as companions and do not treat one bucket count as token throughput. | v0.23.0 | Metric | request_prefill_kv_computed_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:time_to_first_token_seconds observes elapsed seconds from request arrival to the first produced token. | v0.23.0 | Metric | time_to_first_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:time_to_first_token_seconds adds an observation when iteration processing identifies the request as prefilling/producing its first token. | v0.23.0 | Metric | time_to_first_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:time_to_first_token_seconds its histogram count follows first-token events and need not equal a finished-request counter across abort and error paths. | v0.23.0 | Metric | time_to_first_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:time_to_first_token_seconds use histogram quantiles with matched labels and windows; reconcile lifecycle population before comparing its count with request_success. | v0.23.0 | Metric | time_to_first_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_success counts finished engine requests partitioned by finished_reason. | v0.23.0 | Metric | request_success | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_success increments once for each FinishedRequestStats record consumed by the logger. | v0.23.0 | Metric | request_success | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_success is a completed-engine-request population and does not cover requests rejected before engine completion. | v0.23.0 | Metric | request_success | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_success sum only intended finish reasons for the question; use API or ingress telemetry for a denominator covering every HTTP attempt. | v0.23.0 | Metric | request_success | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running is a point-in-time gauge of requests currently running in model execution batches. | v0.23.0 | Metric | num_requests_running | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running is set from scheduler running-request state at a logging snapshot. | v0.23.0 | Metric | num_requests_running | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running measures concurrency rather than arrivals, completions, successes, or failures. | v0.23.0 | Metric | num_requests_running | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running use it with waiting, KV pressure, and token progress to describe load; never use it as an error-rate denominator. | v0.23.0 | Metric | num_requests_running | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds observes one per-finished-request mean decode interval divided by output-token intervals after the first token. | v0.23.0 | Metric | request_time_per_output_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds is computed from first-to-last-token decode time and num_generation_tokens minus one, with zero when no interval exists. | v0.23.0 | Metric | request_time_per_output_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds is a request-weighted TPOT distribution and is not the same population as token-weighted inter_token_latency_seconds. | v0.23.0 | Metric | request_time_per_output_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds use it for per-request decode experience; compare distributions under matched output lengths rather than substituting aggregate tokens per second. | v0.23.0 | Metric | request_time_per_output_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:inter_token_latency_seconds observes an interval in seconds between consecutive non-prefill token timestamps. | v0.23.0 | Metric | inter_token_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:inter_token_latency_seconds appends one observation for each decode update after the first-token/prefill update. | v0.23.0 | Metric | inter_token_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:inter_token_latency_seconds is token-interval weighted and differs from the one-sample-per-request TPOT histogram. | v0.23.0 | Metric | inter_token_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:inter_token_latency_seconds use it to inspect token-level decode cadence; do not equate its histogram count with requests or its mean with request-weighted TPOT. | v0.23.0 | Metric | inter_token_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_queue_time_seconds observes one finished-request duration from queued_ts to the first scheduled_ts. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_queue_time_seconds is emitted on completion from scheduler lifecycle timestamps; repeated QUEUED events overwrite queued_ts while first SCHEDULED is retained. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_queue_time_seconds measures the recorded waiting interval and includes only requests reaching finished-request accounting. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_queue_time_seconds use quantiles with running/KV/token-progress metrics to distinguish admission delay from decode delay; no universal threshold is established. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:e2e_request_latency_seconds observes one finished-request duration in seconds from arrival to logging-time completion. | v0.23.0 | Metric | e2e_request_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:e2e_request_latency_seconds is computed when a request enters finished-request accounting using current time minus arrival_time. | v0.23.0 | Metric | e2e_request_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:e2e_request_latency_seconds combines queue, prefill, decode, and other request-lifecycle time rather than isolating one phase. | v0.23.0 | Metric | e2e_request_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:e2e_request_latency_seconds use phase histograms as companions before assigning cause; compare like request populations and workload distributions. | v0.23.0 | Metric | e2e_request_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:generation_tokens counts generation tokens processed across iterations. | v0.23.0 | Metric | generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:generation_tokens increments from IterationStats.num_generation_tokens during iteration logging. | v0.23.0 | Metric | generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:generation_tokens is aggregate token work and not a per-request latency, fairness, or completion measure. | v0.23.0 | Metric | generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:generation_tokens use rate for aggregate decode throughput and pair it with ITL/TPOT/TTFT distributions before judging user experience. | v0.23.0 | Metric | generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens counts prefill tokens processed across iterations. | v0.23.0 | Metric | prompt_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens increments from IterationStats.num_prompt_tokens during iteration logging. | v0.23.0 | Metric | prompt_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens is aggregate prefill work and does not by itself distinguish cached from newly computed prompt tokens. | v0.23.0 | Metric | prompt_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens use rate for prompt throughput and pair it with cached/computed-token and queue/TTFT metrics before drawing efficiency conclusions. | v0.23.0 | Metric | prompt_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:iteration_tokens_total observes one histogram sample per engine step equal to computed prompt tokens plus generated tokens. | v0.23.0 | Metric | iteration_tokens_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:iteration_tokens_total is populated from iteration-level work after each IterationStats update. | v0.23.0 | Metric | iteration_tokens_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:iteration_tokens_total describes batch/step work distribution rather than a cumulative token counter despite total appearing in its declared name. | v0.23.0 | Metric | iteration_tokens_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:iteration_tokens_total use its distribution to inspect batching; do not calculate token throughput from bucket counts without elapsed-time counters. | v0.23.0 | Metric | iteration_tokens_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:cache_config_info is registered as a Prometheus gauge. | v0.23.0 | Metric | cache_config_info | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:corrupted_requests is registered as a Prometheus counter when `VLLM_COMPUTE_NANS_IN_LOGITS=1`. | v0.23.0 | Metric | corrupted_requests | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:corrupted_requests is registered with the label names model_name, engine. | v0.23.0 | Metric | corrupted_requests | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:e2e_request_latency_seconds is registered as a Prometheus histogram. | v0.23.0 | Metric | e2e_request_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:e2e_request_latency_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | e2e_request_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:e2e_request_latency_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0. | v0.23.0 | Metric | e2e_request_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:engine_sleep_state is registered as a Prometheus gauge. | v0.23.0 | Metric | engine_sleep_state | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:engine_sleep_state is registered with the label names model_name, engine, sleep_state. | v0.23.0 | Metric | engine_sleep_state | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:estimated_flops_per_gpu_total is registered as a Prometheus counter. | v0.23.0 | Metric | estimated_flops_per_gpu_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:estimated_read_bytes_per_gpu_total is registered as a Prometheus counter. | v0.23.0 | Metric | estimated_read_bytes_per_gpu_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:estimated_write_bytes_per_gpu_total is registered as a Prometheus counter. | v0.23.0 | Metric | estimated_write_bytes_per_gpu_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:external_prefix_cache_hits is registered as a Prometheus counter. | v0.23.0 | Metric | external_prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:external_prefix_cache_hits is registered with the label names model_name, engine. | v0.23.0 | Metric | external_prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:external_prefix_cache_queries is registered as a Prometheus counter. | v0.23.0 | Metric | external_prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:external_prefix_cache_queries is registered with the label names model_name, engine. | v0.23.0 | Metric | external_prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:generation_tokens is registered as a Prometheus counter. | v0.23.0 | Metric | generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:generation_tokens is registered with the label names model_name, engine. | v0.23.0 | Metric | generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:inter_token_latency_seconds is registered as a Prometheus histogram. | v0.23.0 | Metric | inter_token_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:inter_token_latency_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | inter_token_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:inter_token_latency_seconds declares 19 literal histogram bucket boundaries in its registration, from 0.01 to 80.0. | v0.23.0 | Metric | inter_token_latency_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:iteration_tokens_total is registered as a Prometheus histogram. | v0.23.0 | Metric | iteration_tokens_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:iteration_tokens_total is registered with the label names model_name, engine. | v0.23.0 | Metric | iteration_tokens_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:iteration_tokens_total declares 13 literal histogram bucket boundaries in its registration, from 1.0 to 16384.0. | v0.23.0 | Metric | iteration_tokens_total | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_idle_before_evict_seconds is registered as a Prometheus histogram when `observability_config.kv_cache_metrics=True`. | v0.23.0 | Metric | kv_block_idle_before_evict_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_idle_before_evict_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | kv_block_idle_before_evict_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_idle_before_evict_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.001 to 1800.0. | v0.23.0 | Metric | kv_block_idle_before_evict_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_lifetime_seconds is registered as a Prometheus histogram when `observability_config.kv_cache_metrics=True`. | v0.23.0 | Metric | kv_block_lifetime_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_lifetime_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | kv_block_lifetime_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_lifetime_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.001 to 1800.0. | v0.23.0 | Metric | kv_block_lifetime_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_reuse_gap_seconds is registered as a Prometheus histogram when `observability_config.kv_cache_metrics=True`. | v0.23.0 | Metric | kv_block_reuse_gap_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_reuse_gap_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | kv_block_reuse_gap_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_block_reuse_gap_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.001 to 1800.0. | v0.23.0 | Metric | kv_block_reuse_gap_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc is registered with the label names model_name, engine. | v0.23.0 | Metric | kv_cache_usage_perc | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:lora_requests_info is registered as a Prometheus gauge when `lora_config is not None`. | v0.23.0 | Metric | lora_requests_info | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:mm_cache_hits is registered as a Prometheus counter. | v0.23.0 | Metric | mm_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:mm_cache_hits is registered with the label names model_name, engine. | v0.23.0 | Metric | mm_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:mm_cache_queries is registered as a Prometheus counter. | v0.23.0 | Metric | mm_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:mm_cache_queries is registered with the label names model_name, engine. | v0.23.0 | Metric | mm_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_preemptions is registered with the label names model_name, engine. | v0.23.0 | Metric | num_preemptions | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running is registered with the label names model_name, engine. | v0.23.0 | Metric | num_requests_running | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_waiting is registered with the label names model_name, engine. | v0.23.0 | Metric | num_requests_waiting | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_waiting_by_reason is registered with the label names model_name, engine, reason. | v0.23.0 | Metric | num_requests_waiting_by_reason | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits is registered with the label names model_name, engine. | v0.23.0 | Metric | prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_queries is registered with the label names model_name, engine. | v0.23.0 | Metric | prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens is registered as a Prometheus counter. | v0.23.0 | Metric | prompt_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens is registered with the label names model_name, engine. | v0.23.0 | Metric | prompt_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens_by_source is registered as a Prometheus counter. | v0.23.0 | Metric | prompt_tokens_by_source | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens_by_source is registered with the label names model_name, engine, source. | v0.23.0 | Metric | prompt_tokens_by_source | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens_cached is registered as a Prometheus counter. | v0.23.0 | Metric | prompt_tokens_cached | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prompt_tokens_cached is registered with the label names model_name, engine. | v0.23.0 | Metric | prompt_tokens_cached | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_decode_time_seconds is registered as a Prometheus histogram. | v0.23.0 | Metric | request_decode_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_decode_time_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | request_decode_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_decode_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0. | v0.23.0 | Metric | request_decode_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_generation_tokens is registered as a Prometheus histogram. | v0.23.0 | Metric | request_generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_generation_tokens is registered with the label names model_name, engine. | v0.23.0 | Metric | request_generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_inference_time_seconds is registered as a Prometheus histogram. | v0.23.0 | Metric | request_inference_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_inference_time_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | request_inference_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_inference_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0. | v0.23.0 | Metric | request_inference_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_max_num_generation_tokens is registered as a Prometheus histogram. | v0.23.0 | Metric | request_max_num_generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_max_num_generation_tokens is registered with the label names model_name, engine. | v0.23.0 | Metric | request_max_num_generation_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_params_max_tokens is registered as a Prometheus histogram. | v0.23.0 | Metric | request_params_max_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_params_max_tokens is registered with the label names model_name, engine. | v0.23.0 | Metric | request_params_max_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_params_n is registered as a Prometheus histogram. | v0.23.0 | Metric | request_params_n | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_params_n is registered with the label names model_name, engine. | v0.23.0 | Metric | request_params_n | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_params_n declares 5 literal histogram bucket boundaries in its registration, from 1.0 to 20.0. | v0.23.0 | Metric | request_params_n | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens is registered as a Prometheus histogram. | v0.23.0 | Metric | request_prefill_kv_computed_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens is registered with the label names model_name, engine. | v0.23.0 | Metric | request_prefill_kv_computed_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_time_seconds is registered as a Prometheus histogram. | v0.23.0 | Metric | request_prefill_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_time_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | request_prefill_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prefill_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0. | v0.23.0 | Metric | request_prefill_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prompt_tokens is registered as a Prometheus histogram. | v0.23.0 | Metric | request_prompt_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_prompt_tokens is registered with the label names model_name, engine. | v0.23.0 | Metric | request_prompt_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_queue_time_seconds is registered as a Prometheus histogram. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_queue_time_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_queue_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_success is registered as a Prometheus counter. | v0.23.0 | Metric | request_success | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_success is registered with the label names model_name, engine, finished_reason. | v0.23.0 | Metric | request_success | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds is registered as a Prometheus histogram. | v0.23.0 | Metric | request_time_per_output_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | request_time_per_output_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds declares 19 literal histogram bucket boundaries in its registration, from 0.01 to 80.0. | v0.23.0 | Metric | request_time_per_output_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:spec_decode_num_accepted_tokens is registered as a Prometheus counter when `speculative_config is not None`. | v0.23.0 | Metric | spec_decode_num_accepted_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:spec_decode_num_accepted_tokens_per_pos is registered as a Prometheus counter when `speculative_config is not None`. | v0.23.0 | Metric | spec_decode_num_accepted_tokens_per_pos | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:spec_decode_num_draft_tokens is registered as a Prometheus counter when `speculative_config is not None`. | v0.23.0 | Metric | spec_decode_num_draft_tokens | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:spec_decode_num_drafts is registered as a Prometheus counter when `speculative_config is not None`. | v0.23.0 | Metric | spec_decode_num_drafts | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:time_to_first_token_seconds is registered as a Prometheus histogram. | v0.23.0 | Metric | time_to_first_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:time_to_first_token_seconds is registered with the label names model_name, engine. | v0.23.0 | Metric | time_to_first_token_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:time_to_first_token_seconds declares 22 literal histogram bucket boundaries in its registration, from 0.001 to 2560.0. | v0.23.0 | Metric | time_to_first_token_seconds | 10 hr ago | github.com |
For each EngineCoreEventType.QUEUED event processed by IterationStats.update_from_events, req_stats.queued_ts is set to event.timestamp. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, decode_time is computed as req_stats.last_token_ts - req_stats.first_token_ts; the source comment describes the decode interval as from first NEW_TOKEN to last NEW_TOKEN and says any preemptions during decode are included. | v0.23.0 | Metric | request_decode_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, queued_time is computed as req_stats.scheduled_ts - req_stats.queued_ts; the source comment describes the queued interval as from first QUEUED event to first SCHEDULED. | v0.23.0 | Metric | request_queue_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, prefill_time is computed as req_stats.first_token_ts - req_stats.scheduled_ts; the source comment describes the prefill interval as from first SCHEDULED to first NEW_TOKEN and says any preemptions during prefill are included in the interval. | v0.23.0 | Metric | request_prefill_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, inference_time is computed as req_stats.last_token_ts - req_stats.scheduled_ts; the source comment describes the inference interval as from first SCHEDULED to last NEW_TOKEN and says any preemptions during prefill or decode are included. | v0.23.0 | Metric | request_inference_time_seconds | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_queries is a Prometheus counter measured in queried tokens — it counts tokens, not requests or cache blocks. | v0.23.0 | Metric | prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — it counts tokens, not requests or cache blocks. | v0.23.0 | Metric | prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_queries is incremented from SchedulerStats.prefix_cache_stats.queries on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters). | v0.23.0 | Metric | prefix_cache_queries | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:prefix_cache_hits is incremented from SchedulerStats.prefix_cache_stats.hits on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters). | v0.23.0 | Metric | prefix_cache_hits | 10 hr ago | github.com |
The vLLM v0.23.0 metrics design docs state the metric of interest is the prefix cache hit rate (hits per query): the counters are exposed raw so operators compute the rate over an interval of their choosing with PromQL (rate(hits)/rate(queries)); vLLM's own logging equivalent aggregates hit_rate over the most recent queries. | v0.23.0 | Metric | prefix_cache_hits | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running is a Prometheus gauge for the number of requests in model execution batches. | v0.23.0 | Metric | num_requests_running | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_running is set from SchedulerStats.num_running_reqs and observes scheduler running-request state. | v0.23.0 | Metric | num_requests_running | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_waiting is a Prometheus gauge for the number of requests waiting to be processed. | v0.23.0 | Metric | num_requests_waiting | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_requests_waiting_by_reason is a Prometheus gauge with reason labels capacity and deferred, and its reasons sum to vllm:num_requests_waiting. | v0.23.0 | Metric | num_requests_waiting_by_reason | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc is a Prometheus gauge for KV-cache usage where 1 means 100 percent usage. | v0.23.0 | Metric | kv_cache_usage_perc | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:kv_cache_usage_perc is set from SchedulerStats.kv_cache_usage and observes KV-cache usage pressure. | v0.23.0 | Metric | kv_cache_usage_perc | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_preemptions is a Prometheus counter for cumulative preemptions from the engine. | v0.23.0 | Metric | num_preemptions | 10 hr ago | github.com |
In vLLM v0.23.0, vllm:num_preemptions increments from IterationStats.num_preempted_reqs, which is increased when engine-core events mark requests as preempted. | v0.23.0 | Metric | num_preemptions | 10 hr ago | github.com |
Schema.ai· vLLM· Schema 0.2.1