Schema.ai· vLLM
vLLM official site Log in
vLLM v0.23.0 · Version scoped

vLLM v0.23.0 v1 metrics loggers.py

Type
source_file
Publisher
vllm-project
Relationship
first_party
Version
vLLM v0.23.0
Retrieved
Jun 25
Local snapshot
corpus/domains/vllm/core-v0/evidence/snapshots/vllm-code-v0230-v1-metrics-loggers.py
SHA-256
ce2a0535d2b2b69e…

Claims citing this source (128)

ClaimLocatorVerified
In vLLM v0.23.0, vllm:prefix_cache_queries counts tokens presented to the first-party prefix-cache lookup, accumulated as a counter.vllm/v1/metrics/loggers.py:562-588Sep 29
In vLLM v0.23.0, vllm:prefix_cache_queries increments from SchedulerStats.prefix_cache_stats.queries on each scheduler-stats logging update.vllm/v1/metrics/loggers.py:562-588Sep 29
In vLLM v0.23.0, vllm:prefix_cache_queries is the token denominator for a same-label, same-window prefix-cache hit ratio.vllm/v1/metrics/loggers.py:562-588Sep 29
In vLLM v0.23.0, vllm:prefix_cache_queries use rate or increase over a chosen window with prefix_cache_hits; do not interpret the cumulative total as a rate.vllm/v1/metrics/loggers.py:562-588Sep 29
In vLLM v0.23.0, vllm:kv_cache_usage_perc is a point-in-time gauge where 1 represents fully used KV block capacity.vllm/v1/metrics/loggers.py:1070-1088Sep 29
In vLLM v0.23.0, vllm:kv_cache_usage_perc is set from SchedulerStats.kv_cache_usage rather than incremented from completed requests.vllm/v1/metrics/loggers.py:1070-1088Sep 29
In vLLM v0.23.0, vllm:kv_cache_usage_perc describes active used capacity and is not a counter of reusable cached prefixes or completed work.vllm/v1/metrics/loggers.py:1070-1088Sep 29
In vLLM v0.23.0, vllm:kv_cache_usage_perc use it as a pressure signal alongside waiting and progress metrics; do not infer cache effectiveness from it alone.vllm/v1/metrics/loggers.py:1070-1088Sep 29
In vLLM v0.23.0, vllm:prompt_tokens_cached counts cached prompt tokens reused during processing, combining local and external sources.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:prompt_tokens_cached increments from the iteration PromptTokenStats.cached_tokens field.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:prompt_tokens_cached is aggregate saved prompt-token work and is not by itself a per-request cache-hit percentage.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:prompt_tokens_cached compare its rate with prompt-token demand under matching labels and windows; separate local/external attribution when required.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens observes one histogram sample per finished request: prompt tokens minus non-negative cached tokens.vllm/v1/metrics/loggers.py:1179-1213Sep 29
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens is computed in finished-request logging after request completion from num_prompt_tokens and num_cached_tokens.vllm/v1/metrics/loggers.py:1179-1213Sep 29
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens measures newly computed prefill KV work per completed request, not total prompt length when caching contributes.vllm/v1/metrics/loggers.py:1179-1213Sep 29
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens compare distributions under matched workloads; use cached-token counters as companions and do not treat one bucket count as token throughput.vllm/v1/metrics/loggers.py:1179-1213Sep 29
In vLLM v0.23.0, vllm:request_success counts finished engine requests partitioned by finished_reason.vllm/v1/metrics/loggers.py:1179-1188Sep 29
In vLLM v0.23.0, vllm:request_success increments once for each FinishedRequestStats record consumed by the logger.vllm/v1/metrics/loggers.py:1179-1188Sep 29
In vLLM v0.23.0, vllm:request_success is a completed-engine-request population and does not cover requests rejected before engine completion.vllm/v1/metrics/loggers.py:1179-1188Sep 29
In vLLM v0.23.0, vllm:request_success sum only intended finish reasons for the question; use API or ingress telemetry for a denominator covering every HTTP attempt.vllm/v1/metrics/loggers.py:1179-1188Sep 29
In vLLM v0.23.0, vllm:num_requests_running is a point-in-time gauge of requests currently running in model execution batches.vllm/v1/metrics/loggers.py:1064-1082Sep 29
In vLLM v0.23.0, vllm:num_requests_running is set from scheduler running-request state at a logging snapshot.vllm/v1/metrics/loggers.py:1064-1082Sep 29
In vLLM v0.23.0, vllm:num_requests_running measures concurrency rather than arrivals, completions, successes, or failures.vllm/v1/metrics/loggers.py:1064-1082Sep 29
In vLLM v0.23.0, vllm:num_requests_running use it with waiting, KV pressure, and token progress to describe load; never use it as an error-rate denominator.vllm/v1/metrics/loggers.py:1064-1082Sep 29
In vLLM v0.23.0, vllm:generation_tokens counts generation tokens processed across iterations.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:generation_tokens increments from IterationStats.num_generation_tokens during iteration logging.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:generation_tokens is aggregate token work and not a per-request latency, fairness, or completion measure.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:generation_tokens use rate for aggregate decode throughput and pair it with ITL/TPOT/TTFT distributions before judging user experience.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:prompt_tokens counts prefill tokens processed across iterations.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:prompt_tokens increments from IterationStats.num_prompt_tokens during iteration logging.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:prompt_tokens is aggregate prefill work and does not by itself distinguish cached from newly computed prompt tokens.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:prompt_tokens use rate for prompt throughput and pair it with cached/computed-token and queue/TTFT metrics before drawing efficiency conclusions.vllm/v1/metrics/loggers.py:1149-1166Sep 29
In vLLM v0.23.0, vllm:iteration_tokens_total observes one histogram sample per engine step equal to computed prompt tokens plus generated tokens.vllm/v1/metrics/loggers.py:1152-1177Sep 29
In vLLM v0.23.0, vllm:iteration_tokens_total is populated from iteration-level work after each IterationStats update.vllm/v1/metrics/loggers.py:1152-1177Sep 29
In vLLM v0.23.0, vllm:iteration_tokens_total describes batch/step work distribution rather than a cumulative token counter despite total appearing in its declared name.vllm/v1/metrics/loggers.py:1152-1177Sep 29
In vLLM v0.23.0, vllm:iteration_tokens_total use its distribution to inspect batching; do not calculate token throughput from bucket counts without elapsed-time counters.vllm/v1/metrics/loggers.py:1152-1177Sep 29
In vLLM v0.23.0, vllm:cache_config_info is registered as a Prometheus gauge.vllm/v1/metrics/loggers.py:1047-1052Sep 29
In vLLM v0.23.0, vllm:corrupted_requests is registered as a Prometheus counter when `VLLM_COMPUTE_NANS_IN_LOGITS=1`.vllm/v1/metrics/loggers.py:530-537Sep 29
In vLLM v0.23.0, vllm:corrupted_requests is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:530-537Sep 29
In vLLM v0.23.0, vllm:e2e_request_latency_seconds is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:870-875Sep 29
In vLLM v0.23.0, vllm:e2e_request_latency_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:870-875Sep 29
In vLLM v0.23.0, vllm:e2e_request_latency_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0.vllm/v1/metrics/loggers.py:870-875Sep 29
In vLLM v0.23.0, vllm:engine_sleep_state is registered as a Prometheus gauge.vllm/v1/metrics/loggers.py:493-503Sep 29
In vLLM v0.23.0, vllm:engine_sleep_state is registered with the label names model_name, engine, sleep_state.vllm/v1/metrics/loggers.py:493-503Sep 29
In vLLM v0.23.0, vllm:external_prefix_cache_hits is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:578-585Sep 29
In vLLM v0.23.0, vllm:external_prefix_cache_hits is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:578-585Sep 29
In vLLM v0.23.0, vllm:external_prefix_cache_queries is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:566-573Sep 29
In vLLM v0.23.0, vllm:external_prefix_cache_queries is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:566-573Sep 29
In vLLM v0.23.0, vllm:generation_tokens is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:662-666Sep 29
In vLLM v0.23.0, vllm:generation_tokens is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:662-666Sep 29
In vLLM v0.23.0, vllm:inter_token_latency_seconds is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:787-812Sep 29
In vLLM v0.23.0, vllm:inter_token_latency_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:787-812Sep 29
In vLLM v0.23.0, vllm:inter_token_latency_seconds declares 19 literal histogram bucket boundaries in its registration, from 0.01 to 80.0.vllm/v1/metrics/loggers.py:787-812Sep 29
In vLLM v0.23.0, vllm:iteration_tokens_total is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:711-716Sep 29
In vLLM v0.23.0, vllm:iteration_tokens_total is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:711-716Sep 29
In vLLM v0.23.0, vllm:iteration_tokens_total declares 13 literal histogram bucket boundaries in its registration, from 1.0 to 16384.0.vllm/v1/metrics/loggers.py:711-716Sep 29
In vLLM v0.23.0, vllm:kv_block_idle_before_evict_seconds is registered as a Prometheus histogram when `observability_config.kv_cache_metrics=True`.vllm/v1/metrics/loggers.py:974-982Sep 29
In vLLM v0.23.0, vllm:kv_block_idle_before_evict_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:974-982Sep 29
In vLLM v0.23.0, vllm:kv_block_idle_before_evict_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.001 to 1800.0.vllm/v1/metrics/loggers.py:974-982Sep 29
In vLLM v0.23.0, vllm:kv_block_lifetime_seconds is registered as a Prometheus histogram when `observability_config.kv_cache_metrics=True`.vllm/v1/metrics/loggers.py:961-969Sep 29
In vLLM v0.23.0, vllm:kv_block_lifetime_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:961-969Sep 29
In vLLM v0.23.0, vllm:kv_block_lifetime_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.001 to 1800.0.vllm/v1/metrics/loggers.py:961-969Sep 29
In vLLM v0.23.0, vllm:kv_block_reuse_gap_seconds is registered as a Prometheus histogram when `observability_config.kv_cache_metrics=True`.vllm/v1/metrics/loggers.py:987-997Sep 29
In vLLM v0.23.0, vllm:kv_block_reuse_gap_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:987-997Sep 29
In vLLM v0.23.0, vllm:kv_block_reuse_gap_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.001 to 1800.0.vllm/v1/metrics/loggers.py:987-997Sep 29
In vLLM v0.23.0, vllm:kv_cache_usage_perc is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:519-524Sep 29
In vLLM v0.23.0, vllm:lora_requests_info is registered as a Prometheus gauge when `lora_config is not None`.vllm/v1/metrics/loggers.py:1023-1032Sep 29
In vLLM v0.23.0, vllm:mm_cache_hits is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:605-611Sep 29
In vLLM v0.23.0, vllm:mm_cache_hits is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:605-611Sep 29
In vLLM v0.23.0, vllm:mm_cache_queries is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:594-600Sep 29
In vLLM v0.23.0, vllm:mm_cache_queries is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:594-600Sep 29
In vLLM v0.23.0, vllm:num_preemptions is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:619-623Sep 29
In vLLM v0.23.0, vllm:num_requests_running is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:451-456Sep 29
In vLLM v0.23.0, vllm:num_requests_waiting is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:461-466Sep 29
In vLLM v0.23.0, vllm:num_requests_waiting_by_reason is registered with the label names model_name, engine, reason.vllm/v1/metrics/loggers.py:471-482Sep 29
In vLLM v0.23.0, vllm:prefix_cache_hits is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:553-557Sep 29
In vLLM v0.23.0, vllm:prefix_cache_queries is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:542-548Sep 29
In vLLM v0.23.0, vllm:prompt_tokens is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:628-632Sep 29
In vLLM v0.23.0, vllm:prompt_tokens is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:628-632Sep 29
In vLLM v0.23.0, vllm:prompt_tokens_by_source is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:638-642Sep 29
In vLLM v0.23.0, vllm:prompt_tokens_by_source is registered with the label names model_name, engine, source.vllm/v1/metrics/loggers.py:638-642Sep 29
In vLLM v0.23.0, vllm:prompt_tokens_cached is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:653-657Sep 29
In vLLM v0.23.0, vllm:prompt_tokens_cached is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:653-657Sep 29
In vLLM v0.23.0, vllm:request_decode_time_seconds is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:910-915Sep 29
In vLLM v0.23.0, vllm:request_decode_time_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:910-915Sep 29
In vLLM v0.23.0, vllm:request_decode_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0.vllm/v1/metrics/loggers.py:910-915Sep 29
In vLLM v0.23.0, vllm:request_generation_tokens is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:698-703Sep 29
In vLLM v0.23.0, vllm:request_generation_tokens is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:698-703Sep 29
In vLLM v0.23.0, vllm:request_inference_time_seconds is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:890-895Sep 29
In vLLM v0.23.0, vllm:request_inference_time_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:890-895Sep 29
In vLLM v0.23.0, vllm:request_inference_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0.vllm/v1/metrics/loggers.py:890-895Sep 29
In vLLM v0.23.0, vllm:request_max_num_generation_tokens is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:721-726Sep 29
In vLLM v0.23.0, vllm:request_max_num_generation_tokens is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:721-726Sep 29
In vLLM v0.23.0, vllm:request_params_max_tokens is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:741-746Sep 29
In vLLM v0.23.0, vllm:request_params_max_tokens is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:741-746Sep 29
In vLLM v0.23.0, vllm:request_params_n is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:731-736Sep 29
In vLLM v0.23.0, vllm:request_params_n is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:731-736Sep 29
In vLLM v0.23.0, vllm:request_params_n declares 5 literal histogram bucket boundaries in its registration, from 1.0 to 20.0.vllm/v1/metrics/loggers.py:731-736Sep 29
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:920-928Sep 29
In vLLM v0.23.0, vllm:request_prefill_kv_computed_tokens is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:920-928Sep 29
In vLLM v0.23.0, vllm:request_prefill_time_seconds is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:900-905Sep 29
In vLLM v0.23.0, vllm:request_prefill_time_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:900-905Sep 29
In vLLM v0.23.0, vllm:request_prefill_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0.vllm/v1/metrics/loggers.py:900-905Sep 29
In vLLM v0.23.0, vllm:request_prompt_tokens is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:688-693Sep 29
In vLLM v0.23.0, vllm:request_prompt_tokens is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:688-693Sep 29
In vLLM v0.23.0, vllm:request_queue_time_seconds is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:880-885Sep 29
In vLLM v0.23.0, vllm:request_queue_time_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:880-885Sep 29
In vLLM v0.23.0, vllm:request_queue_time_seconds declares 21 literal histogram bucket boundaries in its registration, from 0.3 to 7680.0.vllm/v1/metrics/loggers.py:880-885Sep 29
In vLLM v0.23.0, vllm:request_success is registered as a Prometheus counter.vllm/v1/metrics/loggers.py:672-676Sep 29
In vLLM v0.23.0, vllm:request_success is registered with the label names model_name, engine, finished_reason.vllm/v1/metrics/loggers.py:672-676Sep 29
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:817-842Sep 29
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:817-842Sep 29
In vLLM v0.23.0, vllm:request_time_per_output_token_seconds declares 19 literal histogram bucket boundaries in its registration, from 0.01 to 80.0.vllm/v1/metrics/loggers.py:817-842Sep 29
In vLLM v0.23.0, vllm:time_to_first_token_seconds is registered as a Prometheus histogram.vllm/v1/metrics/loggers.py:754-782Sep 29
In vLLM v0.23.0, vllm:time_to_first_token_seconds is registered with the label names model_name, engine.vllm/v1/metrics/loggers.py:754-782Sep 29
In vLLM v0.23.0, vllm:time_to_first_token_seconds declares 22 literal histogram bucket boundaries in its registration, from 0.001 to 2560.0.vllm/v1/metrics/loggers.py:754-782Sep 29
In vLLM v0.23.0, vllm:prefix_cache_queries is a Prometheus counter measured in queried tokens — it counts tokens, not requests or cache blocks.vllm/v1/metrics/loggers.py:542-551, vllm/v1/metrics/loggers.py:1083-1088Sep 29
In vLLM v0.23.0, vllm:prefix_cache_hits is a Prometheus counter measured in cached (hit) tokens — it counts tokens, not requests or cache blocks.vllm/v1/metrics/loggers.py:553-561, vllm/v1/metrics/loggers.py:1083-1088Sep 29
In vLLM v0.23.0, vllm:prefix_cache_queries is incremented from SchedulerStats.prefix_cache_stats.queries on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters).vllm/v1/metrics/loggers.py:1083-1088, vllm/v1/metrics/loggers.py:562-588Sep 29
In vLLM v0.23.0, vllm:prefix_cache_hits is incremented from SchedulerStats.prefix_cache_stats.hits on each logging update and observes prefix-cache token reuse for the first-party prefix cache (KV-connector prefix-cache reuse is reported by the separate vllm:external_prefix_cache_* counters).vllm/v1/metrics/loggers.py:1083-1088, vllm/v1/metrics/loggers.py:562-588Sep 29
In vLLM v0.23.0, vllm:num_requests_running is a Prometheus gauge for the number of requests in model execution batches.vllm/v1/metrics/loggers.py:451-459Sep 29
In vLLM v0.23.0, vllm:num_requests_running is set from SchedulerStats.num_running_reqs and observes scheduler running-request state.vllm/v1/metrics/loggers.py:1066-1069Sep 29
In vLLM v0.23.0, vllm:num_requests_waiting is a Prometheus gauge for the number of requests waiting to be processed.vllm/v1/metrics/loggers.py:461-469Sep 29
In vLLM v0.23.0, vllm:num_requests_waiting_by_reason is a Prometheus gauge with reason labels capacity and deferred, and its reasons sum to vllm:num_requests_waiting.vllm/v1/metrics/loggers.py:471-491Sep 29
In vLLM v0.23.0, vllm:kv_cache_usage_perc is a Prometheus gauge for KV-cache usage where 1 means 100 percent usage.vllm/v1/metrics/loggers.py:519-527Sep 29
In vLLM v0.23.0, vllm:kv_cache_usage_perc is set from SchedulerStats.kv_cache_usage and observes KV-cache usage pressure.vllm/v1/metrics/loggers.py:1081-1081Sep 29
In vLLM v0.23.0, vllm:num_preemptions is a Prometheus counter for cumulative preemptions from the engine.vllm/v1/metrics/loggers.py:619-626Sep 29
In vLLM v0.23.0, vllm:num_preemptions increments from IterationStats.num_preempted_reqs, which is increased when engine-core events mark requests as preempted.vllm/v1/metrics/loggers.py:1149-1150Sep 29

Metrics grounded in this source (38)

NameDescriptionLocator
num_requests_runningGauge for the number of requests in model execution batches.vllm/v1/metrics/loggers.py:451-459
num_requests_waitingGauge for the number of requests waiting to be processed.vllm/v1/metrics/loggers.py:461-469
num_requests_waiting_by_reasonGauge that breaks waiting requests into capacity and deferred reasons.vllm/v1/metrics/loggers.py:471-491
kv_cache_usage_percGauge for KV-cache usage where 1 means 100 percent usage.vllm/v1/metrics/loggers.py:519-527
num_preemptionsCounter for cumulative preemptions from the engine.vllm/v1/metrics/loggers.py:619-626
engine_sleep_stateGauge for engine sleep state; sleep_state label values awake, weights_offloaded, and discard_all encode sleep levels.vllm/v1/metrics/loggers.py:493-514
corrupted_requestsCounter for corrupted requests, measured as requests with NaNs in logits.vllm/v1/metrics/loggers.py:529-540

View all 38 in Metrics →