Schema.ai· vLLM
vLLM official site Log in
All versions · Version range: v0.23.0

vllm:num_requests_running

Gauge for the number of requests in model execution batches.

MetricPrometheusverified Sep 29

Version history

  • In vLLM v0.23.0, vllm:num_requests_running is known only at this version as a Prometheus gauge giving a point-in-time count of requests currently in model execution batches.
  • At v0.23.0 it is registered with the label names model_name and engine.
  • At v0.23.0 its value is taken from SchedulerStats.num_running_reqs, so it observes scheduler running-request state as captured at a logging snapshot.
  • At v0.23.0 it measures concurrency rather than arrivals, completions, successes, or failures.
  • At v0.23.0 the guidance is to read it alongside waiting requests, KV pressure, and token progress to describe load, and never to use it as an error-rate denominator.

Identity

Prometheus type
gauge
Labels
model_nameengine
Unit
Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

covered · Availability & namecovered · Structural identitycovered · Measurement semanticscovered · Causal/runtime semanticscovered · Interpretation & compositioncovered · Guarded guidance

Metric knowledge (7)

ClaimSemantic layerVerified
In vLLM v0.23.0, vllm:num_requests_running is a point-in-time gauge of requests currently running in model execution batches.Measurement semanticsSep 29
In vLLM v0.23.0, vllm:num_requests_running is set from scheduler running-request state at a logging snapshot.Causal/runtime semanticsSep 29
In vLLM v0.23.0, vllm:num_requests_running measures concurrency rather than arrivals, completions, successes, or failures.Interpretation & compositionSep 29
In vLLM v0.23.0, vllm:num_requests_running use it with waiting, KV pressure, and token progress to describe load; never use it as an error-rate denominator.Guarded guidanceSep 29
In vLLM v0.23.0, vllm:num_requests_running is registered with the label names model_name, engine.Structural identitySep 29
In vLLM v0.23.0, vllm:num_requests_running is a Prometheus gauge for the number of requests in model execution batches.Structural identitySep 29
In vLLM v0.23.0, vllm:num_requests_running is set from SchedulerStats.num_running_reqs and observes scheduler running-request state.Interpretation & compositionSep 29

Related Use Cases (1)

Availability

vllm:num_requests_running is registered as a Prometheus gauge in vLLM v0.23.0.
vllm.surface.prometheus_metricscomplete_inventoryPrometheusStatLogger metrics publisher

vLLM v0.23.0 v1 metrics loggers.py · vllm/v1/metrics/loggers.py:451-459