vllm:num_requests_running

Summary

In vLLM v0.23.0, under the V1 metrics path published by PrometheusStatLogger, vllm:num_requests_running is a Prometheus gauge reporting the number of requests in model execution batches. As a measurement, it is a point-in-time gauge of the requests currently running in model execution batches. It is registered with the label names model_name and engine. Drawing on V1 scheduler stats, it is set from SchedulerStats.num_running_reqs and therefore observes scheduler running-request state. Its value is taken from scheduler running-request state at a logging snapshot. Interpret it as a measure of concurrency, not of arrivals, completions, successes, or failures. Use it alongside waiting requests, KV pressure, and token progress to describe load, and never use it as an error-rate denominator.

Version history

• In vLLM v0.23.0, vllm:num_requests_running is known only at this version as a Prometheus gauge giving a point-in-time count of requests currently in model execution batches.
• At v0.23.0 it is registered with the label names model_name and engine.
• At v0.23.0 its value is taken from SchedulerStats.num_running_reqs, so it observes scheduler running-request state as captured at a logging snapshot.
• At v0.23.0 it measures concurrency rather than arrivals, completions, successes, or failures.
• At v0.23.0 the guidance is to read it alongside waiting requests, KV pressure, and token progress to describe load, and never to use it as an error-rate denominator.

Identity

Prometheus type

gauge

Labels

model_nameengine

Unit

Not recorded — vLLM registration code has no structured unit field, and units are not inferred from name suffixes

Semantic coverage

Metric knowledge (7)

Related Use Cases (1)

• #9346: vLLM metrics do not provide an all-HTTP-request denominator

Availability