Technologies/Luigi/luigi.scheduler.running_tasks
LuigiLuigiMetric

luigi.scheduler.running_tasks

Tasks currently executing
Dimensions:None
Available on:PrometheusPrometheus (1)DatadogDatadog (1)
Interface Metrics (2)
PrometheusPrometheus
Current number of tasks in RUNNING state being executed by workers
Dimensions:None
DatadogDatadog
Number of tasks currently running
Dimensions:None
Sources
luigi_scheduler_running_tasksgithub.com
luigi.scheduler.running_tasksgithub.com

Technical Annotations (30)

Configuration Parameters (4)
resources.cpu_coresrecommended: 10
minimum CPU cores available for task execution; must match or exceed task requirements
resources.<resource_name>recommended: integer limit based on capacity
Defines maximum concurrent usage of named resource
apirecommended: 1
In [resources] config section; limits available API resource pool
resourcesrecommended: {"api": 1}
Task class attribute; declares how much API resource each task instance consumes
Error Signatures (8)
There are no more tasks to run at this timelog pattern
pending tasks possibly being run by other workerslog pattern
pending tasks unique to this workerlog pattern
RuntimeError: Unfulfilled dependencies at run timeexception
is marked as running by disconnected workerlog pattern
marking as FAILED with retry delaylog pattern
TypeError: __str__ returned non-string (type NoneType)exception
ImportError: No module named boto3exception
CLI Commands (1)
luigid --background --pidfile <PATH_TO_PIDFILE> --logdir <PATH_TO_LOGDIR> --state-path <PATH_TO_STATEFILE>remediation
Technical References (17)
luigi.Taskcomponent[resources]componentdependenciesconceptcontainercomponentcentral schedulercomponentMesoscomponentschedulercomponentPENDINGconceptRUNNINGconceptTaskProcess.runcomponentscheduler.pyfile path/get_workcomponent/add_taskcomponentluigi/scheduler.pyfile pathluigi/notifications.pyfile pathworker_runningconceptluigidcomponent
Related Insights (10)
Resource requirements block task scheduling despite available workerscritical
▸
Luigi tasks fail with unfulfilled dependency errors under resource contentionwarning
▸
Multiple workers can overload system without global resource limitswarning
▸
Tasks stuck in running state after worker disconnectioncritical
▸
Scheduler blocks tasks when resource limits are exceededwarning
▸
API rate limit exceeded due to uncontrolled task concurrencywarning
▸
Tasks stuck in RUNNING state despite completioncritical
▸
Email notification failure during task disabling leaves tasks in corrupted RUNNING statecritical
▸
Remote scheduler UI inaccessible without port forwardinginfo
▸
Local scheduler in production allows duplicate task executionwarning
▸