Technologies/Luigi/luigi.scheduler.retry.total

LuigiMetric

luigi.scheduler.retry.total

Total task retry attempts

Dimensions:None

Available on:

Prometheus (1)

Interface Metrics (1)

Prometheus

luigi_scheduler_retry_total

Total number of task retries attempted by the scheduler

Dimensions:None

Sources

luigi_scheduler_retry_totalgithub.com

Technical Annotations (36)

Configuration Parameters (13)

retry_count

Maximum number of retries before task is considered failed

disable_window

Time window for tracking task failures

disable_hard_timeout

Controls hard timeout behavior during retries

scheduler.retry_countrecommended: 3

number of retry attempts for failed tasks

scheduler.retry-delayrecommended: 1

delay in seconds between retry attempts

--scheduler-retry-delayrecommended: paired with --local-scheduler

controls retry interval for failed tasks; only effective with local scheduler

--local-schedulerrecommended: required when setting custom retry delays via CLI

starts local scheduler instance that respects CLI retry parameters

--scheduler-retry-countrecommended: 10

maximum number of retry attempts for failed tasks

--worker-retry-external-tasks

enables retry behavior for external data dependency tasks

retry_delayrecommended: 10.0

Seconds to wait before retry; only works on Linux, not Windows

scheduler.disable-persist-secondsrecommended: 1200

Duration task remains disabled before automatic retry; increase to allow more manual intervention time

scheduler.retry_delayrecommended: 60

Delay in seconds between retry attempts

scheduler.disable-hard-timeoutrecommended: 21600

Hard timeout for disabled state in seconds

Error Signatures (5)

is marked as running by disconnected workerlog pattern

marking as FAILED with retry delaylog pattern

This progress looks :( because there were failed taskslog pattern

Data for FileTask(filename=test-file.json) does not exist (yet?). The task is an external data dependency, so it cannot be run from this luigi process.

log pattern

status FAILEDlog pattern

CLI Commands (3)

pip install -U luigiremediation

python -m luigi --module luigi_external MasterTask --worker-retry-external-tasks --scheduler-retry-delay 1 --scheduler-retry-count 10 --worker-keep-alive --local-scheduler

remediation

luigiddiagnostic

Technical References (15)

task schedulercomponenttask signatureconceptS3 versioningcomponentRetryPolicycomponentTaskcomponentschedulercomponentretry mechanismconceptExternalTaskcomponentluigidcomponentWrapperTaskcomponentcentral schedulercomponentWorkercomponentluigi.cfgfile pathdisable statusconceptpending statusconcept

Related Insights (11)

Failed Luigi workflows can be resumed by re-running with same parametersinfo

▸

Unversioned S3 buckets prevent pipeline replay after failureswarning

▸

Tasks stuck in running state after worker disconnectioncritical

▸

Single-threaded task saturates API rate limitswarning

▸

Task retry exhaustion from excessive failureswarning

▸

Luigi reports failure when retried tasks succeedwarning

▸

Heterogeneous task retry requirements cannot be satisfied with global scheduler settingswarning

▸

Scheduler retry-delay parameter requires local-scheduler flag to take effectwarning

▸

Central scheduler fails to update task status after successful retrywarning

▸

Retry delay configuration does not work on Windowswarning

▸

Failed tasks retry infinitely after disable suspension expireswarning

▸