K8s Client Rate Limiter Throttling

warning

scalingUpdated Feb 23, 2026

Elevated cilium_k8s_client_rate_limiter_time_seconds indicates Cilium agents are being throttled by Kubernetes API server rate limits. This delays reaction to cluster state changes, causing stale service endpoints, delayed policy enforcement, and slow pod networking setup.

Sources

Troubleshooting — Cilium 1.19.1 documentationdocs.cilium.io

Technologies:

CiliumSymptoms of this issue are visible in Cilium metrics and logs

KubernetesThe root cause of this issue originates in Kubernetes

How to detect:

Monitor cilium_k8s_client_rate_limiter_time_seconds_datadog for sustained non-zero values. High cilium_k8s_workqueue_adds combined with rate limiting indicates backlog accumulation. Check cilium_k8s_workqueue_retries for signs of reconciliation failures.

Recommended action:

Increase Kubernetes API server rate limits for Cilium service account. Reduce Cilium reconciliation frequency if possible. Monitor cilium_kubernetes_events_received and cilium_kubernetes_events_datadog to identify event storms. Consider deploying dedicated API server instances for CNI operations in very large clusters.