Confluent PlatformApache ZooKeeper

Topic proliferation causes resource contention and monitoring challenges

warning
Resource ContentionUpdated Mar 13, 2025(via Exa)
How to detect:

Creating too many topics introduces overhead in partition management, replication, and ZooKeeper metadata, leading to resource contention (CPU, memory, network) and increased operational complexity with fragmented monitoring.

Recommended action:

Set retention policies (retention.ms, retention.bytes) to auto-expire unnecessary data. Consolidate topics where possible using logical partitioning via message keys. Use topic naming conventions for organization.