ClickHouseAmazon S3

Cold storage deletion undetected due to monitoring gap on S3-backed data

critical
availabilityUpdated Feb 20, 2026(via Exa)
How to detect:

When monitoring focuses only on ingestion lag, query latency, and error rates for active queries, large-scale deletion of cold storage data (>24 hours old in S3) goes undetected for hours. Since users rarely query old data and deletion happens in the background without application-level errors, dashboards remain green while petabytes of data are deleted over an 8-hour window.

Recommended action:

Implement specific monitoring for cold data integrity in S3. Monitor S3 object count and total storage size metrics with alerts on unexpected deletion rates. Add periodic consistency checks between ClickHouse metadata and actual S3 objects. Ensure monitoring covers both hot and cold data layers.