Confluent PlatformApache FlinkApache Kafka

SMTs introduce latency under heavy traffic with complex enrichment

warning
performanceUpdated Sep 30, 2025(via Exa)
How to detect:

Using Kafka Connect Single Message Transforms (SMTs) for complex enrichment operations (lookups, joins, external API calls) under heavy traffic introduces significant latency and degrades throughput, as Connect is not designed for heavy processing workloads.

Recommended action:

Use SMTs only for lightweight operations (adding timestamps, static metadata, field renaming, simple masking). For complex enrichment requiring external lookups or joins, migrate logic to Apache Flink or Kafka Streams. Monitor connector lag and throughput metrics to identify performance bottlenecks.