Non-idempotent tasks with retries cause duplicate processing and data corruption

Retrying non-idempotent tasks leads to duplicate processing, causing data corruption in 25% of cases. Examples include double charges in payment processing. Without message ID tracking and deduplication, retries execute the same operation multiple times.

Always chain idempotency middleware before retry middleware. Implement Redis-based message ID tracking with SCAN for msg_id lookup. Use Redis SETEX with 24-hour TTL to deduplicate messages. Add supplier_tx_hash for transaction-level deduplication. Log with structlog for audit trail.