emqx/changes/ee/fix-13079.en.md
zmstone a00d9f17b1 chore: upgrade kafka producer lib wolff to 1.10.4
wolff-1.10.4 handles message_too_large error code from Kafka.
It tries to split the too-large batch into single-call batches
(for EMQX, one call batch is always one message), and retry.
In case a single-call batch is too large, EMQX should increment
a failure counter.
2024-05-30 15:40:10 +02:00

7 lines
405 B
Markdown

Improve Kafka producer error handling for `message_too_large`.
Prior to this change, Kafka producers would retry sending oversized batches (`message_too_large` error) in hopes of a server side configuration fix (`max.message.bytes`).
Now, oversized messages are automatically split into single-message batches for retry.
If a message still exceeds size limits, it will be dropped to maintain data flow.