emqx/changes/ee/fix-13079.en.md
zmstone a00d9f17b1 chore: upgrade kafka producer lib wolff to 1.10.4
wolff-1.10.4 handles message_too_large error code from Kafka.
It tries to split the too-large batch into single-call batches
(for EMQX, one call batch is always one message), and retry.
In case a single-call batch is too large, EMQX should increment
a failure counter.
2024-05-30 15:40:10 +02:00

405 B

Improve Kafka producer error handling for message_too_large.

Prior to this change, Kafka producers would retry sending oversized batches (message_too_large error) in hopes of a server side configuration fix (max.message.bytes).

Now, oversized messages are automatically split into single-message batches for retry. If a message still exceeds size limits, it will be dropped to maintain data flow.