Commit Graph

15 Commits

Author SHA1 Message Date
zmstone 00dfdc22cf chore: pin Kafka client wolff 0.10.5 2024-06-17 17:27:25 +02:00
zmstone 9637b14543 fix: upgrade to brod 3.18.0 to fix unnecessary rebalance 2024-06-01 10:57:24 +02:00
zmstone a00d9f17b1 chore: upgrade kafka producer lib wolff to 1.10.4
wolff-1.10.4 handles message_too_large error code from Kafka.
It tries to split the too-large batch into single-call batches
(for EMQX, one call batch is always one message), and retry.
In case a single-call batch is too large, EMQX should increment
a failure counter.
2024-05-30 15:40:10 +02:00
Zaiming (Stone) Shi 242df0ffe9 chore: upgrade kafka_protocol from 4.1.3 to 4.1.5 2024-02-16 10:33:53 +01:00
Zaiming (Stone) Shi d4fb3e83f6 chore: upgrade to wolff 1.10.2
Wolff 1.10.2 added an enhancement which makes Kafka resource health
checks more lightweight and lowers the chance of false resource alerts.
2024-02-12 09:05:44 +01:00
Zaiming (Stone) Shi 3b42a7425b feat(kafka_producer): add partitions_limit 2024-01-31 19:26:01 +01:00
JimMoen 5e100f52b8
style: erlfmt all `rebar.config` files and `bin/nodetool` 2023-12-29 09:08:03 +08:00
Zaiming (Stone) Shi 7338e394c8 perf: upgrade to wolff 1.9.1 for better performance
Upgrade from 1.8.0:
- wollf-1.9.0 has the global stats disabled by default
- wolff-1.9.1 improved client pid lookup performance
2023-12-18 18:54:21 +01:00
Zaiming (Stone) Shi e6eb97e104 chore: upgrade dependency brod_gssapi to work with OTP 26 2023-11-29 16:15:18 +01:00
Kjell Winblad 9dc3a169b3 feat: split bridges into a connector part and a bridge part
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>

Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.

Issues:
https://emqx.atlassian.net/browse/EMQX-10805
2023-10-30 14:48:47 +01:00
Zaiming (Stone) Shi 115ab85670 chore: upgrade Kafka client wolff to 1.7.7 2023-08-04 15:31:34 +02:00
Paulo Zulato 8430ec673c feat(kafka): check whether target topic exists
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-21 15:00:35 -03:00
JimMoen bb363e9b9e
chore: bump dependency vsn
* bump `snappyer` ~> 1.2.9
  - as dependency of `kafka_protocol` ~> 4.1.3
  - as dependency of `brod`

* bump `rocksdb` ~> 1.7.2-emqx-11
2023-05-30 11:11:30 +08:00
Zaiming (Stone) Shi aaf3bf27c4 fix(kafka): ensure brod_gssapi sasl_auth are include in release 2023-05-07 09:43:58 +02:00
Thales Macedo Garitezi 871ee90b3e refactor(kafka_bridge): move kafka bridge into its own app
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-12 13:54:45 -03:00