Commit Graph

245 Commits

Author SHA1 Message Date
Ivan Dyachkov cc3b26a3ac Merge remote-tracking branch 'upstream/release-58' into 20240807-sync-release-branches 2024-08-07 09:48:38 +02:00
Thales Macedo Garitezi 9ca3985bbd test: attempt to reduce test flakiness 2024-08-06 12:44:51 -03:00
Thales Macedo Garitezi 85cff5e7eb fix: merge conflicts 2024-07-31 09:14:29 -03:00
Ivan Dyachkov 74cef7937d Merge remote-tracking branch 'upstream/release-57' into 20240731-sync-release-57 2024-07-31 11:31:29 +02:00
zmstone 2e39c4ad5e
Merge pull request #13495 from thalesmg/20240719-m-hide-enable-fields-mkII
chore: hide enable flags from schema and config examples
2024-07-29 21:31:20 +02:00
Thales Macedo Garitezi 693d5dd394 feat: attempt to automatically decode `payload` similar to key and message templates 2024-07-29 13:01:06 -03:00
Thales Macedo Garitezi 6786c9b517
refactor: improve descriptions and identifiers
Co-authored-by: zmstone <zmstone@gmail.com>
2024-07-29 09:45:52 -03:00
Thales Macedo Garitezi 1d56ac6e5e refactor: change topic schema type 2024-07-26 14:26:21 -03:00
Thales Macedo Garitezi 4e0742c66f feat: make kafka producer freely dynamic 2024-07-26 14:25:20 -03:00
Thales Macedo Garitezi df1f4fad70 feat(kafka producer): allow dynamic topics from pre-configured topics
Fixes https://emqx.atlassian.net/browse/EMQX-12656
2024-07-25 15:33:12 -03:00
Thales Macedo Garitezi 33eccb35da chore: update wolff -> 3.0.2 2024-07-25 14:30:50 -03:00
Ivan Dyachkov 606d829256 Merge remote-tracking branch 'upstream/release-57' into 20240724-sync-release-57 2024-07-24 10:28:00 +02:00
zhongwencool e7d07ea17c feat: add resource_type to emqx_resource behaviour 2024-07-23 18:24:28 +08:00
zhongwencool cba3f532f8 feat: don't record dry_run log 2024-07-23 15:14:41 +08:00
Thales Macedo Garitezi 69f5b6fa6c chore: hide `enable` fields from docgen
Fixes https://emqx.atlassian.net/browse/EMQX-12730
2024-07-22 13:26:53 -03:00
Thales Macedo Garitezi 5532f40d83 Merge branch 'release-57' into 20240702-m-sync-r57-mix-umbrella 2024-07-02 11:52:36 -03:00
Thales Macedo Garitezi bd0c8f0204 fix: bad conflict resolution 2024-07-02 09:47:49 -03:00
Thales Macedo Garitezi 19f3b030f9 chore: preparing to run common tests / eunit with mix 2024-07-02 09:45:45 -03:00
Thales Macedo Garitezi b74828d7ea perf: "mixify" project to improve release speed
Fixes https://emqx.atlassian.net/browse/EMQX-12527
2024-07-02 09:39:48 -03:00
Ivan Dyachkov e28750b522 Merge remote-tracking branch 'upstream/release-57' into 0701-sync-release-57 2024-07-01 13:52:14 +02:00
Thales Macedo Garitezi 067beece75 chore: add count to thrown error 2024-06-28 12:06:47 -03:00
Thales Macedo Garitezi 4bd0abc93f chore: bump app vsns 2024-06-27 09:22:06 -03:00
Thales Macedo Garitezi 4c3c86e919 chore: bump wolff -> 2.0.0 2024-06-27 09:20:00 -03:00
Thales Macedo Garitezi ed5e6599d9 fix(buffer worker, kafka): send reply when async call fails immediately
Fixes https://emqx.atlassian.net/browse/EMQX-12585
2024-06-27 09:20:00 -03:00
Thales Macedo Garitezi 6023012f8b fix(kafka and derivatives): add `alias` config to avoid clashes with same topic
Fixes https://emqx.atlassian.net/browse/EMQX-12592
2024-06-27 09:20:00 -03:00
zmstone 5d1608f34b chore: minimize oss/platform diff 2024-06-24 15:30:03 +02:00
Thales Macedo Garitezi 20cffb54d4 Merge branch 'release-57' into sync-r57-m-20240617 2024-06-17 17:10:12 -03:00
zmstone 3ac4ddcbe3 fix(kafka): handle message_too_large
bump 'failed' counter
2024-06-17 17:27:25 +02:00
zmstone 00dfdc22cf chore: pin Kafka client wolff 0.10.5 2024-06-17 17:27:25 +02:00
zmstone ebf17c8143 Merge remote-tracking branch 'origin/release-57' into 0606-merge-release-57-to-master 2024-06-06 17:43:36 +02:00
zmstone 9637b14543 fix: upgrade to brod 3.18.0 to fix unnecessary rebalance 2024-06-01 10:57:24 +02:00
Ivan Dyachkov 29ad07ce29 Merge remote-tracking branch 'upstream/release-57' into 0531-sync-release-57 2024-05-31 07:32:26 +02:00
zmstone a00d9f17b1 chore: upgrade kafka producer lib wolff to 1.10.4
wolff-1.10.4 handles message_too_large error code from Kafka.
It tries to split the too-large batch into single-call batches
(for EMQX, one call batch is always one message), and retry.
In case a single-call batch is too large, EMQX should increment
a failure counter.
2024-05-30 15:40:10 +02:00
zmstone 9fe29bf5f7 chore: bump kafka bridge app vsn 2024-05-29 21:57:14 +02:00
zmstone 238c207b09 chore: bump app versions 2024-05-27 20:26:52 +02:00
ieQu1 acb19a06cf Merge remote-tracking branch 'origin/release-57' 2024-05-21 15:04:46 +02:00
zmstone 331d44a78a chore: improve Kafka producer health-check error logs
The Kafka producer lib wolff returns detailed error logs
about which host:port had issue, EMQX should log them to help
troubleshooting
2024-05-18 19:12:51 +02:00
Kjell Winblad 72b0c63ed2
Merge pull request #13043 from kjellwinblad/kjell/kafka_incorrect_query_type/EMQX-12363
fix(kafka trace): do not include query type in trace entry
2024-05-16 10:08:16 +02:00
firest 437a0c0d55 fix(rocketmq): fix a typo and a RocketMQ encoding error 2024-05-14 23:01:31 +08:00
Kjell Winblad 121084b2ff fix(kafka trace): do not include query type in trace entry
For Kafka that has its internal buffering the on_query_async callback is
used both for sync and async queries (for sync queries the caller waits
for a response directly after calling on_query_async). Therefore, before
this commit, the trace included incorrect query type information. This
has been fixed by removing the query type information from the trace.
This should be okay since the query type is not essential information
and can be derived from the configuration.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12363
2024-05-14 12:24:30 +02:00
Thales Macedo Garitezi 3818b75188
Merge pull request #12959 from thalesmg/kprodu-connector-hc-m-20240429
feat(kafka producer): add health check topic option
2024-05-03 12:48:48 -03:00
Thales Macedo Garitezi 6f3da6b131 feat(kafka producer): add health check topic option
Fixes https://emqx.atlassian.net/browse/EMQX-12241

This allows more accurate health checking for Kafka Producers.  Without a topic, it's not
possible to actually probe the connection to partition leaders, so the connector might not
be reported as `disconnected` without testing a concrete topic.
2024-05-02 17:07:44 -03:00
Thales Macedo Garitezi eb113fa578 fix: add non-empty validator 2024-05-02 11:19:00 -03:00
Thales Macedo Garitezi 3942b371d7 feat(kafka consumer): allow custom group id
Fixes https://emqx.atlassian.net/browse/EMQX-12273
Fixes EMQX-12273

When consuming messages in Kafka in Alibaba Cloud, the group needs to be configured in
advance, and then the consumer can use the group to consume messages. Automatic group
creation is generally not allowed online.
2024-05-02 08:59:40 -03:00
Thales Macedo Garitezi 14ef0b1e51 feat(kafka consumer): allow custom group id
Fixes https://emqx.atlassian.net/browse/EMQX-12273
Fixes EMQX-12273

When consuming messages in Kafka in Alibaba Cloud, the group needs to be configured in
advance, and then the consumer can use the group to consume messages. Automatic group
creation is generally not allowed online.
2024-05-01 13:53:29 -03:00
Kjell Winblad 32c27f1711 feat: add stop after render and after render trace to kafka action 2024-04-25 11:51:46 +02:00
zmstone ca56e7e8d7 fix(kafka): headers are template fields 2024-04-18 13:04:36 +02:00
zmstone 5c014f4c29 test: fix test cases 2024-04-17 20:34:33 +02:00
zmstone 51c8173174 feat(bridge): add is_template flag to bridge config fields 2024-04-17 13:10:55 +02:00
zmstone dfd13b4ab5 test: fix dashboard schema json test cases 2024-04-11 15:12:01 +02:00