yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,431 Commits 393 Branches 669 Tags 74 MiB
b0aa7f25aa
Commit Graph

205 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
3818b75188
Merge pull request #12959 from thalesmg/kprodu-connector-hc-m-20240429 
feat(kafka producer): add health check topic option
 2024-05-03 12:48:48 -03:00
Thales Macedo Garitezi
6f3da6b131 feat(kafka producer): add health check topic option 
Fixes https://emqx.atlassian.net/browse/EMQX-12241

This allows more accurate health checking for Kafka Producers.  Without a topic, it's not
possible to actually probe the connection to partition leaders, so the connector might not
be reported as `disconnected` without testing a concrete topic.
 2024-05-02 17:07:44 -03:00
Thales Macedo Garitezi
eb113fa578 fix: add non-empty validator 2024-05-02 11:19:00 -03:00
Thales Macedo Garitezi
3942b371d7 feat(kafka consumer): allow custom group id 
Fixes https://emqx.atlassian.net/browse/EMQX-12273
Fixes EMQX-12273

When consuming messages in Kafka in Alibaba Cloud, the group needs to be configured in
advance, and then the consumer can use the group to consume messages. Automatic group
creation is generally not allowed online.
 2024-05-02 08:59:40 -03:00
Thales Macedo Garitezi
14ef0b1e51 feat(kafka consumer): allow custom group id 
Fixes https://emqx.atlassian.net/browse/EMQX-12273
Fixes EMQX-12273

When consuming messages in Kafka in Alibaba Cloud, the group needs to be configured in
advance, and then the consumer can use the group to consume messages. Automatic group
creation is generally not allowed online.
 2024-05-01 13:53:29 -03:00
Kjell Winblad
32c27f1711 feat: add stop after render and after render trace to kafka action 2024-04-25 11:51:46 +02:00
zmstone
ca56e7e8d7 fix(kafka): headers are template fields 2024-04-18 13:04:36 +02:00
zmstone
5c014f4c29 test: fix test cases 2024-04-17 20:34:33 +02:00
zmstone
51c8173174 feat(bridge): add is_template flag to bridge config fields 2024-04-17 13:10:55 +02:00
zmstone
dfd13b4ab5 test: fix dashboard schema json test cases 2024-04-11 15:12:01 +02:00
Ivan Dyachkov
db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Ivan Dyachkov
f4446ec680 Merge remote-tracking branch 'upstream/release-56' into 0328-sync-release-56 2024-03-28 09:59:54 +01:00
Kjell Winblad
94c6e5bb38
Merge pull request #12727 from kjellwinblad/kjell/emqx_connector_info/EMQX-11427 
refactor: add emqx_connector_info behavior
 2024-03-25 14:31:40 +01:00
Thales Macedo Garitezi
bc5775a988 fix(kafka_producer): handle ancient v1 config when migrating to actions 
Fixes https://emqx.atlassian.net/browse/EMQX-12064
 2024-03-22 09:45:27 -03:00
Kjell Winblad
2b684c7bcc fix(emqx_connector_info): refactoring based on thalesmg's comments 2024-03-22 09:29:26 +01:00
Kjell Winblad
4bfc51c052 refactor(emqx_connector_info): improve documentation and type specs 2024-03-21 16:08:31 +01:00
Thales Macedo Garitezi
e837791f94 fix(kafka-based bridges): avoid trying to get raw config for replayq dir 
Fixes https://emqx.atlassian.net/browse/EMQX-12049
 2024-03-21 11:49:00 -03:00
Kjell Winblad
a3e631cda2 refactor(kafka connectors): to use emqx_connector_info 
This commit refactors the `emqx_bridge_kafka` to use the
`emqx_connector_info` behavior. The `emqx_bridge_kafka` related
information can thus be removed from `emqx_connector_chema` and
`emqx_connector_resource`.
 2024-03-19 14:41:23 +01:00
Thales Macedo Garitezi
a852695950 fix(kafka_consumer): validate topic mapping in v2 schema 
Fixes https://emqx.atlassian.net/browse/EMQX-12008
 2024-03-13 09:57:46 -03:00
Thales Macedo Garitezi
963e0de0c3 fix(kafka_consumer): check client connectivity 
Fixes https://emqx.atlassian.net/browse/EMQX-11945
 2024-03-07 14:22:03 -03:00
Thales Macedo Garitezi
5d94a910b9 test: reduce flakiness 2024-02-29 13:59:43 -03:00
Thales Macedo Garitezi
16b3dc1166 refactor: use individual keys for subscriber id resources 2024-02-29 10:39:37 -03:00
Thales Macedo Garitezi
0aa9a872a3 feat: check brod_sup for client id 2024-02-29 09:53:23 -03:00
Thales Macedo Garitezi
6b9844ae82 feat: migrate kafka consumer bridge to source + connector 
Fixes https://emqx.atlassian.net/browse/EMQX-11848
 2024-02-27 17:52:33 -03:00
zhongwencool
f2619ce17a
Merge pull request #12571 from zhongwencool/delete-repeat-ci-test 
test(bridge_v1): delete repeat probe_bridge_api code
 2024-02-23 17:17:39 +08:00
Zaiming (Stone) Shi
46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
zhongwencool
d7faab4195 test(bridge_v1): delete repeat probe_bridge_api code 2024-02-23 14:43:42 +08:00
Zaiming (Stone) Shi
333eb34300 test: call emqx_resource:remove_local 2024-02-21 10:53:22 +01:00
Thales Macedo Garitezi
d469f4158e chore: bump app vsns 2024-02-20 16:53:57 -03:00
Zaiming (Stone) Shi
242df0ffe9 chore: upgrade kafka_protocol from 4.1.3 to 4.1.5 2024-02-16 10:33:53 +01:00
Zaiming (Stone) Shi
d4fb3e83f6 chore: upgrade to wolff 1.10.2 
Wolff 1.10.2 added an enhancement which makes Kafka resource health
checks more lightweight and lowers the chance of false resource alerts.
 2024-02-12 09:05:44 +01:00
William Yang
80ff1b1657
Merge pull request #12434 from qzhuyan/fix/william/kafka-dont-stacktrace-when-no-partiion-count 
kafka-bridge: dont stacktrace when no partition count
 2024-02-01 15:09:45 +01:00
William Yang
16444b7da9 fix(bridge-kafka): no stacktrace when fetch topic partition count fail 2024-02-01 14:09:15 +01:00
Zaiming (Stone) Shi
3b42a7425b feat(kafka_producer): add partitions_limit 2024-01-31 19:26:01 +01:00
Zaiming (Stone) Shi
82403167c2 chore: update BSL license change date 2024-01-29 16:47:31 +01:00
Thales Macedo Garitezi
e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489 
feat: refactor MQTT bridge to source, action, and connector
 2024-01-18 09:28:14 -03:00
Thales Macedo Garitezi
2a41cad54f fix(sources): remote irrelevant resource_opts fields for sources 
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
 2024-01-16 11:29:35 -03:00
Thales Macedo Garitezi
cc24fe6e93 feat(mqtt_consumer): add support for rule engine FROM 2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi
ab66986f16 feat: add 'tags' field for action and connector 2024-01-16 13:57:48 +01:00
JimMoen
5e100f52b8
style: erlfmt all rebar.config files and bin/nodetool 2023-12-29 09:08:03 +08:00
Zaiming (Stone) Shi
23ded313ec chore: update app versions 2023-12-22 15:29:22 +01:00
Zaiming (Stone) Shi
322b7bb7d2 chore: bump app vsn 2023-12-22 13:00:37 +01:00
Thales Macedo Garitezi
cf9331a95f Merge branch 'release-54' into sync-r54-m-20231218 2023-12-18 17:21:08 -03:00
Zaiming (Stone) Shi
7338e394c8 perf: upgrade to wolff 1.9.1 for better performance 
Upgrade from 1.8.0:
- wollf-1.9.0 has the global stats disabled by default
- wolff-1.9.1 improved client pid lookup performance
 2023-12-18 18:54:21 +01:00
Thales Macedo Garitezi
432ddc5a3b fix(bridge_v1_schema): undo changing v1 schema to avoid confusion 2023-12-15 12:55:34 -03:00
Thales Macedo Garitezi
5bdfac52b8 fix(bridge_kafka_producer): add missing resource_opts sub-fields to connector schema 2023-12-04 10:02:05 -03:00
Zaiming (Stone) Shi
55621fbb86
Merge pull request #12074 from zmstone/1201-user-peer-module-for-all 
test: replace 'slave' and 'ct_slave' with 'peer'
 2023-12-01 21:35:40 +01:00
Thales Macedo Garitezi
cb60880bab
Merge pull request #12067 from thalesmg/fix-kafka-check-client-m-20231130 
fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
 2023-12-01 09:55:12 -03:00
Zaiming (Stone) Shi
22f7cc1622 test: replace 'slave' and 'ct_slave' with 'peer' 2023-12-01 08:07:09 +01:00
Thales Macedo Garitezi
0388e1c1c4 fix(kafka_producer): add resource_opts to connector schema, and check for client connectivity 
Fixes https://emqx.atlassian.net/browse/EMQX-11494
 2023-11-30 17:51:32 -03:00