Commit Graph

103 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 2f46e86db8 test: refactor test group generation 2023-10-30 01:43:39 +01:00
Zaiming (Stone) Shi 289428cc5a refactor(kafka): rename bridge type 'kafka' to 'kafka_producer' 2023-10-30 00:19:27 +01:00
Stefan Strigler 1dea3e1cc4 test: add suite for AEH bridge v2 2023-10-27 20:00:21 +02:00
Zaiming (Stone) Shi 5f17a8f2ce fix(kafka): use client config for topic existence check
Prior to this fix, Kafka producer config was used as client config
2023-10-27 11:53:42 +02:00
Thales Macedo Garitezi 04a832a80a refactor(kafka): fix typo and take connector type from input 2023-10-27 11:53:42 +02:00
Ivan Dyachkov 7745cb8ef7 test: fix emqx_bridge_v2_kafka_producer_SUITE 2023-10-27 11:53:42 +02:00
Kjell Winblad 90a23d98fe fix: channel handling when resource not connected 2023-10-27 11:53:42 +02:00
Zaiming (Stone) Shi e2aca352b6 fix(kafka): Fix server name indication config for Kafka
Also impoved azur event hub's doc for SNI
2023-10-27 11:53:41 +02:00
Stefan Strigler 2228a0d477 feat: port azure event hub to bridge_v2 2023-10-27 11:53:41 +02:00
Kjell Winblad e72b952138 fix: problems reported by dialyzer 2023-10-27 11:53:41 +02:00
Kjell Winblad 2cd1c88f7f fix: fixup after rebasing on release-23 2023-10-27 11:53:41 +02:00
Stefan Strigler 6568759a7e fix: add bridge_v2_probe 2023-10-27 11:53:41 +02:00
Stefan Strigler 38f5c7dcf4 feat: HTTP API for bridge_v2 2023-10-27 11:53:41 +02:00
Kjell Winblad 828bbc57ac feat: test case fixes and compatibility layer probe etc
* test case fixes for Bridge V1 suite
* Bug fixes
* local_topic
* Bridge V1 probe compatibility functionality
2023-10-27 11:53:41 +02:00
Stefan Strigler 8567ccafc1 fix: don't assume resource-id starts with "connector:" 2023-10-27 11:53:40 +02:00
Stefan Strigler 4b4eb19b0b fix: add schema desc for connector 2023-10-27 11:53:40 +02:00
Stefan Strigler 229bc0ee34 fix: schema fixes for connector and kafka as connector 2023-10-27 11:53:40 +02:00
Kjell Winblad c0df85ac09 feat: Bridge V2 compatiblilty layer progress and local topic
* Most Bridge V1 HTTP API calls are now compatible with Bridge V2
* Local topics works for Bridge V2 now
* A lot of work on trying to get the old Kafka producer test suite
  to work after the refactorings
2023-10-27 11:53:40 +02:00
Kjell Winblad f7984be946 feat: add connector schema scaffold and break out Kafka conector
This commit is the beginning of an effort to split bridges into a
connector part and a bridge part.

Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.

Issues:
https://emqx.atlassian.net/browse/EMQX-10805
2023-10-27 11:53:40 +02:00
Zaiming (Stone) Shi abcb3166f6 fix: use binary string for raw default values 2023-10-24 14:46:01 +02:00
Thales Macedo Garitezi eebfb44f72 fix(resource): create `simple_async_internal_buffer` query mode for bridges with internal buffering
Since authn/authz backends also use simple async/sync queries, we may want to avoid them
calling the connector when it's not connected.
2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi 79cf0a2ced fix(kafka_producer): correctly handle metrics for connector that have internal buffers
Fixes https://emqx.atlassian.net/browse/EMQX-11086

There’s currently a metric inconsistency due to the internal buffering nature of Kafka
Producer (wolff).

We use simple_sync_query to call the Kafka Producer bridge.  If that times out, the call
is accounted as failed, even though the message is buffered in wolff and later sent
successfully.
2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi 34186fcc74 fix(kafka_producer): send messages to wolff producer to buffer even when connector is in `connecting` state
Fixes https://emqx.atlassian.net/browse/EMQX-11085

Messages would not be sent to wolff if the connection was down, so they were effectively lost.
2023-10-06 11:43:29 -03:00
Zaiming (Stone) Shi bce8fd2fbc chore: bump app versions 2023-09-29 18:56:52 +02:00
Zaiming (Stone) Shi 1a13b2ac56 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-28 13:53:03 +02:00
Zaiming (Stone) Shi ea8d54fd8b test: ensure atom exists in test module 2023-09-27 12:58:06 +02:00
Ilya Averyanov 14983ec14a chore(hooks): validate hookpoints and document hook callbacks
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-26 14:27:42 +03:00
firest 9e55ae240a feat(sso): add `role` into the result of login endpoints 2023-09-22 13:48:07 +08:00
Ivan Dyachkov dafd7c6085 chore: bump apps versions 2023-09-21 10:58:42 +02:00
zhongwencool 9e75ff88af
chore: apply suggestions from code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:15:39 +08:00
zhongwencool 2f1fa2e961 chore: unified slog message formatting to improve logging consistency 2023-09-20 18:13:00 +08:00
Zaiming (Stone) Shi 99b60849b4 test: fix flaky test emqx_falpping_SUITE:t_conf_update_timer 2023-09-18 20:44:38 +02:00
Zaiming (Stone) Shi 7c2f87fabe test: merge broker and router boot modules 2023-09-06 21:36:16 +02:00
Zaiming (Stone) Shi 18c6bfec97 chore: bump app vsns 2023-09-06 11:12:56 +02:00
Zaiming (Stone) Shi e794143ae1 Merge remote-tracking branch 'origin/release-52' into 0906-sync-release-52-to-master 2023-09-06 09:08:22 +02:00
Zaiming (Stone) Shi 876d539336 Merge remote-tracking branch 'andrew/ft/EMQX-10713/unified-route-tab' into 0831-make-use-of-new-routing-table 2023-08-31 15:46:33 +02:00
Paulo Zulato ee77976424 fix(kafka): avoid producer name collision between Kafka and AEH bridges
Fixes https://emqx.atlassian.net/browse/EMQX-10860
2023-08-30 14:48:10 -03:00
Paulo Zulato 0b86f04bae fix(kafka): fix template processing for header
Fixes https://emqx.atlassian.net/browse/EMQX-10846
2023-08-28 17:28:05 -03:00
Andrew Mayorov e85789306b
chore(router): drop test-only `has_routes/1`
Seems that it's too much support work only for test purposes, where
`lookup_routes/1` is nearly as usable.
2023-08-28 14:07:46 +04:00
Paulo Zulato 84e68f8ed8
Merge pull request #11508 from paulozulato/fix-azure-msg-error-handling
fix(kafka): fix result handling when sending message with invalid header
2023-08-25 14:57:09 -03:00
Paulo Zulato cb1e105e19
Merge pull request #11516 from paulozulato/fix-kafka-aeh-client-unique-id
Fix Kafka/AEH ClientId uniqueness
2023-08-25 10:14:44 -03:00
Paulo Zulato 535c7f8b43 fix(kafka): avoid ClientId collision between Kafka and Azure bridges
Fixes https://emqx.atlassian.net/browse/EMQX-10860
2023-08-24 18:46:22 -03:00
Thales Macedo Garitezi 0381ac0410 fix(kafka_producer): use correct timestamp template field
Fixes https://emqx.atlassian.net/browse/EMQX-10847
2023-08-24 14:20:42 -03:00
Paulo Zulato 60e6217496 fix(kafka): fix result handling when sending message with invalid header
Fixes https://emqx.atlassian.net/browse/EMQX-10846
2023-08-24 10:41:42 -03:00
Zaiming (Stone) Shi 01c9095982
Merge pull request #11394 from zmstone/0804-upgrade-wolff-to-1.7.7
chore: upgrade Kafka client wolff to 1.7.7
2023-08-20 10:14:13 +02:00
Thales Macedo Garitezi ffca581229 feat(kafka): add option to configure health check interval
Fixes https://emqx.atlassian.net/browse/EMQX-10781
2023-08-16 11:08:06 -03:00
Thales Macedo Garitezi b6decf9592
Merge pull request #11402 from thalesmg/dynamic-kconsu-mqtt-topics-20230807
feat(kafka_consumer): add mqtt topic placeholder support
2023-08-07 15:10:29 -03:00
Thales Macedo Garitezi 5c8dc092a1 fix(http_bridge): don't attempt to convert headers to atoms
Fixes https://emqx.atlassian.net/browse/EMQX-10653
2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi 9900a32850 feat(kafka_consumer): add mqtt topic placeholder support
Fixes https://emqx.atlassian.net/browse/EMQX-10678
2023-08-07 11:42:34 -03:00
Zaiming (Stone) Shi 115ab85670 chore: upgrade Kafka client wolff to 1.7.7 2023-08-04 15:31:34 +02:00