Commit Graph

336 Commits

Author SHA1 Message Date
Thales Macedo Garitezi c916c83c7c feat: implement azure blob storage action
Fixes https://emqx.atlassian.net/browse/EMQX-12280
2024-05-31 11:07:44 -03:00
Kjell Winblad 26c988fe11
Merge pull request #13077 from kjellwinblad/kjell/fix_action_update_race_issue/EMQX-12376
fix: action config update would sometimes not be reflected in connector
2024-05-23 09:11:48 +02:00
Thales Macedo Garitezi 4094032649 fix(bridge v2 api): don't attempt to start disabled connector when starting action/source
Fixes https://emqx.atlassian.net/browse/EMQX-12435
2024-05-22 13:33:55 -03:00
Kjell Winblad 917474f694 fix: action config update would sometimes not be reflected in connector
Before this commit the following happened sometimes:

1. action status is connected
2. action config is updated to something that should change the status to
   disconnected
3. action status is still connected and the old config is being used by
   the connector even though the config has been correctly updated.

The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).

The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00
Kjell Winblad e32745bca6 test(apply rule trace): add test case for republish and console print 2024-05-03 15:02:37 +02:00
Thales Macedo Garitezi a847389159 fix(http connector): deobfuscate sensitive headers
Fixes https://emqx.atlassian.net/browse/EMQX-12213
2024-04-29 15:20:46 -03:00
zhongwencool c0521fd250 feat: support iotdb 1.3.0 2024-04-19 08:54:18 +08:00
Thales Macedo Garitezi 60cad74286 feat(resource): non-blocking channel health checks
Fixes https://emqx.atlassian.net/browse/EMQX-12015

Continuation of https://github.com/emqx/emqx/pull/12812
2024-04-04 16:13:30 -03:00
Thales Macedo Garitezi bade09b56e feat(resource manager): perform non-blocking resource health checks
Fixes https://emqx.atlassian.net/browse/EMQX-12015

This introduces only _resource_ non-blocking health checks.  _Channel_ non-blocking health
checks may be introduced later.
2024-04-01 14:46:15 -03:00
Kjell Winblad 94c6e5bb38
Merge pull request #12727 from kjellwinblad/kjell/emqx_connector_info/EMQX-11427
refactor: add emqx_connector_info behavior
2024-03-25 14:31:40 +01:00
Kjell Winblad 89befa580e refactor(emqx_connector_info): remove code that is no longer needed 2024-03-21 11:58:54 +01:00
Ivan Dyachkov f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
Thales Macedo Garitezi 1edf284fed test: trigger hocon validators when checking schema 2024-03-12 14:32:16 -03:00
Thales Macedo Garitezi 79d7821222
Merge pull request #12655 from thalesmg/fix-kconsu-status-r56-20240306
fix(kafka_consumer): check client connectivity
2024-03-08 10:53:44 -03:00
Thales Macedo Garitezi 963e0de0c3 fix(kafka_consumer): check client connectivity
Fixes https://emqx.atlassian.net/browse/EMQX-11945
2024-03-07 14:22:03 -03:00
Thales Macedo Garitezi 676df7eb30 test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
Checks that, if a migrated bridge originally has a `topic_mapping` and is later updated
with V2 APIs (without topic mapping in the input), then the new V2 `topic` field prevails.
2024-03-05 09:12:24 -03:00
Thales Macedo Garitezi 6b9844ae82 feat: migrate kafka consumer bridge to source + connector
Fixes https://emqx.atlassian.net/browse/EMQX-11848
2024-02-27 17:52:33 -03:00
Zaiming (Stone) Shi 46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
Thales Macedo Garitezi 205b97f732 fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config
Fixes https://emqx.atlassian.net/browse/EMQX-11891
2024-02-22 10:27:28 -03:00
Andrew Mayorov 7feb444e31
Merge pull request #12557 from keynslug/fix/s3-bridge-startup
fix(s3-bridge): anticipate repeated connector start requests
2024-02-21 15:13:48 +01:00
Andrew Mayorov 6b97983303
fix(s3-bridge): anticipate repeated connector start requests
Starting disconnected connector manually implies that there's no
attempt to stop it first.
2024-02-21 14:23:29 +01:00
Zaiming (Stone) Shi 415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
Thales Macedo Garitezi 4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
Zaiming (Stone) Shi 333eb34300 test: call emqx_resource:remove_local 2024-02-21 10:53:22 +01:00
Andrew Mayorov d8b6ecd185
fix(bridge-s3): expose connector-level `resource_opts` properly 2024-02-21 10:49:04 +01:00
Thales Macedo Garitezi 146d89bd89 feat: convert `gcp_pubsub_consumer` to connector/source
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 17:54:26 -03:00
Thales Macedo Garitezi 9a32895a1a feat: convert `gcp_pubsub_consumer` to connector/source
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
Thales Macedo Garitezi 64367d834b test(bridge_v2): actually check schema 2024-02-08 13:39:31 -03:00
Thales Macedo Garitezi e5b14a38ed
Merge pull request #12472 from thalesmg/fix-bridgev2-api-multicall-m-20240202
fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
2024-02-05 10:28:28 -03:00
Thales Macedo Garitezi 76d242df9b fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
Fixes https://emqx.atlassian.net/browse/EMQX-11834
2024-02-02 17:36:08 -03:00
Thales Macedo Garitezi 76401a302a fix(config): apply config upgrade to deprecated configs
Fixes https://emqx.atlassian.net/browse/EMQX-11845

Fixes https://github.com/emqx/emqx/issues/12452
2024-02-02 15:12:08 -03:00
zhongwencool f9d16340e0 fix: don't add disable bridge to connector's channel 2024-02-01 09:12:43 +08:00
Thales Macedo Garitezi 67c9e9c8fc fix(actions_api): reset metrics on all nodes
Fixes https://emqx.atlassian.net/browse/EMQX-11796
2024-01-23 10:40:02 -03:00
Serge Tupchii a6568dec75 perf(emqx_bridge/connector): apply post config bridge/connector changes in parallel
This can greatly improve the performance when many bridges/connectors are being changed,
e.g. when a backup file is being imported.

Fixes: EMQX-11751
2024-01-19 21:44:31 +02:00
Thales Macedo Garitezi 2a41cad54f fix(sources): remote irrelevant `resource_opts` fields for sources
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
2024-01-16 11:29:35 -03:00
Thales Macedo Garitezi 938429f351 chore(mqtt_bridge): change schema to remote `remote` sub-fields and hide `local`
`local` is still needed for backwards compatibility
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 007af20a30 test(bridge_v2_api): adapt more tests to sources 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi fc88a1ed1e test(sources_api): add some tests to cover `/sources` HTTP API
Also fixes a bug with `DELETE /sources/:id`
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 8f304d3456 test(bridge_v2_api): refactor suite to use CT matrix 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi cc24fe6e93 feat(mqtt_consumer): add support for rule engine `FROM` 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 28de7c89c7 feat: add `/sources*` HTTP APIs 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 6511693b2e refactor(action_api): prepare for `/sources` HTTP API 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 7fc069da46 test: fix another broken test 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 697c8f5ee1 test: fix broken tests 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 79a4a041e4 fix(bridge_api): correctly deobfuscate secrets during dry run
Fixes https://emqx.atlassian.net/browse/EMQX-11733
2024-01-11 15:42:14 -03:00
Thales Macedo Garitezi abd4873091 fix(mysql_bridge): allow deleting bridge when sql contains undefined fields
Fixes https://emqx.atlassian.net/browse/EMQX-11655
2024-01-09 17:25:34 -03:00
zhongwencool ec34b6f41d refactor: remove redis_type from redis action 2024-01-08 15:09:37 +08:00
firest e20b024b6e fix(iotdb): fix iotdb testcases 2024-01-06 13:01:43 +08:00
Thales Macedo Garitezi b61ccf3373 fix(bridges_v1): don't list v2-only bridges in API
Fixes https://emqx.atlassian.net/browse/EMQX-11614
2023-12-20 10:46:35 -03:00
Thales Macedo Garitezi 3f06ebcaf7
Merge pull request #12181 from thalesmg/fix-fill-default-v1-api-return-r54-20231215
fix(bridge_v1_api): fill defaults for v2 raw configs and fix redis connector schema
2023-12-18 11:34:07 -03:00