Commit Graph

810 Commits

Author SHA1 Message Date
Kjell Winblad 366827390e
Merge pull request #12488 from kjellwinblad/kjell/refactor/rocketmq_bridge/EMQX-11467
feat: refactor RocketMQ bridge to connector and action
2024-02-13 16:19:21 +01:00
Andrew Mayorov a9fdf9f1f9
Merge pull request #12495 from keynslug/ft/EMQX-11830/s3-bridge-simple
feat(s3): introduce S3 connector and action
2024-02-12 16:05:22 +01:00
Andrew Mayorov 802c760406
feat(s3): introduce S3 connector and action
This is a trivial connector based on `emqx_s3` and simple action
that maps each incoming event into an S3 object upload. Due to
current `emqx_s3` limitation this bridge is compatible with
backends providing S3 API with path-style bucket access.
2024-02-09 14:16:32 +01:00
Thales Macedo Garitezi 64367d834b test(bridge_v2): actually check schema 2024-02-08 13:39:31 -03:00
Kjell Winblad e284a83f73 feat: refactor RocketMQ bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11467
2024-02-08 16:21:58 +01:00
zhongwencool b444c82a42 feat: rabbitmq bridge v2 integration 2024-02-06 07:57:12 +08:00
Thales Macedo Garitezi e5b14a38ed
Merge pull request #12472 from thalesmg/fix-bridgev2-api-multicall-m-20240202
fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
2024-02-05 10:28:28 -03:00
Thales Macedo Garitezi 76d242df9b fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
Fixes https://emqx.atlassian.net/browse/EMQX-11834
2024-02-02 17:36:08 -03:00
Thales Macedo Garitezi 76401a302a fix(config): apply config upgrade to deprecated configs
Fixes https://emqx.atlassian.net/browse/EMQX-11845

Fixes https://github.com/emqx/emqx/issues/12452
2024-02-02 15:12:08 -03:00
Kjell Winblad fa0baf2f38
Merge pull request #12439 from kjellwinblad/kjell/refactor/oracle_bridge/EMQX-11464
feat: refactor Oracle bridge to connector and action
2024-02-02 10:53:15 +01:00
lafirest 24d8a9ab00
Merge pull request #12449 from lafirest/feat/td
feat(tdengine): improve the TDengine bridge to v2 style
2024-02-02 16:17:01 +08:00
firest dfad020c49 feat(tdengine): improve the TDengine bridge to v2 style 2024-02-01 23:14:24 +08:00
Kjell Winblad a4272c71dc feat: refactor Oracle bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11464
2024-02-01 16:03:05 +01:00
zhongwencool f9d16340e0 fix: don't add disable bridge to connector's channel 2024-02-01 09:12:43 +08:00
Serge Tupchii 2d693402c5 refactor: split greptimedb bridge to actions and connectors 2024-01-26 11:01:53 +02:00
Kjell Winblad 5bf37cbe0a
Merge pull request #12376 from kjellwinblad/kjell/kinesis
feat: refactor kinesis bridge to connector and action
2024-01-25 14:58:40 +01:00
Shawn 6a21766ce3 refactor: split cassandra bridges to actions and connectors 2024-01-25 16:01:44 +08:00
firest dad8a32e0b feat(opents): improve the OpentsDB bridge to v2 style 2024-01-25 16:01:43 +08:00
JimMoen a752119d05
fix(prom_di): use bridge_v2 metrics api 2024-01-25 09:22:22 +08:00
Kjell Winblad fa842736d2 feat: refactor kinesis bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11461
2024-01-24 18:31:03 +01:00
Thales Macedo Garitezi 67c9e9c8fc fix(actions_api): reset metrics on all nodes
Fixes https://emqx.atlassian.net/browse/EMQX-11796
2024-01-23 10:40:02 -03:00
Xinyu Liu aafb683ec7
Merge pull request #12330 from emqx/bridge-v2-cassandra
refactor: split cassandra bridges to actions and connectors
2024-01-23 14:45:32 +08:00
firest e337e1dc40 feat(opents): improve the OpentsDB bridge to v2 style 2024-01-22 20:45:10 +08:00
Shawn b32c0fb0d8 refactor: split cassandra bridges to actions and connectors 2024-01-22 10:16:46 +08:00
Serge Tupchii dc15d37dcc perf(emqx_bridge/connector): load and unload bridges/connectors in parallel
This should reduce app start/stop time, when a large number of bridges/connectors are not healthy.
2024-01-19 21:44:31 +02:00
Serge Tupchii a6568dec75 perf(emqx_bridge/connector): apply post config bridge/connector changes in parallel
This can greatly improve the performance when many bridges/connectors are being changed,
e.g. when a backup file is being imported.

Fixes: EMQX-11751
2024-01-19 21:44:31 +02:00
Thales Macedo Garitezi 4068258c78 fix(bridge_api): explicitly state bpapi version ranges
Fixes https://emqx.atlassian.net/browse/EMQX-11767

The currently supported version for a node may not be available at the time it's probed.
2024-01-19 10:20:57 -03:00
Thales Macedo Garitezi e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489
feat: refactor MQTT bridge to source, action, and connector
2024-01-18 09:28:14 -03:00
Thales Macedo Garitezi 2a41cad54f fix(sources): remote irrelevant `resource_opts` fields for sources
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
2024-01-16 11:29:35 -03:00
Thales Macedo Garitezi a8f9e5676f docs(mqtt_bridge): add API examples 2024-01-16 10:42:52 -03:00
Kjell Winblad 60fab6ee45 refactor: attempt to improve function names 2024-01-16 10:42:52 -03:00
Kjell Winblad c6cd3adccb refactor: fix type upgrade calls and move compatiblitly logic
Some bridge V1 to V2 calls were wrong but did not seem to cause issues
(perhaps due to locking test coverage). This commit also move
compatibility logic from the API module to the emqx_bridge_v2 module
where most of the compatibility logic exists.
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 440a543a85 docs: fix typo 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 938429f351 chore(mqtt_bridge): change schema to remote `remote` sub-fields and hide `local`
`local` is still needed for backwards compatibility
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 007af20a30 test(bridge_v2_api): adapt more tests to sources 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi fc88a1ed1e test(sources_api): add some tests to cover `/sources` HTTP API
Also fixes a bug with `DELETE /sources/:id`
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 8f304d3456 test(bridge_v2_api): refactor suite to use CT matrix 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi cc24fe6e93 feat(mqtt_consumer): add support for rule engine `FROM` 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 28de7c89c7 feat: add `/sources*` HTTP APIs 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 6511693b2e refactor(action_api): prepare for `/sources` HTTP API 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 7fc069da46 test: fix another broken test 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi ab1b0dda67 refactor: fix typo 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 697c8f5ee1 test: fix broken tests 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi cc34660ab9 fix(actions): use backward-compatible ids 2024-01-16 10:42:52 -03:00
Kjell Winblad f199a0f24a feat: refactor MQTT bridge to source, action, and connector
This commit:

* refactors the MQTT V1 bridge into connector, source and action
* Extends the compatibility layer so it works for sources
* Fixes the MQTT bridge test suite so that all test cases passes

We still need to add a HTTP API handling sources. Also, we still need to
add HTTP API example schemes and examples for the MQTT
connector/action/source.

We should also make sure that we handle the corner cases of the MQTT V1
bridge automatic upgrade downgrade in a sufficiently good way:

* An error is currently thrown when converting an MQTT V1 bridge without
  egress or ingress config.
* If there is a source and action with the same name we will currently
  throw an error in the compatibility layer.
* We will also throw an error when converting an MQTT V1 bridge with
  both ingress and egress.

The above is probably the right thing to do  but we have to make sure
that we return a reasonable error to the user when this happens.

(partly)
Fixes:
https://emqx.atlassian.net/browse/EMQX-11489
2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi ab66986f16 feat: add 'tags' field for action and connector 2024-01-16 13:57:48 +01:00
Zaiming (Stone) Shi 58a54adbb4 Merge remote-tracking branch 'origin/master' into release-55 2024-01-15 17:58:48 +01:00
zhongwencool ace443fc18 refactor: refactor es's action 2024-01-15 11:31:13 +08:00
zhongwencool e49d3ca50c feat: support elasticsearch bridge 2024-01-15 09:33:44 +08:00
Thales Macedo Garitezi a5e7db793b fix(mongodb_action): make `batch_size` hidden and fixed in the API
Fixes https://emqx.atlassian.net/browse/ED-1171

The old bridge schema already had this override, but it had not been ported to actions.
2024-01-12 10:04:32 -03:00