Commit Graph

138 Commits

Author SHA1 Message Date
Kjell Winblad ef9884cf47 refactor(rule trace): templates rendered trace to increase code reuse
* The code for passing the trace context to a sub process has been
  improved to increase code reuse. This code is used when the action
  templates are rendered in a sub process.
* A macro has also been added for the error term that is thrown when the
  action shall be stopped after the templates has been rendered. This is
  also done to reduce code duplication and to reduce the risk of
  introducing bugs due to typos.
* Fix incorrect type spec

Thanks to @zmstone for suggesting these improvements in comments to a PR
(https://github.com/emqx/emqx/pull/12916).
2024-04-25 13:15:36 +02:00
Kjell Winblad 810aa68b02 feat: add stop after render and after render trace to dynamo action 2024-04-25 11:51:46 +02:00
Kjell Winblad 1f676ce035 feat: add stop after render and after render trace to mqtt action 2024-04-25 11:51:46 +02:00
zmstone 51c8173174 feat(bridge): add is_template flag to bridge config fields 2024-04-17 13:10:55 +02:00
Serge Tupchii c62410ff75 refactor: remove already bound variable 2024-04-01 17:03:50 +03:00
Ivan Dyachkov db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Kjell Winblad 4bfc51c052 refactor(emqx_connector_info): improve documentation and type specs 2024-03-21 16:08:31 +01:00
Kjell Winblad 12717baa0b refactor(mqtt connector): to use emqx_connector_info
This commit refactors the mqtt connector to use the
`emqx_connector_info` behavior.
2024-03-20 13:33:13 +01:00
zhongwencool 7f1b4cef27 feat: pulsar bridge v2 2024-02-29 16:23:37 +08:00
Zaiming (Stone) Shi 46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
Thales Macedo Garitezi d469f4158e chore: bump app vsns 2024-02-20 16:53:57 -03:00
ieQu1 59e4db98f7
test(bridge_mqtt): Stop snabbkaffe servers 2024-02-16 12:14:19 +01:00
Ilya Averyanov adf22f1f10 fix(mqtt_bridge): render valid messages from incomplete rule data 2024-01-25 16:01:42 +08:00
Thales Macedo Garitezi 1be5b6aaf3 fix(mqtt_action): use overridden `resource_opts`
Fixes https://emqx.atlassian.net/browse/EMQX-11790
2024-01-23 09:41:28 -03:00
Thales Macedo Garitezi d22092e3da fix(mqtt_source): don't attempt local publish when legacy config is absent
Fixes https://emqx.atlassian.net/browse/EMQX-11765
2024-01-19 10:54:05 -03:00
Thales Macedo Garitezi e369c1b971 docs(mqtt_source): fix example qos 2024-01-19 10:54:05 -03:00
Thales Macedo Garitezi 2a41cad54f fix(sources): remote irrelevant `resource_opts` fields for sources
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
2024-01-16 11:29:35 -03:00
Thales Macedo Garitezi a8f9e5676f docs(mqtt_bridge): add API examples 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 938429f351 chore(mqtt_bridge): change schema to remote `remote` sub-fields and hide `local`
`local` is still needed for backwards compatibility
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi cc24fe6e93 feat(mqtt_consumer): add support for rule engine `FROM` 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 28de7c89c7 feat: add `/sources*` HTTP APIs 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi e6ccfa5b39 fix(mqtt_bridge): fixes after rebasing onto current `master`
Rebased on top of 7f57ec47d5
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 12dc9fbeb9 test(mqtt_bridge): fix assertion 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 1ad3100cad chore: add i18n 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 3597ee7c93 fix(mqtt_action): fix resource_opts schema 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 14b99737e9 fix(mqtt_bridge): add missing fields to POST api spec; fix test 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 7befe898d0 fix(mqtt_bridge): fix schema 2024-01-16 10:42:52 -03:00
Kjell Winblad 2ecc775fb7 style: remove commented out code and fix copyright headers 2024-01-16 10:42:52 -03:00
Kjell Winblad 886ed55374 fix: don't call non-existing function 2024-01-16 10:42:52 -03:00
Kjell Winblad 145ed2e632 fix: elvis style error 2024-01-16 10:42:52 -03:00
Kjell Winblad f199a0f24a feat: refactor MQTT bridge to source, action, and connector
This commit:

* refactors the MQTT V1 bridge into connector, source and action
* Extends the compatibility layer so it works for sources
* Fixes the MQTT bridge test suite so that all test cases passes

We still need to add a HTTP API handling sources. Also, we still need to
add HTTP API example schemes and examples for the MQTT
connector/action/source.

We should also make sure that we handle the corner cases of the MQTT V1
bridge automatic upgrade downgrade in a sufficiently good way:

* An error is currently thrown when converting an MQTT V1 bridge without
  egress or ingress config.
* If there is a source and action with the same name we will currently
  throw an error in the compatibility layer.
* We will also throw an error when converting an MQTT V1 bridge with
  both ingress and egress.

The above is probably the right thing to do  but we have to make sure
that we return a reasonable error to the user when this happens.

(partly)
Fixes:
https://emqx.atlassian.net/browse/EMQX-11489
2024-01-16 10:42:52 -03:00
Ivan Dyachkov 06117c3a33 Merge remote-tracking branch 'upstream/release-54' into 0105-sync-r54 2024-01-05 14:20:38 +01:00
JimMoen 5e100f52b8
style: erlfmt all `rebar.config` files and `bin/nodetool` 2023-12-29 09:08:03 +08:00
Zaiming (Stone) Shi 4669c4d552 fix(bridge/mqtt): ensure short clientid
Some mqtt brokers do not allow long client IDs.
To make it compatible with this limitation, this commit
tries to limit the number of bytes under 23 with a best-effort
attempt to derive it from the bridge name.
2023-12-28 11:53:27 +01:00
Zaiming (Stone) Shi 9487635957 test(mqtt-bridge): do not assert map key order
starting from otp 26, small maps (less than 32 fields), the atom
key orders are no longer deterministic
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi ddbb8560fa fix(dialyzer): batch 2 2023-12-08 17:59:55 +01:00
Ivan Dyachkov 9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Thales Macedo Garitezi f8fd95c683 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 09:22:24 -03:00
Zaiming (Stone) Shi db33bc616a feat(schema): Add v2 scheam JSON dump 2023-11-22 13:12:35 +01:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Ivan Dyachkov 0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
Andrew Mayorov 7092c75597
Merge pull request #11809 from keynslug/ft/EMQX-10808/file-secrets
feat(mqttbridge): support file-sourced secrets as passwords
2023-11-01 00:29:30 +07:00
Kjell Winblad 9dc3a169b3 feat: split bridges into a connector part and a bridge part
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>

Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.

Issues:
https://emqx.atlassian.net/browse/EMQX-10805
2023-10-30 14:48:47 +01:00
JimMoen b5411da770
refactor: subscribe process to fix shared-sub 2023-10-27 09:23:43 +08:00
Andrew Mayorov 44b4205561
fix(secret): do not treat missing file secrets as config error
They are intended to be used mostly in the context of resources, which
have their own feedback mechanism: statuses, retries, etc.

Also turn the error into a throw exception, so that it can be
interpreted as a regular error condition, for example by the resource
manager.
2023-10-26 14:37:14 +07:00
Andrew Mayorov 52f4519eeb
feat(mqttbridge): support file-sourced secrets as passwords 2023-10-24 16:24:52 +07:00
Ilya Averyanov 14983ec14a chore(hooks): validate hookpoints and document hook callbacks
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-26 14:27:42 +03:00
Thales Macedo Garitezi 78c5a779d7 feat(republish): allow templating mqtt properties
Fixes https://emqx.atlassian.net/browse/EMQX-10912
2023-09-06 09:54:02 -03:00
Thales Macedo Garitezi 6cd503865b fix(machine_boot): ensure `emqx_bridge` starts after its companion apps
We need to reverse the dependency of `emqx_bridge` and `emqx_bridge_*`, because the former
loads and starts bridges during its application startup.  If the individual bridge
application being loaded has not started with its dependencies, the supervision tree will
not be ready for that.
2023-07-20 13:11:44 -03:00
Thales Macedo Garitezi 425eba8b13 fix(mqtt_bridge): ensure `server` key is a binary
Fixes https://emqx.atlassian.net/browse/EMQX-10461

So that it can be JSON encoded correctly.

```
2023-06-30T02:00:41.160110+00:00 [debug] msg: publish_to, mfa: emqx_trace:publish/1, line: 73, topic: b/$, payload: {"topic":"t/1","server":[49,48,46,52,50,46,51,46,49,56,48,58,49,56,56,51],"retain":false,"qos":1,"pub_props":{},"payload":"{\"msg\": \"hello\"}","message_received_at":1688090441159,"id":"0005FF4F2F181488103417000C2E0000","dup":false}, tag: PUBLISH
```
2023-06-30 13:46:24 -03:00