Commit Graph

100 Commits

Author SHA1 Message Date
Kjell Winblad
ef9884cf47 refactor(rule trace): templates rendered trace to increase code reuse
* The code for passing the trace context to a sub process has been
  improved to increase code reuse. This code is used when the action
  templates are rendered in a sub process.
* A macro has also been added for the error term that is thrown when the
  action shall be stopped after the templates has been rendered. This is
  also done to reduce code duplication and to reduce the risk of
  introducing bugs due to typos.
* Fix incorrect type spec

Thanks to @zmstone for suggesting these improvements in comments to a PR
(https://github.com/emqx/emqx/pull/12916).
2024-04-25 13:15:36 +02:00
Kjell Winblad
810aa68b02 feat: add stop after render and after render trace to dynamo action 2024-04-25 11:51:46 +02:00
Kjell Winblad
1f676ce035 feat: add stop after render and after render trace to mqtt action 2024-04-25 11:51:46 +02:00
zmstone
51c8173174 feat(bridge): add is_template flag to bridge config fields 2024-04-17 13:10:55 +02:00
Serge Tupchii
c62410ff75 refactor: remove already bound variable 2024-04-01 17:03:50 +03:00
Ivan Dyachkov
db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Kjell Winblad
4bfc51c052 refactor(emqx_connector_info): improve documentation and type specs 2024-03-21 16:08:31 +01:00
Kjell Winblad
12717baa0b refactor(mqtt connector): to use emqx_connector_info
This commit refactors the mqtt connector to use the
`emqx_connector_info` behavior.
2024-03-20 13:33:13 +01:00
zhongwencool
7f1b4cef27 feat: pulsar bridge v2 2024-02-29 16:23:37 +08:00
Zaiming (Stone) Shi
46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
Thales Macedo Garitezi
d469f4158e chore: bump app vsns 2024-02-20 16:53:57 -03:00
Ilya Averyanov
adf22f1f10 fix(mqtt_bridge): render valid messages from incomplete rule data 2024-01-25 16:01:42 +08:00
Thales Macedo Garitezi
1be5b6aaf3 fix(mqtt_action): use overridden resource_opts
Fixes https://emqx.atlassian.net/browse/EMQX-11790
2024-01-23 09:41:28 -03:00
Thales Macedo Garitezi
d22092e3da fix(mqtt_source): don't attempt local publish when legacy config is absent
Fixes https://emqx.atlassian.net/browse/EMQX-11765
2024-01-19 10:54:05 -03:00
Thales Macedo Garitezi
e369c1b971 docs(mqtt_source): fix example qos 2024-01-19 10:54:05 -03:00
Thales Macedo Garitezi
2a41cad54f fix(sources): remote irrelevant resource_opts fields for sources
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
2024-01-16 11:29:35 -03:00
Thales Macedo Garitezi
a8f9e5676f docs(mqtt_bridge): add API examples 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
938429f351 chore(mqtt_bridge): change schema to remote remote sub-fields and hide local
`local` is still needed for backwards compatibility
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
cc24fe6e93 feat(mqtt_consumer): add support for rule engine FROM 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
28de7c89c7 feat: add /sources* HTTP APIs 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
e6ccfa5b39 fix(mqtt_bridge): fixes after rebasing onto current master
Rebased on top of 7f57ec47d5
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
1ad3100cad chore: add i18n 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
3597ee7c93 fix(mqtt_action): fix resource_opts schema 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
14b99737e9 fix(mqtt_bridge): add missing fields to POST api spec; fix test 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
7befe898d0 fix(mqtt_bridge): fix schema 2024-01-16 10:42:52 -03:00
Kjell Winblad
2ecc775fb7 style: remove commented out code and fix copyright headers 2024-01-16 10:42:52 -03:00
Kjell Winblad
886ed55374 fix: don't call non-existing function 2024-01-16 10:42:52 -03:00
Kjell Winblad
145ed2e632 fix: elvis style error 2024-01-16 10:42:52 -03:00
Kjell Winblad
f199a0f24a feat: refactor MQTT bridge to source, action, and connector
This commit:

* refactors the MQTT V1 bridge into connector, source and action
* Extends the compatibility layer so it works for sources
* Fixes the MQTT bridge test suite so that all test cases passes

We still need to add a HTTP API handling sources. Also, we still need to
add HTTP API example schemes and examples for the MQTT
connector/action/source.

We should also make sure that we handle the corner cases of the MQTT V1
bridge automatic upgrade downgrade in a sufficiently good way:

* An error is currently thrown when converting an MQTT V1 bridge without
  egress or ingress config.
* If there is a source and action with the same name we will currently
  throw an error in the compatibility layer.
* We will also throw an error when converting an MQTT V1 bridge with
  both ingress and egress.

The above is probably the right thing to do  but we have to make sure
that we return a reasonable error to the user when this happens.

(partly)
Fixes:
https://emqx.atlassian.net/browse/EMQX-11489
2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi
4669c4d552 fix(bridge/mqtt): ensure short clientid
Some mqtt brokers do not allow long client IDs.
To make it compatible with this limitation, this commit
tries to limit the number of bytes under 23 with a best-effort
attempt to derive it from the bridge name.
2023-12-28 11:53:27 +01:00
Zaiming (Stone) Shi
ddbb8560fa fix(dialyzer): batch 2 2023-12-08 17:59:55 +01:00
Ivan Dyachkov
9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Thales Macedo Garitezi
f8fd95c683 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 09:22:24 -03:00
Zaiming (Stone) Shi
db33bc616a feat(schema): Add v2 scheam JSON dump 2023-11-22 13:12:35 +01:00
Ivan Dyachkov
7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Andrew Mayorov
7092c75597
Merge pull request #11809 from keynslug/ft/EMQX-10808/file-secrets
feat(mqttbridge): support file-sourced secrets as passwords
2023-11-01 00:29:30 +07:00
JimMoen
b5411da770
refactor: subscribe process to fix shared-sub 2023-10-27 09:23:43 +08:00
Andrew Mayorov
52f4519eeb
feat(mqttbridge): support file-sourced secrets as passwords 2023-10-24 16:24:52 +07:00
Ilya Averyanov
14983ec14a chore(hooks): validate hookpoints and document hook callbacks
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-26 14:27:42 +03:00
Thales Macedo Garitezi
6cd503865b fix(machine_boot): ensure emqx_bridge starts after its companion apps
We need to reverse the dependency of `emqx_bridge` and `emqx_bridge_*`, because the former
loads and starts bridges during its application startup.  If the individual bridge
application being loaded has not started with its dependencies, the supervision tree will
not be ready for that.
2023-07-20 13:11:44 -03:00
Thales Macedo Garitezi
425eba8b13 fix(mqtt_bridge): ensure server key is a binary
Fixes https://emqx.atlassian.net/browse/EMQX-10461

So that it can be JSON encoded correctly.

```
2023-06-30T02:00:41.160110+00:00 [debug] msg: publish_to, mfa: emqx_trace:publish/1, line: 73, topic: b/$, payload: {"topic":"t/1","server":[49,48,46,52,50,46,51,46,49,56,48,58,49,56,56,51],"retain":false,"qos":1,"pub_props":{},"payload":"{\"msg\": \"hello\"}","message_received_at":1688090441159,"id":"0005FF4F2F181488103417000C2E0000","dup":false}, tag: PUBLISH
```
2023-06-30 13:46:24 -03:00
Andrew Mayorov
d6c1ee183f
refactor(pluglib): move emqx_placeholder to utils app
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
JimMoen
ea81a924f1
refactor: move topic() types def in emqx_types.erl 2023-06-07 22:59:52 +08:00
Andrew Mayorov
f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config 2023-06-05 23:32:13 +03:00
Thales Macedo Garitezi
10425eb925 feat(resource): deprecate auto_restart_interval in favor of health_check_interval
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
firest
6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Andrew Mayorov
26819a647c
fix(mqttbridge): clarify schema descriptions + log messages
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-05-30 22:21:22 +03:00
Andrew Mayorov
7e7b50c5ba
refactor(mqttbridge): move into separate application 2023-05-30 17:56:34 +03:00
Shawn
037b75a276 refactor(bridges): remote the old emqx_bridge_mqtt app 2021-09-10 11:43:03 +08:00
Shawn
298cf24f00 fix(bridges): mqtt bridge cannot forward msgs using payload template 2021-09-10 11:43:03 +08:00