Commit Graph

61 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
6cd503865b fix(machine_boot): ensure emqx_bridge starts after its companion apps
We need to reverse the dependency of `emqx_bridge` and `emqx_bridge_*`, because the former
loads and starts bridges during its application startup.  If the individual bridge
application being loaded has not started with its dependencies, the supervision tree will
not be ready for that.
2023-07-20 13:11:44 -03:00
Thales Macedo Garitezi
425eba8b13 fix(mqtt_bridge): ensure server key is a binary
Fixes https://emqx.atlassian.net/browse/EMQX-10461

So that it can be JSON encoded correctly.

```
2023-06-30T02:00:41.160110+00:00 [debug] msg: publish_to, mfa: emqx_trace:publish/1, line: 73, topic: b/$, payload: {"topic":"t/1","server":[49,48,46,52,50,46,51,46,49,56,48,58,49,56,56,51],"retain":false,"qos":1,"pub_props":{},"payload":"{\"msg\": \"hello\"}","message_received_at":1688090441159,"id":"0005FF4F2F181488103417000C2E0000","dup":false}, tag: PUBLISH
```
2023-06-30 13:46:24 -03:00
Andrew Mayorov
d6c1ee183f
refactor(pluglib): move emqx_placeholder to utils app
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
JimMoen
ea81a924f1
refactor: move topic() types def in emqx_types.erl 2023-06-07 22:59:52 +08:00
Andrew Mayorov
f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config 2023-06-05 23:32:13 +03:00
Thales Macedo Garitezi
10425eb925 feat(resource): deprecate auto_restart_interval in favor of health_check_interval
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
firest
6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Andrew Mayorov
26819a647c
fix(mqttbridge): clarify schema descriptions + log messages
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-05-30 22:21:22 +03:00
Andrew Mayorov
7e7b50c5ba
refactor(mqttbridge): move into separate application 2023-05-30 17:56:34 +03:00
Shawn
037b75a276 refactor(bridges): remote the old emqx_bridge_mqtt app 2021-09-10 11:43:03 +08:00
Shawn
298cf24f00 fix(bridges): mqtt bridge cannot forward msgs using payload template 2021-09-10 11:43:03 +08:00
Shawn
eb8822ce41 feat(bridges): start one mqtt bridge worker for each in/out channel 2021-09-10 11:43:03 +08:00
Shawn
4f82debbe7 feat(bridge): new configs for mqtt bridge 2021-09-10 11:43:03 +08:00
Shawn
39bb1b8d9d fix(bridges): start mqtt bridge failed 2021-09-10 11:43:03 +08:00
Shawn
9ed90ba7a9 feat(bridges): add connector for mqtt bridge 2021-09-10 11:43:03 +08:00
Zaiming Shi
ec13463f4a refactor(schema): prepare for hocon schema doc generation 2021-09-03 11:02:31 +02:00
Zaiming Shi
5165fd6b30 refactor(schema): implement new hocon_schema callbacks 2021-08-30 09:56:03 +08:00
Turtle
e4f5e9332e feat: support array bridge_mqtt conf 2021-08-25 13:34:00 +08:00
Shawn
e8e95d39ef
refactor(config): move emqx_config:get/get_raw to emqx:get_config/get_raw_config (#5517) 2021-08-18 14:52:57 +08:00
Zaiming Shi
e6232665a3 refactor(emqx): ensure app shutdown and reboot order
make use of application dependency to ensure start order.
also the same dependency top-sorted for shutdown and reboot order
2021-08-04 08:40:03 +02:00
Zaiming Shi
5d59ac1f02 refactor(logging): delete log headers from all modules 2021-08-01 09:36:59 +02:00
Turtle
9e97de04c9 chore(plugins): delete emqx_plugins module attributes 2021-07-30 14:32:01 +08:00
Turtle
918a26e921 feat(conf): merge all conf to emqx.conf 2021-07-02 17:01:16 +08:00
turtleDeng
d4f726419f
feat(bridge-mqtt): Update the configuration file to hocon (#5142) 2021-07-01 10:51:59 +08:00
Turtle
faad90c9d4 chore(plugins): rm emqx-web-hook and mv webhook action to emqx_rule_actions 2021-06-28 17:07:46 +08:00
Turtle
c02a9f0101 fix(appup): update appup 2021-05-27 13:55:07 +08:00
Zaiming Shi
6354e75626 chore: update copyrights 2021-04-23 20:43:00 +02:00
Zaiming Shi
d66f67d411 fix(emqx_bridge_mqtt): fix retry_inflight
The Inflight list should not be used to update State.inflight
2021-04-10 14:16:08 +02:00
Zaiming Shi
ae9278a409 fix(bridge_mqtt): push to max-inflight limit
the old pop_and_send implementation may leave inflight queue starving
2021-04-09 10:18:25 +02:00
Zaiming Shi
ae688e2c90 fix(bridge_mqtt): fix inflight reference booking
Prior to this change, the inflight batches are referenced
by the last packet ID for non-QoS-0 messages, other packet
IDs sent back from downstream causes an error log:
"Can't be found from the inflight"

Even worse, the batch is appended back to the queue for retry.
2021-04-09 10:18:25 +02:00
Zaiming Shi
71559dd001 refactor(ssl): Move data dir for ssl certs upload down to lib module 2021-03-22 16:41:01 +01:00
Zaiming Shi
8a41a4ee00 fix(ssl): Fix dir name for uploaded certificates
The dir is "rules" not "rule" in the original code
ref: f1eba1c648/src/emqx_rule_actions_utils.erl
2021-03-22 16:41:01 +01:00
z8674558
119bbc4881 chore(emqx_bridge_msg): add empty props 2021-03-03 20:06:31 +09:00
JianBo He
2fb5dbd546 revert(bridge-mqtt): remove the mqtt_sub resource
We have several reasons to remove this feature:

1. The design does not make sense. A rule engine resource should
not have an impact on the system's messages directly after it is created.
This mqtt_sub actually conflicts with any design concept of the rules engine.

2. The implementation is incorrect. mqtt_sub uses a client pool to establish
a subscription relationship to an MQTT Broker. This causes a message to be
sent repeatedly to EMQ X. Unless a shared subscription is used,
or a Pool Size of 1 is configured.

3. The emqx-bridge-mqtt supports all the features of mqtt_sub.

This feature introduced by https://github.com/emqx/emqx-bridge-mqtt/pull/78.
And it released to v4.2.0 (NOT WORK), v4.2.1-v4.2.7 (FIXED)
2021-03-02 18:56:21 +08:00
z8674558
2e199126dc fix(emqx_bridge_mqtt_actions): string -> file 2021-03-02 15:00:42 +09:00
z8674558
829a39eade fix(emqx_bridge_mqtt_actions): string -> password 2021-03-02 15:00:42 +09:00
Shawn
990b7bd494 fix(bridge_mqtt): typos 2021-03-01 21:00:17 +08:00
Zaiming Shi
675603fd73 fix(bridge-mqtt): no ssl-option at all (empty list) when ssl is disabled 2021-03-01 21:00:17 +08:00
Zaiming Shi
700fa71754 refactor(tls): abstract lib for tls options parsing 2021-03-01 21:00:17 +08:00
Zaiming Shi
da6f1104dc chore(bridge-mqtt): sync enterprise code 2021-03-01 21:00:17 +08:00
Zaiming Shi
f000b6583c fix(tls): Ensure tls config integrity
For default tsl version and ciphers, we try to use otp release number
to determin if we want to use tlsv1.3

For default configs, we try to porivde both tlsv1.3 and
ciphers in config (even for commented out configs)
2021-02-09 22:56:28 +01:00
JianBo He
456e3e5faa chore(bridge-mqtt): fix styles 2021-01-28 16:24:42 +08:00
JianBo He
d98d26942c fix(bridge-mqtt): add max_inflight_size option back
For compatibility with versions below 4.2, we still
use the `max_inflight_size` option, but generate it
as the `max_inflight`

Previous PR: https://github.com/emqx/emqx/pull/3938
2021-01-28 16:24:42 +08:00
JianBo He
67e56658a7 fix(bridge-mqtt): fix bad time unit 2021-01-25 15:35:04 +08:00
JianBo He
3de8fea435 fix(bridge-mqtt): fix bad received metrics counting 2021-01-25 15:35:04 +08:00
Yudai Kiyofuji
56920b6ca6
fix(bridge_mqtt): max_inflight_size -> max_inflight; this enables emqtt.erl to use the value (#3938)
* fix(bridge_mqtt): max_inflight_size -> max_inflight; this enables emqtt.erl to use the value

* chore(bridge_mqtt): fix elvis

* fix(worker): max_inflight_batches -> max_inflight
2020-12-28 13:43:13 +08:00
Zaiming Shi
d60212787b chore(build): remove emqx from app's applications list 2020-12-12 23:02:26 +01:00
Zaiming Shi
7bdd093ef5 Merge remote-tracking branch 'origin/dev/v4.3.0' into umbrella-fix-build 2020-12-11 08:55:57 +01:00
Zaiming Shi
3a91ff8a7b chore(proj): resync apps 2020-12-11 08:19:26 +01:00
Ayodele
56d7b3deb1 Umbrella fix build (#8) 2020-12-10 11:25:25 +01:00