Commit Graph

60 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 425eba8b13 fix(mqtt_bridge): ensure `server` key is a binary
Fixes https://emqx.atlassian.net/browse/EMQX-10461

So that it can be JSON encoded correctly.

```
2023-06-30T02:00:41.160110+00:00 [debug] msg: publish_to, mfa: emqx_trace:publish/1, line: 73, topic: b/$, payload: {"topic":"t/1","server":[49,48,46,52,50,46,51,46,49,56,48,58,49,56,56,51],"retain":false,"qos":1,"pub_props":{},"payload":"{\"msg\": \"hello\"}","message_received_at":1688090441159,"id":"0005FF4F2F181488103417000C2E0000","dup":false}, tag: PUBLISH
```
2023-06-30 13:46:24 -03:00
Andrew Mayorov d6c1ee183f
refactor(pluglib): move `emqx_placeholder` to utils app
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
JimMoen ea81a924f1
refactor: move topic() types def in `emqx_types.erl` 2023-06-07 22:59:52 +08:00
Andrew Mayorov f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config 2023-06-05 23:32:13 +03:00
Thales Macedo Garitezi 10425eb925 feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
firest 6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Andrew Mayorov 26819a647c
fix(mqttbridge): clarify schema descriptions + log messages
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-05-30 22:21:22 +03:00
Andrew Mayorov 7e7b50c5ba
refactor(mqttbridge): move into separate application 2023-05-30 17:56:34 +03:00
Shawn 037b75a276 refactor(bridges): remote the old emqx_bridge_mqtt app 2021-09-10 11:43:03 +08:00
Shawn 298cf24f00 fix(bridges): mqtt bridge cannot forward msgs using payload template 2021-09-10 11:43:03 +08:00
Shawn eb8822ce41 feat(bridges): start one mqtt bridge worker for each in/out channel 2021-09-10 11:43:03 +08:00
Shawn 4f82debbe7 feat(bridge): new configs for mqtt bridge 2021-09-10 11:43:03 +08:00
Shawn 39bb1b8d9d fix(bridges): start mqtt bridge failed 2021-09-10 11:43:03 +08:00
Shawn 9ed90ba7a9 feat(bridges): add connector for mqtt bridge 2021-09-10 11:43:03 +08:00
Zaiming Shi ec13463f4a refactor(schema): prepare for hocon schema doc generation 2021-09-03 11:02:31 +02:00
Zaiming Shi 5165fd6b30 refactor(schema): implement new hocon_schema callbacks 2021-08-30 09:56:03 +08:00
Turtle e4f5e9332e feat: support array bridge_mqtt conf 2021-08-25 13:34:00 +08:00
Shawn e8e95d39ef
refactor(config): move emqx_config:get/get_raw to emqx:get_config/get_raw_config (#5517) 2021-08-18 14:52:57 +08:00
Zaiming Shi e6232665a3 refactor(emqx): ensure app shutdown and reboot order
make use of application dependency to ensure start order.
also the same dependency top-sorted for shutdown and reboot order
2021-08-04 08:40:03 +02:00
Zaiming Shi 5d59ac1f02 refactor(logging): delete log headers from all modules 2021-08-01 09:36:59 +02:00
Turtle 9e97de04c9 chore(plugins): delete emqx_plugins module attributes 2021-07-30 14:32:01 +08:00
Turtle 918a26e921 feat(conf): merge all conf to emqx.conf 2021-07-02 17:01:16 +08:00
turtleDeng d4f726419f
feat(bridge-mqtt): Update the configuration file to hocon (#5142) 2021-07-01 10:51:59 +08:00
Turtle faad90c9d4 chore(plugins): rm emqx-web-hook and mv webhook action to emqx_rule_actions 2021-06-28 17:07:46 +08:00
Turtle c02a9f0101 fix(appup): update appup 2021-05-27 13:55:07 +08:00
Zaiming Shi 6354e75626 chore: update copyrights 2021-04-23 20:43:00 +02:00
Zaiming Shi d66f67d411 fix(emqx_bridge_mqtt): fix retry_inflight
The Inflight list should not be used to update State.inflight
2021-04-10 14:16:08 +02:00
Zaiming Shi ae9278a409 fix(bridge_mqtt): push to max-inflight limit
the old pop_and_send implementation may leave inflight queue starving
2021-04-09 10:18:25 +02:00
Zaiming Shi ae688e2c90 fix(bridge_mqtt): fix inflight reference booking
Prior to this change, the inflight batches are referenced
by the last packet ID for non-QoS-0 messages, other packet
IDs sent back from downstream causes an error log:
"Can't be found from the inflight"

Even worse, the batch is appended back to the queue for retry.
2021-04-09 10:18:25 +02:00
Zaiming Shi 71559dd001 refactor(ssl): Move data dir for ssl certs upload down to lib module 2021-03-22 16:41:01 +01:00
Zaiming Shi 8a41a4ee00 fix(ssl): Fix dir name for uploaded certificates
The dir is "rules" not "rule" in the original code
ref: f1eba1c648/src/emqx_rule_actions_utils.erl
2021-03-22 16:41:01 +01:00
z8674558 119bbc4881 chore(emqx_bridge_msg): add empty props 2021-03-03 20:06:31 +09:00
JianBo He 2fb5dbd546 revert(bridge-mqtt): remove the mqtt_sub resource
We have several reasons to remove this feature:

1. The design does not make sense. A rule engine resource should
not have an impact on the system's messages directly after it is created.
This mqtt_sub actually conflicts with any design concept of the rules engine.

2. The implementation is incorrect. mqtt_sub uses a client pool to establish
a subscription relationship to an MQTT Broker. This causes a message to be
sent repeatedly to EMQ X. Unless a shared subscription is used,
or a Pool Size of 1 is configured.

3. The emqx-bridge-mqtt supports all the features of mqtt_sub.

This feature introduced by https://github.com/emqx/emqx-bridge-mqtt/pull/78.
And it released to v4.2.0 (NOT WORK), v4.2.1-v4.2.7 (FIXED)
2021-03-02 18:56:21 +08:00
z8674558 2e199126dc fix(emqx_bridge_mqtt_actions): string -> file 2021-03-02 15:00:42 +09:00
z8674558 829a39eade fix(emqx_bridge_mqtt_actions): string -> password 2021-03-02 15:00:42 +09:00
Shawn 990b7bd494 fix(bridge_mqtt): typos 2021-03-01 21:00:17 +08:00
Zaiming Shi 675603fd73 fix(bridge-mqtt): no ssl-option at all (empty list) when ssl is disabled 2021-03-01 21:00:17 +08:00
Zaiming Shi 700fa71754 refactor(tls): abstract lib for tls options parsing 2021-03-01 21:00:17 +08:00
Zaiming Shi da6f1104dc chore(bridge-mqtt): sync enterprise code 2021-03-01 21:00:17 +08:00
Zaiming Shi f000b6583c fix(tls): Ensure tls config integrity
For default tsl version and ciphers, we try to use otp release number
to determin if we want to use tlsv1.3

For default configs, we try to porivde both tlsv1.3 and
ciphers in config (even for commented out configs)
2021-02-09 22:56:28 +01:00
JianBo He 456e3e5faa chore(bridge-mqtt): fix styles 2021-01-28 16:24:42 +08:00
JianBo He d98d26942c fix(bridge-mqtt): add max_inflight_size option back
For compatibility with versions below 4.2, we still
use the `max_inflight_size` option, but generate it
as the `max_inflight`

Previous PR: https://github.com/emqx/emqx/pull/3938
2021-01-28 16:24:42 +08:00
JianBo He 67e56658a7 fix(bridge-mqtt): fix bad time unit 2021-01-25 15:35:04 +08:00
JianBo He 3de8fea435 fix(bridge-mqtt): fix bad received metrics counting 2021-01-25 15:35:04 +08:00
Yudai Kiyofuji 56920b6ca6
fix(bridge_mqtt): max_inflight_size -> max_inflight; this enables emqtt.erl to use the value (#3938)
* fix(bridge_mqtt): max_inflight_size -> max_inflight; this enables emqtt.erl to use the value

* chore(bridge_mqtt): fix elvis

* fix(worker): max_inflight_batches -> max_inflight
2020-12-28 13:43:13 +08:00
Zaiming Shi d60212787b chore(build): remove emqx from app's applications list 2020-12-12 23:02:26 +01:00
Zaiming Shi 7bdd093ef5 Merge remote-tracking branch 'origin/dev/v4.3.0' into umbrella-fix-build 2020-12-11 08:55:57 +01:00
Zaiming Shi 3a91ff8a7b chore(proj): resync apps 2020-12-11 08:19:26 +01:00
Ayodele 56d7b3deb1 Umbrella fix build (#8) 2020-12-10 11:25:25 +01:00
Ayodele 0cb9cbce71 Umbrella fix build (#6)
* fix(emqx_bridge_rpc): Dialyzer warnings

* fix(emqx_coap_app): Dialyzer warnings

* fix(emqx_coap_mqtt_adapter): Dialyzer warnings

* fix(emqx_lwm2m_json): Dialyzer warnings

* fix(emqx_lwm2m_message): Dialyzer warnings

* fix(emqx_mgmt): Dialyzer warnings

* fix(emqx_mgmt_cli): Dialyzer warnings

* fix(emqx_mgmt): Dialyzer warnings

* fix(emqx_bridge_rpc): Dialyzer warnings

* fix(emqx_mgmt): Dialyzer warnings

* fix(emqx_exproto_gcli): Dialyzer warnings
2020-12-10 11:22:38 +01:00