This commit:
* refactors the MQTT V1 bridge into connector, source and action
* Extends the compatibility layer so it works for sources
* Fixes the MQTT bridge test suite so that all test cases passes
We still need to add a HTTP API handling sources. Also, we still need to
add HTTP API example schemes and examples for the MQTT
connector/action/source.
We should also make sure that we handle the corner cases of the MQTT V1
bridge automatic upgrade downgrade in a sufficiently good way:
* An error is currently thrown when converting an MQTT V1 bridge without
egress or ingress config.
* If there is a source and action with the same name we will currently
throw an error in the compatibility layer.
* We will also throw an error when converting an MQTT V1 bridge with
both ingress and egress.
The above is probably the right thing to do but we have to make sure
that we return a reasonable error to the user when this happens.
(partly)
Fixes:
https://emqx.atlassian.net/browse/EMQX-11489
Some mqtt brokers do not allow long client IDs.
To make it compatible with this limitation, this commit
tries to limit the number of bytes under 23 with a best-effort
attempt to derive it from the bridge name.
We need to reverse the dependency of `emqx_bridge` and `emqx_bridge_*`, because the former
loads and starts bridges during its application startup. If the individual bridge
application being loaded has not started with its dependencies, the supervision tree will
not be ready for that.
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
Current problem:
In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.
- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.
- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.
The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers. Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state. For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).
Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.
Proposed solution:
We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks. With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.
Also, as two smaller changes to accompany this one:
- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
Prior to this change, the inflight batches are referenced
by the last packet ID for non-QoS-0 messages, other packet
IDs sent back from downstream causes an error log:
"Can't be found from the inflight"
Even worse, the batch is appended back to the queue for retry.
We have several reasons to remove this feature:
1. The design does not make sense. A rule engine resource should
not have an impact on the system's messages directly after it is created.
This mqtt_sub actually conflicts with any design concept of the rules engine.
2. The implementation is incorrect. mqtt_sub uses a client pool to establish
a subscription relationship to an MQTT Broker. This causes a message to be
sent repeatedly to EMQ X. Unless a shared subscription is used,
or a Pool Size of 1 is configured.
3. The emqx-bridge-mqtt supports all the features of mqtt_sub.
This feature introduced by https://github.com/emqx/emqx-bridge-mqtt/pull/78.
And it released to v4.2.0 (NOT WORK), v4.2.1-v4.2.7 (FIXED)
