* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
maintainability
* Added callback to format return value from connector
* Extended test case to check that the format return value callback
works
* Added handling of "lazy" trace entry data to the text formatter. Do we
need to handle this data in the normal log formatters as well?
* The code for passing the trace context to a sub process has been
improved to increase code reuse. This code is used when the action
templates are rendered in a sub process.
* A macro has also been added for the error term that is thrown when the
action shall be stopped after the templates has been rendered. This is
also done to reduce code duplication and to reduce the risk of
introducing bugs due to typos.
* Fix incorrect type spec
Thanks to @zmstone for suggesting these improvements in comments to a PR
(https://github.com/emqx/emqx/pull/12916).
This commit adds the behavior `emqx_connector_info`. The
`emqx_connector_info` behavior should be implement when creating a new
connector to provide information about the connector (such as connector
schema etc) to the `emqx_connetor` application.
The connector in the `emqx_bridge_dynamo` application has also been
refactored to use the new behavior (as a test to see that the behavior
is working as intended).
Fixes:
https://emqx.atlassian.net/browse/EMQX-11427
The DynamoDB connector status checks takes very long when the server is
unavailable which makes the resource manager blocked for a long time.
This causes calls to update the Bridge config with the Bridge V1 API fail
due to a timeout when it calls the resource manager to remove the
channel.
A better fix would be to change the resource manager so that the status
check cannot block it for a long time. However, this is more complicated
so it needs to be done in a later commit. A new ticket has been created
for this task https://emqx.atlassian.net/browse/EMQX-12015 .
Fixes:
https://emqx.atlassian.net/browse/EMQX-11984
This commit cleans up overly complicated code and handles
the case when the worker pool is empty.
Thank you @thalesmg for suggesting this change.
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
We need to reverse the dependency of `emqx_bridge` and `emqx_bridge_*`, because the former
loads and starts bridges during its application startup. If the individual bridge
application being loaded has not started with its dependencies, the supervision tree will
not be ready for that.
This commit refactor the query_mode resource detection code according to
a suggestion from @zmstone. This commit should not contain any
functional change except for a change of the Kafka producer bridge
config.
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
Current problem:
In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.
- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.
- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.
The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers. Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state. For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).
Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.
Proposed solution:
We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks. With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.
Also, as two smaller changes to accompany this one:
- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
