emqx/apps/emqx_bridge_pulsar
Thales Macedo Garitezi 10425eb925 feat(resource): deprecate auto_restart_interval in favor of health_check_interval
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
..
etc feat: implement Pulsar Producer bridge (e5.0) 2023-04-24 10:28:26 -03:00
include feat: implement Pulsar Producer bridge (e5.0) 2023-04-24 10:28:26 -03:00
src feat(resource): deprecate auto_restart_interval in favor of health_check_interval 2023-06-01 11:20:06 -03:00
test test(pulsar_producer): attempt to stabilize flaky test 2023-05-31 10:19:55 -03:00
.gitignore feat: implement Pulsar Producer bridge (e5.0) 2023-04-24 10:28:26 -03:00
BSL.txt feat: implement Pulsar Producer bridge (e5.0) 2023-04-24 10:28:26 -03:00
docker-ct feat: implement Pulsar Producer bridge (e5.0) 2023-04-24 10:28:26 -03:00
README.md docs: hide the not-ready document links 2023-05-15 11:20:23 +08:00
rebar.config feat(pulsar): improve authn error check time and add connect timeout 2023-05-22 11:33:16 -03:00

Pulsar Data Integration Bridge

This application houses the Pulsar Producer data integration bridge for EMQX Enterprise Edition. It provides the means to connect to Pulsar and publish messages to it.

Currently, our Pulsar Producer library has its own replayq buffering implementation, so this bridge does not require buffer workers from emqx_resource. It implements the connection management and interaction without need for a separate connector app, since it's not used by authentication and authorization applications.

Documentation links

For more information on Apache Pulsar, please see its official site.

Contributing

Please see our contributing.md.

License

EMQ Business Source License 1.1, refer to LICENSE.