emqx/lib-ee/emqx_ee_bridge
Thales Macedo Garitezi 8aa7c014e7 perf(buffer_worker): avoid calling `ets:info/2`
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637

During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.

Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full.  Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60).

By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
..
priv/dynamo feat(bridges): introduce DynamoDB definition files from v4 2023-03-08 11:13:51 +08:00
src test(bridge): fix bridge map type filed converters 2023-04-30 10:45:11 +02:00
test perf(buffer_worker): avoid calling `ets:info/2` 2023-05-02 17:05:32 -03:00
.gitignore feat: add emqx enterprise app connector & bridge 2022-07-25 09:36:45 +08:00
README.md feat: add emqx enterprise app connector & bridge 2022-07-25 09:36:45 +08:00
docker-ct test: MS SQL Server data bridge 2023-04-14 10:02:46 +08:00
rebar.config refactor: move emqx_json to emqx_utils_json 2023-04-14 13:31:27 +02:00

README.md

emqx_ee_bridge

An OTP application

Build

$ rebar3 compile