emqx/lib-ee/emqx_ee_bridge/test
Thales Macedo Garitezi 8aa7c014e7 perf(buffer_worker): avoid calling `ets:info/2`
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637

During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.

Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full.  Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60).

By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
..
ee_bridge_hstreamdb_SUITE.erl chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
emqx_ee_bridge_cassa_SUITE.erl perf(buffer_worker): avoid calling `ets:info/2` 2023-05-02 17:05:32 -03:00
emqx_ee_bridge_clickhouse_SUITE.erl feat: add clickhouse database bridge 2023-03-02 12:22:11 +01:00
emqx_ee_bridge_dynamo_SUITE.erl fix(dynamo): fix test case errors 2023-04-26 17:03:01 +08:00
emqx_ee_bridge_gcp_pubsub_SUITE.erl refactor: rename emqx_map_lib to emqx_utils_maps 2023-04-14 13:41:34 +02:00
emqx_ee_bridge_influxdb_SUITE.erl refactor: rename emqx_map_lib to emqx_utils_maps 2023-04-14 13:41:34 +02:00
emqx_ee_bridge_influxdb_tests.erl feat: handle escaped characters in InfluxDB data bridge write_syntax 2023-03-20 16:42:23 +02:00
emqx_ee_bridge_mongodb_SUITE.erl refactor: rename emqx_map_lib to emqx_utils_maps 2023-04-14 13:41:34 +02:00
emqx_ee_bridge_mysql_SUITE.erl refactor: rename emqx_misc to emqx_utils 2023-04-14 13:41:27 +02:00
emqx_ee_bridge_pgsql_SUITE.erl refactor: rename emqx_map_lib to emqx_utils_maps 2023-04-14 13:41:34 +02:00
emqx_ee_bridge_redis_SUITE.erl feat(bridges): enable async query mode for all bridges with buffer workers 2023-04-03 14:49:51 -03:00
emqx_ee_bridge_rocketmq_SUITE.erl fix(rocketmq): fix test cases 2023-04-21 11:02:14 +08:00
emqx_ee_bridge_sqlserver_SUITE.erl fix: stale ref to emqx_map_lib 2023-04-14 18:43:53 +02:00
emqx_ee_bridge_tdengine_SUITE.erl refactor: rename emqx_map_lib to emqx_utils_maps 2023-04-14 13:41:34 +02:00