emqx/changes/ce
Thales Macedo Garitezi 8aa7c014e7 perf(buffer_worker): avoid calling `ets:info/2`
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637

During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.

Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full.  Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60).

By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
..
.gitkeep chore: refactor the format-changelog script 2023-02-16 15:25:31 +08:00
feat-9986.en.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
feat-10022.zh.md docs: fix both original and merged change logs 2023-03-24 21:39:04 +01:00
feat-10077.en.md docs: add changelogs feat-10077 2023-03-27 11:54:03 +02:00
feat-10077.zh.md docs: add changelogs feat-10077 2023-03-27 11:54:03 +02:00
feat-10128.en.md docs(changelog): fix en changelog 2023-03-20 11:30:29 -03:00
feat-10139.en.md docs: fix both original and merged change logs 2023-03-24 21:39:04 +01:00
feat-10139.zh.md docs: fix both original and merged change logs 2023-03-24 21:39:04 +01:00
feat-10156.en.md chore: update changes/ce/feat-10156.en.md 2023-04-16 09:58:22 +08:00
feat-10164.en.md feat(crl): add crl check support 2023-03-20 18:09:38 -03:00
feat-10206.en.md feat(buffer_worker): decouple query mode from underlying connector call mode 2023-03-23 13:40:31 -03:00
feat-10207.en.md chore: add changelog 2023-03-22 16:32:34 +01:00
feat-10210.en.md chore: bump ekka to 0.14.6 2023-03-23 11:59:43 +02:00
feat-10224.en.md docs: improve formatting 2023-03-27 11:02:59 -03:00
feat-10263.en.md feat: add new command for Elixir expression evaluation 2023-03-28 18:45:48 -03:00
feat-10278.en.md chore: update changes 2023-04-03 09:47:05 +08:00
feat-10278.zh.md chore: update changes 2023-04-03 09:47:05 +08:00
feat-10306.en.md feat(bridges): enable async query mode for all bridges with buffer workers 2023-04-03 14:49:51 -03:00
feat-10318.en.md feat(rule engine sql): enable both ' and " strings in FROM clause 2023-04-04 10:36:48 +02:00
feat-10318.zh.md feat(rule engine sql): enable both ' and " strings in FROM clause 2023-04-04 10:36:48 +02:00
feat-10336.en.md feat: add `/rule_engine` API endpoint 2023-04-05 15:19:31 +02:00
feat-10354.en.md chore: add changelog for max_heap_size 2023-04-13 20:35:22 +08:00
feat-10358.en.md fix: rename emqx_misc to emqx_utils 2023-04-16 10:27:23 +08:00
feat-10359.en.md feat(resman): also move out metrics collection for debugging 2023-04-12 16:14:42 +03:00
feat-10373.en.md chore: add changelog for create trace 2023-04-12 17:59:14 +08:00
feat-10381.en.md chore: update changes/ce/feat-10381.en.md 2023-04-13 11:02:10 +08:00
feat-10385.en.md chore: add changelog for hide data items 2023-04-13 19:45:49 +08:00
feat-10391.en.md docs: sync changelogs 2023-04-18 19:17:03 +02:00
feat-10404.en.md feat(buffer_worker): set default queue mode to `memory_only` 2023-04-14 11:37:19 -03:00
fix-10014.en.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10014.zh.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10026.en.md fix(emqx_bridge): remove metrics from non-dedicated bridge API endpoints 2023-03-20 09:43:11 +01:00
fix-10026.zh.md fix(emqx_bridge): remove metrics from non-dedicated bridge API endpoints 2023-03-20 09:43:11 +01:00
fix-10055.en.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10055.zh.md docs: fix both original and merged change logs 2023-03-24 21:39:04 +01:00
fix-10098.zh.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10107.en.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10117.en.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10117.zh.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10124.zh.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10132.en.md docs: fix both original and merged change logs 2023-03-24 21:39:04 +01:00
fix-10132.zh.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10144.en.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10144.zh.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10145.en.md feat(emqx_resource): include error with alarm for resource_down 2023-03-21 15:02:29 +01:00
fix-10154.en.md fix(buffer_worker): check request timeout and health check interval 2023-04-04 08:58:36 -03:00
fix-10154.zh.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10157.en.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10157.zh.md chore: refine change for merged PRs 2023-03-24 11:17:34 +08:00
fix-10172.en.md chore: update changes/ce/fix-10172.en.md 2023-03-23 11:47:12 +08:00
fix-10172.zh.md chore: improve changes 2023-03-21 09:10:20 +08:00
fix-10174.en.md fix: upgrade esockd from 5.9.4 to 5.9.6 2023-03-20 19:27:32 +01:00
fix-10174.zh.md fix: upgrade esockd from 5.9.4 to 5.9.6 2023-03-20 19:27:32 +01:00
fix-10195.en.md fix: add summary to schemas where description has html 2023-03-23 16:40:43 +01:00
fix-10196.en.md chore: add changelog 2023-03-21 17:07:13 +01:00
fix-10209.en.md fix(last_will_testament): don't publish LWT if client is banned when kicked 2023-03-22 16:47:58 -03:00
fix-10211.en.md docs: add changelogs 2023-03-28 14:29:24 +02:00
fix-10211.zh.md docs: add changelogs 2023-03-28 14:29:24 +02:00
fix-10225.en.md fix(emqx_management): resolve plugin name clashes 2023-03-24 16:11:31 +02:00
fix-10226.en.md fix(emqx_bridge): don't crash on validation error 2023-03-24 11:46:41 +01:00
fix-10237.en.md fix: create consistent interface 'with_node' for API access 2023-04-04 16:54:14 +02:00
fix-10237.zh.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10242.en.md fix: avoid uisng 'pid' as log data field 2023-03-26 10:33:18 +02:00
fix-10242.zh.md fix: avoid uisng 'pid' as log data field 2023-03-26 10:33:18 +02:00
fix-10251.en.md fix(rule_engine): count referenced bridges in `from` clauses as dependencies (rv5.0) 2023-03-28 10:31:10 -03:00
fix-10251.zh.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10257.en.md chore: add changes 2023-03-28 19:05:51 +08:00
fix-10257.zh.md chore: add changes 2023-03-28 19:05:51 +08:00
fix-10286.en.md docs: Update changes/ce/fix-10286.en.md 2023-03-30 17:11:10 +02:00
fix-10286.zh.md docs: Add change logs 2023-03-30 14:17:29 +02:00
fix-10297.en.md fix: keep eval command backward compatible with v4 2023-03-30 14:20:49 -03:00
fix-10300.en.md fix(plugins): create directory before uploading 2023-03-30 17:51:16 -03:00
fix-10313.en.md fix: running nodes should not include replica nodes 2023-04-03 20:01:37 +08:00
fix-10313.zh.md fix: running nodes should not include replica nodes 2023-04-03 20:01:37 +08:00
fix-10314.en.md chore: add changelog 2023-04-04 19:58:43 +02:00
fix-10314.zh.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10315.en.md style: add changelog 2023-04-03 15:55:25 +02:00
fix-10317.en.md docs: add change logs 2023-04-03 16:52:49 +02:00
fix-10317.zh.md docs: add change logs 2023-04-03 16:52:49 +02:00
fix-10323.en.md chore: update changes 2023-04-04 16:31:54 +08:00
fix-10323.zh.md chore: update changes 2023-04-04 16:31:54 +08:00
fix-10327.en.md fix(rule_engine): don't increment unknown counter on unrecoverable errors 2023-04-04 15:59:53 +03:00
fix-10327.zh.md chore: update changelog entries 2023-04-11 11:42:07 +02:00
fix-10410.en.md chore: update changes/ce/fix-10410.en.md 2023-04-17 13:57:02 +02:00
fix-10449.en.md chore: update changes/ce/fix-10449.en.md 2023-04-21 15:30:27 +08:00
fix-10455.en.md fix(buffer_worker): avoid sending late reply messages to callers 2023-04-26 13:18:28 -03:00
fix-10548.en.md chore: bump ehttpc -> 0.4.8 2023-04-27 14:10:37 -03:00
perf-10123.en.md docs: fix both original and merged change logs 2023-03-24 21:39:04 +01:00
perf-10123.zh.md docs: fix both original and merged change logs 2023-03-24 21:39:04 +01:00
perf-10573.en.md perf(buffer_worker): avoid calling `ets:info/2` 2023-05-02 17:05:32 -03:00