yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,253 Commits 393 Branches 684 Tags 74 MiB
1c4c7fad92
Commit Graph

15253 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Thales Macedo Garitezi
1c4c7fad92
Merge pull request #10740 from thalesmg/perf-buffer-worker-periodic-metrics-v50 
perf(buffer_worker): update metrics periodically rather than immediately (v5.0)
 2023-05-22 11:41:41 -03:00
Andrew Mayorov
3f882ddcc6
Merge pull request #10751 from ci/ct-flap-listener-api 
fix(listen): wait until port is free when stopping ranch listeners
 2023-05-22 17:00:45 +03:00
Thales Macedo Garitezi
0559d6f639 refactor(buffer_worker): use static fn for bumping counters 2023-05-22 09:12:08 -03:00
Thales Macedo Garitezi
9c71f4ecbd test: fix flaky test 2023-05-22 09:12:00 -03:00
Thales Macedo Garitezi
c74c93388e refactor: rename some variables and sum type constructors for clarity 2023-05-22 09:11:23 -03:00
Thales Macedo Garitezi
7d798c10e9 perf(buffer_worker): flush metrics periodically inside buffer worker process 
Fixes https://emqx.atlassian.net/browse/EMQX-9905

Since calling `telemetry` is costly in a hot path, we instead collect
metrics inside the buffer workers state and periodically flush them,
rather than immediately as events happen.
 2023-05-22 09:11:23 -03:00
Andrew Mayorov
23542d1262
fix(listen): wait until port is free when stopping ranch listeners 
Simple `cowboy:stop_listener/1` will not close the listening socket
explicitly, it was the source of test flaps before this fix.
 2023-05-22 12:29:40 +03:00
Andrew Mayorov
5b312a415b
Merge pull request #10759 from ci/ct-flap-clickhouse 
fix(clickhouse): start app in tests
 2023-05-22 11:11:15 +03:00
William Yang
e824e1db39
Merge pull request #10698 from qzhuyan/perf/william/force-atom-conf-path 
perf(config): enforcing atom key  path in hotcode path
 2023-05-22 09:38:09 +02:00
zhongwencool
c65e3a46b1
Merge pull request #10742 from zhongwencool/fix-authz-file-rule-check 
fix: check authz's file source rules in pre_config_update
 2023-05-22 14:15:18 +08:00
Zhongwen Deng
96e7005de8 refactor: add emqx_authz_file validate function 2023-05-22 11:03:23 +08:00
Andrew Mayorov
ba6b208df2
fix(clickhouse): start app in tests 
Otherwise, depending on the test execution order, tests might
sometimes fail.

Moreover, ensure that applications describe their dependecies
correctly and avoid starting irrelevant apps in tests.
 2023-05-19 23:08:40 +03:00
Andrew Mayorov
c440cd77b0
fix(test): prefer emqx_connector_http:join_paths/2 in api tests 
It is more consistent with what HTTP servers are expecting, plus
not OS-specific as `filename:join/2`.
 2023-05-19 15:37:01 +03:00
Andrew Mayorov
e4701a71da
Merge pull request #10748 from ci/ct-flap-eviction 
test(evict): fix flapping testcase
 2023-05-19 15:31:28 +03:00
某文
6cb9efd7d3 feat: add authz file rule validator 2023-05-19 18:41:59 +08:00
Andrew Mayorov
5c112602b8
test(evict): fix flapping testcase 
Postpone enabling eviction agent so that target connection will not
get evicted too early.
 2023-05-19 13:37:36 +03:00
JianBo He
48ceb14755
Merge pull request #10737 from HJianBo/fix-mqtt-sn-bugs 
Avoid urldecode clientid twice
 2023-05-19 09:19:12 +08:00
某文
52e2caa671 fix: check authz's file rule before save to file 2023-05-18 22:33:12 +08:00
JianBo He
f42cab9c56 chore: update changes 2023-05-18 16:24:37 +08:00
JianBo He
26b69a5591 test(mqttsn): add test cases 2023-05-18 16:16:09 +08:00
JianBo He
c5398ab651 fix(gateway): avoid uri_decode twice 
The url path has beed decoded in
0ebceb432f/src/cowboy_router.erl (L324-L330).
So, we don't need uri_decode in minirest callback again.
 2023-05-18 14:55:12 +08:00
JianBo He
d77d5e33bc fix(mqttsn): Instantly refresh client info after subscribed/unsubscribed 2023-05-18 14:48:37 +08:00
zhongwencool
ba43a0e30f
Merge pull request #10734 from thalesmg/merge-r50-into-v50-a 
merge `release-50` into `master`
 2023-05-18 14:24:53 +08:00
zhongwencool
bf5ee41009
Merge pull request #10702 from zhongwencool/keepalive-backoff-rename 
feat: deprecated keepalive_backoff, introduce keepalive_multiplier
 2023-05-18 10:00:17 +08:00
某文
d4d25d2660 chore: don't lost previous's statval 2023-05-18 09:49:11 +08:00
Thales Macedo Garitezi
f83f9a2d79 chore: bump app vsns 2023-05-17 14:56:11 -03:00
Thales Macedo Garitezi
447b76464b Merge branch 'release-50' into merge-r50-into-v50-a 2023-05-17 14:50:18 -03:00
Thales Macedo Garitezi
10f6edd6ea
Merge pull request #10717 from thalesmg/fix-bw-flush-timer-full-infl-r50 
fix(buffer_worker): avoid setting flush timer when inflight is full
 2023-05-17 14:49:27 -03:00
William Yang
727ad59995 chore(test): improve coverage 2023-05-17 16:58:49 +02:00
Thales Macedo Garitezi
b2afe4e90c
Merge pull request #10690 from thalesmg/perf-webhook-retry-async-reply-v50 
perf(webhook): add async retries and evaluate reply callback in fresh process
 2023-05-17 11:06:48 -03:00
Zhongwen Deng
e26ce5816e feat: hide keepalive_backoff, introduce keepalive_multiplier 2023-05-17 21:25:10 +08:00
zhongwencool
ea8ac877b0
Merge pull request #10713 from zhongwencool/put-webhook-request-timeout-into-resource-opts 
feat: update wehbook's request_timeout into resource_opts
 2023-05-17 21:20:42 +08:00
William Yang
ea4fbb8b6e chore: add changelog 2023-05-17 14:46:07 +02:00
Thales Macedo Garitezi
bc7d0d5fd6 perf(webhook): evaluate replies in fresh process rather than in ehttpc 
This surprisingly simple change yields a big performance improvement
in throughput.

While the previous commit achieves ~ 55 k messages / s
in throughput under some test conditions (100 k concurrent publishers
publishing 1 QoS 1 message per second), the simple change in this
commit improves it further to ~ 63 k messages / s.

Benchmarks indicated that the evaluating one reply function is
consistently quite fast (~ 20 µs), which makes this performance gain
counterintuitive.  Perhaps, although each call is cheap, `ehttpc`
calls several of these in a row when there are several sent requests,
and those costs might add up in latency.
 2023-05-17 09:20:50 -03:00
Thales Macedo Garitezi
a7b41e1cdf perf(webhook): add retry attempts for async 
This is a performance improvement for webhook bridge.

Since this bridge is called using `async` callback mode, and `ehttpc`
frequently returns errors of the form `normal` and `{shutdown,
normal}` that are retried "for free" by `ehttpc`, we add this behavior
to async requests as well.  Other errors are retried too, but they are
not "free": 3 attempts are made at a maximum.

This is important because, when using buffer workers, we should avoid
making them enter the `blocked` state, since that halts all progress
and makes throughput plummet.
 2023-05-17 09:20:50 -03:00
Thales Macedo Garitezi
01cf7fc703 feat(webhook): use clientid for hash-loadbalancing ehttpc workers 2023-05-17 09:20:50 -03:00
Kjell Winblad
397dce0d78
Merge pull request #10654 from kjellwinblad/kjell/refactor/clickhouse_bridge/EMQX-9538 
refactor: the clickhouse bridge to use the new bridge app structure
 2023-05-17 11:19:33 +02:00
William Yang
cdf42760fa chore: little change avoid atom leak 2023-05-17 11:01:03 +02:00
Kjell Winblad
bce43978a2 refactor: the clickhouse bridge to use the new bridge app structure 
Fixes:
https://emqx.atlassian.net/browse/EMQX-9538
 2023-05-17 10:12:43 +02:00
Andrew Mayorov
542739de09
Merge pull request #10722 from fix/ci-plt-cache-update 
ci: ensure dialyzer plt cache is updated on changes
 2023-05-17 10:47:08 +03:00
Ilya Averyanov
c54d04450c
Merge pull request #10723 from savonarola/0516-improve-rebalance-logging 
Improve rebalance logging
 2023-05-17 09:21:10 +03:00
某文
2b99a9b988 feat: hide resource_opts's request_timeout 2023-05-17 13:41:11 +08:00
lafirest
b5da9eb9ad
Merge pull request #10728 from lafirest/fix/port_ruleengine_export_colums 
fix: cannot access columns exported by FOREACH in DO clause
 2023-05-17 11:46:17 +08:00
firest
1f7ede90a4 chore: update app version && changes 2023-05-17 11:01:31 +08:00
firest
18043150be fix: cannot access columns exported by FOREACH in DO clause 2023-05-17 10:55:47 +08:00
JianBo He
0dd3325d70
Merge pull request #10711 from lafirest/fix/fix_infinity_bucket 
fix(limiter): a bucket with an infinity rate shouldn't be added to limiter server
 2023-05-17 10:40:59 +08:00
JianBo He
5aabd03bd0
Merge pull request #10718 from zhongwencool/log-changelog 
chore: update 10607's log config changelog
 2023-05-17 08:54:48 +08:00
JianBo He
2a1814691e
Merge pull request #10721 from zhongwencool/10340-changlog 
chore: update 10340's changelog
 2023-05-17 08:54:11 +08:00
Zaiming (Stone) Shi
9b9731b943
Merge pull request #10473 from zmstone/0421-refactor-delete-unused-schema-field 
0421 refactor delete unused schema field
 2023-05-16 22:22:03 +02:00
Thales Macedo Garitezi
85089a3210 fix(buffer_worker): correctly flush the buffer workers when inflight table room is made 
The previous commit uncovered another bug that was hidden by it:
`maybe_flush_after_async_reply` was sending a message to the wrong
PID.  It was sending a message to `self()` meaning to target a buffer
worker, but `self()` in that context is never the buffer worker, it's
the connector's worker.

This change also revealed a race condition where the buffer workers
could stop flushing messages.  So we piggy-backed on the atomic update
of the table size count to check if the buffer worker should be poked
to continue flushing.  This allows us to get rid of
`maybe_flush_after_async_reply` altogether.
 2023-05-16 17:15:42 -03:00