Commit Graph

12974 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 731ac6567a fix(buffer_worker): don't retry all kinds of inflight requests
Some requests should not be retried during the blocked state.  For
example, if some async requests are just taking some time to process,
we should avoid retrying them periodically, lest risk overloading the
downstream further.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 5425f3d88e refactor: rm unused fn 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 5dd24a64c3 refactor(buffer_worker): check if inflight is full before flushing 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 344eeebe63 fix: always ack async replies
The caller should decide if it should retry in that case, to avoid
overwhelming the resource with retries.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi bd95a95409 refactor: remove redundant `BlockWorker` arg, change boolean to ack/nack
`BlockWorker` was always false (ack).  Also, changed the return to
something more semantic than a boolean to avoid [boolean
blindness](https://runtimeverification.com/blog/code-smell-boolean-blindness/)
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 478fcc6ffd test: fix flaky test 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 30a227bd38 refactor: rename `resume` state timeout to `unblock` 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 7401d6f0ce refactor: rename ack fn 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 196bf1c5ba feat: mass collect calls from mailbox also when blocked 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi d4724d6ce9 refactor: remove redundant function
`retry_queue` does basically what the running state does, now that we
refactored the buffer workers to always use the queue.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi d6a9d0aa48 fix: set queuing to 0 after buffer worker termination 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 81fc561ed5 fix(buffer_worker): check for overflow after enqueuing new requests 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 4cb83d0c9a fix: fix some expressions after refactoring 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi fecdbac9a8 refactor: rename a few functions 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi cdd8de11b0 chore: fix a typo in function name 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi 618b97870b refactor: call local function queue_count everywhere 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi 249c4c1c79 refactor: use 'bufs' for resource worker replayq dir 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi af6807e863 refactor: cancel flush timer sooner
Avoids the cancellation being delayed.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 477c55d8ef fix: sanitizy replayq dir filepath
Colons (`:`) are not allowed in Windows.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 4c04a01370 refactor(buffer_worker): remove `?Q_ITEM` wrapping and use lightweight size estimate 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 32a9e60313 feat(buffer_worker): also use the inflight table for sync requests
Related: https://emqx.atlassian.net/browse/EMQX-8692

This should also correctly account for `retried.*` metrics for sync
requests.

Also fixes cases where race conditions for retrying async requests
could potentially lead to inconsistent metrics.

Fixes more cases where a stale reference to `replayq` was being held
accidentally after a `pop`.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi ff23d25e8b chore(replayq): update replayq -> 0.3.6 and use `clean_start` for buffer workers
So we can truly avoid resuming work after a node restart.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi c383558467 fix(buffer): fix `replayq` usages in buffer workers (5.0)
https://emqx.atlassian.net/browse/EMQX-8700

Fixes a few errors in the usage of `replayq` queues.

- Close `replayq` when `emqx_resource_worker` terminates.
- Do not keep old references to `replayq` after any `pop`s.
- Clear `replayq`'s data directories when removing a resource.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 4c25be8a2c chore(influxdb): remove deprecated value from example 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi 394e8ab6a4
Merge pull request #9793 from zmstone/0117-use-simple-sync-query-for-authn-authz
0117 use simple sync query for authn authz
2023-01-17 20:47:22 +01:00
Zaiming (Stone) Shi fff6bf921f refactor(authz): call emqx_resource:simple_sync_query
there is no need to route the request through the buffer workers
2023-01-17 20:01:45 +01:00
Zaiming (Stone) Shi 7a471faf67 refactor(authn): use simple sync query for authn
there is no need to route the authn queries through the buffer worker
2023-01-17 19:57:14 +01:00
zhongwencool 9f80eb2497
Merge pull request #9777 from zmstone/0116-refactor-config_files-is-readonly
refactor: config_files is a readonly config
2023-01-17 20:32:39 +08:00
Zaiming (Stone) Shi 8b3ec48406 refactor: node.config_files should be hidden as it's not configurable
this config schema exists only for app-env mapping.
users can not change it because the bootstraping config
has to be etc/emqx.conf
2023-01-17 11:08:40 +01:00
Stefan Strigler 74ae7c4264
Merge pull request #9782 from sstrigler/EMQX-8361-fix-ct-suite-for-plugins-api
test: get plugin template as artifact
2023-01-17 11:08:07 +01:00
zhongwencool b1ad61a997
Merge pull request #9788 from terry-xiaoyu/fix-typos
fix: typos in emqx_ee_bridge_mongodb.conf
2023-01-17 17:10:49 +08:00
zhongwencool cb5314bb9e
Merge pull request #9774 from zhongwencool/dashboard-password-validate-v50
feat: add dashboard password validation
2023-01-17 17:02:51 +08:00
Stefan Strigler bc69a00329 test: get plugin template as artifact 2023-01-17 09:50:17 +01:00
Shawn 1c7fe91a91 fix: typos in emqx_ee_bridge_mongodb.conf 2023-01-17 16:33:54 +08:00
Xinyu Liu 8027b79031
Merge pull request #9771 from terry-xiaoyu/update-mysql-e5.0
chore: update mysql-otp to 1.7.2
2023-01-17 14:37:44 +08:00
Xinyu Liu 8dbd852ea1
Merge pull request #9767 from emqx/update_ehttpc-v5.0
chore: update ehttpc to 0.4.4
2023-01-17 14:37:33 +08:00
Shawn 44f635300d chore: update mysql-otp to 1.7.2 2023-01-17 13:36:26 +08:00
Zhongwen Deng 3dfdad9cc9 test: dashboard default password 2023-01-17 11:30:42 +08:00
Zhongwen Deng 2bfb37f7f0 chore: changelog for #9774 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi b9911e281e docs: Update apps/emqx_dashboard/src/emqx_dashboard_admin.erl 2023-01-17 11:30:42 +08:00
zhongwencool c549379b7c chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-17 11:30:42 +08:00
Zhongwen Deng 82af854c4a test: dashbashboard new password failed 2023-01-17 11:30:42 +08:00
Zhongwen Deng a523fa2fa2 feat: add dashboard password validate 2023-01-17 11:30:42 +08:00
JimMoen 050d245fa5
Merge pull request #9680 from JimMoen/fix-influxdb-api
fix(influxdb): authentication is not required for influxdb v1 api
2023-01-17 10:58:53 +08:00
JimMoen 40948f3c0b
fix(influxdb): authentication is not required for influxdb v1 api 2023-01-17 09:29:22 +08:00
Zaiming (Stone) Shi 33be7672ad test: recover boot_modules app env after test 2023-01-17 00:43:25 +01:00
Zaiming (Stone) Shi 86b4d4d23b docs: add missing change logs for #9750 2023-01-17 00:43:25 +01:00
Andrew Mayorov 5fd157fb1d
Merge pull request #9776 from keynslug/chore/bump-epgsql-otp-25
chore: update epgsql to 4.7.0.1 with full OTP-25 compat
2023-01-16 23:51:09 +04:00
Zaiming (Stone) Shi 999d40ef1d
Merge pull request #9769 from zmstone/0116-fix-version-prefix-in-restricted-shell-prompt
fix: version number previs in restricted shell
2023-01-16 20:22:37 +01:00
zhongwencool c73ffdf825
Merge pull request #9779 from zmstone/0116-docs-merge-change-logs
docs: merge released change logs (v5.0.14)
2023-01-16 23:39:44 +08:00