Commit Graph

6599 Commits

Author SHA1 Message Date
William Yang ceac5a0ec7 feat(quic): bump to quicer 0.0.101 2023-02-20 10:42:55 +01:00
William Yang 1840a7f923 test(quic): improve coverage 2023-02-20 10:42:55 +01:00
William Yang 5bdcb0562d feat(quic): workaround to flushing the send buffer after conn shutdown
Could not find a way to ensure msquic flush the send buffer after
calling ConnectionShutdown.

So just close the ctrl stream and let conn owner shutdown the conn.
2023-02-20 10:42:55 +01:00
William Yang 04a8a49dbe test: update testcase for new emqtt 2023-02-20 10:42:55 +01:00
William Yang 7d9bd33de9 feat(quic): bump quicer version to 0.0.100 2023-02-20 10:42:55 +01:00
William Yang 9b52beaee9 fix(quic): handle fast_close while handshake fail 2023-02-20 10:42:55 +01:00
William Yang 9f696928b6 feat(quic): multi streams 2023-02-20 10:42:55 +01:00
William Yang a51c886908 fix: prepare for multi stream 2023-02-20 10:42:55 +01:00
William Yang 2d09a054e3 chore: add some typing 2023-02-20 10:42:55 +01:00
William Yang 00b59b4939 feat(quic): WIP multi-stream 2023-02-20 10:42:55 +01:00
Zaiming (Stone) Shi 6f1eaca253 chore: bump version to e5.0.1-alpha.1 2023-02-20 08:28:14 +01:00
ieQu1 388c33395e
Merge pull request #9939 from ieQu1/mria-replicant-manual-join
Mria replicant manual join
2023-02-17 11:32:58 +01:00
ieQu1 d9554c36ac chore: Bump umbrealla app versions 2023-02-17 09:50:50 +01:00
ieQu1 794bedef9f fix(emqx_ctl): Start CLI before ekka 2023-02-17 09:50:50 +01:00
Erik Timan 056bc71af2 chore: bump VSN version 2023-02-16 15:05:38 +01:00
Erik Timan 2442a4dea7 test(emqx_resource): add regression test for recursive flushing 2023-02-16 14:17:16 +01:00
Erik Timan dcf70e0e68 refactor(emqx_resource): add more trace points for flushing 2023-02-16 14:17:16 +01:00
Thales Macedo Garitezi 6bf385793c
Merge pull request #9978 from thalesmg/fix-psql-tls-v50
fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0)
2023-02-16 09:20:32 -03:00
Zhongwen Deng 82b2f9f245 chore: include emqx_management in emqx_prometheus/emqx_statsd 2023-02-16 11:37:00 +08:00
Thales Macedo Garitezi dbda504f2c fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0)
Fixes https://github.com/emqx/emqx/issues/9907

At v5.0.14, we changed the `ssl` option for the Postgres connector
from `true` to `required`, but there was another transformation in
`conn_opts/2` that led to an incorrect configuration.  This change
ended up preventing users from connecting to Postgres with their
previous configurations after upgrading EMQX.
2023-02-15 14:06:33 -03:00
Zaiming (Stone) Shi 719e02cc96 fix(metrics): use the same data source for RAM usage info 2023-02-15 16:11:39 +01:00
Zaiming (Stone) Shi 1cf88ea0b4 docs: improve bridge_mode description 2023-02-15 15:24:10 +01:00
Stefan Strigler 293df88f36
Merge pull request #9965 from sstrigler/EMQX-3689-emqx-mgmt-api
EMQX 3689 emqx mgmt api
2023-02-15 15:13:17 +01:00
Stefan Strigler 921b5b2c5c style: rename function for compliance 2023-02-15 13:38:14 +01:00
Stefan Strigler 21b8ff6d14 test: use unit test to test paginate 2023-02-15 13:31:48 +01:00
lafirest 3108b6f9dc
Merge pull request #9932 from lafirest/feat/tdengine
feat(bridges): add TDengine
2023-02-15 20:01:50 +08:00
William Yang 02f15be20f
Merge pull request #9967 from qzhuyan/perf/william/add-tls-common-conf-hibernate-after
perf: add tls common conf hibernate after
2023-02-15 12:02:05 +01:00
Stefan Strigler 9be73bfff0 fix: support ipv6 at ip_port 2023-02-15 10:37:13 +01:00
William Yang 4e6c7bfcbe chore: changelog for hibernate_after 2023-02-14 16:26:58 +01:00
William Yang d424d77e84 chore(TLS): hibernate_after default to 5s, follow version 4.x 2023-02-14 16:26:58 +01:00
William Yang 572e50e0a2 perf: add TLS config hibernate_after, reduce memory footprint 2023-02-14 16:26:55 +01:00
firest 81adb02402 chore: bump apps version 2023-02-14 22:04:29 +08:00
firest 530b21268a feat(bridges): add TDengine 2023-02-14 22:04:23 +08:00
Stefan Strigler 4417ea9db7 style: missing comma 2023-02-14 09:13:45 +01:00
JianBo He 0d63cfdc97 chore: remove changes to 5.0.18 2023-02-14 15:34:12 +08:00
JianBo He 36b1c3eb5b fix(api): fix bad return error message if client id is not found 2023-02-14 15:30:55 +08:00
Zaiming (Stone) Shi ba65cf48c3 feat: add new module emqx_cover.erl 2023-02-13 22:31:18 +01:00
Stefan Strigler 8b47acd58d test: page overflow and bad rpc in cluster 2023-02-13 17:46:26 +01:00
Stefan Strigler 8d34a72ab6 test: add test for clustered badrpc 2023-02-13 17:14:15 +01:00
Stefan Strigler a28588ebf0 fix: bad spec 2023-02-13 17:14:02 +01:00
Stefan Strigler 89630a8a74 test: query bad node 2023-02-13 16:09:04 +01:00
Stefan Strigler 0c87d238e9 chore: bump version 2023-02-13 15:46:27 +01:00
Stefan Strigler 9e954585ae test: add node query tests and some more 2023-02-13 15:45:37 +01:00
Stefan Strigler 0e9289e48c refactor: remove unused clause 2023-02-13 15:45:37 +01:00
Stefan Strigler 743ced17c9 test: add test for more types in schema 2023-02-13 15:45:37 +01:00
Stefan Strigler 28560e4603 refactor: remove unused code 2023-02-13 15:45:37 +01:00
Zaiming (Stone) Shi 7d9b0f4826 chore: bump version to v5.0.17 2023-02-13 08:53:09 +01:00
Zaiming (Stone) Shi a11217aa0f chore: bump version to v5.0.17 2023-02-12 11:36:53 +01:00
Zaiming (Stone) Shi 72fe10f18b chore: fix spell check errors 2023-02-10 19:41:14 +01:00
Zaiming (Stone) Shi fb61c2b266 perf: avoid getting metrics (gen_server:call) for each resource lookup 2023-02-10 19:40:37 +01:00
Andrew Mayorov fe450ca2d9
Merge pull request #9938 from keynslug/feat/mqtt-bridge-async-errors
feat(mqtt-bridge): report recoverable errors of async queries
2023-02-10 20:40:58 +04:00
Andrew Mayorov 2ed54e560f
Merge pull request #9952 from keynslug/fix/mqtt-bridge-no-qos2-clean-start
fix(mqtt-bridge): disallow QoS 2 on ingress bridges
2023-02-10 20:40:37 +04:00
Andrew Mayorov cbb2885499
fix(mqtt-bridge): allow to configure `clean_start` for ingresses 2023-02-10 19:15:48 +03:00
Stefan Strigler be615a2bb1
Merge pull request #9951 from sstrigler/EMQX-8929-handle-error-results-in-bridge-api-for-multicall-operations
fix(emqx_bridge): unwrap rpc multicall result
2023-02-10 16:26:50 +01:00
Andrew Mayorov 7002fe2ef4
fix(mqtt-bridge): disallow QoS 2 on ingress bridges 2023-02-10 17:17:59 +03:00
Zaiming (Stone) Shi 5d59fcbf23
Merge pull request #9950 from zhongwencool/prometheus_push_server_url_check
fix: add push_gateway_server's  validator
2023-02-10 12:41:50 +01:00
Stefan Strigler e196d6f42a fix(emqx_bridge): unwrap rpc multicall result 2023-02-10 12:00:20 +01:00
Zhongwen Deng 53488b4377 fix: add push_gateway_server's validator 2023-02-10 17:52:16 +08:00
Andrew Mayorov 8bd7c0eb3f
feat(mqtt-bridge): report recoverable errors of async queries
This should help to avoid delivery failures of messages which could
be safely retried, in the event of intermittent connectivity loss
for example.

It should now be safe since 73d5592b.
2023-02-10 12:26:59 +03:00
Zaiming (Stone) Shi d1eb788ee1
Merge pull request #9916 from qzhuyan/fix/william/bridge-mqtt-TLS-verify-wildcard-peer
fix(mqtt-bridge): support verify TLS wildcard cert
2023-02-10 09:39:35 +01:00
Zaiming (Stone) Shi 3e85cb3804
Merge pull request #9946 from zmstone/0209-fix-add-back-deprecate-field
fix(config): add back reconnect_interval field as deprecated
2023-02-10 09:19:12 +01:00
Zaiming (Stone) Shi 40f719a34d
Merge pull request #9936 from id/feat-disable-disksup
feat: disable disksup by default
2023-02-10 09:17:13 +01:00
Zaiming (Stone) Shi 42dfaf3ef2
Merge pull request #9910 from sstrigler/EMQX-8861-improve-bridge-restart-button-behaviour
EMQX 8861 improve bridge restart button behaviour
2023-02-09 18:00:48 +01:00
Zaiming (Stone) Shi 4e71f1cbc5
Merge pull request #9940 from zmstone/0208-ensure-no-leading-br-in-docs
Add style checks for i18n.conf files
2023-02-09 17:37:44 +01:00
Zaiming (Stone) Shi 5328955ff4 fix(config): add back reconnect_interval field as deprecated 2023-02-09 17:30:09 +01:00
Erik Timan 3dea6b7913
Merge pull request #9930 from olcai/add-live-connections-to-prometheus-v5
feat(emqx_prometheus): expose live_connections stats to prometheus
2023-02-09 16:48:42 +01:00
Andrew Mayorov 48efe552d1
test(mqtt-bridge): manage test flaps by waiting for flush events 2023-02-09 14:31:42 +03:00
Zaiming (Stone) Shi 157c919ba1 ci: add i18n style check script 2023-02-09 11:41:52 +01:00
ieQu1 123fea44de chore(ekka): Bump version to 0.14.0 2023-02-08 17:35:14 +01:00
Zaiming (Stone) Shi 5df667738a docs: fix bad i18n files 2023-02-08 17:25:22 +01:00
Ivan Dyachkov fdb802975c feat: disable disksup by default 2023-02-08 15:33:35 +01:00
Stefan Strigler f89ac54b17 style: remove stale comment 2023-02-08 14:04:09 +01:00
Stefan Strigler a3fd0897bc refactor: less code duplication 2023-02-08 14:04:09 +01:00
Stefan Strigler 502f62e18d fix: try old 'restart' behavior if 'start' is not implemented 2023-02-08 13:19:41 +01:00
Andrew Mayorov 81b1bab11e
chore: bump `emqx_resource` version to 0.1.7
Also add the changelog entry.
2023-02-08 14:21:30 +03:00
Andrew Mayorov c6fc0ec8cd
fix(bufworker): do not avoid retry if inflight table is full
Otherwise there's no other piece of code that would retry the inflight
queries in that case.
2023-02-08 14:08:04 +03:00
Andrew Mayorov d8d06a260f
test(buffer): add test on inflight overflow w/ async queries
This testcase should verify that the buffer will retry all inflight
queries failed with recoverable errors + flush all outstanding queries.

Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-02-08 14:08:04 +03:00
Stefan Strigler cec8afe1b4
Merge pull request #9875 from sstrigler/EMQX-7119-fix-upload-broken-plugin
EMQX 7119 fix upload broken plugin
2023-02-08 09:38:52 +01:00
Zaiming (Stone) Shi d628aba079
Merge pull request #9718 from zmstone/0108-refactor-authn-schema-union-member-selector
0108 refactor authn schema union member selector
2023-02-07 16:45:47 +01:00
William Yang 6113d467c1 fix(mqtt-bridge): support verify TLS wildcard cert 2023-02-07 16:40:55 +01:00
Erik Timan 784ec0c0a5 chore: bump VSN 2023-02-07 15:54:01 +01:00
Erik Timan 19c2288a4b feat(emqx_prometheus): expose live_connections stats to prometheus 2023-02-07 15:42:55 +01:00
Stefan Strigler 86f3f5787f feat: allow to manually re-connect disconected bridge 2023-02-07 11:58:30 +01:00
Zaiming (Stone) Shi 537ff30550 chore: update code comment 2023-02-07 11:40:17 +01:00
Zaiming (Stone) Shi d487299b75 refactor: remove dead code 2023-02-07 08:34:10 +01:00
Zaiming (Stone) Shi 4653ca65f7 fix(logger): fix REPORT_CB/2 CRASH logs 2023-02-06 23:46:49 +01:00
Zaiming (Stone) Shi dc8f0c5101 refactor(authn): no need to check config at app boot 2023-02-06 18:19:22 +01:00
Zaiming (Stone) Shi 83e66da2aa fix(authn): throw exception on invalid config from the API 2023-02-06 14:12:50 +01:00
Stefan Strigler 758d19f1e0 chore: bump vsn 2023-02-06 10:44:06 +01:00
Stefan Strigler 4e12a44ee6 fix: return 204 instead of 200 if there's no body content 2023-02-06 09:47:07 +01:00
Stefan Strigler 847f899fa0 test: ensure bad plugin upload gets deleted 2023-02-06 09:46:16 +01:00
Stefan Strigler e135f8654c test: enable basic plugins API tests 2023-02-06 09:46:16 +01:00
Stefan Strigler a866c995f5 test: add test for arbitrary content in tar file 2023-02-06 09:46:16 +01:00
Stefan Strigler 70c4e12b68 fix: cleanup after upload of broken plugin 2023-02-06 09:46:16 +01:00
Stefan Strigler 618ec04340 style: fix comment 2023-02-06 09:46:16 +01:00
Zaiming (Stone) Shi a9c650da66 refactor: for better test coverage 2023-02-04 07:59:49 +01:00
Zaiming (Stone) Shi 2ebc89e339 refactor: authn schema union selector 2023-02-04 07:58:43 +01:00
Zaiming (Stone) Shi 4f91bf415c fix: schema check error reason pattern 2023-02-04 07:58:06 +01:00
Zaiming (Stone) Shi 65319567cc chore: no type check for 3rd party auth 2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi fbc52330f3 refactor: do not re-check config in post_config_update
now that the type is not lazy, it should have already been checked
when reaching the post_config_update step
2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi 6aaff6211f feat: use union member selector for authn config schema 2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi 15035f7eb0 refactor: remove lazy type for authentication
The idea of using lazy type for authentication was to make the
config check easy to extend at runtime. However the reality
is: it's overly complicated.

It's more likely that people will just continue to implementing
the auth hook callbacks instead of injecting config schema
at runtime.
2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi ccb0c7ebaa Merge remote-tracking branch 'origin/release-50' into 0202-merge-release-50-back-to-master 2023-02-03 16:12:11 +01:00
Zhongwen Deng d45a0cb2ee chore: remove <<>> when logging username 2023-02-03 16:54:18 +08:00
Zaiming (Stone) Shi 1e0fe270bc chore: bump emqx app vsn to 5.0.17 2023-02-02 20:32:09 +01:00
Zaiming (Stone) Shi 94768c9f44 Merge remote-tracking branch 'origin/master' into 0202-merge-release-50-back-to-master 2023-02-02 20:21:26 +01:00
Zaiming (Stone) Shi 5fe3a0b1b3
Merge pull request #9898 from zmstone/0202-upgrade-to-dashboard-e1.0.3
chore: upgrade dashboard to v1.1.7 for ce and e1.0.3 for ee
2023-02-02 17:41:52 +01:00
Zaiming (Stone) Shi 42462db8b8 chore: bump version to v5.0.16 and e5.0.0 2023-02-02 17:26:00 +01:00
Zaiming (Stone) Shi 3d6756cbe8 test: fix addrinuse errors in listeners API test SUITE 2023-02-02 16:37:56 +01:00
Zaiming (Stone) Shi a71d983ff8 test: fix assertMatch patter to make rebar3 fmt work 2023-02-02 16:37:45 +01:00
Zaiming (Stone) Shi 7ea140599a
Merge pull request #9894 from id/ci-always-run-static-checks
ci: always run static_checks
2023-02-02 16:33:19 +01:00
Zaiming (Stone) Shi feca4cc0a5
Merge pull request #9892 from zmstone/0202-docs-cosmetic
0202 docs cosmetic
2023-02-02 15:43:58 +01:00
Zaiming (Stone) Shi 85fb53dcf0 refactor: use compile switch to diff functions between ce and ee 2023-02-02 15:15:31 +01:00
Zaiming (Stone) Shi 0d7cd2f0c5
docs: Update apps/emqx_conf/i18n/emqx_conf_schema.conf
Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-02 14:20:07 +01:00
Zaiming (Stone) Shi 980ba2fd35
docs: Update apps/emqx/i18n/emqx_schema_i18n.conf
Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-02 14:19:37 +01:00
Zaiming (Stone) Shi 0e299642cb chore: function renamed but forgot to update xref excemption 2023-02-02 14:16:23 +01:00
Zaiming (Stone) Shi 58627b7958 chore(emqx_resource_manager): ignore unused return value for dialyzer 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi 9a9943d35d chore(emqx_mgmt_cache): ignore unused return value for dialyer 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi 1c748070c1 fix(emqx_connector_mqtt_worker): publish_async can only return ok # Please enter the commit message for your changes. Lines starting 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi c0d478bd41 fix(buffer_worker): type spec 2023-02-02 14:11:12 +01:00
zhongwencool ee852d8204
Merge pull request #9886 from zhongwencool/mongo-connection-default-async
fix: remove async mode from mongodb/redis/mysql/pgsql bridge
2023-02-02 21:08:01 +08:00
Zaiming (Stone) Shi d5c482b0b0 docs: remove timer unit from description
the user input has time unit. e.g. "5s" for 5 seconds etc.
2023-02-02 13:49:20 +01:00
Zaiming (Stone) Shi 5dbee76af4
Merge pull request #9890 from zmstone/0202-fix-kafka-status-check
0202 fix kafka status check
2023-02-02 13:48:44 +01:00
firest 031b36da32 feat(banned): filter out banned messages from mqueue 2023-02-02 20:20:52 +08:00
Zaiming (Stone) Shi 2b308af354
Merge pull request #9889 from keynslug/fix/EMQX-8857/force-clean-start-mqtt-bridge
fix(mqtt-bridge): stop respecting `clean_start` config parameter
2023-02-02 12:55:58 +01:00
Zaiming (Stone) Shi 9864587389 fix: send to buffer-supported connector even when disconnected 2023-02-02 12:04:17 +01:00
Zaiming (Stone) Shi 13ef30c46c
Merge pull request #9884 from savonarola/resource-fixes
fix(resources): fix resource lifecycle
2023-02-02 12:02:34 +01:00
Andrew Mayorov 0912f13c1f
fix(mqtt-bridge): stop respecting `clean_start` config parameter
We are ignoring the user configuration because there's currently no
reliable way to ensure proper session recovery according to the MQTT
spec.
2023-02-02 12:55:10 +03:00
Kjell Winblad 2cf193e2fd
Merge pull request #9839 from kjellwinblad/kjell/fix/Authorization_header_log_leak_webhook/EMQX-8791
fix: Authorization header leak in log entries for webhook
2023-02-02 10:38:42 +01:00
Zhongwen Deng 1c9035d24c test: remove async from redis ct 2023-02-02 17:37:18 +08:00
Zhongwen Deng 22cc1cc745 fix: make spell_check happy 2023-02-02 17:37:18 +08:00
Zhongwen Deng f8936013b7 chore: replace async with sync 2023-02-02 17:37:18 +08:00
Zhongwen Deng 22c3f50020 fix: add query_mode_sync_only for mysql pgsql redis mongodb bridge 2023-02-02 17:37:18 +08:00
Andrew Mayorov ca5c192f4b
Merge pull request #9882 from fwup/fix/no-mqtt-bridge-middleman
refactor(mqtt-worker): avoid unnecessary abstraction
2023-02-02 13:11:31 +04:00
zhongwencool e5bc787467
Merge pull request #9872 from zmstone/0131-improve-help-text
docs: improve docs for system monitoring config
2023-02-02 14:45:03 +08:00
Zhongwen Deng 1a90c1654c chore: bad typo 2023-02-02 11:43:04 +08:00
Zaiming (Stone) Shi 6fd30a7e81 docs: improve docs for system monitoring config 2023-02-01 20:21:08 +01:00
Ilya Averyanov 14f528cc86 fix(resources): fix resource lifecycle
* do not resume all buffer workers on successful healthcheck
* do not pass undefined state to resource healthcheck callback
2023-02-01 18:26:13 +02:00
Zaiming (Stone) Shi ec3031271b
Merge pull request #9883 from zmstone/0201-upgrade-enterprise-dashboard-to-e1.0.2
0201 upgrade enterprise dashboard to e1.0.2
2023-02-01 16:26:21 +01:00
Zaiming (Stone) Shi 32d18ec594
Merge pull request #9878 from zmstone/0201-upgrade-ehttpc-0.4.6
fix(ehttpc): fix crash when webhook bridge uses `hash` pool type.
2023-02-01 15:00:28 +01:00
Andrew Mayorov 5fd7f65a1f
test(bufworker): make testcase simpler to follow
The confusion was due to the fact that subsequent query was missing
`async_reply_fun` and thus, was not accumulating in the results.
2023-02-01 16:52:47 +03:00
Zaiming (Stone) Shi 38ef37d109 chore: bump release version to e5.0.0-rc.4 2023-02-01 14:52:08 +01:00
Andrew Mayorov 8a46cb974e
test(mqtt-bridge): test async bridge reconnects seamlessly 2023-02-01 16:28:16 +03:00
Andrew Mayorov 5ebceb20d2
test(mqtt-bridge): also test reconfiguration 2023-02-01 16:23:58 +03:00
Andrew Mayorov ad88938d34
refactor: reuse some parts of test code for brewity 2023-02-01 16:22:19 +03:00
Andrew Mayorov f0395be383
refactor(mqtt-worker): avoid unnecessary abstraction
So the code is easier to follow.
2023-02-01 14:46:26 +03:00
zhongwencool 13511d2782
Merge pull request #9856 from zhongwencool/alarm-percent-decimals
fix: /api/nodes is timeout when emqx in high load
2023-02-01 19:40:53 +08:00
Andrew Mayorov 979e9804be
Merge pull request #9879 from keynslug/fwup/fix/buffer-worker-testcase
test(bufworker): fix testcase flapping due to data races
2023-02-01 15:33:59 +04:00
Andrew Mayorov ff473e0f1b
test(bufworker): fix testcase flapping due to data races 2023-02-01 12:57:46 +03:00
Zaiming (Stone) Shi 96a18e7105 chore: upgrade to hocon 0.35.3 2023-02-01 10:52:01 +01:00
Zhongwen Deng 6382915036 chore: repalce fresh by refresh 2023-02-01 17:36:30 +08:00
Zaiming (Stone) Shi 9f6b6cedc6 fix(webhook): pick worker according to pool type 2023-02-01 09:53:25 +01:00
Ilya Averyanov 9d91ebe266
Merge pull request #9842 from savonarola/fix-redis-cluster-recover
fix: fix redis cluster resource recovery
2023-02-01 10:38:52 +02:00
Zaiming (Stone) Shi f4381d90ca refactor: http cliet should not need to retry HTTP requests for bridge 2023-02-01 09:30:30 +01:00
Zaiming (Stone) Shi 6ca816f7a4 docs: remove impl details from user doc 2023-02-01 09:30:10 +01:00
Zhongwen Deng 3c4d09a752 fix: get_memory_cache return {ok, Mem} | stale 2023-02-01 16:00:55 +08:00
Zhongwen Deng ced55719ef chore: only run t_sys_mem_check ct in linux 2023-02-01 14:47:41 +08:00
Zhongwen Deng 56b9238645 fix: only cache sys_memory in linux 2023-02-01 12:21:34 +08:00
Zhongwen Deng 95ef1e806c chore: don't crash when add_handler failed 2023-02-01 11:24:27 +08:00
Zhongwen Deng 9cbad5297c fix: primary logger level not update 2023-02-01 11:16:30 +08:00
Zhongwen Deng 71f00f2962 test: ctl suite failed 2023-02-01 10:07:22 +08:00
Zhongwen Deng 7a9f477267 feat: use emqx_mgmt_cache to cache sys_memory 2023-02-01 10:07:22 +08:00
Zhongwen Deng c2bdb9faa7 test: multiple_nodes case failed 2023-02-01 10:07:22 +08:00
Zhongwen Deng b6e6315b50 feat: change loads from string to float 2023-02-01 10:07:22 +08:00
Zhongwen Deng 0b19be074c feat: cache OTP_VERSION in persistent_term 2023-02-01 10:07:22 +08:00
Zhongwen Deng 6162f90610 fix: don't crash when OTP_VERSION file is missing 2023-02-01 10:07:22 +08:00
Zhongwen Deng 5783127c30 test: cpu_sup:load mock test 2023-02-01 10:07:22 +08:00
Zhongwen Deng 2d67bb3fb6 fix: /api/nodes is timeout if emqx in high load 2023-02-01 10:07:22 +08:00
Zaiming (Stone) Shi 7e8253e3af chore: bump version to e5.0.0-rc.3 2023-01-31 20:45:28 +01:00
Zaiming (Stone) Shi 78028a7fcf
Merge pull request #9876 from zmstone/0131-avoid-persisting-logging-configs-if-no-change
0131 avoid persisting logging configs if no change
2023-01-31 20:43:32 +01:00
Zaiming (Stone) Shi b3ad9e97d2
Merge pull request #9870 from keynslug/fix/mqtt-connection-loss-feedback
feat(mqtt-bridge): avoid middleman process
2023-01-31 19:12:18 +01:00
Zaiming (Stone) Shi ddfbc0a19f refactor: refresh logger config before starting listeners 2023-01-31 18:50:16 +01:00
Zaiming (Stone) Shi 605d9972e4 fix(config): avoid calling emqx:update_config/2 for logger refresh
Prior to this fix, whatever configured for loggers are persisted to
cluster-override.conf.
This may cause troulbe for users who changes boot mode.

For example if the node is once started in console mode,
the logging config is persisted, with console enabled, but file
disabled.
Then if the user decide to start in daemon mode, all the logs
will silently go to erlang.log.N and emqx.log.N will be empty.

After this fix, only changes really made into cluter-override.conf
will take effect.
2023-01-31 18:50:16 +01:00
Zaiming (Stone) Shi b0ac924ca9 refactor: less copy-paste 2023-01-31 18:40:55 +01:00
lafirest 3d59be6ab6
Merge pull request #9871 from lafirest/fix/authz_rules
feat(authz): allow the placeholder to be anywhere in the topic for authz rules
2023-02-01 00:12:06 +08:00
Andrew Mayorov c5a7cd5acd
fix(mqtt-bridge): drop unused configuration parameter 2023-01-31 18:30:42 +03:00
Andrew Mayorov c76311c9c3
fix(buffer): count inflight batches properly 2023-01-31 18:30:42 +03:00
firest 9f4c36ecbc chore: bump version && update changes 2023-01-31 23:30:22 +08:00
firest c034cbf6de feat(authz): allow the placeholder to be anywhere in the topic for authz rules 2023-01-31 23:30:12 +08:00
Andrew Mayorov d0c10b59aa
feat(mqtt-bridge): avoid middleman process
Instead, supervise `emqtt` client process directly.
2023-01-31 17:59:03 +03:00
Ilya Averyanov fce1e74c3d fix(connector): fix redis cluster resource recovery 2023-01-31 16:55:05 +02:00
Zaiming (Stone) Shi f6dafc20ea feat: export API to read cluster and local overrides 2023-01-31 14:20:34 +01:00
Stefan Strigler ade5316419
Merge pull request #9802 from sstrigler/EMQX-8469-enable-proxy-protocol-support-in-emqx-dashboard-schema
feat: support HAProxy protocol for dashboard API
2023-01-31 14:19:56 +01:00
Erik Timan 3786bb8086
Merge pull request #9824 from olcai/internal-error-api-get-topic
fix(emqx_management): handle multiple routes in topics/{topic} API
2023-01-31 13:50:42 +01:00
Andrew Mayorov 4d146c521b
fix(mqtt-bridge): ensure proper feedback on async forwards
So that buffer worker would notice a connection loss in time, and
recycle inflight messages subsequently.
2023-01-31 11:09:39 +03:00
Andrew Mayorov 35c429ef1d
refactor: drop a couple of unused macros / includes 2023-01-31 11:09:39 +03:00
Zaiming (Stone) Shi b3e486041b
Merge pull request #9853 from zmstone/0127-refactor-buffer-worker-no-need-to-keep-request-for-reply-callback
0127 refactor buffer worker no need to keep request for reply callback
2023-01-31 08:44:01 +01:00
Erik Timan 5b3a77e3c7 test(emqx_management): fix flaky route handling in get topic test
This reworks a test case to use a second slave node. This ensures that
an added route is permanently in the routing table. The old version
reverted the routing table quickly since the node name given wasn't a
real node.
2023-01-31 08:28:55 +01:00
Erik Timan 03cabf6b26 chore: bump app VSN 2023-01-31 08:27:54 +01:00
Erik Timan 33e011aff5 fix(emqx_management): handle multiple routes in topics/{topic} API
The topics/{topic} API endpoint would return 500 - Internal Error if a
topic had multiple routes. This is now fixed by returning a list of
routes.
2023-01-31 08:27:33 +01:00
Ilya Averyanov 1a1c3afa81
Merge pull request #9852 from savonarola/fix-redis-desc
Fix Redis configuration field description
2023-01-30 23:26:37 +02:00
Stefan Strigler 3ce6cbcd6a chore: bump vsn 2023-01-30 18:09:10 +01:00
Stefan Strigler 0b1483040a test: add actual test for haproxy 2023-01-30 18:07:06 +01:00
lafirest e288424924
Merge pull request #9864 from lafirest/fix/exclusive-unsub-5.0
fix: the exclusive topics aren't removed when the session has already been cleaned
2023-01-31 00:33:45 +08:00
Kjell Winblad 0c4134c423 test: add unit test case for redact function in http connector 2023-01-30 11:51:28 +01:00
Stefan Strigler 27881064dc fix: increase dropped.queue_full by number of messages 2023-01-30 11:37:35 +01:00
lafirest 3e6cc40525
Merge pull request #9843 from lafirest/fix/speed_up_test
fix: speed up banned test cases
2023-01-30 18:17:27 +08:00
firest 3bdffca488 fix: the exclusive topics aren't removed when the session has already been cleaned 2023-01-30 18:17:00 +08:00
Zhongwen Deng bb636394e1 chore: add debug log for raw data 2023-01-30 16:35:06 +08:00
Zhongwen Deng b73d11675e chore: log the bad mqtt packet(frame error) 2023-01-30 15:05:49 +08:00
Zhongwen Deng 3d07271ea5 fix: crash when mfa not found 2023-01-30 14:30:20 +08:00
Zhongwen Deng 25090563af chore: use brackets to wrap the mqtt packet when logging 2023-01-30 12:29:00 +08:00
Zhongwen Deng ce32ea7334 feat: Make the log output format order fixed 2023-01-30 12:18:22 +08:00
Zhongwen Deng 6cbad047cd fix: don't log CONNECT twice when debug 2023-01-30 12:16:17 +08:00
Zaiming (Stone) Shi d47941601d refactor(buffer_worker): rename trace points 2023-01-28 11:52:11 +01:00
Zaiming (Stone) Shi 7f66c6a9e2
Merge pull request #9840 from olcai/redact-influxdb-tokens
fix: redact influxdb tokens in logs and reduce log level
2023-01-28 11:47:36 +01:00
Zaiming (Stone) Shi fc38ea9571 refactor(buffer_worker): do not keep request body in reply context
the request body can be potentially very large
the reply context is sent to the async call handler and kept
in its memory until the async reply is received from bridge
target service.

this commit tries to minimize the size of the reply context
by replacing the request body with `[]`.
2023-01-27 17:12:55 +01:00
Ilya Averyanov 72f39b9b72 fix(docs): correct Redis conf field description 2023-01-27 17:39:16 +02:00
Zaiming (Stone) Shi 578271ea3d refactor: use lists:map instead of lc for safty 2023-01-27 15:15:46 +01:00
Zaiming (Stone) Shi f793807bc1 refactor(buffer_worker): rename function
batch_reply_after_query to handle_async_batch_reply
2023-01-27 15:04:28 +01:00
Zaiming (Stone) Shi 262c3a2869 refactor(buffer_worker): rename function
from reply_after_query to handle_async_reply
2023-01-27 15:03:18 +01:00
Zaiming (Stone) Shi 6a58bafcb0 chore: bump release version to e5.0.0-rc.2 2023-01-27 14:38:21 +01:00
Zaiming (Stone) Shi 52b75ada04
Merge pull request #9832 from sstrigler/EMQX-8774-failure-to-handle-timeout-error-in-resource-worker
EMQX 8774 failure to handle timeout error in resource worker
2023-01-27 14:36:44 +01:00
Zaiming (Stone) Shi 514609bcf7
Merge pull request #9850 from zmstone/0127-fix-influxdb-bridge-atom-leak
0127 fix influxdb bridge atom leak
2023-01-27 14:30:20 +01:00
Zaiming (Stone) Shi 96ed725a55
Merge pull request #9849 from zmstone/0127-refactor-buffer-worker-simplify-caller-reply
0127 refactor buffer worker simplify caller reply
2023-01-27 14:06:56 +01:00
Zaiming (Stone) Shi c47be57c59 fix(bridge): ensure all bridge resources are stopped before app stop 2023-01-27 12:39:05 +01:00
Zaiming (Stone) Shi d53106145f fix: stop resource when resource manager terminates 2023-01-27 12:39:05 +01:00
Andrew Mayorov d35e46b2d5
Merge pull request #9838 from keynslug/fix/redis-cluster-batching
feat(redis): disable batching in redis_cluster bridges
2023-01-27 15:27:57 +04:00
Stefan Strigler 7005b71ddf style: fix typo in comment 2023-01-27 11:43:51 +01:00
Stefan Strigler 2d62de5188 test: fix expected result from timeout error 2023-01-27 11:43:48 +01:00
Stefan Strigler a180bd9aa5 fix: catch error, not exit 2023-01-27 11:40:06 +01:00
Stefan Strigler b7e3f9d5a6 fix: try-case-of rather than try-of
try-of catches only what happens within but not after
2023-01-27 11:40:06 +01:00
Zaiming (Stone) Shi db2f631a8a refactor(buffer_worker): simplify caller reply 2023-01-27 11:33:45 +01:00
Zaiming (Stone) Shi 965236c888
Merge pull request #9845 from zmstone/0126-reply-caller-for-buffer-overflow-queue-items
0126 reply caller for buffer overflow queue items
2023-01-27 11:32:59 +01:00
Zaiming (Stone) Shi d4fab92b72 refactor(buffer_worker): no need to keep request for REPLY macro 2023-01-27 10:41:30 +01:00
Andrew Mayorov 71f996b9d5
refactor(mqtt-bridge): unwrap single statem actions
So that the code would be easier to follow and harder to break.
Also drop a couple of unused macrodefs.
2023-01-26 21:13:18 +03:00
Andrew Mayorov c95f979413
fix(mqtt_bridge): use correct gen_statem reply action 2023-01-26 21:12:05 +03:00
Zaiming (Stone) Shi 1f799dfd59 fix: reply with {error, buffer_overflow} when discarded 2023-01-26 17:15:36 +01:00
Zaiming (Stone) Shi ed28789164 refactor(buffer_worker): no need to return after collect into buf queue 2023-01-26 14:50:40 +01:00
Zaiming (Stone) Shi 25b4821adc refactor: move the the per-message overflow log from error to info level 2023-01-26 14:48:43 +01:00
Zaiming (Stone) Shi bb26632c8a fix(buffer_worker): fix a wrong assertion
the assertion is to ensure queue items are not binary
but should not assert the queue itself
2023-01-26 14:33:16 +01:00
Zaiming (Stone) Shi f6b3b930b0 chore: improve a error log 2023-01-26 14:21:27 +01:00
Kjell Winblad 86cfbfb43c fix: Authorization header leak in log entries for webhook
There might be another possibility for leakage. If the resource mangager
for the webhook resource crashes, OTP might log the spec for the
resource manager which contains the Config and thus the Authorization
header. This is probably an issue for other resources as well and should
be fixed in another commit. The following issue has been created for
that:

https://emqx.atlassian.net/browse/EMQX-8794

Fixes:
https://emqx.atlassian.net/browse/EMQX-8791
2023-01-26 12:44:16 +01:00
firest 908ac8d948 fix: speed up banned test cases 2023-01-26 13:04:55 +08:00
Andrew Mayorov 2ee00b75a7
fix(redis): unwrap pipeline queries against redis cluster
This is an additional safety measure in addition to the disabled
batching on the bridge level.
2023-01-25 17:28:11 +03:00
Erik Timan 805d08e823 fix: reduce log level from error to warning in several places
This reduces the log level from error to warning in places that are
connected to the influxdb bridge. Transient errors for external
resources should not render an error log.
2023-01-25 14:49:50 +01:00
Erik Timan 8836494542 fix: redact influxdb tokens in a few logs 2023-01-25 14:48:32 +01:00
Zaiming (Stone) Shi 5fdf7fd24c fix(kafka): use async callback to bump success counters
some telemetry events from wolff are discarded:

* dropped:
    this is double counted in wolff,
    we now only subscribe to the dropped_queue_full event
* retried_failed:
    it has different meanings in wolff,
    in wolff, it means it's the 2nd (or onward) produce attempt
    in EMQX, it means it's eventually failed after some retries

* retried_success
    since we are going to handle the success counters in callbac
    this having this reported from wolff will only make things
    harder to understand

* failed
    wolff never fails (unelss drop which is a different counter)
2023-01-24 21:12:36 +01:00
Zaiming (Stone) Shi 6175076f6f
Merge pull request #9835 from olcai/add-influxdb-test-files
fix: add influxdb test files and fixes
2023-01-24 17:02:23 +01:00
Zaiming (Stone) Shi e5b65087af
Merge pull request #9834 from zmstone/0123-fix-idle_timeout-infinity
fix(emqx_connection): crash when idle_timeout is set to infinity
2023-01-24 16:05:09 +01:00
Erik Timan 9d20431257 fix(emqx_resource): fix crash while flushing queue
We used next_event for flushing the queue in emqx_resource, but this
leads to a crash. We now call flush_worker/1 instead.
2023-01-24 14:13:35 +01:00
Erik Timan 28718edbfd chore: bump application VSNs 2023-01-24 14:12:34 +01:00
Zaiming (Stone) Shi 8fde169abb
Merge pull request #9821 from thalesmg/buffer-worker-expiry-v50
feat(buffer_worker): add expiration time to requests
2023-01-24 13:54:04 +01:00
Zaiming (Stone) Shi 7575120ea6 test: use snabbkaffe retry macro 2023-01-24 10:54:20 +01:00
Zaiming (Stone) Shi 140cda2f13 fix(emqx_connection): crash when idle_timeout is set to infinity 2023-01-24 10:14:35 +01:00
Zaiming (Stone) Shi a0100c0a4e chore: bump app versions 2023-01-23 16:20:21 +01:00
Zaiming (Stone) Shi a83ed2fe94 Merge remote-tracking branch 'origin/master' into 0123-chore-sync-release-50-to-master 2023-01-23 16:20:01 +01:00
Zaiming (Stone) Shi 727100e094 chore: prepare for v5.0.15 release 2023-01-20 16:42:01 +01:00
Thales Macedo Garitezi ca4a262b75 refactor: re-organize dealing with unrecoverable errors 2023-01-20 12:00:17 -03:00
Thales Macedo Garitezi 6fa6c679bb feat(buffer_worker): add expiration time to requests
With this, we avoid performing work or replying to callers that are no
longer waiting on a result.

Also introduces two new counters:

- `dropped.expired` :: happens when a request expires before being
  sent downstream
- `late_reply` :: when a response is receive from downstream, but the
  caller is no longer for a reply because the request has expired, and
  the caller might even have retried it.
2023-01-20 11:36:52 -03:00
Zaiming (Stone) Shi 57607ca0ce chore: prepare for v5.0.15 release 2023-01-20 11:20:34 +01:00
Zaiming (Stone) Shi 1c3e055b13
Merge pull request #9822 from JimMoen/fix-schema-typo
chore: i18n typo fix
2023-01-20 11:11:18 +01:00
JimMoen 16f45a60fd
chore: i18n typo fix 2023-01-20 11:50:01 +08:00
Thales Macedo Garitezi a6ad97e6a5
Merge pull request #9807 from thalesmg/buffer-worker-rename-v50
buffer work refactoring follow up (part 1)
2023-01-19 14:17:10 -03:00
Zaiming (Stone) Shi abe7a69696 Merge remote-tracking branch 'origin/master' into release-50 2023-01-19 17:51:38 +01:00
Stefan Strigler 6bde1173af chore: add translations 2023-01-19 13:26:36 +01:00
Thales Macedo Garitezi d755b43c77 fix(jwt_worker): handle exceptions when decoding jwk from pem
Returns a more controlled error if users attempt to use the Service
Account JSON from the GCP PubSub example from swagger, which is
redacted.
2023-01-19 09:24:45 -03:00
Zaiming (Stone) Shi 63748aba3c
Merge pull request #9804 from emqx/release-50
Merge release-50 (candidate of e5.0.0-rc.1) back to master.
2023-01-19 08:48:41 +01:00
Thales Macedo Garitezi 47f796dd12 refactor: rename `emqx_resource_worker` -> `emqx_resource_buffer_worker`
To make it more clear that it's purpose is serve as a buffering layer.
2023-01-18 16:15:34 -03:00
Ilya Averyanov f9843de7ae
Merge pull request #9628 from savonarola/fix-flaky-redis-bridge-test
chore(ee bridge): fix Redis bridge test flakyness
2023-01-18 20:56:13 +02:00
Zaiming (Stone) Shi 1716a5da99 chore: bump version to e5.0.0-rc.1 2023-01-18 17:22:05 +01:00
Stefan Strigler fb763ecebd feat: support HAProxy protocol for dashboard API 2023-01-18 15:49:57 +01:00
Zaiming (Stone) Shi 7abba17b25
Merge pull request #9765 from zmstone/0115-add-password-converter
fix(schema): add password converter to ensure its binary() type
2023-01-18 15:09:05 +01:00
Andrew Mayorov 33b3c4fa9a
Merge pull request #9753 from feat/EMQX-8738/convert-ordered-sets
feat: turn tables queried with search APIs into ordered sets
2023-01-18 18:05:57 +04:00
Ilya Averyanov 44a6e5ed15 chore(resources): add missing parameters to emqx_resource schema 2023-01-18 14:33:45 +02:00
Zaiming (Stone) Shi 7e8381f4c7
Merge pull request #9785 from savonarola/fix-authn-handling
fix(authn): stop authn handling when emqx_authentication provides a result
2023-01-18 13:24:22 +01:00
ieQu1 c46d7f3404
Merge pull request #9801 from ieQu1/ekka-0.13.9
chore(ekka): Bump version to 0.13.9
2023-01-18 13:23:59 +01:00
Thales Macedo Garitezi 167b623497
Merge pull request #9699 from thalesmg/fix-buffer-clear-replayq-on-delete-v50
fix(buffer): fix `replayq` usages in buffer workers (5.0)
2023-01-18 09:08:50 -03:00
Erik Timan 46fc69cd48
Merge pull request #9781 from olcai/delete-zip-file-from-trace-log-download
fix(emqx_management): delete files after trace log download
2023-01-18 13:05:39 +01:00
ieQu1 d7242739e0 chore(ekka): Bump version to 0.13.9 2023-01-18 12:01:03 +01:00
Zaiming (Stone) Shi d4f3b4c8c2 Merge remote-tracking branch 'origin/master' into fix-buffer-clear-replayq-on-delete-v50 2023-01-18 11:39:47 +01:00
zhongwencool 8e1475addb
Merge pull request #9798 from zhongwencool/dashboard-document
chore: improve the dashboard's configuration
2023-01-18 18:20:00 +08:00
Erik Timan 42182279b7 fix(emqx_management): ensure trace file dir is deleted on zip exception 2023-01-18 10:20:41 +01:00
Zhongwen Deng fb84d5b817 chore: make spellcheck happy 2023-01-18 17:06:46 +08:00
Ivan Dyachkov 430b0a03d4
Merge pull request #9780 from id/fix-ensure-no-colon-in-filenames
fix: ensure no colon in filenames
2023-01-18 09:36:16 +01:00
Zhongwen Deng 0d852d9122 docs: improve the dashboard's document 2023-01-18 16:28:35 +08:00
Zaiming (Stone) Shi faf5916ed6 test: relax recoverable/unrecoverable error check
for now, treat all other errors unrecoverable
2023-01-18 07:52:28 +01:00
zhongwencool bc9d97ea53
Merge pull request #9791 from zhongwencool/crash-dump-doc
chore: more detail about crash dump config
2023-01-18 09:56:21 +08:00
Thales Macedo Garitezi 5c2ac0ac81 chore: don't cancel inflight items upon worker death; retry them 2023-01-17 19:50:30 -03:00
Thales Macedo Garitezi 087b667263 fix(buffer_worker): allow signalling unrecoverable errors 2023-01-17 19:50:30 -03:00
Stefan Strigler f899284e3a
Merge pull request #9789 from sstrigler/EMQX-8754-test-function-return-500-of-data-integration-google-pubsub
EMQX 8754 test function return 500 of data integration google pubsub
2023-01-17 22:49:28 +01:00
lafirest dea0c8230e
Merge pull request #9787 from lafirest/fix/webhook_bridge_cfg_upgrade
fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12
2023-01-18 04:47:08 +08:00
Thales Macedo Garitezi 4ed7bff33f chore: fix dialyzer warnings 2023-01-17 16:49:16 -03:00
Thales Macedo Garitezi fa01deb3eb chore: retry as much as possible, don't reply to caller too soon 2023-01-17 16:49:15 -03:00
Thales Macedo Garitezi b82009bc29 refactor: use monotonic times as refs and store initial times when creating ets
with this, we may measure latencies in the future.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 3ba65c4377 feat: poke the buffer workers when inflight is no longer full
if max inflight = 1, then we only make progress based on the state
timer, since the callbacks were not poking the buffer workers.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi b5aaef084c refactor: enter running state directly
now that we don't have the possibility of dirty disk queues (we always
use volatile replayq), we will never resume old work.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi bd0e2a74ba refactor: rename inflight_name field to inflight_tid 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 006b4bda97 feat(buffer_worker): monitor async workers and cancel their inflight requests upon death 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 731ac6567a fix(buffer_worker): don't retry all kinds of inflight requests
Some requests should not be retried during the blocked state.  For
example, if some async requests are just taking some time to process,
we should avoid retrying them periodically, lest risk overloading the
downstream further.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 5425f3d88e refactor: rm unused fn 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 5dd24a64c3 refactor(buffer_worker): check if inflight is full before flushing 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 344eeebe63 fix: always ack async replies
The caller should decide if it should retry in that case, to avoid
overwhelming the resource with retries.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi bd95a95409 refactor: remove redundant `BlockWorker` arg, change boolean to ack/nack
`BlockWorker` was always false (ack).  Also, changed the return to
something more semantic than a boolean to avoid [boolean
blindness](https://runtimeverification.com/blog/code-smell-boolean-blindness/)
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 478fcc6ffd test: fix flaky test 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 30a227bd38 refactor: rename `resume` state timeout to `unblock` 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 7401d6f0ce refactor: rename ack fn 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 196bf1c5ba feat: mass collect calls from mailbox also when blocked 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi d4724d6ce9 refactor: remove redundant function
`retry_queue` does basically what the running state does, now that we
refactored the buffer workers to always use the queue.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi d6a9d0aa48 fix: set queuing to 0 after buffer worker termination 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 81fc561ed5 fix(buffer_worker): check for overflow after enqueuing new requests 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 4cb83d0c9a fix: fix some expressions after refactoring 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi fecdbac9a8 refactor: rename a few functions 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi cdd8de11b0 chore: fix a typo in function name 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi 618b97870b refactor: call local function queue_count everywhere 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi 249c4c1c79 refactor: use 'bufs' for resource worker replayq dir 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi af6807e863 refactor: cancel flush timer sooner
Avoids the cancellation being delayed.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 477c55d8ef fix: sanitizy replayq dir filepath
Colons (`:`) are not allowed in Windows.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 4c04a01370 refactor(buffer_worker): remove `?Q_ITEM` wrapping and use lightweight size estimate 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 32a9e60313 feat(buffer_worker): also use the inflight table for sync requests
Related: https://emqx.atlassian.net/browse/EMQX-8692

This should also correctly account for `retried.*` metrics for sync
requests.

Also fixes cases where race conditions for retrying async requests
could potentially lead to inconsistent metrics.

Fixes more cases where a stale reference to `replayq` was being held
accidentally after a `pop`.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi ff23d25e8b chore(replayq): update replayq -> 0.3.6 and use `clean_start` for buffer workers
So we can truly avoid resuming work after a node restart.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi c383558467 fix(buffer): fix `replayq` usages in buffer workers (5.0)
https://emqx.atlassian.net/browse/EMQX-8700

Fixes a few errors in the usage of `replayq` queues.

- Close `replayq` when `emqx_resource_worker` terminates.
- Do not keep old references to `replayq` after any `pop`s.
- Clear `replayq`'s data directories when removing a resource.
2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi fc992f28bc test: add test coverage 2023-01-17 20:13:01 +01:00
Zaiming (Stone) Shi fff6bf921f refactor(authz): call emqx_resource:simple_sync_query
there is no need to route the request through the buffer workers
2023-01-17 20:01:45 +01:00
Zaiming (Stone) Shi 7a471faf67 refactor(authn): use simple sync query for authn
there is no need to route the authn queries through the buffer worker
2023-01-17 19:57:14 +01:00
Ilya Averyanov 1d7364dab1 fix(authn): fix authn hook chain evaluation
Stop authentication handling when
`emqx_authentication` provides a definitive result.
2023-01-17 20:06:20 +02:00
Stefan Strigler d164e5bc0f fix: don't require ssl conf 2023-01-17 15:29:19 +01:00
Stefan Strigler e54f2f83b3 test: use same default timeout as elsewhere 2023-01-17 15:29:19 +01:00
Erik Timan 30a5cfaa83 fix(emqx_management): remove trace files after zip download
We only deleted the resulting zip after a trace file download, not the
actual trace files. This adds a deletion of the uncompressed trace files
as well. It also creates unique directories when collecting trace files
so that concurrent downloads doesn't overwrite files in transit.
2023-01-17 14:24:10 +01:00
Erik Timan 94296258b4 test(emqx_management): refactor test suite to use common test utils for API calls 2023-01-17 14:24:10 +01:00
Zhongwen Deng f98786c91f fix: bad crash_dump_file default value in rpm 2023-01-17 20:47:54 +08:00
Zhongwen Deng cf99089f84 chore: add some trace func to user_default 2023-01-17 20:33:01 +08:00
Zhongwen Deng 5a56013193 docs: more detail about crashdump config 2023-01-17 20:33:01 +08:00
zhongwencool 9f80eb2497
Merge pull request #9777 from zmstone/0116-refactor-config_files-is-readonly
refactor: config_files is a readonly config
2023-01-17 20:32:39 +08:00
Zaiming (Stone) Shi 263deae1f3 refactor: add a more generic name for password_converter 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 2afbf6a406 test: unlink process which is getting shutdown 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 4a7e74f5d6 fix(schema): add password converter to ensure its binary() type 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 8b3ec48406 refactor: node.config_files should be hidden as it's not configurable
this config schema exists only for app-env mapping.
users can not change it because the bootstraping config
has to be etc/emqx.conf
2023-01-17 11:08:40 +01:00
Stefan Strigler 74ae7c4264
Merge pull request #9782 from sstrigler/EMQX-8361-fix-ct-suite-for-plugins-api
test: get plugin template as artifact
2023-01-17 11:08:07 +01:00
firest c3d5c25c26 fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12 2023-01-17 17:42:10 +08:00
zhongwencool cb5314bb9e
Merge pull request #9774 from zhongwencool/dashboard-password-validate-v50
feat: add dashboard password validation
2023-01-17 17:02:51 +08:00
Stefan Strigler bc69a00329 test: get plugin template as artifact 2023-01-17 09:50:17 +01:00
Shawn 44f635300d chore: update mysql-otp to 1.7.2 2023-01-17 13:36:26 +08:00
Zhongwen Deng 3dfdad9cc9 test: dashboard default password 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi b9911e281e docs: Update apps/emqx_dashboard/src/emqx_dashboard_admin.erl 2023-01-17 11:30:42 +08:00
zhongwencool c549379b7c chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-17 11:30:42 +08:00
Zhongwen Deng 82af854c4a test: dashbashboard new password failed 2023-01-17 11:30:42 +08:00
Zhongwen Deng a523fa2fa2 feat: add dashboard password validate 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi 33be7672ad test: recover boot_modules app env after test 2023-01-17 00:43:25 +01:00
Ivan Dyachkov 676f017ec0 fix: ensure no colon in filenames 2023-01-16 21:27:01 +01:00
Andrew Mayorov 5fd157fb1d
Merge pull request #9776 from keynslug/chore/bump-epgsql-otp-25
chore: update epgsql to 4.7.0.1 with full OTP-25 compat
2023-01-16 23:51:09 +04:00
Zaiming (Stone) Shi e668044ebc fix: version number previs in restricted shell 2023-01-16 13:47:00 +01:00
Andrew Mayorov 34571c779d
feat: make `suboption` table ordering more natural 2023-01-16 15:39:10 +03:00
Andrew Mayorov ce2dba15b4
feat: turn tables queried with search APIs into ordered sets
This is needed to ensure more or less consistent client experience
for the new planned cursor-based search APIs.
2023-01-16 15:39:10 +03:00
Zaiming (Stone) Shi a7fc5e8fe1
Merge pull request #9761 from zmstone/0114-fix-kafka-value-template-and-docs
feat: introduce 'this' concept for placeholder, and use it in Kafka bridge
2023-01-16 13:37:29 +01:00
Andrew Mayorov 54cea3b3f2
chore: update epgsql to 4.7.0.1 with full OTP-25 compat 2023-01-16 15:35:51 +03:00
Stefan Strigler 67909f0b40 fix: testing metrics for emqx_bridge_mqtt_SUITE 2023-01-16 12:10:06 +01:00
Stefan Strigler a19226326c fix: don't test for strict values as that makes test flaky 2023-01-16 12:10:06 +01:00
Stefan Strigler 9a53410efb fix: don't add empty list element to options proplist 2023-01-16 12:00:03 +01:00
Stefan Strigler edfcea7f43 fix: add log for timeout 2023-01-16 12:00:03 +01:00