Commit Graph

6252 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 2ebc89e339 refactor: authn schema union selector 2023-02-04 07:58:43 +01:00
Zaiming (Stone) Shi 4f91bf415c fix: schema check error reason pattern 2023-02-04 07:58:06 +01:00
Zaiming (Stone) Shi 65319567cc chore: no type check for 3rd party auth 2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi fbc52330f3 refactor: do not re-check config in post_config_update
now that the type is not lazy, it should have already been checked
when reaching the post_config_update step
2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi 6aaff6211f feat: use union member selector for authn config schema 2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi 15035f7eb0 refactor: remove lazy type for authentication
The idea of using lazy type for authentication was to make the
config check easy to extend at runtime. However the reality
is: it's overly complicated.

It's more likely that people will just continue to implementing
the auth hook callbacks instead of injecting config schema
at runtime.
2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi ccb0c7ebaa Merge remote-tracking branch 'origin/release-50' into 0202-merge-release-50-back-to-master 2023-02-03 16:12:11 +01:00
Zhongwen Deng d45a0cb2ee chore: remove <<>> when logging username 2023-02-03 16:54:18 +08:00
Zaiming (Stone) Shi 1e0fe270bc chore: bump emqx app vsn to 5.0.17 2023-02-02 20:32:09 +01:00
Zaiming (Stone) Shi 94768c9f44 Merge remote-tracking branch 'origin/master' into 0202-merge-release-50-back-to-master 2023-02-02 20:21:26 +01:00
Zaiming (Stone) Shi 5fe3a0b1b3
Merge pull request #9898 from zmstone/0202-upgrade-to-dashboard-e1.0.3
chore: upgrade dashboard to v1.1.7 for ce and e1.0.3 for ee
2023-02-02 17:41:52 +01:00
Zaiming (Stone) Shi 42462db8b8 chore: bump version to v5.0.16 and e5.0.0 2023-02-02 17:26:00 +01:00
Zaiming (Stone) Shi 3d6756cbe8 test: fix addrinuse errors in listeners API test SUITE 2023-02-02 16:37:56 +01:00
Zaiming (Stone) Shi a71d983ff8 test: fix assertMatch patter to make rebar3 fmt work 2023-02-02 16:37:45 +01:00
Zaiming (Stone) Shi 7ea140599a
Merge pull request #9894 from id/ci-always-run-static-checks
ci: always run static_checks
2023-02-02 16:33:19 +01:00
Zaiming (Stone) Shi feca4cc0a5
Merge pull request #9892 from zmstone/0202-docs-cosmetic
0202 docs cosmetic
2023-02-02 15:43:58 +01:00
Zaiming (Stone) Shi 85fb53dcf0 refactor: use compile switch to diff functions between ce and ee 2023-02-02 15:15:31 +01:00
Zaiming (Stone) Shi 0d7cd2f0c5
docs: Update apps/emqx_conf/i18n/emqx_conf_schema.conf
Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-02 14:20:07 +01:00
Zaiming (Stone) Shi 980ba2fd35
docs: Update apps/emqx/i18n/emqx_schema_i18n.conf
Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-02 14:19:37 +01:00
Zaiming (Stone) Shi 0e299642cb chore: function renamed but forgot to update xref excemption 2023-02-02 14:16:23 +01:00
Zaiming (Stone) Shi 58627b7958 chore(emqx_resource_manager): ignore unused return value for dialyzer 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi 9a9943d35d chore(emqx_mgmt_cache): ignore unused return value for dialyer 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi 1c748070c1 fix(emqx_connector_mqtt_worker): publish_async can only return ok # Please enter the commit message for your changes. Lines starting 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi c0d478bd41 fix(buffer_worker): type spec 2023-02-02 14:11:12 +01:00
zhongwencool ee852d8204
Merge pull request #9886 from zhongwencool/mongo-connection-default-async
fix: remove async mode from mongodb/redis/mysql/pgsql bridge
2023-02-02 21:08:01 +08:00
Zaiming (Stone) Shi d5c482b0b0 docs: remove timer unit from description
the user input has time unit. e.g. "5s" for 5 seconds etc.
2023-02-02 13:49:20 +01:00
Zaiming (Stone) Shi 5dbee76af4
Merge pull request #9890 from zmstone/0202-fix-kafka-status-check
0202 fix kafka status check
2023-02-02 13:48:44 +01:00
Zaiming (Stone) Shi 2b308af354
Merge pull request #9889 from keynslug/fix/EMQX-8857/force-clean-start-mqtt-bridge
fix(mqtt-bridge): stop respecting `clean_start` config parameter
2023-02-02 12:55:58 +01:00
Zaiming (Stone) Shi 9864587389 fix: send to buffer-supported connector even when disconnected 2023-02-02 12:04:17 +01:00
Zaiming (Stone) Shi 13ef30c46c
Merge pull request #9884 from savonarola/resource-fixes
fix(resources): fix resource lifecycle
2023-02-02 12:02:34 +01:00
Andrew Mayorov 0912f13c1f
fix(mqtt-bridge): stop respecting `clean_start` config parameter
We are ignoring the user configuration because there's currently no
reliable way to ensure proper session recovery according to the MQTT
spec.
2023-02-02 12:55:10 +03:00
Kjell Winblad 2cf193e2fd
Merge pull request #9839 from kjellwinblad/kjell/fix/Authorization_header_log_leak_webhook/EMQX-8791
fix: Authorization header leak in log entries for webhook
2023-02-02 10:38:42 +01:00
Zhongwen Deng 1c9035d24c test: remove async from redis ct 2023-02-02 17:37:18 +08:00
Zhongwen Deng 22cc1cc745 fix: make spell_check happy 2023-02-02 17:37:18 +08:00
Zhongwen Deng f8936013b7 chore: replace async with sync 2023-02-02 17:37:18 +08:00
Zhongwen Deng 22c3f50020 fix: add query_mode_sync_only for mysql pgsql redis mongodb bridge 2023-02-02 17:37:18 +08:00
Andrew Mayorov ca5c192f4b
Merge pull request #9882 from fwup/fix/no-mqtt-bridge-middleman
refactor(mqtt-worker): avoid unnecessary abstraction
2023-02-02 13:11:31 +04:00
zhongwencool e5bc787467
Merge pull request #9872 from zmstone/0131-improve-help-text
docs: improve docs for system monitoring config
2023-02-02 14:45:03 +08:00
Zhongwen Deng 1a90c1654c chore: bad typo 2023-02-02 11:43:04 +08:00
Zaiming (Stone) Shi 6fd30a7e81 docs: improve docs for system monitoring config 2023-02-01 20:21:08 +01:00
Ilya Averyanov 14f528cc86 fix(resources): fix resource lifecycle
* do not resume all buffer workers on successful healthcheck
* do not pass undefined state to resource healthcheck callback
2023-02-01 18:26:13 +02:00
Zaiming (Stone) Shi ec3031271b
Merge pull request #9883 from zmstone/0201-upgrade-enterprise-dashboard-to-e1.0.2
0201 upgrade enterprise dashboard to e1.0.2
2023-02-01 16:26:21 +01:00
Zaiming (Stone) Shi 32d18ec594
Merge pull request #9878 from zmstone/0201-upgrade-ehttpc-0.4.6
fix(ehttpc): fix crash when webhook bridge uses `hash` pool type.
2023-02-01 15:00:28 +01:00
Andrew Mayorov 5fd7f65a1f
test(bufworker): make testcase simpler to follow
The confusion was due to the fact that subsequent query was missing
`async_reply_fun` and thus, was not accumulating in the results.
2023-02-01 16:52:47 +03:00
Zaiming (Stone) Shi 38ef37d109 chore: bump release version to e5.0.0-rc.4 2023-02-01 14:52:08 +01:00
Andrew Mayorov 8a46cb974e
test(mqtt-bridge): test async bridge reconnects seamlessly 2023-02-01 16:28:16 +03:00
Andrew Mayorov 5ebceb20d2
test(mqtt-bridge): also test reconfiguration 2023-02-01 16:23:58 +03:00
Andrew Mayorov ad88938d34
refactor: reuse some parts of test code for brewity 2023-02-01 16:22:19 +03:00
Andrew Mayorov f0395be383
refactor(mqtt-worker): avoid unnecessary abstraction
So the code is easier to follow.
2023-02-01 14:46:26 +03:00
zhongwencool 13511d2782
Merge pull request #9856 from zhongwencool/alarm-percent-decimals
fix: /api/nodes is timeout when emqx in high load
2023-02-01 19:40:53 +08:00
Andrew Mayorov 979e9804be
Merge pull request #9879 from keynslug/fwup/fix/buffer-worker-testcase
test(bufworker): fix testcase flapping due to data races
2023-02-01 15:33:59 +04:00
Andrew Mayorov ff473e0f1b
test(bufworker): fix testcase flapping due to data races 2023-02-01 12:57:46 +03:00
Zaiming (Stone) Shi 96a18e7105 chore: upgrade to hocon 0.35.3 2023-02-01 10:52:01 +01:00
Zhongwen Deng 6382915036 chore: repalce fresh by refresh 2023-02-01 17:36:30 +08:00
Zaiming (Stone) Shi 9f6b6cedc6 fix(webhook): pick worker according to pool type 2023-02-01 09:53:25 +01:00
Ilya Averyanov 9d91ebe266
Merge pull request #9842 from savonarola/fix-redis-cluster-recover
fix: fix redis cluster resource recovery
2023-02-01 10:38:52 +02:00
Zaiming (Stone) Shi f4381d90ca refactor: http cliet should not need to retry HTTP requests for bridge 2023-02-01 09:30:30 +01:00
Zaiming (Stone) Shi 6ca816f7a4 docs: remove impl details from user doc 2023-02-01 09:30:10 +01:00
Zhongwen Deng 3c4d09a752 fix: get_memory_cache return {ok, Mem} | stale 2023-02-01 16:00:55 +08:00
Zhongwen Deng ced55719ef chore: only run t_sys_mem_check ct in linux 2023-02-01 14:47:41 +08:00
Zhongwen Deng 56b9238645 fix: only cache sys_memory in linux 2023-02-01 12:21:34 +08:00
Zhongwen Deng 95ef1e806c chore: don't crash when add_handler failed 2023-02-01 11:24:27 +08:00
Zhongwen Deng 9cbad5297c fix: primary logger level not update 2023-02-01 11:16:30 +08:00
Zhongwen Deng 71f00f2962 test: ctl suite failed 2023-02-01 10:07:22 +08:00
Zhongwen Deng 7a9f477267 feat: use emqx_mgmt_cache to cache sys_memory 2023-02-01 10:07:22 +08:00
Zhongwen Deng c2bdb9faa7 test: multiple_nodes case failed 2023-02-01 10:07:22 +08:00
Zhongwen Deng b6e6315b50 feat: change loads from string to float 2023-02-01 10:07:22 +08:00
Zhongwen Deng 0b19be074c feat: cache OTP_VERSION in persistent_term 2023-02-01 10:07:22 +08:00
Zhongwen Deng 6162f90610 fix: don't crash when OTP_VERSION file is missing 2023-02-01 10:07:22 +08:00
Zhongwen Deng 5783127c30 test: cpu_sup:load mock test 2023-02-01 10:07:22 +08:00
Zhongwen Deng 2d67bb3fb6 fix: /api/nodes is timeout if emqx in high load 2023-02-01 10:07:22 +08:00
Zaiming (Stone) Shi 7e8253e3af chore: bump version to e5.0.0-rc.3 2023-01-31 20:45:28 +01:00
Zaiming (Stone) Shi 78028a7fcf
Merge pull request #9876 from zmstone/0131-avoid-persisting-logging-configs-if-no-change
0131 avoid persisting logging configs if no change
2023-01-31 20:43:32 +01:00
Zaiming (Stone) Shi b3ad9e97d2
Merge pull request #9870 from keynslug/fix/mqtt-connection-loss-feedback
feat(mqtt-bridge): avoid middleman process
2023-01-31 19:12:18 +01:00
Zaiming (Stone) Shi ddfbc0a19f refactor: refresh logger config before starting listeners 2023-01-31 18:50:16 +01:00
Zaiming (Stone) Shi 605d9972e4 fix(config): avoid calling emqx:update_config/2 for logger refresh
Prior to this fix, whatever configured for loggers are persisted to
cluster-override.conf.
This may cause troulbe for users who changes boot mode.

For example if the node is once started in console mode,
the logging config is persisted, with console enabled, but file
disabled.
Then if the user decide to start in daemon mode, all the logs
will silently go to erlang.log.N and emqx.log.N will be empty.

After this fix, only changes really made into cluter-override.conf
will take effect.
2023-01-31 18:50:16 +01:00
Zaiming (Stone) Shi b0ac924ca9 refactor: less copy-paste 2023-01-31 18:40:55 +01:00
lafirest 3d59be6ab6
Merge pull request #9871 from lafirest/fix/authz_rules
feat(authz): allow the placeholder to be anywhere in the topic for authz rules
2023-02-01 00:12:06 +08:00
Andrew Mayorov c5a7cd5acd
fix(mqtt-bridge): drop unused configuration parameter 2023-01-31 18:30:42 +03:00
Andrew Mayorov c76311c9c3
fix(buffer): count inflight batches properly 2023-01-31 18:30:42 +03:00
firest 9f4c36ecbc chore: bump version && update changes 2023-01-31 23:30:22 +08:00
firest c034cbf6de feat(authz): allow the placeholder to be anywhere in the topic for authz rules 2023-01-31 23:30:12 +08:00
Andrew Mayorov d0c10b59aa
feat(mqtt-bridge): avoid middleman process
Instead, supervise `emqtt` client process directly.
2023-01-31 17:59:03 +03:00
Ilya Averyanov fce1e74c3d fix(connector): fix redis cluster resource recovery 2023-01-31 16:55:05 +02:00
Zaiming (Stone) Shi f6dafc20ea feat: export API to read cluster and local overrides 2023-01-31 14:20:34 +01:00
Stefan Strigler ade5316419
Merge pull request #9802 from sstrigler/EMQX-8469-enable-proxy-protocol-support-in-emqx-dashboard-schema
feat: support HAProxy protocol for dashboard API
2023-01-31 14:19:56 +01:00
Erik Timan 3786bb8086
Merge pull request #9824 from olcai/internal-error-api-get-topic
fix(emqx_management): handle multiple routes in topics/{topic} API
2023-01-31 13:50:42 +01:00
Andrew Mayorov 4d146c521b
fix(mqtt-bridge): ensure proper feedback on async forwards
So that buffer worker would notice a connection loss in time, and
recycle inflight messages subsequently.
2023-01-31 11:09:39 +03:00
Andrew Mayorov 35c429ef1d
refactor: drop a couple of unused macros / includes 2023-01-31 11:09:39 +03:00
Zaiming (Stone) Shi b3e486041b
Merge pull request #9853 from zmstone/0127-refactor-buffer-worker-no-need-to-keep-request-for-reply-callback
0127 refactor buffer worker no need to keep request for reply callback
2023-01-31 08:44:01 +01:00
Erik Timan 5b3a77e3c7 test(emqx_management): fix flaky route handling in get topic test
This reworks a test case to use a second slave node. This ensures that
an added route is permanently in the routing table. The old version
reverted the routing table quickly since the node name given wasn't a
real node.
2023-01-31 08:28:55 +01:00
Erik Timan 03cabf6b26 chore: bump app VSN 2023-01-31 08:27:54 +01:00
Erik Timan 33e011aff5 fix(emqx_management): handle multiple routes in topics/{topic} API
The topics/{topic} API endpoint would return 500 - Internal Error if a
topic had multiple routes. This is now fixed by returning a list of
routes.
2023-01-31 08:27:33 +01:00
Ilya Averyanov 1a1c3afa81
Merge pull request #9852 from savonarola/fix-redis-desc
Fix Redis configuration field description
2023-01-30 23:26:37 +02:00
Stefan Strigler 3ce6cbcd6a chore: bump vsn 2023-01-30 18:09:10 +01:00
Stefan Strigler 0b1483040a test: add actual test for haproxy 2023-01-30 18:07:06 +01:00
lafirest e288424924
Merge pull request #9864 from lafirest/fix/exclusive-unsub-5.0
fix: the exclusive topics aren't removed when the session has already been cleaned
2023-01-31 00:33:45 +08:00
Kjell Winblad 0c4134c423 test: add unit test case for redact function in http connector 2023-01-30 11:51:28 +01:00
Stefan Strigler 27881064dc fix: increase dropped.queue_full by number of messages 2023-01-30 11:37:35 +01:00
lafirest 3e6cc40525
Merge pull request #9843 from lafirest/fix/speed_up_test
fix: speed up banned test cases
2023-01-30 18:17:27 +08:00