Commit Graph

3051 Commits

Author SHA1 Message Date
Ivan Dyachkov 801c93dd46 chore: v5.1.5-build.3 2023-08-10 12:56:28 +02:00
Ivan Dyachkov 6ead09b28d chore: merge 'upstream/master' 2023-08-10 12:56:00 +02:00
firest 6dbddfb089 fix(limiter): simplify the code of the limiter
move runtime code out from the schema
2023-08-10 18:20:05 +08:00
zhongwencool aa798e531e
Merge pull request #11409 from zhongwencool/opentelemetry-metrics
feat: support opentelemetry metrics
2023-08-10 14:35:25 +08:00
zhongwencool 5a4dd3a5e5 fix: truncate large payload 2023-08-10 09:35:12 +08:00
zhongwencool d5fe891961 fix: don't be kill when send large payload when log is debug 2023-08-10 09:22:30 +08:00
Thales Macedo Garitezi bb8ce68c2e docs: Generate changelog for v5.1.5-patch.2 2023-08-09 16:38:28 -03:00
Thales Macedo Garitezi 95df958f10 docs: Generate changelog for v5.1.5-patch.1 2023-08-09 13:18:09 -03:00
Thales Macedo Garitezi cce0d144f9 Merge branch 'master' into release-v515-patch1-20230809 2023-08-09 13:08:43 -03:00
zhongwencool 3933227636 feat: support opentelemetry metrics 2023-08-09 20:23:40 +08:00
lafirest bef8eddad3
Merge pull request #11405 from lafirest/fix/kal_kialo
fix(calendar): make date parse error reason more sense
2023-08-09 14:42:08 +08:00
Thales Macedo Garitezi fdfdb105f0 docs: Generate changelog for v5.1.5 2023-08-08 10:33:34 -03:00
Thales Macedo Garitezi a1ffd0263d fix(cacerts): add `cacerts` back as deprecated to allow upgrading from older versions
See https://github.com/emqx/emqx/issues/11380
2023-08-08 10:13:13 -03:00
firest 9f38f5f26a fix(calendar): make date parse error reason more sense 2023-08-08 14:28:38 +08:00
lafirest 2b03436552
Merge pull request #11392 from lafirest/feat/ldap_authz
feat(ldap-authz): integrate the LDAP authorization
2023-08-07 11:12:05 +08:00
zhongwencool 665695a977
Merge pull request #11388 from SergeTupchiy/EMQX-10703-fix-replicant-crash-when-core-terminates-abnormally
fix: increase emqx_router_sup restart intensity
2023-08-06 14:40:32 +08:00
Serge Tupchii 2b7798608d fix(emqx_router_helper): don't cleanup down node on a replicant
The cleanup on a replicant node is redundant, as Mria would delegate this delete op
to a core node (via RPC), and the core node is expected to receive the same
`nodedown` message and process it.
2023-08-04 16:55:36 +03:00
SergeTupchiy 2044326734
Merge pull request #11390 from SergeTupchiy/EMQX-10661-improve-perf-high-latency-cluster-network
EMQX-10661 improve performance on high latency cluster network
2023-08-04 16:41:08 +03:00
firest b24a9d5343 chore(ldap-authz): update apps version && changes 2023-08-04 18:14:09 +08:00
Serge Tupchii f276ea9e91 fix: increase emqx_router_sup restart intensity
The goal is to tolerate occasional crashes that can happen under relatively normal conditions
and don't seem critical to shutdown the whole app (emqx).
For example, mria write/delete call delegated from a replicant to a core node may fail,
if the core node is being stopped / restarted / not ready.

Fixes: EMQX-10703, #11310
2023-08-04 10:49:59 +03:00
firest 7055eafb91 fix(ldap): fix license date and some minor problems 2023-08-04 11:01:13 +08:00
Serge Tupchii 466fe7e009 perf: add broker_pool_size, generic_pool_size and channel_cleanup_batch_size config options
Tuning these options can improve performance if cluster interconnect network latency is high.

Fixes: EMQX-10661
2023-08-03 19:44:00 +03:00
Andrew Mayorov 39bb9d5136
chore: bump ekka to 0.15.10
Which sports mria 0.6.0 with dirty activities support.
2023-08-03 11:38:53 +04:00
Ivan Dyachkov 0f161a81c3 chore: v5.1.4 2023-08-01 09:20:46 +02:00
Thales Macedo Garitezi 7687770821 fix(tls): remove `cacerts` config for now
Fixes https://github.com/emqx/emqx/issues/11370

Related: https://github.com/emqx/emqx/pull/11371
2023-07-31 10:44:05 -03:00
Thales Macedo Garitezi a3be1529c6 fix(schema): ensure enterprise module is correctly loaded
Fixes https://emqx.atlassian.net/browse/EMQX-10654

Despite loading the application in `nodetool`, we need to invoke `:module_info()` to force
the module to be actually loaded, especially when it's called in `call_hocon` to generate
the initial configurations.  Otherwise, it'll fail to list all the bridge schemas.
2023-07-28 16:27:43 -03:00
Thales Macedo Garitezi 2ee1aa6d60
Merge pull request #11347 from thalesmg/fix-ocsp-path-encoding-20230725
fix(ocsp): URL encode request path
2023-07-28 09:01:39 -03:00
Ivan Dyachkov d51b5b739b chore: v5.1.3 2023-07-27 17:00:20 +02:00
Ivan Dyachkov 5ac01c9b85
Merge pull request #11360 from id/0727-sync-release-51-to-master 2023-07-27 16:58:09 +02:00
Thales Macedo Garitezi ee3d002fca
Merge pull request #11329 from thalesmg/aeh-bridge-20230720
feat: add Azure Event Hub Producer bridge
2023-07-27 11:18:40 -03:00
Ivan Dyachkov cbfca8c043 chore: merge master into release-51 2023-07-27 15:19:57 +02:00
Ivan Dyachkov 50a0900d92 chore: e5.1.1 2023-07-27 12:18:03 +02:00
Thales Macedo Garitezi d3d52695d5 feat(tls): automatically add `cacerts` to client opts
`public_key:cacerts_get/0` was introduced in OTP 25 and allows us to
load the system trusted CA certificates.

https://www.erlang.org/doc/man/public_key.html#cacerts_get-0
2023-07-25 17:43:33 -03:00
Thales Macedo Garitezi d0deaca00d fix(ocsp): URL encode request path
Fixes https://emqx.atlassian.net/browse/EMQX-10624
2023-07-25 17:39:38 -03:00
Serge Tupchii 77184c00ff chore(ekka): Bump version to 0.15.9 2023-07-25 19:29:18 +03:00
Ivan Dyachkov f9d3d3325b chore: e5.1.1-rc.1 2023-07-25 16:34:25 +02:00
Ilya Averyanov deaac9bd73 fix(authz): correctly identify qos of subscribe actions 2023-07-25 15:20:52 +03:00
firest 8439ce0e84 chore: update app version && reboot_lists 2023-07-24 19:08:41 +08:00
Andrew Mayorov 81793c31fc
Merge pull request #11315 from keynslug/ft/EMQX-9593/peek-mqueue-info
refactor(session): allow peeking at mqueue less intrusively
2023-07-24 09:57:05 +02:00
Ivan Dyachkov 29432009db chore: 5.1.1-alpha.4 2023-07-21 18:42:14 +02:00
ieQu1 485e495b98 chore(ekka): Bump version to 0.15.8 (mria 0.5.10) 2023-07-21 18:31:02 +02:00
Ivan Dyachkov c2c3da073f chore: v5.1.2 2023-07-21 13:26:29 +02:00
Ivan Dyachkov 243b8f5b67 chore: merge 'upstream/master' into v5.1.2 2023-07-21 13:25:46 +02:00
zhongwencool 520ae0e9d3 chore: remove zone desc from document 2023-07-21 15:51:21 +08:00
zhongwencool d3e4fda21d fix: hocon_pp quote string for special char 2023-07-21 12:31:22 +08:00
Andrew Mayorov e1e4c64a30
refactor(session): allow peeking at mqueue less intrusively 2023-07-20 21:21:30 +02:00
Andrew Mayorov f2a32e8ed2
Merge pull request #11304 from keynslug/ft/EMQX-9593/ds-session-connect
refactor(session): minimize `emqx_session` interface surface
2023-07-20 14:33:21 +02:00
SergeTupchiy 8480a26b66
Merge pull request #11065 from SergeTupchiy/EMQX-9992-fix-async-cleanup-error-logs
fix: avoid logging unnecessary errors in async cleanup functions
2023-07-20 12:20:57 +03:00
Serge Tupchii ff4119e8b3 chore(emqx_config): fix typo in the log message 2023-07-19 16:58:21 +03:00
ieQu1 57e39f42c5
Merge pull request #11294 from ieQu1/mria-fixes
Mria fixes
2023-07-19 15:10:42 +02:00
Andrew Mayorov dd31487b4a
refactor(session): drop `is_session/1` helper as useless
This is a part of effort to minimize `emqx_session` module
interface to simplify adding alternative session implementations.
2023-07-19 14:55:26 +02:00
Andrew Mayorov 0cd2351134
refactor(session): hide `no_local` logic behind enqueue / deliver
This is a part of effort to minimize `emqx_session` module
interface to simplify adding alternative session implementations.
2023-07-19 14:55:26 +02:00
Ivan Dyachkov f29a9ed9d5
Merge pull request #11301 from id/0719-sync-release-51-to-master 2023-07-19 14:21:26 +02:00
Andrew Mayorov 92166f287a
Merge pull request #11293 from keynslug/fix/EMQX-9591/phase-out-persistence
chore(ps): phase out existing session persistence mechanism
2023-07-19 10:36:46 +02:00
Ivan Dyachkov b7f0f7b32d chore: merge master into release-51 2023-07-19 09:28:05 +02:00
zhongwencool 402f75592e chore: move rpc call from emqx_management_proto to emqx_conf_proto 2023-07-19 11:05:14 +08:00
zhongwencool 8a0f7bfd99 chore: update version bpapi.versions 2023-07-19 11:05:14 +08:00
ieQu1 ce76d4ab04 chore(ekka): Bump version to 0.15.7 (mria 0.5.9) 2023-07-18 21:45:00 +02:00
Ivan Dyachkov de4cc492d4 chore: e5.1.1-alpha.2 2023-07-18 18:55:24 +02:00
Andrew Mayorov 0a00c39282
chore(ps): phase out existing session persistence mechanism
* It's not really working currently in a cluster setup.
* It's not really working well.
* It would be much easier to integrate `emqx_durable_storage` w/o
  the need to support existing solution.
2023-07-18 15:46:04 +02:00
Thales Macedo Garitezi 4a889c0994 chore: bump ekka -> 0.15.6
https://github.com/emqx/erlang-rocksdb/releases/tag/1.8.0-emqx-1
2023-07-18 09:00:29 -03:00
Thales Macedo Garitezi 1fdcfba629 fix(shared_sub): restore support for `$queue/` shared subscription
Partially addresses: https://emqx.atlassian.net/browse/EMQX-4589

There's still a problem with the handling of shared groups that is
_not particular to the `$queue/` syntax_ and preexistent that the same
client cannot subscribe to _different groups_ that have the same
_topic filter_.
2023-07-17 16:41:24 -03:00
zhongwencool f853f3d3bd
Merge pull request #11271 from zhongwencool/fix-precent-check
fix: keep percentage type's range is 0%~100%
2023-07-14 22:07:41 +08:00
Serge Tupchii 950d5edc41 fix: avoid logging unnecessary errors in async cleanup functions
Cleanup functions that access ETS tables may fail with `badarg` error during EMQX shutdown.
They are called asynchronously by `emqx_pool` workers and accessed ETS tables
may be already destroyed as their owners are shut down.
This fix catches ETS `badarg` errors before they can be caught and logged by `emqx_pool`.

Fixes: EMQX-9992
2023-07-14 14:22:43 +03:00
firest fea73cf17d fix(log): fix the `pubrel` typo in the log 2023-07-14 17:19:50 +08:00
zhongwencool 11ee6abf02 fix: keep precent range is 0%~100% 2023-07-14 16:48:50 +08:00
Thales Macedo Garitezi 06010f7ca9 Merge remote-tracking branch 'origin/release-51' into sync-r51-20230712 2023-07-12 16:47:45 -03:00
SergeTupchiy fce3060634
Merge pull request #11251 from SergeTupchiy/EMQX-10392-cluster-topology-api
Add cluster topology HTTP API endpoint
2023-07-11 17:45:00 +03:00
Serge Tupchii 19de10be7c feat(emqx_management): add /cluster/topology HTTP API endpoint
The endpoint shows Mria RLOG cluster topology info:
connections between core and replicant nodes.

Closes: EMQX-10392
2023-07-11 16:25:36 +03:00
firest 211486a946 fix(ws): fix MQTT packets may be reversed in the WS
Fix when a WS packet contains many MQTT packets, the order of MQTT packets will be reversed
2023-07-11 18:40:06 +08:00
Andrew Mayorov 21fbf79c29
Merge pull request #11191 from keynslug/ft/EMQX-9591/msg-persistence-minimal
feat(ds): hardwire emqx_durable_storage message persistence
2023-07-10 21:05:04 +02:00
Andrew Mayorov d91ab7dec9
test(ds): make `emqx_persistent_messages_SUITE` setup cleaner
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-07-10 18:12:16 +02:00
Ivan Dyachkov fb96c1e20d chore: bump to e5.1.1-alpha.1 2023-07-10 17:25:07 +02:00
Ilya Averyanov 58a83739b8
Merge pull request #10986 from savonarola/ft/EMQX-9523/config-api
feat(ft-api): provide configuration API
2023-07-10 15:32:30 +03:00
Ilya Averyanov 12e237c36d
Merge pull request #11132 from savonarola/0531-authz-qos-retain
feat(authz): use extensible map format for actions in authz rules
2023-07-10 15:32:02 +03:00
Andrew Mayorov 576eab9717
Merge pull request #11193 from keynslug/fix/emqx-ft-test-flaps
fix(ft-test): use new cth tooling in emqx_ft_storage_fs_SUITE
2023-07-10 11:55:16 +02:00
JianBo He f144bb0571
Merge pull request #11226 from zhongwencool/listener-conf-enabled-to-enable
fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable
2023-07-10 17:15:24 +08:00
zhongwencool 802a50601a chore: add comment for init_loader_done 2023-07-10 12:03:36 +08:00
zhongwencool c4ba558ee3 fix: unset config_loader in emqx's env when stop emqx app 2023-07-10 11:06:08 +08:00
zhongwencool d661b10355 fix: emqx CI failed 2023-07-10 10:34:36 +08:00
zhongwencool c0ee47dc08 chore: add changelog for 11226 pr 2023-07-10 10:34:36 +08:00
zhongwencool d781346efc fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable 2023-07-10 10:34:36 +08:00
Andrew Mayorov 030beb3e6a
fix(ds): drop debug printouts in test suite 2023-07-07 21:56:20 +02:00
Andrew Mayorov daf4e86da1
feat(ds): add `ensure_shard/2` shortcut
And use it in `emqx_persistent_session_ds:init()` backend initialization
function.
2023-07-07 21:56:20 +02:00
Ilya Averyanov 77895f2555 feat(authz): fix typos and style for Retain & QoS authz feature
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-07-07 19:40:34 +03:00
Ilya Averyanov 7de26a1776 feat(authz): use extensible map format for actions in authz rules
* support authorization on retain, qos fields
* refactored authz tests heavily
2023-07-07 19:40:34 +03:00
Ilya Averyanov e0353ab750 fix(ft-api): update FT API tests to use emqx_cth_suite 2023-07-07 19:15:42 +03:00
Andrew Mayorov 97b6c43062
fix(ft-test): use new cth tooling in `emqx_ft_storage_fs_SUITE`
In attempt to battle test flakiness.
2023-07-07 14:00:33 +02:00
Andrew Mayorov 7e76914599
feat(ds): hardwire emqx_durable_storage message persistence
Only message persistence is currently implemented, irrespectively of
whether there are persistent sessions around or not.
2023-07-07 13:46:12 +02:00
zhongwencool 352d818e3b chore: update hocon to 0.39.13 2023-07-07 16:39:26 +08:00
zhongwencool 221f6eba06 fix: bad tnx-id when rejoin cluster 2023-07-07 12:25:34 +08:00
firest a1d2b2ca51 chore: bump app versions 2023-07-07 11:17:54 +08:00
Stefan Strigler 9bcabe0a9e
Merge pull request #11184 from sstrigler/EMQX-10270-e-5-1-0-missing-value-range-validation-for-the-max-packet-size-of-mqtt-settings
fix: limit mqtt max_packet_size to 256MB
2023-07-06 10:20:24 +02:00
Ilya Averyanov 9af0030538 chore: bump to v5.1.1 2023-07-05 18:43:45 +03:00
Stefan Strigler 263890be47 fix: limit mqtt max_packet_size to 256MB 2023-07-04 16:03:11 +02:00
Andrew Mayorov 8cc7022760
Merge pull request #11120 from ft/EMQX-10329/tooling
test: add tooling to make testruns more isolated and predictable
2023-07-04 14:36:39 +02:00
zhongwencool f2c5f4932c
Merge pull request #11148 from lafirest/fix/mcall_error_report
fix(emqx_conf): don't sync the MFA to a leaved node
2023-07-04 19:56:52 +08:00
Andrew Mayorov 9ed75085b7
chore: drop dead commented out code 2023-07-04 13:20:04 +02:00
Andrew Mayorov c7ba300efc
fix(cthsuite): use correct type for "smth serializable as HOCON" 2023-07-04 13:20:04 +02:00
Andrew Mayorov d568d43fe4
test(bridge): switch `emqx_bridge_api_SUITE` to use new tooling 2023-07-04 13:20:03 +02:00
Andrew Mayorov 2c50e61cee
fix(cth): reduce amount of logging + streamline logging
Also make sure that code running on `emqx_cth_cluster` nodes don't
crash when using `ct:pal` and friends.
2023-07-04 13:20:03 +02:00
Andrew Mayorov 9bda4192e5
fix(cthsuite): disable default `acl.conf` authz source
So that users of `emqx_cth_suite` won't have to deal with providing
correct filepath each time they start `emqx_conf` and/or `emqx_authz`
up.
2023-07-04 13:20:03 +02:00
Andrew Mayorov 24a16ea247
fix(cthsuite): bump to hocon 0.39.11 with atom prettyprint fix 2023-07-04 13:20:03 +02:00
Andrew Mayorov 6a7254f55e
fix(cthcluster): match correct driver in `stop_node/2`
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2023-07-04 13:20:02 +02:00
Andrew Mayorov fe0b8cfbaf
test(conf): respect `init_config_load_done` in emqx_conf itself
This should make task of configuring various applications (emqx_conf
/ emqx) during testsuites setup much simpler.
2023-07-04 13:20:02 +02:00
Andrew Mayorov f76f3b77d7
test: add tooling to make testruns more isolated and predictable
Also showcase their usage in some of the existing testsuites.
2023-07-04 12:53:39 +02:00
zhongwencool d509c47344 feat: add conf load api 2023-07-04 14:17:11 +08:00
Andrew Mayorov 0f25df3aa8
perf(router): employ `maps:foreach/2` instead 2023-07-03 15:01:47 +02:00
Andrew Mayorov 57da71eb1d
perf(broker): avoid usorting aggregated routes in a loop 2023-07-03 15:01:33 +02:00
zhongwencool e26c30ff0c feat: refactor dashboard https ssl_options 2023-06-29 10:49:39 +08:00
firest 8e68af14ef chore: bump ekka version 2023-06-29 01:45:47 +00:00
Andrew Mayorov 30d78aaac2
Merge pull request #11152 from keynslug/test/ft-prop-aggressive
test(ft): make proptests even less aggressive
2023-06-27 17:44:54 +02:00
Andrew Mayorov ff327609db
test(ft): generate segments manually in proptests
In order to avoid spamming proptest logs with HUGE type instances,
and make proptests themselves simpler. Proper now should refuse
shrinking altogether, instances that cause prop function to timeout
do not shrink well for some reason, making proptests stuck.
2023-06-27 17:13:23 +02:00
Zaiming (Stone) Shi 9b6ed09513 refactor: call hocon_tconf:check_plain/4 directly for example conf check
the attemp to re-use emqx_hocon:check failed because
hocon_tconf:check_plain/3 and check_plain/4 behaves slightly different
for unknown root fields.

To keep the old logic unchanged, for example config checks, we call
check_plain/4 directly
2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi a069b351fd test: add test script to verify config example files 2023-06-27 14:23:18 +02:00
Serge Tupchii 116675e86f chore: bump ekka to 0.15.4 2023-06-27 13:24:49 +03:00
lafirest f2e3446e29
Merge pull request #11135 from lafirest/pref/calendar_timeoffset
perf(rule_engine): improve the time offset parser
2023-06-26 18:54:00 +08:00
lafirest 850d642ec2
Merge pull request #11108 from ieQu1/emqx-ds
Durable storage
2023-06-25 11:36:34 +08:00
firest c8c2793944 perf(rule_engine): improve the time offset parser 2023-06-24 13:05:27 +00:00
Thales Macedo Garitezi 7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
Zaiming (Stone) Shi c58a98954b Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-22 11:05:51 +02:00
ieQu1 efed7cdc2f chore(ds): behavior -> behaviour 2023-06-22 10:20:18 +02:00
Andrew Mayorov 86d787eced
chore: bump hocon to 0.39.10
Which comes with a fix for slightly more user-friendly validation error
messages.
2023-06-21 21:25:43 +02:00
Zaiming (Stone) Shi 7cf8a6c892 chore: bump app vsns 2023-06-21 16:36:51 +02:00
Zaiming (Stone) Shi 5fa87091c4 Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-21 16:08:41 +02:00
Zaiming (Stone) Shi dddccfdc5c chore: bump to v5.1.0 official cut 2023-06-20 17:09:17 +02:00
Zaiming (Stone) Shi 728656d4f6 chore: bump to e5.1.0 official cut 2023-06-20 17:08:18 +02:00
Zaiming (Stone) Shi f95787d229 test: make possible to test core/replica locally 2023-06-20 12:42:00 +02:00
zhongwencool 856de78698 chore: remove the is_list guard 2023-06-20 08:39:23 +08:00
zhongwencool 093cdab838 chore: to_integer to make sure integer is converted 2023-06-20 08:39:23 +08:00
zhongwencool 07172e42f0 test: integer CI check failed 2023-06-20 08:39:23 +08:00
zhongwencool a4be9b8281 chore: bump hocon to 0.39.9 2023-06-20 08:39:23 +08:00
zhongwencool 45963b6a95 fix: ip_port schema type crash 2023-06-20 08:39:23 +08:00
Zaiming (Stone) Shi ead511a128 chore: bump to 5.1.0-rc.1 2023-06-19 23:19:27 +02:00
ieQu1 86b78981c4
Merge pull request #11092 from ieQu1/mria-0.5.5
chore(mria): Bump version to 0.5.5
2023-06-19 23:13:36 +02:00
Thales Macedo Garitezi 01d758e4c0 fix(bpapi): fix `query_mode` type(spec)
The previous spec was wrong due to a mismatch between type declaration and usage in specs:
it would never be `dynamic`, and it was lacking the `simple_async`, `simple_sync` and
`no_queries` constructors.
2023-06-19 16:04:12 -03:00
ieQu1 f82c56ae16 chore(mria): Bump version to 0.5.5 2023-06-19 19:28:17 +02:00
JianBo He cc939d2bda
Merge pull request #11087 from zhongwencool/bump-v5.1.0.alpha.8
chore: bump to v5.1.0.alpha.8
2023-06-19 23:00:53 +08:00
JianBo He b0a6c18917 chore: bump vsn to 5.1.0-alpha.8 2023-06-19 22:59:05 +08:00
zhongwencool 1f1d61cef6
Merge pull request #11084 from zhongwencool/hide-plugins-from-cli
chore: hide plugins from conf load cli
2023-06-19 19:11:52 +08:00
zhongwencool 64612cab14 chore: bump to v5.1.0.alpha.8 2023-06-19 18:22:51 +08:00
zhongwencool a578d6fa1d chore: release e5.1.0-alpha.8 2023-06-19 17:19:03 +08:00
zhongwencool 803cd6c812
Merge pull request #11082 from emqx/release-51
sync release-51 to master
2023-06-19 15:12:37 +08:00
JimMoen 30ac6c4e48
Merge pull request #11074 from JimMoen/fix-share-sub-nl-protocol-error
fix: shared-sub with nl sub-option should cause protocol error
2023-06-19 10:39:08 +08:00
Andrew Mayorov 13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
2023-06-17 00:54:24 +02:00
JimMoen 178711f742
fix: shared-sub with nl sub-option should cause protocol error 2023-06-16 18:19:23 +08:00
Serge Tupchii b49b80d36c fix(emqx_schema): use non negative integer type for 'depth' SSL option
Closes: EMQX-10276
2023-06-16 10:59:39 +08:00
zhongwencool 9b56f116f4
Merge pull request #11062 from zhongwencool/log-file-rename 2023-06-15 20:51:52 +08:00
zhongwencool 3406a5d916
Merge pull request #11063 from zhongwencool/bump-to-alpha.7
chore: bump to e5.1.0-alpha.7
2023-06-15 19:22:37 +08:00
zhongwencool 9060a2dc05 chore: bump to e5.1.0-alpha.7 2023-06-15 19:05:13 +08:00
zhongwencool e6c2da8d19 feat: rename log.file.to to log.file.path 2023-06-15 18:24:21 +08:00
zhongwencool 84a5d0c3c1 fix: delete a default listener then recreate it, it's disabled 2023-06-15 17:19:11 +08:00
zhongwencool 0adbb0deec feat: try to save config when post_config_update failed not in init mfa status 2023-06-15 17:19:11 +08:00
zhongwencool c160301dbe fix: don't log enoent error 2023-06-15 17:06:09 +08:00
zhongwencool 7dade3a52c fix: tls_certfile_gc notice log don't print abspath 2023-06-15 17:06:09 +08:00
Serge Tupchii 64bbe21209 fix(emqx_schema): use non negative integer type for 'depth' SSL option
Closes: EMQX-10276
2023-06-15 11:22:58 +03:00
zhongwencool e42cc58694
Merge pull request #11050 from emqx/release-51
chore: sync release-51 back to master
2023-06-15 09:45:31 +08:00
Andrew Mayorov acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
2023-06-15 00:50:15 +03:00
zhongwencool 41004e86da
Merge pull request #10959 from qzhuyan/dev/william/remove-depre-quic-config
chore: remove deprecated QUIC listener config
2023-06-14 23:01:00 +08:00
Serge Tupchii 7882755114 chore: bump cowboy to 2.9.2
cowboy 2.9.2 uses ranch 1.8.1-emqx that fixes incompatible TLS options
2023-06-14 17:35:17 +03:00
Serge Tupchii 4425942030 fix(emqx_schema): move tls version gap validation to emqx_schema 2023-06-14 17:29:56 +03:00
Ivan Dyachkov 6d88cd7a20 fix(tls): fix incompatible tls options and issue with tls version gap 2023-06-14 17:29:56 +03:00
zhongwencool 267053cc35
Merge pull request #11045 from zhongwencool/hide-zones-authn-in-listeners
feat: hide zone/authn in listeners and remove listeners's authn api
2023-06-14 20:32:27 +08:00
zhongwencool 7668753f50 feat: hide zone/authn in listeners and remove /listeners/:id/authentication api 2023-06-14 15:17:12 +08:00
JianBo He 8ce0380704 chore: bump vsn to e5.1.0-alpha.6 2023-06-14 12:49:02 +08:00
zhongwencool 091c49277f fix: api_listener crash when setting max_connnections as string 2023-06-14 11:26:41 +08:00
zhongwencool 7f0f40cb58
Merge pull request #11034 from zhongwencool/mqtt-conf
feat:  move shared_subscription_strategy from broker to mqtt
2023-06-14 10:00:38 +08:00
Thales Macedo Garitezi b609792a90 fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10278

Since Pulsar client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 11:44:45 -03:00
zhongwencool 946de4a303 chore: dialyzer warning 2023-06-13 20:11:49 +08:00
zhongwencool 50041fdddf feat: hide broker rootkey and move broker.shared_subscription_strategy to mqtt.shared_subscription_strategy 2023-06-13 19:35:52 +08:00
William Yang 8fa743fa22 chore: mark deprecated QUIC listener config for 5.1 2023-06-13 06:00:57 +02:00
Thales Macedo Garitezi 5ef7102cba fix(listener_mgmt_api): use union member selector fn for better error messages (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10247
2023-06-12 16:48:13 -03:00
zhongwencool 58dc042279 chore: bump to e/v5.1.0-alpha.5 2023-06-13 00:06:54 +08:00
zhongwencool 0fc10ad239
Merge pull request #10984 from zhongwencool/reload-cli
feat: support reload cli
2023-06-12 17:21:29 +08:00
Zaiming (Stone) Shi 12b73ba1c0 Merge remote-tracking branch 'origin/master' into release-51 2023-06-12 10:35:47 +02:00
firest 947d99218b chore: upgrade emqtt to avoid sensitive data leakage in the debug log 2023-06-12 16:32:24 +08:00
zhongwencool 1550158ddd fix: update emqx_flapping process's state when emqx:update_config([flapping_detect],Conf) 2023-06-11 18:23:15 +08:00
Zaiming (Stone) Shi 654a9dcab5
Merge pull request #11006 from zmstone/0610-fix-quic-default-certs-path
fix(quic): environment variables as cert file prefix for quic listener
2023-06-11 11:52:56 +02:00
zhongwencool c8cf8d1def
Merge pull request #11009 from zhongwencool/persistent_session_store-default-value
chore: correct persistent_session_store's default value type to boolean
2023-06-11 17:26:58 +08:00
zhongwencool 072f6fac8d chore: update hocon to 0.39.8 2023-06-11 09:11:17 +08:00
zhongwencool 346b363cb5 test: config SUITE failed 2023-06-11 09:11:17 +08:00
zhongwencool 505d677685 chore: move reload_local_etc_config from emqx_conf to emqx_conf_cli 2023-06-11 09:11:17 +08:00
zhongwencool 6c6ff4ea1f chore: rename {mod} to module 2023-06-11 09:11:17 +08:00
zhongwencool cf9a207743 feat: check all config schema before loading 2023-06-11 09:11:17 +08:00
zhongwencool 21ffd958b4 feat: support authn merge_authenticators 2023-06-11 09:11:17 +08:00
zhongwencool 994db58cbf fix: load authz's default sources 2023-06-11 09:11:17 +08:00
zhongwencool c819ac27f4 feat: add conf reload cli 2023-06-11 09:11:17 +08:00
zhongwencool 599eb9fef5 chore: correct persistent_session_store's default value type to boolean 2023-06-10 19:36:05 +08:00
Zaiming (Stone) Shi 97850de524 Merge remote-tracking branch 'origin/release-51' into 0610-merge-release-51-to-master 2023-06-10 12:23:55 +02:00
Zaiming (Stone) Shi 9f135d1f2b fix(quic): environment variables as cert file prefix for quic listener 2023-06-10 12:23:20 +02:00
Serge Tupchii 87b57112df refactor: move tcp keepalive options helper to emqx_utils 2023-06-09 21:43:50 +03:00
Andrew Mayorov b930f4cc73
Merge pull request #10987 from keynslug/fix/EMQX-9257/sep-placeholder
refactor: tear `emqx_plugin_libs` application apart
2023-06-09 17:02:14 +02:00
Thales Macedo Garitezi 20a7ce64cd
Merge pull request #10919 from thalesmg/test-flakiness-20230601-a
test(pulsar_producer): attempt to fix flaky test
2023-06-09 09:30:30 -03:00
Andrew Mayorov 7d0abb6146
feat(emqx): add `emqx_topic:match_any/2` utility 2023-06-09 14:44:37 +03:00
SergeTupchiy e61b2100a3
Merge pull request #10676 from SergeTupchiy/EMQX-9203-config-backup
feat: implement configuration and user data export/import
2023-06-09 14:18:39 +03:00
Serge Tupchii e4d09d4ad4 feat: implement configuration and user data export/import CLI
Closes: EMQX-9203
2023-06-09 14:11:47 +03:00
Zaiming (Stone) Shi ee697831f6
Merge pull request #10926 from zmstone/0602-add-enable-as-an-alias-for-enabled
0602 add enable as an alias for enabled
2023-06-09 08:44:20 +02:00
Zaiming (Stone) Shi af5c6720de Merge remote-tracking branch 'origin/master' into release-51 2023-06-09 08:37:29 +02:00
Zaiming (Stone) Shi 47f826f99a chore: fix typo 2023-06-09 09:31:32 +08:00
Serge Tupchii 53ec8326b0 fix(emqx_listeners): convert authentication certs 2023-06-08 20:44:18 +03:00
Serge Tupchii 33fa053b9b test(emqx_common_test_helpers): keep EMQX ENV vars on a child node for its lifetime 2023-06-08 20:42:51 +03:00
Serge Tupchii 5f526d548a style: use double percent (%%) comments 2023-06-08 20:42:51 +03:00
Kjell Winblad 273dedf7a6
Merge pull request #10911 from kjellwinblad/kjell/bridge/atom_length_too_long/EMQX-9609
fix: friendly error message when creating bridges with too long names
2023-06-08 15:46:42 +02:00
Ivan Dyachkov abbba71191 fix(tls): issue when ssl listner is configured to use tls v1.3 only
clients could not connect due to incompatible tls options if ssl listner is configured to use tls v1.3 only
2023-06-08 15:13:22 +02:00
Zaiming (Stone) Shi b481994e74 chore: bump to version 5.1.0-alpha.4 2023-06-08 12:30:35 +02:00
Zaiming (Stone) Shi c1cf2365c2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-08 12:30:02 +02:00
Zaiming (Stone) Shi 641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51
0607 merge master to release 51
2023-06-08 09:22:21 +02:00
Zaiming (Stone) Shi 641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf
feat: don't hide enable in flapping detect conf
2023-06-07 23:04:56 +02:00
Zaiming (Stone) Shi ccd2589ff2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-07 21:43:29 +02:00
Andrew Mayorov 39b0d41527
fix(bpapi): ignore dropped BPAPI module
It doesn't really breaks interoperability.
2023-06-07 21:33:47 +03:00
Zaiming (Stone) Shi aecea50564
Merge pull request #10902 from zmstone/0529-donot-copy-cluster-conf-from-newer-version
0529 donot copy cluster conf from newer version
2023-06-07 20:32:11 +02:00
Zaiming (Stone) Shi ed1ad4e684 chore(emqx_schema): add 'eanble' as an alias to 'enabled' fields 2023-06-07 20:19:48 +02:00
Thales Macedo Garitezi 105c26afb5
Merge pull request #10964 from thalesmg/fix-hocon-compact-errors-v50
fix(schema): avoid `function_clause` error when compacting errors
2023-06-07 14:00:24 -03:00
Andrew Mayorov 79d430cf03
refactor(pluglib): move `emqx_plugin_libs_proto_v1` into emqx app
And rename it to `emqx_metrics_proto_v1` in the process.
2023-06-07 18:30:12 +03:00
Zaiming (Stone) Shi 622ac4d195
fix(emqx_schema): binary for string default value 2023-06-07 17:14:10 +02:00
JimMoen 4ea3483083
perf(topic): use tail recursive to joining a topic 2023-06-07 22:59:53 +08:00
JimMoen b0e18f5e75
fix(shared-sub): validate share topic error send DISCONNECT 2023-06-07 22:59:52 +08:00
JimMoen 070e410c69
refactor: variable and func rename 2023-06-07 22:59:52 +08:00
JimMoen ea81a924f1
refactor: move topic() types def in `emqx_types.erl` 2023-06-07 22:59:52 +08:00
JimMoen 65483e972e
refactor: use unified table define 2023-06-07 22:51:44 +08:00
zhongwencool 70e800f8a4 chore: delete unuse default value and make flapping_detect middle improtance 2023-06-07 22:50:11 +08:00
zhongwencool 5659cc2b47 feat: add converter to flapping_detect 2023-06-07 21:54:32 +08:00
zhongwencool 1d33b7dbd8 chore: make static check happy 2023-06-07 21:54:32 +08:00
zhongwencool 535870386a fix: zones update api failed 2023-06-07 21:54:32 +08:00
zhongwencool d58506a7c3 feat: always update default zones 2023-06-07 21:54:32 +08:00
zhongwencool 48381d4c86 feat: refactor flapping detect conf 2023-06-07 21:54:32 +08:00
Thales Macedo Garitezi cc8631223e fix(schema): avoid `function_clause` error when compacting errors
Fixes https://emqx.atlassian.net/browse/EMQX-10168

The bridge probe API displayed the typecheck errors for the new
timeout duration types correctly, but when an user tried to create the
bridge anyway a `function_clause` error was raised when trying to
compact hocon errors:

```
09:47:19.045 [warning] [exception: :error, path: '/bridges', reason: {:case_clause, {:error, {:config_update_crashed, :function_clause}}}, stacktrace: [{:emqx_bridge_api, :create_or_update_bridge, 4, [file: '/home/thales/dev/emqx/emqx/apps/emqx_bridge/src/emqx_bridge_api.erl', line: 602]}, {:minirest_handler, :apply_callback, 3, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 111]}, {:minirest_handler, :handle, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 44]}, {:minirest_handler, :init, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 27]}, {:cowboy_handler, :execute, 2, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_handler.erl', line: 41]}, {:cowboy_stream_h, :execute, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 318]}, {:cowboy_stream_h, :request_process, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 302]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]]
```

This fixes the issue so that both APIs return more friendly error messages.
2023-06-07 10:34:58 -03:00
Paulo Zulato a4407764f3 feat(connection): configurable TCP keepalive
Fixes https://emqx.atlassian.net/browse/EMQX-9852
2023-06-07 10:23:38 -03:00
Andrew Mayorov c05f462d0a
fix(tlslib): bring back deterministic filenames for managed certs 2023-06-07 13:32:35 +03:00
William Yang c1be86c5ec
Merge pull request #10931 from qzhuyan/perf/william/save-times-of-maps-build
Perf/william/save times of maps build
2023-06-07 10:32:45 +02:00
Zaiming (Stone) Shi cb3e787cb1 Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-07 09:14:04 +02:00
Serge Tupchii bf2b5a9f24 fix(emqx_listeners): fix listener authentication create/update
Listener authentication is a list of authenticators, so each of them must be created
or updated individually.
2023-06-06 19:00:35 +03:00
Serge Tupchii 1968589f81 fix(emqx_schema): don't allow enabling `fail_if_no_peer_cert` if `verify_none` is set
Setting `fail_if_no_peer_cert = true` and `verify = verify_none` causes connection errors.

Closes: EMQX-9586
2023-06-06 17:20:22 +03:00
Thales Macedo Garitezi 2c97ee1b3f test(conf_app): fix test after common test helpers changes 2023-06-06 09:32:38 -03:00
Thales Macedo Garitezi 97a9bb484a test(pulsar_producer): attempt to fix flaky test 2023-06-06 09:32:38 -03:00
Thales Macedo Garitezi 770dd188b1
Merge pull request #10947 from emqx/release-51
Release 51 to master
2023-06-06 09:32:16 -03:00
Andrew Mayorov d5aedaac7d
fix(tlslib): append random suffix to managed certfiles
In order to lessen the chance of ambiguity in determining collectable
certfiles during GC.
2023-06-06 11:21:03 +03:00
zhongwencool ddb4a09a3d
Merge pull request #10935 from zhongwencool/listeners-conf-update
feat: update listeners from cli
2023-06-06 08:10:45 +08:00
Zaiming (Stone) Shi 3caf114f68 test: fix test case is_list(Config) function guard 2023-06-05 23:00:41 +02:00
Zaiming (Stone) Shi bb1cf9beaa Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-05 22:56:58 +02:00
Andrew Mayorov 750b7158d4
fix(tlsgc): make computing orphans safe against symlinking
Before this commit, `orphans/1` considered managed file an orphan based
on its absolute filename, which is not safe when, for example, emqx has
a symlink configured as `data_dir` while config references certfiles
through realpaths.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 31d38e18c9
refactor(tlsgc): simplify `fold_config/4` 2023-06-05 22:41:08 +03:00
Andrew Mayorov f8e8b7993b
fix(tlsgc): anticipate only binaries as string in raw config
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-05 22:41:08 +03:00
Andrew Mayorov 3c2a7dbadc
fix(tlsgc): consolidate conf keypaths knowledge in `emqx_tls_lib`
So that this GC mechanism will be easier to maintain.
2023-06-05 22:41:08 +03:00
Andrew Mayorov ec06850bd0
test(auth): drop obsolete testcase altogether
Most of the stuff that was tested by this test case is now covered
by `emqx_tls_certfile_gc_SUITE`.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 206d0472e0
fix(emqx): avoid interference in testsuites
Also remove that `?wait_async_action` in `emqx_trace_SUITE` which
consistently becomes stuck for 100 seconds. It's not clear why it
was there in the first place because listeners should start
synchronously, so `emqx_common_test_helpers:start_apps/1` won't return
until required listeners are started.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 8a31e5639b
fix(emqx): ensure that standalone build works
Turns out rebar3 `path` resource has troubles with dangling
symlinks and non-ASCII filenames. 🥲
2023-06-05 22:41:07 +03:00
Andrew Mayorov 99ea9b86c2
feat(tlslib): add separate managed certfiles GC process
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
2023-06-05 22:41:07 +03:00
Andrew Mayorov 95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
2023-06-05 22:41:07 +03:00
Zaiming (Stone) Shi 67db9d6fe9 chore: bump version to 5.1.0-alpha.3 2023-06-05 20:25:48 +02:00
Thales Macedo Garitezi 75df622426
Merge pull request #10930 from thalesmg/max-int-timeout-v50
check maximum timeout values in schema (5.1)
2023-06-05 12:51:36 -03:00
Thales Macedo Garitezi 46393343e2 chore: use `timeout_duration` types for timer fields
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
Thales Macedo Garitezi 9bd37937f1 feat(types): add `timeout_duration{,_ms,_s}` types 2023-06-05 11:46:32 -03:00
某文 d0d6992a14 feat: update listeners from cli 2023-06-04 20:07:33 +08:00
Zaiming (Stone) Shi f469b31fee Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-03 11:21:22 +02:00
Zaiming (Stone) Shi 1ba07e1040 Merge remote-tracking branch 'origin/release-51' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-02 16:30:58 +02:00
Zaiming (Stone) Shi 28c564d15b Merge remote-tracking branch 'origin/master' into release-51 2023-06-02 16:25:48 +02:00
William Yang b1c188f9b1 perf(mqtt-caps): dont filter cap keys
save map builds
2023-06-02 16:24:52 +02:00
William Yang 4961aca3d0 perf(mqtt-caps): save some map builds 2023-06-02 16:19:58 +02:00
Zaiming (Stone) Shi 367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition
fix(emqx_cm): fix channel data registration race-condition
2023-06-02 14:20:18 +02:00
Thales Macedo Garitezi 2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50
fix(plugins): use `emqx:running_nodes` for multicall operations
2023-06-02 09:03:42 -03:00
Zaiming (Stone) Shi c75e9bbe0d fix(emqx_cm): fix channel data registration race-condition
when clustered, there are chances the a mqtt client process
get killed (e.g. holding the channel registeration lock for too long),
if the channel data inserts happen before casting out the message
for channel process monitoring, there is a chance for the
stale message left in the ets tables indefinitely.

this commit changes the order of the non-atomic operations:
it casts out the monitor request message before inserting
channel data.
2023-06-02 11:43:35 +02:00
zhongwencool dd85c981cd
Merge pull request #10783 from zhongwencool/improve-authn
feat: load changes to the authentication configuration from command line
2023-06-02 15:58:13 +08:00
zhongwencool 9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve
feat: support emqx_config:update([authrization],Conf) update
2023-06-02 15:43:09 +08:00
Andrew Mayorov 76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum
fix(ft): respect checksum in `fin` packets
2023-06-01 23:49:55 +03:00
Thales Macedo Garitezi 64258120d7 fix(plugins): use `emqx:running_nodes` for multicall operations
Fixes https://emqx.atlassian.net/browse/EMQX-10079
2023-06-01 14:00:43 -03:00
某文 2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
某文 c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
某文 94b86f01a5 chore: reformat SUITE case 2023-06-01 22:32:31 +08:00
zhongwencool 21df321d89
Merge branch 'master' into improve-authn 2023-06-01 22:24:59 +08:00
Kjell Winblad 75ff76a16b fix: friendly error message when creating bridges with too long names
This commit makes the error message and log entry that appear when one
tries to create a bridge with a name the exceeds 255 bytes (the max
length for atoms) more friendly and easier to understand.

An even better fix would be to not store bridge names as atoms but this
probably requires a more substantial change.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9609
2023-06-01 15:30:06 +02:00
William Yang c1d9bc8033 perf(emqx_mqtt_caps): direct read mqtt from zone conf 2023-06-01 14:54:44 +02:00
William Yang 923913e15c feat(config): merge with global defaults when put new zone 2023-06-01 14:54:44 +02:00
William Yang a2f8e87389 test(config): default zone and global defaults 2023-06-01 14:54:14 +02:00
William Yang e693c6ec54 test: fix emqx_takeover_SUITE 2023-06-01 14:52:33 +02:00
William Yang 8c26ee75db test: remove hardcoded configs
test with schema defaults
2023-06-01 14:52:33 +02:00
William Yang 4416122cf3 test: fix longname/shortname 2023-06-01 14:52:33 +02:00
William Yang f621bf1f8e test(config): update emqx_client_SUITE 2023-06-01 14:52:33 +02:00
William Yang 98d6ed4251 test: fix emqx_session_SUITE 2023-06-01 14:52:33 +02:00
William Yang 88f9782db0 perf(config): read zone conf once 2023-06-01 14:52:30 +02:00
Zhongwen Deng 7eea693422 fix: reorder authn when updating 2023-06-01 17:24:39 +08:00
Zhongwen Deng 4f5d5216bb test: bad authn handler callback module in SUITE 2023-06-01 17:24:39 +08:00
Zhongwen Deng 8f12d81878 feat: support emqx_ctl conf command 2023-06-01 17:24:39 +08:00
Zaiming (Stone) Shi feecd53436 chore: bump to version 5.1.0-alpha.2 for both ce and ee 2023-06-01 10:13:56 +02:00
Zaiming (Stone) Shi 15ffa98bc5 chore: fix a typo in test case function name 2023-06-01 08:18:26 +02:00
Zaiming (Stone) Shi cb2be1811e docs: fix function doc annotation 2023-06-01 08:17:36 +02:00
Thales Macedo Garitezi f83f112189 fix(config): enforce root key name to be a binary (raw config)
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299

```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
    exception: error
    key_path: [listeners,tcp,default,authentication]
    module: emqx_schema
    msg: change_config_crashed
    reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
                               <<"authentication">>]}
    stacktrace: [{emqx_utils_maps,deep_get,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
                      {line,49}]},
                 {emqx_config_handler,return_change_result,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,456}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,284}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,
                     {create_authenticator,'tcp:default',
                         #{backend => built_in_database,enable => true,
                           mechanism => password_based}}},
                 #{rawconf_with_defaults => true}}
```

Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:

```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
 #{<<"ssl">> =>
       #{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```

... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
2023-05-31 18:03:58 -03:00
Thales Macedo Garitezi b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e
more attempts to fix more test flakiness
2023-05-31 16:07:18 -03:00
Zaiming (Stone) Shi 5146de5b1c feat: add a backup copies for cluster.hocon 2023-05-31 20:34:25 +02:00
Zaiming (Stone) Shi 39d6f612ca feat(emqx_release): add new APIs to parse/compare release versions 2023-05-31 20:34:25 +02:00
Andrew Mayorov aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard
test(dashboard): fix inter-suite test flakiness
2023-05-31 20:28:39 +03:00
Andrew Mayorov f90ce9bbfe
test(dashboard): fix inter-suite test flakiness 2023-05-31 19:09:37 +03:00
Andrew Mayorov 0293b54211
fix(ft): respect checksum in `fin` packets 2023-05-31 18:50:38 +03:00
Thales Macedo Garitezi 189e2c87be test(channel): fix flaky test
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```

The problem was that the whole generated message was compared for
equality when receiving the will message, but the timestamp may be
different at publishing time...
2023-05-31 12:42:24 -03:00
JianBo He a3a6480f00
Merge pull request #10880 from HJianBo/batch-kickout-clients
feat: support kickout clients in batch
2023-05-31 22:36:35 +08:00
Thales Macedo Garitezi 77c9eda036 test: rm stray `clear_screen` calls 2023-05-31 11:07:44 -03:00
Thales Macedo Garitezi 0d539e91d1 test(pulsar_producer): attempt to stabilize flaky test
https://github.com/emqx/emqx/actions/runs/5125166433/jobs/9218613872?pr=10886#step:7:679
```
  =CRITICAL REPORT==== 30-May-2023::19:38:58.003170 ===
  Run stage failed: error:{badmatch,
                              {timeout,
                                  [#{msg => pulsar_producer_bridge_started,
                                     '~meta' =>
                                         #{gl => <97891.472.0>,
                                           location =>
                                               #Fun<emqx_bridge_pulsar_impl_producer.11.109752493>,
                                           node => 'autocluster_node1@127.0.0.1',
                                           pid => <97891.787.0>,
                                           time => -576460611692219}}]}}
  Stacktrace: [{emqx_bridge_pulsar_impl_producer_SUITE,'-t_cluster/1-fun-10-',
                   6,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1073}]},
               {emqx_bridge_pulsar_impl_producer_SUITE,t_cluster,1,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1064}]}]
```
2023-05-31 10:19:55 -03:00
zhongwencool 26c2ab0bef
Merge pull request #10894 from zhongwencool/crl-cache-conf-hot-update
feat: update crl_cache conf at runtime
2023-05-31 17:59:09 +08:00
William Yang 2250ed7052
Merge pull request #10823 from qzhuyan/dev/william/meck-olp
test: use meck in olp tests
2023-05-31 11:01:56 +02:00
Zhongwen Deng 38e57f511c feat: update crl_cache conf at runtime 2023-05-31 16:18:03 +08:00
JianBo He afcd9fcb5e chore: update bpapi.versions 2023-05-31 11:29:55 +08:00
zhongwencool 6202cefd5a
Merge pull request #10870 from zhongwencool/stop-dashboard-listener-waiting
fix: flaky test when stopping dashboard listener
2023-05-31 10:54:12 +08:00
firest 79d08e4d26 fix(limiter): temporarily hide the node-level settings 2023-05-31 09:43:14 +08:00
firest 55809f801c test: fix test case error 2023-05-31 09:43:14 +08:00
firest be2dfa51df fix(limiter): improve the default configuration 2023-05-31 09:43:14 +08:00
Zhongwen Deng 5690469896 fix: flaky test when stopping dashboard listener 2023-05-31 07:53:10 +08:00
Thales Macedo Garitezi 299a67656e
Merge pull request #10886 from thalesmg/test-flakiness-20230530-d
test(quic_multistreams): attempt to fix flaky tests
2023-05-30 17:34:23 -03:00
Thales Macedo Garitezi 4f1c55f3fb
Merge pull request #10885 from thalesmg/test-flakiness-20230530-c
test(olp): fix inter-suite flakiness
2023-05-30 16:09:17 -03:00
Thales Macedo Garitezi 596f05f981 test(channel): attempt to reduce test flakiness
https://github.com/emqx/emqx/actions/runs/5124277038/jobs/9217362541?pr=10886#step:5:1582
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```
2023-05-30 15:48:48 -03:00
Thales Macedo Garitezi c43e5b8406
Merge pull request #10883 from thalesmg/test-flakiness-20230530-b
test(exclusive_sub): fix test flakiness
2023-05-30 15:46:14 -03:00
Thales Macedo Garitezi 98a99d47cb test(shared_sub): attempt to reduce test flakiness
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9215439237#step:8:565
```
%%% emqx_shared_sub_SUITE ==> t_local_fallback: FAILED
%%% emqx_shared_sub_SUITE ==> {{badmatch,<<"not yet?">>},
 [{emqx_shared_sub_SUITE,t_local_fallback,1,
                         [{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_shared_sub_SUITE.erl"},
                          {line,669}]},
```
2023-05-30 14:03:14 -03:00
Thales Macedo Garitezi fa9afcd7e4 test(quic_multistreams): attempt to fix flaky tests
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9214950882#step:8:1757
```
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
test_server:ts_tc failed on line 1782
Reason: {test_case_failed,Client <0.6075.5> did not die: stacktrace: {current_stacktrace,
                                            [{gen_statem,loop_receive,3,
                                              [{file,"gen_statem.erl"},
                                               {line,1281}]},
                                             {proc_lib,init_p_do_apply,3,
                                              [{file,"proc_lib.erl"},
                                               {line,240}]}]}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Testing lib.emqx.emqx_quic_multistreams_SUITE: *** FAILED test case 335 of 364 ***
%%% emqx_quic_multistreams_SUITE ==> shutdown.abort_recv_shutdown.ctrl_stream_shutdown.t_multi_streams_remote_shutdown: FAILED
%%% emqx_quic_multistreams_SUITE ==> {test_case_failed,"Client <0.6075.5> did not die: stacktrace: {current_stacktrace,\n                                            [{gen_statem,loop_receive,3,\n                                              [{file,\"gen_statem.erl\"},\n                                               {line,1281}]},\n                                             {proc_lib,init_p_do_apply,3,\n                                              [{file,\"proc_lib.erl\"},\n                                               {line,240}]}]}"}
```

```
  %%% emqx_quic_multistreams_SUITE ==> mstream.profiles.profile_max_throughput.pub_qos2.sub_qos2.qos.t_multi_streams_unsub: FAILED
  %%% emqx_quic_multistreams_SUITE ==> {{case_clause,
       {ok,#{packet_id => 8,properties => #{},reason_code => 0,
             reason_code_name => success,
             via =>
                 {quic,#Ref<0.686875463.2755788802.201649>,
                     #Ref<0.686875463.2755788804.193527>}}}},
   [{emqx_quic_multistreams_SUITE,t_multi_streams_unsub,1,
        [{file,
             "/home/thales/dev/emqx/emqx/apps/emqx/test/emqx_quic_multistreams_SUITE.erl"},
         {line,1094}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
    {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1292}]},
    {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}
```
2023-05-30 13:56:05 -03:00
Thales Macedo Garitezi 37b4a44e23 test(olp): fix inter-suite flakiness
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9212645120?pr=10883#step:8:2538
```
   emqx_olp_SUITE:t_overload_cooldown_conn failed on line 96
  Reason: {assertMatch,[{module,...},{...}|...]}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_olp_SUITE: *** FAILED test case 3 of 4 ***
  %%% emqx_olp_SUITE ==> t_overload_cooldown_conn: FAILED
  %%% emqx_olp_SUITE ==>
  Failure/Error: ?assertMatch({ ok , _Pid }, emqtt : connect ( C ))
    expected: = { ok , _Pid }
         got: {error,econnrefused}
        line: 96
  %%% emqx_olp_SUITE ==> t_overloaded_conn: OK
  Testing lib.emqx.emqx_olp_SUITE: TEST COMPLETE, 3 ok, 1 failed of 4 test cases
```

```sh
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_olp_SUITE.erl
```
2023-05-30 13:39:52 -03:00
William Yang f2e54c9a6e test: use meck in olp tests
avoid flaky test results by using mecks.
2023-05-30 17:57:09 +02:00
Thales Macedo Garitezi a28580248a test(exclusive_sub): fix test flakiness
https://github.com/emqx/emqx/actions/runs/5115608275/jobs/9208594481?pr=10864#step:8:735

```
  Testing lib.emqx.emqx_access_control_SUITE: TEST COMPLETE, 4 ok, 0 failed of 4 test cases

  Testing lib.emqx.emqx_exclusive_sub_SUITE: Starting test, 5 test cases

  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  Error detected: {'EXIT',{shutdown,econnrefused}}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 1 of 5 ***
  %%% emqx_exclusive_sub_SUITE ==> t_allow_normal_sub: FAILED
  %%% emqx_exclusive_sub_SUITE ==> {'EXIT',{shutdown,econnrefused}}

  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  emqx_exclusive_sub_SUITE:t_clean_session failed on line 128
  Reason: {badmatch,{error,econnrefused}}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 2 of 5 ***
  %%% emqx_exclusive_sub_SUITE ==> t_clean_session: FAILED
```

To reproduce:

```
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_exclusive_sub_SUITE.erl
```
2023-05-30 11:23:17 -03:00
Thales Macedo Garitezi cb7929a2fc test(broker): attempt to reduce flakiness 2023-05-30 11:12:18 -03:00
Thales Macedo Garitezi 26992e9d98 test(banned): attempt to improve flaky test 2023-05-30 11:08:31 -03:00
Thales Macedo Garitezi 6b5b9fc8b8
Merge pull request #10864 from thalesmg/fix-flaky-channel-test-20230529
test(channel): fix flaky test
2023-05-30 10:18:47 -03:00
Zaiming (Stone) Shi d2ce7c22fd chore: bump app vsn, core app 'emqx' is now at 5.1 2023-05-30 11:32:18 +02:00
Zaiming (Stone) Shi 1e8ed89db8 Merge remote-tracking branch 'origin/master' into release-51 2023-05-30 11:29:44 +02:00
Zaiming (Stone) Shi 91cdc69976
Merge pull request #10867 from zmstone/0530-merge-release-50-to-master
0530 merge release 50 to master
2023-05-30 09:54:57 +02:00
Zaiming (Stone) Shi 367de2a2eb docs: update bpapi readme 2023-05-30 08:29:43 +02:00
Zaiming (Stone) Shi 941939f7d7 build: finalize 5.0.bpapi 2023-05-30 08:28:45 +02:00
Zaiming (Stone) Shi 3b8cba6ba0 chore: bump rlease versions 2023-05-30 08:28:45 +02:00
Zaiming (Stone) Shi 747eb6fcff Merge remote-tracking branches 'origin/release-50' and 'origin/master' into release-51 2023-05-30 07:54:34 +02:00
JimMoen 909d348570
chore: fix spec typo 2023-05-30 11:31:37 +08:00
Thales Macedo Garitezi 1aff46b7c9 test(channel): fix flaky test 2023-05-29 19:04:48 -03:00
Thales Macedo Garitezi 67e182e0c9
Merge pull request #10813 from thalesmg/refactor-kafka-on-stop-v50
feat(kafka): ensure allocated resources are removed on failures
2023-05-29 16:49:29 -03:00
Thales Macedo Garitezi 79f6861499 chore: prepare to tag v5.0.26 2023-05-29 09:17:59 -03:00
Zaiming (Stone) Shi 8e6dc29637 chore: bump release version to v5.0.26-alpha.1 2023-05-29 08:57:28 +02:00
Zaiming (Stone) Shi 25319c3151
Merge pull request #10839 from zmstone/0526-merge-release-50-to-master
0526 merge release 50 to master
2023-05-29 08:56:20 +02:00
Zaiming (Stone) Shi c2450d230d
Merge pull request #10605 from zmstone/0503-no-telemetry-app-for-ee
0503 refactoring: no telemetry app for ee
2023-05-27 10:57:15 +02:00
Thales Macedo Garitezi d27f593309 test: fix flaky authn test 2023-05-26 16:44:00 -03:00
Zaiming (Stone) Shi fe81e9521a
Merge pull request #10600 from zmstone/0504-delete-statsd-all-together
0504 delete statsd all together
2023-05-26 16:29:59 +02:00
Zaiming (Stone) Shi 36e268c933 chore: bump app versions 2023-05-26 16:05:37 +02:00
Zaiming (Stone) Shi cc5b4d3748 Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test 2023-05-26 15:58:16 +02:00
Zaiming (Stone) Shi 6e2cde8224 refactor: delete emqx_statsd all together 2023-05-26 15:17:42 +02:00
Andrew Mayorov 45b128946a
Merge pull request #10825 from fix/EMQX-9991/bugs
fix(ft): avoid leaking secrets
2023-05-26 16:08:14 +03:00
Zaiming (Stone) Shi 7eb22e27c4 chore: bump release version to e5.0.4 2023-05-26 14:34:01 +02:00
Zhongwen Deng 9b6da3ff10 chore: bump hocon to 0.39.7 2023-05-26 19:00:58 +08:00
Zhongwen Deng df4a0afdf5 fix: zones api default value 2023-05-26 18:59:38 +08:00
Zaiming (Stone) Shi 2d986140df chore: bump version to e5.0.4-rc.2 2023-05-26 12:01:10 +02:00
Andrew Mayorov 48858dee33
fix(s3): wrap S3 secrets during config load 2023-05-26 12:59:21 +03:00
JimMoen a3b77563ba
fix: make static check happy 2023-05-26 10:34:15 +08:00
Paulo Zulato ea86f4442b fix: avoid error 500 when node is re-joining cluster
Fixes https://emqx.atlassian.net/browse/EMQX-9899
2023-05-25 13:32:28 -03:00
Zaiming (Stone) Shi e8a4df96bc Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-25 16:43:20 +02:00
ieQu1 2b79196a03
Merge pull request #10809 from ieQu1/ekka-0.15.2
chore(ekka): Bump version to 0.15.2
2023-05-24 18:13:04 +02:00
zhongwencool 14a6b36899
Merge pull request #10803 from JimMoen/merge-release-50
Merge release-50 into master
2023-05-24 23:08:46 +08:00
ieQu1 3d3dacfcf6 chore(ekka): Bump version to 0.15.2 2023-05-24 15:43:49 +02:00
JimMoen 28015597ee
Merge remote-tracking branch 'emqx/release-50' into merge-release-50 2023-05-24 19:34:12 +08:00