Commit Graph

9151 Commits

Author SHA1 Message Date
zhongwencool 9521e11d39
Merge pull request #11584 from zhongwencool/telemetry-error-on-windows
fix: telemetry report error on windows
2023-09-21 09:08:55 +08:00
JianBo He b924fb618a
Merge pull request #11639 from lafirest/test/sso
test(sso): add test case for the integration of SSO with LDAP
2023-09-21 09:01:13 +08:00
zhongwencool 1b29e4b189
Merge pull request #11641 from zhongwencool/slog-msg-unify 2023-09-21 07:32:39 +08:00
Andrew Mayorov a2ddd9d5f5
fix(session): respect existing session even if expiry interval = 0
If the original connection had Session-Expiry-Interval > 0, and the
new connection set Session-Expiry-Interval = 0, the MQTTv5 spec says
that (supposedly) we still have to continue with the existing session
(if it hasn't expired yet).

Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:54 +04:00
Andrew Mayorov 3945f08f8f
fix(sessds): try to ensure iterators are closed on destroy 2023-09-20 22:55:54 +04:00
Andrew Mayorov 21e82b9534
test(sessmem): make retry delivery testcase more involved 2023-09-20 22:55:54 +04:00
Andrew Mayorov 69889d14a3
fix(sessds): fix use of undefined types 2023-09-20 22:55:54 +04:00
Andrew Mayorov c1583f7f9d
fix(ds): refine `topic()` type to describe parsed topics
And separate it from `topic_filter()` type, which describes parsed
topic filters.
2023-09-20 22:55:52 +04:00
Andrew Mayorov 9362ef6f73
test(sessmem): drop unnecessary nesting in testcase
Also get rid of sneaky binding assignment.
2023-09-20 22:55:25 +04:00
Andrew Mayorov 98706cd215
chore: ensure comments follow code style consistently 2023-09-20 22:55:25 +04:00
Andrew Mayorov 8670efbfa0
chore(chan): rename `Name` → `TimerName` for better readability 2023-09-20 22:55:25 +04:00
Andrew Mayorov 7a9916c84d
fix(sessds): convert ds iterator topics upon opening ds session 2023-09-20 22:55:25 +04:00
Andrew Mayorov 045d8b7f10
refactor(ds): reorder functions to improve readability 2023-09-20 22:55:24 +04:00
Andrew Mayorov 540ca6d60f
chore: drop few commented out and irrelevant pieces 2023-09-20 22:55:24 +04:00
Andrew Mayorov 9d145890cc
refactor(sessmem): pass log context as part of session event
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
Andrew Mayorov adc29e15cc
refactor(session): make typespecsa and flow a bit more clear
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
Andrew Mayorov e422f492ef
test(sessds): reuse and expand persistent session test suite 2023-09-20 22:55:24 +04:00
Andrew Mayorov 3383ae19a9
test(session): switch `emqx_persistent_session_SUITE` to cth tooling 2023-09-20 22:55:23 +04:00
Andrew Mayorov 7326ef550b
fix(sessds): make existing parts of persistent session impl work 2023-09-20 22:55:22 +04:00
Andrew Mayorov e713fc38aa
feat(broker): reflect persisted messages in publish result
In order for callers to distinguish between silently dropped and
durably persisted message w/o matching subscribers.
2023-09-20 22:53:45 +04:00
Andrew Mayorov f4953e719b
fix(cmproto): fix few typespecs 2023-09-20 22:53:45 +04:00
Andrew Mayorov 8af107e28d
test(ds): simplify cluster test setups 2023-09-20 22:53:45 +04:00
Andrew Mayorov 7c4f68dd3d
fix(session): make utility function names consistent
Before this commit behavior of `is_banned_msg/1` / `should_discard/1`
were actually the exact opposite of their names.

Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:53:44 +04:00
Andrew Mayorov 2dae8020ec
refactor(cm): avoid deep indirection in `emqx_session_mem` 2023-09-20 22:53:44 +04:00
Andrew Mayorov abeff0bc4f
chore(session): try to describe what happens after session takeover 2023-09-20 22:53:44 +04:00
Andrew Mayorov 45d44df11d
refactor(session): update eviction channel session logic
The changes partially reflect `emqx_channel` changes with respect to
in-memory session specific logic. The difference is that eviction
channel does not replay post-takeover, instead enqueues messages.
2023-09-20 22:53:44 +04:00
Andrew Mayorov ab1c4c4222
refactor(session): drop `lookup/1` session API
Due to the fact it's not used for anything right now.
2023-09-20 22:53:44 +04:00
Andrew Mayorov 97881ff3ca
refactor(session): bring back common session timers 2023-09-20 22:53:43 +04:00
Andrew Mayorov bf16417513
feat(session): introduce session implementation concept 2023-09-20 22:53:42 +04:00
Andrew Mayorov 780ca15298
chore: bump application versions 2023-09-20 22:52:14 +04:00
Andrew Mayorov 04731b7ef7
test(takeover): randomize messages for random natural ordering 2023-09-20 22:52:14 +04:00
Andrew Mayorov f022c9b1a4
feat(emqx): add `emqx_inflight:fold/3` generic function 2023-09-20 22:52:14 +04:00
Andrew Mayorov 596ce157fd
refactor(exproto): make timer names equal to messages they send
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
Andrew Mayorov 57ae5b14f1
refactor(mqttsn): make timer names equal to messages they send
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
Andrew Mayorov e4866adc2f
refactor(chan): make timer names equal to messages they send
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
Andrew Mayorov cfb1bf1fa4
chore(emqx): drop remnants of former session persistence impl
1. It is not functional anyway.
2. It blocks `emqx_session` refactoring in a few places.
2023-09-20 22:52:13 +04:00
William Yang e43b87f18e test(quic): update emqx_broker_SUITE 2023-09-20 20:20:18 +02:00
Thales Macedo Garitezi 2965fa6fcb
Merge pull request #11613 from thalesmg/ds-keyspace-m-20230914
feat(ds): introduce keyspace concept
2023-09-20 14:21:27 -03:00
William Yang da626f6d29 feat(quic): bump to quicer 0.0.200 and emqtt 1.9.0 2023-09-20 16:43:15 +02:00
zhongwencool dd687d9582 fix: dialyzer warning 2023-09-20 22:41:26 +08:00
zhongwencool 9e75ff88af
chore: apply suggestions from code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:15:39 +08:00
zhongwencool f789a7504f
chore: update apps/emqx/src/emqx_quic_connection.erl
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:00:44 +08:00
Ivan Dyachkov 09aea47f40 chore: 5.2.1 2023-09-20 14:04:28 +02:00
firest 21c37a3f14 fix(sso): adjust the schema of LDAP API && add more logs 2023-09-20 18:15:42 +08:00
zhongwencool 2f1fa2e961 chore: unified slog message formatting to improve logging consistency 2023-09-20 18:13:00 +08:00
zhongwencool c26a18e949 fix: always return ok when remove local resource 2023-09-20 18:02:42 +08:00
firest b11c5d26ce test(sso): add test case for the integration of SSO with LDAP 2023-09-20 16:25:19 +08:00
lafirest bc6edac45f
Merge pull request #11631 from lafirest/feat/dashboard_ldap
feat(dashboard): add SSO feature and integrate with LDAP
2023-09-20 11:40:42 +08:00
firest 61311081d8 fix(sso): fix RBAC test errors && spellcheck 2023-09-20 10:42:17 +08:00
Thales Macedo Garitezi 5ed5ac48ee refactor: combine shard id and keyspace into a single value 2023-09-19 12:31:32 -03:00
firest 5b00998050 fix(sso): fix invalid_dynamic_call 2023-09-19 22:47:48 +08:00
firest 285e529766 fix(sso): fix sso errors found when manual test 2023-09-19 19:15:52 +08:00
Zaiming (Stone) Shi 65818740f3
Merge pull request #11622 from zmstone/0917-upgrade-gen_rpc-to-3.1.0
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-19 10:22:53 +02:00
Ivan Dyachkov a69c26b86d chore: e5.2.1-alpha.3 2023-09-19 08:17:29 +02:00
firest 2cddce5479 feat(dashboard): add SSO feature and integrate with LDAP 2023-09-19 14:10:29 +08:00
JianBo He de1bc04843 chore: bump ekka to 0.15.14, mria to 0.6.3
Inclued ekka's updates:
- https://github.com/emqx/ekka/pull/214
- https://github.com/emqx/ekka/pull/215
- https://github.com/emqx/ekka/pull/216
- https://github.com/emqx/ekka/pull/217

mria's updates:
- https://github.com/emqx/mria/pull/160
- https://github.com/emqx/mria/pull/161
- https://github.com/emqx/mria/pull/162
2023-09-19 09:43:19 +08:00
Thales Macedo Garitezi a511088fd4 refactor: address review comments 2023-09-18 17:47:56 -03:00
Zaiming (Stone) Shi 9f8a7a96af
Merge pull request #11628 from zmstone/0918-upgrade-emqtt-to-1.8.7
0918 upgrade emqtt to 1.8.7
2023-09-18 21:56:04 +02:00
Zaiming (Stone) Shi de93d71014 test: fix emqx_sahred_sub_SUITE:t_remote
prior to this fix, the peer ct node is actually node itself
2023-09-18 21:54:43 +02:00
Zaiming (Stone) Shi 99b60849b4 test: fix flaky test emqx_falpping_SUITE:t_conf_update_timer 2023-09-18 20:44:38 +02:00
SergeTupchiy 0b0e322473
Merge pull request #11627 from SergeTupchiy/EMQX-10962-fix_hstreamdb-bridge-on-stop
fix(emqx_bridge_hstreamdb): fix resources cleanup in on_stop/2 cb
2023-09-18 19:57:07 +03:00
Zaiming (Stone) Shi d1e5b69eac test: increase wait time 2023-09-18 17:33:09 +02:00
Zaiming (Stone) Shi 4f1ec795eb test: delete special handling of emqtt race condition
the race should have been fixed in emqtt 1.8.7
2023-09-18 17:27:34 +02:00
Zaiming (Stone) Shi 46557c3462 fix(emqtt): upgrade from 1.8.6 to 1.8.7 2023-09-18 17:04:23 +02:00
Serge Tupchii 80cf60e9be fix(emqx_bridge_hstreamdb): fix resources cleanup in on_stop/2 cb
fixes: EMQX-10962
2023-09-18 17:48:05 +03:00
Zaiming (Stone) Shi 45f0d0dfce
Merge pull request #11623 from zmstone/0917-upgrade-esockd-to-5.9.7
0917 upgrade esockd to 5.9.7
2023-09-18 16:30:54 +02:00
zhongwencool c868264f65
Merge pull request #11617 from zhongwencool/rule-engine-console-args
fix: keep builtin_console actions not complain args
2023-09-18 20:40:02 +08:00
zhongwencool 8b6d5d4ca9 test: add test for build_console's args 2023-09-18 16:48:38 +08:00
zhongwencool 5490807b20
chore: update apps/emqx_rule_engine/src/emqx_rule_engine_schema.erl
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-18 16:11:39 +08:00
Zaiming (Stone) Shi f66d9e76fe fix(emqx_channel): aovid logging frame errors as exceptions 2023-09-18 09:04:00 +02:00
lafirest 6fe846bf0e
Merge pull request #11610 from lafirest/feat/rbac
feat(dashboard): add RBAC feature for Dashboard
2023-09-18 14:41:47 +08:00
firest ca6547c05b fix(rbac): change default role and update changes 2023-09-18 11:40:35 +08:00
Zaiming (Stone) Shi abbee4bc47 fix(esockd): upgrade esockd from 5.9.6 to 5.9.7 2023-09-17 19:44:04 +02:00
Zaiming (Stone) Shi 985f0e6994 chore(gen_rpc): upgrade from 2.8.1 to 3.1.0 2023-09-17 19:10:13 +02:00
Zaiming (Stone) Shi a9682c2989
Merge pull request #11605 from zmstone/0913-lower-severity-level-for-CMD-override-warnings
0913 lower severity level for cmd override warnings
2023-09-16 11:32:07 +02:00
Zaiming (Stone) Shi ac41f497b4 feat: add bytesize rule sql function 2023-09-15 16:04:21 +02:00
Thales Macedo Garitezi 2aae3856a3
Merge pull request #11596 from thalesmg/fix-typespec-typos-m-20230912
fix: fix typespec typos
2023-09-15 09:24:42 -03:00
Ilya Averyanov f0d2b2e6c3 chore: update emqx_eviction_agent version 2023-09-15 13:24:38 +03:00
firest 4b97d3f57d test(rbac): add test cases for RBAC && update changes 2023-09-15 18:16:28 +08:00
Ilya Averyanov 1959e15703 chore: simplify session eviction for node rebalance 2023-09-15 11:46:51 +02:00
zhongwencool 32d838c85b fix: format console action crash 2023-09-15 17:40:56 +08:00
zhongwencool 640dd130fa fix: don't urldecode twice clientid 2023-09-15 17:36:48 +08:00
zhongwencool d72348c8ed fix: keep builtin_console actions not complain args 2023-09-15 16:38:04 +08:00
firest 45ccc66474 fix(dashbaord): fix test cases to support RBAC 2023-09-15 15:49:37 +08:00
lafirest aace9215a4
Merge pull request #11608 from lafirest/feat/ldap_bind
feat(ldap): integrate authentication with LDAP bind operation
2023-09-15 14:06:39 +08:00
firest 0846939760 fix(ldap): remove unused code and mark sensitive field 2023-09-15 10:13:07 +08:00
Thales Macedo Garitezi b30bcf32bd feat(ds): introduce keyspace concept
Fixes https://emqx.atlassian.net/browse/EMQX-10579

This introduces the concept of "keyspaces" to our durable storage (DS) implementation, and
also refactors some places where "shard" and "keyspace" would be mixed up.

We might want to tune the storage options differently for distinct sets of topics, the
keyspaces.  The keyspace is composed by one or more shards.

- Keyspaces are identified simply by binary strings.
- DS configuration is scoped by keyspaces instead of shards.
- Starting a new DS shard requires definining to which keyspace the shard belongs.
2023-09-14 16:39:00 -03:00
Ilya Averyanov 11c39c4b6a chore: simplify session eviction for node rebalance 2023-09-14 18:28:25 +03:00
Thales Macedo Garitezi 6be0e14e8c fix: fix typespec typos 2023-09-14 10:21:05 -03:00
Thales Macedo Garitezi e41f7dd68c
Merge pull request #11598 from thalesmg/ds-fix-ps-router-m-20230912
fixes and improvements to persistent session router
2023-09-14 10:00:58 -03:00
firest 5c31c5ce76 feat(dashboard): add RBAC feature for Dashboard 2023-09-14 19:28:37 +08:00
firest afbf13b8a2 feat(ldap): integrate authentication with LDAP bind operation 2023-09-14 14:10:43 +08:00
firest 3eb46be2ae fix(banned): fix typos 2023-09-14 14:01:59 +08:00
Zaiming (Stone) Shi 95f1312df7 chore(emqx_ctl): lower command override log from warning to info
prior to this change this warning message is always logged if a node
joins cluster.

the intention of this warning is to detect potential command name
clash (provided by different plugins), so arguably it should be
at debug level
2023-09-13 20:52:08 +02:00
Thales Macedo Garitezi 803b69d878
refactor: use more consistent return types
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2023-09-13 10:49:45 -03:00
Thales Macedo Garitezi f55f6dc779 docs(ps_session): add notes explaining rationale about operation order and consistency 2023-09-13 10:12:27 -03:00
Thales Macedo Garitezi e081abb9e6 perf(ps_router): only check if there are any routes when deciding whether to persist message 2023-09-13 10:02:39 -03:00
Thales Macedo Garitezi cae9ae1fab fix(ps_router): use `disc_copies` for storing persistent session routes 2023-09-13 10:02:39 -03:00
William Yang ea73d674b9 chore: bump version to e5.2.1-alpha.2 2023-09-13 14:55:17 +02:00
ieQu1 69d17f981d
Merge pull request #11595 from ieQu1/dev/dont-prevent-overlapping-partitions
fix(distribution): Set prevent_overlapping_partitions to false
2023-09-13 14:22:34 +02:00