fd75dc895d
fix(authn): "authentication" importance should not be "hidden"
2023-10-03 11:18:21 +02:00
ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53
...
0928 sync release 53
2023-10-02 21:57:32 +02:00
3f6e0e890b
chore: bump emqx_durable_storage and emqx_license app vsn
2023-10-02 20:13:09 +02:00
2358d67908
refactor: move session stuff from `emqx_ds` to `emqx_persistent_session_ds`
...
Part of https://emqx.atlassian.net/browse/EMQX-10942
2023-09-29 18:00:24 -03:00
bce8fd2fbc
chore: bump app versions
2023-09-29 18:56:52 +02:00
ed5b456d62
Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53
2023-09-29 18:24:44 +02:00
a852400fb3
chore: add 5.3.bpapi
2023-09-29 16:34:04 +02:00
fe01aaff4f
chore: prepare for e5.3.0 promotion
2023-09-29 13:32:57 +02:00
ce5bd0a3ce
Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53
2023-09-29 11:36:32 +02:00
c64e599e81
docs: document how to retrieve peercert
2023-09-29 10:33:57 +02:00
dc147fd310
fix(rule-engine): console action has no args field
2023-09-29 10:33:33 +02:00
1177a32310
chore: bump version to 5.3.0-rc.2
2023-09-29 09:49:27 +02:00
6891234390
chore: return simplified error reason for less logging
2023-09-29 09:48:15 +02:00
c2d750aa09
fix(resource): redact query args in exception log
2023-09-29 09:20:42 +02:00
02ef854f0f
fix(ldap): no crash when no query result is empty list
2023-09-29 08:54:41 +02:00
b59a7ff2dd
Merge pull request #11706 from zmstone/0928-fix-ldap-sso-logging-level
...
0928 fix ldap sso logging level and reject muti-match results
2023-09-29 02:40:37 +02:00
4a4730ad46
fix(ldap): handle invalidCredentials in ldap authn
2023-09-29 00:51:05 +02:00
9ee2cb9c79
fix(ldap): return unrecoverable_error if more than on match found
2023-09-28 23:58:34 +02:00
cc5dab1dc7
chore: fix code style
2023-09-28 21:29:59 +02:00
b267fc2588
chore: bump release version to 5.3.0
2023-09-28 21:22:33 +02:00
922d5a9a83
fix(ldap): do not allow multi-matches to proceed
...
if ldap query returns more than on match
we should reject the auth request instead of picking
the first one
2023-09-28 21:20:50 +02:00
d858f8af39
test: fix openldap docker runs
2023-09-28 18:40:03 +02:00
b28e781c50
fix(ldap-sso): do not log error level when invalid user credentials
2023-09-28 18:37:11 +02:00
36f3052be1
Merge pull request #11698 from zmstone/0928-disable-audit-log-by-default
...
fix(audit): disable audit log by default
2023-09-28 17:28:23 +02:00
39820be5ff
Merge pull request #11703 from lafirest/fix/ldap_bind
...
fix(ldap): use the search result as bind target
2023-09-28 14:48:53 +02:00
1a13b2ac56
Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53
2023-09-28 13:53:03 +02:00
43ea367df8
fix(ldap): use the search result as bind target
2023-09-28 18:49:00 +08:00
98409c9b1e
Merge pull request #11702 from zmstone/0928-minor-chore
...
0928 minor chore
2023-09-28 12:39:55 +02:00
26cadb2248
fix(audit): disable audit log by default
2023-09-28 12:36:14 +02:00
b14425d865
fix(sso/saml): ensure lower case http header name
2023-09-28 11:40:21 +02:00
7ddcd71412
docs(ssl_dist.conf): update doc link
2023-09-28 11:16:18 +02:00
dca8fdb17f
fix(resource): respect the start_timeout
2023-09-28 16:36:41 +08:00
b77e5e880a
Merge pull request #11697 from ieQu1/dev/gen-rpc-3.1.1
...
chore(gen_rpc): Bump version to 3.1.1
2023-09-28 10:03:36 +02:00
6d2adfc259
chore(audit): emit the message field
2023-09-28 11:04:42 +08:00
859b122cdd
chore: format codes
2023-09-28 10:25:37 +08:00
b52e4ac99d
chore: update apps/emqx_machine/src/emqx_restricted_shell.erl
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-09-28 10:25:37 +08:00
bdf24d0ec8
chore: remove the `unauthorized` type for `from` field
2023-09-28 10:25:37 +08:00
71acf121ba
chore(audit): distinguish requests from rest_api or dashboard
2023-09-28 10:25:37 +08:00
dd9938114c
chore(audit): add from field
2023-09-28 10:25:37 +08:00
ece7d5b52a
Merge pull request #11690 from lafirest/fix/ldap_parse_cfg
...
fix(ldap): improve the LDAP `parse_config` function
2023-09-28 10:24:52 +08:00
b0d86eecd6
Merge pull request #11691 from lafirest/fix/sso_ssl
...
fix(sso): support for SSL update && ensure update is atomic
2023-09-28 10:22:34 +08:00
57781d0544
fix(ldap): remove the parse_config, it never work
2023-09-28 09:38:16 +08:00
9dee2dc31e
fix(sso): clear last error first before update && fix the `running`
2023-09-28 08:56:16 +08:00
0aa3ccdd65
feat(gen_rpc): Add schema for the TLS versions and cipher suites
2023-09-28 00:36:59 +02:00
afdda107af
fix(logger): json format log encode binary list as string array
2023-09-27 23:40:01 +02:00
6f7a4344dc
fix: do not gc sso saml SP singing keys
2023-09-27 23:19:39 +02:00
34367fc4ec
fix(audit_log): pretty print shell args
2023-09-27 23:19:39 +02:00
bb49914fd6
fix(sso): add convet_certs callback for sso backends
...
must convert certs in pre_config_update so the cert path refernces
are stored in raw config, otherwise the files might get gc:ed
2023-09-27 22:41:39 +02:00
45caa3bf01
fix(sso): make sp_private_key sensitive
...
so it will not be logged
2023-09-27 21:27:59 +02:00
c8cbbff044
fix(logger): no need for special handling of empty string
...
when formating json logs, there is no need to handle empty strings
special, already covered by unicode handling
2023-09-27 21:26:47 +02:00
3c37f19105
chore(gen_rpc): Bump version to 3.1.1
2023-09-27 21:09:00 +02:00
66d2107007
fix(sso): refactor update logic
2023-09-28 00:09:09 +08:00
bd3277c51b
Merge pull request #11683 from zmstone/0925-test-refactor-emqx-rpc-test
...
test: refactor emqx_rpc unit tests
2023-09-27 15:46:27 +02:00
08ad09a68f
fix(sso): refactor backen update logic
...
1. valid config always can update successfully
2. the `running` endpoint only return successfully created backend
3. enhancement of the `/sso` endpoint, and will check is the resource online
2023-09-27 20:53:10 +08:00
ea8d54fd8b
test: ensure atom exists in test module
2023-09-27 12:58:06 +02:00
6e8c73258f
test: refactor emqx_rpc unit tests
2023-09-27 12:58:06 +02:00
c9194cd6b2
fix(saml_sso): donot load IDP metadata when disabling saml
2023-09-27 18:46:24 +08:00
af9e87c025
fix: saml callback should check saml state
2023-09-27 18:34:48 +08:00
b2699c687b
fix(sso): support for SSL update && ensure update is atomic
...
1. support update SSL key and cert files
2. increase connection timeout
3. ensure the update is atomicity, everything will be consistent
2023-09-27 15:48:11 +08:00
1d0e789e4d
Merge pull request #11679 from keynslug/ft/EMQX-10942/custom-session-timers
...
feat(session): add custom session timers mechanism
2023-09-26 19:09:53 +03:00
0c33df3912
fix(ldap): improve the LDAP `parse_config` function
2023-09-26 23:28:37 +08:00
3fa18d6935
fix: lower sso saml redirect http header
2023-09-26 23:25:59 +08:00
6a9bb7c3ae
Merge pull request #11681 from zhongwencool/audit-log-fix-2
...
fix: don't need to change audit log's level
2023-09-26 23:12:28 +08:00
ac5eb5bc29
Merge pull request #11687 from lafirest/fix/sso_timeout
...
fix(sso): Handle backend update timeout and fix create errors
2023-09-26 22:43:38 +08:00
1a96a5990b
chore: Update apps/emqx_enterprise/src/emqx_enterprise_schema.erl
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-26 21:44:40 +08:00
403714d44e
fix(sso): Handle backend update timeout and fix create errors
...
1. correctly handle the timeout when call update on a backend
2. fix that config update always returns success
3. do not ignore start failures and ensure start is atomic
2023-09-26 21:43:30 +08:00
aae59f1efd
Merge pull request #11682 from thalesmg/fix-file-logger-type-r53-20230925
...
fix(file_logger): change file logger type depending on rotation size
2023-09-26 10:22:36 -03:00
bce2142df8
test(session): add custom session timers testcase
2023-09-26 16:22:06 +03:00
4e15edb5e4
Merge pull request #11661 from zmstone/0922-fix-json-log-formatter
...
0922 fix json log formatter
2023-09-26 14:13:41 +02:00
fdcd73e20c
fix(file_logger): change file logger type depending on rotation size
...
Fixes https://emqx.atlassian.net/browse/EMQX-11036
From `logger_disk_log_h:open_disk_log`:
```erlang
open_disk_log(Name,File,Type,MaxNoBytes,MaxNoFiles) ->
case filelib:ensure_dir(File) of
ok ->
Size =
if Type==halt -> MaxNoBytes;
Type==wrap -> {MaxNoBytes,MaxNoFiles} %% <-------
end,
Opts = [{name, Name},
{file, File},
{size, Size},
{type, Type},
{linkto, self()},
{repair, false},
{format, external},
{notify, true},
{quiet, true},
{mode, read_write}],
case disk_log:open(Opts) of
```
Affects all file loggers (audit included):
```
% emqx_config_logger:update_log_handler/1 -> ok
iex(emqx@127.0.0.1)14> Config override: log.file.emqx_audit is updated, but failed to add handler: {handler_not_added,
{badarg,
[{size,
{infinity,
10}},
{type,
wrap},
{linkto,
<0.1952.0>},
{repair,
false},
{format,
external},
{notify,
true},
{quiet,
true},
{mode,
read_write}]}}
```
2023-09-26 09:04:44 -03:00
14983ec14a
chore(hooks): validate hookpoints and document hook callbacks
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-26 14:27:42 +03:00
5e6397b843
chore: remove result field for CLI audit logs
2023-09-26 19:24:35 +08:00
a73c3b8e1e
Merge remote-tracking branch 'ce/release-53' into audit-log-fix-2
2023-09-26 19:05:47 +08:00
e9785a6863
Merge pull request #11680 from thalesmg/audit-not-on-ce-r53-20230925
...
fix(audit): only support audit log on enterprise edition
2023-09-26 18:57:22 +08:00
7d810c2107
test: fix test case match pattern
2023-09-26 11:40:30 +02:00
d31bfc70fb
Merge pull request #11659 from zhongwencool/fix-listener-ssl-create-500
...
fix: create ssl listener return 500 crash
2023-09-26 11:38:07 +02:00
8cc626d33f
Merge pull request #11686 from lafirest/fix/sso_user_backend
...
fix: fix update SSO user && improve SSO user deletion commnad
2023-09-26 16:04:51 +08:00
56917fee5f
fix(sso): Improved SSO user deletion command
2023-09-26 14:15:41 +08:00
d1af81c86e
fix(sso): fix the failure to update SSO users
2023-09-26 12:14:45 +08:00
8135e27586
fix: redact sensitive data in SSO and LDAP
2023-09-26 11:06:47 +08:00
17206f8c75
Merge pull request #11662 from thalesmg/port-scan-check-proto-dist-m-20230922
...
chore: check ekka proto dist module type when resolving node address
2023-09-25 17:01:43 -03:00
5d212e1086
fix(audit): only support audit log on enterprise edition
...
Fixes https://emqx.atlassian.net/browse/EMQX-11039
2023-09-25 15:59:35 -03:00
0498e59c45
test: fix flaky test
2023-09-25 13:35:17 -03:00
95060302fd
fix: don't need to change audit log's level
2023-09-26 00:28:47 +08:00
ff7f37ccf5
test(cth): allow defining schema to load for app
2023-09-25 13:22:41 -03:00
b1f144ab8b
feat(session): add custom session timers mechanism
...
That are managed exclusively by the session implementation, unlike
common session timers that are managed by the channel itself.
2023-09-25 18:19:26 +03:00
1df8326fb8
Merge pull request #11677 from lafirest/fix/sso_cfg
...
adjust the config path for the SSO feature and improve the update logic
2023-09-25 23:00:46 +08:00
5e6996dc05
refactor: log formatter format mfa+line as m:f/a(line)
...
also improve json formatter when a field is iolist
2023-09-25 16:55:22 +02:00
6a557980e4
refactor(logger): ensure JSON log field order
...
Ensure that the log fileds in JSON format are ordered as
time, level, msg, mfal, ...
2023-09-25 16:55:22 +02:00
5f45ba50ff
refactor: delete log formatter config for audit log handler
...
It is using 'json' formatter, the template is useless
2023-09-25 16:55:22 +02:00
1fed38c248
fix(logger): write 'json' format logs as JSON
2023-09-25 16:55:22 +02:00
aea1e80290
feat: add 'format' as alias for log 'formatter'
2023-09-25 16:55:22 +02:00
9106eb92d2
Merge pull request #11675 from qzhuyan/fix/william/quic-respect-param-verify
2023-09-25 15:48:01 +02:00
806017ef90
chore: check ekka proto dist module type when resolving node address
...
Follow up to https://github.com/emqx/emqx/pull/11637#discussion_r1334462917
Fixes https://emqx.atlassian.net/browse/EMQX-10944
2023-09-25 10:31:50 -03:00
cfdb25b213
fix(sso): updates the SSO backend when the `[dashboard]` has updated
2023-09-25 20:30:47 +08:00
f1bc3b68b6
Merge pull request #11674 from id/0925-e5.3.0-alpha.2
...
e5.3.0 alpha.2
2023-09-25 14:21:46 +02:00
6f29bbf997
Merge pull request #11673 from HJianBo/redact_audit_log_from_cli
...
Redact audit log from CLI
2023-09-25 20:03:45 +08:00
81cf619f07
fix(ftconf): also mark `secret_access_key` key as sensitive
2023-09-25 14:52:57 +03:00
36d3a3a524
fix: bump to quicer 0.0.201
...
do not load cacertfile if verify_none
2023-09-25 13:02:01 +02:00
22193d273a
fix(quic): ignore undefined cacertfile
2023-09-25 12:57:55 +02:00
8c93c79b44
chore: e5.3.0-alpha.2
2023-09-25 12:52:45 +02:00
7d58f6c61e
Merge pull request #11668 from JimMoen/saml-login-redirect
...
fix: saml login acs redirect to dashboard overview
2023-09-25 18:11:03 +08:00
b970a34ee0
chore: redact some audit logs from CLI
2023-09-25 18:06:39 +08:00
89e15e9134
Merge pull request #11665 from HJianBo/typo-fixes
...
Typo fixes
2023-09-25 17:23:40 +08:00
1cd9df6461
Merge pull request #11669 from lafirest/fix/dashboard_cli
...
fix(sso): refactor the `admins` CLI
2023-09-25 16:57:18 +08:00
e63d484632
fix(sso): move the config path of the SSO feature to `dashboard.sso`
2023-09-25 15:23:52 +08:00
e1f1c64ecd
Merge branch 'release-53' into 0925-sync-r53-to-master
2023-09-25 08:12:45 +02:00
e94192d1fa
fix(sso): refactor the `admins` CLI
...
1. revert the opensource version
2. allow delete the SSO user via CLI
2023-09-25 13:53:01 +08:00
4f4868a46c
fix(sso): Disable access to `logout` endpoint by the `API key`
2023-09-25 10:31:38 +08:00
30862a94c6
chore: typo fixes
2023-09-24 20:19:05 +08:00
ad4fadc2fa
fix: saml login acs redirect to dashboard overview
2023-09-23 17:29:02 +08:00
f8d06614c0
chore: fix dialyzer warnings
2023-09-23 07:34:04 +08:00
1dddccb448
fix(saml): cert files cleanup when destroy
2023-09-22 22:49:08 +08:00
80a6c1150d
fix(saml): saml login reply role `viewer` as default
2023-09-22 22:39:02 +08:00
cc3e4e4dc5
fix(saml): drop cert and key content and return path
2023-09-22 22:37:04 +08:00
6349cd3910
fix(saml): sp sign request
2023-09-22 21:57:50 +08:00
2a8f3f9eaa
fix: saml xml metedata format
2023-09-22 21:50:43 +08:00
a318ad486a
refactor: behavior login/2 use all http request
2023-09-22 21:50:43 +08:00
9181ec844f
chore: split out sso_saml_api module
2023-09-22 21:50:43 +08:00
df94426ee3
chore: make static_check happy
2023-09-22 21:50:42 +08:00
ec0894ca0b
chore: update esaml vsn
2023-09-22 21:50:42 +08:00
4a26f63bd6
chore: fix bugs
2023-09-22 21:50:41 +08:00
47badc3181
chore: make dialyzer happy
2023-09-22 21:49:12 +08:00
1c78c6bf6d
chore: fix 500 crashes when backend not existed
2023-09-22 21:49:12 +08:00
b4fb5196cb
fix(sso): SSO management API 500
2023-09-22 21:49:12 +08:00
bba5cc44a8
fix: keep same API path style
2023-09-22 21:49:12 +08:00
8300cd42d4
fix: acl url ignore auth check
2023-09-22 21:49:12 +08:00
44836ef5ee
chore: bump esaml vsn to v1.1.1
2023-09-22 21:49:12 +08:00
13666fa9f9
refactor: avoid dynamic call
2023-09-22 21:49:12 +08:00
c9e0d4fc30
feat: saml integration for dashboard sso
2023-09-22 21:49:10 +08:00
d9466eef63
chore: fix Dashboard RBAC license and rebar.config
2023-09-22 21:19:45 +08:00
fd932c9bf7
Merge pull request #11660 from zmstone/0922-fix-audit-eval-command
...
fix(audit): make emqx eval command auditable
2023-09-22 14:02:47 +02:00
a34ab19d93
fix(audit): make emqx eval command auditable
2023-09-22 12:05:45 +02:00
0eed01abee
fix: create ssl listener return 500 crash
2023-09-22 17:34:19 +08:00
13b5e4dbc9
Merge pull request #11658 from lafirest/fix/sso_misc
...
Fix/sso misc
2023-09-22 14:44:39 +08:00
7286d15ca6
chore(sso): adjust the schema of the SSO LDAP backend
2023-09-22 13:48:20 +08:00
9e55ae240a
feat(sso): add `role` into the result of login endpoints
2023-09-22 13:48:07 +08:00
681e57dee6
fix(RBAC): allow read-only users to logout
2023-09-22 11:06:24 +08:00
d6935b6a67
feat: add port scan diagnostics to mria waiting for tables checks
...
Fixes https://emqx.atlassian.net/browse/EMQX-10944
Also updates ekka -> 0.15.15, mria -> 0.6.4
How to test
===========
1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop them in order.
3. Start only the first node that was stopped in the previous step.
4. Wait until the log is printed.
Or, more easily:
1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop all but one.
3. Run `mria_mnesia:diagnosis([]).` on that node.
Example output
==============
```
Check check_open_ports should get ok but got #{msg =>
"some ports are unreachable",
results =>
#{'emqx@172.100.239.4' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
4}],
status =>
bad_ports},
'emqx@172.100.239.5' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
5}],
status =>
bad_ports}}}
```
After one node is back:
```
Check check_open_ports should get ok but got #{msg =>
"some ports are unreachable",
results =>
#{'emqx@172.100.239.4' =>
#{ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
4}],
status => ok},
'emqx@172.100.239.5' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
5}],
status =>
bad_ports}}}
```
2023-09-21 14:29:01 -03:00
7cf60c5a91
chore: e5.3.0-alpha.1
2023-09-21 19:29:00 +02:00
acf4227fc6
test(session): fix quic testgroup in persistent session suite
...
Which broker after quicer 0.0.200 upgrade.
2023-09-21 20:52:04 +04:00
76f614e9c5
Merge pull request #11653 from thalesmg/fix-dash-rbac-dialyzer-error-m-20230921
...
chore: fix dialyzer errors on ce version
2023-09-21 13:06:09 -03:00
85a8c174d9
chore: fix dialyzer errors on ce version
...
```
apps/emqx_dashboard/src/emqx_dashboard.erl
Line 225 Column 17: The pattern {'error', 'unauthorized_role'} can never match the type {'error','not_found' | 'token_timeout'} | {'ok',binary()}
```
2023-09-21 11:07:31 -03:00
b563e30615
Merge pull request #11650 from keynslug/fix/simplify-takeover-suite
...
test(session): make testsuite trigger takeover logic consistently
2023-09-21 18:02:38 +04:00
beea1be9f0
Merge pull request #11599 from zhongwencool/audit-log
2023-09-21 18:39:23 +08:00
0a61d08a0f
Merge pull request #11651 from id/0921-sync-r52-to-master
...
sync r52 to master
2023-09-21 12:31:24 +02:00
3a5227198e
Merge pull request #11642 from qzhuyan/dev/william/quicer-0.0.200
...
quicer 0.0.200
2023-09-21 12:20:14 +02:00
dafd7c6085
chore: bump apps versions
2023-09-21 10:58:42 +02:00
21bb209fb1
feat: added support for auditing API and CLI activity in logs
2023-09-21 16:35:46 +08:00
105bebc250
chore: merge release-52 into master
2023-09-21 10:22:47 +02:00
a8f4b5bf86
test(session): make testsuite trigger takeover logic consistently
2023-09-21 11:53:40 +04:00
9a128517d8
fix(sso): use the correct way to mark the API as authorization-free
2023-09-21 15:52:05 +08:00
dfc14a39bc
test: add emqx_management to telemetry SUITE
2023-09-21 14:15:06 +08:00
2283db62ec
fix(sso): add a API with authorization-free to list running backends
2023-09-21 11:52:30 +08:00
123d31fa7d
Merge pull request #11640 from zhongwencool/ensure-destory-resource
...
fix: always return ok when remove local resource
2023-09-21 09:21:45 +08:00
9521e11d39
Merge pull request #11584 from zhongwencool/telemetry-error-on-windows
...
fix: telemetry report error on windows
2023-09-21 09:08:55 +08:00
b924fb618a
Merge pull request #11639 from lafirest/test/sso
...
test(sso): add test case for the integration of SSO with LDAP
2023-09-21 09:01:13 +08:00
1b29e4b189
Merge pull request #11641 from zhongwencool/slog-msg-unify
2023-09-21 07:32:39 +08:00
a2ddd9d5f5
fix(session): respect existing session even if expiry interval = 0
...
If the original connection had Session-Expiry-Interval > 0, and the
new connection set Session-Expiry-Interval = 0, the MQTTv5 spec says
that (supposedly) we still have to continue with the existing session
(if it hasn't expired yet).
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:54 +04:00
3945f08f8f
fix(sessds): try to ensure iterators are closed on destroy
2023-09-20 22:55:54 +04:00
21e82b9534
test(sessmem): make retry delivery testcase more involved
2023-09-20 22:55:54 +04:00
69889d14a3
fix(sessds): fix use of undefined types
2023-09-20 22:55:54 +04:00
c1583f7f9d
fix(ds): refine `topic()` type to describe parsed topics
...
And separate it from `topic_filter()` type, which describes parsed
topic filters.
2023-09-20 22:55:52 +04:00
9362ef6f73
test(sessmem): drop unnecessary nesting in testcase
...
Also get rid of sneaky binding assignment.
2023-09-20 22:55:25 +04:00
98706cd215
chore: ensure comments follow code style consistently
2023-09-20 22:55:25 +04:00
8670efbfa0
chore(chan): rename `Name` → `TimerName` for better readability
2023-09-20 22:55:25 +04:00
7a9916c84d
fix(sessds): convert ds iterator topics upon opening ds session
2023-09-20 22:55:25 +04:00
045d8b7f10
refactor(ds): reorder functions to improve readability
2023-09-20 22:55:24 +04:00
540ca6d60f
chore: drop few commented out and irrelevant pieces
2023-09-20 22:55:24 +04:00
9d145890cc
refactor(sessmem): pass log context as part of session event
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
adc29e15cc
refactor(session): make typespecsa and flow a bit more clear
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
e422f492ef
test(sessds): reuse and expand persistent session test suite
2023-09-20 22:55:24 +04:00
3383ae19a9
test(session): switch `emqx_persistent_session_SUITE` to cth tooling
2023-09-20 22:55:23 +04:00
7326ef550b
fix(sessds): make existing parts of persistent session impl work
2023-09-20 22:55:22 +04:00
e713fc38aa
feat(broker): reflect persisted messages in publish result
...
In order for callers to distinguish between silently dropped and
durably persisted message w/o matching subscribers.
2023-09-20 22:53:45 +04:00
f4953e719b
fix(cmproto): fix few typespecs
2023-09-20 22:53:45 +04:00
8af107e28d
test(ds): simplify cluster test setups
2023-09-20 22:53:45 +04:00
7c4f68dd3d
fix(session): make utility function names consistent
...
Before this commit behavior of `is_banned_msg/1` / `should_discard/1`
were actually the exact opposite of their names.
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:53:44 +04:00
2dae8020ec
refactor(cm): avoid deep indirection in `emqx_session_mem`
2023-09-20 22:53:44 +04:00
abeff0bc4f
chore(session): try to describe what happens after session takeover
2023-09-20 22:53:44 +04:00
45d44df11d
refactor(session): update eviction channel session logic
...
The changes partially reflect `emqx_channel` changes with respect to
in-memory session specific logic. The difference is that eviction
channel does not replay post-takeover, instead enqueues messages.
2023-09-20 22:53:44 +04:00
ab1c4c4222
refactor(session): drop `lookup/1` session API
...
Due to the fact it's not used for anything right now.
2023-09-20 22:53:44 +04:00
97881ff3ca
refactor(session): bring back common session timers
2023-09-20 22:53:43 +04:00
bf16417513
feat(session): introduce session implementation concept
2023-09-20 22:53:42 +04:00
780ca15298
chore: bump application versions
2023-09-20 22:52:14 +04:00
04731b7ef7
test(takeover): randomize messages for random natural ordering
2023-09-20 22:52:14 +04:00
f022c9b1a4
feat(emqx): add `emqx_inflight:fold/3` generic function
2023-09-20 22:52:14 +04:00
596ce157fd
refactor(exproto): make timer names equal to messages they send
...
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
57ae5b14f1
refactor(mqttsn): make timer names equal to messages they send
...
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
e4866adc2f
refactor(chan): make timer names equal to messages they send
...
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
cfb1bf1fa4
chore(emqx): drop remnants of former session persistence impl
...
1. It is not functional anyway.
2. It blocks `emqx_session` refactoring in a few places.
2023-09-20 22:52:13 +04:00
e43b87f18e
test(quic): update emqx_broker_SUITE
2023-09-20 20:20:18 +02:00
2965fa6fcb
Merge pull request #11613 from thalesmg/ds-keyspace-m-20230914
...
feat(ds): introduce keyspace concept
2023-09-20 14:21:27 -03:00
da626f6d29
feat(quic): bump to quicer 0.0.200 and emqtt 1.9.0
2023-09-20 16:43:15 +02:00
dd687d9582
fix: dialyzer warning
2023-09-20 22:41:26 +08:00
9e75ff88af
chore: apply suggestions from code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:15:39 +08:00
f789a7504f
chore: update apps/emqx/src/emqx_quic_connection.erl
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:00:44 +08:00
09aea47f40
chore: 5.2.1
2023-09-20 14:04:28 +02:00
21c37a3f14
fix(sso): adjust the schema of LDAP API && add more logs
2023-09-20 18:15:42 +08:00
2f1fa2e961
chore: unified slog message formatting to improve logging consistency
2023-09-20 18:13:00 +08:00
c26a18e949
fix: always return ok when remove local resource
2023-09-20 18:02:42 +08:00
b11c5d26ce
test(sso): add test case for the integration of SSO with LDAP
2023-09-20 16:25:19 +08:00
bc6edac45f
Merge pull request #11631 from lafirest/feat/dashboard_ldap
...
feat(dashboard): add SSO feature and integrate with LDAP
2023-09-20 11:40:42 +08:00
61311081d8
fix(sso): fix RBAC test errors && spellcheck
2023-09-20 10:42:17 +08:00
5ed5ac48ee
refactor: combine shard id and keyspace into a single value
2023-09-19 12:31:32 -03:00
5b00998050
fix(sso): fix invalid_dynamic_call
2023-09-19 22:47:48 +08:00
285e529766
fix(sso): fix sso errors found when manual test
2023-09-19 19:15:52 +08:00
65818740f3
Merge pull request #11622 from zmstone/0917-upgrade-gen_rpc-to-3.1.0
...
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-19 10:22:53 +02:00
a69c26b86d
chore: e5.2.1-alpha.3
2023-09-19 08:17:29 +02:00
2cddce5479
feat(dashboard): add SSO feature and integrate with LDAP
2023-09-19 14:10:29 +08:00
de1bc04843
chore: bump ekka to 0.15.14, mria to 0.6.3
...
Inclued ekka's updates:
- https://github.com/emqx/ekka/pull/214
- https://github.com/emqx/ekka/pull/215
- https://github.com/emqx/ekka/pull/216
- https://github.com/emqx/ekka/pull/217
mria's updates:
- https://github.com/emqx/mria/pull/160
- https://github.com/emqx/mria/pull/161
- https://github.com/emqx/mria/pull/162
2023-09-19 09:43:19 +08:00
a511088fd4
refactor: address review comments
2023-09-18 17:47:56 -03:00
9f8a7a96af
Merge pull request #11628 from zmstone/0918-upgrade-emqtt-to-1.8.7
...
0918 upgrade emqtt to 1.8.7
2023-09-18 21:56:04 +02:00
de93d71014
test: fix emqx_sahred_sub_SUITE:t_remote
...
prior to this fix, the peer ct node is actually node itself
2023-09-18 21:54:43 +02:00
99b60849b4
test: fix flaky test emqx_falpping_SUITE:t_conf_update_timer
2023-09-18 20:44:38 +02:00
0b0e322473
Merge pull request #11627 from SergeTupchiy/EMQX-10962-fix_hstreamdb-bridge-on-stop
...
fix(emqx_bridge_hstreamdb): fix resources cleanup in on_stop/2 cb
2023-09-18 19:57:07 +03:00
d1e5b69eac
test: increase wait time
2023-09-18 17:33:09 +02:00
4f1ec795eb
test: delete special handling of emqtt race condition
...
the race should have been fixed in emqtt 1.8.7
2023-09-18 17:27:34 +02:00
46557c3462
fix(emqtt): upgrade from 1.8.6 to 1.8.7
2023-09-18 17:04:23 +02:00
80cf60e9be
fix(emqx_bridge_hstreamdb): fix resources cleanup in on_stop/2 cb
...
fixes: EMQX-10962
2023-09-18 17:48:05 +03:00
45f0d0dfce
Merge pull request #11623 from zmstone/0917-upgrade-esockd-to-5.9.7
...
0917 upgrade esockd to 5.9.7
2023-09-18 16:30:54 +02:00
c868264f65
Merge pull request #11617 from zhongwencool/rule-engine-console-args
...
fix: keep builtin_console actions not complain args
2023-09-18 20:40:02 +08:00
8b6d5d4ca9
test: add test for build_console's args
2023-09-18 16:48:38 +08:00
5490807b20
chore: update apps/emqx_rule_engine/src/emqx_rule_engine_schema.erl
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-18 16:11:39 +08:00
f66d9e76fe
fix(emqx_channel): aovid logging frame errors as exceptions
2023-09-18 09:04:00 +02:00
6fe846bf0e
Merge pull request #11610 from lafirest/feat/rbac
...
feat(dashboard): add RBAC feature for Dashboard
2023-09-18 14:41:47 +08:00
ca6547c05b
fix(rbac): change default role and update changes
2023-09-18 11:40:35 +08:00
abbee4bc47
fix(esockd): upgrade esockd from 5.9.6 to 5.9.7
2023-09-17 19:44:04 +02:00
985f0e6994
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-17 19:10:13 +02:00
a9682c2989
Merge pull request #11605 from zmstone/0913-lower-severity-level-for-CMD-override-warnings
...
0913 lower severity level for cmd override warnings
2023-09-16 11:32:07 +02:00
ac41f497b4
feat: add bytesize rule sql function
2023-09-15 16:04:21 +02:00
2aae3856a3
Merge pull request #11596 from thalesmg/fix-typespec-typos-m-20230912
...
fix: fix typespec typos
2023-09-15 09:24:42 -03:00
f0d2b2e6c3
chore: update emqx_eviction_agent version
2023-09-15 13:24:38 +03:00
4b97d3f57d
test(rbac): add test cases for RBAC && update changes
2023-09-15 18:16:28 +08:00
1959e15703
chore: simplify session eviction for node rebalance
2023-09-15 11:46:51 +02:00
32d838c85b
fix: format console action crash
2023-09-15 17:40:56 +08:00
640dd130fa
fix: don't urldecode twice clientid
2023-09-15 17:36:48 +08:00
d72348c8ed
fix: keep builtin_console actions not complain args
2023-09-15 16:38:04 +08:00
45ccc66474
fix(dashbaord): fix test cases to support RBAC
2023-09-15 15:49:37 +08:00
aace9215a4
Merge pull request #11608 from lafirest/feat/ldap_bind
...
feat(ldap): integrate authentication with LDAP bind operation
2023-09-15 14:06:39 +08:00
0846939760
fix(ldap): remove unused code and mark sensitive field
2023-09-15 10:13:07 +08:00
b30bcf32bd
feat(ds): introduce keyspace concept
...
Fixes https://emqx.atlassian.net/browse/EMQX-10579
This introduces the concept of "keyspaces" to our durable storage (DS) implementation, and
also refactors some places where "shard" and "keyspace" would be mixed up.
We might want to tune the storage options differently for distinct sets of topics, the
keyspaces. The keyspace is composed by one or more shards.
- Keyspaces are identified simply by binary strings.
- DS configuration is scoped by keyspaces instead of shards.
- Starting a new DS shard requires definining to which keyspace the shard belongs.
2023-09-14 16:39:00 -03:00
11c39c4b6a
chore: simplify session eviction for node rebalance
2023-09-14 18:28:25 +03:00
6be0e14e8c
fix: fix typespec typos
2023-09-14 10:21:05 -03:00
e41f7dd68c
Merge pull request #11598 from thalesmg/ds-fix-ps-router-m-20230912
...
fixes and improvements to persistent session router
2023-09-14 10:00:58 -03:00
5c31c5ce76
feat(dashboard): add RBAC feature for Dashboard
2023-09-14 19:28:37 +08:00
afbf13b8a2
feat(ldap): integrate authentication with LDAP bind operation
2023-09-14 14:10:43 +08:00
3eb46be2ae
fix(banned): fix typos
2023-09-14 14:01:59 +08:00
95f1312df7
chore(emqx_ctl): lower command override log from warning to info
...
prior to this change this warning message is always logged if a node
joins cluster.
the intention of this warning is to detect potential command name
clash (provided by different plugins), so arguably it should be
at debug level
2023-09-13 20:52:08 +02:00
803b69d878
refactor: use more consistent return types
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2023-09-13 10:49:45 -03:00
f55f6dc779
docs(ps_session): add notes explaining rationale about operation order and consistency
2023-09-13 10:12:27 -03:00
e081abb9e6
perf(ps_router): only check if there are any routes when deciding whether to persist message
2023-09-13 10:02:39 -03:00
cae9ae1fab
fix(ps_router): use `disc_copies` for storing persistent session routes
2023-09-13 10:02:39 -03:00
ea73d674b9
chore: bump version to e5.2.1-alpha.2
2023-09-13 14:55:17 +02:00
69d17f981d
Merge pull request #11595 from ieQu1/dev/dont-prevent-overlapping-partitions
...
fix(distribution): Set prevent_overlapping_partitions to false
2023-09-13 14:22:34 +02:00
e1456c1c04
feat(emqx_conf_schema): add node.default_bootstrap_batch_size config parameter
2023-09-13 10:11:14 +03:00
af015bc2bb
chore(ekka): bump version to 0.15.13
2023-09-13 10:11:14 +03:00
b41310af9c
fix: Add prevent_overlappiing_partitions to vm_args
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-12 20:01:56 +02:00
1a86c21e20
fix(distribution): Set prevent_overlapping_partitions to false
2023-09-12 17:41:51 +02:00
6ef296268e
chore: bump version to e5.2.1-alpha.1
2023-09-12 15:24:02 +02:00
59b94788bf
refactor: address review comments
2023-09-11 16:25:34 -03:00
64fcdb671d
test: fix a bunch of inter-suite interferences
2023-09-11 16:25:34 -03:00
f1294736b7
feat(ds): add filter for message persistence
...
Fixes https://emqx.atlassian.net/browse/EMQX-10520
2023-09-11 16:25:34 -03:00
fe4640922d
feat(ds): add delete callback
2023-09-11 16:25:33 -03:00
7805cc8c9b
fix(session): check if session is persistent
2023-09-11 16:25:33 -03:00
f490da71cd
chore: prepare v5.2.0-build.1
2023-09-11 16:09:17 -03:00
d0f1c75651
chore: bump apps vsns
2023-09-11 16:06:53 -03:00
223d47a6ac
Merge branch 'master' into sync-m-r52-20230911
2023-09-11 16:04:28 -03:00
063913f245
fix(session): remove recently added `iterators` field from `#session{}` record
...
Fixes https://emqx.atlassian.net/browse/EMQX-10945
This could lead to `badrecord` errors being raised if a takeover were to happen during a
rolling cluster upgrade, as the old nodes could receive a record with more fields than
expected.
2023-09-11 14:46:12 -03:00
887a74a6cb
fix: telemetry report error on windows
2023-09-11 16:48:30 +08:00
7010ac358c
chore: bump release version to v5.2.0
2023-09-08 16:31:52 +02:00
3d133801af
fix(router): use `ets:new/2` options compatible with OTP 24
...
While essentially keeping the same effect.
2023-09-08 18:04:29 +04:00
e089fda260
Merge pull request #11568 from thalesmg/republish-props-m-20230905
...
feat(republish): allow templating mqtt properties
2023-09-07 16:43:00 -03:00
f3bebb0ba2
Merge remote-tracking branch 'origin/master' into 0907-sync-master-to-release-52
2023-09-07 18:51:30 +02:00
68293231b8
test: add cases for more complex placeholders
2023-09-07 13:40:03 -03:00
014bc64d3b
chore: ensure general prop is a binary
2023-09-07 13:40:03 -03:00
02b8bbf76a
chore: drop templating support for `Subscription-Identifier`
...
This is highly-dependent on the session state, as is `Topic-Alias`.
2023-09-07 13:40:03 -03:00
47bd19dee4
chore: lower error log messages down to debug
2023-09-07 13:40:03 -03:00
080cb73da1
fix: handle badmap inside `emqx_placeholder:proc_tmpl`
2023-09-07 13:40:03 -03:00
db2ec55aa0
chore: bump release version to official e5.2.0
2023-09-07 09:40:15 +02:00
7c2f87fabe
test: merge broker and router boot modules
2023-09-06 21:36:16 +02:00
bb55b04d46
refactor: inline fn clauses, improve error logging
2023-09-06 13:54:40 -03:00
c9100c7591
refactor: improve logging information and descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-09-06 13:33:59 -03:00
78c5a779d7
feat(republish): allow templating mqtt properties
...
Fixes https://emqx.atlassian.net/browse/EMQX-10912
2023-09-06 09:54:02 -03:00
7fe704b6dd
Merge pull request #11563 from kjellwinblad/kjell/fix/release-52/EMQX-10775
...
fix: update gpb library to fix type error
2023-09-06 14:47:23 +02:00
18c6bfec97
chore: bump app vsns
2023-09-06 11:12:56 +02:00
e794143ae1
Merge remote-tracking branch 'origin/release-52' into 0906-sync-release-52-to-master
2023-09-06 09:08:22 +02:00
954b73a9aa
Merge pull request #11533 from thalesmg/ds-unsubscribe-m-20230828
...
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
2023-09-05 09:47:17 -03:00
01a128878f
chore(mria): Bump mria to 0.6.1
2023-09-05 11:08:32 +02:00
d26995a2aa
Merge pull request #11524 from keynslug/ft/EMQX-10713/unified-route-tab
...
feat(router): add routing storage schema with 2 tables
2023-09-05 12:38:00 +04:00
2b2e84b83c
fix: update gpb library to fix type error
...
This fixes a bug in the protobuf schema registry functionality. Before
this fix one would get a badarith error if one tried to assign a float
value to an uint64 field. However, this commit fixes this by upgrading
gpb so we instead will get a gpb_type_error which is what we want.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10775
2023-09-05 09:52:17 +02:00
23d63f5e01
refactor: fork clientid type for `emqx_ds:session_id/0`
2023-09-04 15:38:25 -03:00
60ae3c15c8
refactor: disregard impossible case
2023-09-04 15:38:25 -03:00
2cfe5ba057
fix(plugins_api): fix response types and test
2023-09-04 14:37:25 -03:00
02c3d6a0a7
fix(data_import): rm duplicate import call and fix test
2023-09-04 14:19:59 -03:00
73d4248334
test(cth_cluster): always allocate listener ports
2023-09-04 11:22:26 -03:00
9f0f218387
test(router): simplify cluster-related testcase in helper testsuite
2023-09-04 18:05:59 +04:00
7d26b7bc1a
fix(router): emit clearer error when conflicting schemas in use
2023-09-04 16:37:38 +04:00
d33ad8a46f
feat: cluster purge (r5.2)
...
Port of https://github.com/emqx/emqx/pull/11447
targeting `release-52`
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-09-04 09:15:28 -03:00
893f69617a
chore(test): drop obsolete TODOs
2023-09-04 14:40:49 +04:00
f0a0c7d4b1
fix(cthsuite): avoid duplicate application env entries
...
Because `application:set_env/1` complains loudly if there are
duplicates.
2023-09-04 14:40:49 +04:00
5024304bf9
fix(router): wait for tables replicate before choosing schema vsn
2023-09-04 14:40:48 +04:00
24230a64df
chore: bump app versions
2023-09-04 11:39:21 +02:00
b2f57636bd
Merge remote-tracking branch 'upstream/release-51' into 0904-sync-release-51
2023-09-04 11:34:22 +02:00
52546ac50f
Merge pull request #11522 from kjellwinblad/kjell/fix/EMQX-10778
...
fix: bad error message when rule engine schema name is too long
2023-09-04 11:05:55 +02:00
240afecd69
Merge pull request #11487 from savonarola/0821-optimize-bcrypt
...
feat: reduce bcrypt rounds to a usable value
2023-09-04 10:12:40 +03:00
ce571d892e
chore: ping opentelemetry v1.3.0-emqx
2023-09-04 09:34:09 +08:00
ed9afe3345
refactor(emqx_connector): use `hocon_schema:override/2` to make pgsql 'username' field required
2023-09-01 19:56:07 +03:00
607705518b
test(emqx_bridge_cassandra): add connector test case for Cassandra configured without authentication
2023-09-01 19:56:07 +03:00
f0c75d97e1
chore: upgrade emqx_http_lib to 0.5.3
...
0.5.3 release includes a fix to parse <host>:<port> URLs using the default http scheme.
2023-09-01 19:56:07 +03:00
c3b3460173
fix(emqx_oracle): make username a required field in emqx_oracle_schema
2023-09-01 19:56:07 +03:00
4ab5f8374b
chore(schema): mark routing storage schema readOnly
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-01 19:58:31 +04:00
ebf13c90ea
test(router): make sort stable in e2e routing testcase
2023-09-01 19:48:56 +04:00
90156befb5
feat: improve bcrypt usability
...
* limit salt rounds to usable values
* update bcrypt library to enable concurrent bcrypt hash calculation
2023-09-01 17:36:11 +03:00
c54527857a
fix(emqx_bridge_sqlserver): use empty password by default as it is not a required field
2023-09-01 16:57:35 +03:00
1bf86250dd
fix(emqx_bridge_cassandra): allow cassandra bridge without username/password
...
Cassandra can be used without credentials, if it is configured with AllowAllAuthenticator (default).
2023-09-01 16:57:35 +03:00
545f1c84a6
fix(cth): do not allocate ports for `emqx` app by default
...
This causes tricky and impressively hard to track side effects down
the line. Namely, loading `emqx_schema` _after_ `emqx_conf_schema`
(as part of cluster node startup sequence) leads to a couple of
schema root rewrites, because `emqx_schema` defines similar config
roots yet slightly differently (e.g. `authorization`).
2023-09-01 17:23:34 +04:00
0a879bffd1
test(router): add e2e testcase for cluster-wide routing
...
Which verifies that both router storage schemas work correctly.
2023-09-01 13:49:12 +04:00
eb0385a28f
chore(router): separate test-only function exports
2023-09-01 13:49:11 +04:00
3d859453f4
Merge pull request #11554 from lafirest/chore/exhook_prop
...
chore: shutdown reason will never be a UTF8 atom
2023-09-01 16:53:26 +08:00
3bb65e6b05
Merge pull request #11542 from kjellwinblad/kjell/fix/EMQX-10775
...
fix: update gpb library to fix type error
2023-09-01 10:07:03 +02:00
53940754b7
chore: shutdown reason will never be a UTF8 atom
2023-09-01 14:17:33 +08:00
17fe406605
Merge pull request #11551 from paulozulato/refactor-connector-config-callback
...
refactor: remove BridgeName from connector_config callback
2023-08-31 17:05:49 -03:00
bf0f77b6dc
Merge pull request #11548 from thalesmg/fix-plugin-order-update-r52-20230830
...
fix(plugins): update plugin order on whole cluster
2023-08-31 16:18:28 -03:00
7b607c3300
refactor: remove BridgeName from connector_config callback
...
As `bridge_name` is already in the Config parameter, this callback
doesn't need to have it on API anymore.
Fixes https://github.com/emqx/emqx/pull/11546/files#r1310778734
2023-08-31 11:01:49 -03:00
c20ba0572a
chore: bump to 5.2.0-alpha.4
2023-08-31 15:47:39 +02:00
876d539336
Merge remote-tracking branch 'andrew/ft/EMQX-10713/unified-route-tab' into 0831-make-use-of-new-routing-table
2023-08-31 15:46:33 +02:00
fb094e1d47
test(router): avoid testsuite collisions
2023-08-31 13:36:01 +04:00
727fd296ee
Merge pull request #11546 from paulozulato/fix-kafka-producer-unique-name
...
fix(kafka): avoid producer name collision between Kafka and AEH bridges
2023-08-30 18:08:54 -03:00
4dfbc859f9
fix(plugins): update plugin order on whole cluster
...
Fixes https://emqx.atlassian.net/browse/EMQX-10879
2023-08-30 16:39:52 -03:00
ee77976424
fix(kafka): avoid producer name collision between Kafka and AEH bridges
...
Fixes https://emqx.atlassian.net/browse/EMQX-10860
2023-08-30 14:48:10 -03:00
2da604dfa4
test(mgmt): simplify flaky `t_verify_imported_mnesia_tab_on_cluster`
...
And also rest of the test setup in the suite.
2023-08-30 20:24:18 +04:00
b1defa29d7
feat(ruleeng): avoid storing whole rules in topic index
...
Because it doesn't really give any benefit, but wastes memory by
duplication.
2023-08-30 20:07:56 +04:00
6b9cb06334
fix(router): add / refine some comments and log messages
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-30 19:51:31 +04:00
922ca5e141
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
...
Fixes https://emqx.atlassian.net/browse/EMQX-9742
2023-08-30 11:14:28 -03:00
9c6dd30f44
feat(session): store iterator ids in session record
2023-08-30 11:13:55 -03:00
dfcede8794
fix: increment matched counter when bridge is unhealthy
...
Fixes https://emqx.atlassian.net/browse/EMQX-10767
2023-08-30 10:52:53 -03:00
f8923ab419
Merge pull request #11539 from keynslug/ft/cth-desc-workdir
...
feat(cthsuite): add function to determine workdir of testrun
2023-08-30 17:15:55 +04:00
1cab687153
Merge pull request #11540 from thalesmg/fix-cert-path-utf8-r52-20230829
...
fix(bridge): validate bridge name before attempting to convert certificates
2023-08-30 09:45:55 -03:00
8b3a9072f9
Merge pull request #11532 from lafirest/fix/frame_parse
...
fix(frame): improve some error reasons when parsing invalid packet
2023-08-30 19:10:45 +08:00
d911f7fbea
fix: update gpb library to fix type error
...
This fixes a bug in the protobuf schema registry functionality. Before
this fix one would get a badarith error if one tried to assign a float
value to an uint64 field. However, this commit fixes this by upgrading
gpb so we instead will get a gpb_type_error which is what we want.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10775
2023-08-30 11:12:33 +02:00
9fe4382d98
Merge pull request #11530 from JimMoen/chore-bump-hstreamdb-driver
...
chore: bump hstreamdb_erl driver to `0.4.5+v0.16.1`
2023-08-30 14:37:17 +08:00
804443ba40
chore: fix typos
2023-08-30 11:22:09 +08:00
06fad14d07
Merge pull request #11536 from zhongwencool/comma_separated_binary
...
chore: ciphers should allow empty space(comma_separated_binary)
2023-08-30 10:20:06 +08:00
2ef60db147
Merge pull request #11481 from savonarola/0817-simplify-async-responses
...
chore(ft): refactor async reply mechanism
2023-08-30 01:20:09 +03:00
54ac4a8527
chore(ft): tidy up the code according to the review
2023-08-29 23:21:36 +03:00
0f297ffef4
fix(bridge): validate bridge name before attempting to convert certificates
...
Fixes https://emqx.atlassian.net/browse/EMQX-10865
2023-08-29 16:32:01 -03:00
0e770bdc95
test: switch `emqx_broker_SUITE` to use new cth tooling
2023-08-29 22:05:36 +04:00
3268093881
feat(cth): add module-level documenation
2023-08-29 22:05:36 +04:00
f57d16ba13
feat(cthsuite): add function to determine workdir of testrun
...
In a deterministic fashion, to lift the burden of undestanding where
the testrun's data should go from the test writer.
2023-08-29 21:34:01 +04:00
063d6200c8
feat(router): enable using 2 tables to store routes
...
Instead of a single unified table, to reap the benefits of cheap
`ets:lookup/2` per regular topic subscription route.
Change configuration option naming to reflect the change: user now has
an ability to choose _storage schema_.
2023-08-29 17:41:59 +04:00
270fd107b2
feat(triesearch): allow reusing filter tokenization result
...
As a kind of micro-optimization.
2023-08-29 17:31:26 +04:00
501456efec
chore: more safe with iolist_to_binary
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-29 17:10:43 +08:00
4e7ba5d35f
chore: ciphers should allow empty space(comma_separated_binary)
2023-08-29 11:34:40 +08:00
6722722522
fix(frame): improve some error reasons when parsing invalid packet
2023-08-29 10:53:20 +08:00
4245a4d8c6
fix: authz clean-cache clientid always return not_found
2023-08-29 08:17:47 +08:00
cc3ba18734
fix: increment dropped message counter when bridge is unhealthy
...
Fixes https://emqx.atlassian.net/browse/EMQX-10767
2023-08-28 19:47:11 -03:00
2c89be04d1
Merge pull request #11527 from paulozulato/fix-kafka-header-template-handling
...
fix(kafka): fix template processing for header
2023-08-28 18:55:33 -03:00
0b86f04bae
fix(kafka): fix template processing for header
...
Fixes https://emqx.atlassian.net/browse/EMQX-10846
2023-08-28 17:28:05 -03:00
279895b8fd
chore(ft): add read/write concurrency to emqx_ft_async_reply tabs
2023-08-28 22:46:30 +03:00
c04f770118
Merge pull request #11506 from SergeTupchiy/EMQX-10274-do-not-download-empty-trace-log
...
fix(emqx_trace): don't download empty trace log file
2023-08-28 22:17:00 +03:00
0072749143
fix(emqx_trace): don't download empty trace log file
...
Closes: EMQX-10274
2023-08-28 19:25:55 +03:00
5908b69353
chore: hide cluster purge from cli and api for now
2023-08-28 10:23:21 -03:00
f988de4ff4
feat(purge): clear delayed messages
2023-08-28 10:23:21 -03:00
9bded07834
feat: cluster purge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-08-28 10:23:20 -03:00
dec21ffc95
Merge pull request #11362 from thalesmg/ds-handle-sub-20230725
...
feat(ds): open iterators when handling `SUBSCRIBE` packets
2023-08-28 10:21:28 -03:00
2d931a0512
test(evict): do not disable router module in cluster nodes
2023-08-28 14:07:46 +04:00
8d2ebdea7e
fix(router): generalize config option and make effects visible
2023-08-28 14:07:46 +04:00
e85789306b
chore(router): drop test-only `has_routes/1`
...
Seems that it's too much support work only for test purposes, where
`lookup_routes/1` is nearly as usable.
2023-08-28 14:07:46 +04:00
5d51687dbf
feat(router): add unified routing table config option
2023-08-28 14:07:45 +04:00
33e5e1ba57
feat(router): add unified routing table
2023-08-28 14:07:44 +04:00
166375a000
fix(topicidx): make `get_record/2` simpler to use in concurrent env
...
The mechanic of `emqx_topic_index` cannot really guarantee atomicity
of reading records associated with index matches, so instead it's
probably better to make the user aware of that lack of this guarantee.
2023-08-28 13:52:59 +04:00
84e40fb6fe
test(trie): add more involved route add + delete case
...
That shows how current local trie implementation breaks because
of lack of refcounting.
2023-08-28 13:52:59 +04:00
dcb63440bc
refactor(mgmt): avoid dealing with router tab directly
...
Instead, contain all this behind `emqx_router` module interface.
2023-08-28 13:52:59 +04:00
9eccfa0909
refactor(router): isolate cleanup logic in router module
2023-08-28 13:52:59 +04:00
f3467f44b2
chore: bump hstreamdb_erl driver to `0.4.5+v0.16.1`
2023-08-28 17:28:50 +08:00
4488e9e591
chore(ft): stop hook chain when doing actual handling
2023-08-28 11:57:44 +03:00
39a48179ea
chore(emqx_channel): use macros for reply construction
2023-08-28 11:57:44 +03:00
b8cacd2833
chore(ft): add tests for async reply registry
2023-08-28 11:57:44 +03:00
b74ff10705
Merge pull request #11517 from keynslug/ft/topic-index-v3
...
feat(topicidx): iterate on trie search implementation
2023-08-28 12:24:16 +04:00
053b7fb94a
test: add more test for conf_cli
2023-08-28 11:14:36 +08:00
d8be248a3d
fix: improve the suggest msg for update conf failed
2023-08-28 08:53:23 +08:00
b0d4a22aa8
chore(ft): refactor async reply mechanism
2023-08-27 21:54:23 +03:00
b874926de7
Merge pull request #11514 from zhongwencool/stop-load-application-before-reboot
...
fix: stop otel deps appication before reboot
2023-08-27 20:46:31 +08:00
2bb415f897
Merge pull request #11526 from paulozulato/fix-validator-return-msg
...
fix: flatten error message on resource validator
2023-08-25 17:27:31 -03:00
33a0048155
refactor: move logic to `ensure_iterator`
2023-08-25 15:24:53 -03:00
84e68f8ed8
Merge pull request #11508 from paulozulato/fix-azure-msg-error-handling
...
fix(kafka): fix result handling when sending message with invalid header
2023-08-25 14:57:09 -03:00
42877e282d
fix: flatten error message on resource validator
...
Fixes https://emqx.atlassian.net/browse/EMQX-10864
2023-08-25 13:53:52 -03:00
f15f59650d
test: rm obselete workaround code
2023-08-25 13:49:33 -03:00
cb1e105e19
Merge pull request #11516 from paulozulato/fix-kafka-aeh-client-unique-id
...
Fix Kafka/AEH ClientId uniqueness
2023-08-25 10:14:44 -03:00
d5cff533e3
chore(topicidx): drop TODO comment
2023-08-25 16:44:03 +04:00
ca59a87d47
chore(topicidx): refine example of wildcard compare
2023-08-25 16:44:03 +04:00
558402a68e
chore(topic_index): add topic validation
2023-08-25 16:44:03 +04:00
33ed53bb6a
refactor(topicidx): simplify `compare/3` further
...
It's possible to emit seek instruction just once, on the way back
out of the `compare/3` stack.
2023-08-25 16:34:45 +04:00
fc37d235c7
refactor(topicidx): simplify seek instructions
...
Which also avoids comparing filter tail repeatedly when evaluating
it.
2023-08-25 15:24:06 +04:00
c65db82b07
fix: bad error message when rule engine schema name is too long
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-10778
2023-08-25 11:02:38 +02:00
c8af5a702c
test: remove connection_closed from test
2023-08-25 15:53:35 +08:00
fc1738188e
fix: packets_connack_sent is not incremented if the ack_flag field in the CONNACK packet is non-zero
2023-08-25 15:18:34 +08:00
ba9cb0a1ed
Merge pull request #11512 from lafirest/fix/ldap_def_port
...
fix(ldap): expose request_timeout to prevent infinite hang
2023-08-25 10:22:44 +08:00
8ab6eb5638
test: start emqx_opentelemetry in emqx_machine SUITE
2023-08-25 10:09:00 +08:00
cf45e80c71
feat(topicidx): iterate on trie search implementation
...
This improves matching performance and decreases GC pressure on
synthetic workloads.
2023-08-25 01:47:11 +04:00
535c7f8b43
fix(kafka): avoid ClientId collision between Kafka and Azure bridges
...
Fixes https://emqx.atlassian.net/browse/EMQX-10860
2023-08-24 18:46:22 -03:00
d9a5a9ea2b
Merge pull request #11513 from thalesmg/kafka-fix-ts-template-r52-20230824
...
fix: use correct timestamp template for kafka and remove such fields from azure event hub producer
2023-08-24 16:26:52 -03:00
016ae0524f
fix(aeh_producer): remove timestamp template field
...
Fixes https://emqx.atlassian.net/browse/EMQX-10847
Checking the whole Kafka message from AEH, it seems like the timestamp type is append,
which means that it’s the broker who controls the timestamp, and the timestamp defined by
the producer is ignored.
Ref: https://kafka.apache.org/documentation/#brokerconfigs_log.message.timestamp.type
Example message consumed from AEH:
```
%{
"headers" => %{},
"key" => "",
"offset" => 4,
"topic" => "test0",
"ts" => 1692879703006,
"ts_type" => "append",
"value" => "{\"username\":\"undefined\",\"topic\":\"t/aeh/produ\",\"timestamp\":1692879692189,\"qos\":0,\"publish_received_at\":1692879692189,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaa\",\"node\":\"emqx@127.0.0.1\",\"metadata\":{\"rule_id\":\"rule_aehp\"},\"id\":\"000603AA44B34E08F4AF000006E30003\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```
Note the ts_type above is append.
Example message from a Kafka broker whose ts type is create:
```
%{
"headers" => %{},
"key" => "",
"offset" => 4,
"topic" => "test-topic-three-partitions",
"ts" => 1692881883668,
"ts_type" => "create",
"value" => "{\"username\":\"undefined\",\"topic\":\"t/kafka/produ\",\"timestamp\":1692881883668,\"qos\":0,\"publish_received_at\":1692881883668,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaaaa\",\"node\":\"emqx@127.0.0.1\",\"id\":\"000603AAC7529FEEF4AC000007050000\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```
Unfortunately, I couldn’t find anywhere in AEH where that configuration could be changed.
2023-08-24 14:20:42 -03:00
0381ac0410
fix(kafka_producer): use correct timestamp template field
...
Fixes https://emqx.atlassian.net/browse/EMQX-10847
2023-08-24 14:20:42 -03:00
9fe5080705
build: bump versions before release
2023-08-24 16:46:27 +02:00
c215fe3736
fix: stop otel deps appication before reboot
2023-08-24 22:14:14 +08:00
60e6217496
fix(kafka): fix result handling when sending message with invalid header
...
Fixes https://emqx.atlassian.net/browse/EMQX-10846
2023-08-24 10:41:42 -03:00
e5041de9cc
fix(gcp_consumer): handle 401 errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-10852
2023-08-24 09:03:34 -03:00
62423b0b12
refactor(topic_index): less special handling for leading $ words
2023-08-24 13:30:10 +02:00
a30d87e14f
refactor(topic_index): remove more unnecessary next calls
...
also avoid using records (setelement) for recursive return values
2023-08-24 12:25:04 +02:00
a1e6635614
refactor(topic_index): no forced ceiling entry in index table
2023-08-24 12:25:04 +02:00
f4c8c6be55
refactor(topic_index): optimize trie-search performance
2023-08-24 12:25:04 +02:00
6b152b3cb7
test: add more debug output
2023-08-24 12:25:04 +02:00
ae094e363c
chore: fix a typo in log message
2023-08-24 12:25:04 +02:00
6085442958
fix(ldap): expose request_timeout to prevent infinite hang
2023-08-24 16:50:38 +08:00
2322b27542
Merge pull request #11497 from zhongwencool/prometheus-metrics
...
Prometheus metrics
2023-08-24 12:06:32 +08:00
8d2be27ade
chore: add changlog for 11466
2023-08-24 06:46:50 +08:00
84dd3b83bb
fix: hocon converter not working when make_serializable is true(check_plain)
2023-08-24 06:46:47 +08:00
f72ba6819f
Merge pull request #11498 from zhongwencool/fix-otel-stop-error-log
...
fix: stop otel deps application in terminate crashed
2023-08-24 06:40:24 +08:00
d287b736ec
chore: add changelog for 11497
2023-08-23 17:02:30 +08:00
58be7f709b
fix: stop otel deps application in terminate crashed
2023-08-23 16:45:41 +08:00
6bb15541c7
feat: rename olp metrics to overload_protection metrics
2023-08-23 15:25:54 +08:00
b9a97923c4
feat: add some authz/connection missing metrics
2023-08-23 14:47:53 +08:00
141676185d
feat: add authentication metrics to prometheus
2023-08-23 14:15:16 +08:00
4928c69758
feat: add authorization metrics to prometheus
2023-08-23 14:11:59 +08:00
df9f8da178
feat: add overload_protection metrics to prometheus
2023-08-23 14:07:37 +08:00
188a81913e
test: add prometheus api test
2023-08-23 11:23:43 +08:00
ae10415fc3
feat: disabled the Erlang VM Prometheus exporter by default to improve performance and security
2023-08-23 09:03:22 +08:00
61536b5362
Merge pull request #11494 from paulozulato/fix-kinesis-validators
...
fix(kinesis): set validators for static constraints
2023-08-22 18:28:35 -03:00
20b46091f9
fix(kinesis): set validators for static constraints
...
Fixes https://emqx.atlassian.net/browse/EMQX-10833
2023-08-22 14:42:44 -03:00
f007b44426
fix(data_import): rm duplicate import call and fix test
2023-08-22 13:29:15 -03:00
ee2897e5de
test(refactor): move test to integration tests dir
2023-08-22 13:29:15 -03:00
c1f49abad2
test: fix inter-suite flakiness
2023-08-22 13:29:15 -03:00
dad27091be
test: rm custom option
2023-08-22 13:29:15 -03:00
dbfacae283
fix: reinstate transactions
2023-08-22 13:29:15 -03:00
65085d012b
refactor: rename fn
2023-08-22 13:29:15 -03:00
3344bfb0bd
refactor: rm `emqx_ds_replay`
2023-08-22 13:29:15 -03:00
c46b8de938
test: remove unused things, refactor some functions
2023-08-22 13:29:15 -03:00
e8c73b06e1
docs: add comment about future test failure
2023-08-22 13:29:15 -03:00
6de0bbe76a
test(refactor): always allocate listeners for emqx app
2023-08-22 13:29:15 -03:00
a15405a800
test: fix assertions
2023-08-22 13:29:15 -03:00
c74abe79d0
refactor: reduce arity
2023-08-22 13:29:15 -03:00
021755b82b
refactor: rm iterators from DS `#session{}` record
2023-08-22 13:29:15 -03:00
e4e88ebf36
test: add scenario for node stopping midway during subscribe
2023-08-22 13:29:15 -03:00
3239f5ac5b
feat: rm unnecessary transactions, use separate table for iterator references
2023-08-22 13:29:15 -03:00
8eab389ae1
perf: avoid unnecessary transaction
2023-08-22 13:29:15 -03:00
c28c6d1b7e
fix: ensure iterator is opened
2023-08-22 13:29:15 -03:00
33ddbe80ad
refactor: remove persistence leakeage from emqx_cm level
2023-08-22 13:29:15 -03:00
e8d7bb9a67
refactor: rename module
2023-08-22 13:29:15 -03:00
Zaiming (Stone) Shi