Commit Graph

12372 Commits

Author SHA1 Message Date
lafirest
afd4b46f72
Merge pull request #13225 from lafirest/fix/mysql_redact
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-14 09:19:21 +08:00
Thales Macedo Garitezi
3adf64e637
Merge pull request #13240 from thalesmg/ds-cluster-purge-m-20240612
feat: support purging durable sessions during cluster purge
2024-06-13 09:30:18 -03:00
William Yang
1664ea4ad4 Revert: TLS partial-chain and keyUsage #12955 #12977
This reverts commit 28b17a2562.
This reverts commit 01467246fc.
This reverts commit c3f8ba5762.
This reverts commit 1a4a4bb3a5.
This reverts commit fb30207ef3.
This reverts commit 337c230e79.
This reverts commit 3a674f44f1.
This reverts commit 70ffd77f99.
This reverts commit 03b0935564.
This reverts commit 650cf4b27e.
This reverts commit 43ad665dcf.
This reverts commit a29a43e5fc.
This reverts commit 4e9c1ec0c9.
This reverts commit 8eb463c58d.
This reverts commit 90430fa66d.
This reverts commit eb1ab9adfe.
This reverts commit 8bc3a86f63.
This reverts commit fa4357ce89.
This reverts commit 0b95a08d32.
2024-06-13 13:46:45 +02:00
William Yang
723144199e
Merge pull request #13211 from qzhuyan/fix/william/rel57/ee-tls
Fix/william/rel57/ee tls
2024-06-13 13:19:44 +02:00
Andrew Mayorov
0f53b33417
chore(s3): make schema fields with defaults optional 2024-06-13 11:57:36 +02:00
zhongwencool
0eee2edd28 test: fix api_config SUITE failed 2024-06-13 14:55:24 +08:00
zhongwencool
2b1fa1711b fix: observer command crash when can't find object code 2024-06-13 14:55:23 +08:00
zhongwencool
623845d07d fix: don't allow set active_n to negative int 2024-06-13 14:55:23 +08:00
zhongwencool
825bfe4717 fix: load bad configs return unknown msg 2024-06-13 14:55:23 +08:00
zhongwencool
5d3f464ec3 fix: ws/wss's max_frame_size should > 0 2024-06-13 14:55:23 +08:00
zhongwencool
800a79adde test: test failed with_mfa 2024-06-13 09:50:52 +08:00
zhongwencool
c72b455a83 feat: add hidden option to record MFA and line in the log is very useful for debug. 2024-06-12 22:51:45 +08:00
Thales Macedo Garitezi
d0e6f22a79 feat: support purging durable sessions during cluster purge
Fixes https://emqx.atlassian.net/browse/EMQX-12405
2024-06-12 11:19:53 -03:00
JimMoen
daeb10151b
chore: fix typos 2024-06-12 19:06:34 +08:00
JimMoen
cde4cb1358
fix: metrics macro with description 2024-06-12 19:06:33 +08:00
JimMoen
756797b258
refactor: gen metric schema with desc from macros 2024-06-12 19:04:39 +08:00
JimMoen
263d2dbae2
fix: inc metric 'client.auth.anonymous' when client anonymous 2024-06-12 19:04:39 +08:00
JimMoen
9a78a6f640
refactor: mv metrics macros in hrl file 2024-06-12 19:04:29 +08:00
JimMoen
cbac4019b3
fix: rm havn't used counter idx 2024-06-12 19:01:50 +08:00
zmstone
8a283b1cc0
Merge pull request #13180 from zmstone/0604-optimize-tcp-ssl-connection-data-send-performance
0604 optimize tcp ssl connection data send performance
2024-06-11 22:06:57 +02:00
zmstone
51810c6919
Merge pull request #13229 from zmstone/0611-auth-with-raw-cert
feat(authn): support `${cert_pem}` placeholder
2024-06-11 21:16:01 +02:00
Thales Macedo Garitezi
db572d35a7 Merge remote-tracking branch 'origin/release-57' into sync-r57-m-20240611 2024-06-11 15:34:54 -03:00
zmstone
751f7a24e9 feat(authn): support ${cert_pem} placeholder 2024-06-11 19:54:24 +02:00
Thales Macedo Garitezi
337009c3a0 fix: declare emqx_schema_registry as a dependency of emqx_schema_validation 2024-06-11 14:03:27 -03:00
Thales Macedo Garitezi
0f9c3b4cea feat(schema validation): check references schema names and types before changing config
Fixes https://emqx.atlassian.net/browse/EMQX-12368
2024-06-11 14:03:26 -03:00
Thales Macedo Garitezi
03b226248a feat(schema registry): add check for inner types
Currently, only `protobuf` has any.
2024-06-11 13:50:11 -03:00
Thales Macedo Garitezi
b2d716909f
Merge pull request #13200 from thalesmg/update-erlazure-no-gen-server-m-20240606
chore: refactor azure blob connector to use new `erlazure` without `gen_server`
2024-06-11 13:30:14 -03:00
Thales Macedo Garitezi
686bcc8a48
Merge pull request #13207 from thalesmg/republish-metrics-m-20240607
fix(rule actions): check `republish` action result for metrics
2024-06-11 13:29:58 -03:00
Thales Macedo Garitezi
2c6aa80b3c
Merge pull request #13221 from thalesmg/refactor-sv-topic-index-r57-20240610
refactor(schema validation): module organization and position indexing
2024-06-11 13:29:48 -03:00
Thales Macedo Garitezi
83ff07ade0
Merge pull request #13199 from thalesmg/message-transformation-r57-20240604
feat: implement message transformation
2024-06-11 13:29:25 -03:00
Thales Macedo Garitezi
ab5329be02
Merge pull request #13220 from thalesmg/refactor-channel-hc-opt-r57-20240610
fix: remove another redundant health check
2024-06-11 12:22:43 -03:00
Andrew Mayorov
2f0eb3560b
Merge pull request #13227 from keynslug/fix/EMQX-12533/bad-tpl
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 15:39:12 +03:00
Andrew Mayorov
d4fb812352
fix(aggreg): anticipate and handle delivery startup errors 2024-06-11 13:21:41 +02:00
Andrew Mayorov
29fc30ea69
fix(bridge-s3): validate aggreg key template before adding a channel 2024-06-11 13:21:41 +02:00
Andrew Mayorov
fb0da9848c
feat(tpl): add separate placeholders/1 function
The purpose is to have a clearer view of placeholders used in a
template, without going the usual `render(Template, #{})` route that is
actually subtly misleading: it won't mention that `${}` / `${.}`
placeholder has been used.

Also unify handling of `${}` / `${.}` in a couple of places.
2024-06-11 11:42:43 +02:00
Kjell Winblad
9d22089104
Merge pull request #13173 from kjellwinblad/kjell/pgsql_timestamp_encode_etc/EEC-1036
fix(pgsql connector): handle prepared statement already exists
2024-06-11 10:53:30 +02:00
William Yang
44258204bd chore: move tls_certs test helper to apps/emqx 2024-06-11 10:06:18 +02:00
firest
e64f60b73f fix(auth): redact sensitive data for the authn/authz APIs 2024-06-11 16:05:11 +08:00
zhongwencool
b657dc537c
Merge pull request #13156 from savonarola/0529-fix-dashboard-monitor
fix(dashboard_monitor): correctly merge old and new sample data
2024-06-11 10:33:15 +08:00
Thales Macedo Garitezi
05ebb17cd6 refactor: index positions rather than names
Addresses https://github.com/emqx/emqx/pull/13199#discussion_r1633096025
2024-06-10 15:54:37 -03:00
Thales Macedo Garitezi
5eff4a7544 refactor(schema validation): improve api and module organization
To conform to comment from another PR:
https://github.com/emqx/emqx/pull/13199#discussion_r1632658742

... since schema validation is the "inspiration" for message transformation.
2024-06-10 15:46:28 -03:00
Thales Macedo Garitezi
d93c8540c8 feat: pass along peername down to transformation context 2024-06-10 15:35:35 -03:00
Thales Macedo Garitezi
6e0ef893f4 feat: pass along client attributes down to message transformation context 2024-06-10 15:27:04 -03:00
Thales Macedo Garitezi
3c501e4f2a fix: remove another redundant health check
Fixes https://emqx.atlassian.net/browse/EMQX-12521
2024-06-10 15:13:34 -03:00
Thales Macedo Garitezi
1a20a40218
Merge pull request #13209 from thalesmg/rm-redundant-hc-r57-20240607
chore: remove redundant health check call
2024-06-10 14:58:48 -03:00
Thales Macedo Garitezi
e586178479 chore: refactor azure blob connector to use new erlazure without gen_server
https://github.com/dkataskin/erlazure/pull/43 removes unnecessary usage of `gen_server`
from the driver.
2024-06-10 13:42:28 -03:00
Thales Macedo Garitezi
5629fe60c1 refactor: index positions rather than names 2024-06-10 13:33:22 -03:00
Thales Macedo Garitezi
8c7a7cf9db refactor: apply review remarks 2024-06-10 11:17:13 -03:00
Thales Macedo Garitezi
e54cf2f218 refactor: use option to skip rendering as string 2024-06-10 10:49:39 -03:00
Thales Macedo Garitezi
d34558954d refactor: improve api and module organization 2024-06-10 10:40:53 -03:00
Thales Macedo Garitezi
4aa1c3842f fix: declare emqx as a dependency in .app.src 2024-06-10 09:53:29 -03:00
Thales Macedo Garitezi
aa416cc0ad docs: fix stale comments and dates 2024-06-10 09:53:02 -03:00
zmstone
7a09b0470d chore: cosmetic changes 2024-06-10 14:22:25 +02:00
Kjell Winblad
a885f0b41a test(emqx_bridge_pgsql_SUITE): call test janitor 2024-06-10 13:45:27 +02:00
William Yang
a1aa9a4375 fix(ee): emqx no longer deps on emqx_auth_ext 2024-06-10 10:32:01 +02:00
zmstone
4347f3de3e fix(bridge/mqtt): respect client ID prefix 2024-06-10 10:27:45 +02:00
William Yang
76cfc309a9 feat(tls): partial_chain not required 2024-06-10 10:25:44 +02:00
William Yang
7c37bf9965 feat(tls): ee only: TLS partial_chain and Keyusage 2024-06-10 10:24:48 +02:00
William Yang
e60380d205 chore: fix nit 2024-06-10 09:55:12 +02:00
William Yang
e9b813d8ef chore: fix test 2024-06-10 09:54:54 +02:00
William Yang
1739bc0c24 feat(partial_chain): gateway support 2024-06-10 09:54:44 +02:00
zmstone
45dd7816d7 Merge remote-tracking branch 'origin/release-57' 2024-06-09 10:11:15 +02:00
William Yang
abbf2ef62f chore(TLS-chain-test): update for OTP 26 2024-06-08 00:16:06 +02:00
William Yang
221b748b0f test(partial_chain): update tcs for OTP-25 2024-06-08 00:16:01 +02:00
William Yang
463d1a1875 fix(test): tls_verify_partial_chain 2024-06-08 00:15:54 +02:00
William Yang
788cdbc6dd fix(listener): remove partial_chain in wss opts 2024-06-08 00:15:51 +02:00
William Yang
3fc99315e0 chore: happy elvis 2024-06-08 00:15:45 +02:00
William Yang
c5dccdf526 feat(tls): update schema for TLS keyusage 2024-06-08 00:15:43 +02:00
William Yang
aa25e3badd fix(tls): undefined keyusage 2024-06-08 00:15:39 +02:00
William Yang
9e5cfea8c0 test(tls): verify peer keyusage 2024-06-08 00:15:36 +02:00
William Yang
c5657029ab feat(config): partial_chain 2024-06-08 00:15:34 +02:00
William Yang
f7ff9496e6 test: port listener tls partial_chain 2024-06-08 00:15:31 +02:00
William Yang
1ce13242a8 feat(tls): port partial_chain, part 1 2024-06-08 00:15:28 +02:00
Kjell Winblad
c13631102e fix(PgSQL connector): typo in function name 2024-06-07 19:40:18 +02:00
Andrew Mayorov
b2f027c974
Merge pull request #13197 from keynslug/fix/EMQX-12522/conv-ssl
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-07 19:45:53 +03:00
Thales Macedo Garitezi
39615e1cb6 fix(rule actions): check republish action result for metrics
Fixes https://emqx.atlassian.net/browse/EMQX-12328
2024-06-07 11:17:43 -03:00
Thales Macedo Garitezi
28a3c77e88 chore: remove redundant health check call
Since `emqx_resource_manager:channel_health_check` already does a base resource health
check, the call was repeated.
2024-06-07 11:07:12 -03:00
Kjell Winblad
e63dcc84b0 fix: unused variable and better error message 2024-06-07 12:20:42 +02:00
Kjell Winblad
a3f3f96781 test(emqx_bridge_v2_pgsql_SUITE): fix broken test case
We have to delete the connector and action created by the previous step
of the test case so we don't get a name conflict.
2024-06-07 12:03:34 +02:00
JimMoen
78a196e215
Merge pull request #13189 from JimMoen/fix-apply-sqlserver-template
fix(utils_sql): improve insert sql regular expression
2024-06-07 13:50:19 +08:00
lafirest
eb215726d5
Merge pull request #13195 from lafirest/fix/typos
fix: fix typos
2024-06-07 09:04:16 +08:00
Thales Macedo Garitezi
ef36350bf6 feat: implement message transformation
Fixes https://emqx.atlassian.net/browse/EMQX-12517
2024-06-06 17:51:08 -03:00
Andrew Mayorov
8a85d7cb5a
fix(bridge-s3): pass SSL options through convert_certs/2 2024-06-06 19:37:38 +02:00
Andrew Mayorov
c17c788438
feat(connector): allow delegate pre_config_update to impls 2024-06-06 19:37:34 +02:00
zmstone
ebf17c8143 Merge remote-tracking branch 'origin/release-57' into 0606-merge-release-57-to-master 2024-06-06 17:43:36 +02:00
firest
3ae26c8a54 chore: update changes 2024-06-06 18:13:24 +08:00
firest
5532c7b0a6 fix(authz_mnesia): add a soft limit in the API for the length of ACL rules 2024-06-06 16:22:53 +08:00
zmstone
2525bd0c16 perf: eliminate the loop-back fake async inet_reply message
Now that EMQX is on OTP 26 by default, the fake inet_reply
message when the send call returns ok only adds overhead.

For TCP (gen_tcp module), OTP 26 no longer supports direct port_command
so we had to swith to gen_tcp:send, then fake an async reply:
see https://github.com/emqx/esockd/pull/181

For TLS (ssl module), it has never supported async send anyway.

This commit switches to call Transport:send/2 directly,
this should result in one less loop-back message for each message.

For OTP 25 on which EMQX can still be compiled, one will have to
suffer the performance penalty for gen_tcp:send function to
do a non-optimized 'receive' for the send result.
2024-06-06 09:35:51 +02:00
zmstone
5c3f5d8085 perf: do not call inet getstat before each and every send
In a stress test environment, when alarm is enabled, there were
a lot of long_schedule warnings with the stacktrace pointing to
congestion alarm based inet:getstat function.
When alarm is disabled, the one single client is able to
hold 40,000 QoS 1 messages per second (inflight=32) throughput.
When alarm is enabled, its mqueue overflows.

This commit removes the call before each and every data send,
so to rely on emit_stats timer to trigger the congestion check.
2024-06-06 09:35:51 +02:00
firest
d4118c6e8e fix: fix typos 2024-06-06 11:19:40 +08:00
JimMoen
dff31b293c
perf(utils_sql): pre-build regexp 2024-06-06 11:01:38 +08:00
zmstone
c41be7ee22 chore: fix dialyzer warning 2024-06-05 23:37:21 +02:00
Ilia Averianov
dc2e6d1695
Merge pull request #13163 from savonarola/0523-ds-shared-subs-dispatch
Inject shared subcription handling into durable session
2024-06-05 23:59:50 +03:00
Serge Tupchii
159da912c2 feat(emqx_dashboard_swagger): apply field validator in request body check 2024-06-05 19:35:04 +03:00
JimMoen
144264e0d8
test: add utils_sql test cases 2024-06-06 00:30:18 +08:00
JimMoen
627b8c45d2
fix(utils_sql): improve insert sql regular expression 2024-06-06 00:30:17 +08:00
Ilya Averyanov
b075b7120c feat(sessds): implement dispatchig between CE/EE shared sub agents 2024-06-05 19:17:58 +03:00
Ilya Averyanov
60556abc4f fix(dashboard_monitor): correctly merge old and new sample data 2024-06-05 18:49:52 +03:00
Kjell Winblad
2956e849eb fix(pgsql connector): better msg when failing to remove statement 2024-06-05 16:58:14 +02:00
Kjell Winblad
336089f8a7 fix: bug found by dialyzer and make test case cleaner 2024-06-05 15:53:02 +02:00