yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,421 Commits 393 Branches 669 Tags 74 MiB
03843c6071
Commit Graph

9597 Commits

Author SHA1 Message Date
JimMoen
a2015f37ae
Merge pull request #10976 from JimMoen/fix-shared-sub-unsub 2023-10-27 21:22:27 +08:00
lafirest
e63602c4b0
Merge pull request #11811 from lafirest/fix/api_rbac 
fix(rbac): for compatibility with old data schema, extend the existing field as an extra
 2023-10-27 17:56:54 +08:00
JimMoen
d563121284
refactor: move ?REDISPATCH_TO macro to emqx_mqtt.hrl 2023-10-27 17:54:14 +08:00
firest
4ba34f8f3e chore(rbac): fix CI errors & update change 2023-10-27 17:17:04 +08:00
JimMoen
3b5cc912e7
fix: add redispatch_to header to all msgs when deliver shared-sub 
- to find correct SubOpts for shared-sub dispatch
- use previous key `redispatch_to` to ensure rolling upgrade compatibility
 2023-10-27 16:15:23 +08:00
zhongwencool
22223dc536 fix: return 400 when audit log is disabled 2023-10-27 16:11:30 +08:00
zhongwencool
995948f0e8 refactor: remove seq from audit record 2023-10-27 14:07:19 +08:00
zhongwencool
9bb22507df fix: dont use transation on audit log 2023-10-27 11:41:12 +08:00
JimMoen
53383991d9
fix: rewrite #share{} and return #share{}, not formated $share/group/topic 2023-10-27 09:23:44 +08:00
JimMoen
afec6fa2f6
fix: TopicFilter may modified by client.subscribe hook 2023-10-27 09:23:44 +08:00
JimMoen
802a36c670
fix: find SubOpts by shared_record, not deliver topic 2023-10-27 09:23:44 +08:00
JimMoen
eaa5459509
chore: bump apps vsn 2023-10-27 09:23:43 +08:00
JimMoen
0ca725ff25
fix: deprecated share in subopts 2023-10-27 09:23:43 +08:00
JimMoen
b5411da770
refactor: subscribe process to fix shared-sub 2023-10-27 09:23:43 +08:00
Andrew Mayorov
44b4205561
fix(secret): do not treat missing file secrets as config error 
They are intended to be used mostly in the context of resources, which
have their own feedback mechanism: statuses, retries, etc.

Also turn the error into a throw exception, so that it can be
interpreted as a regular error condition, for example by the resource
manager.
 2023-10-26 14:37:14 +07:00
firest
cbfd02d1b0 refactor(ldap): merge the ldap-bind backend as a type for the ldap backend 2023-10-26 11:04:44 +08:00
Zaiming (Stone) Shi
97601171a5
Merge pull request #11818 from zmstone/1025-fix-flaky-test 
test: fix a flaky test case emqx_crl_cache_SUITE:t_revoked
 2023-10-25 20:46:31 +02:00
Zaiming (Stone) Shi
76a8b0efd4 test: fix a flaky test case emqx_crl_cache_SUITE:t_revoked 2023-10-25 16:08:31 +02:00
William Yang
f52b4a9e32 fix: gen_rpc tcp client port align with server port 2023-10-25 12:11:35 +02:00
William Yang
37dd44c3d7
Merge pull request #11813 from qzhuyan/fix/william/genrpc-ssl-port-align 
Fix/william/genrpc ssl port align
 2023-10-25 09:04:19 +02:00
William Yang
6346e0d28a fix(gen_rpc): ssl client port align with server port 2023-10-24 22:33:52 +02:00
firest
ec41479633 feat(rbac): supports setting role in API bootstrap file 2023-10-24 23:03:18 +08:00
firest
e175c213a1 fix(rbac): for compatibility with old data schema, extend the existing field as extra 2023-10-24 22:28:53 +08:00
Zaiming (Stone) Shi
abcb3166f6 fix: use binary string for raw default values 2023-10-24 14:46:01 +02:00
firest
7a8a5926ab fix(rbac): adjust the role names 2023-10-24 19:01:12 +08:00
Stefan Strigler
6e12569260
Merge pull request #11797 from sstrigler/EMQX-1057-fix-authz-api-donot-return-200 
fix: return 404 if built_in_database not configured as auth source
 2023-10-24 12:08:53 +02:00
Andrew Mayorov
52f4519eeb
feat(mqttbridge): support file-sourced secrets as passwords 2023-10-24 16:24:52 +07:00
Andrew Mayorov
1c2f9321d1
feat(emqx): add file-sourced generic secrets 
These secrets follow the same `emqx_secret` convention of 0-arity
functions. Also provide a simple HOCON schema module for use in
application schemas.
 2023-10-24 16:24:51 +07:00
Stefan Strigler
ea9228108b fix: use minirest filter 2023-10-24 09:57:08 +02:00
JimMoen
99fab8dc5d
chore: missing change log for duplicated apikey fixing 2023-10-24 09:44:39 +08:00
Thales Macedo Garitezi
d392c9989f
Merge pull request #11793 from thalesmg/test-rule-engine-foreach-m-20231019 
test(rule_engine): add test to exemplify `foreach` with json array payload
 2023-10-23 13:43:33 -03:00
Stefan Strigler
4e0e755b28 fix: return 404 if built_in_database not configured as auth source 2023-10-23 16:26:41 +02:00
Ilya Averyanov
8c6f6b56b8 chore(tests): report more friendly about unset PROFILE variable 2023-10-23 13:48:42 +03:00
Ilya Averyanov
edde661da3 fix(authn): fix pbkdf2 option validation 2023-10-23 10:26:11 +03:00
William Yang
a9b42af8b3
Merge pull request #11796 from qzhuyan/fix/william/gen_rpc-driver 
fix(gen_rpc): ensure client/server use same driver
 2023-10-23 08:24:17 +02:00
JimMoen
e6576951ef
test: cleanup duplicated apikey with different name 2023-10-20 22:33:26 +08:00
JimMoen
d467289bb2
fix: avoid duplicated apikey from data import 2023-10-20 21:13:04 +08:00
William Yang
dd5807b797 fix(gen_rpc): ensure client/server use same driver 
Also bump to gen_rpc 3.2.1
 2023-10-20 12:17:49 +02:00
zhongwencool
ef692596f7 fix: don't crash when 401 and 403 unauthorize 2023-10-20 15:22:30 +08:00
zhongwencool
6a8b2dc1f9 fix: bad high frequency request name 2023-10-20 14:57:31 +08:00
zhongwencool
c97fe796e3 chore: update emqx_audit's license 2023-10-20 14:47:24 +08:00
JimMoen
f381961108 fix: macro EMQX_RELEASE_EDITION when emqx run as standalnoe app 2023-10-20 14:47:24 +08:00
zhongwencool
32c1f1cca6 chore: inlude emqx as emqx_audit's deps 2023-10-20 14:47:24 +08:00
zhongwencool
1d7aa9495a chore: make spellcheck happy 2023-10-20 14:47:24 +08:00
zhongwencool
befc484544 chore: add changelog entry and fix xref warning 2023-10-20 14:47:24 +08:00
zhongwencool
141061c1e2 feat: add max_filter_size and ignore_high_frequency_request config 2023-10-20 14:47:24 +08:00
zhongwencool
926c804314 feat: add /audit http api to filter audit log 2023-10-20 14:47:24 +08:00
zhongwencool
8b5ec2a4d5 fix: xref failed 2023-10-20 11:46:31 +08:00
zhongwencool
281add1464 Merge branch 'master' into sync-master-to-release-54 2023-10-20 10:11:39 +08:00
lafirest
74442b0d31
Merge pull request #11766 from lafirest/feat/rbac 
feat(api_key): add RBAC feature for the API key
 2023-10-20 09:24:41 +08:00
Thales Macedo Garitezi
8c4beec6f0 test(rule_engine): add test to exemplify foreach with json array payload 
Relates to https://emqx.atlassian.net/browse/EMQX-11174
 2023-10-19 18:19:42 -03:00
Ilya Averyanov
0b4600c293 feat(auth): improve redis command parsing and validation 2023-10-19 23:01:35 +03:00
Thales Macedo Garitezi
eda82caf65
Merge pull request #11791 from thalesmg/fix-coap-heartbeat-m-20231018 
fix(coap): increase received packet counter for keepalive
 2023-10-19 13:12:41 -03:00
Ilya Averyanov
8d82c30b00
Merge pull request #11771 from savonarola/1015-validate-bcrypt-schema-in-api 
feat(authn): allow authn providers to define a separate schama for API
 2023-10-19 15:34:34 +03:00
Ilya Averyanov
8f9ac6c1f3
Merge pull request #11776 from savonarola/1017-auth-cleanup-2 
chore(authn): remove dead code
 2023-10-19 15:28:35 +03:00
Thales Macedo Garitezi
5b9866f630 fix(coap): increase received packet counter for keepalive 
Fixes https://emqx.atlassian.net/browse/EMQX-11193
Fixes https://github.com/emqx/emqx/issues/11779
 2023-10-18 16:35:33 -03:00
zhongwencool
2388d36b09 fix: allow viewers to change their own passwords 2023-10-18 18:40:02 +08:00
Ilya Averyanov
e1c8317779 chore(authn): remove dead code 2023-10-18 10:57:55 +03:00
JimMoen
1f8478623f
chore: bump esaml to support Entra ID && bump app.src 2023-10-18 10:01:34 +08:00
JimMoen
b2948666fb
chore: add saml sso redirect login log 2023-10-18 10:01:34 +08:00
Ilya Averyanov
c0ebaf9b6f
Merge pull request #11541 from savonarola/0829-duplicate-ft-responses 
Support asynchronous mode of receiving responses to FT operations
 2023-10-17 17:08:00 +03:00
Ilya Averyanov
6354f3b04f feat(authn): allow authn providers to define a separate schama for API 2023-10-17 13:19:11 +03:00
Zaiming (Stone) Shi
55618dd28a
Merge pull request #11731 from zmstone/1008-expose-file-transfer-host-conf 
fix(api-docs): add file-transfer config to hot-config scope
 2023-10-17 11:23:16 +02:00
firest
467a2bc9b1 test(rbac): add test cases for RBAC in REST API 2023-10-17 16:23:38 +08:00
firest
26ec860d96 feat(backup): add migration mechanism when import backup data 2023-10-17 16:23:32 +08:00
Zaiming (Stone) Shi
dfa5909392
Merge pull request #11752 from zmstone/1011-change-default-rpc-module-for-mria 
refactor: change mria default rpc module from 'gen_rpc' to 'rpc'
 2023-10-17 10:21:14 +02:00
firest
db3915d472 fix(rbac): update changes && fix CI errors 2023-10-17 13:44:37 +08:00
Zaiming (Stone) Shi
162b901be6
Merge pull request #11763 from zmstone/1114-sync-release-53 
Merge release-53 back to master
 2023-10-16 21:42:02 +02:00
firest
e095de7367 feat(api_key): add RBAC feature for the API key 2023-10-16 14:10:01 +08:00
JianBo He
16cc816bd3
Merge pull request #11757 from zhongwencool/trace-api-return-500 
fix: 500 error response when downloading non-existent trace files
 2023-10-16 13:40:18 +08:00
Zaiming (Stone) Shi
6eb3bb7cff Merge remote-tracking branch 'origin/release-53' into 1114-sync-release-53 2023-10-14 10:16:38 +02:00
Zaiming (Stone) Shi
7c022c2c6a refactor: change mria default rpc module from 'gen_rpc' to 'rpc' 
Erlang distribution seems to outperform gen_rpc (unless gen_rpc clients
are scaled up, but this is not easy to achive for shard transport as it
may reorder events).
 2023-10-14 10:13:31 +02:00
Ilya Averyanov
cd2752117c
Merge pull request #11762 from savonarola/1013-fix-mnesia-authz-destroy 
fix(mnesia authz): destroy authz records on mnesia authz destroy
 2023-10-13 20:54:14 +03:00
Ilya Averyanov
bde8800f2e fix(mnesia authz): destroy authz records on mnesia authz destroy 2023-10-13 18:29:45 +03:00
Ilya Averyanov
90a0c093bf
Merge pull request #11750 from savonarola/1011-redact-auth-http-request 
Do not trace authn http request bodies
 2023-10-13 17:25:28 +03:00
Ilya Averyanov
4f14e8df22 chore(source dir): split out postgresql connector 2023-10-13 16:58:57 +03:00
Thales Macedo Garitezi
7a6c756b7c
Merge pull request #11754 from thalesmg/fix-postgres-error-reason-log-m-20231011 
fix(postgres): format unicode error messages from driver
 2023-10-13 09:58:28 -03:00
Andrew Mayorov
0144ed9981
Merge pull request #11753 from keynslug/chore/smaller-ci-artifacts 
chore(ci): try to reduce CI artifacts footprint
 2023-10-13 16:56:47 +07:00
Andrew Mayorov
51c57a5f0a
fix(cass): avoid using aggregates in healthcheck query 
Otherwise, this query will produce warning message per each
healthcheck, telling the user:
```
WARN  <...> - Aggregation query used without partition key
```
 2023-10-13 16:50:19 +07:00
Andrew Mayorov
083e2da347
chore(bpapicheck): make some failures more user-friendly 2023-10-13 15:42:08 +07:00
Andrew Mayorov
0b9ac24c1e
fix(bpapicheck): stop depending on git index 2023-10-13 15:40:49 +07:00
lafirest
96c546c187
Merge pull request #11725 from lafirest/feat/os-ldap 
chore: change the LDAP integration to opensource
 2023-10-13 15:05:39 +08:00
Zaiming (Stone) Shi
dd03be29a6
Merge pull request #11751 from zmstone/1011-refactor-move-emqx_license-app-to-apps 
1011 refactor move emqx_license app to apps
 2023-10-13 06:45:49 +02:00
firest
33ff5d5588 chore: update auth header file 2023-10-13 10:16:47 +08:00
Ilya Averyanov
68f31a9da2 fix(authn): do not trace authn requests 2023-10-12 16:32:16 +03:00
Ilya Averyanov
08795f559c
Merge pull request #11336 from savonarola/0724-trace-authz 
Trace non-resultative authz calls
 2023-10-12 14:51:15 +03:00
Andrew Mayorov
6413afd0bc
fix(gcpdev): restore original shard in mnesia tab definition 
Changing the shard is not backward compatible, and leads to a crash
when upgrading from 5.2.x.
 2023-10-12 17:30:07 +07:00
Ilya Averyanov
4ecd5e17a2 chore(authz): trace non-resultative authz calls to backend modules 2023-10-12 12:29:39 +03:00
firest
3e658b3da9 chore: update changes 2023-10-12 17:15:47 +08:00
firest
9b4def885a chore: change the LDAP integration to opensource 2023-10-12 16:37:32 +08:00
zhongwencool
1021088f5f fix: 500 error response when downloading non-existent trace files 2023-10-12 15:23:23 +08:00
Thales Macedo Garitezi
b07dddd49e fix(postgres): format unicode error messages from driver 
Fixes https://emqx.atlassian.net/browse/EMQX-11024

Sample error:

```
{error, error, <<"42501">>, insufficient_privilege,
<<229,175,185,232,161,168,32,109,113,116,116,95,117,115,101,114,32,230,157,131,233,153,144,228,184,141,229,164,159>>,
[]}
```
 2023-10-11 17:56:12 -03:00
Ilya Averyanov
534c9bdc13 feat(ft): add additional operation status report channel 2023-10-11 19:51:36 +03:00
Zaiming (Stone) Shi
d9859b6b49
Merge pull request #11749 from thalesmg/test-fix-flaky-http-suite-r53-20231011 
test: attempt to fix flaky http bridge suite
 2023-10-11 16:07:17 +02:00
William Yang
e6358ca18c
Merge pull request #11747 from qzhuyan/dev/william/quicer-0.0.202 
bump to quicer 0.0.202
 2023-10-11 15:43:47 +02:00
Zaiming (Stone) Shi
e069680bca refactor: move emqx_license app from lib-ee to apps 2023-10-11 15:41:35 +02:00
Thales Macedo Garitezi
ab662f506d test: attempt to fix flaky http bridge suite 
```
 =CRASH REPORT==== 10-Oct-2023::05:35:30.911371 ===
  crasher:
    initial call: ehttpc:init/1
    pid: <0.23358.2>
    registered_name: []
    exception error: bad argument
      in function  persistent_term:get/1
         called as persistent_term:get({emqx_bridge_http_SUITE,
                                        do_t_async_retries,attempts})
         *** argument 1: no persistent term stored with this key
      in call from emqx_bridge_http_SUITE:'-do_t_async_retries/3-fun-2-'/0 (/__w/emqx/emqx/apps/emqx_bridge_http/test/emqx_bridge_http_SUITE.erl, line 697)
      in call from emqx_bridge_http_SUITE:'-do_t_async_retries/3-fun-4-'/6 (/__w/emqx/emqx/apps/emqx_bridge_http/test/emqx_bridge_http_SUITE.erl, line 705)
```
 2023-10-11 09:13:25 -03:00
Ilya Averyanov
57655854d1
Merge pull request #11743 from savonarola/1010-auth-leftovers 
chore(auth): cleanup code
 2023-10-11 15:09:07 +03:00
William Yang
a69c53455f fix: bump to quicer 0.0.202 
bring in urgent fixs from msquic 2.2.3
 2023-10-11 13:12:32 +02:00
zhongwencool
cc392cadb8
chore: update apps/emqx_retainer/src/emqx_retainer_mnesia_cli.erl 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-10-11 18:40:09 +08:00
Ilya Averyanov
03ae5bf3c8 chore(auth): cleanup code 2023-10-11 13:13:50 +03:00
zhongwencool
f4a88f717b feat: support retainer Start Length command 2023-10-11 11:47:37 +08:00
zhongwencool
8e7ba16c3a fix: nothing show when run clients list command 2023-10-11 11:47:15 +08:00
zhongwencool
574dc2f243 fix: observer load command crash when loading noexist module 2023-10-11 10:36:33 +08:00
Zaiming (Stone) Shi
a2e86c67db
Merge pull request #11733 from keynslug/fix/EMQX-10827/sesson-takeover 
fix(cm): bring back pre-v5.3.0 compat in `takeover_session_begin/1`
 2023-10-10 19:03:29 +02:00
Zaiming (Stone) Shi
0f07f678e7
Merge pull request #11742 from zmstone/1010-sync-release-53-to-master 
1010 sync release-53 to master
 2023-10-10 18:56:23 +02:00
Thales Macedo Garitezi
196b24c0cb
Merge pull request #11741 from thalesmg/test-fix-flaky-http-test-m-20231010 
test: attempt to fix flaky http bridge suite
 2023-10-10 13:52:08 -03:00
Thales Macedo Garitezi
ee45145fb5
Merge pull request #11724 from thalesmg/fix-kprodu-sync-metrics-m-20231006 
fix({kafka,pulsar}_producer): correctly handle metrics for connectors that have internal buffers
 2023-10-10 12:13:33 -03:00
Thales Macedo Garitezi
ec588f94e2 test: attempt to fix flaky http bridge suite 
```
 =CRASH REPORT==== 10-Oct-2023::05:35:30.911371 ===
  crasher:
    initial call: ehttpc:init/1
    pid: <0.23358.2>
    registered_name: []
    exception error: bad argument
      in function  persistent_term:get/1
         called as persistent_term:get({emqx_bridge_http_SUITE,
                                        do_t_async_retries,attempts})
         *** argument 1: no persistent term stored with this key
      in call from emqx_bridge_http_SUITE:'-do_t_async_retries/3-fun-2-'/0 (/__w/emqx/emqx/apps/emqx_bridge_http/test/emqx_bridge_http_SUITE.erl, line 697)
      in call from emqx_bridge_http_SUITE:'-do_t_async_retries/3-fun-4-'/6 (/__w/emqx/emqx/apps/emqx_bridge_http/test/emqx_bridge_http_SUITE.erl, line 705)
```
 2023-10-10 09:50:53 -03:00
Ivan Dyachkov
bfb2218392 chore: fix release version 2023-10-10 14:50:49 +02:00
Thales Macedo Garitezi
cf2075d7d8 chore: remove mention of is_buffer_supported from typespec 2023-10-10 09:49:18 -03:00
Zaiming (Stone) Shi
143673b703 chore: bump release version to 5.3.1-alpha.1 2023-10-10 14:35:46 +02:00
Zaiming (Stone) Shi
0a9e1dfa88
Merge pull request #11734 from zmstone/1002-support-ipv6-only-clustering 
Make IPv6 clustering options configurable
 2023-10-10 14:29:52 +02:00
Andrew Mayorov
5fff2ffe45
test(kafka): try to stabilize t_dynamic_mqtt_topic/1 testcase 2023-10-10 17:18:24 +07:00
Andrew Mayorov
9b573834f9
Merge pull request #11736 from keynslug/fix/eval-erl-audit 
fix(audit): handle abstract forms when logging `eval_erl`
 2023-10-10 15:49:19 +07:00
Zaiming (Stone) Shi
2626d793a7 test: try to resolve schema module from PROFILE in tests 2023-10-10 10:24:10 +02:00
Zaiming (Stone) Shi
808237364c fix(api-docs): add file-transfer config to hot-config scope 2023-10-10 10:24:10 +02:00
Zaiming (Stone) Shi
e7e696cd66 feat(rpc): add ipv6_only config 2023-10-10 10:05:14 +02:00
firest
e07937a3ef fix(ldap): escape the escape character (\) 2023-10-10 13:49:36 +08:00
Andrew Mayorov
2a291dfd27
fix(audit): handle abstract forms when logging eval_erl 2023-10-10 12:31:19 +07:00
Andrew Mayorov
522302fee1
fix(cm): bring back pre-v5.3.0 compat in takeover_session_begin/1 
Which was accidentally broken in bf164175.
 2023-10-10 01:34:10 +07:00
Zaiming (Stone) Shi
1e93d2f1fc chore: upgrade to ekka 0.15.16 gen_rpc 3.2.0 
* ekka 0.15.16 supports 'inet6_tls' as ekka.proto_dist
* gen_rpc 3.2.0 supports true | false as gen_rpc.ipv6_only
 2023-10-09 20:03:21 +02:00
Zaiming (Stone) Shi
497e08448d feat(cluster): support ipv6 and tls on ipv6 for clustering 
Made possible to configure inet6_tls for Erlang distribution
Also, added support to configure ipv6 listener for gen_rpc
 2023-10-09 20:03:21 +02:00
Zaiming (Stone) Shi
03d8e06ff7 chore(emqx_rule_funcs): regroup export functions 2023-10-09 20:03:21 +02:00
Thales Macedo Garitezi
d6781efee2 fix(resource): change how buffer workers are started 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
902b1d6ec5 fix(pulsar_producer): use simple_async_internal_buffer query mode for Pulsar 
Since it has internal buffering, it necessitates the same fix as Kafka producer.
 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
eebfb44f72 fix(resource): create simple_async_internal_buffer query mode for bridges with internal buffering 
Since authn/authz backends also use simple async/sync queries, we may want to avoid them
calling the connector when it's not connected.
 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
79cf0a2ced fix(kafka_producer): correctly handle metrics for connector that have internal buffers 
Fixes https://emqx.atlassian.net/browse/EMQX-11086

There’s currently a metric inconsistency due to the internal buffering nature of Kafka
Producer (wolff).

We use simple_sync_query to call the Kafka Producer bridge.  If that times out, the call
is accounted as failed, even though the message is buffered in wolff and later sent
successfully.
 2023-10-09 15:02:25 -03:00
Zaiming (Stone) Shi
c60915293a
Merge pull request #11634 from savonarola/0919-auth-refactor 
Auth refactor
 2023-10-09 19:51:26 +02:00
lafirest
8f4cdc3fcf
Merge pull request #11732 from lafirest/test/sso_cli 
test(ldap): add test suite for SSO CLI
 2023-10-09 20:35:23 +08:00
Zaiming (Stone) Shi
9ab49a7ae3 Merge remote-tracking branch 'origin/master' into 0919-auth-refactor 2023-10-09 14:18:10 +02:00
lafirest
2c7e5eb1cb
Merge pull request #11728 from lafirest/fix/ldap_filter 
fix(ldap): improve the filter lex && parse
 2023-10-09 18:24:57 +08:00
firest
e3550fc07b test(ldap): add test suite for SSO CLI 2023-10-09 16:05:56 +08:00
zhongwencool
0c89b6b213 docs: swagger api tags should camel case 2023-10-09 15:21:51 +08:00
firest
b2a6724dc2 fix(ldap): improve the filter lex && parse 
1. auto escape special chars in the filter
2. fix a bug that the value can't be `dn`
 2023-10-08 18:39:20 +08:00
zhongwencool
482e82f914 fix: cli's args in audit log should be array 2023-10-07 17:38:02 +08:00
Thales Macedo Garitezi
34186fcc74 fix(kafka_producer): send messages to wolff producer to buffer even when connector is in connecting state 
Fixes https://emqx.atlassian.net/browse/EMQX-11085

Messages would not be sent to wolff if the connection was down, so they were effectively lost.
 2023-10-06 11:43:29 -03:00
Ilya Averyanov
398a62031e chore: update app versions 2023-10-05 13:41:50 +03:00
Ilya Averyanov
5dff36474d chore(auth): get rid of hardcoded schema modules in auth 2023-10-05 13:41:50 +03:00
Ilya Averyanov
c2c56ba481 chore(auth): update tests 2023-10-05 13:41:50 +03:00
Ilya Averyanov
1eb75b43c4 chore(auth): split emqx_authn and emqx_authz apps 2023-10-05 13:41:50 +03:00
Zaiming (Stone) Shi
fd75dc895d fix(authn): "authentication" importance should not be "hidden" 2023-10-03 11:18:21 +02:00
Zaiming (Stone) Shi
ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53 
0928 sync release 53
 2023-10-02 21:57:32 +02:00
Zaiming (Stone) Shi
3f6e0e890b chore: bump emqx_durable_storage and emqx_license app vsn 2023-10-02 20:13:09 +02:00
Thales Macedo Garitezi
2358d67908 refactor: move session stuff from emqx_ds to emqx_persistent_session_ds 
Part of https://emqx.atlassian.net/browse/EMQX-10942
 2023-09-29 18:00:24 -03:00
Zaiming (Stone) Shi
bce8fd2fbc chore: bump app versions 2023-09-29 18:56:52 +02:00
Zaiming (Stone) Shi
ed5b456d62 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 18:24:44 +02:00
Zaiming (Stone) Shi
a852400fb3 chore: add 5.3.bpapi 2023-09-29 16:34:04 +02:00
Zaiming (Stone) Shi
fe01aaff4f chore: prepare for e5.3.0 promotion 2023-09-29 13:32:57 +02:00
Zaiming (Stone) Shi
ce5bd0a3ce Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 11:36:32 +02:00
Zaiming (Stone) Shi
c64e599e81 docs: document how to retrieve peercert 2023-09-29 10:33:57 +02:00
Zaiming (Stone) Shi
dc147fd310 fix(rule-engine): console action has no args field 2023-09-29 10:33:33 +02:00
Zaiming (Stone) Shi
1177a32310 chore: bump version to 5.3.0-rc.2 2023-09-29 09:49:27 +02:00
Zaiming (Stone) Shi
6891234390 chore: return simplified error reason for less logging 2023-09-29 09:48:15 +02:00
Zaiming (Stone) Shi
c2d750aa09 fix(resource): redact query args in exception log 2023-09-29 09:20:42 +02:00
Zaiming (Stone) Shi
02ef854f0f fix(ldap): no crash when no query result is empty list 2023-09-29 08:54:41 +02:00
Zaiming (Stone) Shi
b59a7ff2dd
Merge pull request #11706 from zmstone/0928-fix-ldap-sso-logging-level 
0928 fix ldap sso logging level and reject muti-match results
 2023-09-29 02:40:37 +02:00
Zaiming (Stone) Shi
4a4730ad46 fix(ldap): handle invalidCredentials in ldap authn 2023-09-29 00:51:05 +02:00
Zaiming (Stone) Shi
9ee2cb9c79 fix(ldap): return unrecoverable_error if more than on match found 2023-09-28 23:58:34 +02:00
Zaiming (Stone) Shi
cc5dab1dc7 chore: fix code style 2023-09-28 21:29:59 +02:00
Zaiming (Stone) Shi
b267fc2588 chore: bump release version to 5.3.0 2023-09-28 21:22:33 +02:00
Zaiming (Stone) Shi
922d5a9a83 fix(ldap): do not allow multi-matches to proceed 
if ldap query returns more than on match
we should reject the auth request instead of picking
the first one
 2023-09-28 21:20:50 +02:00
Zaiming (Stone) Shi
d858f8af39 test: fix openldap docker runs 2023-09-28 18:40:03 +02:00
Zaiming (Stone) Shi
b28e781c50 fix(ldap-sso): do not log error level when invalid user credentials 2023-09-28 18:37:11 +02:00
Zaiming (Stone) Shi
36f3052be1
Merge pull request #11698 from zmstone/0928-disable-audit-log-by-default 
fix(audit): disable audit log by default
 2023-09-28 17:28:23 +02:00
Zaiming (Stone) Shi
39820be5ff
Merge pull request #11703 from lafirest/fix/ldap_bind 
fix(ldap): use the search result as bind target
 2023-09-28 14:48:53 +02:00
Zaiming (Stone) Shi
1a13b2ac56 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-28 13:53:03 +02:00
firest
43ea367df8 fix(ldap): use the search result as bind target 2023-09-28 18:49:00 +08:00
Zaiming (Stone) Shi
98409c9b1e
Merge pull request #11702 from zmstone/0928-minor-chore 
0928 minor chore
 2023-09-28 12:39:55 +02:00
Zaiming (Stone) Shi
26cadb2248 fix(audit): disable audit log by default 2023-09-28 12:36:14 +02:00
Zaiming (Stone) Shi
b14425d865 fix(sso/saml): ensure lower case http header name 2023-09-28 11:40:21 +02:00
Zaiming (Stone) Shi
7ddcd71412 docs(ssl_dist.conf): update doc link 2023-09-28 11:16:18 +02:00
firest
dca8fdb17f fix(resource): respect the start_timeout 2023-09-28 16:36:41 +08:00
ieQu1
b77e5e880a
Merge pull request #11697 from ieQu1/dev/gen-rpc-3.1.1 
chore(gen_rpc): Bump version to 3.1.1
 2023-09-28 10:03:36 +02:00
JianBo He
6d2adfc259 chore(audit): emit the message field 2023-09-28 11:04:42 +08:00
JianBo He
859b122cdd chore: format codes 2023-09-28 10:25:37 +08:00
JianBo He
b52e4ac99d chore: update apps/emqx_machine/src/emqx_restricted_shell.erl 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-09-28 10:25:37 +08:00
JianBo He
bdf24d0ec8 chore: remove the unauthorized type for from field 2023-09-28 10:25:37 +08:00
JianBo He
71acf121ba chore(audit): distinguish requests from rest_api or dashboard 2023-09-28 10:25:37 +08:00
JianBo He
dd9938114c chore(audit): add from field 2023-09-28 10:25:37 +08:00
JianBo He
ece7d5b52a
Merge pull request #11690 from lafirest/fix/ldap_parse_cfg 
fix(ldap): improve the LDAP `parse_config` function
 2023-09-28 10:24:52 +08:00
lafirest
b0d86eecd6
Merge pull request #11691 from lafirest/fix/sso_ssl 
fix(sso): support for SSL update && ensure update is atomic
 2023-09-28 10:22:34 +08:00
firest
57781d0544 fix(ldap): remove the parse_config, it never work 2023-09-28 09:38:16 +08:00
firest
9dee2dc31e fix(sso): clear last error first before update && fix the running 2023-09-28 08:56:16 +08:00
ieQu1
0aa3ccdd65 feat(gen_rpc): Add schema for the TLS versions and cipher suites 2023-09-28 00:36:59 +02:00
Zaiming (Stone) Shi
afdda107af fix(logger): json format log encode binary list as string array 2023-09-27 23:40:01 +02:00
Zaiming (Stone) Shi
6f7a4344dc fix: do not gc sso saml SP singing keys 2023-09-27 23:19:39 +02:00
Zaiming (Stone) Shi
34367fc4ec fix(audit_log): pretty print shell args 2023-09-27 23:19:39 +02:00
Zaiming (Stone) Shi
bb49914fd6 fix(sso): add convet_certs callback for sso backends 
must convert certs in pre_config_update so the cert path refernces
are stored in raw config, otherwise the files might get gc:ed
 2023-09-27 22:41:39 +02:00
Zaiming (Stone) Shi
45caa3bf01 fix(sso): make sp_private_key sensitive 
so it will not be logged
 2023-09-27 21:27:59 +02:00
Zaiming (Stone) Shi
c8cbbff044 fix(logger): no need for special handling of empty string 
when formating json logs, there is no need to handle empty strings
special, already covered by unicode handling
 2023-09-27 21:26:47 +02:00
ieQu1
3c37f19105 chore(gen_rpc): Bump version to 3.1.1 2023-09-27 21:09:00 +02:00
firest
66d2107007 fix(sso): refactor update logic 2023-09-28 00:09:09 +08:00
Zaiming (Stone) Shi
bd3277c51b
Merge pull request #11683 from zmstone/0925-test-refactor-emqx-rpc-test 
test: refactor emqx_rpc unit tests
 2023-09-27 15:46:27 +02:00
firest
08ad09a68f fix(sso): refactor backen update logic 
1. valid config always can update successfully
2. the `running` endpoint only return successfully created backend
3. enhancement of the `/sso` endpoint, and will check is the resource online
 2023-09-27 20:53:10 +08:00
Zaiming (Stone) Shi
ea8d54fd8b test: ensure atom exists in test module 2023-09-27 12:58:06 +02:00
Zaiming (Stone) Shi
6e8c73258f test: refactor emqx_rpc unit tests 2023-09-27 12:58:06 +02:00
JimMoen
c9194cd6b2
fix(saml_sso): donot load IDP metadata when disabling saml 2023-09-27 18:46:24 +08:00
JimMoen
af9e87c025
fix: saml callback should check saml state 2023-09-27 18:34:48 +08:00
firest
b2699c687b fix(sso): support for SSL update && ensure update is atomic 
1. support update SSL key and cert files
2. increase connection timeout
3. ensure the update is atomicity, everything will be consistent
 2023-09-27 15:48:11 +08:00
Andrew Mayorov
1d0e789e4d
Merge pull request #11679 from keynslug/ft/EMQX-10942/custom-session-timers 
feat(session): add custom session timers mechanism
 2023-09-26 19:09:53 +03:00
firest
0c33df3912 fix(ldap): improve the LDAP parse_config function 2023-09-26 23:28:37 +08:00
JimMoen
3fa18d6935
fix: lower sso saml redirect http header 2023-09-26 23:25:59 +08:00
JianBo He
6a9bb7c3ae
Merge pull request #11681 from zhongwencool/audit-log-fix-2 
fix: don't need to change audit log's level
 2023-09-26 23:12:28 +08:00
lafirest
ac5eb5bc29
Merge pull request #11687 from lafirest/fix/sso_timeout 
fix(sso): Handle backend update timeout and fix create errors
 2023-09-26 22:43:38 +08:00
JianBo He
1a96a5990b
chore: Update apps/emqx_enterprise/src/emqx_enterprise_schema.erl 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-09-26 21:44:40 +08:00
firest
403714d44e fix(sso): Handle backend update timeout and fix create errors 
1. correctly handle the timeout when call update on a backend
2. fix that config update always returns success
3. do not ignore start failures and ensure start is atomic
 2023-09-26 21:43:30 +08:00
Thales Macedo Garitezi
aae59f1efd
Merge pull request #11682 from thalesmg/fix-file-logger-type-r53-20230925 
fix(file_logger): change file logger type depending on rotation size
 2023-09-26 10:22:36 -03:00
Andrew Mayorov
bce2142df8
test(session): add custom session timers testcase 2023-09-26 16:22:06 +03:00
Zaiming (Stone) Shi
4e15edb5e4
Merge pull request #11661 from zmstone/0922-fix-json-log-formatter 
0922 fix json log formatter
 2023-09-26 14:13:41 +02:00
Thales Macedo Garitezi
fdcd73e20c fix(file_logger): change file logger type depending on rotation size 
Fixes https://emqx.atlassian.net/browse/EMQX-11036

From `logger_disk_log_h:open_disk_log`:

```erlang
open_disk_log(Name,File,Type,MaxNoBytes,MaxNoFiles) ->
    case filelib:ensure_dir(File) of
        ok ->
            Size =
                if Type==halt -> MaxNoBytes;
                   Type==wrap -> {MaxNoBytes,MaxNoFiles} %% <-------
                end,
            Opts = [{name,   Name},
                    {file,   File},
                    {size,   Size},
                    {type,   Type},
                    {linkto, self()},
                    {repair, false},
                    {format, external},
                    {notify, true},
                    {quiet,  true},
                    {mode,   read_write}],
            case disk_log:open(Opts) of
```

Affects all file loggers (audit included):

```
% emqx_config_logger:update_log_handler/1 -> ok
iex(emqx@127.0.0.1)14> Config override: log.file.emqx_audit is updated, but failed to add handler: {handler_not_added,
                                                                             {badarg,
                                                                              [{size,
                                                                                {infinity,
                                                                                 10}},
                                                                               {type,
                                                                                wrap},
                                                                               {linkto,
                                                                                <0.1952.0>},
                                                                               {repair,
                                                                                false},
                                                                               {format,
                                                                                external},
                                                                               {notify,
                                                                                true},
                                                                               {quiet,
                                                                                true},
                                                                               {mode,
                                                                                read_write}]}}
```
 2023-09-26 09:04:44 -03:00
Ilya Averyanov
14983ec14a chore(hooks): validate hookpoints and document hook callbacks 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-09-26 14:27:42 +03:00
JianBo He
5e6397b843 chore: remove result field for CLI audit logs 2023-09-26 19:24:35 +08:00
JianBo He
a73c3b8e1e Merge remote-tracking branch 'ce/release-53' into audit-log-fix-2 2023-09-26 19:05:47 +08:00
JianBo He
e9785a6863
Merge pull request #11680 from thalesmg/audit-not-on-ce-r53-20230925 
fix(audit): only support audit log on enterprise edition
 2023-09-26 18:57:22 +08:00
Zaiming (Stone) Shi
7d810c2107 test: fix test case match pattern 2023-09-26 11:40:30 +02:00
Zaiming (Stone) Shi
d31bfc70fb
Merge pull request #11659 from zhongwencool/fix-listener-ssl-create-500 
fix: create ssl listener return 500 crash
 2023-09-26 11:38:07 +02:00
lafirest
8cc626d33f
Merge pull request #11686 from lafirest/fix/sso_user_backend 
fix: fix update SSO user && improve SSO user deletion commnad
 2023-09-26 16:04:51 +08:00
firest
56917fee5f fix(sso): Improved SSO user deletion command 2023-09-26 14:15:41 +08:00
firest
d1af81c86e fix(sso): fix the failure to update SSO users 2023-09-26 12:14:45 +08:00
firest
8135e27586 fix: redact sensitive data in SSO and LDAP 2023-09-26 11:06:47 +08:00
Thales Macedo Garitezi
17206f8c75
Merge pull request #11662 from thalesmg/port-scan-check-proto-dist-m-20230922 
chore: check ekka proto dist module type when resolving node address
 2023-09-25 17:01:43 -03:00
Thales Macedo Garitezi
5d212e1086 fix(audit): only support audit log on enterprise edition 
Fixes https://emqx.atlassian.net/browse/EMQX-11039
 2023-09-25 15:59:35 -03:00
Thales Macedo Garitezi
0498e59c45 test: fix flaky test 2023-09-25 13:35:17 -03:00
zhongwencool
95060302fd fix: don't need to change audit log's level 2023-09-26 00:28:47 +08:00
Thales Macedo Garitezi
ff7f37ccf5 test(cth): allow defining schema to load for app 2023-09-25 13:22:41 -03:00
Andrew Mayorov
b1f144ab8b
feat(session): add custom session timers mechanism 
That are managed exclusively by the session implementation, unlike
common session timers that are managed by the channel itself.
 2023-09-25 18:19:26 +03:00
lafirest
1df8326fb8
Merge pull request #11677 from lafirest/fix/sso_cfg 
adjust the config path for the SSO feature and improve the update logic
 2023-09-25 23:00:46 +08:00
Zaiming (Stone) Shi
5e6996dc05 refactor: log formatter format mfa+line as m:f/a(line) 
also improve json formatter when a field is iolist
 2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi
6a557980e4 refactor(logger): ensure JSON log field order 
Ensure that the log fileds in JSON format are ordered as
time, level, msg, mfal, ...
 2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi
5f45ba50ff refactor: delete log formatter config for audit log handler 
It is using 'json' formatter, the template is useless
 2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi
1fed38c248 fix(logger): write 'json' format logs as JSON 2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi
aea1e80290 feat: add 'format' as alias for log 'formatter' 2023-09-25 16:55:22 +02:00
William Yang
9106eb92d2
Merge pull request #11675 from qzhuyan/fix/william/quic-respect-param-verify 2023-09-25 15:48:01 +02:00
Thales Macedo Garitezi
806017ef90 chore: check ekka proto dist module type when resolving node address 
Follow up to https://github.com/emqx/emqx/pull/11637#discussion_r1334462917

Fixes https://emqx.atlassian.net/browse/EMQX-10944
 2023-09-25 10:31:50 -03:00
firest
cfdb25b213 fix(sso): updates the SSO backend when the [dashboard] has updated 2023-09-25 20:30:47 +08:00
Ivan Dyachkov
f1bc3b68b6
Merge pull request #11674 from id/0925-e5.3.0-alpha.2 
e5.3.0 alpha.2
 2023-09-25 14:21:46 +02:00
JianBo He
6f29bbf997
Merge pull request #11673 from HJianBo/redact_audit_log_from_cli 
Redact audit log from CLI
 2023-09-25 20:03:45 +08:00
Andrew Mayorov
81cf619f07
fix(ftconf): also mark secret_access_key key as sensitive 2023-09-25 14:52:57 +03:00
William Yang
36d3a3a524 fix: bump to quicer 0.0.201 
do not load cacertfile if verify_none
 2023-09-25 13:02:01 +02:00
William Yang
22193d273a fix(quic): ignore undefined cacertfile 2023-09-25 12:57:55 +02:00
Ivan Dyachkov
8c93c79b44 chore: e5.3.0-alpha.2 2023-09-25 12:52:45 +02:00
JimMoen
7d58f6c61e
Merge pull request #11668 from JimMoen/saml-login-redirect 
fix: saml login acs redirect to dashboard overview
 2023-09-25 18:11:03 +08:00
JianBo He
b970a34ee0 chore: redact some audit logs from CLI 2023-09-25 18:06:39 +08:00
JianBo He
89e15e9134
Merge pull request #11665 from HJianBo/typo-fixes 
Typo fixes
 2023-09-25 17:23:40 +08:00
lafirest
1cd9df6461
Merge pull request #11669 from lafirest/fix/dashboard_cli 
fix(sso): refactor the `admins` CLI
 2023-09-25 16:57:18 +08:00
firest
e63d484632 fix(sso): move the config path of the SSO feature to dashboard.sso 2023-09-25 15:23:52 +08:00
Ivan Dyachkov
e1f1c64ecd Merge branch 'release-53' into 0925-sync-r53-to-master 2023-09-25 08:12:45 +02:00
firest
e94192d1fa fix(sso): refactor the admins CLI 
1. revert the opensource version
2. allow delete the SSO user via CLI
 2023-09-25 13:53:01 +08:00
firest
4f4868a46c fix(sso): Disable access to logout endpoint by the API key 2023-09-25 10:31:38 +08:00