Commit Graph

9355 Commits

Author SHA1 Message Date
William Yang 37dd44c3d7
Merge pull request #11813 from qzhuyan/fix/william/genrpc-ssl-port-align
Fix/william/genrpc ssl port align
2023-10-25 09:04:19 +02:00
William Yang 6346e0d28a fix(gen_rpc): ssl client port align with server port 2023-10-24 22:33:52 +02:00
Zaiming (Stone) Shi abcb3166f6 fix: use binary string for raw default values 2023-10-24 14:46:01 +02:00
JimMoen 99fab8dc5d
chore: missing change log for duplicated apikey fixing 2023-10-24 09:44:39 +08:00
William Yang a9b42af8b3
Merge pull request #11796 from qzhuyan/fix/william/gen_rpc-driver
fix(gen_rpc): ensure client/server use same driver
2023-10-23 08:24:17 +02:00
JimMoen e6576951ef
test: cleanup duplicated apikey with different name 2023-10-20 22:33:26 +08:00
JimMoen d467289bb2
fix: avoid duplicated apikey from data import 2023-10-20 21:13:04 +08:00
William Yang dd5807b797 fix(gen_rpc): ensure client/server use same driver
Also bump to gen_rpc 3.2.1
2023-10-20 12:17:49 +02:00
Ilya Averyanov 90a0c093bf
Merge pull request #11750 from savonarola/1011-redact-auth-http-request
Do not trace authn http request bodies
2023-10-13 17:25:28 +03:00
Andrew Mayorov 51c57a5f0a
fix(cass): avoid using aggregates in healthcheck query
Otherwise, this query will produce warning message per each
healthcheck, telling the user:
```
WARN  <...> - Aggregation query used without partition key
```
2023-10-13 16:50:19 +07:00
Ilya Averyanov 68f31a9da2 fix(authn): do not trace authn requests 2023-10-12 16:32:16 +03:00
Andrew Mayorov 6413afd0bc
fix(gcpdev): restore original shard in mnesia tab definition
Changing the shard is not backward compatible, and leads to a crash
when upgrading from 5.2.x.
2023-10-12 17:30:07 +07:00
Zaiming (Stone) Shi d9859b6b49
Merge pull request #11749 from thalesmg/test-fix-flaky-http-suite-r53-20231011
test: attempt to fix flaky http bridge suite
2023-10-11 16:07:17 +02:00
William Yang e6358ca18c
Merge pull request #11747 from qzhuyan/dev/william/quicer-0.0.202
bump to quicer 0.0.202
2023-10-11 15:43:47 +02:00
Thales Macedo Garitezi ab662f506d test: attempt to fix flaky http bridge suite
```
 =CRASH REPORT==== 10-Oct-2023::05:35:30.911371 ===
  crasher:
    initial call: ehttpc:init/1
    pid: <0.23358.2>
    registered_name: []
    exception error: bad argument
      in function  persistent_term:get/1
         called as persistent_term:get({emqx_bridge_http_SUITE,
                                        do_t_async_retries,attempts})
         *** argument 1: no persistent term stored with this key
      in call from emqx_bridge_http_SUITE:'-do_t_async_retries/3-fun-2-'/0 (/__w/emqx/emqx/apps/emqx_bridge_http/test/emqx_bridge_http_SUITE.erl, line 697)
      in call from emqx_bridge_http_SUITE:'-do_t_async_retries/3-fun-4-'/6 (/__w/emqx/emqx/apps/emqx_bridge_http/test/emqx_bridge_http_SUITE.erl, line 705)
```
2023-10-11 09:13:25 -03:00
Ilya Averyanov 57655854d1
Merge pull request #11743 from savonarola/1010-auth-leftovers
chore(auth): cleanup code
2023-10-11 15:09:07 +03:00
William Yang a69c53455f fix: bump to quicer 0.0.202
bring in urgent fixs from msquic 2.2.3
2023-10-11 13:12:32 +02:00
Ilya Averyanov 03ae5bf3c8 chore(auth): cleanup code 2023-10-11 13:13:50 +03:00
Zaiming (Stone) Shi a2e86c67db
Merge pull request #11733 from keynslug/fix/EMQX-10827/sesson-takeover
fix(cm): bring back pre-v5.3.0 compat in `takeover_session_begin/1`
2023-10-10 19:03:29 +02:00
Thales Macedo Garitezi ee45145fb5
Merge pull request #11724 from thalesmg/fix-kprodu-sync-metrics-m-20231006
fix({kafka,pulsar}_producer): correctly handle metrics for connectors that have internal buffers
2023-10-10 12:13:33 -03:00
Ivan Dyachkov bfb2218392 chore: fix release version 2023-10-10 14:50:49 +02:00
Thales Macedo Garitezi cf2075d7d8 chore: remove mention of `is_buffer_supported` from typespec 2023-10-10 09:49:18 -03:00
Zaiming (Stone) Shi 143673b703 chore: bump release version to 5.3.1-alpha.1 2023-10-10 14:35:46 +02:00
Zaiming (Stone) Shi 0a9e1dfa88
Merge pull request #11734 from zmstone/1002-support-ipv6-only-clustering
Make IPv6 clustering options configurable
2023-10-10 14:29:52 +02:00
Andrew Mayorov 9b573834f9
Merge pull request #11736 from keynslug/fix/eval-erl-audit
fix(audit): handle abstract forms when logging `eval_erl`
2023-10-10 15:49:19 +07:00
Zaiming (Stone) Shi e7e696cd66 feat(rpc): add ipv6_only config 2023-10-10 10:05:14 +02:00
firest e07937a3ef fix(ldap): escape the escape character (\) 2023-10-10 13:49:36 +08:00
Andrew Mayorov 2a291dfd27
fix(audit): handle abstract forms when logging `eval_erl` 2023-10-10 12:31:19 +07:00
Andrew Mayorov 522302fee1
fix(cm): bring back pre-v5.3.0 compat in `takeover_session_begin/1`
Which was accidentally broken in bf164175.
2023-10-10 01:34:10 +07:00
Zaiming (Stone) Shi 1e93d2f1fc chore: upgrade to ekka 0.15.16 gen_rpc 3.2.0
* ekka 0.15.16 supports 'inet6_tls' as ekka.proto_dist
* gen_rpc 3.2.0 supports true | false as gen_rpc.ipv6_only
2023-10-09 20:03:21 +02:00
Zaiming (Stone) Shi 497e08448d feat(cluster): support ipv6 and tls on ipv6 for clustering
Made possible to configure inet6_tls for Erlang distribution
Also, added support to configure ipv6 listener for gen_rpc
2023-10-09 20:03:21 +02:00
Zaiming (Stone) Shi 03d8e06ff7 chore(emqx_rule_funcs): regroup export functions 2023-10-09 20:03:21 +02:00
Thales Macedo Garitezi d6781efee2 fix(resource): change how buffer workers are started 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi 902b1d6ec5 fix(pulsar_producer): use `simple_async_internal_buffer` query mode for Pulsar
Since it has internal buffering, it necessitates the same fix as Kafka producer.
2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi eebfb44f72 fix(resource): create `simple_async_internal_buffer` query mode for bridges with internal buffering
Since authn/authz backends also use simple async/sync queries, we may want to avoid them
calling the connector when it's not connected.
2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi 79cf0a2ced fix(kafka_producer): correctly handle metrics for connector that have internal buffers
Fixes https://emqx.atlassian.net/browse/EMQX-11086

There’s currently a metric inconsistency due to the internal buffering nature of Kafka
Producer (wolff).

We use simple_sync_query to call the Kafka Producer bridge.  If that times out, the call
is accounted as failed, even though the message is buffered in wolff and later sent
successfully.
2023-10-09 15:02:25 -03:00
Zaiming (Stone) Shi c60915293a
Merge pull request #11634 from savonarola/0919-auth-refactor
Auth refactor
2023-10-09 19:51:26 +02:00
lafirest 8f4cdc3fcf
Merge pull request #11732 from lafirest/test/sso_cli
test(ldap): add test suite for SSO CLI
2023-10-09 20:35:23 +08:00
Zaiming (Stone) Shi 9ab49a7ae3 Merge remote-tracking branch 'origin/master' into 0919-auth-refactor 2023-10-09 14:18:10 +02:00
lafirest 2c7e5eb1cb
Merge pull request #11728 from lafirest/fix/ldap_filter
fix(ldap): improve the filter lex && parse
2023-10-09 18:24:57 +08:00
firest e3550fc07b test(ldap): add test suite for SSO CLI 2023-10-09 16:05:56 +08:00
zhongwencool 0c89b6b213 docs: swagger api tags should camel case 2023-10-09 15:21:51 +08:00
firest b2a6724dc2 fix(ldap): improve the filter lex && parse
1. auto escape special chars in the filter
2. fix a bug that the value can't be `dn`
2023-10-08 18:39:20 +08:00
zhongwencool 482e82f914 fix: cli's args in audit log should be array 2023-10-07 17:38:02 +08:00
Thales Macedo Garitezi 34186fcc74 fix(kafka_producer): send messages to wolff producer to buffer even when connector is in `connecting` state
Fixes https://emqx.atlassian.net/browse/EMQX-11085

Messages would not be sent to wolff if the connection was down, so they were effectively lost.
2023-10-06 11:43:29 -03:00
Ilya Averyanov 398a62031e chore: update app versions 2023-10-05 13:41:50 +03:00
Ilya Averyanov 5dff36474d chore(auth): get rid of hardcoded schema modules in auth 2023-10-05 13:41:50 +03:00
Ilya Averyanov c2c56ba481 chore(auth): update tests 2023-10-05 13:41:50 +03:00
Ilya Averyanov 1eb75b43c4 chore(auth): split emqx_authn and emqx_authz apps 2023-10-05 13:41:50 +03:00
Zaiming (Stone) Shi fd75dc895d fix(authn): "authentication" importance should not be "hidden" 2023-10-03 11:18:21 +02:00
Zaiming (Stone) Shi ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53
0928 sync release 53
2023-10-02 21:57:32 +02:00
Zaiming (Stone) Shi 3f6e0e890b chore: bump emqx_durable_storage and emqx_license app vsn 2023-10-02 20:13:09 +02:00
Thales Macedo Garitezi 2358d67908 refactor: move session stuff from `emqx_ds` to `emqx_persistent_session_ds`
Part of https://emqx.atlassian.net/browse/EMQX-10942
2023-09-29 18:00:24 -03:00
Zaiming (Stone) Shi bce8fd2fbc chore: bump app versions 2023-09-29 18:56:52 +02:00
Zaiming (Stone) Shi ed5b456d62 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 18:24:44 +02:00
Zaiming (Stone) Shi a852400fb3 chore: add 5.3.bpapi 2023-09-29 16:34:04 +02:00
Zaiming (Stone) Shi fe01aaff4f chore: prepare for e5.3.0 promotion 2023-09-29 13:32:57 +02:00
Zaiming (Stone) Shi ce5bd0a3ce Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 11:36:32 +02:00
Zaiming (Stone) Shi c64e599e81 docs: document how to retrieve peercert 2023-09-29 10:33:57 +02:00
Zaiming (Stone) Shi dc147fd310 fix(rule-engine): console action has no args field 2023-09-29 10:33:33 +02:00
Zaiming (Stone) Shi 1177a32310 chore: bump version to 5.3.0-rc.2 2023-09-29 09:49:27 +02:00
Zaiming (Stone) Shi 6891234390 chore: return simplified error reason for less logging 2023-09-29 09:48:15 +02:00
Zaiming (Stone) Shi c2d750aa09 fix(resource): redact query args in exception log 2023-09-29 09:20:42 +02:00
Zaiming (Stone) Shi 02ef854f0f fix(ldap): no crash when no query result is empty list 2023-09-29 08:54:41 +02:00
Zaiming (Stone) Shi b59a7ff2dd
Merge pull request #11706 from zmstone/0928-fix-ldap-sso-logging-level
0928 fix ldap sso logging level and reject muti-match results
2023-09-29 02:40:37 +02:00
Zaiming (Stone) Shi 4a4730ad46 fix(ldap): handle invalidCredentials in ldap authn 2023-09-29 00:51:05 +02:00
Zaiming (Stone) Shi 9ee2cb9c79 fix(ldap): return unrecoverable_error if more than on match found 2023-09-28 23:58:34 +02:00
Zaiming (Stone) Shi cc5dab1dc7 chore: fix code style 2023-09-28 21:29:59 +02:00
Zaiming (Stone) Shi b267fc2588 chore: bump release version to 5.3.0 2023-09-28 21:22:33 +02:00
Zaiming (Stone) Shi 922d5a9a83 fix(ldap): do not allow multi-matches to proceed
if ldap query returns more than on match
we should reject the auth request instead of picking
the first one
2023-09-28 21:20:50 +02:00
Zaiming (Stone) Shi d858f8af39 test: fix openldap docker runs 2023-09-28 18:40:03 +02:00
Zaiming (Stone) Shi b28e781c50 fix(ldap-sso): do not log error level when invalid user credentials 2023-09-28 18:37:11 +02:00
Zaiming (Stone) Shi 36f3052be1
Merge pull request #11698 from zmstone/0928-disable-audit-log-by-default
fix(audit): disable audit log by default
2023-09-28 17:28:23 +02:00
Zaiming (Stone) Shi 39820be5ff
Merge pull request #11703 from lafirest/fix/ldap_bind
fix(ldap): use the search result as bind target
2023-09-28 14:48:53 +02:00
Zaiming (Stone) Shi 1a13b2ac56 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-28 13:53:03 +02:00
firest 43ea367df8 fix(ldap): use the search result as bind target 2023-09-28 18:49:00 +08:00
Zaiming (Stone) Shi 98409c9b1e
Merge pull request #11702 from zmstone/0928-minor-chore
0928 minor chore
2023-09-28 12:39:55 +02:00
Zaiming (Stone) Shi 26cadb2248 fix(audit): disable audit log by default 2023-09-28 12:36:14 +02:00
Zaiming (Stone) Shi b14425d865 fix(sso/saml): ensure lower case http header name 2023-09-28 11:40:21 +02:00
Zaiming (Stone) Shi 7ddcd71412 docs(ssl_dist.conf): update doc link 2023-09-28 11:16:18 +02:00
firest dca8fdb17f fix(resource): respect the start_timeout 2023-09-28 16:36:41 +08:00
ieQu1 b77e5e880a
Merge pull request #11697 from ieQu1/dev/gen-rpc-3.1.1
chore(gen_rpc): Bump version to 3.1.1
2023-09-28 10:03:36 +02:00
JianBo He 6d2adfc259 chore(audit): emit the message field 2023-09-28 11:04:42 +08:00
JianBo He 859b122cdd chore: format codes 2023-09-28 10:25:37 +08:00
JianBo He b52e4ac99d chore: update apps/emqx_machine/src/emqx_restricted_shell.erl
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-09-28 10:25:37 +08:00
JianBo He bdf24d0ec8 chore: remove the `unauthorized` type for `from` field 2023-09-28 10:25:37 +08:00
JianBo He 71acf121ba chore(audit): distinguish requests from rest_api or dashboard 2023-09-28 10:25:37 +08:00
JianBo He dd9938114c chore(audit): add from field 2023-09-28 10:25:37 +08:00
JianBo He ece7d5b52a
Merge pull request #11690 from lafirest/fix/ldap_parse_cfg
fix(ldap): improve the LDAP `parse_config` function
2023-09-28 10:24:52 +08:00
lafirest b0d86eecd6
Merge pull request #11691 from lafirest/fix/sso_ssl
fix(sso): support for SSL update && ensure update is atomic
2023-09-28 10:22:34 +08:00
firest 57781d0544 fix(ldap): remove the parse_config, it never work 2023-09-28 09:38:16 +08:00
firest 9dee2dc31e fix(sso): clear last error first before update && fix the `running` 2023-09-28 08:56:16 +08:00
ieQu1 0aa3ccdd65 feat(gen_rpc): Add schema for the TLS versions and cipher suites 2023-09-28 00:36:59 +02:00
Zaiming (Stone) Shi afdda107af fix(logger): json format log encode binary list as string array 2023-09-27 23:40:01 +02:00
Zaiming (Stone) Shi 6f7a4344dc fix: do not gc sso saml SP singing keys 2023-09-27 23:19:39 +02:00
Zaiming (Stone) Shi 34367fc4ec fix(audit_log): pretty print shell args 2023-09-27 23:19:39 +02:00
Zaiming (Stone) Shi bb49914fd6 fix(sso): add convet_certs callback for sso backends
must convert certs in pre_config_update so the cert path refernces
are stored in raw config, otherwise the files might get gc:ed
2023-09-27 22:41:39 +02:00
Zaiming (Stone) Shi 45caa3bf01 fix(sso): make sp_private_key sensitive
so it will not be logged
2023-09-27 21:27:59 +02:00
Zaiming (Stone) Shi c8cbbff044 fix(logger): no need for special handling of empty string
when formating json logs, there is no need to handle empty strings
special, already covered by unicode handling
2023-09-27 21:26:47 +02:00
ieQu1 3c37f19105 chore(gen_rpc): Bump version to 3.1.1 2023-09-27 21:09:00 +02:00