yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,112 Commits 393 Branches 684 Tags 74 MiB
5aac90ab4e
Commit Graph

9376 Commits

Author SHA1 Message Date
Stefan Strigler
8567ccafc1 fix: don't assume resource-id starts with "connector:" 2023-10-27 11:53:40 +02:00
Stefan Strigler
4b4eb19b0b fix: add schema desc for connector 2023-10-27 11:53:40 +02:00
Stefan Strigler
c9b683fd78 fix: add missing function for non enterprise 2023-10-27 11:53:40 +02:00
Stefan Strigler
229bc0ee34 fix: schema fixes for connector and kafka as connector 2023-10-27 11:53:40 +02:00
Stefan Strigler
64faf32842 style: cleanup examples 2023-10-27 11:53:40 +02:00
Stefan Strigler
7641c22455 fix: add missing parts 2023-10-27 11:53:40 +02:00
Stefan Strigler
ed8aa46602 refactor: copy bridge api code over to emqx_connector 2023-10-27 11:53:40 +02:00
Stefan Strigler
d05f2010b3 fix: fail remove connector with active channels 2023-10-27 11:53:40 +02:00
Stefan Strigler
cc864f4821 feat: add update/3 2023-10-27 11:53:40 +02:00
Stefan Strigler
9195838c17 test: add disable_enable tests 2023-10-27 11:53:40 +02:00
Stefan Strigler
009b9c3d39 style: remove unused commented fn 2023-10-27 11:53:40 +02:00
Kjell Winblad
c0df85ac09 feat: Bridge V2 compatiblilty layer progress and local topic 
* Most Bridge V1 HTTP API calls are now compatible with Bridge V2
* Local topics works for Bridge V2 now
* A lot of work on trying to get the old Kafka producer test suite
  to work after the refactorings
 2023-10-27 11:53:40 +02:00
Stefan Strigler
16d7f4d3e6 style: cleanup emqx_connector_resource 2023-10-27 11:53:40 +02:00
Stefan Strigler
99904cab27 fix: don't break root for non enterprise 2023-10-27 11:53:40 +02:00
Kjell Winblad
5374d35be3 feat: started to make bridge_v2 compatible with old style bridges 2023-10-27 11:53:40 +02:00
Stefan Strigler
bc6e1da2fb test: add suite for emqx_connector 2023-10-27 11:53:40 +02:00
Stefan Strigler
8f109da62b style: cleanup emqx_connector 2023-10-27 11:53:40 +02:00
Kjell Winblad
f7984be946 feat: add connector schema scaffold and break out Kafka conector 
This commit is the beginning of an effort to split bridges into a
connector part and a bridge part.

Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.

Issues:
https://emqx.atlassian.net/browse/EMQX-10805
 2023-10-27 11:53:40 +02:00
Zaiming (Stone) Shi
97601171a5
Merge pull request #11818 from zmstone/1025-fix-flaky-test 
test: fix a flaky test case emqx_crl_cache_SUITE:t_revoked
 2023-10-25 20:46:31 +02:00
Zaiming (Stone) Shi
76a8b0efd4 test: fix a flaky test case emqx_crl_cache_SUITE:t_revoked 2023-10-25 16:08:31 +02:00
William Yang
f52b4a9e32 fix: gen_rpc tcp client port align with server port 2023-10-25 12:11:35 +02:00
William Yang
37dd44c3d7
Merge pull request #11813 from qzhuyan/fix/william/genrpc-ssl-port-align 
Fix/william/genrpc ssl port align
 2023-10-25 09:04:19 +02:00
William Yang
6346e0d28a fix(gen_rpc): ssl client port align with server port 2023-10-24 22:33:52 +02:00
Zaiming (Stone) Shi
abcb3166f6 fix: use binary string for raw default values 2023-10-24 14:46:01 +02:00
JimMoen
99fab8dc5d
chore: missing change log for duplicated apikey fixing 2023-10-24 09:44:39 +08:00
William Yang
a9b42af8b3
Merge pull request #11796 from qzhuyan/fix/william/gen_rpc-driver 
fix(gen_rpc): ensure client/server use same driver
 2023-10-23 08:24:17 +02:00
JimMoen
e6576951ef
test: cleanup duplicated apikey with different name 2023-10-20 22:33:26 +08:00
JimMoen
d467289bb2
fix: avoid duplicated apikey from data import 2023-10-20 21:13:04 +08:00
William Yang
dd5807b797 fix(gen_rpc): ensure client/server use same driver 
Also bump to gen_rpc 3.2.1
 2023-10-20 12:17:49 +02:00
Ilya Averyanov
90a0c093bf
Merge pull request #11750 from savonarola/1011-redact-auth-http-request 
Do not trace authn http request bodies
 2023-10-13 17:25:28 +03:00
Andrew Mayorov
51c57a5f0a
fix(cass): avoid using aggregates in healthcheck query 
Otherwise, this query will produce warning message per each
healthcheck, telling the user:
```
WARN  <...> - Aggregation query used without partition key
```
 2023-10-13 16:50:19 +07:00
Ilya Averyanov
68f31a9da2 fix(authn): do not trace authn requests 2023-10-12 16:32:16 +03:00
Andrew Mayorov
6413afd0bc
fix(gcpdev): restore original shard in mnesia tab definition 
Changing the shard is not backward compatible, and leads to a crash
when upgrading from 5.2.x.
 2023-10-12 17:30:07 +07:00
Zaiming (Stone) Shi
d9859b6b49
Merge pull request #11749 from thalesmg/test-fix-flaky-http-suite-r53-20231011 
test: attempt to fix flaky http bridge suite
 2023-10-11 16:07:17 +02:00
William Yang
e6358ca18c
Merge pull request #11747 from qzhuyan/dev/william/quicer-0.0.202 
bump to quicer 0.0.202
 2023-10-11 15:43:47 +02:00
Thales Macedo Garitezi
ab662f506d test: attempt to fix flaky http bridge suite 
```
 =CRASH REPORT==== 10-Oct-2023::05:35:30.911371 ===
  crasher:
    initial call: ehttpc:init/1
    pid: <0.23358.2>
    registered_name: []
    exception error: bad argument
      in function  persistent_term:get/1
         called as persistent_term:get({emqx_bridge_http_SUITE,
                                        do_t_async_retries,attempts})
         *** argument 1: no persistent term stored with this key
      in call from emqx_bridge_http_SUITE:'-do_t_async_retries/3-fun-2-'/0 (/__w/emqx/emqx/apps/emqx_bridge_http/test/emqx_bridge_http_SUITE.erl, line 697)
      in call from emqx_bridge_http_SUITE:'-do_t_async_retries/3-fun-4-'/6 (/__w/emqx/emqx/apps/emqx_bridge_http/test/emqx_bridge_http_SUITE.erl, line 705)
```
 2023-10-11 09:13:25 -03:00
Ilya Averyanov
57655854d1
Merge pull request #11743 from savonarola/1010-auth-leftovers 
chore(auth): cleanup code
 2023-10-11 15:09:07 +03:00
William Yang
a69c53455f fix: bump to quicer 0.0.202 
bring in urgent fixs from msquic 2.2.3
 2023-10-11 13:12:32 +02:00
Ilya Averyanov
03ae5bf3c8 chore(auth): cleanup code 2023-10-11 13:13:50 +03:00
Zaiming (Stone) Shi
a2e86c67db
Merge pull request #11733 from keynslug/fix/EMQX-10827/sesson-takeover 
fix(cm): bring back pre-v5.3.0 compat in `takeover_session_begin/1`
 2023-10-10 19:03:29 +02:00
Thales Macedo Garitezi
ee45145fb5
Merge pull request #11724 from thalesmg/fix-kprodu-sync-metrics-m-20231006 
fix({kafka,pulsar}_producer): correctly handle metrics for connectors that have internal buffers
 2023-10-10 12:13:33 -03:00
Ivan Dyachkov
bfb2218392 chore: fix release version 2023-10-10 14:50:49 +02:00
Thales Macedo Garitezi
cf2075d7d8 chore: remove mention of is_buffer_supported from typespec 2023-10-10 09:49:18 -03:00
Zaiming (Stone) Shi
143673b703 chore: bump release version to 5.3.1-alpha.1 2023-10-10 14:35:46 +02:00
Zaiming (Stone) Shi
0a9e1dfa88
Merge pull request #11734 from zmstone/1002-support-ipv6-only-clustering 
Make IPv6 clustering options configurable
 2023-10-10 14:29:52 +02:00
Andrew Mayorov
9b573834f9
Merge pull request #11736 from keynslug/fix/eval-erl-audit 
fix(audit): handle abstract forms when logging `eval_erl`
 2023-10-10 15:49:19 +07:00
Zaiming (Stone) Shi
e7e696cd66 feat(rpc): add ipv6_only config 2023-10-10 10:05:14 +02:00
firest
e07937a3ef fix(ldap): escape the escape character (\) 2023-10-10 13:49:36 +08:00
Andrew Mayorov
2a291dfd27
fix(audit): handle abstract forms when logging eval_erl 2023-10-10 12:31:19 +07:00
Andrew Mayorov
522302fee1
fix(cm): bring back pre-v5.3.0 compat in takeover_session_begin/1 
Which was accidentally broken in bf164175.
 2023-10-10 01:34:10 +07:00
Zaiming (Stone) Shi
1e93d2f1fc chore: upgrade to ekka 0.15.16 gen_rpc 3.2.0 
* ekka 0.15.16 supports 'inet6_tls' as ekka.proto_dist
* gen_rpc 3.2.0 supports true | false as gen_rpc.ipv6_only
 2023-10-09 20:03:21 +02:00
Zaiming (Stone) Shi
497e08448d feat(cluster): support ipv6 and tls on ipv6 for clustering 
Made possible to configure inet6_tls for Erlang distribution
Also, added support to configure ipv6 listener for gen_rpc
 2023-10-09 20:03:21 +02:00
Zaiming (Stone) Shi
03d8e06ff7 chore(emqx_rule_funcs): regroup export functions 2023-10-09 20:03:21 +02:00
Thales Macedo Garitezi
d6781efee2 fix(resource): change how buffer workers are started 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
902b1d6ec5 fix(pulsar_producer): use simple_async_internal_buffer query mode for Pulsar 
Since it has internal buffering, it necessitates the same fix as Kafka producer.
 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
eebfb44f72 fix(resource): create simple_async_internal_buffer query mode for bridges with internal buffering 
Since authn/authz backends also use simple async/sync queries, we may want to avoid them
calling the connector when it's not connected.
 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
79cf0a2ced fix(kafka_producer): correctly handle metrics for connector that have internal buffers 
Fixes https://emqx.atlassian.net/browse/EMQX-11086

There’s currently a metric inconsistency due to the internal buffering nature of Kafka
Producer (wolff).

We use simple_sync_query to call the Kafka Producer bridge.  If that times out, the call
is accounted as failed, even though the message is buffered in wolff and later sent
successfully.
 2023-10-09 15:02:25 -03:00
Zaiming (Stone) Shi
c60915293a
Merge pull request #11634 from savonarola/0919-auth-refactor 
Auth refactor
 2023-10-09 19:51:26 +02:00
lafirest
8f4cdc3fcf
Merge pull request #11732 from lafirest/test/sso_cli 
test(ldap): add test suite for SSO CLI
 2023-10-09 20:35:23 +08:00
Zaiming (Stone) Shi
9ab49a7ae3 Merge remote-tracking branch 'origin/master' into 0919-auth-refactor 2023-10-09 14:18:10 +02:00
lafirest
2c7e5eb1cb
Merge pull request #11728 from lafirest/fix/ldap_filter 
fix(ldap): improve the filter lex && parse
 2023-10-09 18:24:57 +08:00
firest
e3550fc07b test(ldap): add test suite for SSO CLI 2023-10-09 16:05:56 +08:00
zhongwencool
0c89b6b213 docs: swagger api tags should camel case 2023-10-09 15:21:51 +08:00
firest
b2a6724dc2 fix(ldap): improve the filter lex && parse 
1. auto escape special chars in the filter
2. fix a bug that the value can't be `dn`
 2023-10-08 18:39:20 +08:00
zhongwencool
482e82f914 fix: cli's args in audit log should be array 2023-10-07 17:38:02 +08:00
Thales Macedo Garitezi
34186fcc74 fix(kafka_producer): send messages to wolff producer to buffer even when connector is in connecting state 
Fixes https://emqx.atlassian.net/browse/EMQX-11085

Messages would not be sent to wolff if the connection was down, so they were effectively lost.
 2023-10-06 11:43:29 -03:00
Ilya Averyanov
398a62031e chore: update app versions 2023-10-05 13:41:50 +03:00
Ilya Averyanov
5dff36474d chore(auth): get rid of hardcoded schema modules in auth 2023-10-05 13:41:50 +03:00
Ilya Averyanov
c2c56ba481 chore(auth): update tests 2023-10-05 13:41:50 +03:00
Ilya Averyanov
1eb75b43c4 chore(auth): split emqx_authn and emqx_authz apps 2023-10-05 13:41:50 +03:00
Zaiming (Stone) Shi
fd75dc895d fix(authn): "authentication" importance should not be "hidden" 2023-10-03 11:18:21 +02:00
Zaiming (Stone) Shi
ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53 
0928 sync release 53
 2023-10-02 21:57:32 +02:00
Zaiming (Stone) Shi
3f6e0e890b chore: bump emqx_durable_storage and emqx_license app vsn 2023-10-02 20:13:09 +02:00
Thales Macedo Garitezi
2358d67908 refactor: move session stuff from emqx_ds to emqx_persistent_session_ds 
Part of https://emqx.atlassian.net/browse/EMQX-10942
 2023-09-29 18:00:24 -03:00
Zaiming (Stone) Shi
bce8fd2fbc chore: bump app versions 2023-09-29 18:56:52 +02:00
Zaiming (Stone) Shi
ed5b456d62 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 18:24:44 +02:00
Zaiming (Stone) Shi
a852400fb3 chore: add 5.3.bpapi 2023-09-29 16:34:04 +02:00
Zaiming (Stone) Shi
fe01aaff4f chore: prepare for e5.3.0 promotion 2023-09-29 13:32:57 +02:00
Zaiming (Stone) Shi
ce5bd0a3ce Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 11:36:32 +02:00
Zaiming (Stone) Shi
c64e599e81 docs: document how to retrieve peercert 2023-09-29 10:33:57 +02:00
Zaiming (Stone) Shi
dc147fd310 fix(rule-engine): console action has no args field 2023-09-29 10:33:33 +02:00
Zaiming (Stone) Shi
1177a32310 chore: bump version to 5.3.0-rc.2 2023-09-29 09:49:27 +02:00
Zaiming (Stone) Shi
6891234390 chore: return simplified error reason for less logging 2023-09-29 09:48:15 +02:00
Zaiming (Stone) Shi
c2d750aa09 fix(resource): redact query args in exception log 2023-09-29 09:20:42 +02:00
Zaiming (Stone) Shi
02ef854f0f fix(ldap): no crash when no query result is empty list 2023-09-29 08:54:41 +02:00
Zaiming (Stone) Shi
b59a7ff2dd
Merge pull request #11706 from zmstone/0928-fix-ldap-sso-logging-level 
0928 fix ldap sso logging level and reject muti-match results
 2023-09-29 02:40:37 +02:00
Zaiming (Stone) Shi
4a4730ad46 fix(ldap): handle invalidCredentials in ldap authn 2023-09-29 00:51:05 +02:00
Zaiming (Stone) Shi
9ee2cb9c79 fix(ldap): return unrecoverable_error if more than on match found 2023-09-28 23:58:34 +02:00
Zaiming (Stone) Shi
cc5dab1dc7 chore: fix code style 2023-09-28 21:29:59 +02:00
Zaiming (Stone) Shi
b267fc2588 chore: bump release version to 5.3.0 2023-09-28 21:22:33 +02:00
Zaiming (Stone) Shi
922d5a9a83 fix(ldap): do not allow multi-matches to proceed 
if ldap query returns more than on match
we should reject the auth request instead of picking
the first one
 2023-09-28 21:20:50 +02:00
Zaiming (Stone) Shi
d858f8af39 test: fix openldap docker runs 2023-09-28 18:40:03 +02:00
Zaiming (Stone) Shi
b28e781c50 fix(ldap-sso): do not log error level when invalid user credentials 2023-09-28 18:37:11 +02:00
Zaiming (Stone) Shi
36f3052be1
Merge pull request #11698 from zmstone/0928-disable-audit-log-by-default 
fix(audit): disable audit log by default
 2023-09-28 17:28:23 +02:00
Zaiming (Stone) Shi
39820be5ff
Merge pull request #11703 from lafirest/fix/ldap_bind 
fix(ldap): use the search result as bind target
 2023-09-28 14:48:53 +02:00
Zaiming (Stone) Shi
1a13b2ac56 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-28 13:53:03 +02:00
firest
43ea367df8 fix(ldap): use the search result as bind target 2023-09-28 18:49:00 +08:00
Zaiming (Stone) Shi
98409c9b1e
Merge pull request #11702 from zmstone/0928-minor-chore 
0928 minor chore
 2023-09-28 12:39:55 +02:00
Zaiming (Stone) Shi
26cadb2248 fix(audit): disable audit log by default 2023-09-28 12:36:14 +02:00
Zaiming (Stone) Shi
b14425d865 fix(sso/saml): ensure lower case http header name 2023-09-28 11:40:21 +02:00