yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,847 Commits 393 Branches 684 Tags 74 MiB
dd85c981cd
Commit Graph

7779 Commits

Author SHA1 Message Date
zhongwencool
dd85c981cd
Merge pull request #10783 from zhongwencool/improve-authn 
feat: load changes to the authentication configuration from command line
 2023-06-02 15:58:13 +08:00
zhongwencool
9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve 
feat: support emqx_config:update([authrization],Conf) update
 2023-06-02 15:43:09 +08:00
lafirest
0f8824b40f
Merge pull request #10921 from lafirest/test/rocketmq_acl 
test(rocketmq): add an ACL test case
 2023-06-02 15:37:46 +08:00
firest
ea9b1e13d5 test(rocketmq): add an ACL test case 2023-06-02 11:19:06 +08:00
某文
367a0e312a chore: remove config_path() function 2023-06-02 07:09:08 +08:00
Andrew Mayorov
76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum 
fix(ft): respect checksum in `fin` packets
 2023-06-01 23:49:55 +03:00
Paulo Zulato
364601c3aa
Merge pull request #10892 from paulozulato/block-oracle-default-database-connection 
fix(oracle): require either sid or service name
 2023-06-01 15:04:03 -03:00
某文
2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
某文
c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
某文
94b86f01a5 chore: reformat SUITE case 2023-06-01 22:32:31 +08:00
zhongwencool
21df321d89
Merge branch 'master' into improve-authn 2023-06-01 22:24:59 +08:00
Thales Macedo Garitezi
ecbb6e0ecb test(global_gc): fix flaky test 2023-06-01 10:48:02 -03:00
Thales Macedo Garitezi
7740987a17
Merge pull request #10905 from thalesmg/fix-enforce-binary-rootkey-config-v50 
fix(config): enforce root key name to be a binary (raw config)
 2023-06-01 10:27:48 -03:00
Zhongwen Deng
7eea693422 fix: reorder authn when updating 2023-06-01 17:24:39 +08:00
Zhongwen Deng
96ee38ce15 chore: bump up authn/ctl app version 2023-06-01 17:24:39 +08:00
Zhongwen Deng
4f5d5216bb test: bad authn handler callback module in SUITE 2023-06-01 17:24:39 +08:00
Zhongwen Deng
8f12d81878 feat: support emqx_ctl conf command 2023-06-01 17:24:39 +08:00
Zhongwen Deng
b290d2543b fix: don't crash when command_ctl table not init 2023-06-01 17:24:39 +08:00
Andrew Mayorov
b5f24c4f88
Merge pull request #10900 from keynslug/fix/EEC-856/parsed-bridge-id 
fix(ruleeng): parse bridge id on create / update
 2023-06-01 10:21:26 +03:00
JianBo He
bcc5e30f22
Merge pull request #10895 from lafirest/fix/refactor_on_stop 
feat: refactored some bridges to avoid leaking resources during crashes at creation
 2023-06-01 15:17:37 +08:00
firest
cb31032e6d chore: make dialyzer && test happy 2023-06-01 11:15:51 +08:00
Thales Macedo Garitezi
f83f112189 fix(config): enforce root key name to be a binary (raw config) 
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299

```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
    exception: error
    key_path: [listeners,tcp,default,authentication]
    module: emqx_schema
    msg: change_config_crashed
    reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
                               <<"authentication">>]}
    stacktrace: [{emqx_utils_maps,deep_get,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
                      {line,49}]},
                 {emqx_config_handler,return_change_result,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,456}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,284}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,
                     {create_authenticator,'tcp:default',
                         #{backend => built_in_database,enable => true,
                           mechanism => password_based}}},
                 #{rawconf_with_defaults => true}}
```

Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:

```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
 #{<<"ssl">> =>
       #{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```

... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
 2023-05-31 18:03:58 -03:00
Paulo Zulato
3a6ad3b049 fix(oracle): require either sid or service name 
Fixes https://emqx.atlassian.net/browse/EMQX-9980
 2023-05-31 17:18:45 -03:00
Thales Macedo Garitezi
b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e 
more attempts to fix more test flakiness
 2023-05-31 16:07:18 -03:00
Andrew Mayorov
aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard 
test(dashboard): fix inter-suite test flakiness
 2023-05-31 20:28:39 +03:00
Andrew Mayorov
f90ce9bbfe
test(dashboard): fix inter-suite test flakiness 2023-05-31 19:09:37 +03:00
Andrew Mayorov
0293b54211
fix(ft): respect checksum in fin packets 2023-05-31 18:50:38 +03:00
Thales Macedo Garitezi
189e2c87be test(channel): fix flaky test 
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```

The problem was that the whole generated message was compared for
equality when receiving the will message, but the timestamp may be
different at publishing time...
 2023-05-31 12:42:24 -03:00
JianBo He
a3a6480f00
Merge pull request #10880 from HJianBo/batch-kickout-clients 
feat: support kickout clients in batch
 2023-05-31 22:36:35 +08:00
JianBo He
8ce0132569
Merge pull request #10871 from HJianBo/fix-coap-bugs 
fix(coap): to better handle coap requests in connection mode
 2023-05-31 22:36:15 +08:00
JianBo He
c67135dd4b
Merge pull request #10598 from HJianBo/refactor_exproto_stream 
refactor(exproto): support unary handler
 2023-05-31 22:36:01 +08:00
Kjell Winblad
036f180c27
Merge pull request #10878 from kjellwinblad/kjell/bridge/rabbitmq_password_log_leak/EMQX-9976 
fix: password log leak in RabbitMQ bridge
 2023-05-31 16:17:19 +02:00
Andrew Mayorov
29fe201676
fix(ruleeng): parse bridge id on create / update 
Instead of parsing it every time in the `send_message/2` hot path.
 2023-05-31 17:08:42 +03:00
Thales Macedo Garitezi
77c9eda036 test: rm stray clear_screen calls 2023-05-31 11:07:44 -03:00
Thales Macedo Garitezi
83a7c800cc test(kafka_consumer): reduce flakiness 
The crash reason can also be `{error, not_found}` depending on
timing...

```
=ERROR REPORT==== 31-May-2023::12:17:31.461490 ===
    exception: error
    key_path: [bridges,kafka_consumer,
               't_resource_manager_crash_before_subscriber_started-576460752303419630']
    module: emqx_conf_schema
    msg: change_config_crashed
    reason: {badmatch,{error,not_found}}
    stacktrace: [{emqx_resource_manager,create_and_return_data,5,
                     [{file,
                          "/emqx/apps/emqx_resource/src/emqx_resource_manager.erl"},
                      {line,119}]},
                 {emqx_bridge_resource,create,4,
                     [{file,
                          "/emqx/apps/emqx_bridge/src/emqx_bridge_resource.erl"},
                      {line,171}]},
                 {emqx_bridge,'-perform_bridge_changes/2-fun-1-',5,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,367}]},
                 {maps,fold_1,3,[{file,"maps.erl"},{line,411}]},
                 {emqx_bridge,perform_bridge_changes,2,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,354}]},
                 {emqx_bridge,post_config_update,5,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,229}]},
                 {emqx_config_handler,call_post_config_update,7,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,398}]},
                 {emqx_config_handler,do_post_config_update,8,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,336}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,281}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,#{<<"authentication">> => <<"none">>,
                           <<"bootstrap_hosts">> =>
                               <<"toxiproxy.emqx.net:9292">>,
                           <<"connect_timeout">> => <<"5s">>,
                           <<"enable">> => true,
                           <<"kafka">> =>
                               #{<<"max_batch_bytes">> => <<"896KB">>,
                                 <<"max_rejoin_attempts">> => 5,
                                 <<"offset_commit_interval_seconds">> => 3,
                                 <<"offset_reset_policy">> => <<"latest">>},
                           <<"key_encoding_mode">> => <<"none">>,
                           <<"metadata_request_timeout">> => <<"5s">>,
                           <<"min_metadata_refresh_interval">> => <<"3s">>,
                           <<"ssl">> =>
                               #{<<"enable">> => false,
                                 <<"server_name_indication">> => <<"auto">>,
                                 <<"verify">> => <<"verify_none">>},
                           <<"topic_mapping">> =>
                               [#{<<"kafka_topic">> =>
                                      <<"t_resource_manager_crash_before_subscriber_started-576460752303419662">>,
                                  <<"mqtt_topic">> =>
                                      <<"mqtt/topic/-576460752303422911">>,
                                  <<"payload_template">> => <<"${.}">>,
                                  <<"qos">> => 0}],
                           <<"value_encoding_mode">> => <<"none">>}},
                 #{override_to => cluster}}
```
 2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi
e7b35011e6 test(oracle): attempt to fix flaky test 
https://github.com/emqx/emqx/actions/runs/5126014459/jobs/9220155908#step:7:754
```
=CRITICAL REPORT==== 30-May-2023::20:50:26.267103 ===
Run stage failed: error:{assertMatch,
                         [{module,emqx_bridge_oracle_SUITE},
                          {line,394},
                          {expression,
                           "emqx_resource : simple_sync_query ( ResourceId , { query , \"SELECT COUNT(*) FROM mqtt_test\" } )"},
                          {pattern,
                           "{ ok , [ { result_set , _ , _ , [ [ { 3 } ] ] } ] }"},
                          {value,
                           {ok,[{result_set,[<<"COUNT(*)">>],[],[[{0}]]}]}}]}
Stacktrace: [{emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-5-',1,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,394}]},
             {emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-6-',5,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,390}]},
             {emqx_bridge_oracle_SUITE,t_batch_sync_query,1,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,364}]}]
```
 2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi
0d539e91d1 test(pulsar_producer): attempt to stabilize flaky test 
https://github.com/emqx/emqx/actions/runs/5125166433/jobs/9218613872?pr=10886#step:7:679
```
  =CRITICAL REPORT==== 30-May-2023::19:38:58.003170 ===
  Run stage failed: error:{badmatch,
                              {timeout,
                                  [#{msg => pulsar_producer_bridge_started,
                                     '~meta' =>
                                         #{gl => <97891.472.0>,
                                           location =>
                                               #Fun<emqx_bridge_pulsar_impl_producer.11.109752493>,
                                           node => 'autocluster_node1@127.0.0.1',
                                           pid => <97891.787.0>,
                                           time => -576460611692219}}]}}
  Stacktrace: [{emqx_bridge_pulsar_impl_producer_SUITE,'-t_cluster/1-fun-10-',
                   6,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1073}]},
               {emqx_bridge_pulsar_impl_producer_SUITE,t_cluster,1,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1064}]}]
```
 2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi
1e25ebb64c test(gcp_pubsub): attempt to fix flakiness 
https://github.com/emqx/emqx/actions/runs/5125118728/jobs/9218520994?pr=10887#step:8:309
```
  =CRITICAL REPORT==== 30-May-2023::19:19:34.887082 ===
  "check stage" failed: error
  {assertMatch,[{module,emqx_bridge_gcp_pubsub_SUITE},
                {line,1066},
                {expression,"? of_kind ( gcp_pubsub_request_failed , Trace )"},
                {pattern,"[ # { reason := Error , connector := ResourceId } | _ ]"},
                {value,[#{connector =>
                              <<"bridge:gcp_pubsub:emqx_bridge_gcp_pubsub_SUITE0005FCEE15534E9CD4CD02004CF10000">>,
                          msg => gcp_pubsub_request_failed,query_mode => async,
                          reason => {closed,"The connection was lost."},
                          recoverable_error => true,
                          '~meta' =>
                              #{gl => <0.17903.2>,
                                location =>
                                    #Fun<emqx_bridge_gcp_pubsub_connector.19.19548918>,
                                node => 'test@127.0.0.1',pid => <0.19724.2>,
                                time => -576460610660164}}]}]}
  Stacktrace: [{emqx_bridge_gcp_pubsub_SUITE,
                   '-do_econnrefused_or_timeout_test/2-fun-2-',3,
                   [{file,
                        "/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
                    {line,1066}]},
               {emqx_bridge_gcp_pubsub_SUITE,do_econnrefused_or_timeout_test,2,
                   [{file,
                        "/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
                    {line,1022}]}]
```
 2023-05-31 10:19:55 -03:00
lafirest
502b24bbe2
Merge pull request #10877 from lafirest/fix/rocketmq_sensitive_data 
fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data
 2023-05-31 18:54:44 +08:00
zhongwencool
26c2ab0bef
Merge pull request #10894 from zhongwencool/crl-cache-conf-hot-update 
feat: update crl_cache conf at runtime
 2023-05-31 17:59:09 +08:00
firest
ce32526c96 chore: bump version && update changes 2023-05-31 17:31:56 +08:00
William Yang
2250ed7052
Merge pull request #10823 from qzhuyan/dev/william/meck-olp 
test: use meck in olp tests
 2023-05-31 11:01:56 +02:00
firest
6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Zhongwen Deng
38e57f511c feat: update crl_cache conf at runtime 2023-05-31 16:18:03 +08:00
JianBo He
837dfeb46f feat(exproto): add raw_publish function 2023-05-31 13:44:51 +08:00
JianBo He
5256d8092c fix: keep the real closed_reason 2023-05-31 13:44:51 +08:00
JianBo He
2e4ec4888d fix(exproto): ensure the on_socket_closed event delivering correctly 2023-05-31 13:44:51 +08:00
JianBo He
9dff026f50 test: fix failed test cases 2023-05-31 13:44:51 +08:00
JianBo He
b6bc3cd921 refactor(exproto): support unary handler 2023-05-31 13:44:51 +08:00
JianBo He
afcd9fcb5e chore: update bpapi.versions 2023-05-31 11:29:55 +08:00