yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,866 Commits 393 Branches 684 Tags 74 MiB
33aa879ad4
Commit Graph

15866 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Thales Macedo Garitezi
33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50 
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
 2023-06-02 16:20:36 -03:00
Thales Macedo Garitezi
90d862cad9
Merge pull request #10915 from thalesmg/fvt-retry-plugin-v50 
ci(fvt): add `pytest-retry` plugin
 2023-06-02 09:26:43 -03:00
Thales Macedo Garitezi
74ffd9ef96
Merge pull request #10918 from thalesmg/fix-ecpool-empty-unrecoverable-v50 
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
 2023-06-02 09:21:00 -03:00
Zaiming (Stone) Shi
367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition 
fix(emqx_cm): fix channel data registration race-condition
 2023-06-02 14:20:18 +02:00
Thales Macedo Garitezi
940353cc52 docs: add changelog 2023-06-02 09:13:47 -03:00
Thales Macedo Garitezi
0790c88aaf
refactor: use default's type as first union member 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-06-02 09:08:11 -03:00
Thales Macedo Garitezi
2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50 
fix(plugins): use `emqx:running_nodes` for multicall operations
 2023-06-02 09:03:42 -03:00
Zaiming (Stone) Shi
c75e9bbe0d fix(emqx_cm): fix channel data registration race-condition 
when clustered, there are chances the a mqtt client process
get killed (e.g. holding the channel registeration lock for too long),
if the channel data inserts happen before casting out the message
for channel process monitoring, there is a chance for the
stale message left in the ets tables indefinitely.

this commit changes the order of the non-atomic operations:
it casts out the monitor request message before inserting
channel data.
 2023-06-02 11:43:35 +02:00
zhongwencool
8d8efe449e
Merge pull request #10896 from zhongwencool/api-keys-hot-conf-update 
feat: api_key conf support hot conf
 2023-06-02 15:58:45 +08:00
zhongwencool
dd85c981cd
Merge pull request #10783 from zhongwencool/improve-authn 
feat: load changes to the authentication configuration from command line
 2023-06-02 15:58:13 +08:00
zhongwencool
9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve 
feat: support emqx_config:update([authrization],Conf) update
 2023-06-02 15:43:09 +08:00
lafirest
0f8824b40f
Merge pull request #10921 from lafirest/test/rocketmq_acl 
test(rocketmq): add an ACL test case
 2023-06-02 15:37:46 +08:00
Ivan Dyachkov
1761e669f0
Merge pull request #10916 from id/0601-add-ui-tests-in-ci 
ci: add ui(dashboard) tests based on pytest and selenium
 2023-06-02 09:07:41 +02:00
firest
ea9b1e13d5 test(rocketmq): add an ACL test case 2023-06-02 11:19:06 +08:00
某文
367a0e312a chore: remove config_path() function 2023-06-02 07:09:08 +08:00
Thales Macedo Garitezi
e47e2c9c12 ci(fvt): add pytest-retry plugin 2023-06-01 18:05:27 -03:00
Andrew Mayorov
76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum 
fix(ft): respect checksum in `fin` packets
 2023-06-01 23:49:55 +03:00
Thales Macedo Garitezi
7055bd8564
Merge pull request #10535 from thalesmg/spellcheck-assert-file-exists-v50 
ci: check if given schema.json exists before running spellcheck
 2023-06-01 17:18:38 -03:00
Thales Macedo Garitezi
0072f4a598 fix(ecpool,bridge): treat {error, ecpool_empty} as a retriable error 
Example from flaky CI test:

```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
 2023-06-01 17:13:06 -03:00
Paulo Zulato
364601c3aa
Merge pull request #10892 from paulozulato/block-oracle-default-database-connection 
fix(oracle): require either sid or service name
 2023-06-01 15:04:03 -03:00
Ivan Dyachkov
8cdb1458c7 ci: add ui(dashboard) tests based on pytest and selenium 2023-06-01 20:01:24 +02:00
Thales Macedo Garitezi
7d3ce14835 test: rm unused var warning 2023-06-01 14:00:43 -03:00
Thales Macedo Garitezi
648b8231ef fix(plugins): avoid crashing when fetching error reason 2023-06-01 14:00:43 -03:00
Thales Macedo Garitezi
64258120d7 fix(plugins): use emqx:running_nodes for multicall operations 
Fixes https://emqx.atlassian.net/browse/EMQX-10079
 2023-06-01 14:00:43 -03:00
某文
2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
Thales Macedo Garitezi
d320497d0c
Merge pull request #10904 from thalesmg/fix-global-gc-config-get-v50 
test(global_gc): fix flaky test
 2023-06-01 13:23:16 -03:00
Thales Macedo Garitezi
3e4790edd4 test(pulsar_producer): fix flaky test 2023-06-01 13:01:58 -03:00
Thales Macedo Garitezi
99796224d8 refactor(resource): rename request_timeout -> request_ttl 
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
 2023-06-01 13:01:53 -03:00
某文
c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
某文
94b86f01a5 chore: reformat SUITE case 2023-06-01 22:32:31 +08:00
zhongwencool
21df321d89
Merge branch 'master' into improve-authn 2023-06-01 22:24:59 +08:00
Thales Macedo Garitezi
f42ccb6262 feat(resource): increase default request timeout to 45 s 
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
 2023-06-01 11:20:06 -03:00
Thales Macedo Garitezi
10425eb925 feat(resource): deprecate auto_restart_interval in favor of health_check_interval 
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
 2023-06-01 11:20:06 -03:00
Thales Macedo Garitezi
ecbb6e0ecb test(global_gc): fix flaky test 2023-06-01 10:48:02 -03:00
Thales Macedo Garitezi
7740987a17
Merge pull request #10905 from thalesmg/fix-enforce-binary-rootkey-config-v50 
fix(config): enforce root key name to be a binary (raw config)
 2023-06-01 10:27:48 -03:00
Zhongwen Deng
7eea693422 fix: reorder authn when updating 2023-06-01 17:24:39 +08:00
Zhongwen Deng
96ee38ce15 chore: bump up authn/ctl app version 2023-06-01 17:24:39 +08:00
Zhongwen Deng
4f5d5216bb test: bad authn handler callback module in SUITE 2023-06-01 17:24:39 +08:00
Zhongwen Deng
8f12d81878 feat: support emqx_ctl conf command 2023-06-01 17:24:39 +08:00
Zhongwen Deng
b290d2543b fix: don't crash when command_ctl table not init 2023-06-01 17:24:39 +08:00
Andrew Mayorov
b5f24c4f88
Merge pull request #10900 from keynslug/fix/EEC-856/parsed-bridge-id 
fix(ruleeng): parse bridge id on create / update
 2023-06-01 10:21:26 +03:00
JianBo He
bcc5e30f22
Merge pull request #10895 from lafirest/fix/refactor_on_stop 
feat: refactored some bridges to avoid leaking resources during crashes at creation
 2023-06-01 15:17:37 +08:00
firest
cb31032e6d chore: make dialyzer && test happy 2023-06-01 11:15:51 +08:00
JianBo He
a402acdc43
Merge pull request #10897 from Kinplemelon/kinple/upgrade-dashboard 
chore: upgrade dashboard to e1.0.8-beta.1 for ee
 2023-06-01 09:46:01 +08:00
Ivan Dyachkov
239f71cecf
Merge pull request #10899 from id/0530-update-perf-test 
0530 update perf test
 2023-05-31 23:17:10 +02:00
Thales Macedo Garitezi
f83f112189 fix(config): enforce root key name to be a binary (raw config) 
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299

```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
    exception: error
    key_path: [listeners,tcp,default,authentication]
    module: emqx_schema
    msg: change_config_crashed
    reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
                               <<"authentication">>]}
    stacktrace: [{emqx_utils_maps,deep_get,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
                      {line,49}]},
                 {emqx_config_handler,return_change_result,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,456}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,284}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,
                     {create_authenticator,'tcp:default',
                         #{backend => built_in_database,enable => true,
                           mechanism => password_based}}},
                 #{rawconf_with_defaults => true}}
```

Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:

```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
 #{<<"ssl">> =>
       #{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```

... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
 2023-05-31 18:03:58 -03:00
Paulo Zulato
3a6ad3b049 fix(oracle): require either sid or service name 
Fixes https://emqx.atlassian.net/browse/EMQX-9980
 2023-05-31 17:18:45 -03:00
Thales Macedo Garitezi
b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e 
more attempts to fix more test flakiness
 2023-05-31 16:07:18 -03:00
Andrew Mayorov
aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard 
test(dashboard): fix inter-suite test flakiness
 2023-05-31 20:28:39 +03:00
Andrew Mayorov
f90ce9bbfe
test(dashboard): fix inter-suite test flakiness 2023-05-31 19:09:37 +03:00