Commit Graph

8118 Commits

Author SHA1 Message Date
某文 367a0e312a chore: remove config_path() function 2023-06-02 07:09:08 +08:00
Andrew Mayorov 76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum
fix(ft): respect checksum in `fin` packets
2023-06-01 23:49:55 +03:00
Thales Macedo Garitezi 0072f4a598 fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
Example from flaky CI test:

```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
2023-06-01 17:13:06 -03:00
Paulo Zulato 92c72c6994 test(oracle): add match for reason field in a test case
Fixes https://emqx.atlassian.net/browse/EMQX-9980
2023-06-01 15:12:09 -03:00
Paulo Zulato 364601c3aa
Merge pull request #10892 from paulozulato/block-oracle-default-database-connection
fix(oracle): require either sid or service name
2023-06-01 15:04:03 -03:00
Thales Macedo Garitezi 7d3ce14835 test: rm unused var warning 2023-06-01 14:00:43 -03:00
Thales Macedo Garitezi 648b8231ef fix(plugins): avoid crashing when fetching error reason 2023-06-01 14:00:43 -03:00
Thales Macedo Garitezi 64258120d7 fix(plugins): use `emqx:running_nodes` for multicall operations
Fixes https://emqx.atlassian.net/browse/EMQX-10079
2023-06-01 14:00:43 -03:00
某文 2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
Thales Macedo Garitezi 3e4790edd4 test(pulsar_producer): fix flaky test 2023-06-01 13:01:58 -03:00
Thales Macedo Garitezi 99796224d8 refactor(resource): rename `request_timeout` -> `request_ttl`
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 13:01:53 -03:00
某文 c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
Stefan Strigler 9c76aa901e fix(emqx_mgmt_api_clients): hide set_keepalive_seconds from swagger 2023-06-01 16:57:19 +02:00
某文 94b86f01a5 chore: reformat SUITE case 2023-06-01 22:32:31 +08:00
zhongwencool 21df321d89
Merge branch 'master' into improve-authn 2023-06-01 22:24:59 +08:00
Thales Macedo Garitezi f42ccb6262 feat(resource): increase default request timeout to 45 s
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 11:20:06 -03:00
Thales Macedo Garitezi 10425eb925 feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
Stefan Strigler e011b5532f
Merge pull request #10881 from sstrigler/EMQX-9355-5-0-x-test-coverage-80
IoTDB test coverage
2023-06-01 15:48:56 +02:00
Thales Macedo Garitezi ecbb6e0ecb test(global_gc): fix flaky test 2023-06-01 10:48:02 -03:00
Kjell Winblad 75ff76a16b fix: friendly error message when creating bridges with too long names
This commit makes the error message and log entry that appear when one
tries to create a bridge with a name the exceeds 255 bytes (the max
length for atoms) more friendly and easier to understand.

An even better fix would be to not store bridge names as atoms but this
probably requires a more substantial change.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9609
2023-06-01 15:30:06 +02:00
Thales Macedo Garitezi 7740987a17
Merge pull request #10905 from thalesmg/fix-enforce-binary-rootkey-config-v50
fix(config): enforce root key name to be a binary (raw config)
2023-06-01 10:27:48 -03:00
William Yang c1d9bc8033 perf(emqx_mqtt_caps): direct read mqtt from zone conf 2023-06-01 14:54:44 +02:00
William Yang 923913e15c feat(config): merge with global defaults when put new zone 2023-06-01 14:54:44 +02:00
William Yang a2f8e87389 test(config): default zone and global defaults 2023-06-01 14:54:14 +02:00
William Yang e693c6ec54 test: fix emqx_takeover_SUITE 2023-06-01 14:52:33 +02:00
William Yang 8c26ee75db test: remove hardcoded configs
test with schema defaults
2023-06-01 14:52:33 +02:00
William Yang 4416122cf3 test: fix longname/shortname 2023-06-01 14:52:33 +02:00
William Yang 8ee78ad6b4 test: fix emqx_mgmt_api_configs_SUITE 2023-06-01 14:52:33 +02:00
William Yang f621bf1f8e test(config): update emqx_client_SUITE 2023-06-01 14:52:33 +02:00
William Yang 98d6ed4251 test: fix emqx_session_SUITE 2023-06-01 14:52:33 +02:00
William Yang 6c1671651c test: use zone default in emqx_eviction_agent_channel_SUITE 2023-06-01 14:52:33 +02:00
William Yang 88f9782db0 perf(config): read zone conf once 2023-06-01 14:52:30 +02:00
firest 232ef23a48 feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation 2023-06-01 18:49:45 +08:00
Zhongwen Deng 7eea693422 fix: reorder authn when updating 2023-06-01 17:24:39 +08:00
Zhongwen Deng 96ee38ce15 chore: bump up authn/ctl app version 2023-06-01 17:24:39 +08:00
Zhongwen Deng 4f5d5216bb test: bad authn handler callback module in SUITE 2023-06-01 17:24:39 +08:00
Zhongwen Deng 8f12d81878 feat: support emqx_ctl conf command 2023-06-01 17:24:39 +08:00
Zhongwen Deng b290d2543b fix: don't crash when command_ctl table not init 2023-06-01 17:24:39 +08:00
Zaiming (Stone) Shi feecd53436 chore: bump to version 5.1.0-alpha.2 for both ce and ee 2023-06-01 10:13:56 +02:00
Andrew Mayorov b5f24c4f88
Merge pull request #10900 from keynslug/fix/EEC-856/parsed-bridge-id
fix(ruleeng): parse bridge id on create / update
2023-06-01 10:21:26 +03:00
JianBo He bcc5e30f22
Merge pull request #10895 from lafirest/fix/refactor_on_stop
feat: refactored some bridges to avoid leaking resources during crashes at creation
2023-06-01 15:17:37 +08:00
Zaiming (Stone) Shi 760de7becf chore: imporve logging message 2023-06-01 08:26:53 +02:00
Zaiming (Stone) Shi 15ffa98bc5 chore: fix a typo in test case function name 2023-06-01 08:18:26 +02:00
Zaiming (Stone) Shi cb2be1811e docs: fix function doc annotation 2023-06-01 08:17:36 +02:00
firest cb31032e6d chore: make dialyzer && test happy 2023-06-01 11:15:51 +08:00
Thales Macedo Garitezi f83f112189 fix(config): enforce root key name to be a binary (raw config)
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299

```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
    exception: error
    key_path: [listeners,tcp,default,authentication]
    module: emqx_schema
    msg: change_config_crashed
    reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
                               <<"authentication">>]}
    stacktrace: [{emqx_utils_maps,deep_get,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
                      {line,49}]},
                 {emqx_config_handler,return_change_result,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,456}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,284}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,
                     {create_authenticator,'tcp:default',
                         #{backend => built_in_database,enable => true,
                           mechanism => password_based}}},
                 #{rawconf_with_defaults => true}}
```

Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:

```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
 #{<<"ssl">> =>
       #{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```

... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
2023-05-31 18:03:58 -03:00
Paulo Zulato 3a6ad3b049 fix(oracle): require either sid or service name
Fixes https://emqx.atlassian.net/browse/EMQX-9980
2023-05-31 17:18:45 -03:00
Thales Macedo Garitezi b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e
more attempts to fix more test flakiness
2023-05-31 16:07:18 -03:00
Zaiming (Stone) Shi 5146de5b1c feat: add a backup copies for cluster.hocon 2023-05-31 20:34:25 +02:00
Zaiming (Stone) Shi 332daabcc5 fix(config): Do not sync cluster config from nodes running new version 2023-05-31 20:34:25 +02:00
Zaiming (Stone) Shi 39d6f612ca feat(emqx_release): add new APIs to parse/compare release versions 2023-05-31 20:34:25 +02:00
Andrew Mayorov aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard
test(dashboard): fix inter-suite test flakiness
2023-05-31 20:28:39 +03:00
Andrew Mayorov f90ce9bbfe
test(dashboard): fix inter-suite test flakiness 2023-05-31 19:09:37 +03:00
Andrew Mayorov 0293b54211
fix(ft): respect checksum in `fin` packets 2023-05-31 18:50:38 +03:00
Thales Macedo Garitezi 189e2c87be test(channel): fix flaky test
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```

The problem was that the whole generated message was compared for
equality when receiving the will message, but the timestamp may be
different at publishing time...
2023-05-31 12:42:24 -03:00
JianBo He a3a6480f00
Merge pull request #10880 from HJianBo/batch-kickout-clients
feat: support kickout clients in batch
2023-05-31 22:36:35 +08:00
JianBo He 8ce0132569
Merge pull request #10871 from HJianBo/fix-coap-bugs
fix(coap): to better handle coap requests in connection mode
2023-05-31 22:36:15 +08:00
JianBo He c67135dd4b
Merge pull request #10598 from HJianBo/refactor_exproto_stream
refactor(exproto): support unary handler
2023-05-31 22:36:01 +08:00
Kjell Winblad 036f180c27
Merge pull request #10878 from kjellwinblad/kjell/bridge/rabbitmq_password_log_leak/EMQX-9976
fix: password log leak in RabbitMQ bridge
2023-05-31 16:17:19 +02:00
Andrew Mayorov 29fe201676
fix(ruleeng): parse bridge id on create / update
Instead of parsing it every time in the `send_message/2` hot path.
2023-05-31 17:08:42 +03:00
Thales Macedo Garitezi 77c9eda036 test: rm stray `clear_screen` calls 2023-05-31 11:07:44 -03:00
Stefan Strigler 1e4cee05df fix(emqx_bridge_iotdb): handle rule engine passed payload
also remove topic logic as it's duplicated functionality via rule engine
2023-05-31 15:57:22 +02:00
Thales Macedo Garitezi 83a7c800cc test(kafka_consumer): reduce flakiness
The crash reason can also be `{error, not_found}` depending on
timing...

```
=ERROR REPORT==== 31-May-2023::12:17:31.461490 ===
    exception: error
    key_path: [bridges,kafka_consumer,
               't_resource_manager_crash_before_subscriber_started-576460752303419630']
    module: emqx_conf_schema
    msg: change_config_crashed
    reason: {badmatch,{error,not_found}}
    stacktrace: [{emqx_resource_manager,create_and_return_data,5,
                     [{file,
                          "/emqx/apps/emqx_resource/src/emqx_resource_manager.erl"},
                      {line,119}]},
                 {emqx_bridge_resource,create,4,
                     [{file,
                          "/emqx/apps/emqx_bridge/src/emqx_bridge_resource.erl"},
                      {line,171}]},
                 {emqx_bridge,'-perform_bridge_changes/2-fun-1-',5,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,367}]},
                 {maps,fold_1,3,[{file,"maps.erl"},{line,411}]},
                 {emqx_bridge,perform_bridge_changes,2,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,354}]},
                 {emqx_bridge,post_config_update,5,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,229}]},
                 {emqx_config_handler,call_post_config_update,7,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,398}]},
                 {emqx_config_handler,do_post_config_update,8,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,336}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,281}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,#{<<"authentication">> => <<"none">>,
                           <<"bootstrap_hosts">> =>
                               <<"toxiproxy.emqx.net:9292">>,
                           <<"connect_timeout">> => <<"5s">>,
                           <<"enable">> => true,
                           <<"kafka">> =>
                               #{<<"max_batch_bytes">> => <<"896KB">>,
                                 <<"max_rejoin_attempts">> => 5,
                                 <<"offset_commit_interval_seconds">> => 3,
                                 <<"offset_reset_policy">> => <<"latest">>},
                           <<"key_encoding_mode">> => <<"none">>,
                           <<"metadata_request_timeout">> => <<"5s">>,
                           <<"min_metadata_refresh_interval">> => <<"3s">>,
                           <<"ssl">> =>
                               #{<<"enable">> => false,
                                 <<"server_name_indication">> => <<"auto">>,
                                 <<"verify">> => <<"verify_none">>},
                           <<"topic_mapping">> =>
                               [#{<<"kafka_topic">> =>
                                      <<"t_resource_manager_crash_before_subscriber_started-576460752303419662">>,
                                  <<"mqtt_topic">> =>
                                      <<"mqtt/topic/-576460752303422911">>,
                                  <<"payload_template">> => <<"${.}">>,
                                  <<"qos">> => 0}],
                           <<"value_encoding_mode">> => <<"none">>}},
                 #{override_to => cluster}}
```
2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi e7b35011e6 test(oracle): attempt to fix flaky test
https://github.com/emqx/emqx/actions/runs/5126014459/jobs/9220155908#step:7:754
```
=CRITICAL REPORT==== 30-May-2023::20:50:26.267103 ===
Run stage failed: error:{assertMatch,
                         [{module,emqx_bridge_oracle_SUITE},
                          {line,394},
                          {expression,
                           "emqx_resource : simple_sync_query ( ResourceId , { query , \"SELECT COUNT(*) FROM mqtt_test\" } )"},
                          {pattern,
                           "{ ok , [ { result_set , _ , _ , [ [ { 3 } ] ] } ] }"},
                          {value,
                           {ok,[{result_set,[<<"COUNT(*)">>],[],[[{0}]]}]}}]}
Stacktrace: [{emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-5-',1,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,394}]},
             {emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-6-',5,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,390}]},
             {emqx_bridge_oracle_SUITE,t_batch_sync_query,1,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,364}]}]
```
2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi 0d539e91d1 test(pulsar_producer): attempt to stabilize flaky test
https://github.com/emqx/emqx/actions/runs/5125166433/jobs/9218613872?pr=10886#step:7:679
```
  =CRITICAL REPORT==== 30-May-2023::19:38:58.003170 ===
  Run stage failed: error:{badmatch,
                              {timeout,
                                  [#{msg => pulsar_producer_bridge_started,
                                     '~meta' =>
                                         #{gl => <97891.472.0>,
                                           location =>
                                               #Fun<emqx_bridge_pulsar_impl_producer.11.109752493>,
                                           node => 'autocluster_node1@127.0.0.1',
                                           pid => <97891.787.0>,
                                           time => -576460611692219}}]}}
  Stacktrace: [{emqx_bridge_pulsar_impl_producer_SUITE,'-t_cluster/1-fun-10-',
                   6,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1073}]},
               {emqx_bridge_pulsar_impl_producer_SUITE,t_cluster,1,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1064}]}]
```
2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi 1e25ebb64c test(gcp_pubsub): attempt to fix flakiness
https://github.com/emqx/emqx/actions/runs/5125118728/jobs/9218520994?pr=10887#step:8:309
```
  =CRITICAL REPORT==== 30-May-2023::19:19:34.887082 ===
  "check stage" failed: error
  {assertMatch,[{module,emqx_bridge_gcp_pubsub_SUITE},
                {line,1066},
                {expression,"? of_kind ( gcp_pubsub_request_failed , Trace )"},
                {pattern,"[ # { reason := Error , connector := ResourceId } | _ ]"},
                {value,[#{connector =>
                              <<"bridge:gcp_pubsub:emqx_bridge_gcp_pubsub_SUITE0005FCEE15534E9CD4CD02004CF10000">>,
                          msg => gcp_pubsub_request_failed,query_mode => async,
                          reason => {closed,"The connection was lost."},
                          recoverable_error => true,
                          '~meta' =>
                              #{gl => <0.17903.2>,
                                location =>
                                    #Fun<emqx_bridge_gcp_pubsub_connector.19.19548918>,
                                node => 'test@127.0.0.1',pid => <0.19724.2>,
                                time => -576460610660164}}]}]}
  Stacktrace: [{emqx_bridge_gcp_pubsub_SUITE,
                   '-do_econnrefused_or_timeout_test/2-fun-2-',3,
                   [{file,
                        "/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
                    {line,1066}]},
               {emqx_bridge_gcp_pubsub_SUITE,do_econnrefused_or_timeout_test,2,
                   [{file,
                        "/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
                    {line,1022}]}]
```
2023-05-31 10:19:55 -03:00
lafirest 502b24bbe2
Merge pull request #10877 from lafirest/fix/rocketmq_sensitive_data
fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data
2023-05-31 18:54:44 +08:00
Zhongwen Deng 5729ea0426 feat: api_key conf support hot conf 2023-05-31 18:23:58 +08:00
zhongwencool 26c2ab0bef
Merge pull request #10894 from zhongwencool/crl-cache-conf-hot-update
feat: update crl_cache conf at runtime
2023-05-31 17:59:09 +08:00
firest ce32526c96 chore: bump version && update changes 2023-05-31 17:31:56 +08:00
Stefan Strigler 6c0fb0e2ea test: run tests against iotdb013 container 2023-05-31 11:30:13 +02:00
William Yang 2250ed7052
Merge pull request #10823 from qzhuyan/dev/william/meck-olp
test: use meck in olp tests
2023-05-31 11:01:56 +02:00
firest 6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Zhongwen Deng 38e57f511c feat: update crl_cache conf at runtime 2023-05-31 16:18:03 +08:00
JianBo He 837dfeb46f feat(exproto): add raw_publish function 2023-05-31 13:44:51 +08:00
JianBo He 5256d8092c fix: keep the real closed_reason 2023-05-31 13:44:51 +08:00
JianBo He 2e4ec4888d fix(exproto): ensure the on_socket_closed event delivering correctly 2023-05-31 13:44:51 +08:00
JianBo He 9dff026f50 test: fix failed test cases 2023-05-31 13:44:51 +08:00
JianBo He b6bc3cd921 refactor(exproto): support unary handler 2023-05-31 13:44:51 +08:00
JianBo He afcd9fcb5e chore: update bpapi.versions 2023-05-31 11:29:55 +08:00
lafirest 5968c71d0c
Merge pull request #10782 from lafirest/feat/retainer_flow_control
feat(retainer): add a field `deliver_rate` to limit the maximum delivery rate
2023-05-31 11:01:10 +08:00
zhongwencool 6202cefd5a
Merge pull request #10870 from zhongwencool/stop-dashboard-listener-waiting
fix: flaky test when stopping dashboard listener
2023-05-31 10:54:12 +08:00
zhongwencool ec580f0454
Merge pull request #10866 from zhongwencool/add-delayed-test
test: add test for delayed update
2023-05-31 10:53:55 +08:00
firest a725ff0440 chore: bump version && update changes 2023-05-31 10:18:00 +08:00
firest a258ef32e5 chore: update retainer app version && changes 2023-05-31 09:44:25 +08:00
firest 03c7b84e89 feat(retainer): add a field `deliver_rate` to limit the maximum delivery rate 2023-05-31 09:44:25 +08:00
firest 6f19cc394e fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data 2023-05-31 09:44:06 +08:00
firest 79d08e4d26 fix(limiter): temporarily hide the node-level settings 2023-05-31 09:43:14 +08:00
firest 55809f801c test: fix test case error 2023-05-31 09:43:14 +08:00
firest be2dfa51df fix(limiter): improve the default configuration 2023-05-31 09:43:14 +08:00
Zhongwen Deng c0b15c402b test: add test for delayed update 2023-05-31 08:17:29 +08:00
Zhongwen Deng 12ef951b8e test: add test for emqx_conf:update([log], Conf) 2023-05-31 07:54:55 +08:00
Zhongwen Deng 5690469896 fix: flaky test when stopping dashboard listener 2023-05-31 07:53:10 +08:00
Thales Macedo Garitezi a7f4f81c38
Merge pull request #10887 from thalesmg/fix-async-worker-down-buffer-worker-20230530-v50
fix: block buffer workers so they may retry requests
2023-05-30 17:39:18 -03:00
Thales Macedo Garitezi 299a67656e
Merge pull request #10886 from thalesmg/test-flakiness-20230530-d
test(quic_multistreams): attempt to fix flaky tests
2023-05-30 17:34:23 -03:00
Andrew Mayorov a2688325e5
Merge pull request #10754 from fix/EMQX-10056/mqtt
feat(mqttconn): employ ecpool instead of single worker
2023-05-30 23:28:10 +03:00
Andrew Mayorov 26819a647c
fix(mqttbridge): clarify schema descriptions + log messages
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-05-30 22:21:22 +03:00
Thales Macedo Garitezi 4f1c55f3fb
Merge pull request #10885 from thalesmg/test-flakiness-20230530-c
test(olp): fix inter-suite flakiness
2023-05-30 16:09:17 -03:00
Thales Macedo Garitezi 596f05f981 test(channel): attempt to reduce test flakiness
https://github.com/emqx/emqx/actions/runs/5124277038/jobs/9217362541?pr=10886#step:5:1582
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```
2023-05-30 15:48:48 -03:00
Thales Macedo Garitezi c43e5b8406
Merge pull request #10883 from thalesmg/test-flakiness-20230530-b
test(exclusive_sub): fix test flakiness
2023-05-30 15:46:14 -03:00
Thales Macedo Garitezi 8c565abc84 test(cassandra): fix flaky test 2023-05-30 15:42:53 -03:00
Thales Macedo Garitezi 6be8ff378e fix(buffer_worker): make buffer worker enter `blocked` state when async worker dies
Fixes https://emqx.atlassian.net/browse/EMQX-10074

Otherwise, requests from those async workers, now retriable, might not
be retried until the buffer worker blocks for other reasons, which
might take a long time.
2023-05-30 15:34:22 -03:00
Thales Macedo Garitezi 0f802775c3
Merge pull request #10884 from thalesmg/fix-rule-metrics-empty-map-v50
fix(rule_engine_api): don't crash when formatting empty metrics
2023-05-30 15:09:04 -03:00
William Yang 0d6465b016 chore: bump app vsn 2023-05-30 19:41:05 +02:00
Thales Macedo Garitezi 98a99d47cb test(shared_sub): attempt to reduce test flakiness
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9215439237#step:8:565
```
%%% emqx_shared_sub_SUITE ==> t_local_fallback: FAILED
%%% emqx_shared_sub_SUITE ==> {{badmatch,<<"not yet?">>},
 [{emqx_shared_sub_SUITE,t_local_fallback,1,
                         [{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_shared_sub_SUITE.erl"},
                          {line,669}]},
```
2023-05-30 14:03:14 -03:00
Thales Macedo Garitezi fa9afcd7e4 test(quic_multistreams): attempt to fix flaky tests
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9214950882#step:8:1757
```
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
test_server:ts_tc failed on line 1782
Reason: {test_case_failed,Client <0.6075.5> did not die: stacktrace: {current_stacktrace,
                                            [{gen_statem,loop_receive,3,
                                              [{file,"gen_statem.erl"},
                                               {line,1281}]},
                                             {proc_lib,init_p_do_apply,3,
                                              [{file,"proc_lib.erl"},
                                               {line,240}]}]}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Testing lib.emqx.emqx_quic_multistreams_SUITE: *** FAILED test case 335 of 364 ***
%%% emqx_quic_multistreams_SUITE ==> shutdown.abort_recv_shutdown.ctrl_stream_shutdown.t_multi_streams_remote_shutdown: FAILED
%%% emqx_quic_multistreams_SUITE ==> {test_case_failed,"Client <0.6075.5> did not die: stacktrace: {current_stacktrace,\n                                            [{gen_statem,loop_receive,3,\n                                              [{file,\"gen_statem.erl\"},\n                                               {line,1281}]},\n                                             {proc_lib,init_p_do_apply,3,\n                                              [{file,\"proc_lib.erl\"},\n                                               {line,240}]}]}"}
```

```
  %%% emqx_quic_multistreams_SUITE ==> mstream.profiles.profile_max_throughput.pub_qos2.sub_qos2.qos.t_multi_streams_unsub: FAILED
  %%% emqx_quic_multistreams_SUITE ==> {{case_clause,
       {ok,#{packet_id => 8,properties => #{},reason_code => 0,
             reason_code_name => success,
             via =>
                 {quic,#Ref<0.686875463.2755788802.201649>,
                     #Ref<0.686875463.2755788804.193527>}}}},
   [{emqx_quic_multistreams_SUITE,t_multi_streams_unsub,1,
        [{file,
             "/home/thales/dev/emqx/emqx/apps/emqx/test/emqx_quic_multistreams_SUITE.erl"},
         {line,1094}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
    {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1292}]},
    {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}
```
2023-05-30 13:56:05 -03:00
Thales Macedo Garitezi 37b4a44e23 test(olp): fix inter-suite flakiness
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9212645120?pr=10883#step:8:2538
```
   emqx_olp_SUITE:t_overload_cooldown_conn failed on line 96
  Reason: {assertMatch,[{module,...},{...}|...]}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_olp_SUITE: *** FAILED test case 3 of 4 ***
  %%% emqx_olp_SUITE ==> t_overload_cooldown_conn: FAILED
  %%% emqx_olp_SUITE ==>
  Failure/Error: ?assertMatch({ ok , _Pid }, emqtt : connect ( C ))
    expected: = { ok , _Pid }
         got: {error,econnrefused}
        line: 96
  %%% emqx_olp_SUITE ==> t_overloaded_conn: OK
  Testing lib.emqx.emqx_olp_SUITE: TEST COMPLETE, 3 ok, 1 failed of 4 test cases
```

```sh
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_olp_SUITE.erl
```
2023-05-30 13:39:52 -03:00
William Yang f2e54c9a6e test: use meck in olp tests
avoid flaky test results by using mecks.
2023-05-30 17:57:09 +02:00
Andrew Mayorov 95cc9b9b72
fix(mqttbridge): ensure elixir release assembles successfully 2023-05-30 17:56:35 +03:00
Andrew Mayorov 1c2719236c
chore(mqttbridge): add README 2023-05-30 17:56:35 +03:00
Andrew Mayorov 7e7b50c5ba
refactor(mqttbridge): move into separate application 2023-05-30 17:56:34 +03:00
Thales Macedo Garitezi 57aacb471c fix(rule_engine_api): don't crash when formatting empty metrics
Fixes https://emqx.atlassian.net/browse/EMQX-10073
Fixes https://github.com/emqx/emqx/issues/10714#issuecomment-1567987664

Similar issue to https://github.com/emqx/emqx/pull/10743, but on the
rule engine API.
2023-05-30 11:54:27 -03:00
Thales Macedo Garitezi a28580248a test(exclusive_sub): fix test flakiness
https://github.com/emqx/emqx/actions/runs/5115608275/jobs/9208594481?pr=10864#step:8:735

```
  Testing lib.emqx.emqx_access_control_SUITE: TEST COMPLETE, 4 ok, 0 failed of 4 test cases

  Testing lib.emqx.emqx_exclusive_sub_SUITE: Starting test, 5 test cases

  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  Error detected: {'EXIT',{shutdown,econnrefused}}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 1 of 5 ***
  %%% emqx_exclusive_sub_SUITE ==> t_allow_normal_sub: FAILED
  %%% emqx_exclusive_sub_SUITE ==> {'EXIT',{shutdown,econnrefused}}

  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  emqx_exclusive_sub_SUITE:t_clean_session failed on line 128
  Reason: {badmatch,{error,econnrefused}}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 2 of 5 ***
  %%% emqx_exclusive_sub_SUITE ==> t_clean_session: FAILED
```

To reproduce:

```
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_exclusive_sub_SUITE.erl
```
2023-05-30 11:23:17 -03:00
Andrew Mayorov ebd612b194
chore: bump applications versions
* emqx_connector 0.1.25
* emqx_rule_engine 5.0.19
* emqx_ee_bridge 0.1.15
2023-05-30 17:21:45 +03:00
Andrew Mayorov eed9358abd
chore: bump `ecpool` to 0.5.4
With fixed typings and empty pool handling.
2023-05-30 17:21:45 +03:00
Andrew Mayorov a5fc26736d
refactor(mqttconn): split ingress/egress into 2 separate pools
Each with a more refined set of responsibilities, at the cost of slight
code duplication. Also provide two different config fields for each pool
size.
2023-05-30 17:21:44 +03:00
Andrew Mayorov c7528e9b35
feat(mqttconn): add `pool_size` config parameter
That currently tunes the number of MQTT clients employed both for
subscriptions (if shared subscription is used) and for publishing to
a remote broker.
2023-05-30 17:21:44 +03:00
Andrew Mayorov 6e97dffdb8
feat(mqttconn): deprecate `mode` config parameter
It adds no value: the only mode was `cluster_shareload` and we just as
well can decide to "share" the load across cluster just by looking if
the remote topic is shared subcription filter or not.
2023-05-30 17:21:44 +03:00
Thales Macedo Garitezi cb7929a2fc test(broker): attempt to reduce flakiness 2023-05-30 11:12:18 -03:00
Thales Macedo Garitezi 26992e9d98 test(banned): attempt to improve flaky test 2023-05-30 11:08:31 -03:00
Thales Macedo Garitezi 04bd39861d chore: bump app vsns 2023-05-30 11:08:20 -03:00
Stefan Strigler 767f7c57e7 test: check device_id is used from topic or config 2023-05-30 15:55:21 +02:00
Stefan Strigler 938d62a666 test: handle bad payload 2023-05-30 15:55:21 +02:00
Stefan Strigler 860d7b169a test: check data at iotdb instance 2023-05-30 15:55:21 +02:00
Stefan Strigler a3021c58f1 fix(emqx_bridge_iotdb): allow integer timestamp 2023-05-30 15:55:21 +02:00
Stefan Strigler bd92116cee test: fix data in calls to look like real world 2023-05-30 15:55:21 +02:00
Stefan Strigler 1381b54a8d fix(emqx_bridge_iotdb): allow non-binary values 2023-05-30 15:55:21 +02:00
Stefan Strigler 64d582770d test: add tracepoints 2023-05-30 15:55:21 +02:00
Stefan Strigler 3d3f2a223c test: test more value conversions 2023-05-30 15:55:21 +02:00
Stefan Strigler 6f54220a51 feat(emqx_bridge_iotdb): handle bad message format gracefully 2023-05-30 15:55:20 +02:00
JianBo He a132df5568 fix(coap): to better handle coap requests in connection mode
Fixes for connection deletion and message publishing requests not taking effect
issues once the connection has been created in a different UDP port
first.
2023-05-30 21:48:57 +08:00
JianBo He 48c53d8b32 chore: add emqx_management_proto_v4 2023-05-30 21:46:33 +08:00
Thales Macedo Garitezi 6b5b9fc8b8
Merge pull request #10864 from thalesmg/fix-flaky-channel-test-20230529
test(channel): fix flaky test
2023-05-30 10:18:47 -03:00
Thales Macedo Garitezi 5f6ff9144b
Merge pull request #10862 from thalesmg/fix-flaky-cassandra-test-v50
test(cassandra): attempt to fix flaky test
2023-05-30 10:02:14 -03:00
Thales Macedo Garitezi 9c3f838e14
Merge pull request #10841 from thalesmg/kafka-validate-key-v50
feat({kafka,pulsar}_producer): add validation for empty message key when strategy = key_dispatch
2023-05-30 09:37:15 -03:00
Kjell Winblad a4fcc0dcb3 fix: password log leak in RabbitMQ bridge
This fixes a vulnerability in the RabbitMQ bridge, which could
potentially expose passwords to log files. This was accomplished by
initializing the encryption library specifically designed for RabbitMQ's
passwords. Consequently, passwords are no longer stored in unencrypted
format. As a result, they will no longer be visible as plain text in log
messages, thereby enhancing the system's security.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9976
2023-05-30 14:36:02 +02:00
JianBo He fc806c4acd test: fix failed tests 2023-05-30 20:15:49 +08:00
Thales Macedo Garitezi 75fcac9711
Merge pull request #10826 from thalesmg/test-partial-batch-expired-inflight-v50
test(buffer_worker): add assertion for inflight count after batch expiration
2023-05-30 09:05:59 -03:00
Thales Macedo Garitezi 5bcc28b8df
Merge pull request #10861 from thalesmg/fix-flaky-telemetry-test-v50
test(telemetry): fix flaky test
2023-05-30 09:05:28 -03:00
JianBo He 1e7872c319 feat: support kickout clients in batch 2023-05-30 19:58:40 +08:00
Zaiming (Stone) Shi 0f080cda66 chore: bump emqx_rule_engine app vsn 2023-05-30 13:50:36 +02:00
Andrew Mayorov 4e6269bedb
feat(mqttconn): subscribe each worker if shared subcription
Also rename `subscriptions` -> `ingress` and `forwards` -> `egress` for
consistency with the config schema.
2023-05-30 14:50:05 +03:00
Andrew Mayorov 81e78516aa
feat(mqttconn): employ ecpool instead of a single worker 2023-05-30 14:50:05 +03:00
Andrew Mayorov 6967f621d8
fix(mqttconn): unify error interpretation in sync/async modes
Also move this logic to the mqtt connector itself, in order to avoid
dealing with extra callback layer.
2023-05-30 14:50:05 +03:00
Andrew Mayorov 67d703f8c5
refactor(mqttconn): simplify mqtt connector
Inline `emqx_connector_mqtt_msg` module code into
`emqx_connector_mqtt_worker` module, since it's not really used
anywhere else and does not provide any reusable abstractions.
2023-05-30 14:50:05 +03:00
Andrew Mayorov bd956d00b6
feat(mqttconn): stop using gproc in hot path
Also drop fiddling with `mountpoint` since this option seems not to be
used anywhere.
2023-05-30 14:50:05 +03:00
Andrew Mayorov 4da0d83faf
chore(mqttconn): remove dead code 2023-05-30 14:50:04 +03:00
Zaiming (Stone) Shi aa096f496f
Merge pull request #10876 from zmstone/merge-release-51-back-to-master
Merge release 51 back to master
2023-05-30 13:49:21 +02:00
Kjell Winblad 5cfe04956d
Merge pull request #10858 from kjellwinblad/kjell/rule_engine_funcs/add_timezone_to_second/EMQX-10058
feat: add timezone_to_second/1 function to rule engine
2023-05-30 12:29:40 +02:00
Zaiming (Stone) Shi d2ce7c22fd chore: bump app vsn, core app 'emqx' is now at 5.1 2023-05-30 11:32:18 +02:00
Zaiming (Stone) Shi 1e8ed89db8 Merge remote-tracking branch 'origin/master' into release-51 2023-05-30 11:29:44 +02:00
Zaiming (Stone) Shi 91cdc69976
Merge pull request #10867 from zmstone/0530-merge-release-50-to-master
0530 merge release 50 to master
2023-05-30 09:54:57 +02:00
Zaiming (Stone) Shi 367de2a2eb docs: update bpapi readme 2023-05-30 08:29:43 +02:00
Zaiming (Stone) Shi 941939f7d7 build: finalize 5.0.bpapi 2023-05-30 08:28:45 +02:00
Zaiming (Stone) Shi 3b8cba6ba0 chore: bump rlease versions 2023-05-30 08:28:45 +02:00
Zaiming (Stone) Shi 9529919046 chore: bump app versions 2023-05-30 08:08:29 +02:00
JimMoen 8569fac6eb
fix: make static_check happy 2023-05-30 14:00:20 +08:00
Zaiming (Stone) Shi 747eb6fcff Merge remote-tracking branches 'origin/release-50' and 'origin/master' into release-51 2023-05-30 07:54:34 +02:00
Zaiming (Stone) Shi 286eefb563 Merge remote-tracking branch 'origin/release-50' 2023-05-30 07:52:54 +02:00
JimMoen 909d348570
chore: fix spec typo 2023-05-30 11:31:37 +08:00
JimMoen bb363e9b9e
chore: bump dependency vsn
* bump `snappyer` ~> 1.2.9
  - as dependency of `kafka_protocol` ~> 4.1.3
  - as dependency of `brod`

* bump `rocksdb` ~> 1.7.2-emqx-11
2023-05-30 11:11:30 +08:00
Thales Macedo Garitezi 1aff46b7c9 test(channel): fix flaky test 2023-05-29 19:04:48 -03:00
Thales Macedo Garitezi bae00c909f test(cassandra): attempt to fix flaky test
```
=ERROR REPORT==== 29-May-2023::20:35:20.457796 ===
    id: <<"bridge:cassandra:emqx_bridge_cassandra_SUITE">>
    msg: unrecoverable_error
    reason: {8704,<<"Expected 8 or 0 byte long for date (4)">>}
=CRITICAL REPORT==== 29-May-2023::20:35:20.458850 ===
"check stage" failed: error
{assertMatch,[{module,emqx_bridge_cassandra_SUITE},
              {line,619},
              {expression,"Trace"},
              {pattern,"[ # { result := { ok , _Pid } } ]"},
              {value,[]}]}
Stacktrace: [{emqx_bridge_cassandra_SUITE,'-t_missing_data/1-fun-3-',1,
                 [{file,
                      "/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
                  {line,619}]},
             {emqx_bridge_cassandra_SUITE,t_missing_data,1,
                 [{file,
                      "/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
                  {line,607}]}]

=CRITICAL REPORT==== 29-May-2023::20:35:20.471872 ===
Check stage failed. Trace dump: "/emqx/_build/test/logs/ct_run.test@127.0.0.1.2023-05-29_20.34.32/snabbkaffe/1685392520460040358.log"

%%% emqx_bridge_cassandra_SUITE ==> tcp.sync.with_batch.t_missing_data: FAILED
%%% emqx_bridge_cassandra_SUITE ==> {{panic,#{msg => "Unexpected result",result => {error,check_stage_failed}}},
 [{emqx_bridge_cassandra_SUITE,t_missing_data,1,
                               [{file,"/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
                                {line,624}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1291}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1223}]}]}
```
2023-05-29 18:19:26 -03:00
Thales Macedo Garitezi fa508a95c1 test(telemetry): fix flaky test
```
%%% emqx_telemetry_SUITE ==>
Failure/Error: ?assertEqual(0, get_value ( num_clients , TelemetryData1 ))
  expected: 0
       got: 1
      line: 454
```
2023-05-29 18:09:35 -03:00
Thales Macedo Garitezi 67e182e0c9
Merge pull request #10813 from thalesmg/refactor-kafka-on-stop-v50
feat(kafka): ensure allocated resources are removed on failures
2023-05-29 16:49:29 -03:00
Kjell Winblad 354603da01 test: helper function was interpreted as a test function 2023-05-29 17:20:37 +02:00
Thales Macedo Garitezi 3edbad9f56 feat(pulsar_producer): add validation for empty message key when strategy = key_dispatch 2023-05-29 10:04:19 -03:00
Kjell Winblad 995025b572 feat: add timezone_to_second/1 function to rule engine
This commit adds the functions timezone_to_offset_seconds and its alias
timezone_to_second to the rule engine. As the names suggests, these
functions convert a timzone offset string such as "+02:00", "Z", "local"
to an integer representing how many seconds that the given timezone
differs from UTC. timezone_to_offset_seconds is the one of the two
functions that should be advertised while timezone_to_second in kept for
backwards compatibility with 4.X.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10058
2023-05-29 14:30:43 +02:00
Zaiming (Stone) Shi 08c1d5e5ae test: stop testing i18n.conf 2023-05-29 14:22:23 +02:00
Zaiming (Stone) Shi d014b80834 build: download zh translation from emqx-i18n.git 2023-05-29 14:22:23 +02:00
Thales Macedo Garitezi 79f6861499 chore: prepare to tag v5.0.26 2023-05-29 09:17:59 -03:00
Thales Macedo Garitezi 639feefab6 Merge remote-tracking branch 'origin/master' into tag-v5026 2023-05-29 09:15:48 -03:00
JimMoen 9526d080af
fix: override `emqx_license` is unnecessary
* continue #10820
2023-05-29 19:10:31 +08:00
JimMoen adae1aec96
chore: fix typo for enterprise readme 2023-05-29 19:08:15 +08:00
JimMoen 6551eb21f2
Merge pull request #10851 from JimMoen/fix-bad-api-logging
fix: redact api request to hide auth token
2023-05-29 17:40:44 +08:00
JimMoen e287cf9a09
Merge pull request #10820 from JimMoen/fix-license-checker-update
fix: reboot `emqx_license` app for join cluster
2023-05-29 17:40:29 +08:00
Andrew Mayorov daa7eebc0f
Merge pull request #10843 from keynslug/fix/ft-prop-zerodiv
test(ft): ensure no division by zero in coverage proptest
2023-05-29 10:23:52 +03:00
zhongwencool 2b6d3bb748
Merge pull request #10833 from zhongwencool/telemtry-report
fix: only report enable authn/authz in telemetry
2023-05-29 15:14:36 +08:00
zhongwencool 1e986a7b93
Merge pull request #10847 from zhongwencool/telemetry-conf-update
feat: support emqx_conf:update([telemetry],Conf).
2023-05-29 15:14:16 +08:00
Zaiming (Stone) Shi e6869a2904 chore: bump app vsn for emqx_machine 2023-05-29 09:04:17 +02:00
Zaiming (Stone) Shi 8e6dc29637 chore: bump release version to v5.0.26-alpha.1 2023-05-29 08:57:28 +02:00
Zaiming (Stone) Shi 25319c3151
Merge pull request #10839 from zmstone/0526-merge-release-50-to-master
0526 merge release 50 to master
2023-05-29 08:56:20 +02:00
Zhongwen Deng 8c814bc861 fix: only report enable authn/authz in telemetry 2023-05-29 14:17:46 +08:00
Zhongwen Deng 4dee1a2429 test: add more test for emqx_telemetry 2023-05-29 14:08:59 +08:00
JimMoen 6f90369429
fix: make dialyzer happy 2023-05-29 13:48:00 +08:00
JimMoen cb05406a00
fix: reboot `emqx_license` app for join cluster 2023-05-29 13:47:46 +08:00
JimMoen dcfe985ee9
fix: redact api request to hide auth token 2023-05-29 11:02:02 +08:00
某文 8ccee19aa5 feat: support emqx_conf:update([telemetry],Conf). 2023-05-28 22:54:35 +08:00
Zaiming (Stone) Shi c2450d230d
Merge pull request #10605 from zmstone/0503-no-telemetry-app-for-ee
0503 refactoring: no telemetry app for ee
2023-05-27 10:57:15 +02:00
Thales Macedo Garitezi d27f593309 test: fix flaky authn test 2023-05-26 16:44:00 -03:00
Andrew Mayorov 98ce7e8bbd
test(ft): ensure no division by zero in coverage proptest 2023-05-26 21:53:01 +03:00
Thales Macedo Garitezi 9b7e473cf6 feat(kafka_producer): add validation for empty message key when strategy = key_dispatch
Fixes https://emqx.atlassian.net/browse/EMQX-9979
2023-05-26 12:02:58 -03:00
Zaiming (Stone) Shi fe81e9521a
Merge pull request #10600 from zmstone/0504-delete-statsd-all-together
0504 delete statsd all together
2023-05-26 16:29:59 +02:00
Zaiming (Stone) Shi 36e268c933 chore: bump app versions 2023-05-26 16:05:37 +02:00
Zaiming (Stone) Shi cc5b4d3748 Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test 2023-05-26 15:58:16 +02:00
Zaiming (Stone) Shi 3595a86724 chore(emqx_telemetry): add registered process names to app info 2023-05-26 15:28:06 +02:00
Zaiming (Stone) Shi d167d6dd41 chore(emqx_telemetry): update supervisor's child restart strategy 2023-05-26 15:28:06 +02:00
Zaiming (Stone) Shi 256e92f4e9 refactor: call emqx_telemetry_config:on_server_stop 2023-05-26 15:28:06 +02:00
Zaiming (Stone) Shi 6e2cde8224 refactor: delete emqx_statsd all together 2023-05-26 15:17:42 +02:00
Andrew Mayorov 45b128946a
Merge pull request #10825 from fix/EMQX-9991/bugs
fix(ft): avoid leaking secrets
2023-05-26 16:08:14 +03:00
Zaiming (Stone) Shi 7eb22e27c4 chore: bump release version to e5.0.4 2023-05-26 14:34:01 +02:00
Zhongwen Deng 9b6da3ff10 chore: bump hocon to 0.39.7 2023-05-26 19:00:58 +08:00
Zhongwen Deng df4a0afdf5 fix: zones api default value 2023-05-26 18:59:38 +08:00
Zaiming (Stone) Shi 2d986140df chore: bump version to e5.0.4-rc.2 2023-05-26 12:01:10 +02:00
Andrew Mayorov 48858dee33
fix(s3): wrap S3 secrets during config load 2023-05-26 12:59:21 +03:00
Andrew Mayorov 3d428a2e0e
fix(ft-s3): rely on asynchronous upload abort
So that upload aborts would not block assemblers. This should not
affect the expected behavior since S3 API usually allows having
concurrent uploads, plus we couldn't really guarantee successful aborts
anyway.
2023-05-26 12:59:20 +03:00
Andrew Mayorov cc3275b389
fix(ft): use conservative timeouts for S3 uploader calls 2023-05-26 12:59:20 +03:00
Andrew Mayorov d36c2c2928
fix(ft-gc): ensure GC of already complete transfers 2023-05-26 12:59:20 +03:00
Andrew Mayorov 75cf562c90
fix(ft): tune some logging levels 2023-05-26 12:59:20 +03:00
Andrew Mayorov 3278158c66
fix(ft): disable `ipv6_probe` by default for S3 clients
It's often demonstrates degraded performance on common setups, e.g.
containers with virtualized network interfaces.
2023-05-26 12:59:20 +03:00
Zaiming (Stone) Shi 17f6c58f7b fix(telemetry): no default value for telemetry.enable in schema
the default value is computed from version string
2023-05-26 11:58:58 +02:00
Zaiming (Stone) Shi dfc11e57d3 chore: delete telemetry.enable from enterprise.conf
telemetry is now not included in enterprise edition
2023-05-26 11:58:58 +02:00
Zaiming (Stone) Shi ee122155f1 docs: fix spellcheck for telemetry app 2023-05-26 08:44:19 +02:00
Zaiming (Stone) Shi 772a4575ca
Merge pull request #10821 from thalesmg/fix-webhook-bridge-req-timeout-r50
fix(webhook): keep `resource_opts.request_timeout` for webhook bridge (r5.0)
2023-05-26 08:39:00 +02:00
JimMoen a3b77563ba
fix: make static check happy 2023-05-26 10:34:15 +08:00
JimMoen 1ba65539c6
chore: refine traces command usage 2023-05-26 09:48:56 +08:00
JimMoen 0788f2f197
fix: `emqx_ctl traces ...` command args
* use second time as trace `start_at` and `end_at`
* default trace time duration 30min to same as api created
* prevent format line broken
2023-05-26 09:48:56 +08:00
Zaiming (Stone) Shi 407cab220b
Merge pull request #10760 from paulozulato/fix-failure-joining
fix: avoid error 500 when node is re-joining cluster
2023-05-26 03:08:17 +02:00
Thales Macedo Garitezi 32e6213ce3 fix(resource_manager_sup): use `one_for_one` instead of `simple_one_for_one`
Using `simple_one_for_one` has a potential race condition issue where
we read the PID of the resource manager before trying to remove a
resource, and then that PID changes because it was either dead at
first, or it crashed and changed, and later we use this stale PID to
try to remove it from the supervisor.  Under such circumstances, the
restarting child might linger in the supervisor, leaking resources.

By using the resource ID itself as a child ID (and using `one_for_one`
restart strategy), we ensure the child is truly removed.
2023-05-25 18:07:43 -03:00
Thales Macedo Garitezi e43517188f fix(bridge): remove bridge if exception happens during creation 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi cb34bc5c46 test(kafka_consumer): attempt to stabilize cluster tests
Example failure:
https://github.com/emqx/emqx/actions/runs/5070096710/jobs/9105822319#step:7:515

The attempt here is to setup the spy as early as possible, before the
bridge starts, so we avoid missing rebalancing events.
2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi 0ca3f51503 fix(kafka): improve shutdown and health check logs during shutdown 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi 5df7314255 feat(kafka): ensure allocated resources are removed on failures 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi 42b37690c7 refactor(pulsar): use macros for allocatable resources 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi db60dcbada test(buffer_worker): add assertion for inflight count after batch expiration
Fixes https://emqx.atlassian.net/browse/EMQX-9829
2023-05-25 16:11:37 -03:00
Paulo Zulato ea86f4442b fix: avoid error 500 when node is re-joining cluster
Fixes https://emqx.atlassian.net/browse/EMQX-9899
2023-05-25 13:32:28 -03:00
Thales Macedo Garitezi 194faefc8a fix(webhook): keep `resource_opts.request_timeout` for webhook bridge (r5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-10001

Recently, we unified request_timeout in a single field located at the
webhook connector schema.  However, the correct fix would be to use
the resource_opts.request_timeout one, as that’s the only one that
allows infinity timeout.
2023-05-25 13:19:27 -03:00
Zaiming (Stone) Shi e8a4df96bc Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-25 16:43:20 +02:00
Thales Macedo Garitezi 18d57ba3eb
Merge pull request #10812 from thalesmg/test-flakiness-20230524
test: attempts to reduce flakiness (pgsql, cassandra)
2023-05-25 09:29:13 -03:00
JianBo He 933357483f
Merge pull request #10817 from HJianBo/fix-restart-timer-interval
fix: fix auto_restart_interval checker
2023-05-25 20:23:53 +08:00
JianBo He 6b5ff24952 test: assert the block_until result 2023-05-25 18:12:38 +08:00
JianBo He fdec86e29c
Merge pull request #10802 from Kinplemelon/kinple/set-base-url-required
fix: set the base_url in IoTDB bridge as a required field
2023-05-25 18:12:05 +08:00
JianBo He de7f1c8aec test: add tests for auto_restart_interval 2023-05-25 17:15:19 +08:00
Zaiming (Stone) Shi ba41c23b7c
Merge pull request #10794 from paulozulato/fix-oracle-database-field
Fix oracle database field
2023-05-25 08:14:13 +02:00
JianBo He 71b636e321 fix: fix auto_restart_interval checker 2023-05-25 12:04:23 +08:00
Paulo Zulato 1f36726cab
Merge pull request #10780 from paulozulato/fix-large-health-check-interval
Add user-friendly message when interval is out of range
2023-05-24 18:41:42 -03:00
Thales Macedo Garitezi 658160f09a test(cassandra): attempt to reduce flakiness 2023-05-24 16:57:51 -03:00
Thales Macedo Garitezi 37061b484a
Merge pull request #10778 from thalesmg/refactor-pulsar-on-stop-v50
feat(pulsar): ensure allocated resources are removed on failures (v5.0)
2023-05-24 16:00:50 -03:00
Thales Macedo Garitezi cc8cebe950
Merge pull request #10811 from thalesmg/fix-flaky-kconsumer-balance-test-r50
test(kafka_consumer): attempt to stabilize cluster tests
2023-05-24 15:56:28 -03:00
Thales Macedo Garitezi 3cb12d7603
Merge pull request #10810 from thalesmg/merge-r50-into-v50-20230524
Merge `release-50` into `master`
2023-05-24 15:55:09 -03:00
Paulo Zulato 122ebcac24 fix: add user-friendly message when interval is out of range 2023-05-24 15:46:00 -03:00
Thales Macedo Garitezi 7f88521836 test(pgsql): reduce flakiness
Depending on timing, `t_write_timeout` was getting stuck while
checking the resource health, and the previous request timeout options
were making a response to never be sent if that process took too long.
2023-05-24 15:41:25 -03:00
Paulo Zulato 8d3d3e523b fix(oracle): replace 'database' field by 'service name'
Fixes https://emqx.atlassian.net/browse/EMQX-9980
2023-05-24 15:40:56 -03:00
Paulo Zulato 50ff6c4aa4
Merge pull request #10741 from paulozulato/fix-oracle-passwd-leaking
fix(oracle): hide password on stacktrace
2023-05-24 15:31:00 -03:00
Kjell Winblad 0e873731a1
Merge pull request #10804 from kjellwinblad/kjell/fix/iotdb_rest_api_v1/EMQX-9920
fix: REST API version 2 not available in IoTDB 0.13 and 1.0
2023-05-24 18:34:45 +02:00
Thales Macedo Garitezi 7374e00a02 test(kafka_consumer): attempt to stabilize cluster tests
Example failure:
https://github.com/emqx/emqx/actions/runs/5070096710/jobs/9105822319#step:7:515

The attempt here is to setup the spy as early as possible, before the
bridge starts, so we avoid missing rebalancing events.
2023-05-24 13:13:54 -03:00
ieQu1 2b79196a03
Merge pull request #10809 from ieQu1/ekka-0.15.2
chore(ekka): Bump version to 0.15.2
2023-05-24 18:13:04 +02:00
Thales Macedo Garitezi 324459990f Merge branch 'release-50' into merge-r50-into-v50-20230524 2023-05-24 12:54:15 -03:00
Thales Macedo Garitezi ad93af2853
Merge pull request #10808 from thalesmg/fix-ehttpc-infinity-timeout-r50
fix: handle `infinity` timeout option in `ehttpc` (r5.0)
2023-05-24 12:53:05 -03:00
Thales Macedo Garitezi fd2940cd77 feat(pulsar): ensure allocated resources are removed on failures (v5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9937
2023-05-24 12:29:00 -03:00
zhongwencool 14a6b36899
Merge pull request #10803 from JimMoen/merge-release-50
Merge release-50 into master
2023-05-24 23:08:46 +08:00
zhongwencool 96e4b5914a
Merge pull request #10797 from zhongwencool/bump-minirest
chore: upgrade minirest to 1.3.10 for more clear error msg
2023-05-24 22:41:42 +08:00
Thales Macedo Garitezi a172a6bc33
Merge pull request #10795 from thalesmg/fix-schema-registry-rule-iolist-r50
fix(schema_registry): ensure `schema_encode` output in rule engine is a binary (r5.0)
2023-05-24 11:02:05 -03:00
ieQu1 3d3dacfcf6 chore(ekka): Bump version to 0.15.2 2023-05-24 15:43:49 +02:00
Thales Macedo Garitezi 4565acc600 fix: handle `infinity` timeout option in `ehttpc` (r5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9987
2023-05-24 10:33:54 -03:00
Andrew Mayorov 3fd2887921
Merge pull request #10793 from fix/EMQX-9965/bugs
fix(ft): handle wider class of bad input in APIs
2023-05-24 16:14:11 +03:00
某文 8816da41a6 test: log test failed 2023-05-24 21:06:15 +08:00
Thales Macedo Garitezi 6c414ab991 fix(schema_registry): ensure `schema_encode` output in rule engine is a binary (r5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9981
2023-05-24 09:13:38 -03:00
JimMoen 28015597ee
Merge remote-tracking branch 'emqx/release-50' into merge-release-50 2023-05-24 19:34:12 +08:00
Kjell Winblad 4f21bdb885 fix: REST API version 2 not available in IoTDB 0.13 and 1.0
This commit makes sure that REST API version 1 is used when the
user has selected to use IoTDB 0.13 or 1.0

Fixes:
https://emqx.atlassian.net/browse/EMQX-9920
2023-05-24 13:09:47 +02:00
JimMoen cc2bb87b82
Merge pull request #10801 from JimMoen/fix-topic-lookup-percent-decode
fix: lookup topic without force percent decode
2023-05-24 18:51:16 +08:00
Kinplemelon 7cad1cb93a fix: set the base_url in IoTDB bridge
as a required field
2023-05-24 18:18:32 +08:00
JimMoen 1ac140312d
test: lookup topic with percent encoded topic name 2023-05-24 17:39:25 +08:00
JimMoen e38645aa28
fix: lookup topic without force percent decode
* the minirest handler would do it
2023-05-24 17:28:32 +08:00
Zaiming (Stone) Shi 2fdf4b5dac
Merge pull request #10744 from savonarola/0518-fix-eviction
fix(evacuation): handle expire interval correctly
2023-05-24 09:20:53 +02:00
Zaiming (Stone) Shi 256cb73e43
Merge pull request #10792 from thalesmg/fix-webhook-bridge-health-check-clause-r50
fix(webhook): fix empty ehttpc worker list clause on health check (r5.0)
2023-05-24 08:46:59 +02:00
Zaiming (Stone) Shi 9363ca5846
Merge pull request #10791 from thalesmg/fix-bridge-infinity-timeout-r50
fix(bridge): pass resource option `request_timeout = infinity` along to buffer workers (r5.0)
2023-05-24 08:46:43 +02:00
某文 7da0860d6c chore: upgrade minirest to 1.3.10 for more clear error msg 2023-05-24 10:44:52 +08:00
JianBo He 0d3de36f4d
Merge pull request #10770 from kjellwinblad/kjell/fix/iotdb_need_to_parse_input_bug/EMQX-9854
fix: IoTDB bridge incoming payload needs to be parsed as JSON
2023-05-24 10:17:23 +08:00
Thales Macedo Garitezi 8e6da40a31 test: fix flaky ocsp test setup (v5.0) 2023-05-23 17:52:02 -03:00
Zaiming (Stone) Shi 2de2a979f5
Merge pull request #10779 from zmstone/0522-remove-log-file-paht-validator
chore: allow special chars in log dir
2023-05-23 22:42:12 +02:00
Zaiming (Stone) Shi b8b1779627 Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-23 21:42:07 +02:00
Zaiming (Stone) Shi f450f51e2c
Merge pull request #10789 from kjellwinblad/kjell/bridge/rabbitmq_password_not_required_filed/EMQX-9974
fix: RabbitMQ field not marked as required
2023-05-23 20:59:49 +02:00
Thales Macedo Garitezi 44ed6a19ee fix(bridge): pass resource option `request_timeout = infinity` along to buffer workers (r5.0) 2023-05-23 14:50:43 -03:00
Thales Macedo Garitezi b5f63f4151 test: fix wrong test listeners option 2023-05-23 14:50:43 -03:00
Andrew Mayorov f4047d3946
test(ft): add testcase for nasty tranfer filenames
* Emoji
* Chinese characters
2023-05-23 20:46:47 +03:00
Thales Macedo Garitezi 25437adf02 fix(webhook): fix empty ehttpc worker list clause on health check (r5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9970
2023-05-23 14:06:40 -03:00
Andrew Mayorov 2dbf84479c
fix(ft): handle wider class of jiffy decode errors
With malformed cursors in the File listing API.

Fixes EMQX-9965
2023-05-23 20:00:00 +03:00
Andrew Mayorov cb14a3e08b
fix(ft): handle empty filepath in fs exporter API
Fixes EMQX-9973
2023-05-23 19:58:05 +03:00
Kjell Winblad 63180c87be
style: simplify code for better readability
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-05-23 18:53:14 +02:00
Zaiming (Stone) Shi 4ada938b82
Merge pull request #10757 from thalesmg/perf-pulsar-producer-status-check-r50
feat(pulsar): improve authn error check time and add connect timeout
2023-05-23 16:37:19 +02:00
Kjell Winblad ea2fea77d7 fix: RabbitMQ field not marked as required
This commit makes sure that the RabbitMQ password filed is marked as
required. This ensures that the user provides a password and that the
bridge does not throw a function clause exception if the password filed
is not set.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9974
2023-05-23 15:50:18 +02:00
Zaiming (Stone) Shi 47a3096776 test: add test case for file path validation 2023-05-23 14:22:50 +02:00
Zaiming (Stone) Shi 2fa5b511bf chore: hide stale config 2023-05-23 14:20:49 +02:00
Zaiming (Stone) Shi 91150e6d83 chore: allow special chars in log dir
the current valdiator does not allow space and ':' in log file path
which is an unresonable restriction
2023-05-23 14:20:49 +02:00
Zaiming (Stone) Shi e78ffb2dd4 chore: bump apps version 2023-05-23 13:04:28 +02:00
Zaiming (Stone) Shi 3bb1f7ab2b Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-23 13:03:55 +02:00
Kjell Winblad 89ea1646be fix: IoTDB name for version option
The previous name for the version option was v1.0.x which is clearly
wrong since this option was tested against IoTDB version v1.1.0. This
commit fixes this by renaming the option to v1.x.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9926
2023-05-23 10:57:05 +02:00
Kjell Winblad d19ddb1832 fix: IoTDB bridge incoming payload needs to be parsed as JSON
There was an incorrect assumption that the data incoming to the IoTDB
bridge has already been parsed. This is fixed by parsing the payload as
JSON data if the payload is not already a map.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9854
2023-05-23 10:14:45 +02:00
zhongwencool 3b6295a085
Merge pull request #10781 from zhongwencool/fix-bridge-test-failed
test: bridge test init dashboard listener failed
2023-05-23 11:16:57 +08:00
某文 1deb3c50c8 test: bridge test init dashboard listener failed 2023-05-23 10:28:01 +08:00
某文 67a30ee976 test: bridge test init dashboard listener failed 2023-05-23 10:16:43 +08:00
某文 ed1268ed30 Merge branch 'release-50' into 0522-merge-release-50-to-master 2023-05-23 07:56:35 +08:00
zhongwencool 26e31137c7
Merge pull request #10764 from HJianBo/remove-unused-comments
chore: remove unused conf comments
2023-05-23 07:53:25 +08:00
Zaiming (Stone) Shi 68f4ae70b2 chore: bump ee release version to e5.0.4-alpha.2 2023-05-22 22:41:19 +02:00
Zaiming (Stone) Shi 732a7be187 Merge remote-tracking branch 'origin/release-50' 2023-05-22 17:46:54 +02:00
Zaiming (Stone) Shi 1f44dd4579
Merge pull request #10756 from zmstone/0519-refactor-move-lib-ee-emqx_ee_conf-to-apps-emqx_enterprise
0519 refactor move lib ee emqx ee conf to apps emqx enterprise
2023-05-22 16:56:51 +02:00
Thales Macedo Garitezi 1c4c7fad92
Merge pull request #10740 from thalesmg/perf-buffer-worker-periodic-metrics-v50
perf(buffer_worker): update metrics periodically rather than immediately (v5.0)
2023-05-22 11:41:41 -03:00
Thales Macedo Garitezi 0877e4296a fix(dashboard): add missing function clause
Example error:
https://github.com/emqx/emqx/actions/runs/5045715277/jobs/9052482682#step:8:294

The previous change was not equivalent to the previous `maps:with/2`
behavior.
2023-05-22 11:35:04 -03:00
Thales Macedo Garitezi 65f973044f feat(pulsar): improve authn error check time and add connect timeout
Fixes https://emqx.atlassian.net/browse/EMQX-9910
2023-05-22 11:33:16 -03:00
Andrew Mayorov 3f882ddcc6
Merge pull request #10751 from ci/ct-flap-listener-api
fix(listen): wait until port is free when stopping ranch listeners
2023-05-22 17:00:45 +03:00
lafirest 88e4078885
Merge pull request #10747 from lafirest/fix/rule_funs_time_offset
fix: port the `emqx_calendar` from v4.4
2023-05-22 20:52:21 +08:00
Zaiming (Stone) Shi 0b9a32f49a chore: add BSL in apps/emqx_enterprise 2023-05-22 14:51:27 +02:00
Zaiming (Stone) Shi 40e8d5d039 refactor: rename lib-ee/emqx_ee_conf to apps/emqx_enterprise 2023-05-22 14:51:27 +02:00
Zaiming (Stone) Shi 3482c2c849 refactor: redefine default value for 'node.applications' in enterprise 2023-05-22 14:51:26 +02:00
lafirest 42f9c9aa9a
Merge pull request #10746 from lafirest/fix/events_api
fix: supports test the `$events/delivery_dropped` event by API
2023-05-22 20:50:03 +08:00
Andrew Mayorov d22541e8b3
fix(ft): correct mistyped option in README 2023-05-22 15:32:08 +03:00
Thales Macedo Garitezi 0559d6f639 refactor(buffer_worker): use static fn for bumping counters 2023-05-22 09:12:08 -03:00
Thales Macedo Garitezi 9c71f4ecbd test: fix flaky test 2023-05-22 09:12:00 -03:00
Thales Macedo Garitezi c74c93388e refactor: rename some variables and sum type constructors for clarity 2023-05-22 09:11:23 -03:00
Thales Macedo Garitezi 7d798c10e9 perf(buffer_worker): flush metrics periodically inside buffer worker process
Fixes https://emqx.atlassian.net/browse/EMQX-9905

Since calling `telemetry` is costly in a hot path, we instead collect
metrics inside the buffer workers state and periodically flush them,
rather than immediately as events happen.
2023-05-22 09:11:23 -03:00
Andrew Mayorov 092e112d62
Merge pull request #10767 from ci/ft-prop-flaky
test(ft): make proptest less aggressive
2023-05-22 12:45:24 +03:00
Andrew Mayorov 23542d1262
fix(listen): wait until port is free when stopping ranch listeners
Simple `cowboy:stop_listener/1` will not close the listening socket
explicitly, it was the source of test flaps before this fix.
2023-05-22 12:29:40 +03:00
Zhongwen Deng cb5a596c57 fix: make sure swagger.json is fully generated 2023-05-22 16:58:13 +08:00
Andrew Mayorov 5b312a415b
Merge pull request #10759 from ci/ct-flap-clickhouse
fix(clickhouse): start app in tests
2023-05-22 11:11:15 +03:00
Andrew Mayorov 966f748551
test(ft): make proptest less aggressive
So that the chance of getting huge coverage and as a result, high
chance of running out of memory, is reduced.
2023-05-22 10:45:27 +03:00
William Yang e824e1db39
Merge pull request #10698 from qzhuyan/perf/william/force-atom-conf-path
perf(config): enforcing atom key  path in hotcode path
2023-05-22 09:38:09 +02:00
firest 087dc59115 test: add test cases for the `rule_test` API 2023-05-22 15:25:35 +08:00
Zhongwen Deng 218fc4a839 refactor: add emqx_authz_file validate function 2023-05-22 14:54:23 +08:00
某文 082214d039 feat: add authz file rule validator 2023-05-22 14:25:05 +08:00
某文 27fca0ef3c fix: check authz's file rule before save to file 2023-05-22 14:22:20 +08:00
zhongwencool c65e3a46b1
Merge pull request #10742 from zhongwencool/fix-authz-file-rule-check
fix: check authz's file source rules in pre_config_update
2023-05-22 14:15:18 +08:00
JianBo He ce3ec351c7 chore: remove unused conf comments 2023-05-22 14:01:49 +08:00
Zhongwen Deng 96e7005de8 refactor: add emqx_authz_file validate function 2023-05-22 11:03:23 +08:00
firest 56a6b699ac fix: port the `emqx_calendar` from v4.4 2023-05-22 09:55:31 +08:00
某文 7c2aac64bc fix: bad cert file path in dashboard https listener 2023-05-21 07:41:46 +08:00
Zaiming (Stone) Shi 3e98b3b050
Merge pull request #10755 from zmstone/0517-fix-bridge-update-timeout-issue
0517 fix bridge update timeout issue
2023-05-20 06:19:26 +02:00
Andrew Mayorov ba6b208df2
fix(clickhouse): start app in tests
Otherwise, depending on the test execution order, tests might
sometimes fail.

Moreover, ensure that applications describe their dependecies
correctly and avoid starting irrelevant apps in tests.
2023-05-19 23:08:40 +03:00
Thales Macedo Garitezi bb4fef3cfd
Merge pull request #10752 from thalesmg/fix-pulsar-redact-auth-r50
fix(pulsar): redact error reason
2023-05-19 16:47:05 -03:00
Zaiming (Stone) Shi cb76e5a241 docs: add changelog for 10755 2023-05-19 20:41:26 +02:00
Zaiming (Stone) Shi 0d8ffc0d59 fix(resource-manager): ensure no false creation
Update is implemented as remove + create.
If a dleete call is made while the create is in progress
the remove call is likely to timeout too.

This causes the follwing creation to falsely succeed,
because there is alreay a running child under the supervisor.

As a result, the resource is permanently removed after
resource_manager eventually handles the remove call.
2023-05-19 18:55:16 +02:00
Zaiming (Stone) Shi f5e5c59763 refactor(resource-manager-sup): do not force kill resource manager
the shutdown timeout is now set to infinity so it will never
force kill a resource manager, otherwise there will be
resource leaks
2023-05-19 18:55:16 +02:00
Zaiming (Stone) Shi 21de0f8274 fix(buffer-worker-sup): fast stop
the timeout shutdown in child spec may
significantly slow down the deletion of a resource

this commit chagnes the shutdown to brutal kill

also, the pool worker removal code has been delete
because it's not necessary since the entier pool is
going to be force-delete later anyway
2023-05-19 18:55:16 +02:00
Zaiming (Stone) Shi be90c63c78 chore(mqtt-connector): refine logging level
connect failure should be at warning level but not error,
the connecting state is visiable from dashbaord
also the resource manager logs connection failures in general
at warning level
2023-05-19 18:55:16 +02:00
Zaiming (Stone) Shi 5bbcf4b712 fix(mqtt-connector): faster shutdown 2023-05-19 18:55:16 +02:00
Zaiming (Stone) Shi 7eef86363a test: make erlfmt happy 2023-05-19 17:58:14 +02:00
Andrew Mayorov 9aaa0b6fe1
Merge pull request #9927 from emqx/file-transfer
feat: implement file transfer over mqtt
2023-05-19 18:35:13 +03:00
Thales Macedo Garitezi 4327e40740 fix(pulsar): redact error reason
Fixes https://emqx.atlassian.net/browse/EMQX-9940

Error log after fix:

```
2023-05-19T13:09:26.304769+00:00 [error] msg: failed_to_start_pulsar_client, mfa: emqx_bridge_pulsar_impl_producer:on_start/2, line: 104, instance_id: <<"bridge:pulsar_producer:pprodu">>, pulsar_hosts: pulsar://pulsar:6652, reason: {#{{"pulsar",6652} => #{error => 'AuthenticationError',message => "Unable to authenticate",request_id => 18446744073709551615}},{child,undefined,'pulsar_producer:pprodu:emqx@127.0.0.1',{pulsar_client,start_link,['pulsar_producer:pprodu:emqx@127.0.0.1',["pulsar://pulsar:6652"],#{conn_opts => #{auth_data => <<"******">>,auth_method_name => <<"basic">>},ssl_opts => []}]},transient,false,5000,worker,[pulsar_client]}}
```
2023-05-19 10:09:57 -03:00
lafirest 264404e180
Merge pull request #10511 from lafirest/fix/resource_safe_log
fix(resource): make some logging of the resource manager more secure
2023-05-19 20:42:21 +08:00
lafirest 1aa904d019
Merge pull request #10738 from lafirest/fix/tdengine_template
fix(tdengine): add supports for the `super table` feature in the SQL template
2023-05-19 20:41:50 +08:00
Andrew Mayorov c440cd77b0
fix(test): prefer `emqx_connector_http:join_paths/2` in api tests
It is more consistent with what HTTP servers are expecting, plus
not OS-specific as `filename:join/2`.
2023-05-19 15:37:01 +03:00
Thales Macedo Garitezi 8cfcc8fbaf
Merge pull request #10743 from thalesmg/fix-bridge-metric-aggregation-r50
fix(bridge_api): don't crash when formatting empty/unknown bridge metrics
2023-05-19 09:07:22 -03:00
某文 6cb9efd7d3 feat: add authz file rule validator 2023-05-19 18:41:59 +08:00
Andrew Mayorov 5c112602b8
test(evict): fix flapping testcase
Postpone enabling eviction agent so that target connection will not
get evicted too early.
2023-05-19 13:37:36 +03:00
Kjell Winblad 28dda39e39
Merge pull request #10724 from kjellwinblad/kjell/docs/missing_http_api_endpoint_descriptions/EMQX-9767
docs: make sure there is a summary for all API endpoints
2023-05-19 12:05:41 +02:00
firest baeb96a6e4 chore: update changes 2023-05-19 15:36:18 +08:00
firest 0eea8438bf fix(resource): make some logging of the resource manager more secure 2023-05-19 15:28:19 +08:00
firest fa799e95a4 fix: supports test the `$events/delivery_dropped` event by API 2023-05-19 14:20:26 +08:00
firest 5a08a7b9de fix(tdengine): minor improvement of code and changes 2023-05-19 11:14:31 +08:00
JianBo He 48ceb14755
Merge pull request #10737 from HJianBo/fix-mqtt-sn-bugs
Avoid urldecode clientid twice
2023-05-19 09:19:12 +08:00