yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,883 Commits 393 Branches 669 Tags 74 MiB
6432c9c8fc
Commit Graph

8536 Commits

Author SHA1 Message Date
zhongwencool
505d677685 chore: move reload_local_etc_config from emqx_conf to emqx_conf_cli 2023-06-11 09:11:17 +08:00
zhongwencool
1ca1ba9e7a fix: get_sources return default authz when sources key not found 2023-06-11 09:11:17 +08:00
zhongwencool
01671d82c7 feat: delete authorization form etc/emqx.conf 2023-06-11 09:11:17 +08:00
zhongwencool
6c6ff4ea1f chore: rename {mod} to module 2023-06-11 09:11:17 +08:00
zhongwencool
cf9a207743 feat: check all config schema before loading 2023-06-11 09:11:17 +08:00
zhongwencool
b829d8edc2 feat: support --auth-chains merge|replace mod 2023-06-11 09:11:17 +08:00
zhongwencool
21ffd958b4 feat: support authn merge_authenticators 2023-06-11 09:11:17 +08:00
zhongwencool
aac5b85d26 feat: authz support merge sources 2023-06-11 09:11:17 +08:00
zhongwencool
994db58cbf fix: load authz's default sources 2023-06-11 09:11:17 +08:00
zhongwencool
c819ac27f4 feat: add conf reload cli 2023-06-11 09:11:17 +08:00
zhongwencool
61bbe19eba
Merge pull request #10961 from HJianBo/support-infinity-max-conns-in-gateway 
Adds support for unlimited max connections for gateway listeners
 2023-06-11 09:09:34 +08:00
zhongwencool
599eb9fef5 chore: correct persistent_session_store's default value type to boolean 2023-06-10 19:36:05 +08:00
Zaiming (Stone) Shi
97850de524 Merge remote-tracking branch 'origin/release-51' into 0610-merge-release-51-to-master 2023-06-10 12:23:55 +02:00
Zaiming (Stone) Shi
9f135d1f2b fix(quic): environment variables as cert file prefix for quic listener 2023-06-10 12:23:20 +02:00
JianBo He
aea3ae4024
Merge pull request #11005 from HJianBo/fix-tracing-logs-typos 
fix(authn): fix the hardcode method in trace logs
 2023-06-10 18:08:28 +08:00
Zaiming (Stone) Shi
3901ed3255
Merge pull request #10996 from sstrigler/EMQX-9435-request-the-non-existent-api-v-4-nodes-endpoint-in-emqx-e-5-0-1-returns-http-code-200 
Return 404 for everything we don't know
 2023-06-10 11:20:25 +02:00
Zaiming (Stone) Shi
f98cdd4983
Merge pull request #10994 from sstrigler/EMQX-10003-e-5-0-4-auth-header-value-of-webhook-data-bridge-can-be-found-in-emqx-log 
fix(emqx_utils): redact proxy-authorization headers
 2023-06-10 11:18:58 +02:00
JianBo He
fdaac9800c fix(authn): fix the hardcode method in trace logs 2023-06-10 13:04:24 +08:00
Serge Tupchii
48ac942807 feat(emqx_bridge_kafka): add tcp keepalive option 
Closes: EMQX-8725
 2023-06-09 22:10:17 +03:00
Kjell Winblad
43292c06e4
Merge pull request #10997 from kjellwinblad/kjell/fix/clickhouse_lost_messages/EMQX-10217 
fix: lost messages when clickhouse closes while sending messages
 2023-06-09 21:08:25 +02:00
Serge Tupchii
87b57112df refactor: move tcp keepalive options helper to emqx_utils 2023-06-09 21:43:50 +03:00
Paulo Zulato
7c63f20aa8
Merge pull request #10999 from paulozulato/fix-kafka-time-field-too-large 
fix(kafka): fix timeout field for some kafka fields
 2023-06-09 15:29:59 -03:00
Paulo Zulato
720a84d48b fix(kafka): fix timeout field for some kafka fields 
Change type of fields 'Partition Count Refresh Interval' and 'Offset
Commit Interval' to avoid accepting values larger than allowed.

Fixes https://emqx.atlassian.net/browse/EMQX-10196
Fixes https://emqx.atlassian.net/browse/EMQX-10199
 2023-06-09 13:38:04 -03:00
Kjell Winblad
1b3af2ac91 refactor: nicer handling of errors and warning log for recoverable errors 2023-06-09 17:22:16 +02:00
Stefan Strigler
57d72ed23e fix(emqx_utils): redact proxy-authorization headers 2023-06-09 17:13:53 +02:00
Stefan Strigler
8c1334c421 fix(emqx_dashboard): return 404 for everything we don't know 2023-06-09 17:13:18 +02:00
Andrew Mayorov
b930f4cc73
Merge pull request #10987 from keynslug/fix/EMQX-9257/sep-placeholder 
refactor: tear `emqx_plugin_libs` application apart
 2023-06-09 17:02:14 +02:00
Thales Macedo Garitezi
4a9f2247ca test(conf_app): fix bad merge conflict 2023-06-09 11:06:54 -03:00
Kjell Winblad
61488ec407 fix: lost messages when clickhouse closes while sending messages 
Fixes:
https://emqx.atlassian.net/browse/EMQX-10217
 2023-06-09 16:04:38 +02:00
Thales Macedo Garitezi
40675843a1 fix(iotdb): remove root request_timeout option 
Fixes https://emqx.atlassian.net/browse/EMQX-10175

Since IoTDB bridge is not used as an authn/authz source and reuses the
webhook connector, we should have only one request timeout field (the
`resource_opts.request_ttl` one) like the webhook bridge does, to
avoid confusion and potentially bad configuration combinations.
 2023-06-09 10:39:01 -03:00
Zaiming (Stone) Shi
643da0564b
Merge pull request #10990 from zmstone/0609-fix-schema-for-dashboard-use 
0609 fix schema for dashboard use
 2023-06-09 15:07:51 +02:00
Thales Macedo Garitezi
20a7ce64cd
Merge pull request #10919 from thalesmg/test-flakiness-20230601-a 
test(pulsar_producer): attempt to fix flaky test
 2023-06-09 09:30:30 -03:00
Andrew Mayorov
c8360f86b7
chore: bump application versions 2023-06-09 14:44:38 +03:00
Andrew Mayorov
830ba54721
refactor(pluglib): remove emqx_plugin_libs application 2023-06-09 14:44:37 +03:00
Andrew Mayorov
e6fb0203b4
refactor(pluglib): move connectivity checks to emqx_connector_lib 2023-06-09 14:44:37 +03:00
Andrew Mayorov
8919a6ef93
refactor(pluglib): provide SQL related utils in emqx_utils_sql 2023-06-09 14:44:37 +03:00
Andrew Mayorov
a51baaa206
refactor(pluglib): move conversion utils to emqx_utils_conv 2023-06-09 14:44:37 +03:00
Andrew Mayorov
7d0abb6146
feat(emqx): add emqx_topic:match_any/2 utility 2023-06-09 14:44:37 +03:00
Andrew Mayorov
d6c1ee183f
refactor(pluglib): move emqx_placeholder to utils app 
Also make user that existing code calls it directly.
 2023-06-09 14:44:36 +03:00
Serge Tupchii
2aad6671ac fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file 
As we use `keep_tables` option, we don't need to modify a backup schema before importing the backup file.
With this option, `mnesia:restore/2` ignores backup schema and keeps current table schema unchanged.
 2023-06-09 14:41:21 +03:00
SergeTupchiy
e61b2100a3
Merge pull request #10676 from SergeTupchiy/EMQX-9203-config-backup 
feat: implement configuration and user data export/import
 2023-06-09 14:18:39 +03:00
Serge Tupchii
e4d09d4ad4 feat: implement configuration and user data export/import CLI 
Closes: EMQX-9203
 2023-06-09 14:11:47 +03:00
Kjell Winblad
4215da12f0
Merge pull request #10970 from kjellwinblad/kjell/feat/kafka_add_async_param/EMQX-8631 
feat: add sync/async option to the Kafka producer bridge
 2023-06-09 12:51:23 +02:00
Kjell Winblad
2671e8ecf9 fix: dialyzer type problem 2023-06-09 11:00:05 +02:00
Ilya Averyanov
c2635e938b
Merge pull request #10974 from keynslug/ft/EMQX-9523/backend-enable-flag 
fix(ft): add `enable` flag to every backend
 2023-06-09 11:44:37 +03:00
JianBo He
53eb8d7f1b test: fix flaky tests introduced by https://github.com/emqx/emqx/pull/10948 2023-06-09 15:48:23 +08:00
JianBo He
72311a546b chore: more clear funcation name 2023-06-09 15:48:23 +08:00
JianBo He
6a05663bd5 chore: ensure default value's type be the first one in the union type 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-06-09 15:48:23 +08:00
JianBo He
dbb8742e0e test: cover max_connection in infinity value 2023-06-09 15:48:23 +08:00
JianBo He
8df7b1a1be fix(gateway): fix http-api 500 issue if setting max_connections to infinity 2023-06-09 15:48:23 +08:00
JianBo He
c450a1784a
Merge pull request #10950 from HJianBo/mqttsn-qos3 
fix(mqttsn): checking enable_qos3 option
 2023-06-09 15:46:55 +08:00
JianBo He
2153cd4317
Merge pull request #10979 from HJianBo/fix-coap-connectionless-pub 
fix(coap): fix crash publish/subscribe in connectionless mode
 2023-06-09 15:46:18 +08:00
Zaiming (Stone) Shi
38e767b808 fix(schema): map new duration types to dashboard's type name 2023-06-09 09:14:55 +02:00
Stefan Strigler
2e9ac8f1ec
Merge pull request #10980 from sstrigler/EMQX-10189-mark-support-for-json-output-format-in-prometheus-stats-as-deprecated 
Mark support for json output format in prometheus stats as deprecated
 2023-06-09 09:00:34 +02:00
Zaiming (Stone) Shi
ee697831f6
Merge pull request #10926 from zmstone/0602-add-enable-as-an-alias-for-enabled 
0602 add enable as an alias for enabled
 2023-06-09 08:44:20 +02:00
Zaiming (Stone) Shi
9f6beee6b3
Merge pull request #10936 from zmstone/0603-imporove-desc 
docs: refine some desc
 2023-06-09 08:44:03 +02:00
Zaiming (Stone) Shi
af5c6720de Merge remote-tracking branch 'origin/master' into release-51 2023-06-09 08:37:29 +02:00
lafirest
601db2b1d3
Merge pull request #10988 from lafirest/fix/konservi_sentema 
fix(resource): improve log security when resource creation fails
 2023-06-09 13:46:12 +08:00
firest
86a7b2d69a fix(resource): improve log security when resource creation fails 2023-06-09 11:43:42 +08:00
zhongwencool
daa5450218
Merge pull request #10985 from zmstone/0608-rename-cluster-call-cmd 
0608 rename cluster call cmd
 2023-06-09 10:56:18 +08:00
zhongwencool
4835c30f8c
Merge pull request #10975 from zhongwencool/conf-docs 
fix: link different document between community and enterprise
 2023-06-09 09:50:45 +08:00
JianBo He
c733ddb166
Merge pull request #10977 from HJianBo/fix-stomp-bugs 
Fix stomp gateway subscription_cnt is not updated timely and max_headers option is not working
 2023-06-09 09:47:26 +08:00
zhongwencool
c9cc06b6b1 chore: compile failed 2023-06-09 09:31:32 +08:00
Zaiming (Stone) Shi
83c36a3c9f refactor(emqx_conf_cli): do not print hidden roots in 'conf show' cmd 2023-06-09 09:31:32 +08:00
Zaiming (Stone) Shi
31ab973b30 refactor(emqx_conf_cli): rename cluster_call ctl command 
Renamed 'clustedr_call' ctl command to 'cluster_sync' as a sub-command
of 'conf'
The old 'cluster_call' command is kept, but no usage prints
 2023-06-09 09:31:32 +08:00
Zaiming (Stone) Shi
96a41ac6bd feat(emqx_ctl): support hidden commands 
hidden commands do not deserve a place in the usage page
 2023-06-09 09:31:32 +08:00
Zaiming (Stone) Shi
47f826f99a chore: fix typo 2023-06-09 09:31:32 +08:00
Serge Tupchii
53ec8326b0 fix(emqx_listeners): convert authentication certs 2023-06-08 20:44:18 +03:00
Serge Tupchii
33fa053b9b test(emqx_common_test_helpers): keep EMQX ENV vars on a child node for its lifetime 2023-06-08 20:42:51 +03:00
Serge Tupchii
5f526d548a style: use double percent (%%) comments 2023-06-08 20:42:51 +03:00
Kjell Winblad
1c7834e056 fix: fixes due to comments from @zmstone 2023-06-08 16:47:02 +02:00
Kjell Winblad
cb3a5fdbd4 style: only callback modules should do dynamic calls 2023-06-08 16:27:04 +02:00
Kjell Winblad
d524f8c805 refactor: rename config parameter 2023-06-08 16:27:04 +02:00
Kjell Winblad
6f2271e9f0 test: add Kafka producer bridge test case for query_mode parameter 2023-06-08 16:27:04 +02:00
Kjell Winblad
ed9e29e769 refactor: refacor query_mode detection code 
This commit refactor the query_mode resource detection code according to
a suggestion from @zmstone. This commit should not contain any
functional change except for a change of the Kafka producer bridge
config.
 2023-06-08 16:26:55 +02:00
Kjell Winblad
273dedf7a6
Merge pull request #10911 from kjellwinblad/kjell/bridge/atom_length_too_long/EMQX-9609 
fix: friendly error message when creating bridges with too long names
 2023-06-08 15:46:42 +02:00
JianBo He
034caaf20f
Merge pull request #10941 from HJianBo/enhance-prometheus-sampe-perf 
perf(prometheus): udpate the prometheus dep vsn
 2023-06-08 21:34:50 +08:00
Ivan Dyachkov
abbba71191 fix(tls): issue when ssl listner is configured to use tls v1.3 only 
clients could not connect due to incompatible tls options if ssl listner is configured to use tls v1.3 only
 2023-06-08 15:13:22 +02:00
Zaiming (Stone) Shi
7d2ea3dc76 docs: refine some desc 2023-06-08 13:58:31 +02:00
Andrew Mayorov
ce8cc05cc8
chore(ftschema): mention translate is for tests purpose only 2023-06-08 14:54:14 +03:00
Andrew Mayorov
e3c16be3c4
fix(ftschema): ensure translate follow emqx_config behavior 
So that atom keys won't unexpectedly get to converters / validators.
 2023-06-08 14:54:14 +03:00
Andrew Mayorov
ae97d516b8
fix(ft): add enable flag to every backend 
And tolerate multiple configured backends, as long as only one of
them is enabled.
 2023-06-08 14:54:13 +03:00
Ivan Dyachkov
b4d081f424
Merge pull request #10978 from id/0608-fix-404-in-rule-engine-api 
fix: missing 404 response in rule engine api schema
 2023-06-08 13:20:02 +02:00
Kjell Winblad
47fa17b3c1 feat: add sync/async option to the Kafka producer bridge 
This commit makes it possible to configure if a Kafka bridge should work
in query mode sync or async by setting the resource_opts.query_mode
configuration option.

Fixes:
https://emqx.atlassian.net/browse/EMQX-8631
 2023-06-08 13:16:06 +02:00
Stefan Strigler
f5e93b1b2b docs(emqx_prometheus): leave remark about json being deprecated 2023-06-08 13:08:45 +02:00
Zaiming (Stone) Shi
b481994e74 chore: bump to version 5.1.0-alpha.4 2023-06-08 12:30:35 +02:00
Zaiming (Stone) Shi
c1cf2365c2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-08 12:30:02 +02:00
JianBo He
95f0745e31 fix(coap): fix crash publish/subscribe in connectionless mode 
this bug was introduced by https://github.com/emqx/emqx/pull/10871
 2023-06-08 18:08:11 +08:00
Ivan Dyachkov
711cedc282 fix: missing 404 response in rule engine api schema 2023-06-08 11:46:16 +02:00
JianBo He
4065f08083 fix(stomp): fix frame limitation is not working 2023-06-08 16:57:12 +08:00
JianBo He
bad0c35bb9 fix(stomp): ensure the subscripton_cnt timely updated 2023-06-08 15:25:44 +08:00
Zaiming (Stone) Shi
641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51 
0607 merge master to release 51
 2023-06-08 09:22:21 +02:00
JianBo He
dc01b7109e chore: update prometheus vsn and grafana template 2023-06-08 14:29:54 +08:00
JianBo He
e0ec050e34 chore: update prometheus vsn 2023-06-08 14:29:54 +08:00
JianBo He
f53e13b8b9 perf: update to our forked prometheus deps 
it's based on v4.10.0 of upstream, and patched this PR:
https://github.com/emqx/prometheus.erl/pull/5
 2023-06-08 14:29:54 +08:00
Ilya Averyanov
b9f1a70214
Merge pull request #10967 from savonarola/0607-rebalance-fixes 
Fix rebalance issues
 2023-06-08 08:29:37 +03:00
zhongwencool
0116c53395 fix: link different document between community and enterprise 2023-06-08 11:33:39 +08:00
JianBo He
d5ebf5650f test: improve the test 2023-06-08 11:24:07 +08:00
JianBo He
fcef456174 feat: add live_connections field for some HTTP APIs 
i.e:
- `/monitor_current`, `/monitor_current/nodes/{node}`
- `/monitor/nodes/{node}`, `/monitor`
- `/node/{node}`, `/nodes`
 2023-06-08 11:24:06 +08:00
JianBo He
5d86443910 chore: fix dialyzer warnings 2023-06-08 11:06:25 +08:00
JianBo He
d0222ae79d chore: ensure test 2023-06-08 10:23:59 +08:00
JianBo He
4e9eeb489d
Merge pull request #10969 from thalesmg/refactor-gcp-pubsub-on-stop-v50 
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
 2023-06-08 09:47:19 +08:00
JianBo He
0b95bc1c1b
Merge pull request #10907 from HJianBo/refactor-on-stop-simple 
feat: refactored some bridges to avoid leaking resources part.2
 2023-06-08 09:37:31 +08:00
Zaiming (Stone) Shi
641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf 
feat: don't hide enable in flapping detect conf
 2023-06-07 23:04:56 +02:00
Zaiming (Stone) Shi
ccd2589ff2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-07 21:43:29 +02:00
Ilya Averyanov
7f2de66dab fix(rebalance): add wait_health_check timeout to node evacuation 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-06-07 21:37:01 +03:00
Andrew Mayorov
39b0d41527
fix(bpapi): ignore dropped BPAPI module 
It doesn't really breaks interoperability.
 2023-06-07 21:33:47 +03:00
Zaiming (Stone) Shi
aecea50564
Merge pull request #10902 from zmstone/0529-donot-copy-cluster-conf-from-newer-version 
0529 donot copy cluster conf from newer version
 2023-06-07 20:32:11 +02:00
Zaiming (Stone) Shi
1e0b2de0ea chore: refine data sync debug log messages 2023-06-07 20:19:48 +02:00
Zaiming (Stone) Shi
ed1ad4e684 chore(emqx_schema): add 'eanble' as an alias to 'enabled' fields 2023-06-07 20:19:48 +02:00
Thales Macedo Garitezi
d9aac63f82 refactor(gcp_pubsub): always cleanup resources when calling on_stop 
Fixes https://emqx.atlassian.net/browse/EMQX-9934

The way that this connector was structured already depended only on the resource ID.  This
PR just makes it completely independent of the resource state term, and also treats an
already absent pool as a success when stopping it.
 2023-06-07 14:34:22 -03:00
Thales Macedo Garitezi
105c26afb5
Merge pull request #10964 from thalesmg/fix-hocon-compact-errors-v50 
fix(schema): avoid `function_clause` error when compacting errors
 2023-06-07 14:00:24 -03:00
Andrew Mayorov
79d430cf03
refactor(pluglib): move emqx_plugin_libs_proto_v1 into emqx app 
And rename it to `emqx_metrics_proto_v1` in the process.
 2023-06-07 18:30:12 +03:00
Zaiming (Stone) Shi
622ac4d195
fix(emqx_schema): binary for string default value 2023-06-07 17:14:10 +02:00
JimMoen
4ea3483083
perf(topic): use tail recursive to joining a topic 2023-06-07 22:59:53 +08:00
JimMoen
b0e18f5e75
fix(shared-sub): validate share topic error send DISCONNECT 2023-06-07 22:59:52 +08:00
JimMoen
070e410c69
refactor: variable and func rename 2023-06-07 22:59:52 +08:00
JimMoen
ea81a924f1
refactor: move topic() types def in emqx_types.erl 2023-06-07 22:59:52 +08:00
JimMoen
65483e972e
refactor: use unified table define 2023-06-07 22:51:44 +08:00
zhongwencool
70e800f8a4 chore: delete unuse default value and make flapping_detect middle improtance 2023-06-07 22:50:11 +08:00
zhongwencool
5659cc2b47 feat: add converter to flapping_detect 2023-06-07 21:54:32 +08:00
zhongwencool
1d33b7dbd8 chore: make static check happy 2023-06-07 21:54:32 +08:00
zhongwencool
535870386a fix: zones update api failed 2023-06-07 21:54:32 +08:00
zhongwencool
d58506a7c3 feat: always update default zones 2023-06-07 21:54:32 +08:00
zhongwencool
48381d4c86 feat: refactor flapping detect conf 2023-06-07 21:54:32 +08:00
Thales Macedo Garitezi
cc8631223e fix(schema): avoid function_clause error when compacting errors 
Fixes https://emqx.atlassian.net/browse/EMQX-10168

The bridge probe API displayed the typecheck errors for the new
timeout duration types correctly, but when an user tried to create the
bridge anyway a `function_clause` error was raised when trying to
compact hocon errors:

```
09:47:19.045 [warning] [exception: :error, path: '/bridges', reason: {:case_clause, {:error, {:config_update_crashed, :function_clause}}}, stacktrace: [{:emqx_bridge_api, :create_or_update_bridge, 4, [file: '/home/thales/dev/emqx/emqx/apps/emqx_bridge/src/emqx_bridge_api.erl', line: 602]}, {:minirest_handler, :apply_callback, 3, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 111]}, {:minirest_handler, :handle, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 44]}, {:minirest_handler, :init, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 27]}, {:cowboy_handler, :execute, 2, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_handler.erl', line: 41]}, {:cowboy_stream_h, :execute, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 318]}, {:cowboy_stream_h, :request_process, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 302]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]]
```

This fixes the issue so that both APIs return more friendly error messages.
 2023-06-07 10:34:58 -03:00
Paulo Zulato
a4407764f3 feat(connection): configurable TCP keepalive 
Fixes https://emqx.atlassian.net/browse/EMQX-9852
 2023-06-07 10:23:38 -03:00
Thales Macedo Garitezi
f9ff1007a0
Merge pull request #10944 from thalesmg/pubsub-jwt-on-demand-v50 
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
 2023-06-07 09:22:09 -03:00
Ilya Averyanov
d1a1e8041d fix(rebalance api): fix error message formatting 2023-06-07 13:57:16 +03:00
Andrew Mayorov
c05f462d0a
fix(tlslib): bring back deterministic filenames for managed certs 2023-06-07 13:32:35 +03:00
William Yang
c1be86c5ec
Merge pull request #10931 from qzhuyan/perf/william/save-times-of-maps-build 
Perf/william/save times of maps build
 2023-06-07 10:32:45 +02:00
SergeTupchiy
3c482e9aa6
Merge pull request #10953 from SergeTupchiy/fix-listener-authn-update 
fix(emqx_listeners): fix listener authentication create/update
 2023-06-07 11:12:41 +03:00
Zaiming (Stone) Shi
cb3e787cb1 Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-07 09:14:04 +02:00
Zaiming (Stone) Shi
ce4e49cd9e
Merge pull request #10943 from zmstone/0605-hide-mcast-cluster-discovery 
chore: start deprecating mcast cluster discovery
 2023-06-07 08:32:33 +02:00
JianBo He
81387800b5
Merge branch 'master' into mqttsn-qos3 2023-06-07 09:49:29 +08:00
JianBo He
21b451a4c2
Merge pull request #10951 from HJianBo/mqttsn-mountpoint-pub 
fix(mqttsn): make mountpoint works for publish
 2023-06-07 09:39:46 +08:00
Andrew Mayorov
32a8963582
Merge pull request #10927 from keynslug/fix/EMQX-9930/tls-cert-gc 
feat(tlslib): add separate managed certfiles GC process
 2023-06-06 20:12:18 +03:00
Thales Macedo Garitezi
260fae296b feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1) 
Fixes https://emqx.atlassian.net/browse/EMQX-9603

Rather than relying on a JWT worker to produce and refresh tokens, we
could just produce then on demand when pushing the messages to GCP
PubSub.  That can generate a bit of extra work (as multiple processes
might realize it’s time to refresh the JWT and do so), but that
shouldn’t be much.  In return, we avoid any possibility of not having
a fresh JWT when pushing messages.
 2023-06-06 13:19:24 -03:00
Serge Tupchii
bf2b5a9f24 fix(emqx_listeners): fix listener authentication create/update 
Listener authentication is a list of authenticators, so each of them must be created
or updated individually.
 2023-06-06 19:00:35 +03:00
Serge Tupchii
1968589f81 fix(emqx_schema): don't allow enabling fail_if_no_peer_cert if verify_none is set 
Setting `fail_if_no_peer_cert = true` and `verify = verify_none` causes connection errors.

Closes: EMQX-9586
 2023-06-06 17:20:22 +03:00
Thales Macedo Garitezi
2c97ee1b3f test(conf_app): fix test after common test helpers changes 2023-06-06 09:32:38 -03:00
Thales Macedo Garitezi
97a9bb484a test(pulsar_producer): attempt to fix flaky test 2023-06-06 09:32:38 -03:00
Thales Macedo Garitezi
770dd188b1
Merge pull request #10947 from emqx/release-51 
Release 51 to master
 2023-06-06 09:32:16 -03:00
JianBo He
bcc47442eb fix(mqttsn): make mountpoint works for publish 2023-06-06 18:44:27 +08:00
Andrew Mayorov
68cd0c47f5
Merge pull request #10945 from fix/EMQX-9967/tmpfn-enametoolong 
fix(ft): ensure temp filenames are under 255 bytes long
 2023-06-06 13:34:47 +03:00
JianBo He
26748ef242 fix(mqttsn): checking enable_qos3 option 2023-06-06 18:21:26 +08:00
Andrew Mayorov
d5aedaac7d
fix(tlslib): append random suffix to managed certfiles 
In order to lessen the chance of ambiguity in determining collectable
certfiles during GC.
 2023-06-06 11:21:03 +03:00
Andrew Mayorov
dcd59e4f1b
fix(ft): set more conservative filename length limit 
Otherwise, local fs exporter will have a hard time preserving the
filemeta, because its filename is even 13 bytes longer.
 2023-06-06 10:40:20 +03:00
Andrew Mayorov
69b98c1830
fix(ft): ensure temp filenames are under 255 bytes long 2023-06-06 10:38:31 +03:00
某文
47d7e6ce01 feat: update gateway from cli 2023-06-06 08:12:47 +08:00
zhongwencool
ddb4a09a3d
Merge pull request #10935 from zhongwencool/listeners-conf-update 
feat: update listeners from cli
 2023-06-06 08:10:45 +08:00
Zaiming (Stone) Shi
3caf114f68 test: fix test case is_list(Config) function guard 2023-06-05 23:00:41 +02:00
Zaiming (Stone) Shi
bb1cf9beaa Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-05 22:56:58 +02:00
Andrew Mayorov
f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config 2023-06-05 23:32:13 +03:00
Andrew Mayorov
750b7158d4
fix(tlsgc): make computing orphans safe against symlinking 
Before this commit, `orphans/1` considered managed file an orphan based
on its absolute filename, which is not safe when, for example, emqx has
a symlink configured as `data_dir` while config references certfiles
through realpaths.
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
31d38e18c9
refactor(tlsgc): simplify fold_config/4 2023-06-05 22:41:08 +03:00
Andrew Mayorov
f8e8b7993b
fix(tlsgc): anticipate only binaries as string in raw config 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
468fd98173
test(utilfs): add few more traverse_dir/3 testcases 2023-06-05 22:41:08 +03:00
Andrew Mayorov
3c2a7dbadc
fix(tlsgc): consolidate conf keypaths knowledge in emqx_tls_lib 
So that this GC mechanism will be easier to maintain.
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
ec06850bd0
test(auth): drop obsolete testcase altogether 
Most of the stuff that was tested by this test case is now covered
by `emqx_tls_certfile_gc_SUITE`.
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
206d0472e0
fix(emqx): avoid interference in testsuites 
Also remove that `?wait_async_action` in `emqx_trace_SUITE` which
consistently becomes stuck for 100 seconds. It's not clear why it
was there in the first place because listeners should start
synchronously, so `emqx_common_test_helpers:start_apps/1` won't return
until required listeners are started.
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
8a31e5639b
fix(emqx): ensure that standalone build works 
Turns out rebar3 `path` resource has troubles with dangling
symlinks and non-ASCII filenames. 🥲
 2023-06-05 22:41:07 +03:00
Andrew Mayorov
1cb226dffb
chore: bump applications versions 
* emqx_authz 0.1.22
* emqx_exhook 5.0.13
* emqx_utils 5.0.3
 2023-06-05 22:41:07 +03:00
Andrew Mayorov
afbf8cb32f
chore(util): drop obsolete stuff from emqx_utils_maps 2023-06-05 22:41:07 +03:00
Andrew Mayorov
99ea9b86c2
feat(tlslib): add separate managed certfiles GC process 
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
 2023-06-05 22:41:07 +03:00
Andrew Mayorov
95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys 
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
 2023-06-05 22:41:07 +03:00
Zaiming (Stone) Shi
67db9d6fe9 chore: bump version to 5.1.0-alpha.3 2023-06-05 20:25:48 +02:00
Thales Macedo Garitezi
75df622426
Merge pull request #10930 from thalesmg/max-int-timeout-v50 
check maximum timeout values in schema (5.1)
 2023-06-05 12:51:36 -03:00
Kjell Winblad
375661c6a1 test: add test case for error when bridge name is too long 2023-06-05 17:02:08 +02:00
Thales Macedo Garitezi
46393343e2 chore: use timeout_duration types for timer fields 
Fixes https://emqx.atlassian.net/browse/EMQX-10020
 2023-06-05 11:46:38 -03:00
Thales Macedo Garitezi
9bd37937f1 feat(types): add timeout_duration{,_ms,_s} types 2023-06-05 11:46:32 -03:00
Paulo Zulato
8d97a85b3c
Merge pull request #10893 from paulozulato/fix-oracle-error-handling 
fix(oracle): fix error handling on sync query
 2023-06-05 10:48:22 -03:00
Zaiming (Stone) Shi
bdcc069aac chore: start deprecating mcast cluster discovery 2023-06-05 14:58:53 +02:00
JimMoen
0f808345e6
Merge pull request #10924 from JimMoen/refactor_influxdb_on_stop 
feat: refactor influxdb connector to to avoid resources leaking
 2023-06-05 18:02:15 +08:00
lafirest
d51c658a30
Merge pull request #10908 from lafirest/feat/rocketmq_on_stop 
feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation
 2023-06-05 15:00:32 +08:00
JianBo He
dbc0cdce67 chore: dont allocate resource for simple connectiors 2023-06-05 10:59:00 +08:00
firest
5921ed3d2e fix(rocketmq): improve function name 2023-06-05 10:43:28 +08:00
JianBo He
e717ddafd7 chore: update changes 2023-06-05 10:19:46 +08:00
JianBo He
3739230435 feat: refactor connectors on_stop function to avoid resources leaking 
Supplement to https://github.com/emqx/emqx/pull/10895
 2023-06-05 10:19:46 +08:00
JianBo He
e07c86b6a8
Merge pull request #10909 from HJianBo/remove-deprecated-gateways 
feat: remove the deprecated '/gateway/*' HTTP APIs
 2023-06-05 09:59:50 +08:00
某文
d0d6992a14 feat: update listeners from cli 2023-06-04 20:07:33 +08:00
某文
03160ef599 test: add more test for update_config 2023-06-04 12:02:02 +08:00
某文
bd29433997 feat: support emqx_conf:update([exhook],Conf) 2023-06-04 09:30:17 +08:00
JimMoen
a174d5741b
feat: refactor influxdb connector to to avoid resources leaking 2023-06-04 02:21:49 +08:00
Zaiming (Stone) Shi
f469b31fee Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-03 11:21:22 +02:00
Paulo Zulato
c1d935fd34
Merge pull request #10917 from paulozulato/block-oracle-default-database-connection 
test(oracle): add match for reason field in a test case
 2023-06-02 18:16:12 -03:00
Andrew Mayorov
5b56a35c0a
Merge pull request #10932 from chore/drop-mqtt-mode-tests 
chore(bridge): drop deprecated `mode` setting from examples / tests
 2023-06-02 22:46:31 +03:00
Zaiming (Stone) Shi
0b87ea4cdc
Merge pull request #10929 from zmstone/0602-ci-pin-otp-25.3.2-1-elixir-1.14.5 
ci: upgrade to otp 25.3.2-1 and elixir 1.14.5
 2023-06-02 21:46:08 +02:00
Zaiming (Stone) Shi
aa46c0a484
Merge pull request #10928 from zmstone/0602-sync-release-51-to-master 
0602 sync release 51 to master
 2023-06-02 21:45:21 +02:00
Thales Macedo Garitezi
33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50 
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
 2023-06-02 16:20:36 -03:00
Andrew Mayorov
09290f8d61
chore(bridge): drop deprecated mode setting from examples / tests 2023-06-02 20:31:31 +03:00
Zaiming (Stone) Shi
e6fb27b3c7 fix(emqx_telemetry): make dialyzer happy 2023-06-02 18:02:46 +02:00
Zaiming (Stone) Shi
1ba07e1040 Merge remote-tracking branch 'origin/release-51' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-02 16:30:58 +02:00
Zaiming (Stone) Shi
28c564d15b Merge remote-tracking branch 'origin/master' into release-51 2023-06-02 16:25:48 +02:00
William Yang
b1c188f9b1 perf(mqtt-caps): dont filter cap keys 
save map builds
 2023-06-02 16:24:52 +02:00
William Yang
4961aca3d0 perf(mqtt-caps): save some map builds 2023-06-02 16:19:58 +02:00
Thales Macedo Garitezi
74ffd9ef96
Merge pull request #10918 from thalesmg/fix-ecpool-empty-unrecoverable-v50 
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
 2023-06-02 09:21:00 -03:00
Zaiming (Stone) Shi
367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition 
fix(emqx_cm): fix channel data registration race-condition
 2023-06-02 14:20:18 +02:00
Thales Macedo Garitezi
0790c88aaf
refactor: use default's type as first union member 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-06-02 09:08:11 -03:00
Thales Macedo Garitezi
2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50 
fix(plugins): use `emqx:running_nodes` for multicall operations
 2023-06-02 09:03:42 -03:00
Zaiming (Stone) Shi
c75e9bbe0d fix(emqx_cm): fix channel data registration race-condition 
when clustered, there are chances the a mqtt client process
get killed (e.g. holding the channel registeration lock for too long),
if the channel data inserts happen before casting out the message
for channel process monitoring, there is a chance for the
stale message left in the ets tables indefinitely.

this commit changes the order of the non-atomic operations:
it casts out the monitor request message before inserting
channel data.
 2023-06-02 11:43:35 +02:00
Stefan Strigler
c9ef1ccacd fix: headers is array not object 2023-06-02 10:24:33 +02:00
Stefan Strigler
c7f177054c fix(emqx_authn): verify_claims is list(), therefor default must match 2023-06-02 10:24:33 +02:00
Stefan Strigler
15c7c281ca fix: don't convert empty list default 2023-06-02 10:03:01 +02:00
Stefan Strigler
190e44f8be fix: brackets not allowed in swagger schema for component names 2023-06-02 10:03:01 +02:00
Stefan Strigler
8fdc9de9e9 fix: empty properties as empty map so it ends up as a json object 2023-06-02 10:03:01 +02:00
Stefan Strigler
e62bd63199 fix: don't add summary, remove it instead 2023-06-02 10:03:01 +02:00
Stefan Strigler
ef2581d13b fix: remove unsupported schema properties 2023-06-02 10:03:01 +02:00
Stefan Strigler
12a4e48727 fix: add definition for bearerAuth 2023-06-02 10:03:01 +02:00
zhongwencool
8d8efe449e
Merge pull request #10896 from zhongwencool/api-keys-hot-conf-update 
feat: api_key conf support hot conf
 2023-06-02 15:58:45 +08:00
zhongwencool
dd85c981cd
Merge pull request #10783 from zhongwencool/improve-authn 
feat: load changes to the authentication configuration from command line
 2023-06-02 15:58:13 +08:00
zhongwencool
9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve 
feat: support emqx_config:update([authrization],Conf) update
 2023-06-02 15:43:09 +08:00
lafirest
0f8824b40f
Merge pull request #10921 from lafirest/test/rocketmq_acl 
test(rocketmq): add an ACL test case
 2023-06-02 15:37:46 +08:00
JianBo He
5e0a3606ad chore: fix outdate tests 2023-06-02 13:30:03 +08:00
JianBo He
0633d894b8 test: fix failed tests 2023-06-02 13:30:03 +08:00
JianBo He
ec0049af7b feat: remove the deprecated '/gateway/*' HTTP APIs 2023-06-02 13:30:03 +08:00
firest
ea9b1e13d5 test(rocketmq): add an ACL test case 2023-06-02 11:19:06 +08:00
Paulo Zulato
fb455d68a3 fix(oracle): fix error handling on sync query 
Fixes https://emqx.atlassian.net/browse/EMQX-10075
 2023-06-01 20:50:29 -03:00
某文
367a0e312a chore: remove config_path() function 2023-06-02 07:09:08 +08:00
Andrew Mayorov
76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum 
fix(ft): respect checksum in `fin` packets
 2023-06-01 23:49:55 +03:00
Thales Macedo Garitezi
0072f4a598 fix(ecpool,bridge): treat {error, ecpool_empty} as a retriable error 
Example from flaky CI test:

```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
 2023-06-01 17:13:06 -03:00
Paulo Zulato
92c72c6994 test(oracle): add match for reason field in a test case 
Fixes https://emqx.atlassian.net/browse/EMQX-9980
 2023-06-01 15:12:09 -03:00
Paulo Zulato
364601c3aa
Merge pull request #10892 from paulozulato/block-oracle-default-database-connection 
fix(oracle): require either sid or service name
 2023-06-01 15:04:03 -03:00
Thales Macedo Garitezi
7d3ce14835 test: rm unused var warning 2023-06-01 14:00:43 -03:00
Thales Macedo Garitezi
648b8231ef fix(plugins): avoid crashing when fetching error reason 2023-06-01 14:00:43 -03:00
Thales Macedo Garitezi
64258120d7 fix(plugins): use emqx:running_nodes for multicall operations 
Fixes https://emqx.atlassian.net/browse/EMQX-10079
 2023-06-01 14:00:43 -03:00
某文
2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
Thales Macedo Garitezi
3e4790edd4 test(pulsar_producer): fix flaky test 2023-06-01 13:01:58 -03:00
Thales Macedo Garitezi
99796224d8 refactor(resource): rename request_timeout -> request_ttl 
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
 2023-06-01 13:01:53 -03:00
某文
c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
Stefan Strigler
9c76aa901e fix(emqx_mgmt_api_clients): hide set_keepalive_seconds from swagger 2023-06-01 16:57:19 +02:00
某文
94b86f01a5 chore: reformat SUITE case 2023-06-01 22:32:31 +08:00
zhongwencool
21df321d89
Merge branch 'master' into improve-authn 2023-06-01 22:24:59 +08:00
Thales Macedo Garitezi
f42ccb6262 feat(resource): increase default request timeout to 45 s 
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
 2023-06-01 11:20:06 -03:00
Thales Macedo Garitezi
10425eb925 feat(resource): deprecate auto_restart_interval in favor of health_check_interval 
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
 2023-06-01 11:20:06 -03:00
Stefan Strigler
e011b5532f
Merge pull request #10881 from sstrigler/EMQX-9355-5-0-x-test-coverage-80 
IoTDB test coverage
 2023-06-01 15:48:56 +02:00
Thales Macedo Garitezi
ecbb6e0ecb test(global_gc): fix flaky test 2023-06-01 10:48:02 -03:00
Kjell Winblad
75ff76a16b fix: friendly error message when creating bridges with too long names 
This commit makes the error message and log entry that appear when one
tries to create a bridge with a name the exceeds 255 bytes (the max
length for atoms) more friendly and easier to understand.

An even better fix would be to not store bridge names as atoms but this
probably requires a more substantial change.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9609
 2023-06-01 15:30:06 +02:00
Thales Macedo Garitezi
7740987a17
Merge pull request #10905 from thalesmg/fix-enforce-binary-rootkey-config-v50 
fix(config): enforce root key name to be a binary (raw config)
 2023-06-01 10:27:48 -03:00
William Yang
c1d9bc8033 perf(emqx_mqtt_caps): direct read mqtt from zone conf 2023-06-01 14:54:44 +02:00
William Yang
923913e15c feat(config): merge with global defaults when put new zone 2023-06-01 14:54:44 +02:00
William Yang
a2f8e87389 test(config): default zone and global defaults 2023-06-01 14:54:14 +02:00
William Yang
e693c6ec54 test: fix emqx_takeover_SUITE 2023-06-01 14:52:33 +02:00
William Yang
8c26ee75db test: remove hardcoded configs 
test with schema defaults
 2023-06-01 14:52:33 +02:00
William Yang
4416122cf3 test: fix longname/shortname 2023-06-01 14:52:33 +02:00
William Yang
8ee78ad6b4 test: fix emqx_mgmt_api_configs_SUITE 2023-06-01 14:52:33 +02:00
William Yang
f621bf1f8e test(config): update emqx_client_SUITE 2023-06-01 14:52:33 +02:00
William Yang
98d6ed4251 test: fix emqx_session_SUITE 2023-06-01 14:52:33 +02:00
William Yang
6c1671651c test: use zone default in emqx_eviction_agent_channel_SUITE 2023-06-01 14:52:33 +02:00
William Yang
88f9782db0 perf(config): read zone conf once 2023-06-01 14:52:30 +02:00
firest
232ef23a48 feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation 2023-06-01 18:49:45 +08:00
Zhongwen Deng
7eea693422 fix: reorder authn when updating 2023-06-01 17:24:39 +08:00
Zhongwen Deng
96ee38ce15 chore: bump up authn/ctl app version 2023-06-01 17:24:39 +08:00
Zhongwen Deng
4f5d5216bb test: bad authn handler callback module in SUITE 2023-06-01 17:24:39 +08:00
Zhongwen Deng
8f12d81878 feat: support emqx_ctl conf command 2023-06-01 17:24:39 +08:00
Zhongwen Deng
b290d2543b fix: don't crash when command_ctl table not init 2023-06-01 17:24:39 +08:00
Zaiming (Stone) Shi
feecd53436 chore: bump to version 5.1.0-alpha.2 for both ce and ee 2023-06-01 10:13:56 +02:00
Andrew Mayorov
b5f24c4f88
Merge pull request #10900 from keynslug/fix/EEC-856/parsed-bridge-id 
fix(ruleeng): parse bridge id on create / update
 2023-06-01 10:21:26 +03:00
JianBo He
bcc5e30f22
Merge pull request #10895 from lafirest/fix/refactor_on_stop 
feat: refactored some bridges to avoid leaking resources during crashes at creation
 2023-06-01 15:17:37 +08:00
Zaiming (Stone) Shi
760de7becf chore: imporve logging message 2023-06-01 08:26:53 +02:00
Zaiming (Stone) Shi
15ffa98bc5 chore: fix a typo in test case function name 2023-06-01 08:18:26 +02:00
Zaiming (Stone) Shi
cb2be1811e docs: fix function doc annotation 2023-06-01 08:17:36 +02:00
firest
cb31032e6d chore: make dialyzer && test happy 2023-06-01 11:15:51 +08:00
Thales Macedo Garitezi
f83f112189 fix(config): enforce root key name to be a binary (raw config) 
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299

```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
    exception: error
    key_path: [listeners,tcp,default,authentication]
    module: emqx_schema
    msg: change_config_crashed
    reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
                               <<"authentication">>]}
    stacktrace: [{emqx_utils_maps,deep_get,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
                      {line,49}]},
                 {emqx_config_handler,return_change_result,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,456}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,284}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,
                     {create_authenticator,'tcp:default',
                         #{backend => built_in_database,enable => true,
                           mechanism => password_based}}},
                 #{rawconf_with_defaults => true}}
```

Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:

```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
 #{<<"ssl">> =>
       #{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```

... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
 2023-05-31 18:03:58 -03:00
Paulo Zulato
3a6ad3b049 fix(oracle): require either sid or service name 
Fixes https://emqx.atlassian.net/browse/EMQX-9980
 2023-05-31 17:18:45 -03:00
Thales Macedo Garitezi
b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e 
more attempts to fix more test flakiness
 2023-05-31 16:07:18 -03:00
Zaiming (Stone) Shi
5146de5b1c feat: add a backup copies for cluster.hocon 2023-05-31 20:34:25 +02:00
Zaiming (Stone) Shi
332daabcc5 fix(config): Do not sync cluster config from nodes running new version 2023-05-31 20:34:25 +02:00
Zaiming (Stone) Shi
39d6f612ca feat(emqx_release): add new APIs to parse/compare release versions 2023-05-31 20:34:25 +02:00
Andrew Mayorov
aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard 
test(dashboard): fix inter-suite test flakiness
 2023-05-31 20:28:39 +03:00
Andrew Mayorov
f90ce9bbfe
test(dashboard): fix inter-suite test flakiness 2023-05-31 19:09:37 +03:00
Andrew Mayorov
0293b54211
fix(ft): respect checksum in fin packets 2023-05-31 18:50:38 +03:00
Thales Macedo Garitezi
189e2c87be test(channel): fix flaky test 
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```

The problem was that the whole generated message was compared for
equality when receiving the will message, but the timestamp may be
different at publishing time...
 2023-05-31 12:42:24 -03:00
JianBo He
a3a6480f00
Merge pull request #10880 from HJianBo/batch-kickout-clients 
feat: support kickout clients in batch
 2023-05-31 22:36:35 +08:00
JianBo He
8ce0132569
Merge pull request #10871 from HJianBo/fix-coap-bugs 
fix(coap): to better handle coap requests in connection mode
 2023-05-31 22:36:15 +08:00
JianBo He
c67135dd4b
Merge pull request #10598 from HJianBo/refactor_exproto_stream 
refactor(exproto): support unary handler
 2023-05-31 22:36:01 +08:00
Kjell Winblad
036f180c27
Merge pull request #10878 from kjellwinblad/kjell/bridge/rabbitmq_password_log_leak/EMQX-9976 
fix: password log leak in RabbitMQ bridge
 2023-05-31 16:17:19 +02:00
Andrew Mayorov
29fe201676
fix(ruleeng): parse bridge id on create / update 
Instead of parsing it every time in the `send_message/2` hot path.
 2023-05-31 17:08:42 +03:00
Thales Macedo Garitezi
77c9eda036 test: rm stray clear_screen calls 2023-05-31 11:07:44 -03:00
Stefan Strigler
1e4cee05df fix(emqx_bridge_iotdb): handle rule engine passed payload 
also remove topic logic as it's duplicated functionality via rule engine
 2023-05-31 15:57:22 +02:00
Thales Macedo Garitezi
83a7c800cc test(kafka_consumer): reduce flakiness 
The crash reason can also be `{error, not_found}` depending on
timing...

```
=ERROR REPORT==== 31-May-2023::12:17:31.461490 ===
    exception: error
    key_path: [bridges,kafka_consumer,
               't_resource_manager_crash_before_subscriber_started-576460752303419630']
    module: emqx_conf_schema
    msg: change_config_crashed
    reason: {badmatch,{error,not_found}}
    stacktrace: [{emqx_resource_manager,create_and_return_data,5,
                     [{file,
                          "/emqx/apps/emqx_resource/src/emqx_resource_manager.erl"},
                      {line,119}]},
                 {emqx_bridge_resource,create,4,
                     [{file,
                          "/emqx/apps/emqx_bridge/src/emqx_bridge_resource.erl"},
                      {line,171}]},
                 {emqx_bridge,'-perform_bridge_changes/2-fun-1-',5,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,367}]},
                 {maps,fold_1,3,[{file,"maps.erl"},{line,411}]},
                 {emqx_bridge,perform_bridge_changes,2,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,354}]},
                 {emqx_bridge,post_config_update,5,
                     [{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
                      {line,229}]},
                 {emqx_config_handler,call_post_config_update,7,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,398}]},
                 {emqx_config_handler,do_post_config_update,8,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,336}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,281}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,#{<<"authentication">> => <<"none">>,
                           <<"bootstrap_hosts">> =>
                               <<"toxiproxy.emqx.net:9292">>,
                           <<"connect_timeout">> => <<"5s">>,
                           <<"enable">> => true,
                           <<"kafka">> =>
                               #{<<"max_batch_bytes">> => <<"896KB">>,
                                 <<"max_rejoin_attempts">> => 5,
                                 <<"offset_commit_interval_seconds">> => 3,
                                 <<"offset_reset_policy">> => <<"latest">>},
                           <<"key_encoding_mode">> => <<"none">>,
                           <<"metadata_request_timeout">> => <<"5s">>,
                           <<"min_metadata_refresh_interval">> => <<"3s">>,
                           <<"ssl">> =>
                               #{<<"enable">> => false,
                                 <<"server_name_indication">> => <<"auto">>,
                                 <<"verify">> => <<"verify_none">>},
                           <<"topic_mapping">> =>
                               [#{<<"kafka_topic">> =>
                                      <<"t_resource_manager_crash_before_subscriber_started-576460752303419662">>,
                                  <<"mqtt_topic">> =>
                                      <<"mqtt/topic/-576460752303422911">>,
                                  <<"payload_template">> => <<"${.}">>,
                                  <<"qos">> => 0}],
                           <<"value_encoding_mode">> => <<"none">>}},
                 #{override_to => cluster}}
```
 2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi
e7b35011e6 test(oracle): attempt to fix flaky test 
https://github.com/emqx/emqx/actions/runs/5126014459/jobs/9220155908#step:7:754
```
=CRITICAL REPORT==== 30-May-2023::20:50:26.267103 ===
Run stage failed: error:{assertMatch,
                         [{module,emqx_bridge_oracle_SUITE},
                          {line,394},
                          {expression,
                           "emqx_resource : simple_sync_query ( ResourceId , { query , \"SELECT COUNT(*) FROM mqtt_test\" } )"},
                          {pattern,
                           "{ ok , [ { result_set , _ , _ , [ [ { 3 } ] ] } ] }"},
                          {value,
                           {ok,[{result_set,[<<"COUNT(*)">>],[],[[{0}]]}]}}]}
Stacktrace: [{emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-5-',1,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,394}]},
             {emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-6-',5,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,390}]},
             {emqx_bridge_oracle_SUITE,t_batch_sync_query,1,
                 [{file,
                      "/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
                  {line,364}]}]
```
 2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi
0d539e91d1 test(pulsar_producer): attempt to stabilize flaky test 
https://github.com/emqx/emqx/actions/runs/5125166433/jobs/9218613872?pr=10886#step:7:679
```
  =CRITICAL REPORT==== 30-May-2023::19:38:58.003170 ===
  Run stage failed: error:{badmatch,
                              {timeout,
                                  [#{msg => pulsar_producer_bridge_started,
                                     '~meta' =>
                                         #{gl => <97891.472.0>,
                                           location =>
                                               #Fun<emqx_bridge_pulsar_impl_producer.11.109752493>,
                                           node => 'autocluster_node1@127.0.0.1',
                                           pid => <97891.787.0>,
                                           time => -576460611692219}}]}}
  Stacktrace: [{emqx_bridge_pulsar_impl_producer_SUITE,'-t_cluster/1-fun-10-',
                   6,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1073}]},
               {emqx_bridge_pulsar_impl_producer_SUITE,t_cluster,1,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1064}]}]
```
 2023-05-31 10:19:55 -03:00
Thales Macedo Garitezi
1e25ebb64c test(gcp_pubsub): attempt to fix flakiness 
https://github.com/emqx/emqx/actions/runs/5125118728/jobs/9218520994?pr=10887#step:8:309
```
  =CRITICAL REPORT==== 30-May-2023::19:19:34.887082 ===
  "check stage" failed: error
  {assertMatch,[{module,emqx_bridge_gcp_pubsub_SUITE},
                {line,1066},
                {expression,"? of_kind ( gcp_pubsub_request_failed , Trace )"},
                {pattern,"[ # { reason := Error , connector := ResourceId } | _ ]"},
                {value,[#{connector =>
                              <<"bridge:gcp_pubsub:emqx_bridge_gcp_pubsub_SUITE0005FCEE15534E9CD4CD02004CF10000">>,
                          msg => gcp_pubsub_request_failed,query_mode => async,
                          reason => {closed,"The connection was lost."},
                          recoverable_error => true,
                          '~meta' =>
                              #{gl => <0.17903.2>,
                                location =>
                                    #Fun<emqx_bridge_gcp_pubsub_connector.19.19548918>,
                                node => 'test@127.0.0.1',pid => <0.19724.2>,
                                time => -576460610660164}}]}]}
  Stacktrace: [{emqx_bridge_gcp_pubsub_SUITE,
                   '-do_econnrefused_or_timeout_test/2-fun-2-',3,
                   [{file,
                        "/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
                    {line,1066}]},
               {emqx_bridge_gcp_pubsub_SUITE,do_econnrefused_or_timeout_test,2,
                   [{file,
                        "/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
                    {line,1022}]}]
```
 2023-05-31 10:19:55 -03:00
lafirest
502b24bbe2
Merge pull request #10877 from lafirest/fix/rocketmq_sensitive_data 
fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data
 2023-05-31 18:54:44 +08:00
Zhongwen Deng
5729ea0426 feat: api_key conf support hot conf 2023-05-31 18:23:58 +08:00
zhongwencool
26c2ab0bef
Merge pull request #10894 from zhongwencool/crl-cache-conf-hot-update 
feat: update crl_cache conf at runtime
 2023-05-31 17:59:09 +08:00
firest
ce32526c96 chore: bump version && update changes 2023-05-31 17:31:56 +08:00
Stefan Strigler
6c0fb0e2ea test: run tests against iotdb013 container 2023-05-31 11:30:13 +02:00
William Yang
2250ed7052
Merge pull request #10823 from qzhuyan/dev/william/meck-olp 
test: use meck in olp tests
 2023-05-31 11:01:56 +02:00
firest
6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Zhongwen Deng
38e57f511c feat: update crl_cache conf at runtime 2023-05-31 16:18:03 +08:00
JianBo He
837dfeb46f feat(exproto): add raw_publish function 2023-05-31 13:44:51 +08:00
JianBo He
5256d8092c fix: keep the real closed_reason 2023-05-31 13:44:51 +08:00
JianBo He
2e4ec4888d fix(exproto): ensure the on_socket_closed event delivering correctly 2023-05-31 13:44:51 +08:00
JianBo He
9dff026f50 test: fix failed test cases 2023-05-31 13:44:51 +08:00
JianBo He
b6bc3cd921 refactor(exproto): support unary handler 2023-05-31 13:44:51 +08:00
JianBo He
afcd9fcb5e chore: update bpapi.versions 2023-05-31 11:29:55 +08:00
lafirest
5968c71d0c
Merge pull request #10782 from lafirest/feat/retainer_flow_control 
feat(retainer): add a field `deliver_rate` to limit the maximum delivery rate
 2023-05-31 11:01:10 +08:00
zhongwencool
6202cefd5a
Merge pull request #10870 from zhongwencool/stop-dashboard-listener-waiting 
fix: flaky test when stopping dashboard listener
 2023-05-31 10:54:12 +08:00
zhongwencool
ec580f0454
Merge pull request #10866 from zhongwencool/add-delayed-test 
test: add test for delayed update
 2023-05-31 10:53:55 +08:00
firest
a725ff0440 chore: bump version && update changes 2023-05-31 10:18:00 +08:00
firest
a258ef32e5 chore: update retainer app version && changes 2023-05-31 09:44:25 +08:00
firest
03c7b84e89 feat(retainer): add a field deliver_rate to limit the maximum delivery rate 2023-05-31 09:44:25 +08:00
firest
6f19cc394e fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data 2023-05-31 09:44:06 +08:00
firest
79d08e4d26 fix(limiter): temporarily hide the node-level settings 2023-05-31 09:43:14 +08:00
firest
55809f801c test: fix test case error 2023-05-31 09:43:14 +08:00
firest
be2dfa51df fix(limiter): improve the default configuration 2023-05-31 09:43:14 +08:00
Zhongwen Deng
c0b15c402b test: add test for delayed update 2023-05-31 08:17:29 +08:00
Zhongwen Deng
12ef951b8e test: add test for emqx_conf:update([log], Conf) 2023-05-31 07:54:55 +08:00
Zhongwen Deng
5690469896 fix: flaky test when stopping dashboard listener 2023-05-31 07:53:10 +08:00
Thales Macedo Garitezi
a7f4f81c38
Merge pull request #10887 from thalesmg/fix-async-worker-down-buffer-worker-20230530-v50 
fix: block buffer workers so they may retry requests
 2023-05-30 17:39:18 -03:00
Thales Macedo Garitezi
299a67656e
Merge pull request #10886 from thalesmg/test-flakiness-20230530-d 
test(quic_multistreams): attempt to fix flaky tests
 2023-05-30 17:34:23 -03:00
Andrew Mayorov
a2688325e5
Merge pull request #10754 from fix/EMQX-10056/mqtt 
feat(mqttconn): employ ecpool instead of single worker
 2023-05-30 23:28:10 +03:00
Andrew Mayorov
26819a647c
fix(mqttbridge): clarify schema descriptions + log messages 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-05-30 22:21:22 +03:00
Thales Macedo Garitezi
4f1c55f3fb
Merge pull request #10885 from thalesmg/test-flakiness-20230530-c 
test(olp): fix inter-suite flakiness
 2023-05-30 16:09:17 -03:00
Thales Macedo Garitezi
596f05f981 test(channel): attempt to reduce test flakiness 
https://github.com/emqx/emqx/actions/runs/5124277038/jobs/9217362541?pr=10886#step:5:1582
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```
 2023-05-30 15:48:48 -03:00
Thales Macedo Garitezi
c43e5b8406
Merge pull request #10883 from thalesmg/test-flakiness-20230530-b 
test(exclusive_sub): fix test flakiness
 2023-05-30 15:46:14 -03:00
Thales Macedo Garitezi
8c565abc84 test(cassandra): fix flaky test 2023-05-30 15:42:53 -03:00
Thales Macedo Garitezi
6be8ff378e fix(buffer_worker): make buffer worker enter blocked state when async worker dies 
Fixes https://emqx.atlassian.net/browse/EMQX-10074

Otherwise, requests from those async workers, now retriable, might not
be retried until the buffer worker blocks for other reasons, which
might take a long time.
 2023-05-30 15:34:22 -03:00
Thales Macedo Garitezi
0f802775c3
Merge pull request #10884 from thalesmg/fix-rule-metrics-empty-map-v50 
fix(rule_engine_api): don't crash when formatting empty metrics
 2023-05-30 15:09:04 -03:00
William Yang
0d6465b016 chore: bump app vsn 2023-05-30 19:41:05 +02:00
Thales Macedo Garitezi
98a99d47cb test(shared_sub): attempt to reduce test flakiness 
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9215439237#step:8:565
```
%%% emqx_shared_sub_SUITE ==> t_local_fallback: FAILED
%%% emqx_shared_sub_SUITE ==> {{badmatch,<<"not yet?">>},
 [{emqx_shared_sub_SUITE,t_local_fallback,1,
                         [{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_shared_sub_SUITE.erl"},
                          {line,669}]},
```
 2023-05-30 14:03:14 -03:00
Thales Macedo Garitezi
fa9afcd7e4 test(quic_multistreams): attempt to fix flaky tests 
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9214950882#step:8:1757
```
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
test_server:ts_tc failed on line 1782
Reason: {test_case_failed,Client <0.6075.5> did not die: stacktrace: {current_stacktrace,
                                            [{gen_statem,loop_receive,3,
                                              [{file,"gen_statem.erl"},
                                               {line,1281}]},
                                             {proc_lib,init_p_do_apply,3,
                                              [{file,"proc_lib.erl"},
                                               {line,240}]}]}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Testing lib.emqx.emqx_quic_multistreams_SUITE: *** FAILED test case 335 of 364 ***
%%% emqx_quic_multistreams_SUITE ==> shutdown.abort_recv_shutdown.ctrl_stream_shutdown.t_multi_streams_remote_shutdown: FAILED
%%% emqx_quic_multistreams_SUITE ==> {test_case_failed,"Client <0.6075.5> did not die: stacktrace: {current_stacktrace,\n                                            [{gen_statem,loop_receive,3,\n                                              [{file,\"gen_statem.erl\"},\n                                               {line,1281}]},\n                                             {proc_lib,init_p_do_apply,3,\n                                              [{file,\"proc_lib.erl\"},\n                                               {line,240}]}]}"}
```

```
  %%% emqx_quic_multistreams_SUITE ==> mstream.profiles.profile_max_throughput.pub_qos2.sub_qos2.qos.t_multi_streams_unsub: FAILED
  %%% emqx_quic_multistreams_SUITE ==> {{case_clause,
       {ok,#{packet_id => 8,properties => #{},reason_code => 0,
             reason_code_name => success,
             via =>
                 {quic,#Ref<0.686875463.2755788802.201649>,
                     #Ref<0.686875463.2755788804.193527>}}}},
   [{emqx_quic_multistreams_SUITE,t_multi_streams_unsub,1,
        [{file,
             "/home/thales/dev/emqx/emqx/apps/emqx/test/emqx_quic_multistreams_SUITE.erl"},
         {line,1094}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
    {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1292}]},
    {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}
```
 2023-05-30 13:56:05 -03:00
Thales Macedo Garitezi
37b4a44e23 test(olp): fix inter-suite flakiness 
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9212645120?pr=10883#step:8:2538
```
   emqx_olp_SUITE:t_overload_cooldown_conn failed on line 96
  Reason: {assertMatch,[{module,...},{...}|...]}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_olp_SUITE: *** FAILED test case 3 of 4 ***
  %%% emqx_olp_SUITE ==> t_overload_cooldown_conn: FAILED
  %%% emqx_olp_SUITE ==>
  Failure/Error: ?assertMatch({ ok , _Pid }, emqtt : connect ( C ))
    expected: = { ok , _Pid }
         got: {error,econnrefused}
        line: 96
  %%% emqx_olp_SUITE ==> t_overloaded_conn: OK
  Testing lib.emqx.emqx_olp_SUITE: TEST COMPLETE, 3 ok, 1 failed of 4 test cases
```

```sh
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_olp_SUITE.erl
```
 2023-05-30 13:39:52 -03:00
William Yang
f2e54c9a6e test: use meck in olp tests 
avoid flaky test results by using mecks.
 2023-05-30 17:57:09 +02:00
Andrew Mayorov
95cc9b9b72
fix(mqttbridge): ensure elixir release assembles successfully 2023-05-30 17:56:35 +03:00
Andrew Mayorov
1c2719236c
chore(mqttbridge): add README 2023-05-30 17:56:35 +03:00
Andrew Mayorov
7e7b50c5ba
refactor(mqttbridge): move into separate application 2023-05-30 17:56:34 +03:00
Thales Macedo Garitezi
57aacb471c fix(rule_engine_api): don't crash when formatting empty metrics 
Fixes https://emqx.atlassian.net/browse/EMQX-10073
Fixes https://github.com/emqx/emqx/issues/10714#issuecomment-1567987664

Similar issue to https://github.com/emqx/emqx/pull/10743, but on the
rule engine API.
 2023-05-30 11:54:27 -03:00
Thales Macedo Garitezi
a28580248a test(exclusive_sub): fix test flakiness 
https://github.com/emqx/emqx/actions/runs/5115608275/jobs/9208594481?pr=10864#step:8:735

```
  Testing lib.emqx.emqx_access_control_SUITE: TEST COMPLETE, 4 ok, 0 failed of 4 test cases

  Testing lib.emqx.emqx_exclusive_sub_SUITE: Starting test, 5 test cases

  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  Error detected: {'EXIT',{shutdown,econnrefused}}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 1 of 5 ***
  %%% emqx_exclusive_sub_SUITE ==> t_allow_normal_sub: FAILED
  %%% emqx_exclusive_sub_SUITE ==> {'EXIT',{shutdown,econnrefused}}

  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  emqx_exclusive_sub_SUITE:t_clean_session failed on line 128
  Reason: {badmatch,{error,econnrefused}}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 2 of 5 ***
  %%% emqx_exclusive_sub_SUITE ==> t_clean_session: FAILED
```

To reproduce:

```
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_exclusive_sub_SUITE.erl
```
 2023-05-30 11:23:17 -03:00
Andrew Mayorov
ebd612b194
chore: bump applications versions 
* emqx_connector 0.1.25
* emqx_rule_engine 5.0.19
* emqx_ee_bridge 0.1.15
 2023-05-30 17:21:45 +03:00
Andrew Mayorov
eed9358abd
chore: bump ecpool to 0.5.4 
With fixed typings and empty pool handling.
 2023-05-30 17:21:45 +03:00
Andrew Mayorov
a5fc26736d
refactor(mqttconn): split ingress/egress into 2 separate pools 
Each with a more refined set of responsibilities, at the cost of slight
code duplication. Also provide two different config fields for each pool
size.
 2023-05-30 17:21:44 +03:00
Andrew Mayorov
c7528e9b35
feat(mqttconn): add pool_size config parameter 
That currently tunes the number of MQTT clients employed both for
subscriptions (if shared subscription is used) and for publishing to
a remote broker.
 2023-05-30 17:21:44 +03:00
Andrew Mayorov
6e97dffdb8
feat(mqttconn): deprecate mode config parameter 
It adds no value: the only mode was `cluster_shareload` and we just as
well can decide to "share" the load across cluster just by looking if
the remote topic is shared subcription filter or not.
 2023-05-30 17:21:44 +03:00
Thales Macedo Garitezi
cb7929a2fc test(broker): attempt to reduce flakiness 2023-05-30 11:12:18 -03:00
Thales Macedo Garitezi
26992e9d98 test(banned): attempt to improve flaky test 2023-05-30 11:08:31 -03:00
Thales Macedo Garitezi
04bd39861d chore: bump app vsns 2023-05-30 11:08:20 -03:00
Stefan Strigler
767f7c57e7 test: check device_id is used from topic or config 2023-05-30 15:55:21 +02:00
Stefan Strigler
938d62a666 test: handle bad payload 2023-05-30 15:55:21 +02:00
Stefan Strigler
860d7b169a test: check data at iotdb instance 2023-05-30 15:55:21 +02:00
Stefan Strigler
a3021c58f1 fix(emqx_bridge_iotdb): allow integer timestamp 2023-05-30 15:55:21 +02:00
Stefan Strigler
bd92116cee test: fix data in calls to look like real world 2023-05-30 15:55:21 +02:00
Stefan Strigler
1381b54a8d fix(emqx_bridge_iotdb): allow non-binary values 2023-05-30 15:55:21 +02:00
Stefan Strigler
64d582770d test: add tracepoints 2023-05-30 15:55:21 +02:00
Stefan Strigler
3d3f2a223c test: test more value conversions 2023-05-30 15:55:21 +02:00
Stefan Strigler
6f54220a51 feat(emqx_bridge_iotdb): handle bad message format gracefully 2023-05-30 15:55:20 +02:00
JianBo He
a132df5568 fix(coap): to better handle coap requests in connection mode 
Fixes for connection deletion and message publishing requests not taking effect
issues once the connection has been created in a different UDP port
first.
 2023-05-30 21:48:57 +08:00
JianBo He
48c53d8b32 chore: add emqx_management_proto_v4 2023-05-30 21:46:33 +08:00
Thales Macedo Garitezi
6b5b9fc8b8
Merge pull request #10864 from thalesmg/fix-flaky-channel-test-20230529 
test(channel): fix flaky test
 2023-05-30 10:18:47 -03:00
Thales Macedo Garitezi
5f6ff9144b
Merge pull request #10862 from thalesmg/fix-flaky-cassandra-test-v50 
test(cassandra): attempt to fix flaky test
 2023-05-30 10:02:14 -03:00
Thales Macedo Garitezi
9c3f838e14
Merge pull request #10841 from thalesmg/kafka-validate-key-v50 
feat({kafka,pulsar}_producer): add validation for empty message key when strategy = key_dispatch
 2023-05-30 09:37:15 -03:00
Kjell Winblad
a4fcc0dcb3 fix: password log leak in RabbitMQ bridge 
This fixes a vulnerability in the RabbitMQ bridge, which could
potentially expose passwords to log files. This was accomplished by
initializing the encryption library specifically designed for RabbitMQ's
passwords. Consequently, passwords are no longer stored in unencrypted
format. As a result, they will no longer be visible as plain text in log
messages, thereby enhancing the system's security.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9976
 2023-05-30 14:36:02 +02:00
JianBo He
fc806c4acd test: fix failed tests 2023-05-30 20:15:49 +08:00
Thales Macedo Garitezi
75fcac9711
Merge pull request #10826 from thalesmg/test-partial-batch-expired-inflight-v50 
test(buffer_worker): add assertion for inflight count after batch expiration
 2023-05-30 09:05:59 -03:00
Thales Macedo Garitezi
5bcc28b8df
Merge pull request #10861 from thalesmg/fix-flaky-telemetry-test-v50 
test(telemetry): fix flaky test
 2023-05-30 09:05:28 -03:00
JianBo He
1e7872c319 feat: support kickout clients in batch 2023-05-30 19:58:40 +08:00
Zaiming (Stone) Shi
0f080cda66 chore: bump emqx_rule_engine app vsn 2023-05-30 13:50:36 +02:00
Andrew Mayorov
4e6269bedb
feat(mqttconn): subscribe each worker if shared subcription 
Also rename `subscriptions` -> `ingress` and `forwards` -> `egress` for
consistency with the config schema.
 2023-05-30 14:50:05 +03:00
Andrew Mayorov
81e78516aa
feat(mqttconn): employ ecpool instead of a single worker 2023-05-30 14:50:05 +03:00
Andrew Mayorov
6967f621d8
fix(mqttconn): unify error interpretation in sync/async modes 
Also move this logic to the mqtt connector itself, in order to avoid
dealing with extra callback layer.
 2023-05-30 14:50:05 +03:00
Andrew Mayorov
67d703f8c5
refactor(mqttconn): simplify mqtt connector 
Inline `emqx_connector_mqtt_msg` module code into
`emqx_connector_mqtt_worker` module, since it's not really used
anywhere else and does not provide any reusable abstractions.
 2023-05-30 14:50:05 +03:00
Andrew Mayorov
bd956d00b6
feat(mqttconn): stop using gproc in hot path 
Also drop fiddling with `mountpoint` since this option seems not to be
used anywhere.
 2023-05-30 14:50:05 +03:00
Andrew Mayorov
4da0d83faf
chore(mqttconn): remove dead code 2023-05-30 14:50:04 +03:00
Zaiming (Stone) Shi
aa096f496f
Merge pull request #10876 from zmstone/merge-release-51-back-to-master 
Merge release 51 back to master
 2023-05-30 13:49:21 +02:00
Kjell Winblad
5cfe04956d
Merge pull request #10858 from kjellwinblad/kjell/rule_engine_funcs/add_timezone_to_second/EMQX-10058 
feat: add timezone_to_second/1 function to rule engine
 2023-05-30 12:29:40 +02:00
Zaiming (Stone) Shi
d2ce7c22fd chore: bump app vsn, core app 'emqx' is now at 5.1 2023-05-30 11:32:18 +02:00
Zaiming (Stone) Shi
1e8ed89db8 Merge remote-tracking branch 'origin/master' into release-51 2023-05-30 11:29:44 +02:00
Zaiming (Stone) Shi
91cdc69976
Merge pull request #10867 from zmstone/0530-merge-release-50-to-master 
0530 merge release 50 to master
 2023-05-30 09:54:57 +02:00
Zaiming (Stone) Shi
367de2a2eb docs: update bpapi readme 2023-05-30 08:29:43 +02:00
Zaiming (Stone) Shi
941939f7d7 build: finalize 5.0.bpapi 2023-05-30 08:28:45 +02:00
Zaiming (Stone) Shi
3b8cba6ba0 chore: bump rlease versions 2023-05-30 08:28:45 +02:00
Zaiming (Stone) Shi
9529919046 chore: bump app versions 2023-05-30 08:08:29 +02:00
JimMoen
8569fac6eb
fix: make static_check happy 2023-05-30 14:00:20 +08:00
Zaiming (Stone) Shi
747eb6fcff Merge remote-tracking branches 'origin/release-50' and 'origin/master' into release-51 2023-05-30 07:54:34 +02:00
Zaiming (Stone) Shi
286eefb563 Merge remote-tracking branch 'origin/release-50' 2023-05-30 07:52:54 +02:00
JimMoen
909d348570
chore: fix spec typo 2023-05-30 11:31:37 +08:00
JimMoen
bb363e9b9e
chore: bump dependency vsn 
* bump `snappyer` ~> 1.2.9
  - as dependency of `kafka_protocol` ~> 4.1.3
  - as dependency of `brod`

* bump `rocksdb` ~> 1.7.2-emqx-11
 2023-05-30 11:11:30 +08:00
Thales Macedo Garitezi
1aff46b7c9 test(channel): fix flaky test 2023-05-29 19:04:48 -03:00
Thales Macedo Garitezi
bae00c909f test(cassandra): attempt to fix flaky test 
```
=ERROR REPORT==== 29-May-2023::20:35:20.457796 ===
    id: <<"bridge:cassandra:emqx_bridge_cassandra_SUITE">>
    msg: unrecoverable_error
    reason: {8704,<<"Expected 8 or 0 byte long for date (4)">>}
=CRITICAL REPORT==== 29-May-2023::20:35:20.458850 ===
"check stage" failed: error
{assertMatch,[{module,emqx_bridge_cassandra_SUITE},
              {line,619},
              {expression,"Trace"},
              {pattern,"[ # { result := { ok , _Pid } } ]"},
              {value,[]}]}
Stacktrace: [{emqx_bridge_cassandra_SUITE,'-t_missing_data/1-fun-3-',1,
                 [{file,
                      "/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
                  {line,619}]},
             {emqx_bridge_cassandra_SUITE,t_missing_data,1,
                 [{file,
                      "/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
                  {line,607}]}]

=CRITICAL REPORT==== 29-May-2023::20:35:20.471872 ===
Check stage failed. Trace dump: "/emqx/_build/test/logs/ct_run.test@127.0.0.1.2023-05-29_20.34.32/snabbkaffe/1685392520460040358.log"

%%% emqx_bridge_cassandra_SUITE ==> tcp.sync.with_batch.t_missing_data: FAILED
%%% emqx_bridge_cassandra_SUITE ==> {{panic,#{msg => "Unexpected result",result => {error,check_stage_failed}}},
 [{emqx_bridge_cassandra_SUITE,t_missing_data,1,
                               [{file,"/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
                                {line,624}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1291}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1223}]}]}
```
 2023-05-29 18:19:26 -03:00
Thales Macedo Garitezi
fa508a95c1 test(telemetry): fix flaky test 
```
%%% emqx_telemetry_SUITE ==>
Failure/Error: ?assertEqual(0, get_value ( num_clients , TelemetryData1 ))
  expected: 0
       got: 1
      line: 454
```
 2023-05-29 18:09:35 -03:00
Thales Macedo Garitezi
67e182e0c9
Merge pull request #10813 from thalesmg/refactor-kafka-on-stop-v50 
feat(kafka): ensure allocated resources are removed on failures
 2023-05-29 16:49:29 -03:00
Kjell Winblad
354603da01 test: helper function was interpreted as a test function 2023-05-29 17:20:37 +02:00
Thales Macedo Garitezi
3edbad9f56 feat(pulsar_producer): add validation for empty message key when strategy = key_dispatch 2023-05-29 10:04:19 -03:00
Kjell Winblad
995025b572 feat: add timezone_to_second/1 function to rule engine 
This commit adds the functions timezone_to_offset_seconds and its alias
timezone_to_second to the rule engine. As the names suggests, these
functions convert a timzone offset string such as "+02:00", "Z", "local"
to an integer representing how many seconds that the given timezone
differs from UTC. timezone_to_offset_seconds is the one of the two
functions that should be advertised while timezone_to_second in kept for
backwards compatibility with 4.X.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10058
 2023-05-29 14:30:43 +02:00
Zaiming (Stone) Shi
08c1d5e5ae test: stop testing i18n.conf 2023-05-29 14:22:23 +02:00
Zaiming (Stone) Shi
d014b80834 build: download zh translation from emqx-i18n.git 2023-05-29 14:22:23 +02:00
Thales Macedo Garitezi
79f6861499 chore: prepare to tag v5.0.26 2023-05-29 09:17:59 -03:00
Thales Macedo Garitezi
639feefab6 Merge remote-tracking branch 'origin/master' into tag-v5026 2023-05-29 09:15:48 -03:00
JimMoen
9526d080af
fix: override emqx_license is unnecessary 
* continue #10820
 2023-05-29 19:10:31 +08:00
JimMoen
adae1aec96
chore: fix typo for enterprise readme 2023-05-29 19:08:15 +08:00
JimMoen
6551eb21f2
Merge pull request #10851 from JimMoen/fix-bad-api-logging 
fix: redact api request to hide auth token
 2023-05-29 17:40:44 +08:00
JimMoen
e287cf9a09
Merge pull request #10820 from JimMoen/fix-license-checker-update 
fix: reboot `emqx_license` app for join cluster
 2023-05-29 17:40:29 +08:00
Andrew Mayorov
daa7eebc0f
Merge pull request #10843 from keynslug/fix/ft-prop-zerodiv 
test(ft): ensure no division by zero in coverage proptest
 2023-05-29 10:23:52 +03:00
zhongwencool
2b6d3bb748
Merge pull request #10833 from zhongwencool/telemtry-report 
fix: only report enable authn/authz in telemetry
 2023-05-29 15:14:36 +08:00
zhongwencool
1e986a7b93
Merge pull request #10847 from zhongwencool/telemetry-conf-update 
feat: support emqx_conf:update([telemetry],Conf).
 2023-05-29 15:14:16 +08:00
Zaiming (Stone) Shi
e6869a2904 chore: bump app vsn for emqx_machine 2023-05-29 09:04:17 +02:00
Zaiming (Stone) Shi
8e6dc29637 chore: bump release version to v5.0.26-alpha.1 2023-05-29 08:57:28 +02:00
Zaiming (Stone) Shi
25319c3151
Merge pull request #10839 from zmstone/0526-merge-release-50-to-master 
0526 merge release 50 to master
 2023-05-29 08:56:20 +02:00
Zhongwen Deng
8c814bc861 fix: only report enable authn/authz in telemetry 2023-05-29 14:17:46 +08:00
Zhongwen Deng
4dee1a2429 test: add more test for emqx_telemetry 2023-05-29 14:08:59 +08:00
JimMoen
6f90369429
fix: make dialyzer happy 2023-05-29 13:48:00 +08:00
JimMoen
cb05406a00
fix: reboot emqx_license app for join cluster 2023-05-29 13:47:46 +08:00
JimMoen
dcfe985ee9
fix: redact api request to hide auth token 2023-05-29 11:02:02 +08:00
某文
8ccee19aa5 feat: support emqx_conf:update([telemetry],Conf). 2023-05-28 22:54:35 +08:00
Zaiming (Stone) Shi
c2450d230d
Merge pull request #10605 from zmstone/0503-no-telemetry-app-for-ee 
0503 refactoring: no telemetry app for ee
 2023-05-27 10:57:15 +02:00
Thales Macedo Garitezi
d27f593309 test: fix flaky authn test 2023-05-26 16:44:00 -03:00
Andrew Mayorov
98ce7e8bbd
test(ft): ensure no division by zero in coverage proptest 2023-05-26 21:53:01 +03:00
Thales Macedo Garitezi
9b7e473cf6 feat(kafka_producer): add validation for empty message key when strategy = key_dispatch 
Fixes https://emqx.atlassian.net/browse/EMQX-9979
 2023-05-26 12:02:58 -03:00
Zaiming (Stone) Shi
fe81e9521a
Merge pull request #10600 from zmstone/0504-delete-statsd-all-together 
0504 delete statsd all together
 2023-05-26 16:29:59 +02:00
Zaiming (Stone) Shi
36e268c933 chore: bump app versions 2023-05-26 16:05:37 +02:00
Zaiming (Stone) Shi
cc5b4d3748 Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test 2023-05-26 15:58:16 +02:00
Zaiming (Stone) Shi
3595a86724 chore(emqx_telemetry): add registered process names to app info 2023-05-26 15:28:06 +02:00
Zaiming (Stone) Shi
d167d6dd41 chore(emqx_telemetry): update supervisor's child restart strategy 2023-05-26 15:28:06 +02:00
Zaiming (Stone) Shi
256e92f4e9 refactor: call emqx_telemetry_config:on_server_stop 2023-05-26 15:28:06 +02:00
Zaiming (Stone) Shi
6e2cde8224 refactor: delete emqx_statsd all together 2023-05-26 15:17:42 +02:00
Andrew Mayorov
45b128946a
Merge pull request #10825 from fix/EMQX-9991/bugs 
fix(ft): avoid leaking secrets
 2023-05-26 16:08:14 +03:00
Zaiming (Stone) Shi
7eb22e27c4 chore: bump release version to e5.0.4 2023-05-26 14:34:01 +02:00
Zhongwen Deng
9b6da3ff10 chore: bump hocon to 0.39.7 2023-05-26 19:00:58 +08:00
Zhongwen Deng
df4a0afdf5 fix: zones api default value 2023-05-26 18:59:38 +08:00
Zaiming (Stone) Shi
2d986140df chore: bump version to e5.0.4-rc.2 2023-05-26 12:01:10 +02:00
Andrew Mayorov
48858dee33
fix(s3): wrap S3 secrets during config load 2023-05-26 12:59:21 +03:00
Andrew Mayorov
3d428a2e0e
fix(ft-s3): rely on asynchronous upload abort 
So that upload aborts would not block assemblers. This should not
affect the expected behavior since S3 API usually allows having
concurrent uploads, plus we couldn't really guarantee successful aborts
anyway.
 2023-05-26 12:59:20 +03:00
Andrew Mayorov
cc3275b389
fix(ft): use conservative timeouts for S3 uploader calls 2023-05-26 12:59:20 +03:00
Andrew Mayorov
d36c2c2928
fix(ft-gc): ensure GC of already complete transfers 2023-05-26 12:59:20 +03:00
Andrew Mayorov
75cf562c90
fix(ft): tune some logging levels 2023-05-26 12:59:20 +03:00
Andrew Mayorov
3278158c66
fix(ft): disable ipv6_probe by default for S3 clients 
It's often demonstrates degraded performance on common setups, e.g.
containers with virtualized network interfaces.
 2023-05-26 12:59:20 +03:00
Zaiming (Stone) Shi
17f6c58f7b fix(telemetry): no default value for telemetry.enable in schema 
the default value is computed from version string
 2023-05-26 11:58:58 +02:00
Zaiming (Stone) Shi
dfc11e57d3 chore: delete telemetry.enable from enterprise.conf 
telemetry is now not included in enterprise edition
 2023-05-26 11:58:58 +02:00
Zaiming (Stone) Shi
ee122155f1 docs: fix spellcheck for telemetry app 2023-05-26 08:44:19 +02:00
Zaiming (Stone) Shi
772a4575ca
Merge pull request #10821 from thalesmg/fix-webhook-bridge-req-timeout-r50 
fix(webhook): keep `resource_opts.request_timeout` for webhook bridge (r5.0)
 2023-05-26 08:39:00 +02:00
JimMoen
a3b77563ba
fix: make static check happy 2023-05-26 10:34:15 +08:00
JimMoen
1ba65539c6
chore: refine traces command usage 2023-05-26 09:48:56 +08:00
JimMoen
0788f2f197
fix: emqx_ctl traces ... command args 
* use second time as trace `start_at` and `end_at`
* default trace time duration 30min to same as api created
* prevent format line broken
 2023-05-26 09:48:56 +08:00
Zaiming (Stone) Shi
407cab220b
Merge pull request #10760 from paulozulato/fix-failure-joining 
fix: avoid error 500 when node is re-joining cluster
 2023-05-26 03:08:17 +02:00
Thales Macedo Garitezi
32e6213ce3 fix(resource_manager_sup): use one_for_one instead of simple_one_for_one 
Using `simple_one_for_one` has a potential race condition issue where
we read the PID of the resource manager before trying to remove a
resource, and then that PID changes because it was either dead at
first, or it crashed and changed, and later we use this stale PID to
try to remove it from the supervisor.  Under such circumstances, the
restarting child might linger in the supervisor, leaking resources.

By using the resource ID itself as a child ID (and using `one_for_one`
restart strategy), we ensure the child is truly removed.
 2023-05-25 18:07:43 -03:00
Thales Macedo Garitezi
e43517188f fix(bridge): remove bridge if exception happens during creation 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi
cb34bc5c46 test(kafka_consumer): attempt to stabilize cluster tests 
Example failure:
https://github.com/emqx/emqx/actions/runs/5070096710/jobs/9105822319#step:7:515

The attempt here is to setup the spy as early as possible, before the
bridge starts, so we avoid missing rebalancing events.
 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi
0ca3f51503 fix(kafka): improve shutdown and health check logs during shutdown 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi
5df7314255 feat(kafka): ensure allocated resources are removed on failures 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi
42b37690c7 refactor(pulsar): use macros for allocatable resources 2023-05-25 16:38:09 -03:00
Thales Macedo Garitezi
db60dcbada test(buffer_worker): add assertion for inflight count after batch expiration 
Fixes https://emqx.atlassian.net/browse/EMQX-9829
 2023-05-25 16:11:37 -03:00
Paulo Zulato
ea86f4442b fix: avoid error 500 when node is re-joining cluster 
Fixes https://emqx.atlassian.net/browse/EMQX-9899
 2023-05-25 13:32:28 -03:00
Thales Macedo Garitezi
194faefc8a fix(webhook): keep resource_opts.request_timeout for webhook bridge (r5.0) 
Fixes https://emqx.atlassian.net/browse/EMQX-10001

Recently, we unified request_timeout in a single field located at the
webhook connector schema.  However, the correct fix would be to use
the resource_opts.request_timeout one, as that’s the only one that
allows infinity timeout.
 2023-05-25 13:19:27 -03:00
Zaiming (Stone) Shi
e8a4df96bc Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-25 16:43:20 +02:00
Thales Macedo Garitezi
18d57ba3eb
Merge pull request #10812 from thalesmg/test-flakiness-20230524 
test: attempts to reduce flakiness (pgsql, cassandra)
 2023-05-25 09:29:13 -03:00
JianBo He
933357483f
Merge pull request #10817 from HJianBo/fix-restart-timer-interval 
fix: fix auto_restart_interval checker
 2023-05-25 20:23:53 +08:00
JianBo He
6b5ff24952 test: assert the block_until result 2023-05-25 18:12:38 +08:00
JianBo He
fdec86e29c
Merge pull request #10802 from Kinplemelon/kinple/set-base-url-required 
fix: set the base_url in IoTDB bridge as a required field
 2023-05-25 18:12:05 +08:00
JianBo He
de7f1c8aec test: add tests for auto_restart_interval 2023-05-25 17:15:19 +08:00
Zaiming (Stone) Shi
ba41c23b7c
Merge pull request #10794 from paulozulato/fix-oracle-database-field 
Fix oracle database field
 2023-05-25 08:14:13 +02:00
JianBo He
71b636e321 fix: fix auto_restart_interval checker 2023-05-25 12:04:23 +08:00
Paulo Zulato
1f36726cab
Merge pull request #10780 from paulozulato/fix-large-health-check-interval 
Add user-friendly message when interval is out of range
 2023-05-24 18:41:42 -03:00
Thales Macedo Garitezi
658160f09a test(cassandra): attempt to reduce flakiness 2023-05-24 16:57:51 -03:00
Thales Macedo Garitezi
37061b484a
Merge pull request #10778 from thalesmg/refactor-pulsar-on-stop-v50 
feat(pulsar): ensure allocated resources are removed on failures (v5.0)
 2023-05-24 16:00:50 -03:00
Thales Macedo Garitezi
cc8cebe950
Merge pull request #10811 from thalesmg/fix-flaky-kconsumer-balance-test-r50 
test(kafka_consumer): attempt to stabilize cluster tests
 2023-05-24 15:56:28 -03:00
Thales Macedo Garitezi
3cb12d7603
Merge pull request #10810 from thalesmg/merge-r50-into-v50-20230524 
Merge `release-50` into `master`
 2023-05-24 15:55:09 -03:00
Paulo Zulato
122ebcac24 fix: add user-friendly message when interval is out of range 2023-05-24 15:46:00 -03:00
Thales Macedo Garitezi
7f88521836 test(pgsql): reduce flakiness 
Depending on timing, `t_write_timeout` was getting stuck while
checking the resource health, and the previous request timeout options
were making a response to never be sent if that process took too long.
 2023-05-24 15:41:25 -03:00
Paulo Zulato
8d3d3e523b fix(oracle): replace 'database' field by 'service name' 
Fixes https://emqx.atlassian.net/browse/EMQX-9980
 2023-05-24 15:40:56 -03:00
Paulo Zulato
50ff6c4aa4
Merge pull request #10741 from paulozulato/fix-oracle-passwd-leaking 
fix(oracle): hide password on stacktrace
 2023-05-24 15:31:00 -03:00
Kjell Winblad
0e873731a1
Merge pull request #10804 from kjellwinblad/kjell/fix/iotdb_rest_api_v1/EMQX-9920 
fix: REST API version 2 not available in IoTDB 0.13 and 1.0
 2023-05-24 18:34:45 +02:00
Thales Macedo Garitezi
7374e00a02 test(kafka_consumer): attempt to stabilize cluster tests 
Example failure:
https://github.com/emqx/emqx/actions/runs/5070096710/jobs/9105822319#step:7:515

The attempt here is to setup the spy as early as possible, before the
bridge starts, so we avoid missing rebalancing events.
 2023-05-24 13:13:54 -03:00
ieQu1
2b79196a03
Merge pull request #10809 from ieQu1/ekka-0.15.2 
chore(ekka): Bump version to 0.15.2
 2023-05-24 18:13:04 +02:00
Thales Macedo Garitezi
324459990f Merge branch 'release-50' into merge-r50-into-v50-20230524 2023-05-24 12:54:15 -03:00
Thales Macedo Garitezi
ad93af2853
Merge pull request #10808 from thalesmg/fix-ehttpc-infinity-timeout-r50 
fix: handle `infinity` timeout option in `ehttpc` (r5.0)
 2023-05-24 12:53:05 -03:00
Thales Macedo Garitezi
fd2940cd77 feat(pulsar): ensure allocated resources are removed on failures (v5.0) 
Fixes https://emqx.atlassian.net/browse/EMQX-9937
 2023-05-24 12:29:00 -03:00
zhongwencool
14a6b36899
Merge pull request #10803 from JimMoen/merge-release-50 
Merge release-50 into master
 2023-05-24 23:08:46 +08:00
zhongwencool
96e4b5914a
Merge pull request #10797 from zhongwencool/bump-minirest 
chore: upgrade minirest to 1.3.10 for more clear error msg
 2023-05-24 22:41:42 +08:00
Thales Macedo Garitezi
a172a6bc33
Merge pull request #10795 from thalesmg/fix-schema-registry-rule-iolist-r50 
fix(schema_registry): ensure `schema_encode` output in rule engine is a binary (r5.0)
 2023-05-24 11:02:05 -03:00
ieQu1
3d3dacfcf6 chore(ekka): Bump version to 0.15.2 2023-05-24 15:43:49 +02:00
Thales Macedo Garitezi
4565acc600 fix: handle infinity timeout option in ehttpc (r5.0) 
Fixes https://emqx.atlassian.net/browse/EMQX-9987
 2023-05-24 10:33:54 -03:00
Andrew Mayorov
3fd2887921
Merge pull request #10793 from fix/EMQX-9965/bugs 
fix(ft): handle wider class of bad input in APIs
 2023-05-24 16:14:11 +03:00
某文
8816da41a6 test: log test failed 2023-05-24 21:06:15 +08:00
Thales Macedo Garitezi
6c414ab991 fix(schema_registry): ensure schema_encode output in rule engine is a binary (r5.0) 
Fixes https://emqx.atlassian.net/browse/EMQX-9981
 2023-05-24 09:13:38 -03:00
JimMoen
28015597ee
Merge remote-tracking branch 'emqx/release-50' into merge-release-50 2023-05-24 19:34:12 +08:00
Kjell Winblad
4f21bdb885 fix: REST API version 2 not available in IoTDB 0.13 and 1.0 
This commit makes sure that REST API version 1 is used when the
user has selected to use IoTDB 0.13 or 1.0

Fixes:
https://emqx.atlassian.net/browse/EMQX-9920
 2023-05-24 13:09:47 +02:00
JimMoen
cc2bb87b82
Merge pull request #10801 from JimMoen/fix-topic-lookup-percent-decode 
fix: lookup topic without force percent decode
 2023-05-24 18:51:16 +08:00
Kinplemelon
7cad1cb93a fix: set the base_url in IoTDB bridge 
as a required field
 2023-05-24 18:18:32 +08:00
JimMoen
1ac140312d
test: lookup topic with percent encoded topic name 2023-05-24 17:39:25 +08:00
JimMoen
e38645aa28
fix: lookup topic without force percent decode 
* the minirest handler would do it
 2023-05-24 17:28:32 +08:00
Zaiming (Stone) Shi
2fdf4b5dac
Merge pull request #10744 from savonarola/0518-fix-eviction 
fix(evacuation): handle expire interval correctly
 2023-05-24 09:20:53 +02:00
Zaiming (Stone) Shi
256cb73e43
Merge pull request #10792 from thalesmg/fix-webhook-bridge-health-check-clause-r50 
fix(webhook): fix empty ehttpc worker list clause on health check (r5.0)
 2023-05-24 08:46:59 +02:00
Zaiming (Stone) Shi
9363ca5846
Merge pull request #10791 from thalesmg/fix-bridge-infinity-timeout-r50 
fix(bridge): pass resource option `request_timeout = infinity` along to buffer workers (r5.0)
 2023-05-24 08:46:43 +02:00
某文
7da0860d6c chore: upgrade minirest to 1.3.10 for more clear error msg 2023-05-24 10:44:52 +08:00
JianBo He
0d3de36f4d
Merge pull request #10770 from kjellwinblad/kjell/fix/iotdb_need_to_parse_input_bug/EMQX-9854 
fix: IoTDB bridge incoming payload needs to be parsed as JSON
 2023-05-24 10:17:23 +08:00
Thales Macedo Garitezi
8e6da40a31 test: fix flaky ocsp test setup (v5.0) 2023-05-23 17:52:02 -03:00
Zaiming (Stone) Shi
2de2a979f5
Merge pull request #10779 from zmstone/0522-remove-log-file-paht-validator 
chore: allow special chars in log dir
 2023-05-23 22:42:12 +02:00
Zaiming (Stone) Shi
b8b1779627 Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-23 21:42:07 +02:00
Zaiming (Stone) Shi
f450f51e2c
Merge pull request #10789 from kjellwinblad/kjell/bridge/rabbitmq_password_not_required_filed/EMQX-9974 
fix: RabbitMQ field not marked as required
 2023-05-23 20:59:49 +02:00
Thales Macedo Garitezi
44ed6a19ee fix(bridge): pass resource option request_timeout = infinity along to buffer workers (r5.0) 2023-05-23 14:50:43 -03:00
Thales Macedo Garitezi
b5f63f4151 test: fix wrong test listeners option 2023-05-23 14:50:43 -03:00
Andrew Mayorov
f4047d3946
test(ft): add testcase for nasty tranfer filenames 
* Emoji
* Chinese characters
 2023-05-23 20:46:47 +03:00
Thales Macedo Garitezi
25437adf02 fix(webhook): fix empty ehttpc worker list clause on health check (r5.0) 
Fixes https://emqx.atlassian.net/browse/EMQX-9970
 2023-05-23 14:06:40 -03:00
Andrew Mayorov
2dbf84479c
fix(ft): handle wider class of jiffy decode errors 
With malformed cursors in the File listing API.

Fixes EMQX-9965
 2023-05-23 20:00:00 +03:00
Andrew Mayorov
cb14a3e08b
fix(ft): handle empty filepath in fs exporter API 
Fixes EMQX-9973
 2023-05-23 19:58:05 +03:00
Kjell Winblad
63180c87be
style: simplify code for better readability 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-05-23 18:53:14 +02:00
Zaiming (Stone) Shi
4ada938b82
Merge pull request #10757 from thalesmg/perf-pulsar-producer-status-check-r50 
feat(pulsar): improve authn error check time and add connect timeout
 2023-05-23 16:37:19 +02:00
Kjell Winblad
ea2fea77d7 fix: RabbitMQ field not marked as required 
This commit makes sure that the RabbitMQ password filed is marked as
required. This ensures that the user provides a password and that the
bridge does not throw a function clause exception if the password filed
is not set.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9974
 2023-05-23 15:50:18 +02:00
Zaiming (Stone) Shi
47a3096776 test: add test case for file path validation 2023-05-23 14:22:50 +02:00
Zaiming (Stone) Shi
2fa5b511bf chore: hide stale config 2023-05-23 14:20:49 +02:00
Zaiming (Stone) Shi
91150e6d83 chore: allow special chars in log dir 
the current valdiator does not allow space and ':' in log file path
which is an unresonable restriction
 2023-05-23 14:20:49 +02:00
Zaiming (Stone) Shi
e78ffb2dd4 chore: bump apps version 2023-05-23 13:04:28 +02:00
Zaiming (Stone) Shi
3bb1f7ab2b Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-23 13:03:55 +02:00
Kjell Winblad
89ea1646be fix: IoTDB name for version option 
The previous name for the version option was v1.0.x which is clearly
wrong since this option was tested against IoTDB version v1.1.0. This
commit fixes this by renaming the option to v1.x.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9926
 2023-05-23 10:57:05 +02:00
Kjell Winblad
d19ddb1832 fix: IoTDB bridge incoming payload needs to be parsed as JSON 
There was an incorrect assumption that the data incoming to the IoTDB
bridge has already been parsed. This is fixed by parsing the payload as
JSON data if the payload is not already a map.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9854
 2023-05-23 10:14:45 +02:00
zhongwencool
3b6295a085
Merge pull request #10781 from zhongwencool/fix-bridge-test-failed 
test: bridge test init dashboard listener failed
 2023-05-23 11:16:57 +08:00
某文
1deb3c50c8 test: bridge test init dashboard listener failed 2023-05-23 10:28:01 +08:00
某文
67a30ee976 test: bridge test init dashboard listener failed 2023-05-23 10:16:43 +08:00
某文
ed1268ed30 Merge branch 'release-50' into 0522-merge-release-50-to-master 2023-05-23 07:56:35 +08:00
zhongwencool
26e31137c7
Merge pull request #10764 from HJianBo/remove-unused-comments 
chore: remove unused conf comments
 2023-05-23 07:53:25 +08:00
Zaiming (Stone) Shi
68f4ae70b2 chore: bump ee release version to e5.0.4-alpha.2 2023-05-22 22:41:19 +02:00
Zaiming (Stone) Shi
732a7be187 Merge remote-tracking branch 'origin/release-50' 2023-05-22 17:46:54 +02:00
Zaiming (Stone) Shi
1f44dd4579
Merge pull request #10756 from zmstone/0519-refactor-move-lib-ee-emqx_ee_conf-to-apps-emqx_enterprise 
0519 refactor move lib ee emqx ee conf to apps emqx enterprise
 2023-05-22 16:56:51 +02:00
Thales Macedo Garitezi
1c4c7fad92
Merge pull request #10740 from thalesmg/perf-buffer-worker-periodic-metrics-v50 
perf(buffer_worker): update metrics periodically rather than immediately (v5.0)
 2023-05-22 11:41:41 -03:00
Thales Macedo Garitezi
0877e4296a fix(dashboard): add missing function clause 
Example error:
https://github.com/emqx/emqx/actions/runs/5045715277/jobs/9052482682#step:8:294

The previous change was not equivalent to the previous `maps:with/2`
behavior.
 2023-05-22 11:35:04 -03:00
Thales Macedo Garitezi
65f973044f feat(pulsar): improve authn error check time and add connect timeout 
Fixes https://emqx.atlassian.net/browse/EMQX-9910
 2023-05-22 11:33:16 -03:00
Andrew Mayorov
3f882ddcc6
Merge pull request #10751 from ci/ct-flap-listener-api 
fix(listen): wait until port is free when stopping ranch listeners
 2023-05-22 17:00:45 +03:00
lafirest
88e4078885
Merge pull request #10747 from lafirest/fix/rule_funs_time_offset 
fix: port the `emqx_calendar` from v4.4
 2023-05-22 20:52:21 +08:00
Zaiming (Stone) Shi
0b9a32f49a chore: add BSL in apps/emqx_enterprise 2023-05-22 14:51:27 +02:00
Zaiming (Stone) Shi
40e8d5d039 refactor: rename lib-ee/emqx_ee_conf to apps/emqx_enterprise 2023-05-22 14:51:27 +02:00
Zaiming (Stone) Shi
3482c2c849 refactor: redefine default value for 'node.applications' in enterprise 2023-05-22 14:51:26 +02:00
lafirest
42f9c9aa9a
Merge pull request #10746 from lafirest/fix/events_api 
fix: supports test the `$events/delivery_dropped` event by API
 2023-05-22 20:50:03 +08:00
Andrew Mayorov
d22541e8b3
fix(ft): correct mistyped option in README 2023-05-22 15:32:08 +03:00
Thales Macedo Garitezi
0559d6f639 refactor(buffer_worker): use static fn for bumping counters 2023-05-22 09:12:08 -03:00
Thales Macedo Garitezi
9c71f4ecbd test: fix flaky test 2023-05-22 09:12:00 -03:00
Thales Macedo Garitezi
c74c93388e refactor: rename some variables and sum type constructors for clarity 2023-05-22 09:11:23 -03:00
Thales Macedo Garitezi
7d798c10e9 perf(buffer_worker): flush metrics periodically inside buffer worker process 
Fixes https://emqx.atlassian.net/browse/EMQX-9905

Since calling `telemetry` is costly in a hot path, we instead collect
metrics inside the buffer workers state and periodically flush them,
rather than immediately as events happen.
 2023-05-22 09:11:23 -03:00
Andrew Mayorov
092e112d62
Merge pull request #10767 from ci/ft-prop-flaky 
test(ft): make proptest less aggressive
 2023-05-22 12:45:24 +03:00
Andrew Mayorov
23542d1262
fix(listen): wait until port is free when stopping ranch listeners 
Simple `cowboy:stop_listener/1` will not close the listening socket
explicitly, it was the source of test flaps before this fix.
 2023-05-22 12:29:40 +03:00
Zhongwen Deng
cb5a596c57 fix: make sure swagger.json is fully generated 2023-05-22 16:58:13 +08:00
Andrew Mayorov
5b312a415b
Merge pull request #10759 from ci/ct-flap-clickhouse 
fix(clickhouse): start app in tests
 2023-05-22 11:11:15 +03:00
Andrew Mayorov
966f748551
test(ft): make proptest less aggressive 
So that the chance of getting huge coverage and as a result, high
chance of running out of memory, is reduced.
 2023-05-22 10:45:27 +03:00
William Yang
e824e1db39
Merge pull request #10698 from qzhuyan/perf/william/force-atom-conf-path 
perf(config): enforcing atom key  path in hotcode path
 2023-05-22 09:38:09 +02:00
firest
087dc59115 test: add test cases for the rule_test API 2023-05-22 15:25:35 +08:00
Zhongwen Deng
218fc4a839 refactor: add emqx_authz_file validate function 2023-05-22 14:54:23 +08:00
某文
082214d039 feat: add authz file rule validator 2023-05-22 14:25:05 +08:00
某文
27fca0ef3c fix: check authz's file rule before save to file 2023-05-22 14:22:20 +08:00
zhongwencool
c65e3a46b1
Merge pull request #10742 from zhongwencool/fix-authz-file-rule-check 
fix: check authz's file source rules in pre_config_update
 2023-05-22 14:15:18 +08:00
JianBo He
ce3ec351c7 chore: remove unused conf comments 2023-05-22 14:01:49 +08:00
Zhongwen Deng
96e7005de8 refactor: add emqx_authz_file validate function 2023-05-22 11:03:23 +08:00
firest
56a6b699ac fix: port the emqx_calendar from v4.4 2023-05-22 09:55:31 +08:00
某文
7c2aac64bc fix: bad cert file path in dashboard https listener 2023-05-21 07:41:46 +08:00
Zaiming (Stone) Shi
3e98b3b050
Merge pull request #10755 from zmstone/0517-fix-bridge-update-timeout-issue 
0517 fix bridge update timeout issue
 2023-05-20 06:19:26 +02:00
Andrew Mayorov
ba6b208df2
fix(clickhouse): start app in tests 
Otherwise, depending on the test execution order, tests might
sometimes fail.

Moreover, ensure that applications describe their dependecies
correctly and avoid starting irrelevant apps in tests.
 2023-05-19 23:08:40 +03:00
Thales Macedo Garitezi
bb4fef3cfd
Merge pull request #10752 from thalesmg/fix-pulsar-redact-auth-r50 
fix(pulsar): redact error reason
 2023-05-19 16:47:05 -03:00
Zaiming (Stone) Shi
cb76e5a241 docs: add changelog for 10755 2023-05-19 20:41:26 +02:00
Zaiming (Stone) Shi
0d8ffc0d59 fix(resource-manager): ensure no false creation 
Update is implemented as remove + create.
If a dleete call is made while the create is in progress
the remove call is likely to timeout too.

This causes the follwing creation to falsely succeed,
because there is alreay a running child under the supervisor.

As a result, the resource is permanently removed after
resource_manager eventually handles the remove call.
 2023-05-19 18:55:16 +02:00
Zaiming (Stone) Shi
f5e5c59763 refactor(resource-manager-sup): do not force kill resource manager 
the shutdown timeout is now set to infinity so it will never
force kill a resource manager, otherwise there will be
resource leaks
 2023-05-19 18:55:16 +02:00