yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,271 Commits 393 Branches 684 Tags 74 MiB
7dade3a52c
Commit Graph

8114 Commits

Author SHA1 Message Date
Kjell Winblad
d524f8c805 refactor: rename config parameter 2023-06-08 16:27:04 +02:00
Kjell Winblad
6f2271e9f0 test: add Kafka producer bridge test case for query_mode parameter 2023-06-08 16:27:04 +02:00
Kjell Winblad
ed9e29e769 refactor: refacor query_mode detection code 
This commit refactor the query_mode resource detection code according to
a suggestion from @zmstone. This commit should not contain any
functional change except for a change of the Kafka producer bridge
config.
 2023-06-08 16:26:55 +02:00
Kjell Winblad
273dedf7a6
Merge pull request #10911 from kjellwinblad/kjell/bridge/atom_length_too_long/EMQX-9609 
fix: friendly error message when creating bridges with too long names
 2023-06-08 15:46:42 +02:00
JianBo He
034caaf20f
Merge pull request #10941 from HJianBo/enhance-prometheus-sampe-perf 
perf(prometheus): udpate the prometheus dep vsn
 2023-06-08 21:34:50 +08:00
Ivan Dyachkov
abbba71191 fix(tls): issue when ssl listner is configured to use tls v1.3 only 
clients could not connect due to incompatible tls options if ssl listner is configured to use tls v1.3 only
 2023-06-08 15:13:22 +02:00
Zaiming (Stone) Shi
7d2ea3dc76 docs: refine some desc 2023-06-08 13:58:31 +02:00
Andrew Mayorov
ce8cc05cc8
chore(ftschema): mention translate is for tests purpose only 2023-06-08 14:54:14 +03:00
Andrew Mayorov
e3c16be3c4
fix(ftschema): ensure translate follow emqx_config behavior 
So that atom keys won't unexpectedly get to converters / validators.
 2023-06-08 14:54:14 +03:00
Andrew Mayorov
ae97d516b8
fix(ft): add enable flag to every backend 
And tolerate multiple configured backends, as long as only one of
them is enabled.
 2023-06-08 14:54:13 +03:00
Ivan Dyachkov
b4d081f424
Merge pull request #10978 from id/0608-fix-404-in-rule-engine-api 
fix: missing 404 response in rule engine api schema
 2023-06-08 13:20:02 +02:00
Kjell Winblad
47fa17b3c1 feat: add sync/async option to the Kafka producer bridge 
This commit makes it possible to configure if a Kafka bridge should work
in query mode sync or async by setting the resource_opts.query_mode
configuration option.

Fixes:
https://emqx.atlassian.net/browse/EMQX-8631
 2023-06-08 13:16:06 +02:00
Stefan Strigler
f5e93b1b2b docs(emqx_prometheus): leave remark about json being deprecated 2023-06-08 13:08:45 +02:00
Zaiming (Stone) Shi
b481994e74 chore: bump to version 5.1.0-alpha.4 2023-06-08 12:30:35 +02:00
Zaiming (Stone) Shi
c1cf2365c2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-08 12:30:02 +02:00
JianBo He
95f0745e31 fix(coap): fix crash publish/subscribe in connectionless mode 
this bug was introduced by https://github.com/emqx/emqx/pull/10871
 2023-06-08 18:08:11 +08:00
Ivan Dyachkov
711cedc282 fix: missing 404 response in rule engine api schema 2023-06-08 11:46:16 +02:00
JianBo He
4065f08083 fix(stomp): fix frame limitation is not working 2023-06-08 16:57:12 +08:00
JianBo He
bad0c35bb9 fix(stomp): ensure the subscripton_cnt timely updated 2023-06-08 15:25:44 +08:00
Zaiming (Stone) Shi
641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51 
0607 merge master to release 51
 2023-06-08 09:22:21 +02:00
JianBo He
dc01b7109e chore: update prometheus vsn and grafana template 2023-06-08 14:29:54 +08:00
JianBo He
e0ec050e34 chore: update prometheus vsn 2023-06-08 14:29:54 +08:00
JianBo He
f53e13b8b9 perf: update to our forked prometheus deps 
it's based on v4.10.0 of upstream, and patched this PR:
https://github.com/emqx/prometheus.erl/pull/5
 2023-06-08 14:29:54 +08:00
Ilya Averyanov
b9f1a70214
Merge pull request #10967 from savonarola/0607-rebalance-fixes 
Fix rebalance issues
 2023-06-08 08:29:37 +03:00
zhongwencool
0116c53395 fix: link different document between community and enterprise 2023-06-08 11:33:39 +08:00
JianBo He
d5ebf5650f test: improve the test 2023-06-08 11:24:07 +08:00
JianBo He
fcef456174 feat: add live_connections field for some HTTP APIs 
i.e:
- `/monitor_current`, `/monitor_current/nodes/{node}`
- `/monitor/nodes/{node}`, `/monitor`
- `/node/{node}`, `/nodes`
 2023-06-08 11:24:06 +08:00
JianBo He
5d86443910 chore: fix dialyzer warnings 2023-06-08 11:06:25 +08:00
JianBo He
d0222ae79d chore: ensure test 2023-06-08 10:23:59 +08:00
JianBo He
4e9eeb489d
Merge pull request #10969 from thalesmg/refactor-gcp-pubsub-on-stop-v50 
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
 2023-06-08 09:47:19 +08:00
JianBo He
0b95bc1c1b
Merge pull request #10907 from HJianBo/refactor-on-stop-simple 
feat: refactored some bridges to avoid leaking resources part.2
 2023-06-08 09:37:31 +08:00
Zaiming (Stone) Shi
641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf 
feat: don't hide enable in flapping detect conf
 2023-06-07 23:04:56 +02:00
Zaiming (Stone) Shi
ccd2589ff2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-07 21:43:29 +02:00
Ilya Averyanov
7f2de66dab fix(rebalance): add wait_health_check timeout to node evacuation 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-06-07 21:37:01 +03:00
Andrew Mayorov
39b0d41527
fix(bpapi): ignore dropped BPAPI module 
It doesn't really breaks interoperability.
 2023-06-07 21:33:47 +03:00
Zaiming (Stone) Shi
aecea50564
Merge pull request #10902 from zmstone/0529-donot-copy-cluster-conf-from-newer-version 
0529 donot copy cluster conf from newer version
 2023-06-07 20:32:11 +02:00
Zaiming (Stone) Shi
1e0b2de0ea chore: refine data sync debug log messages 2023-06-07 20:19:48 +02:00
Zaiming (Stone) Shi
ed1ad4e684 chore(emqx_schema): add 'eanble' as an alias to 'enabled' fields 2023-06-07 20:19:48 +02:00
Thales Macedo Garitezi
d9aac63f82 refactor(gcp_pubsub): always cleanup resources when calling on_stop 
Fixes https://emqx.atlassian.net/browse/EMQX-9934

The way that this connector was structured already depended only on the resource ID.  This
PR just makes it completely independent of the resource state term, and also treats an
already absent pool as a success when stopping it.
 2023-06-07 14:34:22 -03:00
Thales Macedo Garitezi
105c26afb5
Merge pull request #10964 from thalesmg/fix-hocon-compact-errors-v50 
fix(schema): avoid `function_clause` error when compacting errors
 2023-06-07 14:00:24 -03:00
Andrew Mayorov
79d430cf03
refactor(pluglib): move emqx_plugin_libs_proto_v1 into emqx app 
And rename it to `emqx_metrics_proto_v1` in the process.
 2023-06-07 18:30:12 +03:00
Zaiming (Stone) Shi
622ac4d195
fix(emqx_schema): binary for string default value 2023-06-07 17:14:10 +02:00
JimMoen
4ea3483083
perf(topic): use tail recursive to joining a topic 2023-06-07 22:59:53 +08:00
JimMoen
b0e18f5e75
fix(shared-sub): validate share topic error send DISCONNECT 2023-06-07 22:59:52 +08:00
JimMoen
070e410c69
refactor: variable and func rename 2023-06-07 22:59:52 +08:00
JimMoen
ea81a924f1
refactor: move topic() types def in emqx_types.erl 2023-06-07 22:59:52 +08:00
JimMoen
65483e972e
refactor: use unified table define 2023-06-07 22:51:44 +08:00
zhongwencool
70e800f8a4 chore: delete unuse default value and make flapping_detect middle improtance 2023-06-07 22:50:11 +08:00
zhongwencool
5659cc2b47 feat: add converter to flapping_detect 2023-06-07 21:54:32 +08:00
zhongwencool
1d33b7dbd8 chore: make static check happy 2023-06-07 21:54:32 +08:00
zhongwencool
535870386a fix: zones update api failed 2023-06-07 21:54:32 +08:00
zhongwencool
d58506a7c3 feat: always update default zones 2023-06-07 21:54:32 +08:00
zhongwencool
48381d4c86 feat: refactor flapping detect conf 2023-06-07 21:54:32 +08:00
Thales Macedo Garitezi
cc8631223e fix(schema): avoid function_clause error when compacting errors 
Fixes https://emqx.atlassian.net/browse/EMQX-10168

The bridge probe API displayed the typecheck errors for the new
timeout duration types correctly, but when an user tried to create the
bridge anyway a `function_clause` error was raised when trying to
compact hocon errors:

```
09:47:19.045 [warning] [exception: :error, path: '/bridges', reason: {:case_clause, {:error, {:config_update_crashed, :function_clause}}}, stacktrace: [{:emqx_bridge_api, :create_or_update_bridge, 4, [file: '/home/thales/dev/emqx/emqx/apps/emqx_bridge/src/emqx_bridge_api.erl', line: 602]}, {:minirest_handler, :apply_callback, 3, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 111]}, {:minirest_handler, :handle, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 44]}, {:minirest_handler, :init, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 27]}, {:cowboy_handler, :execute, 2, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_handler.erl', line: 41]}, {:cowboy_stream_h, :execute, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 318]}, {:cowboy_stream_h, :request_process, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 302]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]]
```

This fixes the issue so that both APIs return more friendly error messages.
 2023-06-07 10:34:58 -03:00
Paulo Zulato
a4407764f3 feat(connection): configurable TCP keepalive 
Fixes https://emqx.atlassian.net/browse/EMQX-9852
 2023-06-07 10:23:38 -03:00
Thales Macedo Garitezi
f9ff1007a0
Merge pull request #10944 from thalesmg/pubsub-jwt-on-demand-v50 
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
 2023-06-07 09:22:09 -03:00
Ilya Averyanov
d1a1e8041d fix(rebalance api): fix error message formatting 2023-06-07 13:57:16 +03:00
Andrew Mayorov
c05f462d0a
fix(tlslib): bring back deterministic filenames for managed certs 2023-06-07 13:32:35 +03:00
William Yang
c1be86c5ec
Merge pull request #10931 from qzhuyan/perf/william/save-times-of-maps-build 
Perf/william/save times of maps build
 2023-06-07 10:32:45 +02:00
SergeTupchiy
3c482e9aa6
Merge pull request #10953 from SergeTupchiy/fix-listener-authn-update 
fix(emqx_listeners): fix listener authentication create/update
 2023-06-07 11:12:41 +03:00
Zaiming (Stone) Shi
cb3e787cb1 Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-07 09:14:04 +02:00
Zaiming (Stone) Shi
ce4e49cd9e
Merge pull request #10943 from zmstone/0605-hide-mcast-cluster-discovery 
chore: start deprecating mcast cluster discovery
 2023-06-07 08:32:33 +02:00
JianBo He
81387800b5
Merge branch 'master' into mqttsn-qos3 2023-06-07 09:49:29 +08:00
JianBo He
21b451a4c2
Merge pull request #10951 from HJianBo/mqttsn-mountpoint-pub 
fix(mqttsn): make mountpoint works for publish
 2023-06-07 09:39:46 +08:00
Andrew Mayorov
32a8963582
Merge pull request #10927 from keynslug/fix/EMQX-9930/tls-cert-gc 
feat(tlslib): add separate managed certfiles GC process
 2023-06-06 20:12:18 +03:00
Thales Macedo Garitezi
260fae296b feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1) 
Fixes https://emqx.atlassian.net/browse/EMQX-9603

Rather than relying on a JWT worker to produce and refresh tokens, we
could just produce then on demand when pushing the messages to GCP
PubSub.  That can generate a bit of extra work (as multiple processes
might realize it’s time to refresh the JWT and do so), but that
shouldn’t be much.  In return, we avoid any possibility of not having
a fresh JWT when pushing messages.
 2023-06-06 13:19:24 -03:00
Serge Tupchii
bf2b5a9f24 fix(emqx_listeners): fix listener authentication create/update 
Listener authentication is a list of authenticators, so each of them must be created
or updated individually.
 2023-06-06 19:00:35 +03:00
Serge Tupchii
1968589f81 fix(emqx_schema): don't allow enabling fail_if_no_peer_cert if verify_none is set 
Setting `fail_if_no_peer_cert = true` and `verify = verify_none` causes connection errors.

Closes: EMQX-9586
 2023-06-06 17:20:22 +03:00
Thales Macedo Garitezi
2c97ee1b3f test(conf_app): fix test after common test helpers changes 2023-06-06 09:32:38 -03:00
Thales Macedo Garitezi
97a9bb484a test(pulsar_producer): attempt to fix flaky test 2023-06-06 09:32:38 -03:00
Thales Macedo Garitezi
770dd188b1
Merge pull request #10947 from emqx/release-51 
Release 51 to master
 2023-06-06 09:32:16 -03:00
JianBo He
bcc47442eb fix(mqttsn): make mountpoint works for publish 2023-06-06 18:44:27 +08:00
Andrew Mayorov
68cd0c47f5
Merge pull request #10945 from fix/EMQX-9967/tmpfn-enametoolong 
fix(ft): ensure temp filenames are under 255 bytes long
 2023-06-06 13:34:47 +03:00
JianBo He
26748ef242 fix(mqttsn): checking enable_qos3 option 2023-06-06 18:21:26 +08:00
Andrew Mayorov
d5aedaac7d
fix(tlslib): append random suffix to managed certfiles 
In order to lessen the chance of ambiguity in determining collectable
certfiles during GC.
 2023-06-06 11:21:03 +03:00
Andrew Mayorov
dcd59e4f1b
fix(ft): set more conservative filename length limit 
Otherwise, local fs exporter will have a hard time preserving the
filemeta, because its filename is even 13 bytes longer.
 2023-06-06 10:40:20 +03:00
Andrew Mayorov
69b98c1830
fix(ft): ensure temp filenames are under 255 bytes long 2023-06-06 10:38:31 +03:00
某文
47d7e6ce01 feat: update gateway from cli 2023-06-06 08:12:47 +08:00
zhongwencool
ddb4a09a3d
Merge pull request #10935 from zhongwencool/listeners-conf-update 
feat: update listeners from cli
 2023-06-06 08:10:45 +08:00
Zaiming (Stone) Shi
3caf114f68 test: fix test case is_list(Config) function guard 2023-06-05 23:00:41 +02:00
Zaiming (Stone) Shi
bb1cf9beaa Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-05 22:56:58 +02:00
Andrew Mayorov
f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config 2023-06-05 23:32:13 +03:00
Andrew Mayorov
750b7158d4
fix(tlsgc): make computing orphans safe against symlinking 
Before this commit, `orphans/1` considered managed file an orphan based
on its absolute filename, which is not safe when, for example, emqx has
a symlink configured as `data_dir` while config references certfiles
through realpaths.
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
31d38e18c9
refactor(tlsgc): simplify fold_config/4 2023-06-05 22:41:08 +03:00
Andrew Mayorov
f8e8b7993b
fix(tlsgc): anticipate only binaries as string in raw config 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
468fd98173
test(utilfs): add few more traverse_dir/3 testcases 2023-06-05 22:41:08 +03:00
Andrew Mayorov
3c2a7dbadc
fix(tlsgc): consolidate conf keypaths knowledge in emqx_tls_lib 
So that this GC mechanism will be easier to maintain.
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
ec06850bd0
test(auth): drop obsolete testcase altogether 
Most of the stuff that was tested by this test case is now covered
by `emqx_tls_certfile_gc_SUITE`.
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
206d0472e0
fix(emqx): avoid interference in testsuites 
Also remove that `?wait_async_action` in `emqx_trace_SUITE` which
consistently becomes stuck for 100 seconds. It's not clear why it
was there in the first place because listeners should start
synchronously, so `emqx_common_test_helpers:start_apps/1` won't return
until required listeners are started.
 2023-06-05 22:41:08 +03:00
Andrew Mayorov
8a31e5639b
fix(emqx): ensure that standalone build works 
Turns out rebar3 `path` resource has troubles with dangling
symlinks and non-ASCII filenames. 🥲
 2023-06-05 22:41:07 +03:00
Andrew Mayorov
1cb226dffb
chore: bump applications versions 
* emqx_authz 0.1.22
* emqx_exhook 5.0.13
* emqx_utils 5.0.3
 2023-06-05 22:41:07 +03:00
Andrew Mayorov
afbf8cb32f
chore(util): drop obsolete stuff from emqx_utils_maps 2023-06-05 22:41:07 +03:00
Andrew Mayorov
99ea9b86c2
feat(tlslib): add separate managed certfiles GC process 
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
 2023-06-05 22:41:07 +03:00
Andrew Mayorov
95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys 
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
 2023-06-05 22:41:07 +03:00
Zaiming (Stone) Shi
67db9d6fe9 chore: bump version to 5.1.0-alpha.3 2023-06-05 20:25:48 +02:00
Thales Macedo Garitezi
75df622426
Merge pull request #10930 from thalesmg/max-int-timeout-v50 
check maximum timeout values in schema (5.1)
 2023-06-05 12:51:36 -03:00
Kjell Winblad
375661c6a1 test: add test case for error when bridge name is too long 2023-06-05 17:02:08 +02:00
Thales Macedo Garitezi
46393343e2 chore: use timeout_duration types for timer fields 
Fixes https://emqx.atlassian.net/browse/EMQX-10020
 2023-06-05 11:46:38 -03:00
Thales Macedo Garitezi
9bd37937f1 feat(types): add timeout_duration{,_ms,_s} types 2023-06-05 11:46:32 -03:00
Paulo Zulato
8d97a85b3c
Merge pull request #10893 from paulozulato/fix-oracle-error-handling 
fix(oracle): fix error handling on sync query
 2023-06-05 10:48:22 -03:00
Zaiming (Stone) Shi
bdcc069aac chore: start deprecating mcast cluster discovery 2023-06-05 14:58:53 +02:00
JimMoen
0f808345e6
Merge pull request #10924 from JimMoen/refactor_influxdb_on_stop 
feat: refactor influxdb connector to to avoid resources leaking
 2023-06-05 18:02:15 +08:00
lafirest
d51c658a30
Merge pull request #10908 from lafirest/feat/rocketmq_on_stop 
feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation
 2023-06-05 15:00:32 +08:00
JianBo He
dbc0cdce67 chore: dont allocate resource for simple connectiors 2023-06-05 10:59:00 +08:00
firest
5921ed3d2e fix(rocketmq): improve function name 2023-06-05 10:43:28 +08:00
JianBo He
e717ddafd7 chore: update changes 2023-06-05 10:19:46 +08:00
JianBo He
3739230435 feat: refactor connectors on_stop function to avoid resources leaking 
Supplement to https://github.com/emqx/emqx/pull/10895
 2023-06-05 10:19:46 +08:00
JianBo He
e07c86b6a8
Merge pull request #10909 from HJianBo/remove-deprecated-gateways 
feat: remove the deprecated '/gateway/*' HTTP APIs
 2023-06-05 09:59:50 +08:00
某文
d0d6992a14 feat: update listeners from cli 2023-06-04 20:07:33 +08:00
某文
03160ef599 test: add more test for update_config 2023-06-04 12:02:02 +08:00
某文
bd29433997 feat: support emqx_conf:update([exhook],Conf) 2023-06-04 09:30:17 +08:00
JimMoen
a174d5741b
feat: refactor influxdb connector to to avoid resources leaking 2023-06-04 02:21:49 +08:00
Zaiming (Stone) Shi
f469b31fee Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-03 11:21:22 +02:00
Paulo Zulato
c1d935fd34
Merge pull request #10917 from paulozulato/block-oracle-default-database-connection 
test(oracle): add match for reason field in a test case
 2023-06-02 18:16:12 -03:00
Andrew Mayorov
5b56a35c0a
Merge pull request #10932 from chore/drop-mqtt-mode-tests 
chore(bridge): drop deprecated `mode` setting from examples / tests
 2023-06-02 22:46:31 +03:00
Zaiming (Stone) Shi
0b87ea4cdc
Merge pull request #10929 from zmstone/0602-ci-pin-otp-25.3.2-1-elixir-1.14.5 
ci: upgrade to otp 25.3.2-1 and elixir 1.14.5
 2023-06-02 21:46:08 +02:00
Zaiming (Stone) Shi
aa46c0a484
Merge pull request #10928 from zmstone/0602-sync-release-51-to-master 
0602 sync release 51 to master
 2023-06-02 21:45:21 +02:00
Thales Macedo Garitezi
33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50 
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
 2023-06-02 16:20:36 -03:00
Andrew Mayorov
09290f8d61
chore(bridge): drop deprecated mode setting from examples / tests 2023-06-02 20:31:31 +03:00
Zaiming (Stone) Shi
e6fb27b3c7 fix(emqx_telemetry): make dialyzer happy 2023-06-02 18:02:46 +02:00
Zaiming (Stone) Shi
1ba07e1040 Merge remote-tracking branch 'origin/release-51' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-02 16:30:58 +02:00
Zaiming (Stone) Shi
28c564d15b Merge remote-tracking branch 'origin/master' into release-51 2023-06-02 16:25:48 +02:00
William Yang
b1c188f9b1 perf(mqtt-caps): dont filter cap keys 
save map builds
 2023-06-02 16:24:52 +02:00
William Yang
4961aca3d0 perf(mqtt-caps): save some map builds 2023-06-02 16:19:58 +02:00
Thales Macedo Garitezi
74ffd9ef96
Merge pull request #10918 from thalesmg/fix-ecpool-empty-unrecoverable-v50 
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
 2023-06-02 09:21:00 -03:00
Zaiming (Stone) Shi
367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition 
fix(emqx_cm): fix channel data registration race-condition
 2023-06-02 14:20:18 +02:00
Thales Macedo Garitezi
0790c88aaf
refactor: use default's type as first union member 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-06-02 09:08:11 -03:00
Thales Macedo Garitezi
2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50 
fix(plugins): use `emqx:running_nodes` for multicall operations
 2023-06-02 09:03:42 -03:00
Zaiming (Stone) Shi
c75e9bbe0d fix(emqx_cm): fix channel data registration race-condition 
when clustered, there are chances the a mqtt client process
get killed (e.g. holding the channel registeration lock for too long),
if the channel data inserts happen before casting out the message
for channel process monitoring, there is a chance for the
stale message left in the ets tables indefinitely.

this commit changes the order of the non-atomic operations:
it casts out the monitor request message before inserting
channel data.
 2023-06-02 11:43:35 +02:00
Stefan Strigler
c9ef1ccacd fix: headers is array not object 2023-06-02 10:24:33 +02:00
Stefan Strigler
c7f177054c fix(emqx_authn): verify_claims is list(), therefor default must match 2023-06-02 10:24:33 +02:00
Stefan Strigler
15c7c281ca fix: don't convert empty list default 2023-06-02 10:03:01 +02:00
Stefan Strigler
190e44f8be fix: brackets not allowed in swagger schema for component names 2023-06-02 10:03:01 +02:00
Stefan Strigler
8fdc9de9e9 fix: empty properties as empty map so it ends up as a json object 2023-06-02 10:03:01 +02:00
Stefan Strigler
e62bd63199 fix: don't add summary, remove it instead 2023-06-02 10:03:01 +02:00
Stefan Strigler
ef2581d13b fix: remove unsupported schema properties 2023-06-02 10:03:01 +02:00
Stefan Strigler
12a4e48727 fix: add definition for bearerAuth 2023-06-02 10:03:01 +02:00
zhongwencool
8d8efe449e
Merge pull request #10896 from zhongwencool/api-keys-hot-conf-update 
feat: api_key conf support hot conf
 2023-06-02 15:58:45 +08:00
zhongwencool
dd85c981cd
Merge pull request #10783 from zhongwencool/improve-authn 
feat: load changes to the authentication configuration from command line
 2023-06-02 15:58:13 +08:00
zhongwencool
9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve 
feat: support emqx_config:update([authrization],Conf) update
 2023-06-02 15:43:09 +08:00
lafirest
0f8824b40f
Merge pull request #10921 from lafirest/test/rocketmq_acl 
test(rocketmq): add an ACL test case
 2023-06-02 15:37:46 +08:00
JianBo He
5e0a3606ad chore: fix outdate tests 2023-06-02 13:30:03 +08:00
JianBo He
0633d894b8 test: fix failed tests 2023-06-02 13:30:03 +08:00
JianBo He
ec0049af7b feat: remove the deprecated '/gateway/*' HTTP APIs 2023-06-02 13:30:03 +08:00
firest
ea9b1e13d5 test(rocketmq): add an ACL test case 2023-06-02 11:19:06 +08:00
Paulo Zulato
fb455d68a3 fix(oracle): fix error handling on sync query 
Fixes https://emqx.atlassian.net/browse/EMQX-10075
 2023-06-01 20:50:29 -03:00
某文
367a0e312a chore: remove config_path() function 2023-06-02 07:09:08 +08:00
Andrew Mayorov
76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum 
fix(ft): respect checksum in `fin` packets
 2023-06-01 23:49:55 +03:00
Thales Macedo Garitezi
0072f4a598 fix(ecpool,bridge): treat {error, ecpool_empty} as a retriable error 
Example from flaky CI test:

```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
 2023-06-01 17:13:06 -03:00
Paulo Zulato
92c72c6994 test(oracle): add match for reason field in a test case 
Fixes https://emqx.atlassian.net/browse/EMQX-9980
 2023-06-01 15:12:09 -03:00
Paulo Zulato
364601c3aa
Merge pull request #10892 from paulozulato/block-oracle-default-database-connection 
fix(oracle): require either sid or service name
 2023-06-01 15:04:03 -03:00
Thales Macedo Garitezi
7d3ce14835 test: rm unused var warning 2023-06-01 14:00:43 -03:00
Thales Macedo Garitezi
648b8231ef fix(plugins): avoid crashing when fetching error reason 2023-06-01 14:00:43 -03:00
Thales Macedo Garitezi
64258120d7 fix(plugins): use emqx:running_nodes for multicall operations 
Fixes https://emqx.atlassian.net/browse/EMQX-10079
 2023-06-01 14:00:43 -03:00
某文
2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
Thales Macedo Garitezi
3e4790edd4 test(pulsar_producer): fix flaky test 2023-06-01 13:01:58 -03:00
Thales Macedo Garitezi
99796224d8 refactor(resource): rename request_timeout -> request_ttl 
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
 2023-06-01 13:01:53 -03:00
某文
c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
Stefan Strigler
9c76aa901e fix(emqx_mgmt_api_clients): hide set_keepalive_seconds from swagger 2023-06-01 16:57:19 +02:00
某文
94b86f01a5 chore: reformat SUITE case 2023-06-01 22:32:31 +08:00
zhongwencool
21df321d89
Merge branch 'master' into improve-authn 2023-06-01 22:24:59 +08:00
Thales Macedo Garitezi
f42ccb6262 feat(resource): increase default request timeout to 45 s 
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
 2023-06-01 11:20:06 -03:00
Thales Macedo Garitezi
10425eb925 feat(resource): deprecate auto_restart_interval in favor of health_check_interval 
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
 2023-06-01 11:20:06 -03:00
Stefan Strigler
e011b5532f
Merge pull request #10881 from sstrigler/EMQX-9355-5-0-x-test-coverage-80 
IoTDB test coverage
 2023-06-01 15:48:56 +02:00
Thales Macedo Garitezi
ecbb6e0ecb test(global_gc): fix flaky test 2023-06-01 10:48:02 -03:00
Kjell Winblad
75ff76a16b fix: friendly error message when creating bridges with too long names 
This commit makes the error message and log entry that appear when one
tries to create a bridge with a name the exceeds 255 bytes (the max
length for atoms) more friendly and easier to understand.

An even better fix would be to not store bridge names as atoms but this
probably requires a more substantial change.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9609
 2023-06-01 15:30:06 +02:00
Thales Macedo Garitezi
7740987a17
Merge pull request #10905 from thalesmg/fix-enforce-binary-rootkey-config-v50 
fix(config): enforce root key name to be a binary (raw config)
 2023-06-01 10:27:48 -03:00
William Yang
c1d9bc8033 perf(emqx_mqtt_caps): direct read mqtt from zone conf 2023-06-01 14:54:44 +02:00
William Yang
923913e15c feat(config): merge with global defaults when put new zone 2023-06-01 14:54:44 +02:00
William Yang
a2f8e87389 test(config): default zone and global defaults 2023-06-01 14:54:14 +02:00
William Yang
e693c6ec54 test: fix emqx_takeover_SUITE 2023-06-01 14:52:33 +02:00
William Yang
8c26ee75db test: remove hardcoded configs 
test with schema defaults
 2023-06-01 14:52:33 +02:00
William Yang
4416122cf3 test: fix longname/shortname 2023-06-01 14:52:33 +02:00
William Yang
8ee78ad6b4 test: fix emqx_mgmt_api_configs_SUITE 2023-06-01 14:52:33 +02:00
William Yang
f621bf1f8e test(config): update emqx_client_SUITE 2023-06-01 14:52:33 +02:00
William Yang
98d6ed4251 test: fix emqx_session_SUITE 2023-06-01 14:52:33 +02:00
William Yang
6c1671651c test: use zone default in emqx_eviction_agent_channel_SUITE 2023-06-01 14:52:33 +02:00
William Yang
88f9782db0 perf(config): read zone conf once 2023-06-01 14:52:30 +02:00
firest
232ef23a48 feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation 2023-06-01 18:49:45 +08:00
Zhongwen Deng
7eea693422 fix: reorder authn when updating 2023-06-01 17:24:39 +08:00
Zhongwen Deng
96ee38ce15 chore: bump up authn/ctl app version 2023-06-01 17:24:39 +08:00
Zhongwen Deng
4f5d5216bb test: bad authn handler callback module in SUITE 2023-06-01 17:24:39 +08:00
Zhongwen Deng
8f12d81878 feat: support emqx_ctl conf command 2023-06-01 17:24:39 +08:00
Zhongwen Deng
b290d2543b fix: don't crash when command_ctl table not init 2023-06-01 17:24:39 +08:00
Zaiming (Stone) Shi
feecd53436 chore: bump to version 5.1.0-alpha.2 for both ce and ee 2023-06-01 10:13:56 +02:00
Andrew Mayorov
b5f24c4f88
Merge pull request #10900 from keynslug/fix/EEC-856/parsed-bridge-id 
fix(ruleeng): parse bridge id on create / update
 2023-06-01 10:21:26 +03:00
JianBo He
bcc5e30f22
Merge pull request #10895 from lafirest/fix/refactor_on_stop 
feat: refactored some bridges to avoid leaking resources during crashes at creation
 2023-06-01 15:17:37 +08:00
Zaiming (Stone) Shi
760de7becf chore: imporve logging message 2023-06-01 08:26:53 +02:00
Zaiming (Stone) Shi
15ffa98bc5 chore: fix a typo in test case function name 2023-06-01 08:18:26 +02:00
Zaiming (Stone) Shi
cb2be1811e docs: fix function doc annotation 2023-06-01 08:17:36 +02:00
firest
cb31032e6d chore: make dialyzer && test happy 2023-06-01 11:15:51 +08:00
Thales Macedo Garitezi
f83f112189 fix(config): enforce root key name to be a binary (raw config) 
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299

```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
    exception: error
    key_path: [listeners,tcp,default,authentication]
    module: emqx_schema
    msg: change_config_crashed
    reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
                               <<"authentication">>]}
    stacktrace: [{emqx_utils_maps,deep_get,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
                      {line,49}]},
                 {emqx_config_handler,return_change_result,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,456}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,284}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,
                     {create_authenticator,'tcp:default',
                         #{backend => built_in_database,enable => true,
                           mechanism => password_based}}},
                 #{rawconf_with_defaults => true}}
```

Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:

```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
 #{<<"ssl">> =>
       #{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```

... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
 2023-05-31 18:03:58 -03:00
Paulo Zulato
3a6ad3b049 fix(oracle): require either sid or service name 
Fixes https://emqx.atlassian.net/browse/EMQX-9980
 2023-05-31 17:18:45 -03:00
Thales Macedo Garitezi
b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e 
more attempts to fix more test flakiness
 2023-05-31 16:07:18 -03:00
Zaiming (Stone) Shi
5146de5b1c feat: add a backup copies for cluster.hocon 2023-05-31 20:34:25 +02:00
Zaiming (Stone) Shi
332daabcc5 fix(config): Do not sync cluster config from nodes running new version 2023-05-31 20:34:25 +02:00
Zaiming (Stone) Shi
39d6f612ca feat(emqx_release): add new APIs to parse/compare release versions 2023-05-31 20:34:25 +02:00
Andrew Mayorov
aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard 
test(dashboard): fix inter-suite test flakiness
 2023-05-31 20:28:39 +03:00
Andrew Mayorov
f90ce9bbfe
test(dashboard): fix inter-suite test flakiness 2023-05-31 19:09:37 +03:00
Andrew Mayorov
0293b54211
fix(ft): respect checksum in fin packets 2023-05-31 18:50:38 +03:00