Commit Graph

7986 Commits

Author SHA1 Message Date
JianBo He dc01b7109e chore: update prometheus vsn and grafana template 2023-06-08 14:29:54 +08:00
JianBo He e0ec050e34 chore: update prometheus vsn 2023-06-08 14:29:54 +08:00
JianBo He f53e13b8b9 perf: update to our forked prometheus deps
it's based on v4.10.0 of upstream, and patched this PR:
https://github.com/emqx/prometheus.erl/pull/5
2023-06-08 14:29:54 +08:00
Ilya Averyanov b9f1a70214
Merge pull request #10967 from savonarola/0607-rebalance-fixes
Fix rebalance issues
2023-06-08 08:29:37 +03:00
zhongwencool 0116c53395 fix: link different document between community and enterprise 2023-06-08 11:33:39 +08:00
JianBo He d5ebf5650f test: improve the test 2023-06-08 11:24:07 +08:00
JianBo He fcef456174 feat: add `live_connections` field for some HTTP APIs
i.e:
- `/monitor_current`, `/monitor_current/nodes/{node}`
- `/monitor/nodes/{node}`, `/monitor`
- `/node/{node}`, `/nodes`
2023-06-08 11:24:06 +08:00
JianBo He 4e9eeb489d
Merge pull request #10969 from thalesmg/refactor-gcp-pubsub-on-stop-v50
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
2023-06-08 09:47:19 +08:00
JianBo He 0b95bc1c1b
Merge pull request #10907 from HJianBo/refactor-on-stop-simple
feat: refactored some bridges to avoid leaking resources part.2
2023-06-08 09:37:31 +08:00
Zaiming (Stone) Shi 641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf
feat: don't hide enable in flapping detect conf
2023-06-07 23:04:56 +02:00
Zaiming (Stone) Shi ccd2589ff2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-07 21:43:29 +02:00
Ilya Averyanov 7f2de66dab fix(rebalance): add wait_health_check timeout to node evacuation
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-06-07 21:37:01 +03:00
Andrew Mayorov 39b0d41527
fix(bpapi): ignore dropped BPAPI module
It doesn't really breaks interoperability.
2023-06-07 21:33:47 +03:00
Zaiming (Stone) Shi aecea50564
Merge pull request #10902 from zmstone/0529-donot-copy-cluster-conf-from-newer-version
0529 donot copy cluster conf from newer version
2023-06-07 20:32:11 +02:00
Thales Macedo Garitezi d9aac63f82 refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
Fixes https://emqx.atlassian.net/browse/EMQX-9934

The way that this connector was structured already depended only on the resource ID.  This
PR just makes it completely independent of the resource state term, and also treats an
already absent pool as a success when stopping it.
2023-06-07 14:34:22 -03:00
Thales Macedo Garitezi 105c26afb5
Merge pull request #10964 from thalesmg/fix-hocon-compact-errors-v50
fix(schema): avoid `function_clause` error when compacting errors
2023-06-07 14:00:24 -03:00
Andrew Mayorov 79d430cf03
refactor(pluglib): move `emqx_plugin_libs_proto_v1` into emqx app
And rename it to `emqx_metrics_proto_v1` in the process.
2023-06-07 18:30:12 +03:00
Zaiming (Stone) Shi 622ac4d195
fix(emqx_schema): binary for string default value 2023-06-07 17:14:10 +02:00
JimMoen 4ea3483083
perf(topic): use tail recursive to joining a topic 2023-06-07 22:59:53 +08:00
JimMoen b0e18f5e75
fix(shared-sub): validate share topic error send DISCONNECT 2023-06-07 22:59:52 +08:00
JimMoen 070e410c69
refactor: variable and func rename 2023-06-07 22:59:52 +08:00
JimMoen ea81a924f1
refactor: move topic() types def in `emqx_types.erl` 2023-06-07 22:59:52 +08:00
JimMoen 65483e972e
refactor: use unified table define 2023-06-07 22:51:44 +08:00
zhongwencool 70e800f8a4 chore: delete unuse default value and make flapping_detect middle improtance 2023-06-07 22:50:11 +08:00
zhongwencool 5659cc2b47 feat: add converter to flapping_detect 2023-06-07 21:54:32 +08:00
zhongwencool 1d33b7dbd8 chore: make static check happy 2023-06-07 21:54:32 +08:00
zhongwencool 535870386a fix: zones update api failed 2023-06-07 21:54:32 +08:00
zhongwencool d58506a7c3 feat: always update default zones 2023-06-07 21:54:32 +08:00
zhongwencool 48381d4c86 feat: refactor flapping detect conf 2023-06-07 21:54:32 +08:00
Thales Macedo Garitezi cc8631223e fix(schema): avoid `function_clause` error when compacting errors
Fixes https://emqx.atlassian.net/browse/EMQX-10168

The bridge probe API displayed the typecheck errors for the new
timeout duration types correctly, but when an user tried to create the
bridge anyway a `function_clause` error was raised when trying to
compact hocon errors:

```
09:47:19.045 [warning] [exception: :error, path: '/bridges', reason: {:case_clause, {:error, {:config_update_crashed, :function_clause}}}, stacktrace: [{:emqx_bridge_api, :create_or_update_bridge, 4, [file: '/home/thales/dev/emqx/emqx/apps/emqx_bridge/src/emqx_bridge_api.erl', line: 602]}, {:minirest_handler, :apply_callback, 3, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 111]}, {:minirest_handler, :handle, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 44]}, {:minirest_handler, :init, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 27]}, {:cowboy_handler, :execute, 2, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_handler.erl', line: 41]}, {:cowboy_stream_h, :execute, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 318]}, {:cowboy_stream_h, :request_process, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 302]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]]
```

This fixes the issue so that both APIs return more friendly error messages.
2023-06-07 10:34:58 -03:00
Paulo Zulato a4407764f3 feat(connection): configurable TCP keepalive
Fixes https://emqx.atlassian.net/browse/EMQX-9852
2023-06-07 10:23:38 -03:00
Thales Macedo Garitezi f9ff1007a0
Merge pull request #10944 from thalesmg/pubsub-jwt-on-demand-v50
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
2023-06-07 09:22:09 -03:00
Ilya Averyanov d1a1e8041d fix(rebalance api): fix error message formatting 2023-06-07 13:57:16 +03:00
Andrew Mayorov c05f462d0a
fix(tlslib): bring back deterministic filenames for managed certs 2023-06-07 13:32:35 +03:00
William Yang c1be86c5ec
Merge pull request #10931 from qzhuyan/perf/william/save-times-of-maps-build
Perf/william/save times of maps build
2023-06-07 10:32:45 +02:00
SergeTupchiy 3c482e9aa6
Merge pull request #10953 from SergeTupchiy/fix-listener-authn-update
fix(emqx_listeners): fix listener authentication create/update
2023-06-07 11:12:41 +03:00
Zaiming (Stone) Shi cb3e787cb1 Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-07 09:14:04 +02:00
Zaiming (Stone) Shi ce4e49cd9e
Merge pull request #10943 from zmstone/0605-hide-mcast-cluster-discovery
chore: start deprecating mcast cluster discovery
2023-06-07 08:32:33 +02:00
JianBo He 21b451a4c2
Merge pull request #10951 from HJianBo/mqttsn-mountpoint-pub
fix(mqttsn): make mountpoint works for publish
2023-06-07 09:39:46 +08:00
Andrew Mayorov 32a8963582
Merge pull request #10927 from keynslug/fix/EMQX-9930/tls-cert-gc
feat(tlslib): add separate managed certfiles GC process
2023-06-06 20:12:18 +03:00
Thales Macedo Garitezi 260fae296b feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-9603

Rather than relying on a JWT worker to produce and refresh tokens, we
could just produce then on demand when pushing the messages to GCP
PubSub.  That can generate a bit of extra work (as multiple processes
might realize it’s time to refresh the JWT and do so), but that
shouldn’t be much.  In return, we avoid any possibility of not having
a fresh JWT when pushing messages.
2023-06-06 13:19:24 -03:00
Serge Tupchii bf2b5a9f24 fix(emqx_listeners): fix listener authentication create/update
Listener authentication is a list of authenticators, so each of them must be created
or updated individually.
2023-06-06 19:00:35 +03:00
Serge Tupchii 1968589f81 fix(emqx_schema): don't allow enabling `fail_if_no_peer_cert` if `verify_none` is set
Setting `fail_if_no_peer_cert = true` and `verify = verify_none` causes connection errors.

Closes: EMQX-9586
2023-06-06 17:20:22 +03:00
Thales Macedo Garitezi 770dd188b1
Merge pull request #10947 from emqx/release-51
Release 51 to master
2023-06-06 09:32:16 -03:00
JianBo He bcc47442eb fix(mqttsn): make mountpoint works for publish 2023-06-06 18:44:27 +08:00
Andrew Mayorov 68cd0c47f5
Merge pull request #10945 from fix/EMQX-9967/tmpfn-enametoolong
fix(ft): ensure temp filenames are under 255 bytes long
2023-06-06 13:34:47 +03:00
Andrew Mayorov d5aedaac7d
fix(tlslib): append random suffix to managed certfiles
In order to lessen the chance of ambiguity in determining collectable
certfiles during GC.
2023-06-06 11:21:03 +03:00
Andrew Mayorov dcd59e4f1b
fix(ft): set more conservative filename length limit
Otherwise, local fs exporter will have a hard time preserving the
filemeta, because its filename is even 13 bytes longer.
2023-06-06 10:40:20 +03:00
Andrew Mayorov 69b98c1830
fix(ft): ensure temp filenames are under 255 bytes long 2023-06-06 10:38:31 +03:00
某文 47d7e6ce01 feat: update gateway from cli 2023-06-06 08:12:47 +08:00
zhongwencool ddb4a09a3d
Merge pull request #10935 from zhongwencool/listeners-conf-update
feat: update listeners from cli
2023-06-06 08:10:45 +08:00
Zaiming (Stone) Shi 3caf114f68 test: fix test case is_list(Config) function guard 2023-06-05 23:00:41 +02:00
Zaiming (Stone) Shi bb1cf9beaa Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-05 22:56:58 +02:00
Andrew Mayorov f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config 2023-06-05 23:32:13 +03:00
Andrew Mayorov 750b7158d4
fix(tlsgc): make computing orphans safe against symlinking
Before this commit, `orphans/1` considered managed file an orphan based
on its absolute filename, which is not safe when, for example, emqx has
a symlink configured as `data_dir` while config references certfiles
through realpaths.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 31d38e18c9
refactor(tlsgc): simplify `fold_config/4` 2023-06-05 22:41:08 +03:00
Andrew Mayorov f8e8b7993b
fix(tlsgc): anticipate only binaries as string in raw config
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-05 22:41:08 +03:00
Andrew Mayorov 468fd98173
test(utilfs): add few more `traverse_dir/3` testcases 2023-06-05 22:41:08 +03:00
Andrew Mayorov 3c2a7dbadc
fix(tlsgc): consolidate conf keypaths knowledge in `emqx_tls_lib`
So that this GC mechanism will be easier to maintain.
2023-06-05 22:41:08 +03:00
Andrew Mayorov ec06850bd0
test(auth): drop obsolete testcase altogether
Most of the stuff that was tested by this test case is now covered
by `emqx_tls_certfile_gc_SUITE`.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 206d0472e0
fix(emqx): avoid interference in testsuites
Also remove that `?wait_async_action` in `emqx_trace_SUITE` which
consistently becomes stuck for 100 seconds. It's not clear why it
was there in the first place because listeners should start
synchronously, so `emqx_common_test_helpers:start_apps/1` won't return
until required listeners are started.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 8a31e5639b
fix(emqx): ensure that standalone build works
Turns out rebar3 `path` resource has troubles with dangling
symlinks and non-ASCII filenames. 🥲
2023-06-05 22:41:07 +03:00
Andrew Mayorov 1cb226dffb
chore: bump applications versions
* emqx_authz 0.1.22
* emqx_exhook 5.0.13
* emqx_utils 5.0.3
2023-06-05 22:41:07 +03:00
Andrew Mayorov afbf8cb32f
chore(util): drop obsolete stuff from `emqx_utils_maps` 2023-06-05 22:41:07 +03:00
Andrew Mayorov 99ea9b86c2
feat(tlslib): add separate managed certfiles GC process
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
2023-06-05 22:41:07 +03:00
Andrew Mayorov 95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
2023-06-05 22:41:07 +03:00
Zaiming (Stone) Shi 67db9d6fe9 chore: bump version to 5.1.0-alpha.3 2023-06-05 20:25:48 +02:00
Thales Macedo Garitezi 75df622426
Merge pull request #10930 from thalesmg/max-int-timeout-v50
check maximum timeout values in schema (5.1)
2023-06-05 12:51:36 -03:00
Kjell Winblad 375661c6a1 test: add test case for error when bridge name is too long 2023-06-05 17:02:08 +02:00
Thales Macedo Garitezi 46393343e2 chore: use `timeout_duration` types for timer fields
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
Thales Macedo Garitezi 9bd37937f1 feat(types): add `timeout_duration{,_ms,_s}` types 2023-06-05 11:46:32 -03:00
Paulo Zulato 8d97a85b3c
Merge pull request #10893 from paulozulato/fix-oracle-error-handling
fix(oracle): fix error handling on sync query
2023-06-05 10:48:22 -03:00
Zaiming (Stone) Shi bdcc069aac chore: start deprecating mcast cluster discovery 2023-06-05 14:58:53 +02:00
JimMoen 0f808345e6
Merge pull request #10924 from JimMoen/refactor_influxdb_on_stop
feat: refactor influxdb connector to to avoid resources leaking
2023-06-05 18:02:15 +08:00
lafirest d51c658a30
Merge pull request #10908 from lafirest/feat/rocketmq_on_stop
feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation
2023-06-05 15:00:32 +08:00
JianBo He dbc0cdce67 chore: dont allocate resource for simple connectiors 2023-06-05 10:59:00 +08:00
firest 5921ed3d2e fix(rocketmq): improve function name 2023-06-05 10:43:28 +08:00
JianBo He e717ddafd7 chore: update changes 2023-06-05 10:19:46 +08:00
JianBo He 3739230435 feat: refactor connectors on_stop function to avoid resources leaking
Supplement to https://github.com/emqx/emqx/pull/10895
2023-06-05 10:19:46 +08:00
JianBo He e07c86b6a8
Merge pull request #10909 from HJianBo/remove-deprecated-gateways
feat: remove the deprecated '/gateway/*' HTTP APIs
2023-06-05 09:59:50 +08:00
某文 d0d6992a14 feat: update listeners from cli 2023-06-04 20:07:33 +08:00
某文 03160ef599 test: add more test for update_config 2023-06-04 12:02:02 +08:00
某文 bd29433997 feat: support emqx_conf:update([exhook],Conf) 2023-06-04 09:30:17 +08:00
JimMoen a174d5741b
feat: refactor influxdb connector to to avoid resources leaking 2023-06-04 02:21:49 +08:00
Zaiming (Stone) Shi f469b31fee Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-03 11:21:22 +02:00
Paulo Zulato c1d935fd34
Merge pull request #10917 from paulozulato/block-oracle-default-database-connection
test(oracle): add match for reason field in a test case
2023-06-02 18:16:12 -03:00
Andrew Mayorov 5b56a35c0a
Merge pull request #10932 from chore/drop-mqtt-mode-tests
chore(bridge): drop deprecated `mode` setting from examples / tests
2023-06-02 22:46:31 +03:00
Zaiming (Stone) Shi 0b87ea4cdc
Merge pull request #10929 from zmstone/0602-ci-pin-otp-25.3.2-1-elixir-1.14.5
ci: upgrade to otp 25.3.2-1 and elixir 1.14.5
2023-06-02 21:46:08 +02:00
Zaiming (Stone) Shi aa46c0a484
Merge pull request #10928 from zmstone/0602-sync-release-51-to-master
0602 sync release 51 to master
2023-06-02 21:45:21 +02:00
Thales Macedo Garitezi 33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
2023-06-02 16:20:36 -03:00
Andrew Mayorov 09290f8d61
chore(bridge): drop deprecated `mode` setting from examples / tests 2023-06-02 20:31:31 +03:00
Zaiming (Stone) Shi e6fb27b3c7 fix(emqx_telemetry): make dialyzer happy 2023-06-02 18:02:46 +02:00
Zaiming (Stone) Shi 1ba07e1040 Merge remote-tracking branch 'origin/release-51' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-02 16:30:58 +02:00
Zaiming (Stone) Shi 28c564d15b Merge remote-tracking branch 'origin/master' into release-51 2023-06-02 16:25:48 +02:00
William Yang b1c188f9b1 perf(mqtt-caps): dont filter cap keys
save map builds
2023-06-02 16:24:52 +02:00
William Yang 4961aca3d0 perf(mqtt-caps): save some map builds 2023-06-02 16:19:58 +02:00
Thales Macedo Garitezi 74ffd9ef96
Merge pull request #10918 from thalesmg/fix-ecpool-empty-unrecoverable-v50
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
2023-06-02 09:21:00 -03:00
Zaiming (Stone) Shi 367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition
fix(emqx_cm): fix channel data registration race-condition
2023-06-02 14:20:18 +02:00
Thales Macedo Garitezi 0790c88aaf
refactor: use default's type as first union member
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-02 09:08:11 -03:00
Thales Macedo Garitezi 2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50
fix(plugins): use `emqx:running_nodes` for multicall operations
2023-06-02 09:03:42 -03:00