yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,973 Commits 393 Branches 669 Tags 74 MiB
589080820e
Commit Graph

7145 Commits

Author SHA1 Message Date
William Yang
6dd99b3331
Merge pull request #10030 from qzhuyan/test/william/fix-quic-flaky-tc 
chore: fix quic flaky tc
 2023-02-27 10:04:12 +01:00
firest
7d762593f6 fix(connector): redact the http body in error logs for security reasons 2023-02-27 11:41:34 +08:00
William Yang
58af936120 test: fix flaky emqx_broker_helper_SUITE:t_shard_seq 2023-02-24 21:21:12 +01:00
William Yang
60532569a2 test(quic): fix flaky 2023-02-24 21:21:12 +01:00
Zaiming (Stone) Shi
5150407f7b chore: bump to version v5.0.18 2023-02-24 20:06:43 +01:00
Zaiming (Stone) Shi
b3f3bdeafe Merge remote-tracking branch 'origin/master' into release-50 2023-02-24 20:05:59 +01:00
Zaiming (Stone) Shi
df7e9db057
Merge pull request #10019 from qzhuyan/dev/william/quic-hidden-low-level-tunings 
230222 feat(quic): add hidden low level settings for listeners.
 2023-02-24 20:03:31 +01:00
Zaiming (Stone) Shi
80afb69bb2
Merge pull request #10015 from zmstone/0220-faile-fast-if-cookie-is-obviously-wrong 
0220 fail fast if cookie is obviously wrong
 2023-02-24 19:54:35 +01:00
Zaiming (Stone) Shi
79bf77c2f1
Merge pull request #10020 from zmstone/0222-fix-bridge-async-mode-counters 
fix(bridge): fix dropped counter and inflight gauge
 2023-02-24 19:51:24 +01:00
Andrew Mayorov
c883e4b36a
test: drop custom loop_wait in favor of snabkaffe's ?retry 2023-02-24 18:16:35 +03:00
Andrew Mayorov
2b4e49e7df
fix(bufworker): handle replies of simple async queries 
Before that change, simple queries were treated as "retries"
essentially, thus skipping all the reply processing there is.
 2023-02-24 15:06:49 +03:00
William Yang
fde19e2b4b docs(quic): fix typos 2023-02-24 10:20:28 +01:00
William Yang
f61c3c47ca docs(quic): add zh 2023-02-24 10:20:28 +01:00
William Yang
681fd80cab docs(quic): move desc to i18n. 2023-02-24 10:20:28 +01:00
William Yang
ee77c113aa test(quic): tuned listener 2023-02-24 10:20:28 +01:00
William Yang
ea65ec10bf feat(quic): add hidden low level settings for listeners. 2023-02-24 10:20:28 +01:00
Zaiming (Stone) Shi
c97d17cc91 test: refactor to loop wait for counters 2023-02-24 09:02:03 +01:00
Zaiming (Stone) Shi
a10dbba084 refactor(buffer_worker): less defensive on inflight counter decrement 2023-02-23 21:23:10 +01:00
Ilya Averyanov
5bfd690df6 fix(dashboard): fix swagger spec generation 2023-02-23 22:09:36 +02:00
Zaiming (Stone) Shi
7a6465e2cf fix(buffer_worker): ensure flush timer reset in blocked state 2023-02-23 21:06:38 +01:00
Zaiming (Stone) Shi
3a6dbbdd05 refactor(buffer_worker): ensure flsh message is never missed 2023-02-23 20:11:00 +01:00
Zaiming (Stone) Shi
dbfdeec5e9 fix(buffer_worker): log unknown async replies 2023-02-23 12:55:49 +01:00
Zaiming (Stone) Shi
356a94af30 fix(buffer_worker): ensure async flush message is sent 
This is a new issue introduced in the previous fix commits
after handling the partial expiry correctly, the
IsFullBefore check is no longer the state before the reply
is received but the state after a partially-expired batch
is shrinked.
The fix is simple, move the check to the entry-point of
where async reply callback enters, then send an async
'flush' notification regardless of the handling result.
 2023-02-23 09:47:34 +01:00
Zaiming (Stone) Shi
e9787c3ee3
Merge pull request #9966 from zmstone/0213-port-blackbox-cover-report-from-v4 
feat: add new module emqx_cover.erl
 2023-02-23 09:44:11 +01:00
Zaiming (Stone) Shi
3413af76be fix(emqx_misc): ensure flatten list for safe dir 2023-02-23 09:05:52 +01:00
Zaiming (Stone) Shi
713220f88b refactor(buffer_worker): more generic process for all_expired 2023-02-23 00:04:20 +01:00
Zaiming (Stone) Shi
036f69cd6e test: ensure batch size > 1 is covered in expiration test 2023-02-22 23:26:04 +01:00
Zaiming (Stone) Shi
bf8becd521 test: make sure gauge return to 0 in test cases 2023-02-22 23:07:12 +01:00
Zaiming (Stone) Shi
3458aee296
Merge branch 'master' into 0213-port-blackbox-cover-report-from-v4 2023-02-22 22:21:44 +01:00
Zaiming (Stone) Shi
fc614e16e5 fix(bridge): update inflight items after partial expiry 2023-02-22 22:05:56 +01:00
ieQu1
b0777ca73b chore(ekka): Bump version 2023-02-22 20:55:04 +01:00
Stefan Strigler
0cfa5e2ce1
Merge pull request #9994 from sstrigler/EMQX-3688-emqx-mgmt 
emqx mgmt code coverage and cleanup
 2023-02-22 14:30:36 +01:00
Zaiming (Stone) Shi
6a1085a842 chore: change 'emqx_ctl' usage to 'emqx ctl' 2023-02-22 13:57:55 +01:00
Zaiming (Stone) Shi
bb13d0708f fix(bridge): fix dropped counter and inflight gauge 
Prior to this fix there were two metrics issues
1. if a batch is all requests expired when receiving a reply
   it only bumped 1 instead of the batch size for 'late_reply'
2. when a batch is partially delivered (or expired), the
   dropped requests were not decremented from the inflight size gauge
 2023-02-22 13:20:58 +01:00
Zaiming (Stone) Shi
a1ba7bc004
Merge pull request #9976 from zmstone/0215-docs-improve-bridge_mode-description 
docs: improve bridge_mode description
 2023-02-21 20:55:20 +01:00
Stefan Strigler
967369075f
Merge pull request #10014 from sstrigler/EMQX-7822-5-0-get-api-v-5-monitor-nodes-node-must-return-404-when-node-is-not-responding 
fix: return `404` for unknown node names
 2023-02-21 17:09:03 +01:00
Stefan Strigler
a0589d5b95 fix: return 404 for unknown node names 2023-02-21 16:23:38 +01:00
Stefan Strigler
965d63f4f5 fix: schema for /gateways/:name/clients was missing top-level structure 2023-02-21 15:33:56 +01:00
Stefan Strigler
bf978efc83
Merge pull request #10009 from sstrigler/EMQX-7994-get-trace-name-log-bytes-xxx-does-not-do-input-validation 
get trace name log bytes xxx does not do input validation
 2023-02-21 14:12:57 +01:00
William Yang
969cbecf91
Merge pull request #10010 from qzhuyan/dev/william/new-emqtt-quicer 
chore: bump quicer 0.0.111 and emqtt 1.8.2
 2023-02-21 13:30:27 +01:00
Stefan Strigler
28382ec26b test: add test for pubsub api 2023-02-21 13:11:34 +01:00
William Yang
0a207856aa chore: bump quicer 0.0.111 and emqtt 1.8.2 2023-02-21 10:36:15 +01:00
Stefan Strigler
e78c2c2869 fix: return 404 in case node is not found 2023-02-21 10:20:34 +01:00
Stefan Strigler
9ecf154a71 fix: limit bytes param to signed 32bit int 
We still need to check if chunk we're reading fits in memory
 2023-02-21 10:20:14 +01:00
Stefan Strigler
8ae4440061 style: fix API description for bytes parameter 2023-02-21 09:41:32 +01:00
Stefan Strigler
81fad58f12 fix: re-add list_subscriptions/0 
Created a ticket to add an actual working implementation
 2023-02-21 09:41:00 +01:00
Stefan Strigler
71f3efb2ce style: fix empty line 2023-02-21 09:41:00 +01:00
Stefan Strigler
c44c7fcbce style: remove unused macro 2023-02-21 09:41:00 +01:00
Stefan Strigler
0482f43802 test: add test for banned and alarms 2023-02-21 09:41:00 +01:00
Stefan Strigler
0d357f7038 refactor: cleanup list subscriptions 2023-02-21 09:41:00 +01:00
Stefan Strigler
0d2ce85776 fix: return 'not found' for subscriptions of unknown client 2023-02-21 09:41:00 +01:00
Stefan Strigler
f3ced5d5eb refactor: kickout_client doesn't need a format fun 2023-02-21 09:41:00 +01:00
Stefan Strigler
a6d88c3caa test: more tests 2023-02-21 09:41:00 +01:00
Stefan Strigler
fc33bce40d test(emqx_mgmt): test list_nodes/0 2023-02-21 09:41:00 +01:00
Stefan Strigler
777ca72ad5 fix: don't crash on broker_info() 2023-02-21 09:41:00 +01:00
Stefan Strigler
f9895a33b6 refactor: rename max_row_limit to default_row_limit 2023-02-21 09:41:00 +01:00
Zaiming (Stone) Shi
9316690c29 fix(schema): binary string for default values 
A lot of the string value fields had default value defined in
schema as list-string rather than binary-string.
This caused the generated schema dump (in JSON format)
to have raw_default field as an integer array.
 2023-02-21 09:09:51 +01:00
William Yang
31cfd728c4 ci(quic): bump to quicer 0.0.109 for ubuntu22.04 prebuilds 2023-02-20 14:50:35 +01:00
William Yang
bd4a84ac0a test(quic): adapt to new emqtt reconnect mechanism. 2023-02-20 14:48:39 +01:00
William Yang
296e271b97 fix(quic): bump to emqtt 1.8.1 2023-02-20 10:53:51 +01:00
William Yang
cf72947f0a test(quic): use quic.ssl_options 2023-02-20 10:53:42 +01:00
William Yang
ebd0fb74a3 test(quic): by default, bind to port not IPv4 2023-02-20 10:43:24 +01:00
William Yang
3f7032fbe9 chore(quic): troubleshooting large payload 2023-02-20 10:43:21 +01:00
William Yang
fef0a9375c chore(quic): make spell check happy 2023-02-20 10:43:17 +01:00
William Yang
b81b62c639 chore(quic): doc about deprecated fields. 2023-02-20 10:43:17 +01:00
William Yang
45718dd77f chore(quic): debug flaky large payload tc. 2023-02-20 10:43:17 +01:00
William Yang
f106f30a96 chore: fix comments in emqx_connection 2023-02-20 10:43:17 +01:00
William Yang
c6c3bd0396 chore(quic): schema format fix 2023-02-20 10:42:55 +01:00
William Yang
0e40f6cf48 feat(quic): listener use common server ssl_options 2023-02-20 10:42:55 +01:00
William Yang
fc3e8715a1 feat(quic): bump to emqtt 1.8.0 2023-02-20 10:42:55 +01:00
William Yang
04f502fb54 feat(quic): support mTLS with 'verify' and 'cacertfile' 2023-02-20 10:42:55 +01:00
William Yang
c7efccb996 chore: bump emqtt 1.7.1-pre2 & quicer 0.0.108 2023-02-20 10:42:55 +01:00
William Yang
c457c1092b fix(quic): show QUIC listeners in dashboard 2023-02-20 10:42:55 +01:00
William Yang
3c73c6b7c6 feat(quic): bump quicer to 0.0.107 2023-02-20 10:42:55 +01:00
William Yang
0351b32cf4 test(quic): disable shutdown policy for large payload test 2023-02-20 10:42:55 +01:00
William Yang
f4f346e387 test(quic): fix flaky test 2023-02-20 10:42:55 +01:00
William Yang
db544cf9ad fix: emqtt vsn in rebar after rebase 2023-02-20 10:42:55 +01:00
William Yang
dc26790495 test(quic): trace why we get verify_peer 2023-02-20 10:42:55 +01:00
William Yang
f8fd201a8c test(quic): fix flaky test 2023-02-20 10:42:55 +01:00
William Yang
d8fa65ea09 fix(quic): handle timeout event in data stream 2023-02-20 10:42:55 +01:00
William Yang
38247a9d62 feat(quic): bump quicer to 0.0.106 2023-02-20 10:42:55 +01:00
William Yang
381eb8ec68 chore(quic): fix dialyzer 2023-02-20 10:42:55 +01:00
William Yang
282d1a6829 ci: build dialyzer PLT with quicer, jq and bcrypt 2023-02-20 10:42:55 +01:00
William Yang
88cdfcc4a6 test(quic): excl. multistream SUITE when BUILD_WITHOUT_QUIC 2023-02-20 10:42:55 +01:00
William Yang
de810e04fd chore(quic): clean test code 2023-02-20 10:42:55 +01:00
William Yang
98a72d40ce fix(emqx_connection): do not raise an exception for normal shutdown 2023-02-20 10:42:55 +01:00
William Yang
1692a16778 feat(quic): handle ctrl stream normal shutdown 2023-02-20 10:42:55 +01:00
William Yang
2a6cdd9da6 test(quic): enhance large payload test 2023-02-20 10:42:55 +01:00
William Yang
00f615a1e3 chore(quic): clean code 2023-02-20 10:42:55 +01:00
William Yang
22dcf5907e feat(quic): bump to quicer 0.0.104 2023-02-20 10:42:55 +01:00
William Yang
f65ac5422e test(quic): improve coverage 2023-02-20 10:42:55 +01:00
William Yang
5764994436 feat(quic): bump to quicer 0.0.103 2023-02-20 10:42:55 +01:00
William Yang
1e8b2e247e feat(quic): 0-RTT multi-streams data 2023-02-20 10:42:55 +01:00
William Yang
71d3148544 feat(quic): stream use active_n 10 2023-02-20 10:42:55 +01:00
William Yang
0173121a30 feat(quic): improve coverage and remove unused code 2023-02-20 10:42:55 +01:00
William Yang
0544a3ca0c fix(quic): setops on stream and handle peer needs stream 
- setopts should go for stream
- handle peer_needs_streams for none msquic clients
 2023-02-20 10:42:55 +01:00
William Yang
ceac5a0ec7 feat(quic): bump to quicer 0.0.101 2023-02-20 10:42:55 +01:00
William Yang
1840a7f923 test(quic): improve coverage 2023-02-20 10:42:55 +01:00
William Yang
5bdcb0562d feat(quic): workaround to flushing the send buffer after conn shutdown 
Could not find a way to ensure msquic flush the send buffer after
calling ConnectionShutdown.

So just close the ctrl stream and let conn owner shutdown the conn.
 2023-02-20 10:42:55 +01:00
William Yang
04a8a49dbe test: update testcase for new emqtt 2023-02-20 10:42:55 +01:00
William Yang
7d9bd33de9 feat(quic): bump quicer version to 0.0.100 2023-02-20 10:42:55 +01:00
William Yang
9b52beaee9 fix(quic): handle fast_close while handshake fail 2023-02-20 10:42:55 +01:00
William Yang
9f696928b6 feat(quic): multi streams 2023-02-20 10:42:55 +01:00
William Yang
a51c886908 fix: prepare for multi stream 2023-02-20 10:42:55 +01:00
William Yang
2d09a054e3 chore: add some typing 2023-02-20 10:42:55 +01:00
William Yang
00b59b4939 feat(quic): WIP multi-stream 2023-02-20 10:42:55 +01:00
Zaiming (Stone) Shi
6f1eaca253 chore: bump version to e5.0.1-alpha.1 2023-02-20 08:28:14 +01:00
ieQu1
388c33395e
Merge pull request #9939 from ieQu1/mria-replicant-manual-join 
Mria replicant manual join
 2023-02-17 11:32:58 +01:00
ieQu1
d9554c36ac chore: Bump umbrealla app versions 2023-02-17 09:50:50 +01:00
ieQu1
794bedef9f fix(emqx_ctl): Start CLI before ekka 2023-02-17 09:50:50 +01:00
Erik Timan
056bc71af2 chore: bump VSN version 2023-02-16 15:05:38 +01:00
Erik Timan
2442a4dea7 test(emqx_resource): add regression test for recursive flushing 2023-02-16 14:17:16 +01:00
Erik Timan
dcf70e0e68 refactor(emqx_resource): add more trace points for flushing 2023-02-16 14:17:16 +01:00
Thales Macedo Garitezi
6bf385793c
Merge pull request #9978 from thalesmg/fix-psql-tls-v50 
fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0)
 2023-02-16 09:20:32 -03:00
Zhongwen Deng
82b2f9f245 chore: include emqx_management in emqx_prometheus/emqx_statsd 2023-02-16 11:37:00 +08:00
Thales Macedo Garitezi
dbda504f2c fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0) 
Fixes https://github.com/emqx/emqx/issues/9907

At v5.0.14, we changed the `ssl` option for the Postgres connector
from `true` to `required`, but there was another transformation in
`conn_opts/2` that led to an incorrect configuration.  This change
ended up preventing users from connecting to Postgres with their
previous configurations after upgrading EMQX.
 2023-02-15 14:06:33 -03:00
Zaiming (Stone) Shi
719e02cc96 fix(metrics): use the same data source for RAM usage info 2023-02-15 16:11:39 +01:00
Zaiming (Stone) Shi
1cf88ea0b4 docs: improve bridge_mode description 2023-02-15 15:24:10 +01:00
Stefan Strigler
293df88f36
Merge pull request #9965 from sstrigler/EMQX-3689-emqx-mgmt-api 
EMQX 3689 emqx mgmt api
 2023-02-15 15:13:17 +01:00
Stefan Strigler
921b5b2c5c style: rename function for compliance 2023-02-15 13:38:14 +01:00
Stefan Strigler
21b8ff6d14 test: use unit test to test paginate 2023-02-15 13:31:48 +01:00
lafirest
3108b6f9dc
Merge pull request #9932 from lafirest/feat/tdengine 
feat(bridges): add TDengine
 2023-02-15 20:01:50 +08:00
William Yang
02f15be20f
Merge pull request #9967 from qzhuyan/perf/william/add-tls-common-conf-hibernate-after 
perf: add tls common conf hibernate after
 2023-02-15 12:02:05 +01:00
Stefan Strigler
9be73bfff0 fix: support ipv6 at ip_port 2023-02-15 10:37:13 +01:00
William Yang
4e6c7bfcbe chore: changelog for hibernate_after 2023-02-14 16:26:58 +01:00
William Yang
d424d77e84 chore(TLS): hibernate_after default to 5s, follow version 4.x 2023-02-14 16:26:58 +01:00
William Yang
572e50e0a2 perf: add TLS config hibernate_after, reduce memory footprint 2023-02-14 16:26:55 +01:00
firest
81adb02402 chore: bump apps version 2023-02-14 22:04:29 +08:00
firest
530b21268a feat(bridges): add TDengine 2023-02-14 22:04:23 +08:00
Stefan Strigler
4417ea9db7 style: missing comma 2023-02-14 09:13:45 +01:00
JianBo He
0d63cfdc97 chore: remove changes to 5.0.18 2023-02-14 15:34:12 +08:00
JianBo He
36b1c3eb5b fix(api): fix bad return error message if client id is not found 2023-02-14 15:30:55 +08:00
Zaiming (Stone) Shi
ba65cf48c3 feat: add new module emqx_cover.erl 2023-02-13 22:31:18 +01:00
Stefan Strigler
8b47acd58d test: page overflow and bad rpc in cluster 2023-02-13 17:46:26 +01:00
Stefan Strigler
8d34a72ab6 test: add test for clustered badrpc 2023-02-13 17:14:15 +01:00
Stefan Strigler
a28588ebf0 fix: bad spec 2023-02-13 17:14:02 +01:00
Stefan Strigler
89630a8a74 test: query bad node 2023-02-13 16:09:04 +01:00
Stefan Strigler
0c87d238e9 chore: bump version 2023-02-13 15:46:27 +01:00
Stefan Strigler
9e954585ae test: add node query tests and some more 2023-02-13 15:45:37 +01:00
Stefan Strigler
0e9289e48c refactor: remove unused clause 2023-02-13 15:45:37 +01:00
Stefan Strigler
743ced17c9 test: add test for more types in schema 2023-02-13 15:45:37 +01:00
Stefan Strigler
28560e4603 refactor: remove unused code 2023-02-13 15:45:37 +01:00
Zaiming (Stone) Shi
7d9b0f4826 chore: bump version to v5.0.17 2023-02-13 08:53:09 +01:00
Zaiming (Stone) Shi
a11217aa0f chore: bump version to v5.0.17 2023-02-12 11:36:53 +01:00
Zaiming (Stone) Shi
72fe10f18b chore: fix spell check errors 2023-02-10 19:41:14 +01:00
Zaiming (Stone) Shi
fb61c2b266 perf: avoid getting metrics (gen_server:call) for each resource lookup 2023-02-10 19:40:37 +01:00
Andrew Mayorov
fe450ca2d9
Merge pull request #9938 from keynslug/feat/mqtt-bridge-async-errors 
feat(mqtt-bridge): report recoverable errors of async queries
 2023-02-10 20:40:58 +04:00
Andrew Mayorov
2ed54e560f
Merge pull request #9952 from keynslug/fix/mqtt-bridge-no-qos2-clean-start 
fix(mqtt-bridge): disallow QoS 2 on ingress bridges
 2023-02-10 20:40:37 +04:00
Andrew Mayorov
cbb2885499
fix(mqtt-bridge): allow to configure clean_start for ingresses 2023-02-10 19:15:48 +03:00
Stefan Strigler
be615a2bb1
Merge pull request #9951 from sstrigler/EMQX-8929-handle-error-results-in-bridge-api-for-multicall-operations 
fix(emqx_bridge): unwrap rpc multicall result
 2023-02-10 16:26:50 +01:00
Andrew Mayorov
7002fe2ef4
fix(mqtt-bridge): disallow QoS 2 on ingress bridges 2023-02-10 17:17:59 +03:00
Zaiming (Stone) Shi
5d59fcbf23
Merge pull request #9950 from zhongwencool/prometheus_push_server_url_check 
fix: add push_gateway_server's  validator
 2023-02-10 12:41:50 +01:00
Stefan Strigler
e196d6f42a fix(emqx_bridge): unwrap rpc multicall result 2023-02-10 12:00:20 +01:00
Zhongwen Deng
53488b4377 fix: add push_gateway_server's validator 2023-02-10 17:52:16 +08:00
Andrew Mayorov
8bd7c0eb3f
feat(mqtt-bridge): report recoverable errors of async queries 
This should help to avoid delivery failures of messages which could
be safely retried, in the event of intermittent connectivity loss
for example.

It should now be safe since 73d5592b.
 2023-02-10 12:26:59 +03:00
Zaiming (Stone) Shi
d1eb788ee1
Merge pull request #9916 from qzhuyan/fix/william/bridge-mqtt-TLS-verify-wildcard-peer 
fix(mqtt-bridge): support verify TLS wildcard cert
 2023-02-10 09:39:35 +01:00
Zaiming (Stone) Shi
3e85cb3804
Merge pull request #9946 from zmstone/0209-fix-add-back-deprecate-field 
fix(config): add back reconnect_interval field as deprecated
 2023-02-10 09:19:12 +01:00
Zaiming (Stone) Shi
40f719a34d
Merge pull request #9936 from id/feat-disable-disksup 
feat: disable disksup by default
 2023-02-10 09:17:13 +01:00
Zaiming (Stone) Shi
42dfaf3ef2
Merge pull request #9910 from sstrigler/EMQX-8861-improve-bridge-restart-button-behaviour 
EMQX 8861 improve bridge restart button behaviour
 2023-02-09 18:00:48 +01:00
Zaiming (Stone) Shi
4e71f1cbc5
Merge pull request #9940 from zmstone/0208-ensure-no-leading-br-in-docs 
Add style checks for i18n.conf files
 2023-02-09 17:37:44 +01:00
Zaiming (Stone) Shi
5328955ff4 fix(config): add back reconnect_interval field as deprecated 2023-02-09 17:30:09 +01:00
Erik Timan
3dea6b7913
Merge pull request #9930 from olcai/add-live-connections-to-prometheus-v5 
feat(emqx_prometheus): expose live_connections stats to prometheus
 2023-02-09 16:48:42 +01:00
Andrew Mayorov
48efe552d1
test(mqtt-bridge): manage test flaps by waiting for flush events 2023-02-09 14:31:42 +03:00
Zaiming (Stone) Shi
157c919ba1 ci: add i18n style check script 2023-02-09 11:41:52 +01:00
ieQu1
123fea44de chore(ekka): Bump version to 0.14.0 2023-02-08 17:35:14 +01:00
Zaiming (Stone) Shi
5df667738a docs: fix bad i18n files 2023-02-08 17:25:22 +01:00
Ivan Dyachkov
fdb802975c feat: disable disksup by default 2023-02-08 15:33:35 +01:00
Stefan Strigler
f89ac54b17 style: remove stale comment 2023-02-08 14:04:09 +01:00
Stefan Strigler
a3fd0897bc refactor: less code duplication 2023-02-08 14:04:09 +01:00
Stefan Strigler
502f62e18d fix: try old 'restart' behavior if 'start' is not implemented 2023-02-08 13:19:41 +01:00
Andrew Mayorov
81b1bab11e
chore: bump emqx_resource version to 0.1.7 
Also add the changelog entry.
 2023-02-08 14:21:30 +03:00
Andrew Mayorov
c6fc0ec8cd
fix(bufworker): do not avoid retry if inflight table is full 
Otherwise there's no other piece of code that would retry the inflight
queries in that case.
 2023-02-08 14:08:04 +03:00
Andrew Mayorov
d8d06a260f
test(buffer): add test on inflight overflow w/ async queries 
This testcase should verify that the buffer will retry all inflight
queries failed with recoverable errors + flush all outstanding queries.

Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
 2023-02-08 14:08:04 +03:00
Stefan Strigler
cec8afe1b4
Merge pull request #9875 from sstrigler/EMQX-7119-fix-upload-broken-plugin 
EMQX 7119 fix upload broken plugin
 2023-02-08 09:38:52 +01:00
Zaiming (Stone) Shi
d628aba079
Merge pull request #9718 from zmstone/0108-refactor-authn-schema-union-member-selector 
0108 refactor authn schema union member selector
 2023-02-07 16:45:47 +01:00
William Yang
6113d467c1 fix(mqtt-bridge): support verify TLS wildcard cert 2023-02-07 16:40:55 +01:00
Erik Timan
784ec0c0a5 chore: bump VSN 2023-02-07 15:54:01 +01:00
Erik Timan
19c2288a4b feat(emqx_prometheus): expose live_connections stats to prometheus 2023-02-07 15:42:55 +01:00
Stefan Strigler
86f3f5787f feat: allow to manually re-connect disconected bridge 2023-02-07 11:58:30 +01:00
Zaiming (Stone) Shi
537ff30550 chore: update code comment 2023-02-07 11:40:17 +01:00
Zaiming (Stone) Shi
d487299b75 refactor: remove dead code 2023-02-07 08:34:10 +01:00
Zaiming (Stone) Shi
4653ca65f7 fix(logger): fix REPORT_CB/2 CRASH logs 2023-02-06 23:46:49 +01:00
Zaiming (Stone) Shi
dc8f0c5101 refactor(authn): no need to check config at app boot 2023-02-06 18:19:22 +01:00
Zaiming (Stone) Shi
83e66da2aa fix(authn): throw exception on invalid config from the API 2023-02-06 14:12:50 +01:00
Stefan Strigler
758d19f1e0 chore: bump vsn 2023-02-06 10:44:06 +01:00
Stefan Strigler
4e12a44ee6 fix: return 204 instead of 200 if there's no body content 2023-02-06 09:47:07 +01:00
Stefan Strigler
847f899fa0 test: ensure bad plugin upload gets deleted 2023-02-06 09:46:16 +01:00
Stefan Strigler
e135f8654c test: enable basic plugins API tests 2023-02-06 09:46:16 +01:00
Stefan Strigler
a866c995f5 test: add test for arbitrary content in tar file 2023-02-06 09:46:16 +01:00
Stefan Strigler
70c4e12b68 fix: cleanup after upload of broken plugin 2023-02-06 09:46:16 +01:00
Stefan Strigler
618ec04340 style: fix comment 2023-02-06 09:46:16 +01:00
Zaiming (Stone) Shi
a9c650da66 refactor: for better test coverage 2023-02-04 07:59:49 +01:00
Zaiming (Stone) Shi
2ebc89e339 refactor: authn schema union selector 2023-02-04 07:58:43 +01:00
Zaiming (Stone) Shi
4f91bf415c fix: schema check error reason pattern 2023-02-04 07:58:06 +01:00
Zaiming (Stone) Shi
65319567cc chore: no type check for 3rd party auth 2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi
fbc52330f3 refactor: do not re-check config in post_config_update 
now that the type is not lazy, it should have already been checked
when reaching the post_config_update step
 2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi
6aaff6211f feat: use union member selector for authn config schema 2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi
15035f7eb0 refactor: remove lazy type for authentication 
The idea of using lazy type for authentication was to make the
config check easy to extend at runtime. However the reality
is: it's overly complicated.

It's more likely that people will just continue to implementing
the auth hook callbacks instead of injecting config schema
at runtime.
 2023-02-04 07:56:29 +01:00
Zaiming (Stone) Shi
ccb0c7ebaa Merge remote-tracking branch 'origin/release-50' into 0202-merge-release-50-back-to-master 2023-02-03 16:12:11 +01:00
Zhongwen Deng
d45a0cb2ee chore: remove <<>> when logging username 2023-02-03 16:54:18 +08:00
Zaiming (Stone) Shi
1e0fe270bc chore: bump emqx app vsn to 5.0.17 2023-02-02 20:32:09 +01:00
Zaiming (Stone) Shi
94768c9f44 Merge remote-tracking branch 'origin/master' into 0202-merge-release-50-back-to-master 2023-02-02 20:21:26 +01:00
Zaiming (Stone) Shi
5fe3a0b1b3
Merge pull request #9898 from zmstone/0202-upgrade-to-dashboard-e1.0.3 
chore: upgrade dashboard to v1.1.7 for ce and e1.0.3 for ee
 2023-02-02 17:41:52 +01:00
Zaiming (Stone) Shi
42462db8b8 chore: bump version to v5.0.16 and e5.0.0 2023-02-02 17:26:00 +01:00
Zaiming (Stone) Shi
3d6756cbe8 test: fix addrinuse errors in listeners API test SUITE 2023-02-02 16:37:56 +01:00
Zaiming (Stone) Shi
a71d983ff8 test: fix assertMatch patter to make rebar3 fmt work 2023-02-02 16:37:45 +01:00
Zaiming (Stone) Shi
7ea140599a
Merge pull request #9894 from id/ci-always-run-static-checks 
ci: always run static_checks
 2023-02-02 16:33:19 +01:00
Zaiming (Stone) Shi
feca4cc0a5
Merge pull request #9892 from zmstone/0202-docs-cosmetic 
0202 docs cosmetic
 2023-02-02 15:43:58 +01:00
Zaiming (Stone) Shi
85fb53dcf0 refactor: use compile switch to diff functions between ce and ee 2023-02-02 15:15:31 +01:00
Zaiming (Stone) Shi
0d7cd2f0c5
docs: Update apps/emqx_conf/i18n/emqx_conf_schema.conf 
Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
 2023-02-02 14:20:07 +01:00
Zaiming (Stone) Shi
980ba2fd35
docs: Update apps/emqx/i18n/emqx_schema_i18n.conf 
Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
 2023-02-02 14:19:37 +01:00
Zaiming (Stone) Shi
0e299642cb chore: function renamed but forgot to update xref excemption 2023-02-02 14:16:23 +01:00
Zaiming (Stone) Shi
58627b7958 chore(emqx_resource_manager): ignore unused return value for dialyzer 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi
9a9943d35d chore(emqx_mgmt_cache): ignore unused return value for dialyer 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi
1c748070c1 fix(emqx_connector_mqtt_worker): publish_async can only return ok # Please enter the commit message for your changes. Lines starting 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi
c0d478bd41 fix(buffer_worker): type spec 2023-02-02 14:11:12 +01:00
zhongwencool
ee852d8204
Merge pull request #9886 from zhongwencool/mongo-connection-default-async 
fix: remove async mode from mongodb/redis/mysql/pgsql bridge
 2023-02-02 21:08:01 +08:00
Zaiming (Stone) Shi
d5c482b0b0 docs: remove timer unit from description 
the user input has time unit. e.g. "5s" for 5 seconds etc.
 2023-02-02 13:49:20 +01:00
Zaiming (Stone) Shi
5dbee76af4
Merge pull request #9890 from zmstone/0202-fix-kafka-status-check 
0202 fix kafka status check
 2023-02-02 13:48:44 +01:00
firest
031b36da32 feat(banned): filter out banned messages from mqueue 2023-02-02 20:20:52 +08:00
Zaiming (Stone) Shi
2b308af354
Merge pull request #9889 from keynslug/fix/EMQX-8857/force-clean-start-mqtt-bridge 
fix(mqtt-bridge): stop respecting `clean_start` config parameter
 2023-02-02 12:55:58 +01:00
Zaiming (Stone) Shi
9864587389 fix: send to buffer-supported connector even when disconnected 2023-02-02 12:04:17 +01:00
Zaiming (Stone) Shi
13ef30c46c
Merge pull request #9884 from savonarola/resource-fixes 
fix(resources): fix resource lifecycle
 2023-02-02 12:02:34 +01:00
Andrew Mayorov
0912f13c1f
fix(mqtt-bridge): stop respecting clean_start config parameter 
We are ignoring the user configuration because there's currently no
reliable way to ensure proper session recovery according to the MQTT
spec.
 2023-02-02 12:55:10 +03:00
Kjell Winblad
2cf193e2fd
Merge pull request #9839 from kjellwinblad/kjell/fix/Authorization_header_log_leak_webhook/EMQX-8791 
fix: Authorization header leak in log entries for webhook
 2023-02-02 10:38:42 +01:00
Zhongwen Deng
1c9035d24c test: remove async from redis ct 2023-02-02 17:37:18 +08:00
Zhongwen Deng
22cc1cc745 fix: make spell_check happy 2023-02-02 17:37:18 +08:00
Zhongwen Deng
f8936013b7 chore: replace async with sync 2023-02-02 17:37:18 +08:00
Zhongwen Deng
22c3f50020 fix: add query_mode_sync_only for mysql pgsql redis mongodb bridge 2023-02-02 17:37:18 +08:00
Andrew Mayorov
ca5c192f4b
Merge pull request #9882 from fwup/fix/no-mqtt-bridge-middleman 
refactor(mqtt-worker): avoid unnecessary abstraction
 2023-02-02 13:11:31 +04:00
zhongwencool
e5bc787467
Merge pull request #9872 from zmstone/0131-improve-help-text 
docs: improve docs for system monitoring config
 2023-02-02 14:45:03 +08:00
Zhongwen Deng
1a90c1654c chore: bad typo 2023-02-02 11:43:04 +08:00
Zaiming (Stone) Shi
6fd30a7e81 docs: improve docs for system monitoring config 2023-02-01 20:21:08 +01:00
Ilya Averyanov
14f528cc86 fix(resources): fix resource lifecycle 
* do not resume all buffer workers on successful healthcheck
* do not pass undefined state to resource healthcheck callback
 2023-02-01 18:26:13 +02:00
Zaiming (Stone) Shi
ec3031271b
Merge pull request #9883 from zmstone/0201-upgrade-enterprise-dashboard-to-e1.0.2 
0201 upgrade enterprise dashboard to e1.0.2
 2023-02-01 16:26:21 +01:00
Zaiming (Stone) Shi
32d18ec594
Merge pull request #9878 from zmstone/0201-upgrade-ehttpc-0.4.6 
fix(ehttpc): fix crash when webhook bridge uses `hash` pool type.
 2023-02-01 15:00:28 +01:00
Andrew Mayorov
5fd7f65a1f
test(bufworker): make testcase simpler to follow 
The confusion was due to the fact that subsequent query was missing
`async_reply_fun` and thus, was not accumulating in the results.
 2023-02-01 16:52:47 +03:00
Zaiming (Stone) Shi
38ef37d109 chore: bump release version to e5.0.0-rc.4 2023-02-01 14:52:08 +01:00
Andrew Mayorov
8a46cb974e
test(mqtt-bridge): test async bridge reconnects seamlessly 2023-02-01 16:28:16 +03:00
Andrew Mayorov
5ebceb20d2
test(mqtt-bridge): also test reconfiguration 2023-02-01 16:23:58 +03:00
Andrew Mayorov
ad88938d34
refactor: reuse some parts of test code for brewity 2023-02-01 16:22:19 +03:00
Andrew Mayorov
f0395be383
refactor(mqtt-worker): avoid unnecessary abstraction 
So the code is easier to follow.
 2023-02-01 14:46:26 +03:00
zhongwencool
13511d2782
Merge pull request #9856 from zhongwencool/alarm-percent-decimals 
fix: /api/nodes is timeout when emqx in high load
 2023-02-01 19:40:53 +08:00
Andrew Mayorov
979e9804be
Merge pull request #9879 from keynslug/fwup/fix/buffer-worker-testcase 
test(bufworker): fix testcase flapping due to data races
 2023-02-01 15:33:59 +04:00
Andrew Mayorov
ff473e0f1b
test(bufworker): fix testcase flapping due to data races 2023-02-01 12:57:46 +03:00
Zaiming (Stone) Shi
96a18e7105 chore: upgrade to hocon 0.35.3 2023-02-01 10:52:01 +01:00
Zhongwen Deng
6382915036 chore: repalce fresh by refresh 2023-02-01 17:36:30 +08:00
Zaiming (Stone) Shi
9f6b6cedc6 fix(webhook): pick worker according to pool type 2023-02-01 09:53:25 +01:00
Ilya Averyanov
9d91ebe266
Merge pull request #9842 from savonarola/fix-redis-cluster-recover 
fix: fix redis cluster resource recovery
 2023-02-01 10:38:52 +02:00
Zaiming (Stone) Shi
f4381d90ca refactor: http cliet should not need to retry HTTP requests for bridge 2023-02-01 09:30:30 +01:00
Zaiming (Stone) Shi
6ca816f7a4 docs: remove impl details from user doc 2023-02-01 09:30:10 +01:00
Zhongwen Deng
3c4d09a752 fix: get_memory_cache return {ok, Mem} | stale 2023-02-01 16:00:55 +08:00
Zhongwen Deng
ced55719ef chore: only run t_sys_mem_check ct in linux 2023-02-01 14:47:41 +08:00
Zhongwen Deng
56b9238645 fix: only cache sys_memory in linux 2023-02-01 12:21:34 +08:00
Zhongwen Deng
95ef1e806c chore: don't crash when add_handler failed 2023-02-01 11:24:27 +08:00
Zhongwen Deng
9cbad5297c fix: primary logger level not update 2023-02-01 11:16:30 +08:00
Zhongwen Deng
71f00f2962 test: ctl suite failed 2023-02-01 10:07:22 +08:00
Zhongwen Deng
7a9f477267 feat: use emqx_mgmt_cache to cache sys_memory 2023-02-01 10:07:22 +08:00
Zhongwen Deng
c2bdb9faa7 test: multiple_nodes case failed 2023-02-01 10:07:22 +08:00
Zhongwen Deng
b6e6315b50 feat: change loads from string to float 2023-02-01 10:07:22 +08:00
Zhongwen Deng
0b19be074c feat: cache OTP_VERSION in persistent_term 2023-02-01 10:07:22 +08:00
Zhongwen Deng
6162f90610 fix: don't crash when OTP_VERSION file is missing 2023-02-01 10:07:22 +08:00
Zhongwen Deng
5783127c30 test: cpu_sup:load mock test 2023-02-01 10:07:22 +08:00
Zhongwen Deng
2d67bb3fb6 fix: /api/nodes is timeout if emqx in high load 2023-02-01 10:07:22 +08:00
Zaiming (Stone) Shi
7e8253e3af chore: bump version to e5.0.0-rc.3 2023-01-31 20:45:28 +01:00
Zaiming (Stone) Shi
78028a7fcf
Merge pull request #9876 from zmstone/0131-avoid-persisting-logging-configs-if-no-change 
0131 avoid persisting logging configs if no change
 2023-01-31 20:43:32 +01:00
Zaiming (Stone) Shi
b3ad9e97d2
Merge pull request #9870 from keynslug/fix/mqtt-connection-loss-feedback 
feat(mqtt-bridge): avoid middleman process
 2023-01-31 19:12:18 +01:00
Zaiming (Stone) Shi
ddfbc0a19f refactor: refresh logger config before starting listeners 2023-01-31 18:50:16 +01:00
Zaiming (Stone) Shi
605d9972e4 fix(config): avoid calling emqx:update_config/2 for logger refresh 
Prior to this fix, whatever configured for loggers are persisted to
cluster-override.conf.
This may cause troulbe for users who changes boot mode.

For example if the node is once started in console mode,
the logging config is persisted, with console enabled, but file
disabled.
Then if the user decide to start in daemon mode, all the logs
will silently go to erlang.log.N and emqx.log.N will be empty.

After this fix, only changes really made into cluter-override.conf
will take effect.
 2023-01-31 18:50:16 +01:00
Zaiming (Stone) Shi
b0ac924ca9 refactor: less copy-paste 2023-01-31 18:40:55 +01:00
lafirest
3d59be6ab6
Merge pull request #9871 from lafirest/fix/authz_rules 
feat(authz): allow the placeholder to be anywhere in the topic for authz rules
 2023-02-01 00:12:06 +08:00
Andrew Mayorov
c5a7cd5acd
fix(mqtt-bridge): drop unused configuration parameter 2023-01-31 18:30:42 +03:00
Andrew Mayorov
c76311c9c3
fix(buffer): count inflight batches properly 2023-01-31 18:30:42 +03:00
firest
9f4c36ecbc chore: bump version && update changes 2023-01-31 23:30:22 +08:00
firest
c034cbf6de feat(authz): allow the placeholder to be anywhere in the topic for authz rules 2023-01-31 23:30:12 +08:00
Andrew Mayorov
d0c10b59aa
feat(mqtt-bridge): avoid middleman process 
Instead, supervise `emqtt` client process directly.
 2023-01-31 17:59:03 +03:00
Ilya Averyanov
fce1e74c3d fix(connector): fix redis cluster resource recovery 2023-01-31 16:55:05 +02:00
Zaiming (Stone) Shi
f6dafc20ea feat: export API to read cluster and local overrides 2023-01-31 14:20:34 +01:00
Stefan Strigler
ade5316419
Merge pull request #9802 from sstrigler/EMQX-8469-enable-proxy-protocol-support-in-emqx-dashboard-schema 
feat: support HAProxy protocol for dashboard API
 2023-01-31 14:19:56 +01:00
Erik Timan
3786bb8086
Merge pull request #9824 from olcai/internal-error-api-get-topic 
fix(emqx_management): handle multiple routes in topics/{topic} API
 2023-01-31 13:50:42 +01:00
Andrew Mayorov
4d146c521b
fix(mqtt-bridge): ensure proper feedback on async forwards 
So that buffer worker would notice a connection loss in time, and
recycle inflight messages subsequently.
 2023-01-31 11:09:39 +03:00
Andrew Mayorov
35c429ef1d
refactor: drop a couple of unused macros / includes 2023-01-31 11:09:39 +03:00
Zaiming (Stone) Shi
b3e486041b
Merge pull request #9853 from zmstone/0127-refactor-buffer-worker-no-need-to-keep-request-for-reply-callback 
0127 refactor buffer worker no need to keep request for reply callback
 2023-01-31 08:44:01 +01:00
Erik Timan
5b3a77e3c7 test(emqx_management): fix flaky route handling in get topic test 
This reworks a test case to use a second slave node. This ensures that
an added route is permanently in the routing table. The old version
reverted the routing table quickly since the node name given wasn't a
real node.
 2023-01-31 08:28:55 +01:00
Erik Timan
03cabf6b26 chore: bump app VSN 2023-01-31 08:27:54 +01:00
Erik Timan
33e011aff5 fix(emqx_management): handle multiple routes in topics/{topic} API 
The topics/{topic} API endpoint would return 500 - Internal Error if a
topic had multiple routes. This is now fixed by returning a list of
routes.
 2023-01-31 08:27:33 +01:00
Ilya Averyanov
1a1c3afa81
Merge pull request #9852 from savonarola/fix-redis-desc 
Fix Redis configuration field description
 2023-01-30 23:26:37 +02:00
Stefan Strigler
3ce6cbcd6a chore: bump vsn 2023-01-30 18:09:10 +01:00
Stefan Strigler
0b1483040a test: add actual test for haproxy 2023-01-30 18:07:06 +01:00
lafirest
e288424924
Merge pull request #9864 from lafirest/fix/exclusive-unsub-5.0 
fix: the exclusive topics aren't removed when the session has already been cleaned
 2023-01-31 00:33:45 +08:00
Kjell Winblad
0c4134c423 test: add unit test case for redact function in http connector 2023-01-30 11:51:28 +01:00
Stefan Strigler
27881064dc fix: increase dropped.queue_full by number of messages 2023-01-30 11:37:35 +01:00
lafirest
3e6cc40525
Merge pull request #9843 from lafirest/fix/speed_up_test 
fix: speed up banned test cases
 2023-01-30 18:17:27 +08:00
firest
3bdffca488 fix: the exclusive topics aren't removed when the session has already been cleaned 2023-01-30 18:17:00 +08:00
Zhongwen Deng
bb636394e1 chore: add debug log for raw data 2023-01-30 16:35:06 +08:00
Zhongwen Deng
b73d11675e chore: log the bad mqtt packet(frame error) 2023-01-30 15:05:49 +08:00
Zhongwen Deng
3d07271ea5 fix: crash when mfa not found 2023-01-30 14:30:20 +08:00
Zhongwen Deng
25090563af chore: use brackets to wrap the mqtt packet when logging 2023-01-30 12:29:00 +08:00
Zhongwen Deng
ce32ea7334 feat: Make the log output format order fixed 2023-01-30 12:18:22 +08:00
Zhongwen Deng
6cbad047cd fix: don't log CONNECT twice when debug 2023-01-30 12:16:17 +08:00
Zaiming (Stone) Shi
d47941601d refactor(buffer_worker): rename trace points 2023-01-28 11:52:11 +01:00
Zaiming (Stone) Shi
7f66c6a9e2
Merge pull request #9840 from olcai/redact-influxdb-tokens 
fix: redact influxdb tokens in logs and reduce log level
 2023-01-28 11:47:36 +01:00
Zaiming (Stone) Shi
fc38ea9571 refactor(buffer_worker): do not keep request body in reply context 
the request body can be potentially very large
the reply context is sent to the async call handler and kept
in its memory until the async reply is received from bridge
target service.

this commit tries to minimize the size of the reply context
by replacing the request body with `[]`.
 2023-01-27 17:12:55 +01:00
Ilya Averyanov
72f39b9b72 fix(docs): correct Redis conf field description 2023-01-27 17:39:16 +02:00
Zaiming (Stone) Shi
578271ea3d refactor: use lists:map instead of lc for safty 2023-01-27 15:15:46 +01:00
Zaiming (Stone) Shi
f793807bc1 refactor(buffer_worker): rename function 
batch_reply_after_query to handle_async_batch_reply
 2023-01-27 15:04:28 +01:00
Zaiming (Stone) Shi
262c3a2869 refactor(buffer_worker): rename function 
from reply_after_query to handle_async_reply
 2023-01-27 15:03:18 +01:00
Zaiming (Stone) Shi
6a58bafcb0 chore: bump release version to e5.0.0-rc.2 2023-01-27 14:38:21 +01:00
Zaiming (Stone) Shi
52b75ada04
Merge pull request #9832 from sstrigler/EMQX-8774-failure-to-handle-timeout-error-in-resource-worker 
EMQX 8774 failure to handle timeout error in resource worker
 2023-01-27 14:36:44 +01:00
Zaiming (Stone) Shi
514609bcf7
Merge pull request #9850 from zmstone/0127-fix-influxdb-bridge-atom-leak 
0127 fix influxdb bridge atom leak
 2023-01-27 14:30:20 +01:00
Zaiming (Stone) Shi
96ed725a55
Merge pull request #9849 from zmstone/0127-refactor-buffer-worker-simplify-caller-reply 
0127 refactor buffer worker simplify caller reply
 2023-01-27 14:06:56 +01:00
Zaiming (Stone) Shi
c47be57c59 fix(bridge): ensure all bridge resources are stopped before app stop 2023-01-27 12:39:05 +01:00
Zaiming (Stone) Shi
d53106145f fix: stop resource when resource manager terminates 2023-01-27 12:39:05 +01:00
Andrew Mayorov
d35e46b2d5
Merge pull request #9838 from keynslug/fix/redis-cluster-batching 
feat(redis): disable batching in redis_cluster bridges
 2023-01-27 15:27:57 +04:00
Stefan Strigler
7005b71ddf style: fix typo in comment 2023-01-27 11:43:51 +01:00
Stefan Strigler
2d62de5188 test: fix expected result from timeout error 2023-01-27 11:43:48 +01:00
Stefan Strigler
a180bd9aa5 fix: catch error, not exit 2023-01-27 11:40:06 +01:00
Stefan Strigler
b7e3f9d5a6 fix: try-case-of rather than try-of 
try-of catches only what happens within but not after
 2023-01-27 11:40:06 +01:00
Zaiming (Stone) Shi
db2f631a8a refactor(buffer_worker): simplify caller reply 2023-01-27 11:33:45 +01:00
Zaiming (Stone) Shi
965236c888
Merge pull request #9845 from zmstone/0126-reply-caller-for-buffer-overflow-queue-items 
0126 reply caller for buffer overflow queue items
 2023-01-27 11:32:59 +01:00
Zaiming (Stone) Shi
d4fab92b72 refactor(buffer_worker): no need to keep request for REPLY macro 2023-01-27 10:41:30 +01:00
Andrew Mayorov
71f996b9d5
refactor(mqtt-bridge): unwrap single statem actions 
So that the code would be easier to follow and harder to break.
Also drop a couple of unused macrodefs.
 2023-01-26 21:13:18 +03:00
Andrew Mayorov
c95f979413
fix(mqtt_bridge): use correct gen_statem reply action 2023-01-26 21:12:05 +03:00
Zaiming (Stone) Shi
1f799dfd59 fix: reply with {error, buffer_overflow} when discarded 2023-01-26 17:15:36 +01:00
Zaiming (Stone) Shi
ed28789164 refactor(buffer_worker): no need to return after collect into buf queue 2023-01-26 14:50:40 +01:00
Zaiming (Stone) Shi
25b4821adc refactor: move the the per-message overflow log from error to info level 2023-01-26 14:48:43 +01:00
Zaiming (Stone) Shi
bb26632c8a fix(buffer_worker): fix a wrong assertion 
the assertion is to ensure queue items are not binary
but should not assert the queue itself
 2023-01-26 14:33:16 +01:00
Zaiming (Stone) Shi
f6b3b930b0 chore: improve a error log 2023-01-26 14:21:27 +01:00
Kjell Winblad
86cfbfb43c fix: Authorization header leak in log entries for webhook 
There might be another possibility for leakage. If the resource mangager
for the webhook resource crashes, OTP might log the spec for the
resource manager which contains the Config and thus the Authorization
header. This is probably an issue for other resources as well and should
be fixed in another commit. The following issue has been created for
that:

https://emqx.atlassian.net/browse/EMQX-8794

Fixes:
https://emqx.atlassian.net/browse/EMQX-8791
 2023-01-26 12:44:16 +01:00
firest
908ac8d948 fix: speed up banned test cases 2023-01-26 13:04:55 +08:00
Andrew Mayorov
2ee00b75a7
fix(redis): unwrap pipeline queries against redis cluster 
This is an additional safety measure in addition to the disabled
batching on the bridge level.
 2023-01-25 17:28:11 +03:00
Erik Timan
805d08e823 fix: reduce log level from error to warning in several places 
This reduces the log level from error to warning in places that are
connected to the influxdb bridge. Transient errors for external
resources should not render an error log.
 2023-01-25 14:49:50 +01:00
Erik Timan
8836494542 fix: redact influxdb tokens in a few logs 2023-01-25 14:48:32 +01:00
Zaiming (Stone) Shi
5fdf7fd24c fix(kafka): use async callback to bump success counters 
some telemetry events from wolff are discarded:

* dropped:
    this is double counted in wolff,
    we now only subscribe to the dropped_queue_full event
* retried_failed:
    it has different meanings in wolff,
    in wolff, it means it's the 2nd (or onward) produce attempt
    in EMQX, it means it's eventually failed after some retries

* retried_success
    since we are going to handle the success counters in callbac
    this having this reported from wolff will only make things
    harder to understand

* failed
    wolff never fails (unelss drop which is a different counter)
 2023-01-24 21:12:36 +01:00
Zaiming (Stone) Shi
6175076f6f
Merge pull request #9835 from olcai/add-influxdb-test-files 
fix: add influxdb test files and fixes
 2023-01-24 17:02:23 +01:00
Zaiming (Stone) Shi
e5b65087af
Merge pull request #9834 from zmstone/0123-fix-idle_timeout-infinity 
fix(emqx_connection): crash when idle_timeout is set to infinity
 2023-01-24 16:05:09 +01:00
Erik Timan
9d20431257 fix(emqx_resource): fix crash while flushing queue 
We used next_event for flushing the queue in emqx_resource, but this
leads to a crash. We now call flush_worker/1 instead.
 2023-01-24 14:13:35 +01:00
Erik Timan
28718edbfd chore: bump application VSNs 2023-01-24 14:12:34 +01:00
Zaiming (Stone) Shi
8fde169abb
Merge pull request #9821 from thalesmg/buffer-worker-expiry-v50 
feat(buffer_worker): add expiration time to requests
 2023-01-24 13:54:04 +01:00
Zaiming (Stone) Shi
7575120ea6 test: use snabbkaffe retry macro 2023-01-24 10:54:20 +01:00
Zaiming (Stone) Shi
140cda2f13 fix(emqx_connection): crash when idle_timeout is set to infinity 2023-01-24 10:14:35 +01:00
Zaiming (Stone) Shi
a0100c0a4e chore: bump app versions 2023-01-23 16:20:21 +01:00
Zaiming (Stone) Shi
a83ed2fe94 Merge remote-tracking branch 'origin/master' into 0123-chore-sync-release-50-to-master 2023-01-23 16:20:01 +01:00
Zaiming (Stone) Shi
727100e094 chore: prepare for v5.0.15 release 2023-01-20 16:42:01 +01:00
Thales Macedo Garitezi
ca4a262b75 refactor: re-organize dealing with unrecoverable errors 2023-01-20 12:00:17 -03:00
Thales Macedo Garitezi
6fa6c679bb feat(buffer_worker): add expiration time to requests 
With this, we avoid performing work or replying to callers that are no
longer waiting on a result.

Also introduces two new counters:

- `dropped.expired` :: happens when a request expires before being
  sent downstream
- `late_reply` :: when a response is receive from downstream, but the
  caller is no longer for a reply because the request has expired, and
  the caller might even have retried it.
 2023-01-20 11:36:52 -03:00
Zaiming (Stone) Shi
57607ca0ce chore: prepare for v5.0.15 release 2023-01-20 11:20:34 +01:00
Zaiming (Stone) Shi
1c3e055b13
Merge pull request #9822 from JimMoen/fix-schema-typo 
chore: i18n typo fix
 2023-01-20 11:11:18 +01:00
JimMoen
16f45a60fd
chore: i18n typo fix 2023-01-20 11:50:01 +08:00
Thales Macedo Garitezi
a6ad97e6a5
Merge pull request #9807 from thalesmg/buffer-worker-rename-v50 
buffer work refactoring follow up (part 1)
 2023-01-19 14:17:10 -03:00
Zaiming (Stone) Shi
abe7a69696 Merge remote-tracking branch 'origin/master' into release-50 2023-01-19 17:51:38 +01:00
Stefan Strigler
6bde1173af chore: add translations 2023-01-19 13:26:36 +01:00
Thales Macedo Garitezi
d755b43c77 fix(jwt_worker): handle exceptions when decoding jwk from pem 
Returns a more controlled error if users attempt to use the Service
Account JSON from the GCP PubSub example from swagger, which is
redacted.
 2023-01-19 09:24:45 -03:00
Zaiming (Stone) Shi
63748aba3c
Merge pull request #9804 from emqx/release-50 
Merge release-50 (candidate of e5.0.0-rc.1) back to master.
 2023-01-19 08:48:41 +01:00
Thales Macedo Garitezi
47f796dd12 refactor: rename emqx_resource_worker -> emqx_resource_buffer_worker 
To make it more clear that it's purpose is serve as a buffering layer.
 2023-01-18 16:15:34 -03:00
Ilya Averyanov
f9843de7ae
Merge pull request #9628 from savonarola/fix-flaky-redis-bridge-test 
chore(ee bridge): fix Redis bridge test flakyness
 2023-01-18 20:56:13 +02:00
Zaiming (Stone) Shi
1716a5da99 chore: bump version to e5.0.0-rc.1 2023-01-18 17:22:05 +01:00
Stefan Strigler
fb763ecebd feat: support HAProxy protocol for dashboard API 2023-01-18 15:49:57 +01:00
Zaiming (Stone) Shi
7abba17b25
Merge pull request #9765 from zmstone/0115-add-password-converter 
fix(schema): add password converter to ensure its binary() type
 2023-01-18 15:09:05 +01:00
Andrew Mayorov
33b3c4fa9a
Merge pull request #9753 from feat/EMQX-8738/convert-ordered-sets 
feat: turn tables queried with search APIs into ordered sets
 2023-01-18 18:05:57 +04:00
Ilya Averyanov
44a6e5ed15 chore(resources): add missing parameters to emqx_resource schema 2023-01-18 14:33:45 +02:00
Zaiming (Stone) Shi
7e8381f4c7
Merge pull request #9785 from savonarola/fix-authn-handling 
fix(authn): stop authn handling when emqx_authentication provides a result
 2023-01-18 13:24:22 +01:00
ieQu1
c46d7f3404
Merge pull request #9801 from ieQu1/ekka-0.13.9 
chore(ekka): Bump version to 0.13.9
 2023-01-18 13:23:59 +01:00
Thales Macedo Garitezi
167b623497
Merge pull request #9699 from thalesmg/fix-buffer-clear-replayq-on-delete-v50 
fix(buffer): fix `replayq` usages in buffer workers (5.0)
 2023-01-18 09:08:50 -03:00
Erik Timan
46fc69cd48
Merge pull request #9781 from olcai/delete-zip-file-from-trace-log-download 
fix(emqx_management): delete files after trace log download
 2023-01-18 13:05:39 +01:00
ieQu1
d7242739e0 chore(ekka): Bump version to 0.13.9 2023-01-18 12:01:03 +01:00
Zaiming (Stone) Shi
d4f3b4c8c2 Merge remote-tracking branch 'origin/master' into fix-buffer-clear-replayq-on-delete-v50 2023-01-18 11:39:47 +01:00
zhongwencool
8e1475addb
Merge pull request #9798 from zhongwencool/dashboard-document 
chore: improve the dashboard's configuration
 2023-01-18 18:20:00 +08:00
Erik Timan
42182279b7 fix(emqx_management): ensure trace file dir is deleted on zip exception 2023-01-18 10:20:41 +01:00
Zhongwen Deng
fb84d5b817 chore: make spellcheck happy 2023-01-18 17:06:46 +08:00
Ivan Dyachkov
430b0a03d4
Merge pull request #9780 from id/fix-ensure-no-colon-in-filenames 
fix: ensure no colon in filenames
 2023-01-18 09:36:16 +01:00
Zhongwen Deng
0d852d9122 docs: improve the dashboard's document 2023-01-18 16:28:35 +08:00
Zaiming (Stone) Shi
faf5916ed6 test: relax recoverable/unrecoverable error check 
for now, treat all other errors unrecoverable
 2023-01-18 07:52:28 +01:00
zhongwencool
bc9d97ea53
Merge pull request #9791 from zhongwencool/crash-dump-doc 
chore: more detail about crash dump config
 2023-01-18 09:56:21 +08:00
Thales Macedo Garitezi
5c2ac0ac81 chore: don't cancel inflight items upon worker death; retry them 2023-01-17 19:50:30 -03:00
Thales Macedo Garitezi
087b667263 fix(buffer_worker): allow signalling unrecoverable errors 2023-01-17 19:50:30 -03:00
Stefan Strigler
f899284e3a
Merge pull request #9789 from sstrigler/EMQX-8754-test-function-return-500-of-data-integration-google-pubsub 
EMQX 8754 test function return 500 of data integration google pubsub
 2023-01-17 22:49:28 +01:00
lafirest
dea0c8230e
Merge pull request #9787 from lafirest/fix/webhook_bridge_cfg_upgrade 
fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12
 2023-01-18 04:47:08 +08:00
Thales Macedo Garitezi
4ed7bff33f chore: fix dialyzer warnings 2023-01-17 16:49:16 -03:00
Thales Macedo Garitezi
fa01deb3eb chore: retry as much as possible, don't reply to caller too soon 2023-01-17 16:49:15 -03:00
Thales Macedo Garitezi
b82009bc29 refactor: use monotonic times as refs and store initial times when creating ets 
with this, we may measure latencies in the future.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
3ba65c4377 feat: poke the buffer workers when inflight is no longer full 
if max inflight = 1, then we only make progress based on the state
timer, since the callbacks were not poking the buffer workers.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
b5aaef084c refactor: enter running state directly 
now that we don't have the possibility of dirty disk queues (we always
use volatile replayq), we will never resume old work.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
bd0e2a74ba refactor: rename inflight_name field to inflight_tid 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
006b4bda97 feat(buffer_worker): monitor async workers and cancel their inflight requests upon death 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
731ac6567a fix(buffer_worker): don't retry all kinds of inflight requests 
Some requests should not be retried during the blocked state.  For
example, if some async requests are just taking some time to process,
we should avoid retrying them periodically, lest risk overloading the
downstream further.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
5425f3d88e refactor: rm unused fn 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
5dd24a64c3 refactor(buffer_worker): check if inflight is full before flushing 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
344eeebe63 fix: always ack async replies 
The caller should decide if it should retry in that case, to avoid
overwhelming the resource with retries.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
bd95a95409 refactor: remove redundant BlockWorker arg, change boolean to ack/nack 
`BlockWorker` was always false (ack).  Also, changed the return to
something more semantic than a boolean to avoid [boolean
blindness](https://runtimeverification.com/blog/code-smell-boolean-blindness/)
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
478fcc6ffd test: fix flaky test 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
30a227bd38 refactor: rename resume state timeout to unblock 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
7401d6f0ce refactor: rename ack fn 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
196bf1c5ba feat: mass collect calls from mailbox also when blocked 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
d4724d6ce9 refactor: remove redundant function 
`retry_queue` does basically what the running state does, now that we
refactored the buffer workers to always use the queue.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
d6a9d0aa48 fix: set queuing to 0 after buffer worker termination 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
81fc561ed5 fix(buffer_worker): check for overflow after enqueuing new requests 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
4cb83d0c9a fix: fix some expressions after refactoring 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi
fecdbac9a8 refactor: rename a few functions 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi
cdd8de11b0 chore: fix a typo in function name 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi
618b97870b refactor: call local function queue_count everywhere 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi
249c4c1c79 refactor: use 'bufs' for resource worker replayq dir 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
af6807e863 refactor: cancel flush timer sooner 
Avoids the cancellation being delayed.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
477c55d8ef fix: sanitizy replayq dir filepath 
Colons (`:`) are not allowed in Windows.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
4c04a01370 refactor(buffer_worker): remove ?Q_ITEM wrapping and use lightweight size estimate 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
32a9e60313 feat(buffer_worker): also use the inflight table for sync requests 
Related: https://emqx.atlassian.net/browse/EMQX-8692

This should also correctly account for `retried.*` metrics for sync
requests.

Also fixes cases where race conditions for retrying async requests
could potentially lead to inconsistent metrics.

Fixes more cases where a stale reference to `replayq` was being held
accidentally after a `pop`.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
ff23d25e8b chore(replayq): update replayq -> 0.3.6 and use clean_start for buffer workers 
So we can truly avoid resuming work after a node restart.
 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
c383558467 fix(buffer): fix replayq usages in buffer workers (5.0) 
https://emqx.atlassian.net/browse/EMQX-8700

Fixes a few errors in the usage of `replayq` queues.

- Close `replayq` when `emqx_resource_worker` terminates.
- Do not keep old references to `replayq` after any `pop`s.
- Clear `replayq`'s data directories when removing a resource.
 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi
fc992f28bc test: add test coverage 2023-01-17 20:13:01 +01:00
Zaiming (Stone) Shi
fff6bf921f refactor(authz): call emqx_resource:simple_sync_query 
there is no need to route the request through the buffer workers
 2023-01-17 20:01:45 +01:00
Zaiming (Stone) Shi
7a471faf67 refactor(authn): use simple sync query for authn 
there is no need to route the authn queries through the buffer worker
 2023-01-17 19:57:14 +01:00
Ilya Averyanov
1d7364dab1 fix(authn): fix authn hook chain evaluation 
Stop authentication handling when
`emqx_authentication` provides a definitive result.
 2023-01-17 20:06:20 +02:00
Stefan Strigler
d164e5bc0f fix: don't require ssl conf 2023-01-17 15:29:19 +01:00
Stefan Strigler
e54f2f83b3 test: use same default timeout as elsewhere 2023-01-17 15:29:19 +01:00
Erik Timan
30a5cfaa83 fix(emqx_management): remove trace files after zip download 
We only deleted the resulting zip after a trace file download, not the
actual trace files. This adds a deletion of the uncompressed trace files
as well. It also creates unique directories when collecting trace files
so that concurrent downloads doesn't overwrite files in transit.
 2023-01-17 14:24:10 +01:00
Erik Timan
94296258b4 test(emqx_management): refactor test suite to use common test utils for API calls 2023-01-17 14:24:10 +01:00
Zhongwen Deng
f98786c91f fix: bad crash_dump_file default value in rpm 2023-01-17 20:47:54 +08:00
Zhongwen Deng
cf99089f84 chore: add some trace func to user_default 2023-01-17 20:33:01 +08:00
Zhongwen Deng
5a56013193 docs: more detail about crashdump config 2023-01-17 20:33:01 +08:00
zhongwencool
9f80eb2497
Merge pull request #9777 from zmstone/0116-refactor-config_files-is-readonly 
refactor: config_files is a readonly config
 2023-01-17 20:32:39 +08:00
Zaiming (Stone) Shi
263deae1f3 refactor: add a more generic name for password_converter 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi
2afbf6a406 test: unlink process which is getting shutdown 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi
4a7e74f5d6 fix(schema): add password converter to ensure its binary() type 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi
8b3ec48406 refactor: node.config_files should be hidden as it's not configurable 
this config schema exists only for app-env mapping.
users can not change it because the bootstraping config
has to be etc/emqx.conf
 2023-01-17 11:08:40 +01:00
Stefan Strigler
74ae7c4264
Merge pull request #9782 from sstrigler/EMQX-8361-fix-ct-suite-for-plugins-api 
test: get plugin template as artifact
 2023-01-17 11:08:07 +01:00
firest
c3d5c25c26 fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12 2023-01-17 17:42:10 +08:00
zhongwencool
cb5314bb9e
Merge pull request #9774 from zhongwencool/dashboard-password-validate-v50 
feat: add dashboard password validation
 2023-01-17 17:02:51 +08:00
Stefan Strigler
bc69a00329 test: get plugin template as artifact 2023-01-17 09:50:17 +01:00
Shawn
44f635300d chore: update mysql-otp to 1.7.2 2023-01-17 13:36:26 +08:00
Zhongwen Deng
3dfdad9cc9 test: dashboard default password 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi
b9911e281e docs: Update apps/emqx_dashboard/src/emqx_dashboard_admin.erl 2023-01-17 11:30:42 +08:00
zhongwencool
c549379b7c chore: apply suggestions from code review 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-01-17 11:30:42 +08:00
Zhongwen Deng
82af854c4a test: dashbashboard new password failed 2023-01-17 11:30:42 +08:00
Zhongwen Deng
a523fa2fa2 feat: add dashboard password validate 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi
33be7672ad test: recover boot_modules app env after test 2023-01-17 00:43:25 +01:00
Ivan Dyachkov
676f017ec0 fix: ensure no colon in filenames 2023-01-16 21:27:01 +01:00
Andrew Mayorov
5fd157fb1d
Merge pull request #9776 from keynslug/chore/bump-epgsql-otp-25 
chore: update epgsql to 4.7.0.1 with full OTP-25 compat
 2023-01-16 23:51:09 +04:00
Zaiming (Stone) Shi
e668044ebc fix: version number previs in restricted shell 2023-01-16 13:47:00 +01:00
Andrew Mayorov
34571c779d
feat: make suboption table ordering more natural 2023-01-16 15:39:10 +03:00
Andrew Mayorov
ce2dba15b4
feat: turn tables queried with search APIs into ordered sets 
This is needed to ensure more or less consistent client experience
for the new planned cursor-based search APIs.
 2023-01-16 15:39:10 +03:00
Zaiming (Stone) Shi
a7fc5e8fe1
Merge pull request #9761 from zmstone/0114-fix-kafka-value-template-and-docs 
feat: introduce 'this' concept for placeholder, and use it in Kafka bridge
 2023-01-16 13:37:29 +01:00
Andrew Mayorov
54cea3b3f2
chore: update epgsql to 4.7.0.1 with full OTP-25 compat 2023-01-16 15:35:51 +03:00
Stefan Strigler
67909f0b40 fix: testing metrics for emqx_bridge_mqtt_SUITE 2023-01-16 12:10:06 +01:00
Stefan Strigler
a19226326c fix: don't test for strict values as that makes test flaky 2023-01-16 12:10:06 +01:00
Stefan Strigler
9a53410efb fix: don't add empty list element to options proplist 2023-01-16 12:00:03 +01:00
Stefan Strigler
edfcea7f43 fix: add log for timeout 2023-01-16 12:00:03 +01:00
Zaiming (Stone) Shi
47414e0d53 docs: improve kafka key and value field description 2023-01-16 11:32:09 +01:00
Zaiming (Stone) Shi
376dab7cd8
Merge pull request #9743 from zmstone/0112-docs-fix-some-descriptions 
0112 docs fix some descriptions
 2023-01-16 09:56:56 +01:00
JimMoen
54ebc27d24
Merge pull request #9672 from JimMoen/0103-fix-mqtt-bridge 
Fix the problem that the bridge is not available when the Payload template is empty in the MQTT bridge.
 2023-01-16 09:57:20 +08:00
Zaiming (Stone) Shi
089389100e
Merge pull request #9763 from zmstone/0114-fix-crash-when-password-undefined 
0114 fix crash when password undefined
 2023-01-15 12:41:02 +01:00
Zaiming (Stone) Shi
df202c69ad
Merge pull request #9746 from zmstone/0113-fix-mqtt-connect-packet-format-when-password-is-empty 
fix(emqx_packet): log empty string if password value is missing
 2023-01-14 19:42:07 +01:00
Zaiming (Stone) Shi
0f2f5fbbe0 fix(authn): no exception when password is 'undefined' 2023-01-14 17:58:55 +01:00
Zaiming (Stone) Shi
7073c62dd9 feat: add a 'this' for rule engine put env 
now a dot is allowed in front of a var (path)
e.g. ${.clientid} is equivalent to ${clientid}

${.} however, means everything.
the parsed var is interally represented as $_THIS_
to make it easier to read/search.
 2023-01-14 11:03:58 +01:00
Zaiming (Stone) Shi
002f5dbd89
Merge pull request #9750 from zhongwencool/bootstrap-app-env 
fix: save app's env when bootstrapping and reloading
 2023-01-14 07:33:13 +01:00
Zhongwen Deng
b07befbf56 test: conf_app ct failed 2023-01-14 01:03:57 +08:00
Stefan Strigler
6fe09447ed fix: stale test using old resource paths after merge 2023-01-13 17:23:25 +01:00
Zhongwen Deng
09160383f5 chore: bump emqx_resource to 0.1.5 2023-01-13 23:42:41 +08:00
Zhongwen Deng
588140b842 chore: why we should ingore save kernel app env when init 2023-01-13 23:35:45 +08:00
Zhongwen Deng
1fe0061123 fix: node.config_files type is hocon:array(string())) 2023-01-13 23:35:45 +08:00
Zhongwen Deng
c9f0355f5a fix: save app env when bootstrap and reload 2023-01-13 23:35:45 +08:00
Zhongwen Deng
1085492171 chore: don't refresh when CT is running 2023-01-13 23:35:45 +08:00
Zhongwen Deng
91d38b410d fix: refresh logger handler when starting 2023-01-13 23:35:45 +08:00
Stefan Strigler
8f5881d1a5 fix: remove stale request/3 from merge error 2023-01-13 16:19:35 +01:00
Stefan Strigler
e08c1d2229 Merge remote-tracking branch 'olcai/refactor-bridges-api' into dev/api-refactor 2023-01-13 15:49:52 +01:00
Stefan Strigler
1690a6dcfc
Merge branch 'master' into dev/api-refactor 2023-01-13 15:34:13 +01:00
Erik Timan
61e98900be chore: bump app vsn of emqx_resource 2023-01-13 15:13:35 +01:00
Erik Timan
c3133fb6a2 fix(emqx_bridge): small fixes from review 2023-01-13 14:54:54 +01:00
Zaiming (Stone) Shi
a77d33b97d
Merge pull request #9751 from lafirest/fix/un_cleared_ca 
fix that obsoleted cert file will not be deleted after the listener is updated/deleted
 2023-01-13 14:38:55 +01:00
Kjell Winblad
1ac03ab208
Merge pull request #9730 from kjellwinblad/kjell/fix/resource_atom_leak/EMQX-8583 
fix: remove atom leaks
 2023-01-13 14:38:28 +01:00
Zaiming (Stone) Shi
bb3dceb456
Merge pull request #9749 from keynslug/fix/count-respect-matchspec 
fix(paging): respect matchspec even if qs is empty when counting
 2023-01-13 14:29:03 +01:00
Erik Timan
8dd52e5a6e chore: add translations to schemas 2023-01-13 14:19:23 +01:00
Erik Timan
f1c58c34ed test(emqx_bridge): fix fetching of metrics in emqx_bridge_mqtt_SUITE 2023-01-13 14:19:23 +01:00
Zaiming (Stone) Shi
2974269f2a
Merge pull request #9723 from emqx/release-50 
chore: sync release-50 to master
 2023-01-13 14:13:12 +01:00
firest
0a965879dd fix: fix that obsoleted cert file will not be deleted after the listener is updated/deleted 2023-01-13 18:42:27 +08:00
Zhongwen Deng
f15b29b1ef chore: upgrade app version 2023-01-13 18:22:29 +08:00
JimMoen
b7259d9a20
test(mqtt-bridge): use empty payload template for ingress/egress mqtt bridge 2023-01-13 18:12:53 +08:00
Kjell Winblad
734e6b9c96 chore: fix flaky test cases, log labels and review comments 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-01-13 11:05:02 +01:00
Andrew Mayorov
b3e62bd8f8
fix(paging): respect matchspec even if qs is empty when counting 2023-01-13 12:56:29 +03:00
zhongwencool
383a6812d1
Merge pull request #9748 from zhongwencool/listener-max-connections-default-value 
fix: default max_connections in raw config should be binary not atom
 2023-01-13 17:55:22 +08:00
JimMoen
59ac0b1424
fix(mqtt-bridge): transmit raw msg payload with empty template 2023-01-13 17:23:04 +08:00
zhongwencool
0049b4a294
Merge branch 'master' into release-50 2023-01-13 16:39:35 +08:00
Zaiming (Stone) Shi
e7883e0ee2
Merge pull request #9741 from id/fix-description-for-duration-fields 
fix: remove time unit from duration fields description
 2023-01-13 09:31:48 +01:00
Zhongwen Deng
eacdaffe3c fix: default max_connections in raw config should be binary not atom 2023-01-13 16:01:57 +08:00
Andrew Mayorov
33d88c95de
Merge pull request #9745 from keynslug/fix/EMQX-8702/changelog 
chore: add changelog for the fuzzy search API fix
 2023-01-13 11:59:37 +04:00
Zaiming (Stone) Shi
2bc19c3a62 fix(emqx_packet): log empty string if password value is missing 
to keep it a consistent behaviour comparing when logging username
 2023-01-13 08:21:09 +01:00
Zaiming (Stone) Shi
153d64068a refactor: rename root level cache type to authz_cache 2023-01-13 07:47:54 +01:00
Zaiming (Stone) Shi
c7ac55520d docs: update pool_size doc 2023-01-13 07:46:23 +01:00
Zaiming (Stone) Shi
b40ce0fc2d
Merge pull request #9626 from id/fix-enable-authz-cache-by-default 
fix: enable authorization cache by default
 2023-01-13 07:39:38 +01:00
Xinyu Liu
6b75077c47
Merge pull request #9734 from terry-xiaoyu/influxdb-default-value-of-write-syntax 
refactor: remove default value of timestamp field of influxdb
 2023-01-13 10:14:14 +08:00
Erik Timan
42f42de4d9 feat(emqx_bridge): add separate endpoint for enable/disable of bridge 
In order to improve the consistency with other API endpoints, we move
the enable/disable operations to a separate endpoint
/bridges/{id}/enable/[true,false].
 2023-01-12 19:39:43 +01:00
Erik Timan
860e21d40f feat(emqx_bridge): move metrics to own endpoint, rename reset_metrics 
In order for the /bridges APIs to be consistent with other APIs, we move
out metrics from GET /bridges/{id} to its own endpoint,
/bridges/{id}/metrics. We also rename /bridges/reset_metrics to
/bridges/metrics/reset.
 2023-01-12 19:39:41 +01:00
Erik Timan
c2fd1a4482 feat(emqx_bridge): shorten operation endpoint URLs 
This shortens and simplifies URLs for performing bridge operations so
that the API looks more congruent.
 2023-01-12 19:36:22 +01:00
Andrew Mayorov
90e837783f
fix(paging): rename partial to overflow for clarity 2023-01-12 19:25:40 +03:00
lafirest
a26c05f4f6
Merge pull request #9586 from lafirest/fix/disable_basic_auth_api 
fix: disable basic auth for HTTP API
 2023-01-12 23:02:29 +08:00
Zaiming (Stone) Shi
370b6b0d2f
Merge pull request #9735 from kjellwinblad/kjell/fix/password_log_leak/EMQX-8718 
fix: remove password from info logs
 2023-01-12 15:34:16 +01:00
Shawn
1711823487 refactor: remove default value of timestamp field of influxdb 2023-01-12 22:25:44 +08:00
firest
c5f557e315 fix: disable basic auth for HTTP API 2023-01-12 21:35:49 +08:00
Stefan Strigler
2a81fa1522 fix: remove redundant log message 2023-01-12 14:29:11 +01:00
Stefan Strigler
04f46f5227 feat: make http connector report errors 2023-01-12 14:26:36 +01:00
Stefan Strigler
77f043dedf fix: don't require username and password 
not enforced by schema so we shouldn't either
 2023-01-12 14:26:36 +01:00
Ivan Dyachkov
b5d3e9d8b8 fix: remove time unit from duration fields description 2023-01-12 14:18:55 +01:00
Ivan Dyachkov
f90c41f769 fix: set default value in schema 2023-01-12 13:49:52 +01:00
Ivan Dyachkov
1f57e7b538 fix: enable authorization cache by default 2023-01-12 12:36:31 +01:00
Andrew Mayorov
9e9d97b4d8
Merge pull request #9726 from keynslug/fix/EMQX-8702/fuzzy-search-paging 
fix(api): augment paged search responses with `hasnext` flag
 2023-01-12 13:35:59 +04:00
Kjell Winblad
82e80e5610 fix: remove password from info logs 
Information log messages for several connectors contained sensitive
password information. This commit fixes this by hiding the sensitive
information from the log messages.

Fixes: https://emqx.atlassian.net/browse/EMQX-8718
 2023-01-12 09:28:01 +01:00
JianBo He
13fdbd695e
Merge pull request #9722 from HJianBo/more-confs-for-prometheus 
feat: more confs for prometheus pushing
 2023-01-12 16:06:21 +08:00
Andrew Mayorov
50c9321b9e
fix(api): anticipate empty set of nodes handling cluster query 2023-01-12 10:57:50 +03:00
Andrew Mayorov
c89b227687
fix(test): adapt affected testcases 2023-01-12 10:57:50 +03:00
Andrew Mayorov
abf6f143e5
fix(paging): return subset of rows if page start is in the middle 
This change should cover this example scenario where select ended
returning 4 rows.

Row 3
Row 4
Row 5 <- page start
Row 6

Here, only rows 5 and 6 should end up in the response. Before this
change some rows could be duplicated across adjacent search pages.
 2023-01-12 10:57:50 +03:00
Andrew Mayorov
e07aa2086b
fix(api): augment paged search responses with hasnext flag 
This flag indicates whether there are more results available on the
next pages. It is needed in cases when the total number of search
results is not known in advance.

Also, in such cases there's no `count` field in responses anymore
because responding with `0` was confusing for clients: it's not possible
to differentiate between "there are no results" and "we don't know how
much".

Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-01-12 10:57:30 +03:00
Zhongwen Deng
1194d07a07 refactor: update emqx_prometheus's headers from map() to list() 2023-01-12 12:48:26 +08:00
Xinyu Liu
bae811e8b4
Merge pull request #9725 from terry-xiaoyu/remove_the_auto_reconnect_field 
refactor: remove the auto_reconnect field
 2023-01-12 11:11:00 +08:00
Kjell Winblad
8c482e03d1 fix: remove atom leaks 
Both emqx_resource_managers and emqx_resource_workers leaked atoms as they
created an unique atoms to use as registered names. This is fixed by
removing the need to register the names.

Fixes: https://emqx.atlassian.net/browse/EMQX-8583
 2023-01-11 17:03:28 +01:00
Stefan Strigler
417d95d76f
Merge pull request #9708 from sstrigler/EMQX-8588-refactor-emqx-resource-manager 
EMQX 8588 refactor emqx resource manager
 2023-01-11 16:13:55 +01:00
Shawn
3e9c4f444f refactor: remove the auto_reconnect field 2023-01-11 21:47:06 +08:00
Thales Macedo Garitezi
254b92a3c1 docs: add comma to please spellcheck 2023-01-11 10:31:23 -03:00
Stefan Strigler
8ad8288195 feat: report error in create_dry_run 2023-01-11 14:22:37 +01:00
Thales Macedo Garitezi
d5a462486e feat: tag another emqx schema as emqx 2023-01-11 09:31:38 -03:00
Thales Macedo Garitezi
e6a8be1ff6 docs: fix closing tag 2023-01-11 09:10:04 -03:00
Thales Macedo Garitezi
48e1ba4832 feat(docs): add tags to schemas 
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
 2023-01-11 09:10:03 -03:00
Kjell Winblad
fde8752452
Merge pull request #9710 from kjellwinblad/kjell/mongodb_upgrade_in_emqx_5_master_fix 
chore: upgrade mongodb-erlang to v3.0.19
 2023-01-11 12:41:11 +01:00
JianBo He
9f6df27759 chore: bump app.src vsn 2023-01-11 17:47:06 +08:00
JianBo He
b1292d6d4f feat(prom): support configuring job_name for pushing mode 2023-01-11 17:31:58 +08:00
JianBo He
50b7ac6a22 feat(prom): support headers for pushing 2023-01-11 17:31:58 +08:00
Zhongwen Deng
fdc0682c3f fix: elvis warning 2023-01-11 17:30:45 +08:00
zhongwencool
6f0e228a9b
Merge pull request #9717 from zmstone/0110-do-not-start-resource-after-recreate-if-disabled 
fix(emqx_resource_manager): do not start when disabled
 2023-01-11 16:12:53 +08:00
Zaiming (Stone) Shi
85a8eff90b fix(emqx_resource_manager): do not start when disabled 2023-01-11 08:33:48 +01:00
Zhongwen Deng
d9429bba00 fix: bump hocon to 0.35.0(don't log overriding structs) 2023-01-11 14:44:34 +08:00
Zhongwen Deng
ba738d7707 chore: bump to 5.0.14 2023-01-11 14:39:32 +08:00
zhongwencool
ce2b4d48ad
Merge pull request #9714 from zhongwencool/auto-subscribe-api-schema 
fix: bad auto_subscribe api schema
 2023-01-11 14:32:40 +08:00
zhongwencool
e7d6d26186
Merge pull request #9713 from zhongwencool/api-keys-bootstrap-file 
feat: introduce API keys bootstrap_file config
 2023-01-11 11:30:45 +08:00
Zhongwen Deng
52ccd0762e test: QOS ct failed 2023-01-11 10:34:09 +08:00
Zaiming (Stone) Shi
6bbb5edb53
Merge pull request #9667 from olcai/remove-clientid-in-publish-request-body 
fix(mgmt_api): remove possibility to set clientid in /publish API
 2023-01-10 22:31:35 +01:00
Zaiming (Stone) Shi
ee89de816c fix: upgrade bridge config converter 2023-01-10 20:42:30 +01:00
Erik Timan
fb97096405 fix(mgmt_api): deprecate clientid field instead of removing it 2023-01-10 16:48:04 +01:00
Zhongwen Deng
a2be9947f5 chore: update auto_subscribe app version 2023-01-10 22:12:23 +08:00
Zhongwen Deng
5415e341fb fix: bad auto_subscribe api schema 2023-01-10 21:53:54 +08:00
zhongwencool
918ba0dffd
chore: apply suggestions from code review 
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
 2023-01-10 20:39:31 +08:00
Zhongwen Deng
21dbc21dc8 chore: add changelog for api_key.bootstrap_file 2023-01-10 19:15:01 +08:00
Zhongwen Deng
5d4a1933a2 chore: add more test for api_key 2023-01-10 19:04:00 +08:00
Zhongwen Deng
f6a47e5cf6 chore: support api_key.bootstrap_file config 2023-01-10 19:03:49 +08:00
Zhongwen Deng
142826c4c3 chore: limit the maximum number of api keys to 100 2023-01-10 19:03:39 +08:00
Xinyu Liu
0f70786d13
Merge pull request #9705 from terry-xiaoyu/remove-default-value-of-webhook-body 
fix: remove the default value of webhook body field
 2023-01-10 18:23:50 +08:00
Xinyu Liu
12de104033
Merge pull request #9712 from emqx/fix-bulk-subscribe-failed-in-client.connected-hook 
fix: bulk subscribe topics failed in the client.connected hook
 2023-01-10 18:22:15 +08:00
ieQu1
13f1ab6344
Merge pull request #9671 from ieQu1/metrics 
feat(emqx_metrics): Sliding window samples
 2023-01-10 11:08:43 +01:00
Stefan Strigler
f27f573109 refactor: move to /authorization/sources/built_in_database/rules 2023-01-10 11:00:22 +01:00
Shawn
73643e932f fix: bulk subscribe topics failed in the client.connected hook 2023-01-10 16:43:24 +08:00
Xinyu Liu
52343e58c7
Merge pull request #9703 from emqx/subscribe-api-function-clause 
fix: missing default value for qos field of API /clients/:clientid/subscribe
 2023-01-10 10:13:58 +08:00
Shawn
724015af47 fix: remove the default value of webhook body field 2023-01-10 09:45:25 +08:00
Zaiming (Stone) Shi
67f2159a27
Merge pull request #9653 from zmstone/0101-authz-schema-union-member-selection 
0101 authz schema union member selection
 2023-01-09 22:17:51 +01:00
Kjell Winblad
0bd5bec982 chore: upgrade mongodb-erlang to v3.0.19 
This upgrades mongodb-erlang from v3.0.18 to v3.0.19. This fixes a bug
that occurred when one issued a command specified as a map*. This fix
should not affect present EMQX as this functionality is currently not
used by EMQX. However, I think it is good to do the upgrade anyway in
case we will use the fixed functionality in the future. There is no need
to update the changelog.

* https://github.com/emqx/mongodb-erlang/pull/36
 2023-01-09 18:14:11 +01:00
lafirest
f1ebfe40a5
Merge pull request #9697 from lafirest/fix/ldap_i18n 
chore: add i18n file for ldap connector
 2023-01-09 22:54:09 +08:00
Thales Macedo Garitezi
7f5150b2e9
Merge pull request #9689 from thalesmg/fix-http-authz-failure-v50 
fix(authz_http): handle `ignore` results (request failures)
 2023-01-09 11:07:52 -03:00
Zaiming (Stone) Shi
c3635f537a ci: wait for redis in emqx_ee_bridge_redis_SUITE 2023-01-09 14:39:56 +01:00
Zaiming (Stone) Shi
0697c692ed refactor: mongo_type and redis_type are not mandatory 
the connector schemas are shared between authn, auth and bridges,
the difference is that authn and authz configs are unions like:

[
    {mongo_type = rs,
     ...
    }
    {another backend config}
]

however the brdige types are maps like, for example:

mongodb_rs.$name {
    mongo_type = rs
    ...
}

in which case, the mongo_type is not required.

in order to keep the schema static as much as possible,
this field is chanegd to 'required => false' with a default value.

However, for authn and authz, the union selector will still raise
exception if the there is no type provided.
 2023-01-09 14:26:16 +01:00
Zaiming (Stone) Shi
e52f9d5920 refactor: use union member type selector for authz sources 2023-01-09 14:26:16 +01:00
Kjell Winblad
ba8c304723
Merge pull request #9691 from kjellwinblad/kjell/mongodb_upgrade_in_emqx_5_master 
feat: upgrade MongoDB library to support MongoDB 5.1+
 2023-01-09 14:15:48 +01:00
Kjell Winblad
692c1f0d1b
Merge pull request #9678 from kjellwinblad/kjell/fix/jira/EMQX-8648 
fix: no feedback when deleting non-existing configuration
 2023-01-09 13:52:40 +01:00
Stefan Strigler
c85a988a43 fix: split 'server' param into host and port for mqtt bridge 2023-01-09 13:27:42 +01:00
Andrew Mayorov
0db31f30a1
Merge pull request #9675 from keynslug/chore/otp-25-compat-deps 
chore: bump deps versions with full OTP-25 compat
 2023-01-09 14:54:14 +04:00
Shawn
50717a5567 fix: set the topic field of API /clients/:clientid/subscribe as requried 2023-01-09 17:57:39 +08:00
Stefan Strigler
c42c99f94f fix: set a timeout for tcp connect 2023-01-09 10:01:35 +01:00
Stefan Strigler
96ca0d9f49 feat(emqx_bridge): add /bridges_probe API endpoint 2023-01-09 10:01:33 +01:00
Stefan Strigler
903ae9a644 style: fix typo in fun name 2023-01-09 09:59:42 +01:00
Erik Timan
3319a8d28e fix(mgmt_api): remove possibility to set clientid in /publish API 
To avoid security confusion, we remove the possibility to specify the
client ID in the request body for /publish and /publish/bulk.
 2023-01-09 09:30:34 +01:00
Zaiming (Stone) Shi
8d53d154d9 refactor: remove license from configs API 
license has its own config APIs
 2023-01-09 09:16:19 +01:00
Shawn
7ed1384623 fix: missing default value for qos field of API /clients/:clientid/subscribe 2023-01-09 16:02:10 +08:00
Zaiming (Stone) Shi
3859878985
Merge pull request #9684 from zmstone/0105-do-not-use-testcase-skip 
test: do not use tc_user_skip for test cases
 2023-01-09 09:02:06 +01:00
JianBo He
6f5057b9dd
Merge pull request #9637 from HJianBo/fix-clients-api 
fix(clients): fix expiry_interval unit error
 2023-01-09 09:49:16 +08:00
Xinyu Liu
336b310d56
Merge pull request #9687 from emqx/function_clause_when_send_msg_to_bridges 
fix: function_clause when sending messages to bridges
 2023-01-09 09:23:17 +08:00
Andrew Mayorov
1336a57b3b
chore: bump to emqx/eredis_cluster 0.7.5 
* Full OTP-25 compat.
* Password censoring in status and crash reports.
* No more crashes on query timeout.
 2023-01-08 13:13:31 +03:00
Zaiming (Stone) Shi
5653d113ac
Merge pull request #9663 from zmstone/0102-fix-stopped-nodes-detection 
0102 fix stopped nodes detection
 2023-01-08 10:03:48 +01:00
firest
d2456f9fbe chore: add i18n file for ldap connector 2023-01-06 17:28:06 +08:00
firest
f7b50c56da feat(bridges): integrate PostgreSQL into bridges 
support both simple and batch query
 2023-01-06 11:47:22 +08:00
JianBo He
5d55e3a7b3 chore: updage mgmt app vsn 2023-01-06 10:31:27 +08:00
JianBo He
65729cd640 fix(clients): fix expiry_interval unit error 
It should be second not millisecond
 2023-01-06 10:31:27 +08:00
Shawn
13b2f45405 fix: function_clause when sending messages to bridges 2023-01-06 09:24:38 +08:00
Zaiming (Stone) Shi
7a4e7a6de9 test: wait for redis 2023-01-05 22:47:08 +01:00
Zaiming (Stone) Shi
5f12cdff6c ci: should not skip tests in github actions 2023-01-05 22:47:08 +01:00
Thales Macedo Garitezi
22707495ac
Merge pull request #9690 from thalesmg/fix-flaky-jwt-test 
test(flaky): fix flaky jwt worker test
 2023-01-05 14:00:39 -03:00
Kjell Winblad
5266722e51 feat: upgrade MongoDB library to support MongoDB 5.1+ 2023-01-05 16:40:45 +01:00
Kjell Winblad
b59c4c34c5 fix(Bridge REST API): no feedback when deleting bridge 
This fixes https://emqx.atlassian.net/browse/EMQX-8648. The issue
described in `EMQX-8648` is that when deleting a non-existing bridge the
server gives a success response. See below:

```
curl --head -u admin:public2 -X 'DELETE' 'http://localhost:18083/api/v5/bridges/webhook:i_do_not_exist'
HTTP/1.1 204 No Content
date: Tue, 03 Jan 2023 16:59:01 GMT
server: Cowboy
```

After the fix, deleting a non existing bridge will give the following
response:

```
HTTP/1.1 404 Not Found
content-length: 49
content-type: application/json
date: Thu, 05 Jan 2023 12:40:35 GMT
server: Cowboy
```

Closes: EMQX-8648
 2023-01-05 16:19:46 +01:00
Thales Macedo Garitezi
c6b8e614df fix(authz_http): handle ignore results (request failures) 
Related issue: https://github.com/emqx/emqx/issues/9683

When the HTTP request for authz fails (e.g.: resource is down or
server is down), then the HTTP authorizer returns `ignore`, which was
not handled correctly by the authorization callback.
 2023-01-05 11:34:23 -03:00
Thales Macedo Garitezi
cf6996cbea test(flaky): fix flaky jwt worker test 2023-01-05 11:19:43 -03:00
Thales Macedo Garitezi
70eb5ffb58 refactor: remove unused function 2023-01-05 10:16:01 -03:00
Thales Macedo Garitezi
56437228dc docs: improve descriptions 
Thanks to @qzhuyan for the corrections.
 2023-01-05 10:16:01 -03:00
Thales Macedo Garitezi
fd360ac6c0 feat(buffer_worker): refactor buffer/resource workers to always use queue 
This makes the buffer/resource workers always use `replayq` for
queuing, along with collecting multiple requests in a single call.
This is done to avoid long message queues for the buffer workers and
rely on `replayq`'s capabilities of offloading to disk and detecting
overflow.

Also, this deprecates the `enable_batch` and `enable_queue` resource
creation options, as: i) queuing is now always enables; ii) batch_size
> 1 <=> batch_enabled.  The corresponding metric
`dropped.queue_not_enabled` is dropped, along with `batching`.  The
batching is too ephemeral, especially considering a default batch time
of 20 ms, and is not shown in the dashboard, so it was removed.
 2023-01-05 10:15:09 -03:00
Thales Macedo Garitezi
bf3983e7c4 feat(buffer_worker): use offload mode for replayq 
To avoid confusion for the users as to what persistence guarantees we
offer when buffering bridges/resources, we will always enable offload
mode for `replayq`.  With this, when the buffer size is above the max
segment size, it'll flush the queue to disk, but on recovery after a
restart it'll clean the existing segments rather than resuming from
them.
 2023-01-05 10:11:59 -03:00
ieQu1
5d9f9671e9 feat(emqx_metrics): Sliding window samples 2023-01-05 13:55:52 +01:00
Thales Macedo Garitezi
fa4f90d43e
Merge pull request #9676 from thalesmg/fix-change-retainer-storage-type-v50 
fix(retainer): change mnesia table storage types during update
 2023-01-05 09:07:10 -03:00
ieQu1
93c205db7c fix(router_helper): Reduce log verbosity of expected events 2023-01-05 12:17:27 +01:00
Thales Macedo Garitezi
51ad27cb4b test(retainer): assert that retained messages are not lost when changing storage type 2023-01-04 13:52:52 -03:00
Thales Macedo Garitezi
2f13bfd452 fix(retainer): change mnesia table storage types during update 
https://emqx.atlassian.net/browse/EMQX-8650
 2023-01-04 13:52:52 -03:00
Thales Macedo Garitezi
86a4ac87a7
Merge pull request #9674 from thalesmg/reset-rule-metrics-on-disable-v50 
chore(rule_engine): reset metrics when disabling a rule
 2023-01-04 13:30:39 -03:00
Thales Macedo Garitezi
00b91d8a60
Merge pull request #9664 from thalesmg/fix-mongo-duration-field-descs-v50 
docs(mongodb_connector): fix duration field descriptions
 2023-01-04 10:03:38 -03:00
Thales Macedo Garitezi
eb7dca3691 chore(rule_engine): reset metrics when disabling a rule 
https://emqx.atlassian.net/browse/EMQX-8502

When a bridge is disabled, its metrics are reset.  With this change,
we make rule actions behave like that: disabling a rule will reset its
metrics.
 2023-01-04 09:30:47 -03:00
Erik Timan
f767db4d8f
Merge pull request #9660 from olcai/refactor-ingress-bridge-counter 
refactor(emqx_resource): ingress bridge counter
 2023-01-04 10:43:07 +01:00
Thales Macedo Garitezi
c3cfbae3c2 docs: improve descriptions 
Thanks to @qzhuyan for the corrections
 2023-01-03 11:38:09 -03:00
firest
b43be50a98 test: add the redacted test case for bridges api 2023-01-03 22:16:34 +08:00
firest
b016695cb1 fix(bridges): obfuscate the password in bridges API responses 2023-01-03 22:09:29 +08:00
Thales Macedo Garitezi
7bbdc20993 docs(mongodb_connector): fix duration field descriptions 
https://emqx.atlassian.net/browse/EMQX-8642

Adds different descriptions for the different fields of type duration
for MongoDB.

Based off: https://www.mongodb.com/docs/manual/reference/connection-string/
 2023-01-03 10:01:56 -03:00
Zaiming (Stone) Shi
ced5a27053 test: fix emqx_conf test cases 2023-01-02 21:30:40 +01:00
ieQu1
2f208c3ab9 fix(cluster_rpc): Detect stopped replicant nodes 2023-01-02 21:30:40 +01:00
lafirest
7985cd3536
Merge pull request #9638 from lafirest/fix/mysql_dup 
fix(mysql): fix the problem of data loss and bad match when mysql is disconnected
 2023-01-02 23:15:44 +08:00
Erik Timan
b9d012e072 refactor(emqx_resource): ingress bridge counter 
Unify code paths for resource metrics by removing
emqx_resource:inc_received/1 and adding
emqx_resource_metrics:received_inc/1 & friends.
 2023-01-02 15:11:52 +01:00
Thales Macedo Garitezi
7e02eac3bc
Merge pull request #9619 from thalesmg/refactor-gauges-v50 
refactor(metrics): use absolute gauge values rather than deltas (v5.0)
 2023-01-02 10:56:47 -03:00
firest
bc37556963 fix(mysql): fix the problem of data loss and bad match when mysql is disconnected 2023-01-02 21:46:24 +08:00
Zaiming (Stone) Shi
dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
Zaiming (Stone) Shi
231d6da4dc
Merge pull request #9644 from id/fix-kafka-bridge-connectivity-detection 
fix(kafka): detect connectivity in on_get_status
 2022-12-30 21:54:56 +01:00
Thales Macedo Garitezi
305ed68916 chore: bump app vsns 2022-12-30 16:51:24 -03:00
Thales Macedo Garitezi
8b060a75f1 refactor(metrics): use absolute gauge values rather than deltas 
https://emqx.atlassian.net/browse/EMQX-8548

Currently, we face several issues trying to keep resource metrics
reasonable.  For example, when a resource is re-created and has its
metrics reset, but then its durable queue resumes its previous work
and leads to strange (often negative) metrics.

Instead using `counters` that are shared by more than one worker to
manage gauges, we introduce an ETS table whose key is not only scoped
by the Resource ID as before, but also by the worker ID.  This way,
when a worker starts/terminates, they should set their own gauges to
their values (often 0 or `replayq:count` when resuming off a queue).
With this scoping and initialization procedure, we'll hopefully avoid
hitting those strange metrics scenarios and have better control over
the gauges.
 2022-12-30 16:51:24 -03:00
Zaiming (Stone) Shi
b30f2ea58b fix(statsd schema): a field is not 'required' when it has 'default' 2022-12-30 14:20:24 +01:00
Zaiming (Stone) Shi
981f75dc73 test(emqx_modules): do not expect badmatch in test cases!!! 2022-12-30 14:20:24 +01:00
Zaiming (Stone) Shi
0ce1ca89b7 refactor: use string type for server and servers 2022-12-30 14:20:23 +01:00
Zaiming (Stone) Shi
cac7e0c5f0 refactor: move version prefix string detection to a static fun 
this is to make dialyzer happy
 2022-12-30 14:19:03 +01:00
Thales Macedo Garitezi
a612eacf3a
Merge pull request #9641 from thalesmg/fix-gcp-pubsub-jwt-refresh-v50 
fix(gcp_pubsub): fix potential jwt accumulation and lack of refresh (v5.0)
 2022-12-30 09:33:41 -03:00
Ivan Dyachkov
f410201dc3 chore: fix flaky test case in emqx_connector_jwt_worker_SUITE 2022-12-30 11:45:13 +01:00
zhongwencool
f92b1fb8f4
Merge pull request #9633 from zhongwencool/fix-json-fmt 
fix: bad best fmt json []
 2022-12-30 09:08:13 +08:00
Thales Macedo Garitezi
446a4c74d0 fix(gcp_pubsub): fix potential jwt accumulation and lack of refresh (v5.0) 
https://emqx.atlassian.net/browse/EMQX-8653
Related:
- https://emqx.atlassian.net/browse/EEC-737
- https://emqx.atlassian.net/browse/EMQX-8652

Since the rule resource testing mechanism creates a new resource to
test the configuration, a new JWT associated with an unique temporary
resource was being created and left in the JWT table, leaking it.

Also, a wrong case clause when setting the new refresh timer for the
JWT worker was preventing it from refreshing from the 2nd refresh
onward.
 2022-12-29 16:30:36 -03:00
Zaiming (Stone) Shi
0b43ae621d ci: dump docker-compose log if failed to run ct 2022-12-29 09:23:11 +01:00
Zhongwen Deng
4e62aff3ab fix: bad best fmt json [] 2022-12-29 14:53:00 +08:00
Zaiming (Stone) Shi
2790d5697d test: add a 2 seconds sleep between listener sotp and restart 
to lower the chance of eaddrinuse
 2022-12-28 18:48:53 +01:00
Thales Macedo Garitezi
4819794401 test(refactor): decrease test teardown noise 2022-12-27 10:30:14 -03:00
Thales Macedo Garitezi
6fdcba641e test(refactor): no need for monitor 
the janitor is already linked to the parent
 2022-12-27 10:30:14 -03:00
Zaiming (Stone) Shi
a777bac38c build: prepare for v5.0.13 release 2022-12-27 11:54:09 +01:00
Zaiming (Stone) Shi
15bd797b36 ci: bump to v5.0.13-rc.3 2022-12-27 11:24:47 +01:00
Zaiming (Stone) Shi
580137de0d Merge remote-tracking branch 'origin/master' into release-50 2022-12-27 11:06:17 +01:00
lafirest
2ff6ef01f1
Merge pull request #9599 from lafirest/fix/blacklist_default_exp 
fix(banned): ensure the default expiration time of `banned` is large enough
 2022-12-27 17:30:10 +08:00
Zaiming (Stone) Shi
0517541f34 ci: bump to v5.0.13-rc.2 2022-12-27 09:54:34 +01:00
Zaiming (Stone) Shi
18246442aa chore: prepare v5.0.13-rc.1 
bump dashboard version to v1.1.4
 2022-12-26 21:45:48 +01:00
zhongwencool
81b226a801
Merge pull request #9608 from zmstone/1123-pin-hocon-0.33.0 
1123 pin hocon 0.33.0
 2022-12-26 19:22:21 +08:00
firest
af3f0c9bb0 fix(banned): ensure the default timeout of banned is large enough 2022-12-26 10:17:31 +08:00
Zaiming (Stone) Shi
f93c22045d fix: non-empty field should not be undefined 2022-12-24 11:41:45 +01:00
Zaiming (Stone) Shi
269a40a9ea fix: format psk user lookup fun back to string 2022-12-23 19:22:15 +01:00
Zaiming (Stone) Shi
d3efb0c0ba chore: bump app versions 2022-12-23 15:10:16 +01:00
Zaiming (Stone) Shi
350023e757 fix(config): option only_fill_defaults renamed to make_serializable 2022-12-23 14:27:04 +01:00
Thales Macedo Garitezi
35dc75b7ed feat(mqtt): add option to customize clientid prefix for egress bridges 
https://emqx.atlassian.net/browse/EMQX-8445

Currently the bridge client’s client ID is prefixed with the resource
ID.

Sometimes it’s useful for users to have control of this prefix,
e.g. prefix based ACL rules in the target broker.
 2022-12-23 09:50:26 -03:00
Ilya Averyanov
fa7433628c fix(logs): set password field sensitive for connectors 2022-12-23 12:46:03 +01:00
Zaiming (Stone) Shi
59970272ff chore: pin hocon 0.33.0 2022-12-23 12:07:36 +01:00
Zaiming (Stone) Shi
6292d7ec68 refactor: remove dead code 2022-12-23 12:07:36 +01:00
Erik Timan
13942f5c49 refactor: rename error return in mysql connector 2022-12-22 10:29:12 +01:00
Erik Timan
7df24000a0 test: add more EE mysql bridge test cases 2022-12-22 10:27:38 +01:00
Erik Timan
23ac426608 fix(emqx_connector): check for key among prepared statements on query 
An infinite loop was triggered in the mysql connector when a query
used a prepared statement key that was not among the defined prepared
statements on start. We now check that the key is defined among the
prepared statements before recursing. It seems that this bug was never
triggered in any production code flow and simply found while writing
tests.

An error return spell fix is also included as well as a FIXME comment
regarding running mysql:prepare and not distinguishing between
transient failures and syntax errors. Syntax errors should not be
retried.
 2022-12-22 10:27:38 +01:00
lafirest
f660724bf7
Merge pull request #9582 from lafirest/fix/fix_field_name 
fix(limiter): change limiter `cache` to `buffer`
 2022-12-21 18:12:40 +08:00
firest
a8e020af58 fix(limiter): change limiter cache to buffer 2022-12-20 22:30:52 +08:00
Zaiming (Stone) Shi
516147ad00
Merge pull request #9581 from zmstone/1219-fix-mqtt-bridge-config 
1219 fix mqtt bridge config
 2022-12-20 14:47:02 +01:00
Zaiming (Stone) Shi
479e191dcf refactor: refine worker pool config and doc 
worker pool is a buffer pool
the description hinted connection pool which is wrong.
 2022-12-20 09:02:51 +01:00
Zaiming (Stone) Shi
da51433dc3 refactor: add default value for eggress remote 'qos' and 'retain' 
otherwise when updating from dashboard, there is no way to set
'false' for 'retain' because it's a checkbox,
it's either 'true' or 'undefined'
 2022-12-20 00:00:31 +01:00
Zaiming (Stone) Shi
c085ffa0fe refactor: default mqtt bridgge buffer pool size down to 4 2022-12-19 23:59:46 +01:00
Zaiming (Stone) Shi
f611cbab45 chore: cap replayq seg size under total size 2022-12-19 23:16:05 +01:00
Zaiming (Stone) Shi
e932569f34 refactor: delete stale code 
'hash' strategy has been removed from config schema
hence no need to keep the compatibility code
 2022-12-19 20:20:58 +01:00
Zaiming (Stone) Shi
ad3a793910 fix(shared_sub): insert alive pid table at mnesia table event 2022-12-19 20:12:38 +01:00
Zaiming (Stone) Shi
4f1fb0b629 refactor(shared_sub): improve sticky strategy performance 
* avoid calling ets:select twice for sticky strategy
* when the calling process is terminating, no loop-back dispatch
 2022-12-19 20:04:33 +01:00
Zaiming (Stone) Shi
29f394aa70 fix(shared): handle unsubscribe for sticky strategy 
prior to this change, the message is dispatched to a shared subscriber
even after unsubscribed
 2022-12-19 19:12:38 +01:00
Zaiming (Stone) Shi
67718ef84f docs: prepare to deprecate broker.shared_dispatch_ack_enabled 2022-12-19 17:07:52 +01:00
Stefan Strigler
7aee1a08aa
Merge pull request #9531 from sstrigler/EMQX-7982-put-authentication-id-should-return-204 
refactor: return `204` instead of `200` for `PUT /authenticator/:id`
 2022-12-19 14:46:31 +01:00
Andrew Mayorov
cbe54c001c
Merge pull request #9510 from keynslug/chore/otp-25-compat 
chore: enable OTP-25 compatibility
 2022-12-19 15:12:19 +04:00
Stefan Strigler
9d97775567 refactor: return 204 instead of 200 for PUT /authenticator/:id 2022-12-19 10:02:08 +01:00
JimMoen
4906da9796
test(rule-funcs): test compression funcs 2022-12-19 15:43:50 +08:00
JimMoen
5151242583
feat(rule-engine): zip, gzip, zip_compress funcs in rule-sql 2022-12-19 15:43:50 +08:00
Andrew Mayorov
ed2d5aa48a
fix: handle status in format_status/1 callback 
Which is expected argument type for this callback. Also try to
make sure that random maps won't pass through this callback unnoticed.
 2022-12-16 13:45:06 +03:00
Andrew Mayorov
41a4e41d0c
fix(ct): prevent testcase flapping 
Testcase `t_log_file` was flapping when run happened to pass through
boundary of a second. Archive files' `ctime`s could differ between
consecutive archive downloads.
 2022-12-16 13:45:06 +03:00
Andrew Mayorov
4f00f0968d
ci: update to emqx-plugin-template 5.0.0 2022-12-16 13:45:06 +03:00
Andrew Mayorov
fed7c80a9a
fix(ci): spec function with explicit exception 2022-12-16 13:45:05 +03:00
Andrew Mayorov
8a0ca38a77
fix: drop no longer supported dialyzer option 2022-12-16 13:45:05 +03:00
Andrew Mayorov
c301c8e992
chore: drop few unused macros / includes 2022-12-16 13:45:04 +03:00
Andrew Mayorov
a614e3065a
chore(tls): drop OTP-22-related workaround 2022-12-16 13:45:04 +03:00
JimMoen
23d5d5329e
test(session): ensure 'message.dropped' hook ran with named reason 2022-12-15 16:16:27 +08:00
JimMoen
b88398c3c6
fix: run message.dropped hook, inc messages.dropped metrics 
- when awaiting_rel full
- packet identifier in use (QoS2 packet resend)
 2022-12-15 16:16:26 +08:00
Zaiming (Stone) Shi
9e3da5b661 chore: bump app versions 2022-12-14 20:07:41 +01:00
Zaiming (Stone) Shi
56066a03b5 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 20:04:20 +01:00
Zaiming (Stone) Shi
82db73c3f7 chore: bump version to v5.0.12 2022-12-14 20:02:42 +01:00
Zaiming (Stone) Shi
7137a422df refactor(emqx_authn): make error messages more readable 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
abef881a73 refactor(config): no need to explicitly remove env metadata 
after upgraded to honcon 0.32.0, the env meta is removed
by default.
 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
34f7fff97d chore: pin hocon 0.32.0 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
2d7099e3ae refactor(emqx_bridge_resource): rename a variable 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
42c58e2a91 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 15:29:13 +01:00
Thales Macedo Garitezi
f827062f0b
Merge pull request #9449 from thalesmg/gcp-pubsub-ee50 
feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0)
 2022-12-14 10:31:12 -03:00
Zaiming (Stone) Shi
0412826dd5 refactor: change ce | ee to v | prefix for shell prompt 
reason: 'ce' (Community Edition) is only for internal use,
when it comes to user/customer facing descriptions,
we should use Opensource edition and Enterprise edition.

Similary, for user/customer facing shell prompt,
use `v` for Opensource edition and `e` for Enterprise
 2022-12-14 13:05:44 +01:00
Stefan Strigler
7bdb029745 refactor: use POST not PUT for /users/{name}/change_pwd 2022-12-13 11:08:54 +01:00
Thales Macedo Garitezi
464d0a5057 refactor(test): use a linked janitor for test teardown 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
b9bc82f87a feat(gcp_pubsub): add local_topic config 
Given the implicit convention that an egress bridge containing the
`local_topic` config will forward messages without the need for a rule
action, this was added to avoid needing a rule action.
 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
a095867358 test(refactor): add ids to namespace on_exit callbacks 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
697b3ecf2d docs: improve descriptions 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
1cd91a24e9 feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0) 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
fffce9316c test(fix): avoid trying to connect to system_monitor db in tests 
When `system_monitor`'s application is loaded, it sets a default
hostname:

```erlang
1> application:get_env(system_monitor, db_hostname).
undefined
2> application:load(system_monitor).
ok
3> application:get_env(system_monitor, db_hostname).
{ok,"localhost"}
```

And that makes tests crash when somehow it gets loaded.

By having a semantic `enable` field, we could avoid starting this
application by checking if the hostname is filled or not.

```
=ERROR REPORT==== 1-Dec-2022::13:57:20.855070 ===
** Generic server <0.7646.3> terminating
** Last message in was {command,epgsql_cmd_connect,
                                #{codecs => [],database => "postgres",
                                  host => [],
                                  password =>
                                      #Fun<epgsql_cmd_connect.0.29916615>,
                                  port => 5432,timeout => 5000,
                                  username => "system_monitor"}}
** When Server state == {state,undefined,undefined,<<>>,undefined,on_message,
                               undefined,
                               {[],[]},
                               undefined,undefined,undefined,undefined,[],
                               information_redacted,[],undefined,undefined,
                               undefined,undefined,undefined,#{}}
** Reason for termination ==
** {badarg,
       [{gen_tcp,connect_0,4,[{file,"gen_tcp.erl"},{line,207}]},
        {epgsql_cmd_connect,open_socket,2,
            [{file,
                 "/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl"},
             {line,94}]},
        {epgsql_cmd_connect,execute,2,
            [{file,
                 "/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl"},
             {line,54}]},
        {epgsql_sock,command_exec,4,
            [{file,"/emqx/_build/default/lib/epgsql/src/epgsql_sock.erl"},
             {line,338}]},
        {gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},
        {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},
        {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
** Client system_monitor_pg stacktrace
** [{gen,do_call,4,[{file,"gen.erl"},{line,214}]},
    {gen_server,call,3,[{file,"gen_server.erl"},{line,243}]},
    {epgsql,call_connect,2,
            [{file,"/emqx/_build/default/lib/epgsql/src/epgsql.erl"},
             {line,203}]},
    {system_monitor_pg,connect,0,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,151}]},
    {system_monitor_pg,initialize,0,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,142}]},
    {system_monitor_pg,handle_info,2,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,92}]},
    {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,695}]},
    {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,771}]},
    {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]

=CRASH REPORT==== 1-Dec-2022::13:57:20.855697 ===
  crasher:
    initial call: epgsql_sock:init/1
    pid: <0.7646.3>
    registered_name: []
    exception exit: badarg
      in function  gen_tcp:connect_0/4 (gen_tcp.erl, line 207)
      in call from epgsql_cmd_connect:open_socket/2 (/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl, line 94)
      in call from epgsql_cmd_connect:execute/2 (/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl, line 54)
      in call from epgsql_sock:command_exec/4 (/emqx/_build/default/lib/epgsql/src/epgsql_sock.erl, line 338)
      in call from gen_server:try_handle_call/4 (gen_server.erl, line 721)
      in call from gen_server:handle_msg/6 (gen_server.erl, line 750)
    ancestors: [system_monitor_pg,system_monitor2_sup,system_monitor_sup,
                  <0.4297.3>]
    message_queue_len: 0
    messages: []
    links: [<0.4303.3>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1598
    stack_size: 29
    reductions: 734
  neighbours:
```
 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
c69022f3c4 test(ci): clean config after test 
Trying to fix this error for profile `emqx`:

```
=CRASH REPORT==== 30-Nov-2022::13:25:46.763989 ===
  crasher:
    initial call: application_master:init/4
    pid: <0.9682.1>
    registered_name: []
    exception exit: {bad_return,
                        {{emqx_conf_app,start,[normal,[]]},
                         {emqx_conf_schema,
                             [#{kind => validation_error,path => "bridges",
                                reason => unknown_fields,
                                unknown => <<"influxdb_api_v1">>,
                                unmatched => <<"mqtt,webhook">>}]}}}
      in function  application_master:init/4 (application_master.erl, line 142)
    ancestors: [<0.9681.1>]
    message_queue_len: 1
    messages: [{'EXIT',<0.9683.1>,normal}]
    links: [<0.9681.1>,<0.44.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 29
    reductions: 195
  neighbours:

=INFO REPORT==== 30-Nov-2022::13:25:46.777895 ===
    application: emqx_conf
    exited: {bad_return,
                {{emqx_conf_app,start,[normal,[]]},
                 {emqx_conf_schema,
                     [#{kind => validation_error,path => "bridges",
                        reason => unknown_fields,
                        unknown => <<"influxdb_api_v1">>,
                        unmatched => <<"mqtt,webhook">>}]}}}
    type: temporary

%%% emqx_plugins_SUITE ==> init_per_suite: FAILED
%%% emqx_plugins_SUITE ==> {{failed_to_start_app,emqx_conf,
     {emqx_conf,
         {bad_return,
             {{emqx_conf_app,start,[normal,[]]},
              {emqx_conf_schema,
                  [#{kind => validation_error,path => "bridges",
                     reason => unknown_fields,
                     unknown => <<"influxdb_api_v1">>,
                     unmatched => <<"mqtt,webhook">>}]}}}}},
 [{emqx_common_test_helpers,start_app,4,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
       {line,227}]},
  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
  {emqx_plugins_SUITE,init_per_suite,1,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
       {line,34}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1380}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}

%%% emqx_plugins_SUITE ==> t_bad_info_json: SKIPPED
%%% emqx_plugins_SUITE ==> {tc_auto_skip,
    {failed,
        {emqx_plugins_SUITE,init_per_suite,
            {'EXIT',
                {{failed_to_start_app,emqx_conf,
                     {emqx_conf,
                         {bad_return,
                             {{emqx_conf_app,start,[normal,[]]},
                              {emqx_conf_schema,
                                  [#{kind => validation_error,
                                     path => "bridges",
                                     reason => unknown_fields,
                                     unknown => <<"influxdb_api_v1">>,
                                     unmatched => <<"mqtt,webhook">>}]}}}}},
                 [{emqx_common_test_helpers,start_app,4,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
                       {line,227}]},
                  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
                  {emqx_plugins_SUITE,init_per_suite,1,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
                       {line,34}]},
                  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
                  {test_server,run_test_case_eval1,6,
                      [{file,"test_server.erl"},{line,1380}]},
                  {test_server,run_test_case_eval,9,
                      [{file,"test_server.erl"},{line,1224}]}]}}}}}
```
 2022-12-12 17:18:13 -03:00
Thales Macedo Garitezi
34e9056779 refactor: fix typo in variable name 
Might confuse people to think it's related to `replayq`.
 2022-12-12 17:17:51 -03:00
Zaiming (Stone) Shi
f8685e3a58
Merge pull request #9437 from zhongwencool/authn-env-failed 
fix: set authentication array from ENV failed
 2022-12-12 19:12:27 +01:00
Zhongwen Deng
e57a1f2c8b fix: set authentication array from ENV failed 2022-12-12 14:18:30 +01:00
Erik Timan
0242d5f360 test: rename return_body to return_all in emqx_mgmt api test util 2022-12-12 13:29:11 +01:00
Erik Timan
8009f0a3a8 test: api returns for lwm2m client posts 
Add test coverage for HTTP 204 returns from lwm2m client API
posts. This includes a small refactoring of the request functionality
in emqx_mgmt_api_test_util.
 2022-12-12 13:28:24 +01:00
Erik Timan
a8c9d02871 fix(emqx_gateway): return 204 for lwm2m client posts 
Change the return for lwm2m client API posts from 200 to 204 since
they contain no content.
 2022-12-12 13:28:24 +01:00
Stefan Strigler
6b22a074f0 refactor: move /mqtt/sys_topics to generic /configs/sys_topics 2022-12-12 13:25:42 +01:00
Zaiming (Stone) Shi
c2eabfe5f7
Merge pull request #9523 from qzhuyan/fix/william/bridges-ingress-and-egress 
Fix/william/bridges ingress and egress
 2022-12-12 09:36:08 +01:00
Zaiming (Stone) Shi
61da68ff40 fix(emqx_schema): handshake_timeout is common for all listeners 2022-12-09 16:47:19 +01:00
William Yang
6b6bb09d4a fix: ingress only bridge causes egress bridge traffic stop 2022-12-09 15:36:20 +01:00
Stefan Strigler
85f54220b0
Merge pull request #9494 from sstrigler/EMQX-7992-post-trace-must-respond-with-409-on-duplicate-entry 
fix: return 409 in case of duplicate
 2022-12-09 15:10:23 +01:00
Zaiming (Stone) Shi
3084d1263b
Merge pull request #9519 from zmstone/1209-chore-merge-dev/ee5.0-to-release-50 
1209 chore merge dev/ee5.0 to release 50
 2022-12-09 14:29:13 +01:00
Stefan Strigler
2a27d2e781 fix: return 409 in case of duplicate 2022-12-09 13:33:34 +01:00
Zaiming (Stone) Shi
6a4fb1241b Merge remote-tracking branch 'origin/release-50' into 1209-chore-merge-dev/ee5.0-to-release-50 2022-12-09 12:54:16 +01:00
Stefan Strigler
1472cfefd0 fix: flip order of checks to return 404 before 400 2022-12-09 12:53:04 +01:00
Stefan Strigler
1f7c02ecf5 fix: return 400 if node in query doesn't look like a known node 2022-12-09 12:53:02 +01:00
Ilya Averyanov
a26964291d
Merge pull request #9496 from savonarola/fix-auth-chain 
Fix `emqx_authentication` hook cooperation with other hooks
 2022-12-09 14:48:18 +03:00
Ilya Averyanov
7ae3ecfccb chore(retainer): optimize index writes 2022-12-09 14:42:43 +03:00
Ilya Averyanov
cd0ae62995 fix(auth): fix emqx_authenticator cooperation with other 'client.authenticate' hooks 2022-12-09 13:57:48 +03:00
Zaiming (Stone) Shi
40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
lafirest
3705d5bc73
Merge pull request #9505 from lafirest/fix/psk_r50 
fix(psk): Add more PSK ciphers support
 2022-12-09 00:00:52 +08:00
Zaiming (Stone) Shi
e1f09d9899 Merge remote-tracking branch 'origin/release-50' into 1208-merge-release-50-back-to-master 2022-12-08 14:27:01 +01:00
firest
dac8845b5d test(psk): add PSK ciphers test case 2022-12-08 11:41:57 +08:00
firest
21a908167d fix(psk): Add more PSK ciphers support 2022-12-08 11:41:42 +08:00
Zaiming (Stone) Shi
dc14cd450d test: render config for emqx_conf only for ee bridge tests 2022-12-07 18:27:06 +01:00
Zaiming (Stone) Shi
c9d84b2fda test: do not load emqx_conf app when it's not found 
also remove stale code in emqx_common_test_helpers
 2022-12-07 16:49:54 +01:00
Zaiming (Stone) Shi
d89dbb7caa Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-07 16:24:13 +01:00
Zaiming (Stone) Shi
f3b069a0d9 test: fix config load for lib-ee tests 2022-12-07 15:50:50 +01:00
Zaiming (Stone) Shi
b897d64b75 test: fix emqx_frame_SUITE 2022-12-07 12:20:03 +01:00
Ilya Averyanov
6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi
31098d6c67 test: fix bridge api tests, metrics is now a separate api 2022-12-06 19:03:32 +01:00
Zaiming (Stone) Shi
bd65e8aad6 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:42:50 +01:00
Zaiming (Stone) Shi
1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Stefan Strigler
0b324da7cb refactor: move metrics out of /rules(/:id) to /rules/:id/metrics 2022-12-06 15:23:28 +01:00
Zaiming (Stone) Shi
380b75d4a9
Merge pull request #9422 from emqx/1124-refactor-avoid-creating-atom-at-run 
refactor: use static function references
 2022-12-06 14:24:05 +01:00
Zaiming (Stone) Shi
cca3421308 refactor: use static function references 2022-12-06 09:40:03 +01:00
lafirest
3e29e37691
Merge pull request #9477 from lafirest/fix/fuzzing_errors_v50 
Fix/fuzzing errors v50
 2022-12-06 08:22:46 +08:00
Zaiming (Stone) Shi
817554c7d7 refactor(emqx_frame): better error handling 2022-12-06 01:48:52 +08:00
Zaiming (Stone) Shi
16aeba24b7
Merge pull request #9472 from zmstone/1203-merge-release-50-to-master 
1203 merge release 50 to master
 2022-12-05 16:39:51 +01:00
firest
72669080a1 fix(frame): fix potential parse errors found by fuzzing test 2022-12-05 23:16:07 +08:00
Stefan Strigler
0be5c8e84b fix: Revert "refactor: return 204 instead of 200 for PUT /authenticator/:id" 
This reverts commit 3bda905e92.
 2022-12-05 15:52:16 +01:00
Stefan Strigler
2443500884 fix: return 404 for /telemetry/data in case it's disabled 2022-12-05 12:41:27 +01:00
Stefan Strigler
1ba66fc722 docs: /trace returns list not single trace 2022-12-05 11:50:52 +01:00
Stefan Strigler
f8dd8d652d docs: add swagger error codes for /trace/:name/[log|download] 2022-12-05 11:50:52 +01:00
Stefan Strigler
9100b13db6 docs: fix schema for /trace/:name/log 2022-12-05 11:50:52 +01:00
Zhongwen Deng
64ab4ca025 test: add copy data dir test case 2022-12-05 16:25:28 +08:00
Stefan Strigler
3bda905e92 refactor: return 204 instead of 200 for PUT /authenticator/:id 2022-12-05 09:19:55 +01:00
Zhongwen Deng
9b37837c9c fix: don't save cert/authz zip on disk when emqx start 2022-12-05 16:12:21 +08:00
Zaiming (Stone) Shi
9fd1864d17 Merge remote-tracking branch 'origin/release-50' 2022-12-03 12:58:42 +01:00
Stefan Strigler
5b4feca3cb
Merge pull request #9419 from sstrigler/EMQX-7863-post-authentication-id 
refactor: use PUT on `/position` to move authenticator
 2022-12-02 14:07:59 +01:00
Erik Timan
485c4ddcab
Merge pull request #9393 from olcai/add-mysql-bridge-test 
test: add basic mysql bridge test (ee5.0)
 2022-12-02 09:43:22 +01:00
lafirest
4cc13c4aa5
Merge pull request #9133 from lafirest/fix/limiter_api 
fix(limiter): add `limiter` back to the configs API
 2022-12-02 10:06:03 +08:00
Zaiming (Stone) Shi
eb017ab034 test: add test case to cover mqtt bridge config upgrade 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi
9ea22d062d refactor: make all bridges optional (required = false) 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi
7adb539ae1 chore: pin hocon 0.31.2 2022-12-01 16:50:10 +01:00
Erik Timan
28fc2b9ee9 fix(mgmt_api): add missing error codes in schema 2022-12-01 12:20:11 +01:00
Erik Timan
5461311edc test: more EE mysql bridge tests 2022-12-01 10:27:51 +01:00
Erik Timan
96bff1d32e test: improve basic tests for EE mysql bridge 2022-12-01 10:27:51 +01:00
firest
88a0c77707 fix(limiter): add limiter back to the configs API 2022-12-01 15:07:15 +08:00
Zaiming (Stone) Shi
9a6901987f feat: add mqtt bridge config upgrade converter 2022-11-30 21:01:59 +01:00
Zaiming (Stone) Shi
983e904858 fix(connector): fix ssl clear 2022-11-30 17:03:41 +01:00
firest
81586a4c8d chore: bump connector version 2022-11-30 23:14:54 +08:00
firest
51ace31f30 fix(connector): fix previous incorrectly fixes 
There was an error in the try_clear_certs2/3 clause that the argument isn't the ssl field but including it.
 2022-11-30 22:14:44 +08:00
zhongwencool
5a2265a286
Merge pull request #9407 from v-zhuravlev/master 
Update grafana dashboards
 2022-11-30 17:03:39 +08:00
Stefan Strigler
b124e64399 refactor: use PUT on /position to move authenticator 2022-11-29 17:28:37 +01:00
lafirest
e6e559710b
Merge pull request #9432 from lafirest/fix/exhook_ssl_management 
fix(exhook): fix obsolete SSL files aren't deleted after the Exhook config update
 2022-11-29 22:32:10 +08:00
Zaiming (Stone) Shi
311e4cd583
Merge pull request #9440 from zmstone/1128-chore-sync-release-branch-back-to-master 
1128 chore sync release branch back to master
 2022-11-29 11:47:01 +01:00
firest
606389edd2 chore: update changes && bump version 2022-11-29 10:41:21 +08:00
firest
f9bc700498 fix(exhook): fix obsolete SSL files aren't deleted after the Exhook config update 2022-11-29 10:40:08 +08:00
Zaiming (Stone) Shi
b398617614 chore: bump app versions 2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi
77341e7a3c chore: bump app versions 2022-11-28 21:06:33 +01:00
Zaiming (Stone) Shi
9f3a9efbad Merge remote-tracking branch 'origin/release-v50' 2022-11-28 21:02:39 +01:00
Zaiming (Stone) Shi
7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
Erik Timan
2582c1713e fix: emqx_topic:join/1 instead of filename:join/1 
filename:join/1 doesn't work on Windows for joining a topic. We now use
emqx_topic:join/1 instead.
 2022-11-28 11:00:18 +01:00
Erik Timan
95faf56077 feat: remove $queue in favor of $shared 
This commit removes support for setting shared subscriptions with the
non-standard $queue feature. Shared subscriptions is now part of the MQTT
spec (using $shared) and we will only support that from now on.
 2022-11-28 11:00:18 +01:00
Zaiming (Stone) Shi
05719daff2 chore: bump version to v5.0.11 2022-11-27 11:15:28 +01:00
Zaiming (Stone) Shi
970b3eb1a0 chore: bump to version v5.0.11-rc.1 2022-11-26 15:51:38 +01:00
Vitaly Zhuravlev
5c9fff43a2 chore: add proper units for rates on the dashboard 2022-11-25 19:12:36 +03:00
Vitaly Zhuravlev
5af1d6662f chore: update grafana dashboards 2022-11-25 19:12:25 +03:00
Zhongwen Deng
65a1f90f58 Merge branch 'global-gc-interval' of https://github.com/zhongwencool/emqx into global-gc-interval 2022-11-25 16:39:02 +08:00
Zhongwen Deng
28bede0377 chore: update i18n en description 2022-11-25 16:38:40 +08:00
zhongwencool
2fbd6c9690
Merge branch 'master' into global-gc-interval 2022-11-25 12:06:16 +08:00
lafirest
345fa65241
Merge pull request #9416 from lafirest/fix/emqx_ctl 
fix(ctl): fix unsafe `list_to_atom`
 2022-11-25 09:58:07 +08:00
Zhongwen Deng
59ac5320b1 chore: update emqx_conf/emqx_machine version 2022-11-25 07:33:08 +08:00
Zaiming (Stone) Shi
6ee475d9b1 fix(emqx_authz_api_mnesia): return the right matchers 2022-11-24 20:32:00 +01:00
JianBo He
dbdb78d38a chore: clarify the case when count returns zero 2022-11-24 20:28:50 +01:00
JianBo He
edb35c08a8 chore: refactor ms2qs function type 2022-11-24 20:14:33 +01:00
JianBo He
9786a6c267 refactor(mgmt): convert fuzzy filter func to named func 2022-11-24 20:14:33 +01:00
JianBo He
ffb3f2d2d2 chore: change emqx_live_connection tab type to ordered_set 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2022-11-24 20:14:33 +01:00
JianBo He
6d9e1e0d7a test(mgmt): cover emqx_mgmt_api:cluster_query 2022-11-24 20:14:33 +01:00
JianBo He
36de83a50a feat(cm): change emqx_channel_info to ordered_set table 2022-11-24 20:14:33 +01:00
JianBo He
b9c5a5f822 fix(delayed): return correct node name 2022-11-24 20:14:33 +01:00
JianBo He
0c1412315c chore(bpapi): ignore emqx_mgmt_api:collect_total_from_tail_nodes/3 2022-11-24 20:14:33 +01:00
JianBo He
79a2682fd3 chore: improve the no-conditions query 2022-11-24 20:14:33 +01:00
JianBo He
8a0c468b01 test: refine tests for lots of List HTTP API 2022-11-24 20:14:33 +01:00
JianBo He
09958d9a33 chore: fix diaylzer warnings 2022-11-24 20:14:33 +01:00
JianBo He
9c7bf9d601 chore: update app.src 2022-11-24 20:14:33 +01:00
JianBo He
8f7337c9d2 chore: return undefined fuzzy searching func 2022-11-24 20:14:33 +01:00
JianBo He
28d391f26c fix(mgmt): collect total number in node_query/cluster_query 2022-11-24 20:14:33 +01:00
JianBo He
1fe9c105aa refactor(mgmt): smplify the node_query/cluster_query implementation 2022-11-24 20:14:33 +01:00
JianBo He
08121e7df6 fix(mgmt): optimize the speed of query tail pages 
In the previous, when you query the tail pages, all the front of rows
will be queried out and formatted. It greatly hurts the speed of query.

Currently, we only format the final result rows. i.e, the query for the
last page of data will be 10x faster.
 2022-11-24 20:14:33 +01:00
Zaiming (Stone) Shi
6837af2308
Merge branch 'release-v50' into 0720-feat-direct-deny-when-anonymous-is-not-allowed 2022-11-24 16:07:58 +01:00
Stefan Strigler
08163feed7
Merge pull request #9401 from sstrigler/EMQX-7751-support-message-properties 
EMQX 7751 support message properties
 2022-11-24 14:04:25 +01:00
Zhongwen Deng
70b6397ada feat: support node.global_gc_interval=disabled 2022-11-24 20:35:41 +08:00
firest
03fb543ba8 fix(ctl): fix unsafe list_to_atom 
A command can be called only when it is exists
 2022-11-24 17:28:53 +08:00
firest
2bc8b00419 feat(authn): support quick deny anonymous 2022-11-24 09:44:19 +01:00
Zaiming (Stone) Shi
b879c935eb
Merge pull request #9404 from JimMoen/fix-typo-50 
fix(typo): typo in code
 2022-11-24 09:39:43 +01:00
firest
fc2793c300 fix: fix upside down function clause 2022-11-24 10:37:52 +08:00
Zhongwen Deng
282e6ff218 chore: set channel info once 2022-11-24 09:59:39 +08:00
Zaiming (Stone) Shi
67cef65fdb refactor(emqx_connector_ssl): simplify implementation 2022-11-23 23:49:43 +01:00
Zaiming (Stone) Shi
6f67e3b333 feat(emqx_map_lib): add a binary_key_map help function 2022-11-23 23:49:43 +01:00
firest
6dc5078a96 fix: fix obsolete SSL files aren't deleted after the bridge configuration update 2022-11-23 23:49:43 +01:00
Stefan Strigler
afcb33aa20 feat: support publish with properties 2022-11-23 17:04:35 +01:00