yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,064 Commits 393 Branches 684 Tags 74 MiB
cf71f9148e
Commit Graph

6595 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
1f31a87974 fix(bridge): improve macro var usage 
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
969fa03ecc feat: implement kafka consumer 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
094e4a2eeb chore: fix typespec 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
b9e92173cf test: improve cluster helper 
1) Make each node have its own isolated data dir to avoid false
negatives.
2) Allow parameterizing the peer module.
3) Fix cluster RPC after a node joins the cluster.
 2023-03-16 13:42:59 -03:00
Thales Macedo Garitezi
561c25f0e3 feat: update snabbkaffe -> 1.0.7 2023-03-16 13:42:38 -03:00
Thales Macedo Garitezi
91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII 
feat: add ocsp stapling support to mqtt ssl listener (5.0)
 2023-03-16 13:10:48 -03:00
JimMoen
1688cb7df2
Merge pull request #10152 from JimMoen/refine-changes 
chore: refine changes for merged PRs
 2023-03-16 21:11:52 +08:00
Thales Macedo Garitezi
a614bdc94a chore(ocsp): catch unexpected error when fetching ocsp response 2023-03-16 09:55:52 -03:00
Thales Macedo Garitezi
d1f58d6e2d refactor: replace macro by simple function 2023-03-16 09:32:40 -03:00
Thales Macedo Garitezi
e5645a7b21 refactor: rename macros 2023-03-16 09:18:16 -03:00
Kjell Winblad
884dddec79
Merge pull request #10136 from kjellwinblad/kjell/fix/default_value_ui 
fix: schema default value so that it shows up correctly in UI
 2023-03-16 12:20:08 +01:00
JimMoen
afd29b69c4
chore: refine i18n doc punctuation 2023-03-16 18:22:29 +08:00
Ivan Dyachkov
e3d4deb8a0
Merge pull request #10147 from id/chore-minor-improvements 
Chore minor improvements
 2023-03-16 08:18:43 +00:00
Andrew Mayorov
816667d85c
Merge pull request #10123 from fix/EMQX-9136/lookup-busy-resources 
perf(bridge-api): ask nodes' bridge listings in parallel
 2023-03-16 00:07:12 +03:00
Ivan Dyachkov
5258b4c6e9 chore: bump ekka to 0.14.5 
to make use of erlang-rocksdb 1.7.2-emqx-9
 2023-03-15 19:54:55 +01:00
Thales Macedo Garitezi
164440fe83 test(resource): fix flaky test 
Sometimes this test might retry more times, so we check the prefix
of the trace only.
 2023-03-15 14:25:55 -03:00
Thales Macedo Garitezi
03b95073fc test: fix inter-suite flakiness 2023-03-15 14:25:41 -03:00
Andrew Mayorov
a9bc8a4464
refactor(resman): rename ets_lookup → lookup_cached 
That way we hide the impementation details + the interface becomes
cleaner and more obvious.
 2023-03-15 19:17:30 +03:00
Andrew Mayorov
29907875bf
test(bufworker): set batch_time for batch-related testcases 
By default it's `0` since e9d3fc51. This made a couple of tests prone
to flapping.
 2023-03-15 19:17:30 +03:00
Andrew Mayorov
e411c5d5f8
refactor(resman): work with state cache atomically 
Also ensure that cache entries are always consistent with `Data`,
so that most of the code could rely on reading the cached entry
most of the time.
 2023-03-15 19:17:30 +03:00
Andrew Mayorov
b3e7e51094
test(bridge): drop unnecessary cleanup routines 
Since `end_per_testcase` cleans out all the resources anyway.
 2023-03-15 19:17:29 +03:00
Andrew Mayorov
cad6492c99
perf(bridge-api): ask bridge listings in parallel 
Also rename response formatting functions to better clarify their
purpose.
 2023-03-15 19:17:29 +03:00
Thales Macedo Garitezi
0deb9925c1 docs: improve descriptions 
Co-authored-by: William Yang <mscame@gmail.com>
 2023-03-15 10:30:02 -03:00
Zhongwen Deng
60677bc400 chore: remove unuse code 2023-03-15 20:40:54 +08:00
Ivan Dyachkov
c9b736ae23
Merge pull request #10141 from id/0315-chore-merge-release-50-to-master 
0315 chore merge release 50 to master
 2023-03-15 12:11:54 +00:00
Kjell Winblad
13a35cc045 fix: schema default value so that it shows up correctly in UI 
This complements PR https://github.com/emqx/emqx/pull/10124.
The default values for duration_ms() fields needs to be formatted
as a binary string with unit to show up correctly in the dashboard
UI.
 2023-03-15 11:04:16 +01:00
Thales Macedo Garitezi
04378f242c feat(ocsp_cache): give ocsp cache table an heir 2023-03-14 16:09:06 -03:00
Thales Macedo Garitezi
f6707d1dd0 test: fix how ocsp client is run in tests 
For some yet unknown reason the old test version using `open_port`
does not work in OTP 25, but works fine in OTP 24.  There are no
messages at all received from The openssl client port program in OTP
25.
 2023-03-14 16:09:06 -03:00
Thales Macedo Garitezi
63ef2f9b79 feat: save uploaded OCSP issuer pem like other ssl certs 2023-03-14 16:09:06 -03:00
Thales Macedo Garitezi
158f054187 docs: improve descriptions 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-03-14 16:09:06 -03:00
Thales Macedo Garitezi
57e38c8502 refactor(ocsp): add reusable type for normalized binary URLs 2023-03-14 16:09:05 -03:00
Thales Macedo Garitezi
067747c2de docs: improve descriptions 
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
 2023-03-14 16:08:47 -03:00
Thales Macedo Garitezi
52263a0448 feat: add ocsp stapling and crl support to mqtt ssl listener 2023-03-14 16:08:47 -03:00
Thales Macedo Garitezi
422597a441 test: fix flaky tests 2023-03-14 16:08:47 -03:00
Thales Macedo Garitezi
65fee34fe4 test: fix inter-suite test teardowns 2023-03-14 16:08:47 -03:00
Thales Macedo Garitezi
4880a849b9 chore: update emqtt -> 1.8.5 
Needed for OCSP / CRL tests because of a bug that makes emqtt hang
forever on TLS handshake errors.
 2023-03-14 16:08:47 -03:00
SergeTupchiy
9f9d16dd48
Merge pull request #10117 from SergeTupchiy/EMQX-8889_copy_plugins_on_joining_a_cluster 
fix: copy plugins to a new node joining a cluster
 2023-03-14 21:07:12 +02:00
Zaiming (Stone) Shi
6cbc17f110
Merge pull request #10135 from zmstone/0314-upgrade-to-hocon-0.37.0 
chore: upgrade to hocon 0.37.0
 2023-03-14 19:55:41 +01:00
Stefan Strigler
255117f2c4 chore: bump vsn e5.0.2-alpha.1 2023-03-14 19:51:35 +01:00
Serge Tupchii
bde4215a11 fix: copy plugins to a new node joining a cluster 
Closes: EMQX-8889
 2023-03-14 20:42:57 +02:00
William Yang
fffb92ebb7 test(quic): make tc run independent 2023-03-14 11:46:24 +01:00
William Yang
4c28b9f839 chore(test): use power of 2 recvbuff size in quic test 
Make msquic assertions happy while in debug mode.
 2023-03-14 11:46:24 +01:00
Zaiming (Stone) Shi
18b145ab67 chore: upgrade to hocon 0.37.0 
hocon 0.37.0 introduced schema field aliases
this change does not impact anything until we start using aliases
 2023-03-14 11:39:55 +01:00
zhongwencool
4f1da5e9fa
Merge pull request #10132 from zhongwencool/systemd-stop-crash-log 
fix: stop port apps when emqx_machine_terminator shutdown
 2023-03-14 17:40:26 +08:00
zhongwencool
d853085259
Merge pull request #10119 from zhongwencool/fix-statsd-empty-server 
fix: don't allow empty string in server_port schema
 2023-03-14 16:37:15 +08:00
Zhongwen Deng
0738444da9 fix: stop port apps when emqx_machine_terminator shutdown 2023-03-14 15:39:50 +08:00
Andrew Mayorov
a530ccbe3d
Merge pull request #10095 from fix/EEC-782/mysql-prepstmt-exhaustion 
fix(mysql): be explicit that batch queries are parameterless
 2023-03-14 10:21:45 +03:00
Zhongwen Deng
80205d9704 chore: apply code review 2023-03-14 10:54:05 +08:00
Zhongwen Deng
ee2847dcd9 test: make schema test happy 2023-03-14 09:17:45 +08:00
Zhongwen Deng
5be4d97c42 fix: don't allow empty string in server_port schema 2023-03-14 09:17:45 +08:00
Thales Macedo Garitezi
823128dfc7
Merge pull request #10127 from thalesmg/rm-rebar3-emqx-app-master 
chore: remove committed rebar3 binary
 2023-03-13 16:33:57 -03:00
Kjell Winblad
87ede1b1f7
Merge pull request #10124 from kjellwinblad/kjell/mongodb/default_heartbeat 
fix: increase heartbeat time to avoid extreme MongoDB logging
 2023-03-13 18:56:49 +01:00
Thales Macedo Garitezi
25c28eab81 chore: remove committed rebar3 binary 
We already set up our custom rebar3 binary in CI, so this doesn't need
to be committed.  It is also outdated with our currently used version.
 2023-03-13 14:11:39 -03:00
Stefan Strigler
62aec87c56
Merge pull request #10107 from sstrigler/EMQX-9147-bridges-api-fix-handling-unknown-bridge-id 
Bridges API: fix handling unknown bridge-id
 2023-03-13 16:40:46 +01:00
Kjell Winblad
1cf01197bb fix: increase heartbeat time to avoid extreme MongoDB logging 
Our MongoDB driver creates a new temporary connection, for every active
connection, to just do a single heartbeat test. There is configurable
delay between every heartbeat test. When the user has an EMQX cluster
with a MongoDB bridge (to a MongoDB replica set), there will be a lot of
connections. Furthermore, as MongoDB creates a log entry every time a
new connection is created, the log will be flooded with info about new
connection. One user have reported more than 1MB of log data in a 10
minute period.

This commit tries to fix this by increasing the default delay between
heartbeats. A better fix would be to change the MongoDB driver so that
it does not create a new connection just to do a heartbeat check, but
this is more complicated so we leave this to the future. We might also
swap out the current MongoDB driver to something better.

Fixes:
https://github.com/emqx/emqx/issues/9851
 2023-03-13 14:58:53 +01:00
Zaiming (Stone) Shi
0c5ab5183a
Merge pull request #10109 from zmstone/0308-stop-releasing-schema.json-in-rel-package 
0308 stop releasing schema.json in rel package
 2023-03-13 14:27:15 +01:00
Andrew Mayorov
53bc27e0f4
refactor(bridge): avoid unnecessary maps:to_list/1 when listing 2023-03-13 14:49:38 +03:00
Andrew Mayorov
686bf8255b
fix(bridge): reply emqx_resource:get_instance/1 from cache 
The resource manager may be busy at times, so this change ensures that
getting resource instance state will not block. Currently, no users of
`emqx_resource:get_instance/1` do seem to be relying on state being
"as-actual-as-possible" guarantee it was providing.
 2023-03-13 14:35:08 +03:00
Kjell Winblad
baf39fe080
Merge pull request #10098 from kjellwinblad/kjell/fix/mongo_authz_crash 
fix: mongodb authz crash
 2023-03-13 10:46:49 +01:00
Stefan Strigler
c1adf0de1f test(emqx_bridge): increase coverage by adding common edge cases 2023-03-13 10:40:47 +01:00
Stefan Strigler
478601ee41 fix(emqx_bridge): remove unreachable code paths 2023-03-13 10:40:47 +01:00
Stefan Strigler
dba95ec0fa style(emqx_bridge): fix wording 2023-03-13 10:40:47 +01:00
Stefan Strigler
71ec77a2f2 fix(emqx_bridge): for node operation check if node is part of cluster 
This fixes the case where we returned `501 NOT IMPLEMENTED` in the past.
 2023-03-13 10:40:47 +01:00
Stefan Strigler
7124600a71 docs(emqx_bridge): add start as operation to bridge 2023-03-13 10:38:51 +01:00
Stefan Strigler
1bcc5623ed fix(emqx_bridge): check if bridge enabled before calling op 2023-03-13 10:38:49 +01:00
Stefan Strigler
a325133391 fix(emqx_bridge): don't crash checking if bridge enabled 2023-03-13 10:37:28 +01:00
Stefan Strigler
80b81748df fix(emqx_bridge): handle bridge not found in call_operation 2023-03-13 10:37:28 +01:00
Stefan Strigler
fb3d101b3a refactor(emqx_bridge): fix var names 2023-03-13 10:37:28 +01:00
Stefan Strigler
e31f4d6091 refactor(emqx_bridge): add BAD_REQUEST macro and minor cleanups 2023-03-13 10:37:28 +01:00
lafirest
e8cf604ad4
Merge pull request #9893 from lafirest/feat/filiter_banned 
feat(banned): filter out banned messages from mqueue
 2023-03-13 09:58:39 +08:00
ieQu1
e3595f2e79 chore(mria): Bump version to 0.4.0 2023-03-11 00:37:25 +01:00
Andrew Mayorov
e571b602b8
chore: bump app versions + add changelog 
Bump `emqx_plugin_libs` app version to 4.3.7.
 2023-03-10 18:43:19 +03:00
Andrew Mayorov
781942cc4f
chore: drop unused macrodefs 2023-03-10 18:42:04 +03:00
Andrew Mayorov
0a7f6c7d03
fix(mysql): ensure proper escaping in batch inserts 
Also hexencode non-utf8 binaries. This is essentially an heuristic.
We don't know column types in runtime, and there's no simple way
to find them out. Since we're already doing full binary scan during
escaping it should be cheap to bail out on non-utf8 strings and
hexencode them instead.

Also introduce separate function to highlight that this escaping
is MySQL-specific.
 2023-03-10 18:42:04 +03:00
Andrew Mayorov
fc37d9b3cd
fix(mysql): be explicit that batch queries are parameterless 
So that mysql client won't attempt to prepare them automatically, thus
trashing the server's prepared statements table and making interaction
overall heavier.
 2023-03-10 18:42:04 +03:00
Andrew Mayorov
a86d06f043
chore: bump app versions following last merge-back 2023-03-10 16:44:15 +03:00
Ilya Averyanov
5650ad20f8
Merge pull request #10100 from savonarola/0308-fix-scram-auth 
Fix channel crash for slow clients with enhanced authentication
 2023-03-10 12:58:43 +02:00
Ivan Dyachkov
951e89134c Merge remote-tracking branch 'upstream/release-50' into 0310-merge-release-50-to-master 2023-03-10 08:48:32 +01:00
Zaiming (Stone) Shi
3f461f312a chore: bump version to e5.0.1 and v5.0.20 2023-03-09 18:34:26 +01:00
Zaiming (Stone) Shi
6ebc31a929
Merge pull request #10104 from JimMoen/refine-changes-for-release-50 
Refine changes for release 50
 2023-03-09 18:33:00 +01:00
Ilya Averyanov
cba0287439 fix(mqtt): fix channel crash for slow clients with enhanced authn 2023-03-09 17:27:24 +02:00
Kjell Winblad
aa57ea9ee1 fix: mongodb authz crash 
This fixes a crash with an error in the log file (see below) that
happened when the MongoDB authorization module queried the database. The
reason is that the collection name that was sent to the mongodb
connection was an atom. This is fixed by making sure it is not an atom.

2023-03-08T17:16:34.215523+01:00 [error] msg: query_mongo_error, mfa:
emqx_authz_mongodb:authorize/4, line: 95, peername: 127.0.0.1:53212,
clientid: client123, collection: mqtt_acl, filter: #{username =>
<<"emqx_u">>}, reason: {resource_error,#{msg => #{error =>
{error,{error_cannot_parse_response,{op_msg_response,#{<<"code">> =>
73,<<"codeName">> => <<"InvalidNamespace">>,<<"errmsg">> => <<"Failed to
parse namespace element">>,<<"ok">> => 0.0}}}},id =>
<<"emqx_authz_mongodb:3">>,name => call_query,request =>
{find,mqtt_acl,#{username => <<"emqx_u">>},#{}},stacktrace =>
[{mc_connection_man,reply,1,[{file,"mc_connection_man.erl"},{line,123}],
...]}, reason => exception}}, resource_id: <<"emqx_authz_mongodb:3">>

Fixes: https://github.com/emqx/emqx/issues/9783
 2023-03-09 16:01:23 +01:00
Zaiming (Stone) Shi
3463cd0d37 refactor: generate document related artifacts to _build/docgen 2023-03-09 09:32:03 +01:00
JimMoen
842cb7fadc
fix: debug log for rulesql parse failed 
See also: PR#10059.
The other error logs will be printed in `emqx_rule_runtime:apply_rule/3`
 2023-03-09 11:28:27 +08:00
Zaiming (Stone) Shi
fe27604010 Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master 2023-03-08 16:46:45 +01:00
Zaiming (Stone) Shi
c773f5f5e1 chore: bump version to e5.0.1-rc.2 2023-03-08 13:55:37 +01:00
firest
b8b1a78025 feat(bridges): integrate the DynamoDB into data bridges 2023-03-08 11:13:38 +08:00
Zaiming (Stone) Shi
158b4a95df
Merge pull request #10079 from qzhuyan/fix/william/i18n-broker_shared_subscription_strategy 
fix: i18n desc of broker_shared_subscription_strategy
 2023-03-07 21:57:07 +01:00
Zaiming (Stone) Shi
26b29185b2 test(emqx_bridge_webhook_SUITE): fix flakyness in test web server 2023-03-07 20:57:38 +01:00
Kjell Winblad
163b33ab28 test: remove unnecessary dependencies of ee apps 2023-03-07 20:57:38 +01:00
Kjell Winblad
ca947e3e70 fix: lost messages when HTTP connection times out 
When using async mode with the webhook bridge, queued messages that are
not fully processed when the connection times out could be lost. This
commit fixes this by letting the bridge return a recoverable_error when
this happen. The message send will then be retried in sync mode by the
emqx_resource_buffer_worker.

Fixes: https://emqx.atlassian.net/browse/EMQX-8974
 2023-03-07 20:57:19 +01:00
Stefan Strigler
609cd01a35
Merge pull request #10085 from sstrigler/EMQX-8552-authorization-sources-type-status-move-shouldnt-exist-when-authorization-sources-type-doesnt-exist 
emqx_authz API: return 404 for all requests on non existent source
 2023-03-07 16:19:28 +01:00
Stefan Strigler
b54f444263 fix(emqx_authz): return 404 for requests on non existent source 2023-03-07 13:51:06 +01:00
Stefan Strigler
a7605fba94 test(emqx_authz): use snabbkaffe:retry instead of timer:sleep 
also use emqx_json rather than jiffy or jsx directly
 2023-03-07 13:49:46 +01:00
ieQu1
6f88cb7d9e chore(ekka): Bump version to 0.14.3 2023-03-07 12:07:07 +01:00
zhongwencool
0a5b221984
Merge pull request #10055 from zhongwencool/fix-max-awaiting-rel 
fix: update max_awaiting_rel in session
 2023-03-07 18:37:20 +08:00
SergeTupchiy
9949c87b91
Merge pull request #10069 from SergeTupchiy/ci/EMQX-9055-configuration-item-flush-time-interval-used-for-stats-d-integration-not-work 
fix: EMQX-9055 configuration item flush time interval used for statsd integration not work
 2023-03-07 11:43:57 +02:00
William Yang
2b1cf2fa52
Merge pull request #10058 from qzhuyan/dev/william/deprecated-unsupp-quic-TLS-options 
fix: deprecate unsupp quic TLS options
 2023-03-07 09:29:52 +01:00
William Yang
422f8e3be3
Merge pull request #10078 from qzhuyan/fix/william/inval-listener-options-segfault 
fix: inval listener option cause segfault
 2023-03-07 09:26:27 +01:00
Zaiming (Stone) Shi
133734b345 chore: bump version number to e5.0.1-rc.1 and v5.0.20 2023-03-07 08:41:13 +01:00