d464e2aad5
refactor: rename test resource prefix
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
66eb4ef069
test: fix inter-suite flakiness
2023-03-16 13:43:01 -03:00
f31f15e44e
chore(kafka_producer): make schema changes more backwards compatible
...
This will still require fixes to the frontend.
2023-03-16 13:43:01 -03:00
03342923b9
fix(bridge): use the same dry run prefix
...
Kafka Producer and Consumer bridges rely on this prefix for detecting
a dry run and avoid leaking atoms. At some point, this prefix was
changed, effectively disabling the check in Kafka Producer.
2023-03-16 13:43:01 -03:00
1f31a87974
fix(bridge): improve macro var usage
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-16 13:43:01 -03:00
969fa03ecc
feat: implement kafka consumer
2023-03-16 13:43:01 -03:00
094e4a2eeb
chore: fix typespec
2023-03-16 13:43:01 -03:00
b9e92173cf
test: improve cluster helper
...
1) Make each node have its own isolated data dir to avoid false
negatives.
2) Allow parameterizing the peer module.
3) Fix cluster RPC after a node joins the cluster.
2023-03-16 13:42:59 -03:00
561c25f0e3
feat: update snabbkaffe -> 1.0.7
2023-03-16 13:42:38 -03:00
91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII
...
feat: add ocsp stapling support to mqtt ssl listener (5.0)
2023-03-16 13:10:48 -03:00
1688cb7df2
Merge pull request #10152 from JimMoen/refine-changes
...
chore: refine changes for merged PRs
2023-03-16 21:11:52 +08:00
a614bdc94a
chore(ocsp): catch unexpected error when fetching ocsp response
2023-03-16 09:55:52 -03:00
d1f58d6e2d
refactor: replace macro by simple function
2023-03-16 09:32:40 -03:00
e5645a7b21
refactor: rename macros
2023-03-16 09:18:16 -03:00
884dddec79
Merge pull request #10136 from kjellwinblad/kjell/fix/default_value_ui
...
fix: schema default value so that it shows up correctly in UI
2023-03-16 12:20:08 +01:00
afd29b69c4
chore: refine i18n doc punctuation
2023-03-16 18:22:29 +08:00
e3d4deb8a0
Merge pull request #10147 from id/chore-minor-improvements
...
Chore minor improvements
2023-03-16 08:18:43 +00:00
816667d85c
Merge pull request #10123 from fix/EMQX-9136/lookup-busy-resources
...
perf(bridge-api): ask nodes' bridge listings in parallel
2023-03-16 00:07:12 +03:00
5258b4c6e9
chore: bump ekka to 0.14.5
...
to make use of erlang-rocksdb 1.7.2-emqx-9
2023-03-15 19:54:55 +01:00
164440fe83
test(resource): fix flaky test
...
Sometimes this test might retry more times, so we check the prefix
of the trace only.
2023-03-15 14:25:55 -03:00
03b95073fc
test: fix inter-suite flakiness
2023-03-15 14:25:41 -03:00
a9bc8a4464
refactor(resman): rename `ets_lookup` → `lookup_cached`
...
That way we hide the impementation details + the interface becomes
cleaner and more obvious.
2023-03-15 19:17:30 +03:00
29907875bf
test(bufworker): set `batch_time` for batch-related testcases
...
By default it's `0` since e9d3fc51
2023-03-15 19:17:30 +03:00
e411c5d5f8
refactor(resman): work with state cache atomically
...
Also ensure that cache entries are always consistent with `Data`,
so that most of the code could rely on reading the cached entry
most of the time.
2023-03-15 19:17:30 +03:00
b3e7e51094
test(bridge): drop unnecessary cleanup routines
...
Since `end_per_testcase` cleans out all the resources anyway.
2023-03-15 19:17:29 +03:00
cad6492c99
perf(bridge-api): ask bridge listings in parallel
...
Also rename response formatting functions to better clarify their
purpose.
2023-03-15 19:17:29 +03:00
0deb9925c1
docs: improve descriptions
...
Co-authored-by: William Yang <mscame@gmail.com>
2023-03-15 10:30:02 -03:00
60677bc400
chore: remove unuse code
2023-03-15 20:40:54 +08:00
c9b736ae23
Merge pull request #10141 from id/0315-chore-merge-release-50-to-master
...
0315 chore merge release 50 to master
2023-03-15 12:11:54 +00:00
13a35cc045
fix: schema default value so that it shows up correctly in UI
...
This complements PR https://github.com/emqx/emqx/pull/10124 .
The default values for duration_ms() fields needs to be formatted
as a binary string with unit to show up correctly in the dashboard
UI.
2023-03-15 11:04:16 +01:00
04378f242c
feat(ocsp_cache): give ocsp cache table an heir
2023-03-14 16:09:06 -03:00
f6707d1dd0
test: fix how ocsp client is run in tests
...
For some yet unknown reason the old test version using `open_port`
does not work in OTP 25, but works fine in OTP 24. There are no
messages at all received from The openssl client port program in OTP
25.
2023-03-14 16:09:06 -03:00
63ef2f9b79
feat: save uploaded OCSP issuer pem like other ssl certs
2023-03-14 16:09:06 -03:00
158f054187
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-14 16:09:06 -03:00
57e38c8502
refactor(ocsp): add reusable type for normalized binary URLs
2023-03-14 16:09:05 -03:00
067747c2de
docs: improve descriptions
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-14 16:08:47 -03:00
52263a0448
feat: add ocsp stapling and crl support to mqtt ssl listener
2023-03-14 16:08:47 -03:00
422597a441
test: fix flaky tests
2023-03-14 16:08:47 -03:00
65fee34fe4
test: fix inter-suite test teardowns
2023-03-14 16:08:47 -03:00
4880a849b9
chore: update emqtt -> 1.8.5
...
Needed for OCSP / CRL tests because of a bug that makes emqtt hang
forever on TLS handshake errors.
2023-03-14 16:08:47 -03:00
9f9d16dd48
Merge pull request #10117 from SergeTupchiy/EMQX-8889_copy_plugins_on_joining_a_cluster
...
fix: copy plugins to a new node joining a cluster
2023-03-14 21:07:12 +02:00
6cbc17f110
Merge pull request #10135 from zmstone/0314-upgrade-to-hocon-0.37.0
...
chore: upgrade to hocon 0.37.0
2023-03-14 19:55:41 +01:00
255117f2c4
chore: bump vsn e5.0.2-alpha.1
2023-03-14 19:51:35 +01:00
bde4215a11
fix: copy plugins to a new node joining a cluster
...
Closes: EMQX-8889
2023-03-14 20:42:57 +02:00
fffb92ebb7
test(quic): make tc run independent
2023-03-14 11:46:24 +01:00
4c28b9f839
chore(test): use power of 2 recvbuff size in quic test
...
Make msquic assertions happy while in debug mode.
2023-03-14 11:46:24 +01:00
18b145ab67
chore: upgrade to hocon 0.37.0
...
hocon 0.37.0 introduced schema field aliases
this change does not impact anything until we start using aliases
2023-03-14 11:39:55 +01:00
4f1da5e9fa
Merge pull request #10132 from zhongwencool/systemd-stop-crash-log
...
fix: stop port apps when emqx_machine_terminator shutdown
2023-03-14 17:40:26 +08:00
d853085259
Merge pull request #10119 from zhongwencool/fix-statsd-empty-server
...
fix: don't allow empty string in server_port schema
2023-03-14 16:37:15 +08:00
0738444da9
fix: stop port apps when emqx_machine_terminator shutdown
2023-03-14 15:39:50 +08:00
a530ccbe3d
Merge pull request #10095 from fix/EEC-782/mysql-prepstmt-exhaustion
...
fix(mysql): be explicit that batch queries are parameterless
2023-03-14 10:21:45 +03:00
80205d9704
chore: apply code review
2023-03-14 10:54:05 +08:00
ee2847dcd9
test: make schema test happy
2023-03-14 09:17:45 +08:00
5be4d97c42
fix: don't allow empty string in server_port schema
2023-03-14 09:17:45 +08:00
823128dfc7
Merge pull request #10127 from thalesmg/rm-rebar3-emqx-app-master
...
chore: remove committed rebar3 binary
2023-03-13 16:33:57 -03:00
87ede1b1f7
Merge pull request #10124 from kjellwinblad/kjell/mongodb/default_heartbeat
...
fix: increase heartbeat time to avoid extreme MongoDB logging
2023-03-13 18:56:49 +01:00
25c28eab81
chore: remove committed rebar3 binary
...
We already set up our custom rebar3 binary in CI, so this doesn't need
to be committed. It is also outdated with our currently used version.
2023-03-13 14:11:39 -03:00
62aec87c56
Merge pull request #10107 from sstrigler/EMQX-9147-bridges-api-fix-handling-unknown-bridge-id
...
Bridges API: fix handling unknown bridge-id
2023-03-13 16:40:46 +01:00
1cf01197bb
fix: increase heartbeat time to avoid extreme MongoDB logging
...
Our MongoDB driver creates a new temporary connection, for every active
connection, to just do a single heartbeat test. There is configurable
delay between every heartbeat test. When the user has an EMQX cluster
with a MongoDB bridge (to a MongoDB replica set), there will be a lot of
connections. Furthermore, as MongoDB creates a log entry every time a
new connection is created, the log will be flooded with info about new
connection. One user have reported more than 1MB of log data in a 10
minute period.
This commit tries to fix this by increasing the default delay between
heartbeats. A better fix would be to change the MongoDB driver so that
it does not create a new connection just to do a heartbeat check, but
this is more complicated so we leave this to the future. We might also
swap out the current MongoDB driver to something better.
Fixes:
https://github.com/emqx/emqx/issues/9851
2023-03-13 14:58:53 +01:00
0c5ab5183a
Merge pull request #10109 from zmstone/0308-stop-releasing-schema.json-in-rel-package
...
0308 stop releasing schema.json in rel package
2023-03-13 14:27:15 +01:00
53bc27e0f4
refactor(bridge): avoid unnecessary `maps:to_list/1` when listing
2023-03-13 14:49:38 +03:00
686bf8255b
fix(bridge): reply `emqx_resource:get_instance/1` from cache
...
The resource manager may be busy at times, so this change ensures that
getting resource instance state will not block. Currently, no users of
`emqx_resource:get_instance/1` do seem to be relying on state being
"as-actual-as-possible" guarantee it was providing.
2023-03-13 14:35:08 +03:00
baf39fe080
Merge pull request #10098 from kjellwinblad/kjell/fix/mongo_authz_crash
...
fix: mongodb authz crash
2023-03-13 10:46:49 +01:00
c1adf0de1f
test(emqx_bridge): increase coverage by adding common edge cases
2023-03-13 10:40:47 +01:00
478601ee41
fix(emqx_bridge): remove unreachable code paths
2023-03-13 10:40:47 +01:00
dba95ec0fa
style(emqx_bridge): fix wording
2023-03-13 10:40:47 +01:00
71ec77a2f2
fix(emqx_bridge): for node operation check if node is part of cluster
...
This fixes the case where we returned `501 NOT IMPLEMENTED` in the past.
2023-03-13 10:40:47 +01:00
7124600a71
docs(emqx_bridge): add `start` as operation to bridge
2023-03-13 10:38:51 +01:00
1bcc5623ed
fix(emqx_bridge): check if bridge enabled before calling op
2023-03-13 10:38:49 +01:00
a325133391
fix(emqx_bridge): don't crash checking if bridge enabled
2023-03-13 10:37:28 +01:00
80b81748df
fix(emqx_bridge): handle bridge not found in call_operation
2023-03-13 10:37:28 +01:00
fb3d101b3a
refactor(emqx_bridge): fix var names
2023-03-13 10:37:28 +01:00
e31f4d6091
refactor(emqx_bridge): add BAD_REQUEST macro and minor cleanups
2023-03-13 10:37:28 +01:00
e8cf604ad4
Merge pull request #9893 from lafirest/feat/filiter_banned
...
feat(banned): filter out banned messages from mqueue
2023-03-13 09:58:39 +08:00
e3595f2e79
chore(mria): Bump version to 0.4.0
2023-03-11 00:37:25 +01:00
e571b602b8
chore: bump app versions + add changelog
...
Bump `emqx_plugin_libs` app version to 4.3.7.
2023-03-10 18:43:19 +03:00
781942cc4f
chore: drop unused macrodefs
2023-03-10 18:42:04 +03:00
0a7f6c7d03
fix(mysql): ensure proper escaping in batch inserts
...
Also hexencode non-utf8 binaries. This is essentially an heuristic.
We don't know column types in runtime, and there's no simple way
to find them out. Since we're already doing full binary scan during
escaping it should be cheap to bail out on non-utf8 strings and
hexencode them instead.
Also introduce separate function to highlight that this escaping
is MySQL-specific.
2023-03-10 18:42:04 +03:00
fc37d9b3cd
fix(mysql): be explicit that batch queries are parameterless
...
So that mysql client won't attempt to prepare them automatically, thus
trashing the server's prepared statements table and making interaction
overall heavier.
2023-03-10 18:42:04 +03:00
a86d06f043
chore: bump app versions following last merge-back
2023-03-10 16:44:15 +03:00
5650ad20f8
Merge pull request #10100 from savonarola/0308-fix-scram-auth
...
Fix channel crash for slow clients with enhanced authentication
2023-03-10 12:58:43 +02:00
951e89134c
Merge remote-tracking branch 'upstream/release-50' into 0310-merge-release-50-to-master
2023-03-10 08:48:32 +01:00
3f461f312a
chore: bump version to e5.0.1 and v5.0.20
2023-03-09 18:34:26 +01:00
6ebc31a929
Merge pull request #10104 from JimMoen/refine-changes-for-release-50
...
Refine changes for release 50
2023-03-09 18:33:00 +01:00
cba0287439
fix(mqtt): fix channel crash for slow clients with enhanced authn
2023-03-09 17:27:24 +02:00
aa57ea9ee1
fix: mongodb authz crash
...
This fixes a crash with an error in the log file (see below) that
happened when the MongoDB authorization module queried the database. The
reason is that the collection name that was sent to the mongodb
connection was an atom. This is fixed by making sure it is not an atom.
2023-03-08T17:16:34.215523+01:00 [error] msg: query_mongo_error, mfa:
emqx_authz_mongodb:authorize/4, line: 95, peername: 127.0.0.1:53212,
clientid: client123, collection: mqtt_acl, filter: #{username =>
<<"emqx_u">>}, reason: {resource_error,#{msg => #{error =>
{error,{error_cannot_parse_response,{op_msg_response,#{<<"code">> =>
73,<<"codeName">> => <<"InvalidNamespace">>,<<"errmsg">> => <<"Failed to
parse namespace element">>,<<"ok">> => 0.0}}}},id =>
<<"emqx_authz_mongodb:3">>,name => call_query,request =>
{find,mqtt_acl,#{username => <<"emqx_u">>},#{}},stacktrace =>
[{mc_connection_man,reply,1,[{file,"mc_connection_man.erl"},{line,123}],
...]}, reason => exception}}, resource_id: <<"emqx_authz_mongodb:3">>
Fixes: https://github.com/emqx/emqx/issues/9783
2023-03-09 16:01:23 +01:00
3463cd0d37
refactor: generate document related artifacts to _build/docgen
2023-03-09 09:32:03 +01:00
842cb7fadc
fix: debug log for rulesql parse failed
...
See also: PR#10059.
The other error logs will be printed in `emqx_rule_runtime:apply_rule/3`
2023-03-09 11:28:27 +08:00
fe27604010
Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master
2023-03-08 16:46:45 +01:00
c773f5f5e1
chore: bump version to e5.0.1-rc.2
2023-03-08 13:55:37 +01:00
b8b1a78025
feat(bridges): integrate the DynamoDB into data bridges
2023-03-08 11:13:38 +08:00
158b4a95df
Merge pull request #10079 from qzhuyan/fix/william/i18n-broker_shared_subscription_strategy
...
fix: i18n desc of broker_shared_subscription_strategy
2023-03-07 21:57:07 +01:00
26b29185b2
test(emqx_bridge_webhook_SUITE): fix flakyness in test web server
2023-03-07 20:57:38 +01:00
163b33ab28
test: remove unnecessary dependencies of ee apps
2023-03-07 20:57:38 +01:00
ca947e3e70
fix: lost messages when HTTP connection times out
...
When using async mode with the webhook bridge, queued messages that are
not fully processed when the connection times out could be lost. This
commit fixes this by letting the bridge return a recoverable_error when
this happen. The message send will then be retried in sync mode by the
emqx_resource_buffer_worker.
Fixes: https://emqx.atlassian.net/browse/EMQX-8974
2023-03-07 20:57:19 +01:00
609cd01a35
Merge pull request #10085 from sstrigler/EMQX-8552-authorization-sources-type-status-move-shouldnt-exist-when-authorization-sources-type-doesnt-exist
...
emqx_authz API: return 404 for all requests on non existent source
2023-03-07 16:19:28 +01:00
b54f444263
fix(emqx_authz): return `404` for requests on non existent source
2023-03-07 13:51:06 +01:00
a7605fba94
test(emqx_authz): use snabbkaffe:retry instead of timer:sleep
...
also use emqx_json rather than jiffy or jsx directly
2023-03-07 13:49:46 +01:00
6f88cb7d9e
chore(ekka): Bump version to 0.14.3
2023-03-07 12:07:07 +01:00
0a5b221984
Merge pull request #10055 from zhongwencool/fix-max-awaiting-rel
...
fix: update max_awaiting_rel in session
2023-03-07 18:37:20 +08:00
Thales Macedo Garitezi