61cb03b45a
fix(buffer_worker): change the default `resume_interval` value and expose it as hidden config
...
Also removes the previously added alarm for request timeout.
There are situations where having a short request timeout and a long
health check interval make sense, so we don't want to alarm the user
for those situations. Instead, we automatically attempt to set a
reasonable `resume_interval` value.
2023-03-22 11:47:36 -03:00
84def357a9
Merge pull request #10143 from lafirest/feat/rocketmq
...
feat(bridges): integrate RocketMQ into data bridges
2023-03-22 20:43:22 +08:00
e43aa5d841
Merge pull request #10199 from zmstone/0321-merge-release-50-to-master
...
0321 merge release 50 to master
2023-03-22 10:20:48 +01:00
db6afb40b0
Merge pull request #10174 from zmstone/0320-upgrade-esockd-to-5.9.6
...
fix: upgrade esockd from 5.9.4 to 5.9.6
2023-03-22 09:50:38 +01:00
af7b2215ca
fix(bridge-api): unflatten nodes' bridge lists back
...
Bridge lists were erroneously flattened in cad6492c
2023-03-22 11:35:09 +03:00
9853d00cad
feat(bridges): integrate RocketMQ into data bridges
2023-03-22 10:36:52 +08:00
65c2da7ef5
Merge remote-tracking branch 'ce/master' into cassa
2023-03-22 09:30:50 +08:00
bf35419b52
Merge pull request #10190 from keynslug/fix/EMQX-9269/bridge-list
...
fix(bridge-api): unflatten nodes' bridge lists back
2023-03-22 00:53:03 +03:00
e6091db351
Merge remote-tracking branch 'origin/release-50' into 0321-merge-release-50-to-master
2023-03-21 22:03:31 +01:00
6c24b3a77d
Merge pull request #10196 from sstrigler/EMQX-9201-the-case-is-not-correct-for-some-of-menu
...
Emqx 9201 the case is not correct for some of menu
2023-03-21 19:42:10 +01:00
4bbbaf8b13
Merge pull request #10164 from thalesmg/crl-v50-mkIII
...
feat(crl): add crl check support
2023-03-21 14:42:01 -03:00
27b8445337
fix: add inflight window setting to the clickhouse bridge
...
This commit makes sure the inflight window setting is present for the
clickhouse bridge. It also changes emqx_resource_schema that previously
removed the inflight window setting from resources with query mode
`always_sync`. We don't need to do that because all bridges that uses
the buffer worker queue will get async call handling even if the bridge
don't support the async callback.
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 17:14:03 +01:00
327b782210
style: lower case in descriptions
2023-03-21 17:04:42 +01:00
d9b1b1c802
fix(bridge-api): unflatten nodes' bridge lists back
...
Bridge lists were erroneously flattened in cad6492c
2023-03-21 18:59:27 +03:00
981268911a
style: make summary lower-case
2023-03-21 16:56:37 +01:00
3880862c81
fix(emqx_bridge): return 503 for inconsistency in bridge setup
2023-03-21 15:07:43 +01:00
4b0ea562a2
refactor(emqx_bridge): consistently use macros for http response
2023-03-21 15:07:41 +01:00
84fc64822e
style: fix wording for nxdomain and malformed_username_or_password
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 15:02:29 +01:00
8af3fb4ee7
feat: move human readable error translations to emqx_misc
2023-03-21 15:02:29 +01:00
c1384b6e6e
feat(emqx_resource): include error with alarm for resource_down
2023-03-21 15:02:29 +01:00
53825b9aba
fix(emqx_bridge): propagate connection error to resource status
2023-03-21 15:02:29 +01:00
fd23800370
Merge pull request #10180 from zmstone/0320-add-emqx_ctl-readme
...
docs: add readme for emqx_ctl app
2023-03-21 11:33:16 +01:00
7d8fc3d2ad
Merge pull request #10185 from qzhuyan/dev/william/bump-quicer
...
chore(build): Support fetching binary for macOS M1 [quicer 0.0.114]
2023-03-21 11:32:33 +01:00
092daf409a
chore(testlib): update include for CI of emqtt lib
2023-03-21 09:03:31 +01:00
b6e583143f
chore: bump quicer 0.0.114
...
for fetching quicer binary for macOS M1 builds.
2023-03-21 09:01:25 +01:00
89782e6d21
docs: add info about that rule engine can handle different types of structured data
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 06:26:04 +01:00
b059bad08a
chore(acl): fully match `dashboard` username
...
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2023-03-21 09:06:46 +08:00
f2b54588e7
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-20 18:09:38 -03:00
7a71d64686
feat(crl): add crl check support
2023-03-20 18:09:38 -03:00
ee403423b9
Merge pull request #10166 from savonarola/0318-mix-build
...
chore(mix): fix mix builds
2023-03-20 22:37:21 +02:00
c739d9957c
docs: add readme for emqx_ctl app
2023-03-20 19:35:17 +01:00
6f71898546
fix: upgrade esockd from 5.9.4 to 5.9.6
...
changes in esockd are
* 5.9.5: added API to retrieve SNI
* 5.9.6: avoid error log if socket is closed before sending
proxy protocol headers
2023-03-20 19:27:32 +01:00
08e249dc4d
docs: refine README.md for the emqx_rule_engine app
...
This commit adds more content to the readme file of the `emqx_rule_engine`
app. The aim of the changes are to make it easier to understand what the
rule engine is and what it does.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9229
2023-03-20 16:53:55 +01:00
2d75c7d6d9
fix(emqx_bridge): remove metrics from non-dedicated bridge API endpoints
...
Metrics should only be exposed via the /bridges/:id/metrics endpoint,
and not in other operations such as getting the list of all bridges, or
in the response when a bridge has been created. This commit removes all
traces of metrics for the non-dedicated API endpoints.
2023-03-20 09:43:11 +01:00
377127ce72
fix(acl): fix wrong default ACL rules
2023-03-20 13:53:57 +08:00
12942b676d
Merge remote-tracking branch 'upstream/master' into cassa
2023-03-20 09:50:27 +08:00
6a668c35db
Merge pull request #10167 from zhongwencool/fix-swagger-doc-500-crash
...
fix: swagger doc 500 crash
2023-03-19 16:28:32 +08:00
b529a0bce1
test: add swagger.json api test
2023-03-19 13:05:32 +08:00
f8573ae8d7
fix: swagger doc 500 crash
2023-03-19 12:34:06 +08:00
39f22585d0
chore(mix): fix mix builds
2023-03-19 01:39:56 +02:00
149afe863f
chore: e5.0.2/v5.0.21 code freeze
2023-03-17 13:19:44 +01:00
cf1cce011b
Merge pull request #9564 from thalesmg/kafka-source-ee50
...
feat: implement kafka consumer (ee5.0)
2023-03-17 09:04:53 -03:00
1bf72c4d2e
Merge pull request #10155 from zmstone/0316-build-schema-dump-per-lang
...
build: generate per-lang schema dump
2023-03-17 11:45:35 +01:00
3c1254d873
fix: newly created listeners have no limiter restrictions
2023-03-17 14:32:42 +08:00
c0a216a740
feat(bridge): support cassandra bridge
2023-03-17 11:34:48 +08:00
ad1deedd0e
build: generate per-lang schema dump
2023-03-16 21:25:21 +01:00
20414d7373
fix(buffer_worker): check request timeout and health check interval
...
Fixes https://emqx.atlassian.net/browse/EMQX-9099
The default value for `request_timeout` is 15 seconds, and the default
resume interval is also 15 seconds (the health check timeout, if
`resume_interval` is not explicitly given). This means that, in
practice, if a buffer worker ever gets into the blocked state, then
almost all requests will timeout.
Proposed improvement:
- `request_timeout` should by default be twice as much as
health_check_interval.
- Emit a alarm if `request_timeout` is not greater than
`health_check_interval`.
2023-03-16 13:46:45 -03:00
966276127e
test: trying to make tests more stable
2023-03-16 13:43:01 -03:00
3954b7bde2
fix(bridges): function clause when a non-ingress bridge coexists with an egress bridge
...
This was not caught by our tests because we always test bridge types
in isolation. So, if the config only contains ingress-only bridges,
the `on_message_publish` hook is never installed.
In a real system, if there are bridges of mixed types in the config,
the hook might be installed, and `emqx_bridge:get_matched_bridge_id`
would crash when iterating over the ingress bridges.
2023-03-16 13:43:01 -03:00
947e014132
test: improve cluster node helpers
...
* Add option to start autocluster. This is useful for scenarios where
a cluster is already running and has some configurations set (via
config handler/cluster rpc) and later another node joins.
* Improve mnesia data directory isolation for nodes. A new dir is set
so that we may avoid intra and inter-suite flakiness due to dirty
tables and schema.
* The janitor is now called synchronously. This ensures the cleanup
is done before the test pid dies.
2023-03-16 13:43:01 -03:00
d464e2aad5
refactor: rename test resource prefix
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
66eb4ef069
test: fix inter-suite flakiness
2023-03-16 13:43:01 -03:00
f31f15e44e
chore(kafka_producer): make schema changes more backwards compatible
...
This will still require fixes to the frontend.
2023-03-16 13:43:01 -03:00
03342923b9
fix(bridge): use the same dry run prefix
...
Kafka Producer and Consumer bridges rely on this prefix for detecting
a dry run and avoid leaking atoms. At some point, this prefix was
changed, effectively disabling the check in Kafka Producer.
2023-03-16 13:43:01 -03:00
1f31a87974
fix(bridge): improve macro var usage
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-16 13:43:01 -03:00
969fa03ecc
feat: implement kafka consumer
2023-03-16 13:43:01 -03:00
094e4a2eeb
chore: fix typespec
2023-03-16 13:43:01 -03:00
b9e92173cf
test: improve cluster helper
...
1) Make each node have its own isolated data dir to avoid false
negatives.
2) Allow parameterizing the peer module.
3) Fix cluster RPC after a node joins the cluster.
2023-03-16 13:42:59 -03:00
561c25f0e3
feat: update snabbkaffe -> 1.0.7
2023-03-16 13:42:38 -03:00
91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII
...
feat: add ocsp stapling support to mqtt ssl listener (5.0)
2023-03-16 13:10:48 -03:00
1688cb7df2
Merge pull request #10152 from JimMoen/refine-changes
...
chore: refine changes for merged PRs
2023-03-16 21:11:52 +08:00
a614bdc94a
chore(ocsp): catch unexpected error when fetching ocsp response
2023-03-16 09:55:52 -03:00
d1f58d6e2d
refactor: replace macro by simple function
2023-03-16 09:32:40 -03:00
e5645a7b21
refactor: rename macros
2023-03-16 09:18:16 -03:00
884dddec79
Merge pull request #10136 from kjellwinblad/kjell/fix/default_value_ui
...
fix: schema default value so that it shows up correctly in UI
2023-03-16 12:20:08 +01:00
afd29b69c4
chore: refine i18n doc punctuation
2023-03-16 18:22:29 +08:00
e3d4deb8a0
Merge pull request #10147 from id/chore-minor-improvements
...
Chore minor improvements
2023-03-16 08:18:43 +00:00
816667d85c
Merge pull request #10123 from fix/EMQX-9136/lookup-busy-resources
...
perf(bridge-api): ask nodes' bridge listings in parallel
2023-03-16 00:07:12 +03:00
5258b4c6e9
chore: bump ekka to 0.14.5
...
to make use of erlang-rocksdb 1.7.2-emqx-9
2023-03-15 19:54:55 +01:00
164440fe83
test(resource): fix flaky test
...
Sometimes this test might retry more times, so we check the prefix
of the trace only.
2023-03-15 14:25:55 -03:00
03b95073fc
test: fix inter-suite flakiness
2023-03-15 14:25:41 -03:00
a9bc8a4464
refactor(resman): rename `ets_lookup` → `lookup_cached`
...
That way we hide the impementation details + the interface becomes
cleaner and more obvious.
2023-03-15 19:17:30 +03:00
29907875bf
test(bufworker): set `batch_time` for batch-related testcases
...
By default it's `0` since e9d3fc51
2023-03-15 19:17:30 +03:00
e411c5d5f8
refactor(resman): work with state cache atomically
...
Also ensure that cache entries are always consistent with `Data`,
so that most of the code could rely on reading the cached entry
most of the time.
2023-03-15 19:17:30 +03:00
b3e7e51094
test(bridge): drop unnecessary cleanup routines
...
Since `end_per_testcase` cleans out all the resources anyway.
2023-03-15 19:17:29 +03:00
cad6492c99
perf(bridge-api): ask bridge listings in parallel
...
Also rename response formatting functions to better clarify their
purpose.
2023-03-15 19:17:29 +03:00
0deb9925c1
docs: improve descriptions
...
Co-authored-by: William Yang <mscame@gmail.com>
2023-03-15 10:30:02 -03:00
60677bc400
chore: remove unuse code
2023-03-15 20:40:54 +08:00
c9b736ae23
Merge pull request #10141 from id/0315-chore-merge-release-50-to-master
...
0315 chore merge release 50 to master
2023-03-15 12:11:54 +00:00
13a35cc045
fix: schema default value so that it shows up correctly in UI
...
This complements PR https://github.com/emqx/emqx/pull/10124 .
The default values for duration_ms() fields needs to be formatted
as a binary string with unit to show up correctly in the dashboard
UI.
2023-03-15 11:04:16 +01:00
04378f242c
feat(ocsp_cache): give ocsp cache table an heir
2023-03-14 16:09:06 -03:00
f6707d1dd0
test: fix how ocsp client is run in tests
...
For some yet unknown reason the old test version using `open_port`
does not work in OTP 25, but works fine in OTP 24. There are no
messages at all received from The openssl client port program in OTP
25.
2023-03-14 16:09:06 -03:00
63ef2f9b79
feat: save uploaded OCSP issuer pem like other ssl certs
2023-03-14 16:09:06 -03:00
158f054187
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-14 16:09:06 -03:00
57e38c8502
refactor(ocsp): add reusable type for normalized binary URLs
2023-03-14 16:09:05 -03:00
067747c2de
docs: improve descriptions
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-14 16:08:47 -03:00
52263a0448
feat: add ocsp stapling and crl support to mqtt ssl listener
2023-03-14 16:08:47 -03:00
422597a441
test: fix flaky tests
2023-03-14 16:08:47 -03:00
65fee34fe4
test: fix inter-suite test teardowns
2023-03-14 16:08:47 -03:00
4880a849b9
chore: update emqtt -> 1.8.5
...
Needed for OCSP / CRL tests because of a bug that makes emqtt hang
forever on TLS handshake errors.
2023-03-14 16:08:47 -03:00
9f9d16dd48
Merge pull request #10117 from SergeTupchiy/EMQX-8889_copy_plugins_on_joining_a_cluster
...
fix: copy plugins to a new node joining a cluster
2023-03-14 21:07:12 +02:00
6cbc17f110
Merge pull request #10135 from zmstone/0314-upgrade-to-hocon-0.37.0
...
chore: upgrade to hocon 0.37.0
2023-03-14 19:55:41 +01:00
255117f2c4
chore: bump vsn e5.0.2-alpha.1
2023-03-14 19:51:35 +01:00
bde4215a11
fix: copy plugins to a new node joining a cluster
...
Closes: EMQX-8889
2023-03-14 20:42:57 +02:00
fffb92ebb7
test(quic): make tc run independent
2023-03-14 11:46:24 +01:00
4c28b9f839
chore(test): use power of 2 recvbuff size in quic test
...
Make msquic assertions happy while in debug mode.
2023-03-14 11:46:24 +01:00
18b145ab67
chore: upgrade to hocon 0.37.0
...
hocon 0.37.0 introduced schema field aliases
this change does not impact anything until we start using aliases
2023-03-14 11:39:55 +01:00
4f1da5e9fa
Merge pull request #10132 from zhongwencool/systemd-stop-crash-log
...
fix: stop port apps when emqx_machine_terminator shutdown
2023-03-14 17:40:26 +08:00
d853085259
Merge pull request #10119 from zhongwencool/fix-statsd-empty-server
...
fix: don't allow empty string in server_port schema
2023-03-14 16:37:15 +08:00
0738444da9
fix: stop port apps when emqx_machine_terminator shutdown
2023-03-14 15:39:50 +08:00
a530ccbe3d
Merge pull request #10095 from fix/EEC-782/mysql-prepstmt-exhaustion
...
fix(mysql): be explicit that batch queries are parameterless
2023-03-14 10:21:45 +03:00
80205d9704
chore: apply code review
2023-03-14 10:54:05 +08:00
ee2847dcd9
test: make schema test happy
2023-03-14 09:17:45 +08:00
5be4d97c42
fix: don't allow empty string in server_port schema
2023-03-14 09:17:45 +08:00
823128dfc7
Merge pull request #10127 from thalesmg/rm-rebar3-emqx-app-master
...
chore: remove committed rebar3 binary
2023-03-13 16:33:57 -03:00
87ede1b1f7
Merge pull request #10124 from kjellwinblad/kjell/mongodb/default_heartbeat
...
fix: increase heartbeat time to avoid extreme MongoDB logging
2023-03-13 18:56:49 +01:00
25c28eab81
chore: remove committed rebar3 binary
...
We already set up our custom rebar3 binary in CI, so this doesn't need
to be committed. It is also outdated with our currently used version.
2023-03-13 14:11:39 -03:00
62aec87c56
Merge pull request #10107 from sstrigler/EMQX-9147-bridges-api-fix-handling-unknown-bridge-id
...
Bridges API: fix handling unknown bridge-id
2023-03-13 16:40:46 +01:00
1cf01197bb
fix: increase heartbeat time to avoid extreme MongoDB logging
...
Our MongoDB driver creates a new temporary connection, for every active
connection, to just do a single heartbeat test. There is configurable
delay between every heartbeat test. When the user has an EMQX cluster
with a MongoDB bridge (to a MongoDB replica set), there will be a lot of
connections. Furthermore, as MongoDB creates a log entry every time a
new connection is created, the log will be flooded with info about new
connection. One user have reported more than 1MB of log data in a 10
minute period.
This commit tries to fix this by increasing the default delay between
heartbeats. A better fix would be to change the MongoDB driver so that
it does not create a new connection just to do a heartbeat check, but
this is more complicated so we leave this to the future. We might also
swap out the current MongoDB driver to something better.
Fixes:
https://github.com/emqx/emqx/issues/9851
2023-03-13 14:58:53 +01:00
0c5ab5183a
Merge pull request #10109 from zmstone/0308-stop-releasing-schema.json-in-rel-package
...
0308 stop releasing schema.json in rel package
2023-03-13 14:27:15 +01:00
53bc27e0f4
refactor(bridge): avoid unnecessary `maps:to_list/1` when listing
2023-03-13 14:49:38 +03:00
686bf8255b
fix(bridge): reply `emqx_resource:get_instance/1` from cache
...
The resource manager may be busy at times, so this change ensures that
getting resource instance state will not block. Currently, no users of
`emqx_resource:get_instance/1` do seem to be relying on state being
"as-actual-as-possible" guarantee it was providing.
2023-03-13 14:35:08 +03:00
baf39fe080
Merge pull request #10098 from kjellwinblad/kjell/fix/mongo_authz_crash
...
fix: mongodb authz crash
2023-03-13 10:46:49 +01:00
c1adf0de1f
test(emqx_bridge): increase coverage by adding common edge cases
2023-03-13 10:40:47 +01:00
478601ee41
fix(emqx_bridge): remove unreachable code paths
2023-03-13 10:40:47 +01:00
dba95ec0fa
style(emqx_bridge): fix wording
2023-03-13 10:40:47 +01:00
71ec77a2f2
fix(emqx_bridge): for node operation check if node is part of cluster
...
This fixes the case where we returned `501 NOT IMPLEMENTED` in the past.
2023-03-13 10:40:47 +01:00
7124600a71
docs(emqx_bridge): add `start` as operation to bridge
2023-03-13 10:38:51 +01:00
1bcc5623ed
fix(emqx_bridge): check if bridge enabled before calling op
2023-03-13 10:38:49 +01:00
a325133391
fix(emqx_bridge): don't crash checking if bridge enabled
2023-03-13 10:37:28 +01:00
80b81748df
fix(emqx_bridge): handle bridge not found in call_operation
2023-03-13 10:37:28 +01:00
fb3d101b3a
refactor(emqx_bridge): fix var names
2023-03-13 10:37:28 +01:00
e31f4d6091
refactor(emqx_bridge): add BAD_REQUEST macro and minor cleanups
2023-03-13 10:37:28 +01:00
e8cf604ad4
Merge pull request #9893 from lafirest/feat/filiter_banned
...
feat(banned): filter out banned messages from mqueue
2023-03-13 09:58:39 +08:00
e3595f2e79
chore(mria): Bump version to 0.4.0
2023-03-11 00:37:25 +01:00
e571b602b8
chore: bump app versions + add changelog
...
Bump `emqx_plugin_libs` app version to 4.3.7.
2023-03-10 18:43:19 +03:00
781942cc4f
chore: drop unused macrodefs
2023-03-10 18:42:04 +03:00
0a7f6c7d03
fix(mysql): ensure proper escaping in batch inserts
...
Also hexencode non-utf8 binaries. This is essentially an heuristic.
We don't know column types in runtime, and there's no simple way
to find them out. Since we're already doing full binary scan during
escaping it should be cheap to bail out on non-utf8 strings and
hexencode them instead.
Also introduce separate function to highlight that this escaping
is MySQL-specific.
2023-03-10 18:42:04 +03:00
fc37d9b3cd
fix(mysql): be explicit that batch queries are parameterless
...
So that mysql client won't attempt to prepare them automatically, thus
trashing the server's prepared statements table and making interaction
overall heavier.
2023-03-10 18:42:04 +03:00
a86d06f043
chore: bump app versions following last merge-back
2023-03-10 16:44:15 +03:00
5650ad20f8
Merge pull request #10100 from savonarola/0308-fix-scram-auth
...
Fix channel crash for slow clients with enhanced authentication
2023-03-10 12:58:43 +02:00
951e89134c
Merge remote-tracking branch 'upstream/release-50' into 0310-merge-release-50-to-master
2023-03-10 08:48:32 +01:00
3f461f312a
chore: bump version to e5.0.1 and v5.0.20
2023-03-09 18:34:26 +01:00
6ebc31a929
Merge pull request #10104 from JimMoen/refine-changes-for-release-50
...
Refine changes for release 50
2023-03-09 18:33:00 +01:00
cba0287439
fix(mqtt): fix channel crash for slow clients with enhanced authn
2023-03-09 17:27:24 +02:00
aa57ea9ee1
fix: mongodb authz crash
...
This fixes a crash with an error in the log file (see below) that
happened when the MongoDB authorization module queried the database. The
reason is that the collection name that was sent to the mongodb
connection was an atom. This is fixed by making sure it is not an atom.
2023-03-08T17:16:34.215523+01:00 [error] msg: query_mongo_error, mfa:
emqx_authz_mongodb:authorize/4, line: 95, peername: 127.0.0.1:53212,
clientid: client123, collection: mqtt_acl, filter: #{username =>
<<"emqx_u">>}, reason: {resource_error,#{msg => #{error =>
{error,{error_cannot_parse_response,{op_msg_response,#{<<"code">> =>
73,<<"codeName">> => <<"InvalidNamespace">>,<<"errmsg">> => <<"Failed to
parse namespace element">>,<<"ok">> => 0.0}}}},id =>
<<"emqx_authz_mongodb:3">>,name => call_query,request =>
{find,mqtt_acl,#{username => <<"emqx_u">>},#{}},stacktrace =>
[{mc_connection_man,reply,1,[{file,"mc_connection_man.erl"},{line,123}],
...]}, reason => exception}}, resource_id: <<"emqx_authz_mongodb:3">>
Fixes: https://github.com/emqx/emqx/issues/9783
2023-03-09 16:01:23 +01:00
3463cd0d37
refactor: generate document related artifacts to _build/docgen
2023-03-09 09:32:03 +01:00
842cb7fadc
fix: debug log for rulesql parse failed
...
See also: PR#10059.
The other error logs will be printed in `emqx_rule_runtime:apply_rule/3`
2023-03-09 11:28:27 +08:00
fe27604010
Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master
2023-03-08 16:46:45 +01:00
c773f5f5e1
chore: bump version to e5.0.1-rc.2
2023-03-08 13:55:37 +01:00
b8b1a78025
feat(bridges): integrate the DynamoDB into data bridges
2023-03-08 11:13:38 +08:00
158b4a95df
Merge pull request #10079 from qzhuyan/fix/william/i18n-broker_shared_subscription_strategy
...
fix: i18n desc of broker_shared_subscription_strategy
2023-03-07 21:57:07 +01:00
26b29185b2
test(emqx_bridge_webhook_SUITE): fix flakyness in test web server
2023-03-07 20:57:38 +01:00
163b33ab28
test: remove unnecessary dependencies of ee apps
2023-03-07 20:57:38 +01:00
ca947e3e70
fix: lost messages when HTTP connection times out
...
When using async mode with the webhook bridge, queued messages that are
not fully processed when the connection times out could be lost. This
commit fixes this by letting the bridge return a recoverable_error when
this happen. The message send will then be retried in sync mode by the
emqx_resource_buffer_worker.
Fixes: https://emqx.atlassian.net/browse/EMQX-8974
2023-03-07 20:57:19 +01:00
609cd01a35
Merge pull request #10085 from sstrigler/EMQX-8552-authorization-sources-type-status-move-shouldnt-exist-when-authorization-sources-type-doesnt-exist
...
emqx_authz API: return 404 for all requests on non existent source
2023-03-07 16:19:28 +01:00
b54f444263
fix(emqx_authz): return `404` for requests on non existent source
2023-03-07 13:51:06 +01:00
a7605fba94
test(emqx_authz): use snabbkaffe:retry instead of timer:sleep
...
also use emqx_json rather than jiffy or jsx directly
2023-03-07 13:49:46 +01:00
6f88cb7d9e
chore(ekka): Bump version to 0.14.3
2023-03-07 12:07:07 +01:00
0a5b221984
Merge pull request #10055 from zhongwencool/fix-max-awaiting-rel
...
fix: update max_awaiting_rel in session
2023-03-07 18:37:20 +08:00
9949c87b91
Merge pull request #10069 from SergeTupchiy/ci/EMQX-9055-configuration-item-flush-time-interval-used-for-stats-d-integration-not-work
...
fix: EMQX-9055 configuration item flush time interval used for statsd integration not work
2023-03-07 11:43:57 +02:00
2b1cf2fa52
Merge pull request #10058 from qzhuyan/dev/william/deprecated-unsupp-quic-TLS-options
...
fix: deprecate unsupp quic TLS options
2023-03-07 09:29:52 +01:00
422f8e3be3
Merge pull request #10078 from qzhuyan/fix/william/inval-listener-options-segfault
...
fix: inval listener option cause segfault
2023-03-07 09:26:27 +01:00
133734b345
chore: bump version number to e5.0.1-rc.1 and v5.0.20
2023-03-07 08:41:13 +01:00
eef65fba60
fix(buffer_worker): handle `request_timeout = infinity` case
...
The current schema allows `infinity` for `request_timeout`, so we have
to take that into account. It's not currently possible to set
`batch_time = infinity`, so there's no need to treat that case.
2023-03-06 15:31:28 -03:00
18ab7ed197
chore: bump app vsns
2023-03-06 15:31:28 -03:00
0e707e837f
docs(buffer_worker): improve description of `request_timeout`
2023-03-06 15:31:28 -03:00
e17ad320ee
fix(bridge): do not log in converter
2023-03-06 15:31:28 -03:00
e9d3fc511f
chore(buffer_worker): change default `batch_time` to 0 and improve docs
2023-03-06 15:31:28 -03:00
2fa4e8e21a
Merge pull request #10068 from thalesmg/fix-flaky-mqtt-bridge-test
...
test: fix flaky mqtt bridge test
2023-03-06 13:06:37 -03:00
7c65f6f9a9
fix: i18n desc of broker_shared_subscription_strategy
2023-03-06 15:09:56 +01:00
abb5271177
test(quic): start listener with invalid parameter
2023-03-06 14:58:35 +01:00
a81d1a4618
Merge pull request #10066 from sstrigler/EMQX-8932-bridge-api-reports-500-for-most-errors-when-starting-a-resource
...
fix: return human readable error message for most common cases
2023-03-06 14:13:34 +01:00
f95a30ae89
fix(webhook): convert `request_timeout`s in root and resource_opts
2023-03-06 10:12:38 -03:00
167b7a212f
refactor(buffer_worker): avoid starting 0-time timers
2023-03-06 10:12:38 -03:00
e9ffabf936
fix(buffer_worker): add batch time automatic adjustment
...
To avoid message loss due to misconfigurations, we adjust `batch_time`
based on `request_timeout`. If `batch_time` > `request_timeout`, all
requests will timeout before being sent if the message rate is low.
Even worse if `pool_size` is high. We cap `batch_time` at
`request_timeout div 2` as a rule of thumb.
2023-03-06 10:12:38 -03:00
27c5389fdc
fix(quic): inval listener option casue segfault
...
bump quicer to 0.0.113
2023-03-06 14:05:33 +01:00
8fbb948b6f
test: fix flaky mqtt bridge test
...
Sometimes, this test fails because the metrics are still in the
inflight phase.
2023-03-06 09:09:33 -03:00
d0ea7f4647
fix(emqx_authz): check if type param matches type in body
2023-03-06 11:10:31 +01:00
f498a3538b
fix: update max_awaiting_rel in session
2023-03-06 17:06:40 +08:00
a4aece396a
Merge pull request #10052 from zmstone/0301-optimize-boot-error-log
...
0301 optimize boot failure logs
2023-03-06 08:32:15 +01:00
a638cc6566
Merge pull request #9985 from kjellwinblad/kjell/feat/clickhouse_bridge_2/EMQX-8391
...
feat: add clickhouse database bridge
2023-03-04 07:05:05 +01:00
b3907128e8
fix: set statsd flush_time_interval = max(flush_time_interval, sample_time_interval)
...
flush_time_interval is used to calculate statsd sampling rate:
rate = sample_time_interval / flush_time_interval
This means that flush_time_interval must always be greater than (or equal to)
sample_time_interval, otherwise, the sampling rate will be invalid (> 1).
Relates to EMQX-9055
2023-03-03 22:24:00 +02:00
bff087f40a
fix: restart emqx_statsd with the updated configuration
...
emqx_config_handler:post_config_update/5 cb is called before an updated config is saved.
Thus, a process being restarted in that callback cannot get the latest config by calling
emqx_conf:get/2, because that update is not saved yet.
Relates to EMQX-9055
2023-03-03 22:24:00 +02:00
570bf165af
fix: return human readable error message for most common cases
2023-03-03 16:49:09 +01:00
c01f62a1c1
chore: bump apps versions
2023-03-03 13:58:32 +01:00
6ebd3dc747
feat(emqx_rule_engine): decompose error tuples
...
sqltester for instance returns sth like {"...sytnax error...", OrigSql}
2023-03-03 09:59:24 +01:00
44eca1fa72
fix(emqx_rule_engine): don't crash if we can't encode json
2023-03-03 09:36:13 +01:00
64b5e9585e
feat(emqx_rule_engine): API format errors in a human readable way
2023-03-02 16:49:11 +01:00
4eb9a69df0
feat(bin/emqx): diagnose boot failure
...
Prior to this change, when EMQX daemon mode failed to start
it's not quite easy for users to understand what went wrong.
All the know is the node did not start in time
and then instructed to boot the node in 'console' mode wishing
for some logs.
However, the node might actuay be running, causing 'console' mode
to fail with a different reason.
With this change, after a filure of daemon mode boot,
we issue a diagnosis.
1. if node can not be found from ps -ef, instruct the user
to find information in erlang.log.N
2. if the node is found running, but not responding to pings
instruct the user to check if the node name is
resolvable and reachable
3. if the node is responding to pings but emqx app is not
running, then it's likely a bug. so the user is advised
to report a github issue.
2023-03-02 14:46:09 +01:00
baf01617cd
fix(quic): mark unsupp TLS options deprecated
2023-03-02 14:03:44 +01:00
67acdf0888
feat: add clickhouse database bridge
...
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
2023-03-02 12:22:11 +01:00
39e1cf9502
fix(emqx_bridge): let it crash instead of 400 on failed reset
2023-03-02 11:23:07 +01:00
4c23ab097d
fix(emqx_bridge): return 400 if operation not possible
2023-03-02 11:23:07 +01:00
9fb74bfc87
Merge pull request #10050 from sstrigler/EMQX-9063-bridges-api-consistently-return-404-if-resource-does-not-exist
...
fix: consistently return 404 in case bridge is not found or invalid
2023-03-02 11:20:56 +01:00
680a4362d1
fix: match macro variables
2023-03-02 10:12:56 +01:00
76f97010bb
style: change wording
2023-03-02 09:14:52 +01:00
77dea0c77a
fix: Handle obfuscated fields in bridges_probe API
2023-03-01 22:05:23 +02:00
3f324f85bc
fix: consistently return 404 in case bridge is not found or invalid
...
Also: fix some typos and be more verbose
2023-03-01 18:03:49 +01:00
a5f31d0bf4
Merge remote-tracking branch 'origin/master' into 0301-merge-release-50-to-master
2023-03-01 14:04:29 +01:00
0185c81f69
chore: bump opensource edition version to v5.0.19
2023-03-01 11:53:24 +01:00
fdc0ad6669
Merge pull request #10044 from zmstone/0228-fix-node-info-format
...
fix(emqx_mgmt): format stoped nodes correctly
2023-03-01 11:39:09 +01:00
083330ad80
Merge remote-tracking branch 'origin/master' into 0301-merge-release-50-to-master
2023-03-01 08:53:03 +01:00
2b06089250
test: stop apps in end_per_suite
2023-03-01 08:48:23 +01:00
469071df00
fix(emqx_management): ensure binary string for log_path
2023-03-01 07:45:36 +01:00
d369e3167b
fix(emqx_mgmt): format stoped nodes correctly
2023-02-28 21:38:30 +01:00
f80c4e4084
chore(ekka): Bump version to 0.14.2 (mria 0.3.3)
2023-02-28 20:14:13 +01:00
b3100ca8ae
Merge pull request #9956 from zmstone/0210-fix-bridge-status
...
0210 docs: fix bridge status
2023-02-28 19:44:07 +01:00
427e65bf21
fix: bump hocon to 0.36.0 disallow dup fields
2023-02-28 14:55:45 +01:00
7136cc94f4
chore(quic): remove dup handshake_idle_timeout_ms
2023-02-28 14:55:45 +01:00
56279f6305
docs: fix bridge status report
2023-02-28 11:56:46 +01:00
f3e1860562
test: fix flaky olp tcs
...
Leave only one scheduler online to make test result stable.
2023-02-27 20:01:29 +01:00
7dce152ffd
Merge pull request #10032 from keynslug/fix/EMQX-9019/no-metrics-disconnected
...
fix(bridgeapi): anticipate node responses w/o metrics
2023-02-27 16:43:25 +04:00
7c573d9365
fix(bridgeapi): anticipate node responses w/o metrics
...
When some resource manager is busy with trying to estabilish a
connection with remote, we hit the "read-from-cache" codepath so the
resource data will not contain any metrics.
2023-02-27 14:55:41 +03:00
6dd99b3331
Merge pull request #10030 from qzhuyan/test/william/fix-quic-flaky-tc
...
chore: fix quic flaky tc
2023-02-27 10:04:12 +01:00
7d762593f6
fix(connector): redact the http body in error logs for security reasons
2023-02-27 11:41:34 +08:00
58af936120
test: fix flaky emqx_broker_helper_SUITE:t_shard_seq
2023-02-24 21:21:12 +01:00
60532569a2
test(quic): fix flaky
2023-02-24 21:21:12 +01:00
5150407f7b
chore: bump to version v5.0.18
2023-02-24 20:06:43 +01:00
b3f3bdeafe
Merge remote-tracking branch 'origin/master' into release-50
2023-02-24 20:05:59 +01:00
df7e9db057
Merge pull request #10019 from qzhuyan/dev/william/quic-hidden-low-level-tunings
...
230222 feat(quic): add hidden low level settings for listeners.
2023-02-24 20:03:31 +01:00
80afb69bb2
Merge pull request #10015 from zmstone/0220-faile-fast-if-cookie-is-obviously-wrong
...
0220 fail fast if cookie is obviously wrong
2023-02-24 19:54:35 +01:00
79bf77c2f1
Merge pull request #10020 from zmstone/0222-fix-bridge-async-mode-counters
...
fix(bridge): fix dropped counter and inflight gauge
2023-02-24 19:51:24 +01:00
c883e4b36a
test: drop custom `loop_wait` in favor of snabkaffe's `?retry`
2023-02-24 18:16:35 +03:00
2b4e49e7df
fix(bufworker): handle replies of simple async queries
...
Before that change, simple queries were treated as "retries"
essentially, thus skipping all the reply processing there is.
2023-02-24 15:06:49 +03:00
fde19e2b4b
docs(quic): fix typos
2023-02-24 10:20:28 +01:00
f61c3c47ca
docs(quic): add zh
2023-02-24 10:20:28 +01:00
681fd80cab
docs(quic): move desc to i18n.
2023-02-24 10:20:28 +01:00
ee77c113aa
test(quic): tuned listener
2023-02-24 10:20:28 +01:00
ea65ec10bf
feat(quic): add hidden low level settings for listeners.
2023-02-24 10:20:28 +01:00
c97d17cc91
test: refactor to loop wait for counters
2023-02-24 09:02:03 +01:00
a10dbba084
refactor(buffer_worker): less defensive on inflight counter decrement
2023-02-23 21:23:10 +01:00
5bfd690df6
fix(dashboard): fix swagger spec generation
2023-02-23 22:09:36 +02:00
7a6465e2cf
fix(buffer_worker): ensure flush timer reset in blocked state
2023-02-23 21:06:38 +01:00
3a6dbbdd05
refactor(buffer_worker): ensure flsh message is never missed
2023-02-23 20:11:00 +01:00
dbfdeec5e9
fix(buffer_worker): log unknown async replies
2023-02-23 12:55:49 +01:00
356a94af30
fix(buffer_worker): ensure async flush message is sent
...
This is a new issue introduced in the previous fix commits
after handling the partial expiry correctly, the
IsFullBefore check is no longer the state before the reply
is received but the state after a partially-expired batch
is shrinked.
The fix is simple, move the check to the entry-point of
where async reply callback enters, then send an async
'flush' notification regardless of the handling result.
2023-02-23 09:47:34 +01:00
e9787c3ee3
Merge pull request #9966 from zmstone/0213-port-blackbox-cover-report-from-v4
...
feat: add new module emqx_cover.erl
2023-02-23 09:44:11 +01:00
3413af76be
fix(emqx_misc): ensure flatten list for safe dir
2023-02-23 09:05:52 +01:00
713220f88b
refactor(buffer_worker): more generic process for all_expired
2023-02-23 00:04:20 +01:00
036f69cd6e
test: ensure batch size > 1 is covered in expiration test
2023-02-22 23:26:04 +01:00
bf8becd521
test: make sure gauge return to 0 in test cases
2023-02-22 23:07:12 +01:00
3458aee296
Merge branch 'master' into 0213-port-blackbox-cover-report-from-v4
2023-02-22 22:21:44 +01:00
fc614e16e5
fix(bridge): update inflight items after partial expiry
2023-02-22 22:05:56 +01:00
b0777ca73b
chore(ekka): Bump version
2023-02-22 20:55:04 +01:00
0cfa5e2ce1
Merge pull request #9994 from sstrigler/EMQX-3688-emqx-mgmt
...
emqx mgmt code coverage and cleanup
2023-02-22 14:30:36 +01:00
6a1085a842
chore: change 'emqx_ctl' usage to 'emqx ctl'
2023-02-22 13:57:55 +01:00
bb13d0708f
fix(bridge): fix dropped counter and inflight gauge
...
Prior to this fix there were two metrics issues
1. if a batch is all requests expired when receiving a reply
it only bumped 1 instead of the batch size for 'late_reply'
2. when a batch is partially delivered (or expired), the
dropped requests were not decremented from the inflight size gauge
2023-02-22 13:20:58 +01:00
a1ba7bc004
Merge pull request #9976 from zmstone/0215-docs-improve-bridge_mode-description
...
docs: improve bridge_mode description
2023-02-21 20:55:20 +01:00
967369075f
Merge pull request #10014 from sstrigler/EMQX-7822-5-0-get-api-v-5-monitor-nodes-node-must-return-404-when-node-is-not-responding
...
fix: return `404` for unknown node names
2023-02-21 17:09:03 +01:00
a0589d5b95
fix: return `404` for unknown node names
2023-02-21 16:23:38 +01:00
965d63f4f5
fix: schema for `/gateways/:name/clients` was missing top-level structure
2023-02-21 15:33:56 +01:00
bf978efc83
Merge pull request #10009 from sstrigler/EMQX-7994-get-trace-name-log-bytes-xxx-does-not-do-input-validation
...
get trace name log bytes xxx does not do input validation
2023-02-21 14:12:57 +01:00
969cbecf91
Merge pull request #10010 from qzhuyan/dev/william/new-emqtt-quicer
...
chore: bump quicer 0.0.111 and emqtt 1.8.2
2023-02-21 13:30:27 +01:00
28382ec26b
test: add test for pubsub api
2023-02-21 13:11:34 +01:00
0a207856aa
chore: bump quicer 0.0.111 and emqtt 1.8.2
2023-02-21 10:36:15 +01:00
e78c2c2869
fix: return 404 in case node is not found
2023-02-21 10:20:34 +01:00
9ecf154a71
fix: limit bytes param to signed 32bit int
...
We still need to check if chunk we're reading fits in memory
2023-02-21 10:20:14 +01:00
8ae4440061
style: fix API description for bytes parameter
2023-02-21 09:41:32 +01:00
81fad58f12
fix: re-add `list_subscriptions/0`
...
Created a ticket to add an actual working implementation
2023-02-21 09:41:00 +01:00
71f3efb2ce
style: fix empty line
2023-02-21 09:41:00 +01:00
c44c7fcbce
style: remove unused macro
2023-02-21 09:41:00 +01:00
0482f43802
test: add test for banned and alarms
2023-02-21 09:41:00 +01:00
0d357f7038
refactor: cleanup list subscriptions
2023-02-21 09:41:00 +01:00
0d2ce85776
fix: return 'not found' for subscriptions of unknown client
2023-02-21 09:41:00 +01:00
f3ced5d5eb
refactor: kickout_client doesn't need a format fun
2023-02-21 09:41:00 +01:00
a6d88c3caa
test: more tests
2023-02-21 09:41:00 +01:00
fc33bce40d
test(emqx_mgmt): test list_nodes/0
2023-02-21 09:41:00 +01:00
777ca72ad5
fix: don't crash on broker_info()
2023-02-21 09:41:00 +01:00
f9895a33b6
refactor: rename max_row_limit to default_row_limit
2023-02-21 09:41:00 +01:00
9316690c29
fix(schema): binary string for default values
...
A lot of the string value fields had default value defined in
schema as list-string rather than binary-string.
This caused the generated schema dump (in JSON format)
to have raw_default field as an integer array.
2023-02-21 09:09:51 +01:00
31cfd728c4
ci(quic): bump to quicer 0.0.109 for ubuntu22.04 prebuilds
2023-02-20 14:50:35 +01:00
bd4a84ac0a
test(quic): adapt to new emqtt reconnect mechanism.
2023-02-20 14:48:39 +01:00
296e271b97
fix(quic): bump to emqtt 1.8.1
2023-02-20 10:53:51 +01:00
cf72947f0a
test(quic): use quic.ssl_options
2023-02-20 10:53:42 +01:00
ebd0fb74a3
test(quic): by default, bind to port not IPv4
2023-02-20 10:43:24 +01:00
3f7032fbe9
chore(quic): troubleshooting large payload
2023-02-20 10:43:21 +01:00
fef0a9375c
chore(quic): make spell check happy
2023-02-20 10:43:17 +01:00
b81b62c639
chore(quic): doc about deprecated fields.
2023-02-20 10:43:17 +01:00
45718dd77f
chore(quic): debug flaky large payload tc.
2023-02-20 10:43:17 +01:00
f106f30a96
chore: fix comments in emqx_connection
2023-02-20 10:43:17 +01:00
c6c3bd0396
chore(quic): schema format fix
2023-02-20 10:42:55 +01:00
0e40f6cf48
feat(quic): listener use common server ssl_options
2023-02-20 10:42:55 +01:00
fc3e8715a1
feat(quic): bump to emqtt 1.8.0
2023-02-20 10:42:55 +01:00
04f502fb54
feat(quic): support mTLS with 'verify' and 'cacertfile'
2023-02-20 10:42:55 +01:00
c7efccb996
chore: bump emqtt 1.7.1-pre2 & quicer 0.0.108
2023-02-20 10:42:55 +01:00
c457c1092b
fix(quic): show QUIC listeners in dashboard
2023-02-20 10:42:55 +01:00
3c73c6b7c6
feat(quic): bump quicer to 0.0.107
2023-02-20 10:42:55 +01:00
0351b32cf4
test(quic): disable shutdown policy for large payload test
2023-02-20 10:42:55 +01:00
f4f346e387
test(quic): fix flaky test
2023-02-20 10:42:55 +01:00
db544cf9ad
fix: emqtt vsn in rebar after rebase
2023-02-20 10:42:55 +01:00
dc26790495
test(quic): trace why we get verify_peer
2023-02-20 10:42:55 +01:00
f8fd201a8c
test(quic): fix flaky test
2023-02-20 10:42:55 +01:00
d8fa65ea09
fix(quic): handle timeout event in data stream
2023-02-20 10:42:55 +01:00
38247a9d62
feat(quic): bump quicer to 0.0.106
2023-02-20 10:42:55 +01:00
381eb8ec68
chore(quic): fix dialyzer
2023-02-20 10:42:55 +01:00
282d1a6829
ci: build dialyzer PLT with quicer, jq and bcrypt
2023-02-20 10:42:55 +01:00
88cdfcc4a6
test(quic): excl. multistream SUITE when BUILD_WITHOUT_QUIC
2023-02-20 10:42:55 +01:00
de810e04fd
chore(quic): clean test code
2023-02-20 10:42:55 +01:00
98a72d40ce
fix(emqx_connection): do not raise an exception for normal shutdown
2023-02-20 10:42:55 +01:00
1692a16778
feat(quic): handle ctrl stream normal shutdown
2023-02-20 10:42:55 +01:00
2a6cdd9da6
test(quic): enhance large payload test
2023-02-20 10:42:55 +01:00
00f615a1e3
chore(quic): clean code
2023-02-20 10:42:55 +01:00
22dcf5907e
feat(quic): bump to quicer 0.0.104
2023-02-20 10:42:55 +01:00
f65ac5422e
test(quic): improve coverage
2023-02-20 10:42:55 +01:00
5764994436
feat(quic): bump to quicer 0.0.103
2023-02-20 10:42:55 +01:00
1e8b2e247e
feat(quic): 0-RTT multi-streams data
2023-02-20 10:42:55 +01:00
71d3148544
feat(quic): stream use active_n 10
2023-02-20 10:42:55 +01:00
0173121a30
feat(quic): improve coverage and remove unused code
2023-02-20 10:42:55 +01:00
0544a3ca0c
fix(quic): setops on stream and handle peer needs stream
...
- setopts should go for stream
- handle peer_needs_streams for none msquic clients
2023-02-20 10:42:55 +01:00
Thales Macedo Garitezi