Commit Graph

6650 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 127a075b66 test(dynamo): attempt to fix dynamo tests
Those tests in the `flaky` test are really flaky and require lots of
CI retries.

Apparently, the flakiness comes from race conditions from restarting
bridges with the same name too fast between test cases.  Previously,
all test cases were sharing the same bridge name (the module name).
2023-03-22 14:34:37 -03:00
Stefan Strigler 2662f79ec0 feat(emqx_dashboard): use label as summary in OpenAPI spec 2023-03-22 16:29:51 +01:00
Thales Macedo Garitezi 61cb03b45a fix(buffer_worker): change the default `resume_interval` value and expose it as hidden config
Also removes the previously added alarm for request timeout.

There are situations where having a short request timeout and a long
health check interval make sense, so we don't want to alarm the user
for those situations.  Instead, we automatically attempt to set a
reasonable `resume_interval` value.
2023-03-22 11:47:36 -03:00
lafirest 84def357a9
Merge pull request #10143 from lafirest/feat/rocketmq
feat(bridges): integrate RocketMQ into data bridges
2023-03-22 20:43:22 +08:00
Zaiming (Stone) Shi e43aa5d841
Merge pull request #10199 from zmstone/0321-merge-release-50-to-master
0321 merge release 50 to master
2023-03-22 10:20:48 +01:00
Zaiming (Stone) Shi db6afb40b0
Merge pull request #10174 from zmstone/0320-upgrade-esockd-to-5.9.6
fix: upgrade esockd from 5.9.4 to 5.9.6
2023-03-22 09:50:38 +01:00
Andrew Mayorov af7b2215ca
fix(bridge-api): unflatten nodes' bridge lists back
Bridge lists were erroneously flattened in cad6492c. This causes
bridge listing fail in emqx clusters consisting of more than 1 node.
2023-03-22 11:35:09 +03:00
firest 9853d00cad feat(bridges): integrate RocketMQ into data bridges 2023-03-22 10:36:52 +08:00
JianBo He 65c2da7ef5 Merge remote-tracking branch 'ce/master' into cassa 2023-03-22 09:30:50 +08:00
Andrew Mayorov bf35419b52
Merge pull request #10190 from keynslug/fix/EMQX-9269/bridge-list
fix(bridge-api): unflatten nodes' bridge lists back
2023-03-22 00:53:03 +03:00
Zaiming (Stone) Shi e6091db351 Merge remote-tracking branch 'origin/release-50' into 0321-merge-release-50-to-master 2023-03-21 22:03:31 +01:00
Stefan Strigler 6c24b3a77d
Merge pull request #10196 from sstrigler/EMQX-9201-the-case-is-not-correct-for-some-of-menu
Emqx 9201 the case is not correct for some of menu
2023-03-21 19:42:10 +01:00
Thales Macedo Garitezi 4bbbaf8b13
Merge pull request #10164 from thalesmg/crl-v50-mkIII
feat(crl): add crl check support
2023-03-21 14:42:01 -03:00
Stefan Strigler 327b782210 style: lower case in descriptions 2023-03-21 17:04:42 +01:00
Andrew Mayorov d9b1b1c802
fix(bridge-api): unflatten nodes' bridge lists back
Bridge lists were erroneously flattened in cad6492c. This causes
bridge listing fail in emqx clusters consisting of more than 1 node.
2023-03-21 18:59:27 +03:00
Stefan Strigler 981268911a style: make summary lower-case 2023-03-21 16:56:37 +01:00
Stefan Strigler 3880862c81 fix(emqx_bridge): return 503 for inconsistency in bridge setup 2023-03-21 15:07:43 +01:00
Stefan Strigler 4b0ea562a2 refactor(emqx_bridge): consistently use macros for http response 2023-03-21 15:07:41 +01:00
Stefan Strigler 84fc64822e style: fix wording for nxdomain and malformed_username_or_password
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 15:02:29 +01:00
Stefan Strigler 8af3fb4ee7 feat: move human readable error translations to emqx_misc 2023-03-21 15:02:29 +01:00
Stefan Strigler c1384b6e6e feat(emqx_resource): include error with alarm for resource_down 2023-03-21 15:02:29 +01:00
Stefan Strigler 53825b9aba fix(emqx_bridge): propagate connection error to resource status 2023-03-21 15:02:29 +01:00
Zaiming (Stone) Shi fd23800370
Merge pull request #10180 from zmstone/0320-add-emqx_ctl-readme
docs: add readme for emqx_ctl app
2023-03-21 11:33:16 +01:00
William Yang 7d8fc3d2ad
Merge pull request #10185 from qzhuyan/dev/william/bump-quicer
chore(build): Support fetching binary for macOS M1 [quicer 0.0.114]
2023-03-21 11:32:33 +01:00
William Yang 092daf409a chore(testlib): update include for CI of emqtt lib 2023-03-21 09:03:31 +01:00
William Yang b6e583143f chore: bump quicer 0.0.114
for fetching quicer binary for macOS M1 builds.
2023-03-21 09:01:25 +01:00
Kjell Winblad 89782e6d21
docs: add info about that rule engine can handle different types of structured data
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 06:26:04 +01:00
JianBo He b059bad08a
chore(acl): fully match `dashboard` username
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2023-03-21 09:06:46 +08:00
Thales Macedo Garitezi f2b54588e7 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-20 18:09:38 -03:00
Thales Macedo Garitezi 7a71d64686 feat(crl): add crl check support 2023-03-20 18:09:38 -03:00
Ilya Averyanov ee403423b9
Merge pull request #10166 from savonarola/0318-mix-build
chore(mix): fix mix builds
2023-03-20 22:37:21 +02:00
Zaiming (Stone) Shi c739d9957c docs: add readme for emqx_ctl app 2023-03-20 19:35:17 +01:00
Zaiming (Stone) Shi 6f71898546 fix: upgrade esockd from 5.9.4 to 5.9.6
changes in esockd are

* 5.9.5: added API to retrieve SNI
* 5.9.6: avoid error log if socket is closed before sending
         proxy protocol headers
2023-03-20 19:27:32 +01:00
Kjell Winblad 08e249dc4d docs: refine README.md for the emqx_rule_engine app
This commit adds more content to the readme file of the `emqx_rule_engine`
app. The aim of the changes are to make it easier to understand what the
rule engine is and what it does.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9229
2023-03-20 16:53:55 +01:00
Erik Timan 2d75c7d6d9 fix(emqx_bridge): remove metrics from non-dedicated bridge API endpoints
Metrics should only be exposed via the /bridges/:id/metrics endpoint,
and not in other operations such as getting the list of all bridges, or
in the response when a bridge has been created. This commit removes all
traces of metrics for the non-dedicated API endpoints.
2023-03-20 09:43:11 +01:00
JianBo He 377127ce72 fix(acl): fix wrong default ACL rules 2023-03-20 13:53:57 +08:00
JianBo He 12942b676d Merge remote-tracking branch 'upstream/master' into cassa 2023-03-20 09:50:27 +08:00
zhongwencool 6a668c35db
Merge pull request #10167 from zhongwencool/fix-swagger-doc-500-crash
fix: swagger doc 500 crash
2023-03-19 16:28:32 +08:00
Zhongwen Deng b529a0bce1 test: add swagger.json api test 2023-03-19 13:05:32 +08:00
Zhongwen Deng f8573ae8d7 fix: swagger doc 500 crash 2023-03-19 12:34:06 +08:00
Ilya Averyanov 39f22585d0 chore(mix): fix mix builds 2023-03-19 01:39:56 +02:00
Zaiming (Stone) Shi 149afe863f chore: e5.0.2/v5.0.21 code freeze 2023-03-17 13:19:44 +01:00
Thales Macedo Garitezi cf1cce011b
Merge pull request #9564 from thalesmg/kafka-source-ee50
feat: implement kafka consumer (ee5.0)
2023-03-17 09:04:53 -03:00
Zaiming (Stone) Shi 1bf72c4d2e
Merge pull request #10155 from zmstone/0316-build-schema-dump-per-lang
build: generate per-lang schema dump
2023-03-17 11:45:35 +01:00
Zhongwen Deng 3c1254d873 fix: newly created listeners have no limiter restrictions 2023-03-17 14:32:42 +08:00
JianBo He c0a216a740 feat(bridge): support cassandra bridge 2023-03-17 11:34:48 +08:00
Zaiming (Stone) Shi ad1deedd0e build: generate per-lang schema dump 2023-03-16 21:25:21 +01:00
Thales Macedo Garitezi 20414d7373 fix(buffer_worker): check request timeout and health check interval
Fixes https://emqx.atlassian.net/browse/EMQX-9099

The default value for `request_timeout` is 15 seconds, and the default
resume interval is also 15 seconds (the health check timeout, if
`resume_interval` is not explicitly given).  This means that, in
practice, if a buffer worker ever gets into the blocked state, then
almost all requests will timeout.

Proposed improvement:

- `request_timeout` should by default be twice as much as
health_check_interval.
- Emit a alarm if `request_timeout` is not greater than
`health_check_interval`.
2023-03-16 13:46:45 -03:00
Thales Macedo Garitezi 966276127e test: trying to make tests more stable 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 3954b7bde2 fix(bridges): function clause when a non-ingress bridge coexists with an egress bridge
This was not caught by our tests because we always test bridge types
in isolation.  So, if the config only contains ingress-only bridges,
the `on_message_publish` hook is never installed.

In a real system, if there are bridges of mixed types in the config,
the hook might be installed, and `emqx_bridge:get_matched_bridge_id`
would crash when iterating over the ingress bridges.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 947e014132 test: improve cluster node helpers
* Add option to start autocluster.  This is useful for scenarios where
a cluster is already running and has some configurations set (via
config handler/cluster rpc) and later another node joins.

* Improve mnesia data directory isolation for nodes.  A new dir is set
so that we may avoid intra and inter-suite flakiness due to dirty
tables and schema.

* The janitor is now called synchronously.  This ensures the cleanup
is done before the test pid dies.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi d464e2aad5 refactor: rename test resource prefix
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 66eb4ef069 test: fix inter-suite flakiness 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi f31f15e44e chore(kafka_producer): make schema changes more backwards compatible
This will still require fixes to the frontend.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 03342923b9 fix(bridge): use the same dry run prefix
Kafka Producer and Consumer bridges rely on this prefix for detecting
a dry run and avoid leaking atoms.  At some point, this prefix was
changed, effectively disabling the check in Kafka Producer.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1f31a87974 fix(bridge): improve macro var usage
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 969fa03ecc feat: implement kafka consumer 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 094e4a2eeb chore: fix typespec 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi b9e92173cf test: improve cluster helper
1) Make each node have its own isolated data dir to avoid false
negatives.
2) Allow parameterizing the peer module.
3) Fix cluster RPC after a node joins the cluster.
2023-03-16 13:42:59 -03:00
Thales Macedo Garitezi 561c25f0e3 feat: update snabbkaffe -> 1.0.7 2023-03-16 13:42:38 -03:00
Thales Macedo Garitezi 91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII
feat: add ocsp stapling support to mqtt ssl listener (5.0)
2023-03-16 13:10:48 -03:00
JimMoen 1688cb7df2
Merge pull request #10152 from JimMoen/refine-changes
chore: refine changes for merged PRs
2023-03-16 21:11:52 +08:00
Thales Macedo Garitezi a614bdc94a chore(ocsp): catch unexpected error when fetching ocsp response 2023-03-16 09:55:52 -03:00
Thales Macedo Garitezi d1f58d6e2d refactor: replace macro by simple function 2023-03-16 09:32:40 -03:00
Thales Macedo Garitezi e5645a7b21 refactor: rename macros 2023-03-16 09:18:16 -03:00
Kjell Winblad 884dddec79
Merge pull request #10136 from kjellwinblad/kjell/fix/default_value_ui
fix: schema default value so that it shows up correctly in UI
2023-03-16 12:20:08 +01:00
JimMoen afd29b69c4
chore: refine i18n doc punctuation 2023-03-16 18:22:29 +08:00
Ivan Dyachkov e3d4deb8a0
Merge pull request #10147 from id/chore-minor-improvements
Chore minor improvements
2023-03-16 08:18:43 +00:00
Andrew Mayorov 816667d85c
Merge pull request #10123 from fix/EMQX-9136/lookup-busy-resources
perf(bridge-api): ask nodes' bridge listings in parallel
2023-03-16 00:07:12 +03:00
Ivan Dyachkov 5258b4c6e9 chore: bump ekka to 0.14.5
to make use of erlang-rocksdb 1.7.2-emqx-9
2023-03-15 19:54:55 +01:00
Thales Macedo Garitezi 164440fe83 test(resource): fix flaky test
Sometimes this test might retry more times, so we check the prefix
of the trace only.
2023-03-15 14:25:55 -03:00
Thales Macedo Garitezi 03b95073fc test: fix inter-suite flakiness 2023-03-15 14:25:41 -03:00
Andrew Mayorov a9bc8a4464
refactor(resman): rename `ets_lookup` → `lookup_cached`
That way we hide the impementation details + the interface becomes
cleaner and more obvious.
2023-03-15 19:17:30 +03:00
Andrew Mayorov 29907875bf
test(bufworker): set `batch_time` for batch-related testcases
By default it's `0` since e9d3fc51. This made a couple of tests prone
to flapping.
2023-03-15 19:17:30 +03:00
Andrew Mayorov e411c5d5f8
refactor(resman): work with state cache atomically
Also ensure that cache entries are always consistent with `Data`,
so that most of the code could rely on reading the cached entry
most of the time.
2023-03-15 19:17:30 +03:00
Andrew Mayorov b3e7e51094
test(bridge): drop unnecessary cleanup routines
Since `end_per_testcase` cleans out all the resources anyway.
2023-03-15 19:17:29 +03:00
Andrew Mayorov cad6492c99
perf(bridge-api): ask bridge listings in parallel
Also rename response formatting functions to better clarify their
purpose.
2023-03-15 19:17:29 +03:00
Thales Macedo Garitezi 0deb9925c1 docs: improve descriptions
Co-authored-by: William Yang <mscame@gmail.com>
2023-03-15 10:30:02 -03:00
Zhongwen Deng 60677bc400 chore: remove unuse code 2023-03-15 20:40:54 +08:00
Ivan Dyachkov c9b736ae23
Merge pull request #10141 from id/0315-chore-merge-release-50-to-master
0315 chore merge release 50 to master
2023-03-15 12:11:54 +00:00
Kjell Winblad 13a35cc045 fix: schema default value so that it shows up correctly in UI
This complements PR https://github.com/emqx/emqx/pull/10124.
The default values for duration_ms() fields needs to be formatted
as a binary string with unit to show up correctly in the dashboard
UI.
2023-03-15 11:04:16 +01:00
Thales Macedo Garitezi 04378f242c feat(ocsp_cache): give ocsp cache table an heir 2023-03-14 16:09:06 -03:00
Thales Macedo Garitezi f6707d1dd0 test: fix how ocsp client is run in tests
For some yet unknown reason the old test version using `open_port`
does not work in OTP 25, but works fine in OTP 24.  There are no
messages at all received from The openssl client port program in OTP
25.
2023-03-14 16:09:06 -03:00
Thales Macedo Garitezi 63ef2f9b79 feat: save uploaded OCSP issuer pem like other ssl certs 2023-03-14 16:09:06 -03:00
Thales Macedo Garitezi 158f054187 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-14 16:09:06 -03:00
Thales Macedo Garitezi 57e38c8502 refactor(ocsp): add reusable type for normalized binary URLs 2023-03-14 16:09:05 -03:00
Thales Macedo Garitezi 067747c2de docs: improve descriptions
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-14 16:08:47 -03:00
Thales Macedo Garitezi 52263a0448 feat: add ocsp stapling and crl support to mqtt ssl listener 2023-03-14 16:08:47 -03:00
Thales Macedo Garitezi 422597a441 test: fix flaky tests 2023-03-14 16:08:47 -03:00
Thales Macedo Garitezi 65fee34fe4 test: fix inter-suite test teardowns 2023-03-14 16:08:47 -03:00
Thales Macedo Garitezi 4880a849b9 chore: update emqtt -> 1.8.5
Needed for OCSP / CRL tests because of a bug that makes emqtt hang
forever on TLS handshake errors.
2023-03-14 16:08:47 -03:00
SergeTupchiy 9f9d16dd48
Merge pull request #10117 from SergeTupchiy/EMQX-8889_copy_plugins_on_joining_a_cluster
fix: copy plugins to a new node joining a cluster
2023-03-14 21:07:12 +02:00
Zaiming (Stone) Shi 6cbc17f110
Merge pull request #10135 from zmstone/0314-upgrade-to-hocon-0.37.0
chore: upgrade to hocon 0.37.0
2023-03-14 19:55:41 +01:00
Stefan Strigler 255117f2c4 chore: bump vsn e5.0.2-alpha.1 2023-03-14 19:51:35 +01:00
Serge Tupchii bde4215a11 fix: copy plugins to a new node joining a cluster
Closes: EMQX-8889
2023-03-14 20:42:57 +02:00
William Yang fffb92ebb7 test(quic): make tc run independent 2023-03-14 11:46:24 +01:00
William Yang 4c28b9f839 chore(test): use power of 2 recvbuff size in quic test
Make msquic assertions happy while in debug mode.
2023-03-14 11:46:24 +01:00
Zaiming (Stone) Shi 18b145ab67 chore: upgrade to hocon 0.37.0
hocon 0.37.0 introduced schema field aliases
this change does not impact anything until we start using aliases
2023-03-14 11:39:55 +01:00
zhongwencool 4f1da5e9fa
Merge pull request #10132 from zhongwencool/systemd-stop-crash-log
fix: stop port apps when emqx_machine_terminator shutdown
2023-03-14 17:40:26 +08:00
zhongwencool d853085259
Merge pull request #10119 from zhongwencool/fix-statsd-empty-server
fix: don't allow empty string in server_port schema
2023-03-14 16:37:15 +08:00