Commit Graph

173 Commits

Author SHA1 Message Date
Andrew Mayorov 7e7b50c5ba
refactor(mqttbridge): move into separate application 2023-05-30 17:56:34 +03:00
Andrew Mayorov a5fc26736d
refactor(mqttconn): split ingress/egress into 2 separate pools
Each with a more refined set of responsibilities, at the cost of slight
code duplication. Also provide two different config fields for each pool
size.
2023-05-30 17:21:44 +03:00
Andrew Mayorov 4e6269bedb
feat(mqttconn): subscribe each worker if shared subcription
Also rename `subscriptions` -> `ingress` and `forwards` -> `egress` for
consistency with the config schema.
2023-05-30 14:50:05 +03:00
Andrew Mayorov 81e78516aa
feat(mqttconn): employ ecpool instead of a single worker 2023-05-30 14:50:05 +03:00
Andrew Mayorov bd956d00b6
feat(mqttconn): stop using gproc in hot path
Also drop fiddling with `mountpoint` since this option seems not to be
used anywhere.
2023-05-30 14:50:05 +03:00
Zaiming (Stone) Shi cc5b4d3748 Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test 2023-05-26 15:58:16 +02:00
Thales Macedo Garitezi 194faefc8a fix(webhook): keep `resource_opts.request_timeout` for webhook bridge (r5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-10001

Recently, we unified request_timeout in a single field located at the
webhook connector schema.  However, the correct fix would be to use
the resource_opts.request_timeout one, as that’s the only one that
allows infinity timeout.
2023-05-25 13:19:27 -03:00
JianBo He 6b5ff24952 test: assert the block_until result 2023-05-25 18:12:38 +08:00
JianBo He de7f1c8aec test: add tests for auto_restart_interval 2023-05-25 17:15:19 +08:00
某文 7da0860d6c chore: upgrade minirest to 1.3.10 for more clear error msg 2023-05-24 10:44:52 +08:00
Thales Macedo Garitezi b5f63f4151 test: fix wrong test listeners option 2023-05-23 14:50:43 -03:00
某文 1deb3c50c8 test: bridge test init dashboard listener failed 2023-05-23 10:28:01 +08:00
某文 67a30ee976 test: bridge test init dashboard listener failed 2023-05-23 10:16:43 +08:00
Zaiming (Stone) Shi 732a7be187 Merge remote-tracking branch 'origin/release-50' 2023-05-22 17:46:54 +02:00
Thales Macedo Garitezi 7d798c10e9 perf(buffer_worker): flush metrics periodically inside buffer worker process
Fixes https://emqx.atlassian.net/browse/EMQX-9905

Since calling `telemetry` is costly in a hot path, we instead collect
metrics inside the buffer workers state and periodically flush them,
rather than immediately as events happen.
2023-05-22 09:11:23 -03:00
Thales Macedo Garitezi 09ea2e2224 fix(bridge_api): don't crash when formatting empty/unknown bridge metrics
Fixes https://emqx.atlassian.net/browse/EMQX-9872
2023-05-18 14:25:13 -03:00
Thales Macedo Garitezi 447b76464b Merge branch 'release-50' into merge-r50-into-v50-a 2023-05-17 14:50:18 -03:00
Thales Macedo Garitezi a7b41e1cdf perf(webhook): add retry attempts for async
This is a performance improvement for webhook bridge.

Since this bridge is called using `async` callback mode, and `ehttpc`
frequently returns errors of the form `normal` and `{shutdown,
normal}` that are retried "for free" by `ehttpc`, we add this behavior
to async requests as well.  Other errors are retried too, but they are
not "free": 3 attempts are made at a maximum.

This is important because, when using buffer workers, we should avoid
making them enter the `blocked` state, since that halts all progress
and makes throughput plummet.
2023-05-17 09:20:50 -03:00
某文 2b99a9b988 feat: hide resource_opts's request_timeout 2023-05-17 13:41:11 +08:00
某文 a2aa6b4666 chore: make ci happy again 2023-05-16 21:29:52 +08:00
Zhongwen Deng 14952658d5 chore: convert rotation struct to roation_count 2023-05-09 11:24:51 +08:00
Stefan Strigler d231e708cb feat: add IotDB bridge 2023-05-08 16:19:01 +02:00
Zaiming (Stone) Shi 856129984b refactor: remove raw_with_default config load option
This option was previously only in tests to avoid
emqx_conf app overwriting previously set configs with default values.

After a03f2dd64b, the issue for
test cases had been resolved.
This commit is to get rid of the option all together
2023-05-04 09:48:28 +02:00
Serge Tupchii 45254d7d85 fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
Fixes: EMQX-9310
2023-04-20 17:21:43 +03:00
Stefan Strigler 7df0493312
Merge pull request #10390 from sstrigler/EMQX-9549-new-emqx-utils-app-to-collect-utility-modules
New emqx_utils app to collect utility modules
2023-04-14 20:33:11 +02:00
Thales Macedo Garitezi e073bc90bc refactor(buffer_worker): rename `s/queue/buffer/g` 2023-04-14 11:37:19 -03:00
Stefan Strigler 4f80690162 fix: byebye jsx 2023-04-14 13:41:34 +02:00
Stefan Strigler 062ce5f819 refactor: rename emqx_map_lib to emqx_utils_maps 2023-04-14 13:41:34 +02:00
Stefan Strigler 9c11bfce80 refactor: rename emqx_misc to emqx_utils 2023-04-14 13:41:27 +02:00
Stefan Strigler f8e9e54393 refactor: move emqx_json to emqx_utils_json 2023-04-14 13:31:27 +02:00
Andrew Mayorov e70deae1c3
feat(resource): ask for metrics only when needed 2023-04-11 12:00:19 +03:00
Zaiming (Stone) Shi 80eb9d7542
Merge pull request #10252 from emqx/release-50
0327 merge release-50 to master
2023-03-29 12:33:17 +02:00
Kjell Winblad 8e0d315b7b
Merge pull request #10197 from kjellwinblad/0321-fix-inflight-window-hand-over-to-kjell
fix: add inflight window setting to the clickhouse bridge
2023-03-29 09:38:24 +02:00
Thales Macedo Garitezi 1824e7efcc fix(rule_engine): count referenced bridges in `from` clauses as dependencies (rv5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9325

Currently, ingress bridges referenced in the `FROM` clause of rules
are not being accounted as dependencies.

When we try to delete an ingress bridge that's referenced in a rule
like `select * from "$bridges/mqtt:ingress"`, that bridge does not
trigger an UI warning about dependent actions.
2023-03-28 10:31:10 -03:00
Andrew Mayorov d942a531e6
fix(test): avoid unnecessary cleanups
This might make some noise in the logs, depending on the testsuite
configuration, because some test setup might be missing required
applications.
2023-03-27 23:41:19 +03:00
Andrew Mayorov 11b5b7b638
test(bridge-api): also run testcases in cluster environment
Excluding a couple of testcases which does not make much sense running
in the cluster. Also try to reduce amount of "noise" in the testcases,
making them easier to comprehend.

Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-03-27 23:41:08 +03:00
Stefan Strigler fbfdaf44e0 fix(emqx_bridge): don't crash on validation error 2023-03-24 11:46:41 +01:00
Kjell Winblad 35474578ca refactor: rename async_inflight_window to inflight_window everywhere 2023-03-23 14:21:57 +01:00
Zaiming (Stone) Shi e6091db351 Merge remote-tracking branch 'origin/release-50' into 0321-merge-release-50-to-master 2023-03-21 22:03:31 +01:00
Stefan Strigler 3880862c81 fix(emqx_bridge): return 503 for inconsistency in bridge setup 2023-03-21 15:07:43 +01:00
Stefan Strigler 84fc64822e style: fix wording for nxdomain and malformed_username_or_password
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 15:02:29 +01:00
Stefan Strigler 53825b9aba fix(emqx_bridge): propagate connection error to resource status 2023-03-21 15:02:29 +01:00
Erik Timan 2d75c7d6d9 fix(emqx_bridge): remove metrics from non-dedicated bridge API endpoints
Metrics should only be exposed via the /bridges/:id/metrics endpoint,
and not in other operations such as getting the list of all bridges, or
in the response when a bridge has been created. This commit removes all
traces of metrics for the non-dedicated API endpoints.
2023-03-20 09:43:11 +01:00
Thales Macedo Garitezi 66eb4ef069 test: fix inter-suite flakiness 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII
feat: add ocsp stapling support to mqtt ssl listener (5.0)
2023-03-16 13:10:48 -03:00
Andrew Mayorov 816667d85c
Merge pull request #10123 from fix/EMQX-9136/lookup-busy-resources
perf(bridge-api): ask nodes' bridge listings in parallel
2023-03-16 00:07:12 +03:00
Thales Macedo Garitezi 03b95073fc test: fix inter-suite flakiness 2023-03-15 14:25:41 -03:00
Andrew Mayorov b3e7e51094
test(bridge): drop unnecessary cleanup routines
Since `end_per_testcase` cleans out all the resources anyway.
2023-03-15 19:17:29 +03:00
Stefan Strigler c1adf0de1f test(emqx_bridge): increase coverage by adding common edge cases 2023-03-13 10:40:47 +01:00
Stefan Strigler 71ec77a2f2 fix(emqx_bridge): for node operation check if node is part of cluster
This fixes the case where we returned `501 NOT IMPLEMENTED` in the past.
2023-03-13 10:40:47 +01:00