Thales Macedo Garitezi
7d798c10e9
perf(buffer_worker): flush metrics periodically inside buffer worker process
...
Fixes https://emqx.atlassian.net/browse/EMQX-9905
Since calling `telemetry` is costly in a hot path, we instead collect
metrics inside the buffer workers state and periodically flush them,
rather than immediately as events happen.
2023-05-22 09:11:23 -03:00
William Yang
e824e1db39
Merge pull request #10698 from qzhuyan/perf/william/force-atom-conf-path
...
perf(config): enforcing atom key path in hotcode path
2023-05-22 09:38:09 +02:00
Thales Macedo Garitezi
09ea2e2224
fix(bridge_api): don't crash when formatting empty/unknown bridge metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-9872
2023-05-18 14:25:13 -03:00
Thales Macedo Garitezi
5d5c16a56d
feat(bridges): use union member selector function for better error messages
2023-05-17 17:57:29 -03:00
Thales Macedo Garitezi
447b76464b
Merge branch 'release-50' into merge-r50-into-v50-a
2023-05-17 14:50:18 -03:00
Thales Macedo Garitezi
b2afe4e90c
Merge pull request #10690 from thalesmg/perf-webhook-retry-async-reply-v50
...
perf(webhook): add async retries and evaluate reply callback in fresh process
2023-05-17 11:06:48 -03:00
Thales Macedo Garitezi
a7b41e1cdf
perf(webhook): add retry attempts for async
...
This is a performance improvement for webhook bridge.
Since this bridge is called using `async` callback mode, and `ehttpc`
frequently returns errors of the form `normal` and `{shutdown,
normal}` that are retried "for free" by `ehttpc`, we add this behavior
to async requests as well. Other errors are retried too, but they are
not "free": 3 attempts are made at a maximum.
This is important because, when using buffer workers, we should avoid
making them enter the `blocked` state, since that halts all progress
and makes throughput plummet.
2023-05-17 09:20:50 -03:00
William Yang
cdf42760fa
chore: little change avoid atom leak
2023-05-17 11:01:03 +02:00
某文
2b99a9b988
feat: hide resource_opts's request_timeout
2023-05-17 13:41:11 +08:00
Zaiming (Stone) Shi
9b9731b943
Merge pull request #10473 from zmstone/0421-refactor-delete-unused-schema-field
...
0421 refactor delete unused schema field
2023-05-16 22:22:03 +02:00
某文
a2aa6b4666
chore: make ci happy again
2023-05-16 21:29:52 +08:00
William Yang
1c746ed289
perf(config): enforcing atom key path in hotcode path
2023-05-16 09:57:04 +02:00
某文
7d7c069257
feat: update wehbook's request_timeout into resource_opts
2023-05-16 15:32:43 +08:00
zhongwencool
a953b951fe
Merge branch 'master' into sync-release-50-to-master
2023-05-12 18:01:58 +08:00
某文
6056b0e7a8
fix: ensure atom key for emqx_config:get
2023-05-11 17:40:03 +08:00
Zaiming (Stone) Shi
7566c25199
refactor: delete unused connector field 'base_url'
2023-05-11 09:04:55 +02:00
Zhongwen Deng
14952658d5
chore: convert rotation struct to roation_count
2023-05-09 11:24:51 +08:00
Stefan Strigler
d231e708cb
feat: add IotDB bridge
2023-05-08 16:19:01 +02:00
Zaiming (Stone) Shi
856129984b
refactor: remove raw_with_default config load option
...
This option was previously only in tests to avoid
emqx_conf app overwriting previously set configs with default values.
After a03f2dd64b
, the issue for
test cases had been resolved.
This commit is to get rid of the option all together
2023-05-04 09:48:28 +02:00
Zaiming (Stone) Shi
d5f5f35787
Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master
2023-05-02 18:08:04 +02:00
Zaiming (Stone) Shi
57cc854a4a
test(bridge): fix bridge map type filed converters
...
now the converters on map type fields only work at the wrapping map
level but not the values
2023-04-30 10:45:11 +02:00
Paulo Zulato
dd90b2f498
feat(oracle): Oracle Database integration
2023-04-27 13:07:20 -03:00
Thales Macedo Garitezi
ad4be08bb2
feat: implement Pulsar Producer bridge (e5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-8398
2023-04-24 10:28:26 -03:00
Serge Tupchii
45254d7d85
fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
...
Fixes: EMQX-9310
2023-04-20 17:21:43 +03:00
Zaiming (Stone) Shi
30fd9b10f7
refactor: change bridges config importance level from hidden to low
2023-04-18 19:17:03 +02:00
Ivan Dyachkov
dc78ecb41c
chore: merge upstream/master
2023-04-18 17:33:32 +02:00
某文
fbadfc06e4
feat: change exhook, rule_engine, bridge to low importance level instead of hidden
2023-04-18 20:15:19 +08:00
Ivan Dyachkov
9fc8a498f8
chore: bump apps versions
2023-04-17 09:09:08 +02:00
Stefan Strigler
7df0493312
Merge pull request #10390 from sstrigler/EMQX-9549-new-emqx-utils-app-to-collect-utility-modules
...
New emqx_utils app to collect utility modules
2023-04-14 20:33:11 +02:00
Thales Macedo Garitezi
e073bc90bc
refactor(buffer_worker): rename `s/queue/buffer/g`
2023-04-14 11:37:19 -03:00
Stefan Strigler
4f80690162
fix: byebye jsx
2023-04-14 13:41:34 +02:00
Stefan Strigler
062ce5f819
refactor: rename emqx_map_lib to emqx_utils_maps
2023-04-14 13:41:34 +02:00
Stefan Strigler
d0df086c80
refactor: rename emqx_api_lib to emqx_utils_api
2023-04-14 13:41:33 +02:00
Stefan Strigler
9c11bfce80
refactor: rename emqx_misc to emqx_utils
2023-04-14 13:41:27 +02:00
Stefan Strigler
f8e9e54393
refactor: move emqx_json to emqx_utils_json
2023-04-14 13:31:27 +02:00
JimMoen
790d841697
Merge pull request #10363 from JimMoen/feat-mssqlserver-birdge
...
feat: implement Microsoft SQL Server bridge (e5.0)
2023-04-14 17:55:48 +08:00
Andrew Mayorov
5e92ba6fa9
Merge pull request #10359 from ft/EMQX-9136/no-ask-metrics
...
feat(resource): ask for metrics only when needed
2023-04-14 12:28:52 +03:00
JimMoen
5841969877
feat: implement Microsoft SQL Server bridge (e5.0)
2023-04-14 10:02:45 +08:00
Zhongwen Deng
cc2beda37a
feat: hiden rule_engine/bridge/authz/authn from doc/example
2023-04-13 14:42:21 +08:00
Ivan Dyachkov
bdffa925db
chore: merge upstream/master release-50
2023-04-12 15:30:20 +02:00
Andrew Mayorov
e70deae1c3
feat(resource): ask for metrics only when needed
2023-04-11 12:00:19 +03:00
Stefan Strigler
04626ce9cc
fix: create consistent interface 'with_node' for API access
2023-04-04 16:54:14 +02:00
firest
11b3264251
fix: redact the password to `******` in API examples
...
fix #10222
2023-04-04 16:02:45 +08:00
Zaiming (Stone) Shi
36000abf51
refactor: relocate i18n files for apps/emqx
2023-04-03 13:12:24 +02:00
Zaiming (Stone) Shi
80eb9d7542
Merge pull request #10252 from emqx/release-50
...
0327 merge release-50 to master
2023-03-29 12:33:17 +02:00
Kjell Winblad
8e0d315b7b
Merge pull request #10197 from kjellwinblad/0321-fix-inflight-window-hand-over-to-kjell
...
fix: add inflight window setting to the clickhouse bridge
2023-03-29 09:38:24 +02:00
Thales Macedo Garitezi
64faccf50b
test: fix flaky kafka consumer test
2023-03-28 14:50:55 -03:00
Thales Macedo Garitezi
1824e7efcc
fix(rule_engine): count referenced bridges in `from` clauses as dependencies (rv5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9325
Currently, ingress bridges referenced in the `FROM` clause of rules
are not being accounted as dependencies.
When we try to delete an ingress bridge that's referenced in a rule
like `select * from "$bridges/mqtt:ingress"`, that bridge does not
trigger an UI warning about dependent actions.
2023-03-28 10:31:10 -03:00
Stefan Strigler
193905c7b9
Merge pull request #10237 from sstrigler/EMQX-9138-e-5-0-2-ensure-that-the-error-messages-returned-are-consistent
...
fix: create consistent interface 'with_node' for API access
2023-03-28 10:41:10 +02:00
Andrew Mayorov
d942a531e6
fix(test): avoid unnecessary cleanups
...
This might make some noise in the logs, depending on the testsuite
configuration, because some test setup might be missing required
applications.
2023-03-27 23:41:19 +03:00
Andrew Mayorov
11b5b7b638
test(bridge-api): also run testcases in cluster environment
...
Excluding a couple of testcases which does not make much sense running
in the cluster. Also try to reduce amount of "noise" in the testcases,
making them easier to comprehend.
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-03-27 23:41:08 +03:00
Stefan Strigler
64a1d84a44
fix: create consistent interface 'with_node' for API access
2023-03-27 12:14:48 +02:00
JianBo He
bfa5922209
Merge pull request #10140 from HJianBo/cassa
...
feat: support cassandra data bridge
2023-03-27 10:23:02 +08:00
Zaiming (Stone) Shi
dd996ad1dc
chore: bump app vsns
2023-03-24 21:47:15 +01:00
Stefan Strigler
8d084ed13b
Merge pull request #10226 from sstrigler/EMQX-9273-e-5-0-2-creating-a-data-bridge-the-name-verification-returned-an-incorrect-error-code-of-500
...
fix(emqx_bridge): don't crash on validation error
2023-03-24 14:48:03 +01:00
Stefan Strigler
fbfdaf44e0
fix(emqx_bridge): don't crash on validation error
2023-03-24 11:46:41 +01:00
Stefan Strigler
a8cfc1c7a1
fix(emqx_bridge): filter status_reason in config update
2023-03-24 11:46:41 +01:00
Stefan Strigler
ba49316ec5
fix: use lower-case
2023-03-23 16:08:27 +01:00
Kjell Winblad
35474578ca
refactor: rename async_inflight_window to inflight_window everywhere
2023-03-23 14:21:57 +01:00
JianBo He
8cbbc9f271
Merge remote-tracking branch 'upstream/master' into cassa
2023-03-23 11:53:17 +08:00
lafirest
84def357a9
Merge pull request #10143 from lafirest/feat/rocketmq
...
feat(bridges): integrate RocketMQ into data bridges
2023-03-22 20:43:22 +08:00
Zaiming (Stone) Shi
e43aa5d841
Merge pull request #10199 from zmstone/0321-merge-release-50-to-master
...
0321 merge release 50 to master
2023-03-22 10:20:48 +01:00
Andrew Mayorov
af7b2215ca
fix(bridge-api): unflatten nodes' bridge lists back
...
Bridge lists were erroneously flattened in cad6492c
. This causes
bridge listing fail in emqx clusters consisting of more than 1 node.
2023-03-22 11:35:09 +03:00
firest
9853d00cad
feat(bridges): integrate RocketMQ into data bridges
2023-03-22 10:36:52 +08:00
JianBo He
65c2da7ef5
Merge remote-tracking branch 'ce/master' into cassa
2023-03-22 09:30:50 +08:00
Andrew Mayorov
bf35419b52
Merge pull request #10190 from keynslug/fix/EMQX-9269/bridge-list
...
fix(bridge-api): unflatten nodes' bridge lists back
2023-03-22 00:53:03 +03:00
Zaiming (Stone) Shi
e6091db351
Merge remote-tracking branch 'origin/release-50' into 0321-merge-release-50-to-master
2023-03-21 22:03:31 +01:00
Stefan Strigler
6c24b3a77d
Merge pull request #10196 from sstrigler/EMQX-9201-the-case-is-not-correct-for-some-of-menu
...
Emqx 9201 the case is not correct for some of menu
2023-03-21 19:42:10 +01:00
Andrew Mayorov
d9b1b1c802
fix(bridge-api): unflatten nodes' bridge lists back
...
Bridge lists were erroneously flattened in cad6492c
. This causes
bridge listing fail in emqx clusters consisting of more than 1 node.
2023-03-21 18:59:27 +03:00
Stefan Strigler
981268911a
style: make summary lower-case
2023-03-21 16:56:37 +01:00
Stefan Strigler
3880862c81
fix(emqx_bridge): return 503 for inconsistency in bridge setup
2023-03-21 15:07:43 +01:00
Stefan Strigler
4b0ea562a2
refactor(emqx_bridge): consistently use macros for http response
2023-03-21 15:07:41 +01:00
Stefan Strigler
84fc64822e
style: fix wording for nxdomain and malformed_username_or_password
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 15:02:29 +01:00
Stefan Strigler
8af3fb4ee7
feat: move human readable error translations to emqx_misc
2023-03-21 15:02:29 +01:00
Stefan Strigler
53825b9aba
fix(emqx_bridge): propagate connection error to resource status
2023-03-21 15:02:29 +01:00
Erik Timan
2d75c7d6d9
fix(emqx_bridge): remove metrics from non-dedicated bridge API endpoints
...
Metrics should only be exposed via the /bridges/:id/metrics endpoint,
and not in other operations such as getting the list of all bridges, or
in the response when a bridge has been created. This commit removes all
traces of metrics for the non-dedicated API endpoints.
2023-03-20 09:43:11 +01:00
JianBo He
12942b676d
Merge remote-tracking branch 'upstream/master' into cassa
2023-03-20 09:50:27 +08:00
JianBo He
c0a216a740
feat(bridge): support cassandra bridge
2023-03-17 11:34:48 +08:00
Thales Macedo Garitezi
3954b7bde2
fix(bridges): function clause when a non-ingress bridge coexists with an egress bridge
...
This was not caught by our tests because we always test bridge types
in isolation. So, if the config only contains ingress-only bridges,
the `on_message_publish` hook is never installed.
In a real system, if there are bridges of mixed types in the config,
the hook might be installed, and `emqx_bridge:get_matched_bridge_id`
would crash when iterating over the ingress bridges.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
66eb4ef069
test: fix inter-suite flakiness
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
f31f15e44e
chore(kafka_producer): make schema changes more backwards compatible
...
This will still require fixes to the frontend.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
03342923b9
fix(bridge): use the same dry run prefix
...
Kafka Producer and Consumer bridges rely on this prefix for detecting
a dry run and avoid leaking atoms. At some point, this prefix was
changed, effectively disabling the check in Kafka Producer.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
1f31a87974
fix(bridge): improve macro var usage
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
969fa03ecc
feat: implement kafka consumer
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII
...
feat: add ocsp stapling support to mqtt ssl listener (5.0)
2023-03-16 13:10:48 -03:00
Andrew Mayorov
816667d85c
Merge pull request #10123 from fix/EMQX-9136/lookup-busy-resources
...
perf(bridge-api): ask nodes' bridge listings in parallel
2023-03-16 00:07:12 +03:00
Thales Macedo Garitezi
03b95073fc
test: fix inter-suite flakiness
2023-03-15 14:25:41 -03:00
Andrew Mayorov
b3e7e51094
test(bridge): drop unnecessary cleanup routines
...
Since `end_per_testcase` cleans out all the resources anyway.
2023-03-15 19:17:29 +03:00
Andrew Mayorov
cad6492c99
perf(bridge-api): ask bridge listings in parallel
...
Also rename response formatting functions to better clarify their
purpose.
2023-03-15 19:17:29 +03:00
Andrew Mayorov
53bc27e0f4
refactor(bridge): avoid unnecessary `maps:to_list/1` when listing
2023-03-13 14:49:38 +03:00
Stefan Strigler
c1adf0de1f
test(emqx_bridge): increase coverage by adding common edge cases
2023-03-13 10:40:47 +01:00
Stefan Strigler
478601ee41
fix(emqx_bridge): remove unreachable code paths
2023-03-13 10:40:47 +01:00
Stefan Strigler
dba95ec0fa
style(emqx_bridge): fix wording
2023-03-13 10:40:47 +01:00
Stefan Strigler
71ec77a2f2
fix(emqx_bridge): for node operation check if node is part of cluster
...
This fixes the case where we returned `501 NOT IMPLEMENTED` in the past.
2023-03-13 10:40:47 +01:00
Stefan Strigler
7124600a71
docs(emqx_bridge): add `start` as operation to bridge
2023-03-13 10:38:51 +01:00
Stefan Strigler
1bcc5623ed
fix(emqx_bridge): check if bridge enabled before calling op
2023-03-13 10:38:49 +01:00
Stefan Strigler
a325133391
fix(emqx_bridge): don't crash checking if bridge enabled
2023-03-13 10:37:28 +01:00
Stefan Strigler
80b81748df
fix(emqx_bridge): handle bridge not found in call_operation
2023-03-13 10:37:28 +01:00
Stefan Strigler
fb3d101b3a
refactor(emqx_bridge): fix var names
2023-03-13 10:37:28 +01:00
Stefan Strigler
e31f4d6091
refactor(emqx_bridge): add BAD_REQUEST macro and minor cleanups
2023-03-13 10:37:28 +01:00
ieQu1
e3595f2e79
chore(mria): Bump version to 0.4.0
2023-03-11 00:37:25 +01:00
Andrew Mayorov
a86d06f043
chore: bump app versions following last merge-back
2023-03-10 16:44:15 +03:00
Zaiming (Stone) Shi
fe27604010
Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master
2023-03-08 16:46:45 +01:00
firest
b8b1a78025
feat(bridges): integrate the DynamoDB into data bridges
2023-03-08 11:13:38 +08:00
Zaiming (Stone) Shi
26b29185b2
test(emqx_bridge_webhook_SUITE): fix flakyness in test web server
2023-03-07 20:57:38 +01:00
Kjell Winblad
163b33ab28
test: remove unnecessary dependencies of ee apps
2023-03-07 20:57:38 +01:00
Kjell Winblad
ca947e3e70
fix: lost messages when HTTP connection times out
...
When using async mode with the webhook bridge, queued messages that are
not fully processed when the connection times out could be lost. This
commit fixes this by letting the bridge return a recoverable_error when
this happen. The message send will then be retried in sync mode by the
emqx_resource_buffer_worker.
Fixes: https://emqx.atlassian.net/browse/EMQX-8974
2023-03-07 20:57:19 +01:00
Thales Macedo Garitezi
e17ad320ee
fix(bridge): do not log in converter
2023-03-06 15:31:28 -03:00
Thales Macedo Garitezi
2fa4e8e21a
Merge pull request #10068 from thalesmg/fix-flaky-mqtt-bridge-test
...
test: fix flaky mqtt bridge test
2023-03-06 13:06:37 -03:00
Stefan Strigler
a81d1a4618
Merge pull request #10066 from sstrigler/EMQX-8932-bridge-api-reports-500-for-most-errors-when-starting-a-resource
...
fix: return human readable error message for most common cases
2023-03-06 14:13:34 +01:00
Thales Macedo Garitezi
f95a30ae89
fix(webhook): convert `request_timeout`s in root and resource_opts
2023-03-06 10:12:38 -03:00
Thales Macedo Garitezi
8fbb948b6f
test: fix flaky mqtt bridge test
...
Sometimes, this test fails because the metrics are still in the
inflight phase.
2023-03-06 09:09:33 -03:00
Kjell Winblad
a638cc6566
Merge pull request #9985 from kjellwinblad/kjell/feat/clickhouse_bridge_2/EMQX-8391
...
feat: add clickhouse database bridge
2023-03-04 07:05:05 +01:00
Stefan Strigler
570bf165af
fix: return human readable error message for most common cases
2023-03-03 16:49:09 +01:00
Ivan Dyachkov
c01f62a1c1
chore: bump apps versions
2023-03-03 13:58:32 +01:00
Kjell Winblad
67acdf0888
feat: add clickhouse database bridge
...
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
2023-03-02 12:22:11 +01:00
Stefan Strigler
39e1cf9502
fix(emqx_bridge): let it crash instead of 400 on failed reset
2023-03-02 11:23:07 +01:00
Stefan Strigler
4c23ab097d
fix(emqx_bridge): return 400 if operation not possible
2023-03-02 11:23:07 +01:00
Stefan Strigler
680a4362d1
fix: match macro variables
2023-03-02 10:12:56 +01:00
Stefan Strigler
76f97010bb
style: change wording
2023-03-02 09:14:52 +01:00
Serge Tupchii
77dea0c77a
fix: Handle obfuscated fields in bridges_probe API
2023-03-01 22:05:23 +02:00
Stefan Strigler
3f324f85bc
fix: consistently return 404 in case bridge is not found or invalid
...
Also: fix some typos and be more verbose
2023-03-01 18:03:49 +01:00
Zaiming (Stone) Shi
56279f6305
docs: fix bridge status report
2023-02-28 11:56:46 +01:00
Andrew Mayorov
7c573d9365
fix(bridgeapi): anticipate node responses w/o metrics
...
When some resource manager is busy with trying to estabilish a
connection with remote, we hit the "read-from-cache" codepath so the
resource data will not contain any metrics.
2023-02-27 14:55:41 +03:00
Zaiming (Stone) Shi
72fe10f18b
chore: fix spell check errors
2023-02-10 19:41:14 +01:00
Andrew Mayorov
2ed54e560f
Merge pull request #9952 from keynslug/fix/mqtt-bridge-no-qos2-clean-start
...
fix(mqtt-bridge): disallow QoS 2 on ingress bridges
2023-02-10 20:40:37 +04:00
Andrew Mayorov
cbb2885499
fix(mqtt-bridge): allow to configure `clean_start` for ingresses
2023-02-10 19:15:48 +03:00
Andrew Mayorov
7002fe2ef4
fix(mqtt-bridge): disallow QoS 2 on ingress bridges
2023-02-10 17:17:59 +03:00
Stefan Strigler
e196d6f42a
fix(emqx_bridge): unwrap rpc multicall result
2023-02-10 12:00:20 +01:00
Zaiming (Stone) Shi
3e85cb3804
Merge pull request #9946 from zmstone/0209-fix-add-back-deprecate-field
...
fix(config): add back reconnect_interval field as deprecated
2023-02-10 09:19:12 +01:00
Zaiming (Stone) Shi
42dfaf3ef2
Merge pull request #9910 from sstrigler/EMQX-8861-improve-bridge-restart-button-behaviour
...
EMQX 8861 improve bridge restart button behaviour
2023-02-09 18:00:48 +01:00
Zaiming (Stone) Shi
4e71f1cbc5
Merge pull request #9940 from zmstone/0208-ensure-no-leading-br-in-docs
...
Add style checks for i18n.conf files
2023-02-09 17:37:44 +01:00
Zaiming (Stone) Shi
5328955ff4
fix(config): add back reconnect_interval field as deprecated
2023-02-09 17:30:09 +01:00
Andrew Mayorov
48efe552d1
test(mqtt-bridge): manage test flaps by waiting for flush events
2023-02-09 14:31:42 +03:00
Zaiming (Stone) Shi
157c919ba1
ci: add i18n style check script
2023-02-09 11:41:52 +01:00
Stefan Strigler
f89ac54b17
style: remove stale comment
2023-02-08 14:04:09 +01:00
Stefan Strigler
a3fd0897bc
refactor: less code duplication
2023-02-08 14:04:09 +01:00
Stefan Strigler
502f62e18d
fix: try old 'restart' behavior if 'start' is not implemented
2023-02-08 13:19:41 +01:00
Stefan Strigler
86f3f5787f
feat: allow to manually re-connect disconected bridge
2023-02-07 11:58:30 +01:00
Stefan Strigler
4e12a44ee6
fix: return 204 instead of 200 if there's no body content
2023-02-06 09:47:07 +01:00
Zaiming (Stone) Shi
a71d983ff8
test: fix assertMatch patter to make rebar3 fmt work
2023-02-02 16:37:45 +01:00
Zaiming (Stone) Shi
85fb53dcf0
refactor: use compile switch to diff functions between ce and ee
2023-02-02 15:15:31 +01:00
Andrew Mayorov
0912f13c1f
fix(mqtt-bridge): stop respecting `clean_start` config parameter
...
We are ignoring the user configuration because there's currently no
reliable way to ensure proper session recovery according to the MQTT
spec.
2023-02-02 12:55:10 +03:00
Andrew Mayorov
8a46cb974e
test(mqtt-bridge): test async bridge reconnects seamlessly
2023-02-01 16:28:16 +03:00
Andrew Mayorov
5ebceb20d2
test(mqtt-bridge): also test reconfiguration
2023-02-01 16:23:58 +03:00
Andrew Mayorov
ad88938d34
refactor: reuse some parts of test code for brewity
2023-02-01 16:22:19 +03:00
Andrew Mayorov
c5a7cd5acd
fix(mqtt-bridge): drop unused configuration parameter
2023-01-31 18:30:42 +03:00
Andrew Mayorov
d0c10b59aa
feat(mqtt-bridge): avoid middleman process
...
Instead, supervise `emqtt` client process directly.
2023-01-31 17:59:03 +03:00
Zaiming (Stone) Shi
7f66c6a9e2
Merge pull request #9840 from olcai/redact-influxdb-tokens
...
fix: redact influxdb tokens in logs and reduce log level
2023-01-28 11:47:36 +01:00
Zaiming (Stone) Shi
52b75ada04
Merge pull request #9832 from sstrigler/EMQX-8774-failure-to-handle-timeout-error-in-resource-worker
...
EMQX 8774 failure to handle timeout error in resource worker
2023-01-27 14:36:44 +01:00
Zaiming (Stone) Shi
c47be57c59
fix(bridge): ensure all bridge resources are stopped before app stop
2023-01-27 12:39:05 +01:00
Stefan Strigler
7005b71ddf
style: fix typo in comment
2023-01-27 11:43:51 +01:00
Erik Timan
8836494542
fix: redact influxdb tokens in a few logs
2023-01-25 14:48:32 +01:00
Zaiming (Stone) Shi
5fdf7fd24c
fix(kafka): use async callback to bump success counters
...
some telemetry events from wolff are discarded:
* dropped:
this is double counted in wolff,
we now only subscribe to the dropped_queue_full event
* retried_failed:
it has different meanings in wolff,
in wolff, it means it's the 2nd (or onward) produce attempt
in EMQX, it means it's eventually failed after some retries
* retried_success
since we are going to handle the success counters in callbac
this having this reported from wolff will only make things
harder to understand
* failed
wolff never fails (unelss drop which is a different counter)
2023-01-24 21:12:36 +01:00
Erik Timan
28718edbfd
chore: bump application VSNs
2023-01-24 14:12:34 +01:00
Thales Macedo Garitezi
ca4a262b75
refactor: re-organize dealing with unrecoverable errors
2023-01-20 12:00:17 -03:00
Thales Macedo Garitezi
6fa6c679bb
feat(buffer_worker): add expiration time to requests
...
With this, we avoid performing work or replying to callers that are no
longer waiting on a result.
Also introduces two new counters:
- `dropped.expired` :: happens when a request expires before being
sent downstream
- `late_reply` :: when a response is receive from downstream, but the
caller is no longer for a reply because the request has expired, and
the caller might even have retried it.
2023-01-20 11:36:52 -03:00
Thales Macedo Garitezi
47f796dd12
refactor: rename `emqx_resource_worker` -> `emqx_resource_buffer_worker`
...
To make it more clear that it's purpose is serve as a buffering layer.
2023-01-18 16:15:34 -03:00
Zaiming (Stone) Shi
d4f3b4c8c2
Merge remote-tracking branch 'origin/master' into fix-buffer-clear-replayq-on-delete-v50
2023-01-18 11:39:47 +01:00
Ivan Dyachkov
430b0a03d4
Merge pull request #9780 from id/fix-ensure-no-colon-in-filenames
...
fix: ensure no colon in filenames
2023-01-18 09:36:16 +01:00
Stefan Strigler
f899284e3a
Merge pull request #9789 from sstrigler/EMQX-8754-test-function-return-500-of-data-integration-google-pubsub
...
EMQX 8754 test function return 500 of data integration google pubsub
2023-01-17 22:49:28 +01:00
Thales Macedo Garitezi
fa01deb3eb
chore: retry as much as possible, don't reply to caller too soon
2023-01-17 16:49:15 -03:00
Thales Macedo Garitezi
478fcc6ffd
test: fix flaky test
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi
32a9e60313
feat(buffer_worker): also use the inflight table for sync requests
...
Related: https://emqx.atlassian.net/browse/EMQX-8692
This should also correctly account for `retried.*` metrics for sync
requests.
Also fixes cases where race conditions for retrying async requests
could potentially lead to inconsistent metrics.
Fixes more cases where a stale reference to `replayq` was being held
accidentally after a `pop`.
2023-01-17 16:48:48 -03:00
Stefan Strigler
d164e5bc0f
fix: don't require ssl conf
2023-01-17 15:29:19 +01:00
firest
c3d5c25c26
fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12
2023-01-17 17:42:10 +08:00
Ivan Dyachkov
676f017ec0
fix: ensure no colon in filenames
2023-01-16 21:27:01 +01:00
Stefan Strigler
67909f0b40
fix: testing metrics for emqx_bridge_mqtt_SUITE
2023-01-16 12:10:06 +01:00
Stefan Strigler
a19226326c
fix: don't test for strict values as that makes test flaky
2023-01-16 12:10:06 +01:00
JimMoen
54ebc27d24
Merge pull request #9672 from JimMoen/0103-fix-mqtt-bridge
...
Fix the problem that the bridge is not available when the Payload template is empty in the MQTT bridge.
2023-01-16 09:57:20 +08:00
Stefan Strigler
8f5881d1a5
fix: remove stale request/3 from merge error
2023-01-13 16:19:35 +01:00
Stefan Strigler
e08c1d2229
Merge remote-tracking branch 'olcai/refactor-bridges-api' into dev/api-refactor
2023-01-13 15:49:52 +01:00
Stefan Strigler
1690a6dcfc
Merge branch 'master' into dev/api-refactor
2023-01-13 15:34:13 +01:00
Erik Timan
c3133fb6a2
fix(emqx_bridge): small fixes from review
2023-01-13 14:54:54 +01:00
Erik Timan
8dd52e5a6e
chore: add translations to schemas
2023-01-13 14:19:23 +01:00
Erik Timan
f1c58c34ed
test(emqx_bridge): fix fetching of metrics in emqx_bridge_mqtt_SUITE
2023-01-13 14:19:23 +01:00
Zhongwen Deng
f15b29b1ef
chore: upgrade app version
2023-01-13 18:22:29 +08:00
JimMoen
b7259d9a20
test(mqtt-bridge): use empty payload template for ingress/egress mqtt bridge
2023-01-13 18:12:53 +08:00
Erik Timan
42f42de4d9
feat(emqx_bridge): add separate endpoint for enable/disable of bridge
...
In order to improve the consistency with other API endpoints, we move
the enable/disable operations to a separate endpoint
/bridges/{id}/enable/[true,false].
2023-01-12 19:39:43 +01:00
Erik Timan
860e21d40f
feat(emqx_bridge): move metrics to own endpoint, rename reset_metrics
...
In order for the /bridges APIs to be consistent with other APIs, we move
out metrics from GET /bridges/{id} to its own endpoint,
/bridges/{id}/metrics. We also rename /bridges/reset_metrics to
/bridges/metrics/reset.
2023-01-12 19:39:41 +01:00
Erik Timan
c2fd1a4482
feat(emqx_bridge): shorten operation endpoint URLs
...
This shortens and simplifies URLs for performing bridge operations so
that the API looks more congruent.
2023-01-12 19:36:22 +01:00
firest
c5f557e315
fix: disable basic auth for HTTP API
2023-01-12 21:35:49 +08:00
Stefan Strigler
8ad8288195
feat: report error in create_dry_run
2023-01-11 14:22:37 +01:00
Thales Macedo Garitezi
48e1ba4832
feat(docs): add tags to schemas
...
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
2023-01-11 09:10:03 -03:00
zhongwencool
6f0e228a9b
Merge pull request #9717 from zmstone/0110-do-not-start-resource-after-recreate-if-disabled
...
fix(emqx_resource_manager): do not start when disabled
2023-01-11 16:12:53 +08:00
Zaiming (Stone) Shi
85a8eff90b
fix(emqx_resource_manager): do not start when disabled
2023-01-11 08:33:48 +01:00
Zaiming (Stone) Shi
ee89de816c
fix: upgrade bridge config converter
2023-01-10 20:42:30 +01:00
Xinyu Liu
0f70786d13
Merge pull request #9705 from terry-xiaoyu/remove-default-value-of-webhook-body
...
fix: remove the default value of webhook body field
2023-01-10 18:23:50 +08:00
Shawn
724015af47
fix: remove the default value of webhook body field
2023-01-10 09:45:25 +08:00
Kjell Winblad
692c1f0d1b
Merge pull request #9678 from kjellwinblad/kjell/fix/jira/EMQX-8648
...
fix: no feedback when deleting non-existing configuration
2023-01-09 13:52:40 +01:00
Stefan Strigler
c85a988a43
fix: split 'server' param into host and port for mqtt bridge
2023-01-09 13:27:42 +01:00
Stefan Strigler
c42c99f94f
fix: set a timeout for tcp connect
2023-01-09 10:01:35 +01:00
Stefan Strigler
96ca0d9f49
feat(emqx_bridge): add /bridges_probe API endpoint
2023-01-09 10:01:33 +01:00
Shawn
13b2f45405
fix: function_clause when sending messages to bridges
2023-01-06 09:24:38 +08:00
Kjell Winblad
b59c4c34c5
fix(Bridge REST API): no feedback when deleting bridge
...
This fixes https://emqx.atlassian.net/browse/EMQX-8648 . The issue
described in `EMQX-8648` is that when deleting a non-existing bridge the
server gives a success response. See below:
```
curl --head -u admin:public2 -X 'DELETE' 'http://localhost:18083/api/v5/bridges/webhook:i_do_not_exist '
HTTP/1.1 204 No Content
date: Tue, 03 Jan 2023 16:59:01 GMT
server: Cowboy
```
After the fix, deleting a non existing bridge will give the following
response:
```
HTTP/1.1 404 Not Found
content-length: 49
content-type: application/json
date: Thu, 05 Jan 2023 12:40:35 GMT
server: Cowboy
```
Closes: EMQX-8648
2023-01-05 16:19:46 +01:00
Thales Macedo Garitezi
fd360ac6c0
feat(buffer_worker): refactor buffer/resource workers to always use queue
...
This makes the buffer/resource workers always use `replayq` for
queuing, along with collecting multiple requests in a single call.
This is done to avoid long message queues for the buffer workers and
rely on `replayq`'s capabilities of offloading to disk and detecting
overflow.
Also, this deprecates the `enable_batch` and `enable_queue` resource
creation options, as: i) queuing is now always enables; ii) batch_size
> 1 <=> batch_enabled. The corresponding metric
`dropped.queue_not_enabled` is dropped, along with `batching`. The
batching is too ephemeral, especially considering a default batch time
of 20 ms, and is not shown in the dashboard, so it was removed.
2023-01-05 10:15:09 -03:00
firest
b43be50a98
test: add the `redacted` test case for bridges api
2023-01-03 22:16:34 +08:00
firest
b016695cb1
fix(bridges): obfuscate the password in bridges API responses
2023-01-03 22:09:29 +08:00
Thales Macedo Garitezi
7e02eac3bc
Merge pull request #9619 from thalesmg/refactor-gauges-v50
...
refactor(metrics): use absolute gauge values rather than deltas (v5.0)
2023-01-02 10:56:47 -03:00
Zaiming (Stone) Shi
dbc10c2eed
chore: update copyright year 2023
2023-01-02 09:22:27 +01:00
Thales Macedo Garitezi
305ed68916
chore: bump app vsns
2022-12-30 16:51:24 -03:00
Thales Macedo Garitezi
8b060a75f1
refactor(metrics): use absolute gauge values rather than deltas
...
https://emqx.atlassian.net/browse/EMQX-8548
Currently, we face several issues trying to keep resource metrics
reasonable. For example, when a resource is re-created and has its
metrics reset, but then its durable queue resumes its previous work
and leads to strange (often negative) metrics.
Instead using `counters` that are shared by more than one worker to
manage gauges, we introduce an ETS table whose key is not only scoped
by the Resource ID as before, but also by the worker ID. This way,
when a worker starts/terminates, they should set their own gauges to
their values (often 0 or `replayq:count` when resuming off a queue).
With this scoping and initialization procedure, we'll hopefully avoid
hitting those strange metrics scenarios and have better control over
the gauges.
2022-12-30 16:51:24 -03:00
Zaiming (Stone) Shi
0b43ae621d
ci: dump docker-compose log if failed to run ct
2022-12-29 09:23:11 +01:00
Thales Macedo Garitezi
35dc75b7ed
feat(mqtt): add option to customize clientid prefix for egress bridges
...
https://emqx.atlassian.net/browse/EMQX-8445
Currently the bridge client’s client ID is prefixed with the resource
ID.
Sometimes it’s useful for users to have control of this prefix,
e.g. prefix based ACL rules in the target broker.
2022-12-23 09:50:26 -03:00
Zaiming (Stone) Shi
c085ffa0fe
refactor: default mqtt bridgge buffer pool size down to 4
2022-12-19 23:59:46 +01:00
Zaiming (Stone) Shi
9e3da5b661
chore: bump app versions
2022-12-14 20:07:41 +01:00
Zaiming (Stone) Shi
56066a03b5
Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams
2022-12-14 20:04:20 +01:00
Zaiming (Stone) Shi
2d7099e3ae
refactor(emqx_bridge_resource): rename a variable
2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
42c58e2a91
Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams
2022-12-14 15:29:13 +01:00
Thales Macedo Garitezi
b9bc82f87a
feat(gcp_pubsub): add `local_topic` config
...
Given the implicit convention that an egress bridge containing the
`local_topic` config will forward messages without the need for a rule
action, this was added to avoid needing a rule action.
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
c69022f3c4
test(ci): clean config after test
...
Trying to fix this error for profile `emqx`:
```
=CRASH REPORT==== 30-Nov-2022::13:25:46.763989 ===
crasher:
initial call: application_master:init/4
pid: <0.9682.1>
registered_name: []
exception exit: {bad_return,
{{emqx_conf_app,start,[normal,[]]},
{emqx_conf_schema,
[#{kind => validation_error,path => "bridges",
reason => unknown_fields,
unknown => <<"influxdb_api_v1">>,
unmatched => <<"mqtt,webhook">>}]}}}
in function application_master:init/4 (application_master.erl, line 142)
ancestors: [<0.9681.1>]
message_queue_len: 1
messages: [{'EXIT',<0.9683.1>,normal}]
links: [<0.9681.1>,<0.44.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 610
stack_size: 29
reductions: 195
neighbours:
=INFO REPORT==== 30-Nov-2022::13:25:46.777895 ===
application: emqx_conf
exited: {bad_return,
{{emqx_conf_app,start,[normal,[]]},
{emqx_conf_schema,
[#{kind => validation_error,path => "bridges",
reason => unknown_fields,
unknown => <<"influxdb_api_v1">>,
unmatched => <<"mqtt,webhook">>}]}}}
type: temporary
%%% emqx_plugins_SUITE ==> init_per_suite: FAILED
%%% emqx_plugins_SUITE ==> {{failed_to_start_app,emqx_conf,
{emqx_conf,
{bad_return,
{{emqx_conf_app,start,[normal,[]]},
{emqx_conf_schema,
[#{kind => validation_error,path => "bridges",
reason => unknown_fields,
unknown => <<"influxdb_api_v1">>,
unmatched => <<"mqtt,webhook">>}]}}}}},
[{emqx_common_test_helpers,start_app,4,
[{file,
"/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
{line,227}]},
{lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
{emqx_plugins_SUITE,init_per_suite,1,
[{file,
"/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
{line,34}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1380}]},
{test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}
%%% emqx_plugins_SUITE ==> t_bad_info_json: SKIPPED
%%% emqx_plugins_SUITE ==> {tc_auto_skip,
{failed,
{emqx_plugins_SUITE,init_per_suite,
{'EXIT',
{{failed_to_start_app,emqx_conf,
{emqx_conf,
{bad_return,
{{emqx_conf_app,start,[normal,[]]},
{emqx_conf_schema,
[#{kind => validation_error,
path => "bridges",
reason => unknown_fields,
unknown => <<"influxdb_api_v1">>,
unmatched => <<"mqtt,webhook">>}]}}}}},
[{emqx_common_test_helpers,start_app,4,
[{file,
"/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
{line,227}]},
{lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
{emqx_plugins_SUITE,init_per_suite,1,
[{file,
"/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
{line,34}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,
[{file,"test_server.erl"},{line,1380}]},
{test_server,run_test_case_eval,9,
[{file,"test_server.erl"},{line,1224}]}]}}}}}
```
2022-12-12 17:18:13 -03:00
William Yang
6b6bb09d4a
fix: ingress only bridge causes egress bridge traffic stop
2022-12-09 15:36:20 +01:00
Zaiming (Stone) Shi
40809b2ad0
Merge remote-tracking branch 'origin/dev/ee5.0' into release-50
2022-12-09 11:45:52 +01:00
Ilya Averyanov
6692b0c895
feat(bridge): add Redis bridge
2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi
31098d6c67
test: fix bridge api tests, metrics is now a separate api
2022-12-06 19:03:32 +01:00
Zaiming (Stone) Shi
eb017ab034
test: add test case to cover mqtt bridge config upgrade
2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi
9ea22d062d
refactor: make all bridges optional (required = false)
2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi
9a6901987f
feat: add mqtt bridge config upgrade converter
2022-11-30 21:01:59 +01:00
Zaiming (Stone) Shi
983e904858
fix(connector): fix ssl clear
2022-11-30 17:03:41 +01:00
Zaiming (Stone) Shi
b398617614
chore: bump app versions
2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi
7ee53e5319
Merge tag 'v5.0.11' into dev/ee5.0
2022-11-28 21:02:21 +01:00
firest
6dc5078a96
fix: fix obsolete SSL files aren't deleted after the bridge configuration update
2022-11-23 23:49:43 +01:00
firest
1719414c04
chore: update app versions
2022-11-23 11:42:49 +08:00
firest
19405114e2
fix: fix newly found unsafe `binary_to_atom`
2022-11-23 11:07:20 +08:00
Zaiming (Stone) Shi
e5ced07665
fix: add no_return function spec to make dialyzer happy
2022-11-16 19:07:51 +01:00
Zaiming (Stone) Shi
7305923d12
fix: bridge name parser should not leak atom
2022-11-16 18:25:42 +01:00
Zaiming (Stone) Shi
c940b901f5
chore: fix app versions
2022-11-16 16:26:43 +01:00
Zaiming (Stone) Shi
09455edae8
Merge tag 'v5.0.10' into dev/ee5.0
2022-11-16 16:20:30 +01:00
firest
6c52d5de1f
fix(bridge): Ensure that the node name is known
2022-11-01 15:36:10 +08:00
firest
73b4ac9f65
fix(bridge): keep bridge name type as binary don't convert it to atom
...
After investigation, it was confirmed that there was no need to convert the bridge name to atom
2022-11-01 11:24:24 +08:00
Zaiming (Stone) Shi
c157392452
docs: fix self-closing html tag, change </br> to <br/>
2022-10-27 13:57:18 +02:00
Thales Macedo Garitezi
2d01726b22
fix: account calls when resource is not connected as matched
2022-10-13 15:32:04 -03:00
Thales Macedo Garitezi
f0ff32c031
test: fix tests after counter changes
2022-10-11 17:45:48 -03:00
Shawn
dfe14be8b2
chore: bump app vsns
2022-09-26 18:23:28 +08:00
Shawn
4135910b42
chore: merge master into dev/ee5.0
2022-09-26 09:52:33 +08:00
Zaiming (Stone) Shi
f6ac4c3a76
Merge pull request #8798 from zmstone/0815-feat-add-kafka-connector
...
feat: Add Kafka connector
2022-09-24 22:57:50 +02:00
Shawn
d24441b204
fix(bridges): some issues found in code review
2022-09-23 18:32:39 +08:00
Zaiming (Stone) Shi
7b601bf970
chore: delete bad parse_bridge function clause
2022-09-23 11:53:02 +02:00
Kjell Winblad
adc67b165b
test: test cases for Kafka bridge REST API
2022-09-23 10:09:07 +02:00
JimMoen
6788881642
fix: hide influxdb udp config
2022-09-20 14:46:59 +08:00
Xinyu Liu
cc327629c3
Merge pull request #8992 from terry-xiaoyu/fast_resume_from_inflight_full
...
Fast resume from inflight full
2022-09-18 17:49:04 +08:00
Shawn
9aa7e826cb
refactor(resource): fast resume resource worker if inflight msgs are ACKed
2022-09-17 00:34:30 +08:00
JimMoen
0336b935a0
Merge pull request #8901 from JimMoen/fix-bridge-api
...
- for connectors using ecpool, parse connect error `Reason` when start pool failed.
- Refine InfluxDB bridge desc.
2022-09-16 10:22:06 +08:00
Shawn
d5d3972ff5
chore: add test cases for MQTT Bridge reconnecting
2022-09-15 10:19:33 +08:00
JimMoen
0390a5e547
fix(bridge): mysql bridge error case
2022-09-15 09:36:52 +08:00
Shawn
f41adb0997
refactor: change some default values of resource_opts
2022-09-14 15:18:07 +08:00
JimMoen
f018b8ab49
fix(bridge/authz/exhook): no need to drop invalid certs
2022-09-14 14:08:55 +08:00
Zaiming (Stone) Shi
0c1595be02
feat: Add Kafka connector
2022-09-13 19:46:56 +02:00
Shawn
b9ae4ea276
refactor: rename some metrics for emqx_resource
2022-09-13 14:04:25 +08:00
Zaiming (Stone) Shi
81ec4c4403
Merge pull request #8906 from zmstone/merge-master-to-ee50-a
...
Merge master to ee50
2022-09-07 11:19:41 +02:00
Zaiming (Stone) Shi
befc4acced
Merge remote-tracking branch 'origin/master' into merge-master-to-ee50-a
2022-09-06 20:31:38 +02:00
Shawn
60a90858f8
feat: check dependent actions before removing the bridges
2022-09-06 15:34:23 +08:00
Xinyu Liu
23b6ff399d
Merge pull request #8844 from thalesmg/mongodb-bridge
...
feat: add mongodb bridge (e5.0)
2022-09-03 12:27:01 +08:00
Xinyu Liu
7fed4faac3
Merge pull request #8859 from terry-xiaoyu/refactor_counters_for_resources
...
Some improvements and bug fixes to emqx_resources
2022-09-03 12:16:49 +08:00
JimMoen
d533e35df0
chore: update app vsn
2022-09-02 22:26:21 +08:00
JimMoen
2693fc263d
fix(bridge): show file paths for SSL certs in dashboard for bridge
2022-09-02 14:12:03 +08:00
Shawn
83f21b4c65
refactor(resource): remove metrics 'sent.exception'
2022-09-02 12:46:53 +08:00
Shawn
b45f3de8db
refactor(resource): rename metrics batched,queued -> batching,queuing
2022-09-02 12:41:14 +08:00
Thales Macedo Garitezi
3d4afd65df
feat: add mongodb bridge (e5.0)
2022-09-01 14:47:14 -03:00
Zaiming (Stone) Shi
8a5a9ca2bf
Merge pull request #8854 from zmstone/0831-chore-merge-main-to-dev-ee50
...
0831 chore merge main to dev ee50
2022-09-01 11:35:23 +01:00
JianBo He
c1afb34a86
test: fix failed tests
2022-09-01 11:34:31 +08:00
JianBo He
a5ac5b6f3a
Merge branch 'dev/ee5.0' into refactor-mqtt-bridge
2022-09-01 09:30:37 +08:00
Zaiming (Stone) Shi
dac178cbaf
chore: ensure version bumps
2022-08-31 17:23:47 +02:00
Shawn
ba1f5eecd3
fix: update the swagger for new resource metrics
2022-08-31 11:14:36 +08:00
Shawn
73e19d84ee
feat: use the new metrics to bridge APIs
2022-08-30 23:47:58 +08:00
Shawn
65dfa63324
fix: update the counters for data bridges
2022-08-30 12:28:01 +08:00
JianBo He
a6eff81163
chore: update emqtt to 1.7.0-rc.1
2022-08-30 09:50:36 +08:00
JianBo He
0aa10702db
feat(bridge): support async mode resource options
2022-08-30 09:34:56 +08:00
JimMoen
ca6533395c
feat: influxdb bridge structure fits new style
2022-08-24 09:59:22 +08:00
JimMoen
43c964c87e
fix(docs): ee bridge api docs generation
...
Use try catch. Because function was unexported before called.
2022-08-24 09:52:38 +08:00
Shawn
45352206a3
fix(auth): remove emqx_connector from testcases of authz/authn
2022-08-22 20:20:45 +08:00
Shawn
55c18c0b5f
fix(bridges): update the test cases for new config structure
2022-08-22 18:24:59 +08:00
Shawn
aea8c77b49
refactor: new config structure for mqtt bridge
2022-08-22 18:24:59 +08:00
Shawn
9e35032d78
fix: make resume_interval defaults to health_check_interval
2022-08-16 10:09:02 +08:00
Xinyu Liu
2898966439
Merge branch 'dev/ee5.0' into resource_opts
2022-08-15 21:43:22 +08:00
Shawn
19d85d485b
refactor(resource): add resource_opts level into config structure
2022-08-15 21:40:10 +08:00
Shawn
b3162fe5ff
fix: conflicts between master and ee5.0
2022-08-14 22:24:09 +08:00
Shawn
3cb79e811f
fix: remove the extra fields max_retries
2022-08-12 18:05:03 +08:00
Shawn
0cdf4b47f1
feat: add more resource creation opts
2022-08-12 13:47:45 +08:00
Xinyu Liu
a05cc20db7
Merge pull request #8693 from JimMoen/feat-influxdb-batch-query
...
Feat influxdb batch query
2022-08-12 00:22:23 +08:00
JimMoen
83746daad5
fix: update bridge config badmap error
2022-08-11 23:15:53 +08:00
JimMoen
458dab53c5
fix: undefined_functions dialyzer warning
2022-08-11 22:55:22 +08:00
JimMoen
ed796acb95
fix: fetch resource options after reboot
2022-08-11 22:11:28 +08:00
Shawn
88388b0c54
feat: http connector support async sending
2022-08-11 21:44:22 +08:00
JimMoen
3a76a50382
fix: syntax error and compile error
2022-08-11 20:58:43 +08:00
Shawn
2872f0b668
fix(bridges): support create resources with options
2022-08-11 19:11:44 +08:00
Shawn
6203a01320
feat: add inflight window to emqx_resource
2022-08-11 08:36:35 +08:00
Shawn
0377d3cf61
fix: update existing testcases for new emqx_resource
2022-08-10 00:34:35 +08:00
JimMoen
2e336fbc01
fix: bridge and connector namespace contains DB name
...
Contains Detailed DB name for identifier hocon schema refs.
2022-08-07 19:37:44 +08:00
DDDHuang
55b9684541
fix: influxdb bridge api available
2022-08-01 10:28:07 +08:00
DDDHuang
3fe76446bc
Merge branch 'master' into dev/ee5.0
2022-08-01 10:14:21 +08:00
DDDHuang
a4992ef1b5
fix: hstreamdb connector conf & api
2022-07-28 16:47:30 +08:00
Shawn
23eeb21b13
fix(webhook): the 'max_retries' opt means request retry times
2022-07-27 17:06:14 +08:00
DDDHuang
9ae7c62656
fix: exs deps & bad suites
2022-07-27 16:24:13 +08:00
DDDHuang
be6de4aad0
fix: ee bridge & connector remove app mod
2022-07-27 11:38:45 +08:00
DDDHuang
fa4bc921ac
fix: hstream db connector & bridge, TODO: SUITE
2022-07-27 11:38:45 +08:00
DDDHuang
98b36c4681
fix: hstream db connector , TODO: start apps
2022-07-27 11:38:45 +08:00
DDDHuang
f00a7417bb
feat: hstream bridge & connector
2022-07-27 11:38:45 +08:00
Shawn
6685a3c5a8
fix: remove the 'headers' field from the rule events
2022-06-17 15:57:10 +08:00
Shawn
d6ef2f7502
refactor: graceful recreate resources
2022-06-17 05:29:18 +08:00
Xinyu Liu
3b00b16abe
Merge pull request #8221 from terry-xiaoyu/fix_delayed_module_disbled_after_emqx_stop
...
feat: fix the hook priorities
2022-06-16 09:42:42 +08:00
Zaiming (Stone) Shi
738a551550
refactor: best-effort json for hocon validation errors
2022-06-15 15:01:15 +02:00
Shawn
39b1b20506
feat: fix the hook priorities
2022-06-15 19:03:40 +08:00
Shawn
dd099137f0
fix: catch errors when load bridges
2022-06-13 22:31:58 +08:00
Shawn
9a0fdc4b11
feat: support configure connector configs directly in bridges
2022-06-10 23:06:20 +08:00
Shawn
69fba6958b
refactor: remove the emqx_bridge_monitor module
2022-06-01 15:31:48 +08:00
Shawn
88ca25c60c
fix(resource): fast return when starting a unavailable resource
2022-06-01 08:24:53 +08:00
JianBo He
efc0ca2b62
Merge branch 'master' into changes-enable_pipelining-type
2022-05-31 20:49:10 +08:00
JianBo He
f0ddac63cb
chore: change the enable_pipelining default value
2022-05-31 19:32:53 +08:00
Zhongwen Deng
d61b44ef97
fix: ct failed again
2022-05-31 19:20:27 +08:00
Zhongwen Deng
8aa60cc0a5
feat: generate a minimized emqx.conf
2022-05-31 19:20:27 +08:00
Shawn
1054c364ad
refactor(resource): improve health check and alarm it if resource down
2022-05-31 01:40:40 +08:00
Shawn
a157539710
refactor(rule): rename http_bridge -> webhook
2022-05-25 18:56:12 +08:00
Shawn
71de9616d3
refactor(rule): rename outputs -> actions
2022-05-25 17:02:37 +08:00
Shawn
d89d692d3e
refactor: move some APIs to new module emqx_bridge_resource
2022-05-18 15:12:34 +08:00
DDDHuang
8b0adf69d2
Merge pull request #7926 from DDDHuang/rule_api_errormsg
...
fix: better error message for rule engine
2022-05-12 19:07:47 +08:00
DDDHuang
2de69c97ba
fix: ruleengine & connector & bridge api, better error message
2022-05-12 16:51:36 +08:00
Thales Macedo Garitezi
47ed9cbc76
fix(telemetry): count bridges referenced in `from` rule clauses
2022-05-11 15:15:17 -03:00
Chris Hicks
841acb7828
Merge pull request #7783 from emqx/EMQX-4199-introduce-a-new-emqx-resource-manager-module
...
feat: isolate resource manager processes
2022-05-09 18:33:01 +02:00
Chris
0b3e30e813
feat: isolate resource manager processes
2022-05-09 13:24:34 +02:00
Shawn
9a033af2e8
fix: restart bridges via HTTP APIs to wrong node
2022-05-05 18:47:19 +08:00
Shawn
1ce8a8c886
fix: testing connector removes ssl cert files that are in use
2022-05-05 14:55:40 +08:00
zhongwencool
c7241f2abc
Merge pull request #7813 from DDDHuang/bridge_api_bad_format
...
fix: bridge api response bad node name format
2022-04-29 09:12:22 +08:00
Zhongwen Deng
03d2a1ebb6
chore: reformat emqx_bridge_api
2022-04-29 09:08:45 +08:00
Shawn
6edb453ce9
fix: the field 'enable' is missing for GET /bridges/:id
2022-04-28 18:13:19 +08:00
Shawn
e18c2a6de2
fix: convert ssl certfiles for dry-run creating bridges
2022-04-28 16:17:55 +08:00
DDDHuang
a26b0f79f2
fix: bridge api response bad node name format
2022-04-28 16:06:57 +08:00
Zaiming (Stone) Shi
02c3f87b31
style: reformat all remaining apps
2022-04-27 15:51:18 +02:00
Shawn
94e24c2621
refactor: move ssl file handling from resources to bridges
2022-04-27 11:59:15 +08:00
Thales Macedo Garitezi
f8f97d39d3
Merge pull request #7766 from thalesmg/bugfix-telemetry-bridge
...
fix: avoid crashing telemetry if bridge app is not ready
2022-04-26 09:03:29 -03:00
Thales Macedo Garitezi
ada3b03f6e
fix(telemetry): do not crash if bridge app is not ready yet
2022-04-25 11:23:13 -03:00
EMQ-YangM
eb3870b538
fix: when enable is false, restart and stop cannot be performed.
2022-04-25 18:48:12 +08:00
JimMoen
e9843aa225
fix(doc): html line break use `</br>`
2022-04-24 18:56:21 +08:00
Zhongwen Deng
5223c3ee61
feat: obfuscate sensitive values default_password
2022-04-24 09:41:29 +08:00
Yang Miao
4a43a10296
Merge pull request #7701 from EMQ-YangM/improve_docs
...
fix: improve document quality
2022-04-22 16:52:09 +08:00
EMQ-YangM
f00854aa9d
fix(emqx_bridge): add required fields
2022-04-22 14:22:22 +08:00
EMQ-YangM
739fe9b71e
feat: emqx_bridge i18n support
2022-04-22 10:53:06 +08:00
EMQ-YangM
5d9df74be7
test: fix required fields
2022-04-21 18:47:02 +08:00
Shawn
19630e9a99
feat: save ssl cert files for data bridges
2022-04-21 09:00:06 +08:00
Shawn
5f74db6d88
fix: treat all returns except {error, Reason} as ok
2022-04-18 22:24:39 +08:00
EMQ-YangM
a7ee4f9fea
fix: fix bridge resource Id
2022-04-12 14:10:35 +08:00
EMQ-YangM
5aeb3c2dc2
feat: impl bridge reset_metrics api
2022-04-11 10:25:48 +08:00
Thales Macedo Garitezi
2a7b777a2a
fix(test): fix inter-suite test coupling
2022-04-07 12:21:15 -03:00
Thales Macedo Garitezi
911e02f626
feat(bridge): export basic usage info for telemetry
2022-04-06 10:20:23 -03:00
ieQu1
99b3f2f83e
docs(schema): Document missing records and fields
2022-04-04 23:58:43 +02:00
Ilya Averyanov
0728b1e3f4
chore(emqx_modules): add emqx_topic_metrics_api tests
2022-03-30 21:18:27 +03:00
Zaiming (Stone) Shi
5458490bc3
chore: add namespace to bridge schema
2022-03-23 21:12:55 +01:00
DDDHuang
74e6fa5a94
fix: generate bridge api response code
...
fix: generate connector api response code SUITE
fix: generate exhook api response code
2022-03-21 14:12:19 +08:00
Shawn
1d023b541f
refactor(connnector): rename waiting_connect_complete -> wait_for_resource_ready
...
Rename the option to wait_for_resource_ready and defaults to 5s.
2022-03-10 10:46:57 +08:00
Shawn
50d0a3271b
fix(test): add testcases for enable/disbale bridges
2022-03-09 18:49:30 +08:00