yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,612 Commits 393 Branches 669 Tags 74 MiB
cc5b4d3748
Commit Graph

370 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi
dd996ad1dc chore: bump app vsns 2023-03-24 21:47:15 +01:00
Stefan Strigler
8d084ed13b
Merge pull request #10226 from sstrigler/EMQX-9273-e-5-0-2-creating-a-data-bridge-the-name-verification-returned-an-incorrect-error-code-of-500 
fix(emqx_bridge): don't crash on validation error
 2023-03-24 14:48:03 +01:00
Stefan Strigler
fbfdaf44e0 fix(emqx_bridge): don't crash on validation error 2023-03-24 11:46:41 +01:00
Stefan Strigler
a8cfc1c7a1 fix(emqx_bridge): filter status_reason in config update 2023-03-24 11:46:41 +01:00
Stefan Strigler
ba49316ec5 fix: use lower-case 2023-03-23 16:08:27 +01:00
Kjell Winblad
35474578ca refactor: rename async_inflight_window to inflight_window everywhere 2023-03-23 14:21:57 +01:00
JianBo He
8cbbc9f271 Merge remote-tracking branch 'upstream/master' into cassa 2023-03-23 11:53:17 +08:00
lafirest
84def357a9
Merge pull request #10143 from lafirest/feat/rocketmq 
feat(bridges): integrate RocketMQ into data bridges
 2023-03-22 20:43:22 +08:00
Zaiming (Stone) Shi
e43aa5d841
Merge pull request #10199 from zmstone/0321-merge-release-50-to-master 
0321 merge release 50 to master
 2023-03-22 10:20:48 +01:00
Andrew Mayorov
af7b2215ca
fix(bridge-api): unflatten nodes' bridge lists back 
Bridge lists were erroneously flattened in cad6492c. This causes
bridge listing fail in emqx clusters consisting of more than 1 node.
 2023-03-22 11:35:09 +03:00
firest
9853d00cad feat(bridges): integrate RocketMQ into data bridges 2023-03-22 10:36:52 +08:00
JianBo He
65c2da7ef5 Merge remote-tracking branch 'ce/master' into cassa 2023-03-22 09:30:50 +08:00
Andrew Mayorov
bf35419b52
Merge pull request #10190 from keynslug/fix/EMQX-9269/bridge-list 
fix(bridge-api): unflatten nodes' bridge lists back
 2023-03-22 00:53:03 +03:00
Zaiming (Stone) Shi
e6091db351 Merge remote-tracking branch 'origin/release-50' into 0321-merge-release-50-to-master 2023-03-21 22:03:31 +01:00
Stefan Strigler
6c24b3a77d
Merge pull request #10196 from sstrigler/EMQX-9201-the-case-is-not-correct-for-some-of-menu 
Emqx 9201 the case is not correct for some of menu
 2023-03-21 19:42:10 +01:00
Andrew Mayorov
d9b1b1c802
fix(bridge-api): unflatten nodes' bridge lists back 
Bridge lists were erroneously flattened in cad6492c. This causes
bridge listing fail in emqx clusters consisting of more than 1 node.
 2023-03-21 18:59:27 +03:00
Stefan Strigler
981268911a style: make summary lower-case 2023-03-21 16:56:37 +01:00
Stefan Strigler
3880862c81 fix(emqx_bridge): return 503 for inconsistency in bridge setup 2023-03-21 15:07:43 +01:00
Stefan Strigler
4b0ea562a2 refactor(emqx_bridge): consistently use macros for http response 2023-03-21 15:07:41 +01:00
Stefan Strigler
8af3fb4ee7 feat: move human readable error translations to emqx_misc 2023-03-21 15:02:29 +01:00
Stefan Strigler
53825b9aba fix(emqx_bridge): propagate connection error to resource status 2023-03-21 15:02:29 +01:00
Erik Timan
2d75c7d6d9 fix(emqx_bridge): remove metrics from non-dedicated bridge API endpoints 
Metrics should only be exposed via the /bridges/:id/metrics endpoint,
and not in other operations such as getting the list of all bridges, or
in the response when a bridge has been created. This commit removes all
traces of metrics for the non-dedicated API endpoints.
 2023-03-20 09:43:11 +01:00
JianBo He
12942b676d Merge remote-tracking branch 'upstream/master' into cassa 2023-03-20 09:50:27 +08:00
JianBo He
c0a216a740 feat(bridge): support cassandra bridge 2023-03-17 11:34:48 +08:00
Thales Macedo Garitezi
3954b7bde2 fix(bridges): function clause when a non-ingress bridge coexists with an egress bridge 
This was not caught by our tests because we always test bridge types
in isolation.  So, if the config only contains ingress-only bridges,
the `on_message_publish` hook is never installed.

In a real system, if there are bridges of mixed types in the config,
the hook might be installed, and `emqx_bridge:get_matched_bridge_id`
would crash when iterating over the ingress bridges.
 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
f31f15e44e chore(kafka_producer): make schema changes more backwards compatible 
This will still require fixes to the frontend.
 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
03342923b9 fix(bridge): use the same dry run prefix 
Kafka Producer and Consumer bridges rely on this prefix for detecting
a dry run and avoid leaking atoms.  At some point, this prefix was
changed, effectively disabling the check in Kafka Producer.
 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
1f31a87974 fix(bridge): improve macro var usage 
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
969fa03ecc feat: implement kafka consumer 2023-03-16 13:43:01 -03:00
Andrew Mayorov
816667d85c
Merge pull request #10123 from fix/EMQX-9136/lookup-busy-resources 
perf(bridge-api): ask nodes' bridge listings in parallel
 2023-03-16 00:07:12 +03:00
Andrew Mayorov
cad6492c99
perf(bridge-api): ask bridge listings in parallel 
Also rename response formatting functions to better clarify their
purpose.
 2023-03-15 19:17:29 +03:00
Andrew Mayorov
53bc27e0f4
refactor(bridge): avoid unnecessary maps:to_list/1 when listing 2023-03-13 14:49:38 +03:00
Stefan Strigler
478601ee41 fix(emqx_bridge): remove unreachable code paths 2023-03-13 10:40:47 +01:00
Stefan Strigler
dba95ec0fa style(emqx_bridge): fix wording 2023-03-13 10:40:47 +01:00
Stefan Strigler
71ec77a2f2 fix(emqx_bridge): for node operation check if node is part of cluster 
This fixes the case where we returned `501 NOT IMPLEMENTED` in the past.
 2023-03-13 10:40:47 +01:00
Stefan Strigler
7124600a71 docs(emqx_bridge): add start as operation to bridge 2023-03-13 10:38:51 +01:00
Stefan Strigler
1bcc5623ed fix(emqx_bridge): check if bridge enabled before calling op 2023-03-13 10:38:49 +01:00
Stefan Strigler
a325133391 fix(emqx_bridge): don't crash checking if bridge enabled 2023-03-13 10:37:28 +01:00
Stefan Strigler
80b81748df fix(emqx_bridge): handle bridge not found in call_operation 2023-03-13 10:37:28 +01:00
Stefan Strigler
fb3d101b3a refactor(emqx_bridge): fix var names 2023-03-13 10:37:28 +01:00
Stefan Strigler
e31f4d6091 refactor(emqx_bridge): add BAD_REQUEST macro and minor cleanups 2023-03-13 10:37:28 +01:00
ieQu1
e3595f2e79 chore(mria): Bump version to 0.4.0 2023-03-11 00:37:25 +01:00
Andrew Mayorov
a86d06f043
chore: bump app versions following last merge-back 2023-03-10 16:44:15 +03:00
Zaiming (Stone) Shi
fe27604010 Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master 2023-03-08 16:46:45 +01:00
firest
b8b1a78025 feat(bridges): integrate the DynamoDB into data bridges 2023-03-08 11:13:38 +08:00
Thales Macedo Garitezi
e17ad320ee fix(bridge): do not log in converter 2023-03-06 15:31:28 -03:00
Stefan Strigler
a81d1a4618
Merge pull request #10066 from sstrigler/EMQX-8932-bridge-api-reports-500-for-most-errors-when-starting-a-resource 
fix: return human readable error message for most common cases
 2023-03-06 14:13:34 +01:00
Thales Macedo Garitezi
f95a30ae89 fix(webhook): convert request_timeouts in root and resource_opts 2023-03-06 10:12:38 -03:00
Kjell Winblad
a638cc6566
Merge pull request #9985 from kjellwinblad/kjell/feat/clickhouse_bridge_2/EMQX-8391 
feat: add clickhouse database bridge
 2023-03-04 07:05:05 +01:00
Stefan Strigler
570bf165af fix: return human readable error message for most common cases 2023-03-03 16:49:09 +01:00
Ivan Dyachkov
c01f62a1c1 chore: bump apps versions 2023-03-03 13:58:32 +01:00
Kjell Winblad
67acdf0888 feat: add clickhouse database bridge 
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
 2023-03-02 12:22:11 +01:00
Stefan Strigler
39e1cf9502 fix(emqx_bridge): let it crash instead of 400 on failed reset 2023-03-02 11:23:07 +01:00
Stefan Strigler
4c23ab097d fix(emqx_bridge): return 400 if operation not possible 2023-03-02 11:23:07 +01:00
Stefan Strigler
680a4362d1 fix: match macro variables 2023-03-02 10:12:56 +01:00
Stefan Strigler
76f97010bb style: change wording 2023-03-02 09:14:52 +01:00
Serge Tupchii
77dea0c77a fix: Handle obfuscated fields in bridges_probe API 2023-03-01 22:05:23 +02:00
Stefan Strigler
3f324f85bc fix: consistently return 404 in case bridge is not found or invalid 
Also: fix some typos and be more verbose
 2023-03-01 18:03:49 +01:00
Zaiming (Stone) Shi
56279f6305 docs: fix bridge status report 2023-02-28 11:56:46 +01:00
Andrew Mayorov
7c573d9365
fix(bridgeapi): anticipate node responses w/o metrics 
When some resource manager is busy with trying to estabilish a
connection with remote, we hit the "read-from-cache" codepath so the
resource data will not contain any metrics.
 2023-02-27 14:55:41 +03:00
Stefan Strigler
e196d6f42a fix(emqx_bridge): unwrap rpc multicall result 2023-02-10 12:00:20 +01:00
Stefan Strigler
f89ac54b17 style: remove stale comment 2023-02-08 14:04:09 +01:00
Stefan Strigler
a3fd0897bc refactor: less code duplication 2023-02-08 14:04:09 +01:00
Stefan Strigler
502f62e18d fix: try old 'restart' behavior if 'start' is not implemented 2023-02-08 13:19:41 +01:00
Stefan Strigler
86f3f5787f feat: allow to manually re-connect disconected bridge 2023-02-07 11:58:30 +01:00
Stefan Strigler
4e12a44ee6 fix: return 204 instead of 200 if there's no body content 2023-02-06 09:47:07 +01:00
Zaiming (Stone) Shi
85fb53dcf0 refactor: use compile switch to diff functions between ce and ee 2023-02-02 15:15:31 +01:00
Andrew Mayorov
c5a7cd5acd
fix(mqtt-bridge): drop unused configuration parameter 2023-01-31 18:30:42 +03:00
Zaiming (Stone) Shi
7f66c6a9e2
Merge pull request #9840 from olcai/redact-influxdb-tokens 
fix: redact influxdb tokens in logs and reduce log level
 2023-01-28 11:47:36 +01:00
Zaiming (Stone) Shi
c47be57c59 fix(bridge): ensure all bridge resources are stopped before app stop 2023-01-27 12:39:05 +01:00
Erik Timan
8836494542 fix: redact influxdb tokens in a few logs 2023-01-25 14:48:32 +01:00
Zaiming (Stone) Shi
5fdf7fd24c fix(kafka): use async callback to bump success counters 
some telemetry events from wolff are discarded:

* dropped:
    this is double counted in wolff,
    we now only subscribe to the dropped_queue_full event
* retried_failed:
    it has different meanings in wolff,
    in wolff, it means it's the 2nd (or onward) produce attempt
    in EMQX, it means it's eventually failed after some retries

* retried_success
    since we are going to handle the success counters in callbac
    this having this reported from wolff will only make things
    harder to understand

* failed
    wolff never fails (unelss drop which is a different counter)
 2023-01-24 21:12:36 +01:00
Erik Timan
28718edbfd chore: bump application VSNs 2023-01-24 14:12:34 +01:00
Thales Macedo Garitezi
6fa6c679bb feat(buffer_worker): add expiration time to requests 
With this, we avoid performing work or replying to callers that are no
longer waiting on a result.

Also introduces two new counters:

- `dropped.expired` :: happens when a request expires before being
  sent downstream
- `late_reply` :: when a response is receive from downstream, but the
  caller is no longer for a reply because the request has expired, and
  the caller might even have retried it.
 2023-01-20 11:36:52 -03:00
Thales Macedo Garitezi
47f796dd12 refactor: rename emqx_resource_worker -> emqx_resource_buffer_worker 
To make it more clear that it's purpose is serve as a buffering layer.
 2023-01-18 16:15:34 -03:00
Zaiming (Stone) Shi
d4f3b4c8c2 Merge remote-tracking branch 'origin/master' into fix-buffer-clear-replayq-on-delete-v50 2023-01-18 11:39:47 +01:00
Ivan Dyachkov
430b0a03d4
Merge pull request #9780 from id/fix-ensure-no-colon-in-filenames 
fix: ensure no colon in filenames
 2023-01-18 09:36:16 +01:00
Stefan Strigler
f899284e3a
Merge pull request #9789 from sstrigler/EMQX-8754-test-function-return-500-of-data-integration-google-pubsub 
EMQX 8754 test function return 500 of data integration google pubsub
 2023-01-17 22:49:28 +01:00
Thales Macedo Garitezi
fa01deb3eb chore: retry as much as possible, don't reply to caller too soon 2023-01-17 16:49:15 -03:00
Stefan Strigler
d164e5bc0f fix: don't require ssl conf 2023-01-17 15:29:19 +01:00
firest
c3d5c25c26 fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12 2023-01-17 17:42:10 +08:00
Ivan Dyachkov
676f017ec0 fix: ensure no colon in filenames 2023-01-16 21:27:01 +01:00
Stefan Strigler
e08c1d2229 Merge remote-tracking branch 'olcai/refactor-bridges-api' into dev/api-refactor 2023-01-13 15:49:52 +01:00
Stefan Strigler
1690a6dcfc
Merge branch 'master' into dev/api-refactor 2023-01-13 15:34:13 +01:00
Erik Timan
c3133fb6a2 fix(emqx_bridge): small fixes from review 2023-01-13 14:54:54 +01:00
Zhongwen Deng
f15b29b1ef chore: upgrade app version 2023-01-13 18:22:29 +08:00
Erik Timan
42f42de4d9 feat(emqx_bridge): add separate endpoint for enable/disable of bridge 
In order to improve the consistency with other API endpoints, we move
the enable/disable operations to a separate endpoint
/bridges/{id}/enable/[true,false].
 2023-01-12 19:39:43 +01:00
Erik Timan
860e21d40f feat(emqx_bridge): move metrics to own endpoint, rename reset_metrics 
In order for the /bridges APIs to be consistent with other APIs, we move
out metrics from GET /bridges/{id} to its own endpoint,
/bridges/{id}/metrics. We also rename /bridges/reset_metrics to
/bridges/metrics/reset.
 2023-01-12 19:39:41 +01:00
Erik Timan
c2fd1a4482 feat(emqx_bridge): shorten operation endpoint URLs 
This shortens and simplifies URLs for performing bridge operations so
that the API looks more congruent.
 2023-01-12 19:36:22 +01:00
Stefan Strigler
8ad8288195 feat: report error in create_dry_run 2023-01-11 14:22:37 +01:00
Thales Macedo Garitezi
48e1ba4832 feat(docs): add tags to schemas 
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
 2023-01-11 09:10:03 -03:00
zhongwencool
6f0e228a9b
Merge pull request #9717 from zmstone/0110-do-not-start-resource-after-recreate-if-disabled 
fix(emqx_resource_manager): do not start when disabled
 2023-01-11 16:12:53 +08:00
Zaiming (Stone) Shi
85a8eff90b fix(emqx_resource_manager): do not start when disabled 2023-01-11 08:33:48 +01:00
Zaiming (Stone) Shi
ee89de816c fix: upgrade bridge config converter 2023-01-10 20:42:30 +01:00
Xinyu Liu
0f70786d13
Merge pull request #9705 from terry-xiaoyu/remove-default-value-of-webhook-body 
fix: remove the default value of webhook body field
 2023-01-10 18:23:50 +08:00
Shawn
724015af47 fix: remove the default value of webhook body field 2023-01-10 09:45:25 +08:00
Kjell Winblad
692c1f0d1b
Merge pull request #9678 from kjellwinblad/kjell/fix/jira/EMQX-8648 
fix: no feedback when deleting non-existing configuration
 2023-01-09 13:52:40 +01:00
Stefan Strigler
c85a988a43 fix: split 'server' param into host and port for mqtt bridge 2023-01-09 13:27:42 +01:00
Stefan Strigler
c42c99f94f fix: set a timeout for tcp connect 2023-01-09 10:01:35 +01:00
Stefan Strigler
96ca0d9f49 feat(emqx_bridge): add /bridges_probe API endpoint 2023-01-09 10:01:33 +01:00