Commit Graph

649 Commits

Author SHA1 Message Date
Andrew Mayorov fe450ca2d9
Merge pull request #9938 from keynslug/feat/mqtt-bridge-async-errors
feat(mqtt-bridge): report recoverable errors of async queries
2023-02-10 20:40:58 +04:00
Andrew Mayorov cbb2885499
fix(mqtt-bridge): allow to configure `clean_start` for ingresses 2023-02-10 19:15:48 +03:00
Andrew Mayorov 7002fe2ef4
fix(mqtt-bridge): disallow QoS 2 on ingress bridges 2023-02-10 17:17:59 +03:00
Andrew Mayorov 8bd7c0eb3f
feat(mqtt-bridge): report recoverable errors of async queries
This should help to avoid delivery failures of messages which could
be safely retried, in the event of intermittent connectivity loss
for example.

It should now be safe since 73d5592b.
2023-02-10 12:26:59 +03:00
Zaiming (Stone) Shi 5328955ff4 fix(config): add back reconnect_interval field as deprecated 2023-02-09 17:30:09 +01:00
Zaiming (Stone) Shi 1c748070c1 fix(emqx_connector_mqtt_worker): publish_async can only return ok # Please enter the commit message for your changes. Lines starting 2023-02-02 14:11:12 +01:00
Zaiming (Stone) Shi 2b308af354
Merge pull request #9889 from keynslug/fix/EMQX-8857/force-clean-start-mqtt-bridge
fix(mqtt-bridge): stop respecting `clean_start` config parameter
2023-02-02 12:55:58 +01:00
Andrew Mayorov 0912f13c1f
fix(mqtt-bridge): stop respecting `clean_start` config parameter
We are ignoring the user configuration because there's currently no
reliable way to ensure proper session recovery according to the MQTT
spec.
2023-02-02 12:55:10 +03:00
Kjell Winblad 2cf193e2fd
Merge pull request #9839 from kjellwinblad/kjell/fix/Authorization_header_log_leak_webhook/EMQX-8791
fix: Authorization header leak in log entries for webhook
2023-02-02 10:38:42 +01:00
Andrew Mayorov ca5c192f4b
Merge pull request #9882 from fwup/fix/no-mqtt-bridge-middleman
refactor(mqtt-worker): avoid unnecessary abstraction
2023-02-02 13:11:31 +04:00
Zaiming (Stone) Shi 32d18ec594
Merge pull request #9878 from zmstone/0201-upgrade-ehttpc-0.4.6
fix(ehttpc): fix crash when webhook bridge uses `hash` pool type.
2023-02-01 15:00:28 +01:00
Andrew Mayorov f0395be383
refactor(mqtt-worker): avoid unnecessary abstraction
So the code is easier to follow.
2023-02-01 14:46:26 +03:00
Zaiming (Stone) Shi 9f6b6cedc6 fix(webhook): pick worker according to pool type 2023-02-01 09:53:25 +01:00
Ilya Averyanov 9d91ebe266
Merge pull request #9842 from savonarola/fix-redis-cluster-recover
fix: fix redis cluster resource recovery
2023-02-01 10:38:52 +02:00
Zaiming (Stone) Shi f4381d90ca refactor: http cliet should not need to retry HTTP requests for bridge 2023-02-01 09:30:30 +01:00
Andrew Mayorov c5a7cd5acd
fix(mqtt-bridge): drop unused configuration parameter 2023-01-31 18:30:42 +03:00
Andrew Mayorov d0c10b59aa
feat(mqtt-bridge): avoid middleman process
Instead, supervise `emqtt` client process directly.
2023-01-31 17:59:03 +03:00
Ilya Averyanov fce1e74c3d fix(connector): fix redis cluster resource recovery 2023-01-31 16:55:05 +02:00
Andrew Mayorov 4d146c521b
fix(mqtt-bridge): ensure proper feedback on async forwards
So that buffer worker would notice a connection loss in time, and
recycle inflight messages subsequently.
2023-01-31 11:09:39 +03:00
Andrew Mayorov 35c429ef1d
refactor: drop a couple of unused macros / includes 2023-01-31 11:09:39 +03:00
Kjell Winblad 0c4134c423 test: add unit test case for redact function in http connector 2023-01-30 11:51:28 +01:00
Andrew Mayorov d35e46b2d5
Merge pull request #9838 from keynslug/fix/redis-cluster-batching
feat(redis): disable batching in redis_cluster bridges
2023-01-27 15:27:57 +04:00
Andrew Mayorov 71f996b9d5
refactor(mqtt-bridge): unwrap single statem actions
So that the code would be easier to follow and harder to break.
Also drop a couple of unused macrodefs.
2023-01-26 21:13:18 +03:00
Andrew Mayorov c95f979413
fix(mqtt_bridge): use correct gen_statem reply action 2023-01-26 21:12:05 +03:00
Kjell Winblad 86cfbfb43c fix: Authorization header leak in log entries for webhook
There might be another possibility for leakage. If the resource mangager
for the webhook resource crashes, OTP might log the spec for the
resource manager which contains the Config and thus the Authorization
header. This is probably an issue for other resources as well and should
be fixed in another commit. The following issue has been created for
that:

https://emqx.atlassian.net/browse/EMQX-8794

Fixes:
https://emqx.atlassian.net/browse/EMQX-8791
2023-01-26 12:44:16 +01:00
Andrew Mayorov 2ee00b75a7
fix(redis): unwrap pipeline queries against redis cluster
This is an additional safety measure in addition to the disabled
batching on the bridge level.
2023-01-25 17:28:11 +03:00
Erik Timan 28718edbfd chore: bump application VSNs 2023-01-24 14:12:34 +01:00
Zaiming (Stone) Shi 8fde169abb
Merge pull request #9821 from thalesmg/buffer-worker-expiry-v50
feat(buffer_worker): add expiration time to requests
2023-01-24 13:54:04 +01:00
Thales Macedo Garitezi ca4a262b75 refactor: re-organize dealing with unrecoverable errors 2023-01-20 12:00:17 -03:00
Thales Macedo Garitezi d755b43c77 fix(jwt_worker): handle exceptions when decoding jwk from pem
Returns a more controlled error if users attempt to use the Service
Account JSON from the GCP PubSub example from swagger, which is
redacted.
2023-01-19 09:24:45 -03:00
Zaiming (Stone) Shi 7abba17b25
Merge pull request #9765 from zmstone/0115-add-password-converter
fix(schema): add password converter to ensure its binary() type
2023-01-18 15:09:05 +01:00
Thales Macedo Garitezi 087b667263 fix(buffer_worker): allow signalling unrecoverable errors 2023-01-17 19:50:30 -03:00
Thales Macedo Garitezi 006b4bda97 feat(buffer_worker): monitor async workers and cancel their inflight requests upon death 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi 4a7e74f5d6 fix(schema): add password converter to ensure its binary() type 2023-01-17 11:12:17 +01:00
Stefan Strigler 9a53410efb fix: don't add empty list element to options proplist 2023-01-16 12:00:03 +01:00
Stefan Strigler edfcea7f43 fix: add log for timeout 2023-01-16 12:00:03 +01:00
JimMoen 54ebc27d24
Merge pull request #9672 from JimMoen/0103-fix-mqtt-bridge
Fix the problem that the bridge is not available when the Payload template is empty in the MQTT bridge.
2023-01-16 09:57:20 +08:00
Stefan Strigler 1690a6dcfc
Merge branch 'master' into dev/api-refactor 2023-01-13 15:34:13 +01:00
Zhongwen Deng f15b29b1ef chore: upgrade app version 2023-01-13 18:22:29 +08:00
JimMoen 59ac0b1424
fix(mqtt-bridge): transmit raw msg payload with empty template 2023-01-13 17:23:04 +08:00
Stefan Strigler 2a81fa1522 fix: remove redundant log message 2023-01-12 14:29:11 +01:00
Stefan Strigler 04f46f5227 feat: make http connector report errors 2023-01-12 14:26:36 +01:00
Stefan Strigler 77f043dedf fix: don't require username and password
not enforced by schema so we shouldn't either
2023-01-12 14:26:36 +01:00
Kjell Winblad 82e80e5610 fix: remove password from info logs
Information log messages for several connectors contained sensitive
password information. This commit fixes this by hiding the sensitive
information from the log messages.

Fixes: https://emqx.atlassian.net/browse/EMQX-8718
2023-01-12 09:28:01 +01:00
Shawn 3e9c4f444f refactor: remove the auto_reconnect field 2023-01-11 21:47:06 +08:00
Xinyu Liu 0f70786d13
Merge pull request #9705 from terry-xiaoyu/remove-default-value-of-webhook-body
fix: remove the default value of webhook body field
2023-01-10 18:23:50 +08:00
Shawn 724015af47 fix: remove the default value of webhook body field 2023-01-10 09:45:25 +08:00
Zaiming (Stone) Shi 0697c692ed refactor: mongo_type and redis_type are not mandatory
the connector schemas are shared between authn, auth and bridges,
the difference is that authn and authz configs are unions like:

[
    {mongo_type = rs,
     ...
    }
    {another backend config}
]

however the brdige types are maps like, for example:

mongodb_rs.$name {
    mongo_type = rs
    ...
}

in which case, the mongo_type is not required.

in order to keep the schema static as much as possible,
this field is chanegd to 'required => false' with a default value.

However, for authn and authz, the union selector will still raise
exception if the there is no type provided.
2023-01-09 14:26:16 +01:00
Zaiming (Stone) Shi e52f9d5920 refactor: use union member type selector for authz sources 2023-01-09 14:26:16 +01:00
firest f7b50c56da feat(bridges): integrate PostgreSQL into bridges
support both simple and batch query
2023-01-06 11:47:22 +08:00
Thales Macedo Garitezi 00b91d8a60
Merge pull request #9664 from thalesmg/fix-mongo-duration-field-descs-v50
docs(mongodb_connector): fix duration field descriptions
2023-01-04 10:03:38 -03:00
Erik Timan f767db4d8f
Merge pull request #9660 from olcai/refactor-ingress-bridge-counter
refactor(emqx_resource): ingress bridge counter
2023-01-04 10:43:07 +01:00
Thales Macedo Garitezi 7bbdc20993 docs(mongodb_connector): fix duration field descriptions
https://emqx.atlassian.net/browse/EMQX-8642

Adds different descriptions for the different fields of type duration
for MongoDB.

Based off: https://www.mongodb.com/docs/manual/reference/connection-string/
2023-01-03 10:01:56 -03:00
Erik Timan b9d012e072 refactor(emqx_resource): ingress bridge counter
Unify code paths for resource metrics by removing
emqx_resource:inc_received/1 and adding
emqx_resource_metrics:received_inc/1 & friends.
2023-01-02 15:11:52 +01:00
firest bc37556963 fix(mysql): fix the problem of data loss and bad match when mysql is disconnected 2023-01-02 21:46:24 +08:00
Zaiming (Stone) Shi dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
Zaiming (Stone) Shi 0ce1ca89b7 refactor: use string type for server and servers 2022-12-30 14:20:23 +01:00
Thales Macedo Garitezi 446a4c74d0 fix(gcp_pubsub): fix potential jwt accumulation and lack of refresh (v5.0)
https://emqx.atlassian.net/browse/EMQX-8653
Related:
- https://emqx.atlassian.net/browse/EEC-737
- https://emqx.atlassian.net/browse/EMQX-8652

Since the rule resource testing mechanism creates a new resource to
test the configuration, a new JWT associated with an unique temporary
resource was being created and left in the JWT table, leaking it.

Also, a wrong case clause when setting the new refresh timer for the
JWT worker was preventing it from refreshing from the 2nd refresh
onward.
2022-12-29 16:30:36 -03:00
zhongwencool 81b226a801
Merge pull request #9608 from zmstone/1123-pin-hocon-0.33.0
1123 pin hocon 0.33.0
2022-12-26 19:22:21 +08:00
Thales Macedo Garitezi 35dc75b7ed feat(mqtt): add option to customize clientid prefix for egress bridges
https://emqx.atlassian.net/browse/EMQX-8445

Currently the bridge client’s client ID is prefixed with the resource
ID.

Sometimes it’s useful for users to have control of this prefix,
e.g. prefix based ACL rules in the target broker.
2022-12-23 09:50:26 -03:00
Ilya Averyanov fa7433628c fix(logs): set password field sensitive for connectors 2022-12-23 12:46:03 +01:00
Zaiming (Stone) Shi 6292d7ec68 refactor: remove dead code 2022-12-23 12:07:36 +01:00
Erik Timan 13942f5c49 refactor: rename error return in mysql connector 2022-12-22 10:29:12 +01:00
Erik Timan 7df24000a0 test: add more EE mysql bridge test cases 2022-12-22 10:27:38 +01:00
Erik Timan 23ac426608 fix(emqx_connector): check for key among prepared statements on query
An infinite loop was triggered in the mysql connector when a query
used a prepared statement key that was not among the defined prepared
statements on start. We now check that the key is defined among the
prepared statements before recursing. It seems that this bug was never
triggered in any production code flow and simply found while writing
tests.

An error return spell fix is also included as well as a FIXME comment
regarding running mysql:prepare and not distinguishing between
transient failures and syntax errors. Syntax errors should not be
retried.
2022-12-22 10:27:38 +01:00
Zaiming (Stone) Shi 516147ad00
Merge pull request #9581 from zmstone/1219-fix-mqtt-bridge-config
1219 fix mqtt bridge config
2022-12-20 14:47:02 +01:00
Zaiming (Stone) Shi da51433dc3 refactor: add default value for eggress remote 'qos' and 'retain'
otherwise when updating from dashboard, there is no way to set
'false' for 'retain' because it's a checkbox,
it's either 'true' or 'undefined'
2022-12-20 00:00:31 +01:00
Zaiming (Stone) Shi 29f394aa70 fix(shared): handle unsubscribe for sticky strategy
prior to this change, the message is dispatched to a shared subscriber
even after unsubscribed
2022-12-19 19:12:38 +01:00
Andrew Mayorov ed2d5aa48a
fix: handle status in `format_status/1` callback
Which is expected argument type for this callback. Also try to
make sure that random maps won't pass through this callback unnoticed.
2022-12-16 13:45:06 +03:00
Zaiming (Stone) Shi 9e3da5b661 chore: bump app versions 2022-12-14 20:07:41 +01:00
Zaiming (Stone) Shi 42c58e2a91 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 15:29:13 +01:00
Thales Macedo Garitezi 1cd91a24e9 feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0) 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi 34e9056779 refactor: fix typo in variable name
Might confuse people to think it's related to `replayq`.
2022-12-12 17:17:51 -03:00
Zaiming (Stone) Shi 40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
Ilya Averyanov 6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi 1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Erik Timan 485c4ddcab
Merge pull request #9393 from olcai/add-mysql-bridge-test
test: add basic mysql bridge test (ee5.0)
2022-12-02 09:43:22 +01:00
Zaiming (Stone) Shi eb017ab034 test: add test case to cover mqtt bridge config upgrade 2022-12-01 16:50:10 +01:00
Erik Timan 5461311edc test: more EE mysql bridge tests 2022-12-01 10:27:51 +01:00
Erik Timan 96bff1d32e test: improve basic tests for EE mysql bridge 2022-12-01 10:27:51 +01:00
Zaiming (Stone) Shi 9a6901987f feat: add mqtt bridge config upgrade converter 2022-11-30 21:01:59 +01:00
Zaiming (Stone) Shi 983e904858 fix(connector): fix ssl clear 2022-11-30 17:03:41 +01:00
firest 81586a4c8d chore: bump connector version 2022-11-30 23:14:54 +08:00
firest 51ace31f30 fix(connector): fix previous incorrectly fixes
There was an error in the try_clear_certs2/3 clause that the argument isn't the ssl field but including it.
2022-11-30 22:14:44 +08:00
Zaiming (Stone) Shi b398617614 chore: bump app versions 2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi 7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
firest fc2793c300 fix: fix upside down function clause 2022-11-24 10:37:52 +08:00
Zaiming (Stone) Shi 67cef65fdb refactor(emqx_connector_ssl): simplify implementation 2022-11-23 23:49:43 +01:00
firest 6dc5078a96 fix: fix obsolete SSL files aren't deleted after the bridge configuration update 2022-11-23 23:49:43 +01:00
kraftwerk28 00c57de4c3 feat: do not drop MQTTv5 properties in rule/bridge 2022-11-22 20:09:59 +01:00
Zaiming (Stone) Shi 09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
firest d0687c7aaa fix(connector): keep connector name type as binary don't convert it to atom
After investigation, it was confirmed that there was no need to convert the connector name to atom
2022-11-01 15:36:31 +08:00
Zaiming (Stone) Shi c157392452 docs: fix self-closing html tag, change </br> to <br/> 2022-10-27 13:57:18 +02:00
JianBo He 834910b6f6 chore: update app.src 2022-09-27 17:24:44 +08:00
JianBo He 39ad172377 chore: replace ip_port() wuth host_port() for all connectors 2022-09-27 15:19:42 +08:00
Zaiming (Stone) Shi 00e4b4da5a fix(schema): support hostname.domain:port for mqtt bridge 2022-09-27 09:02:49 +02:00
Shawn dfe14be8b2 chore: bump app vsns 2022-09-26 18:23:28 +08:00
Shawn 4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
JimMoen a660004fa4 fix(bridge): mqtt bridge client info `username` & `password`
for non configured `username` and `password`, default values `<<>>` should be
used. emqtt needs the values.
2022-09-16 15:09:51 +08:00
Shawn d5d3972ff5 chore: add test cases for MQTT Bridge reconnecting 2022-09-15 10:19:33 +08:00
Shawn 4e211c12d3 fix(mqtt_bridge): return value of sending messages was discarded 2022-09-15 08:57:01 +08:00
Shawn 1c03c236f5 fix(mqtt_bridge): handle send_to_remote in idle state 2022-09-14 15:19:30 +08:00
JimMoen f018b8ab49 fix(bridge/authz/exhook): no need to drop invalid certs 2022-09-14 14:08:55 +08:00
JimMoen b5bf5d3adb fix: mqtt bridge payload default value 2022-09-13 18:58:01 +08:00
Zaiming (Stone) Shi 81ec4c4403
Merge pull request #8906 from zmstone/merge-master-to-ee50-a
Merge master to ee50
2022-09-07 11:19:41 +02:00
Shawn 2b33ca6d49 fix: no error log print if insert bool values into mysql 2022-09-07 16:00:09 +08:00
Zaiming (Stone) Shi befc4acced Merge remote-tracking branch 'origin/master' into merge-master-to-ee50-a 2022-09-06 20:31:38 +02:00
Xinyu Liu 23b6ff399d
Merge pull request #8844 from thalesmg/mongodb-bridge
feat: add mongodb bridge (e5.0)
2022-09-03 12:27:01 +08:00
Xinyu Liu 7fed4faac3
Merge pull request #8859 from terry-xiaoyu/refactor_counters_for_resources
Some improvements and bug fixes to emqx_resources
2022-09-03 12:16:49 +08:00
JimMoen d533e35df0 chore: update app vsn 2022-09-02 22:26:21 +08:00
Shawn 26234d38b9 fix: mark the async msg 'queuing' not 'sent.inflight' on recoverable_error 2022-09-02 18:41:43 +08:00
JimMoen 16618131ad fix(bridge): mqtt source/sink bridge use empty payload template 2022-09-02 14:44:38 +08:00
JimMoen 2693fc263d fix(bridge): show file paths for SSL certs in dashboard for bridge 2022-09-02 14:12:03 +08:00
Shawn b45f3de8db refactor(resource): rename metrics batched,queued -> batching,queuing 2022-09-02 12:41:14 +08:00
JimMoen 74659bb864 fix(bridge): rm mqtt bridge default `username` and `password` 2022-09-02 10:43:32 +08:00
Thales Macedo Garitezi 275171d217 feat: handle lists of servers in mongo servers config 2022-09-01 14:47:14 -03:00
Thales Macedo Garitezi 3d4afd65df feat: add mongodb bridge (e5.0) 2022-09-01 14:47:14 -03:00
Zaiming (Stone) Shi 8a5a9ca2bf
Merge pull request #8854 from zmstone/0831-chore-merge-main-to-dev-ee50
0831 chore merge main to dev ee50
2022-09-01 11:35:23 +01:00
JianBo He a5ac5b6f3a
Merge branch 'dev/ee5.0' into refactor-mqtt-bridge 2022-09-01 09:30:37 +08:00
Zaiming (Stone) Shi 5a3fb4a011 Merge remote-tracking branch 'origin/master' into dev/ee5.0 2022-08-31 17:11:15 +02:00
Shawn 0ef0b68de4 refactor: change '{recoverable_error,Reason}' to '{error,{recoverable_error,Reason}}' 2022-08-31 18:25:00 +08:00
Shawn ca52b8eb29 fix: start connector-mqtt failed when username/password not provided 2022-08-31 09:18:10 +08:00
Shawn 73e19d84ee feat: use the new metrics to bridge APIs 2022-08-30 23:47:58 +08:00
Shawn 65dfa63324 fix: update the counters for data bridges 2022-08-30 12:28:01 +08:00
Shawn 262e68f7d2 fix: return error when receive HTTP code other than 2xx 2022-08-30 10:16:02 +08:00
Shawn b5ad5233a1 fix(mqtt-bridge): username and password defaults to undefined 2022-08-30 10:14:56 +08:00
JianBo He 0aa10702db feat(bridge): support async mode resource options 2022-08-30 09:34:56 +08:00
JianBo He ca7ad9cc15 chore: refactor mqtt connector 2022-08-30 09:34:56 +08:00
Zaiming (Stone) Shi 2eb078ff3b chore: bump changed app versions 2022-08-26 14:32:28 +02:00
Shawn 1625b8eaeb fix(mysql_bridge): export the query_mode option to the APIs 2022-08-26 17:11:24 +08:00
Shawn 4ac6f12252 refactor: configs and APIs for mysql bridge 2022-08-24 10:08:03 +08:00
Shawn 45352206a3 fix(auth): remove emqx_connector from testcases of authz/authn 2022-08-22 20:20:45 +08:00
Shawn 55c18c0b5f fix(bridges): update the test cases for new config structure 2022-08-22 18:24:59 +08:00
Shawn aea8c77b49 refactor: new config structure for mqtt bridge 2022-08-22 18:24:59 +08:00
lafirest 4ad04b646f
Merge pull request #8759 from lafirest/feat/mysql_batch_query
feat(bridge): add `on_batch_query` on emqx_connector_mysql
2022-08-22 17:22:39 +08:00
JimMoen f0c2b53868 fix(bpapi): make bpapi static_checks happy 2022-08-22 10:51:44 +08:00
JimMoen 2648362c62 fix(bridge): password for bridge/db format as `password` for dashboard 2022-08-19 15:54:19 +08:00
JimMoen 06363e63d9 fix(influxdb): connector use a fallbacke `pool_size` for influxdb client 2022-08-19 15:54:19 +08:00
firest 8d8afd1688 feat(bridge): add `on_batch_query` on emqx_connector_mysql 2022-08-19 13:16:22 +08:00
ieQu1 b8a7ab9ad3 fix(connector_psql): Wrap password in a fun 2022-08-17 14:31:12 +02:00
Shawn de3a325953 fix: revert the changes in connector mysql 2022-08-16 09:06:13 +08:00
Xinyu Liu 2898966439
Merge branch 'dev/ee5.0' into resource_opts 2022-08-15 21:43:22 +08:00
Shawn 19d85d485b refactor(resource): add resource_opts level into config structure 2022-08-15 21:40:10 +08:00
Shawn b3162fe5ff fix: conflicts between master and ee5.0 2022-08-14 22:24:09 +08:00
JimMoen dc7953c3e1 chore: refine async query variable name 2022-08-12 17:28:56 +08:00
lafirest 404e79b7d7
Merge pull request #8687 from lafirest/fix/bridge_mysql
fix(bridge): replace prepare_statement by sql_template
2022-08-12 10:12:34 +08:00
JimMoen 449239d0f4 fix: count all msg size of `event_message` for mqtt bridge 2022-08-12 01:29:56 +08:00
Shawn 88388b0c54 feat: http connector support async sending 2022-08-11 21:44:22 +08:00
firest 7581082fcb fix(bridge): replace prepare_statement by sql_template 2022-08-11 13:22:16 +08:00
Zaiming (Stone) Shi 4994731d3e chore: update emqx_connector app vsn 2022-08-10 09:50:12 +02:00
Zaiming (Stone) Shi 18614ad291 fix(connector): add back deleted fields as deprecated
max_retries and retry_interval were removed from 5.0.4 wihtout
backward compatibliity.
This commit adds the fields back as deprecated
2022-08-10 09:50:12 +02:00
Shawn 35fe70b887 feat: support aysnc callback to connector modules 2022-08-10 00:34:35 +08:00
Shawn 75adba0781 fix: increase resource metrics using the resource id 2022-08-10 00:34:35 +08:00
Shawn d3950b9534 fix(resource): make option 'queue_enabled' disabled by default 2022-08-10 00:34:35 +08:00
Shawn 0377d3cf61 fix: update existing testcases for new emqx_resource 2022-08-10 00:34:35 +08:00
Shawn 2fb42e4d37 refactor: create emqx_resource_worker_sup for resource workers 2022-08-10 00:34:35 +08:00
firest aa6077bdea fix(bridge): remove emqx_ee_connector_mysql 2022-08-04 14:52:16 +08:00
DDDHuang 01e9e8a0eb feat: add ee influxdb connector & bridge 2022-08-01 10:27:35 +08:00
DDDHuang 3fe76446bc Merge branch 'master' into dev/ee5.0 2022-08-01 10:14:21 +08:00
DDDHuang a4992ef1b5 fix: hstreamdb connector conf & api 2022-07-28 16:47:30 +08:00
Shawn 23eeb21b13 fix(webhook): the 'max_retries' opt means request retry times 2022-07-27 17:06:14 +08:00
JimMoen 59c301243c fix(connector): redis sentinel field required 2022-07-13 10:37:16 +08:00
Shawn b55048f705 chore: update app versions 2022-07-08 16:29:07 +08:00
Shawn 841f816e39 refactor: improve the health_check for webhook bridges 2022-07-08 09:17:23 +08:00
Shawn 6685a3c5a8 fix: remove the 'headers' field from the rule events 2022-06-17 15:57:10 +08:00
Xinyu Liu c47d28cdc3
Merge pull request #8247 from terry-xiaoyu/mongo_auth_timeout
feat: add start_after_created option to resource:create/4
2022-06-17 07:51:52 +08:00
Shawn cc25f92273 feat: add start_after_created option to resource:create/4 2022-06-16 23:34:52 +08:00
JianBo He 70c63779df
Merge pull request #8232 from HJianBo/mapping-ssl-true-to-required
chore(pgsql): mapping ssl option from 'true' to 'required'
2022-06-16 16:11:34 +08:00
JianBo He 8194427cb3 chore(pgsql): mapping ssl option from 'true' to 'required'
Once the pgsql client is configured with ssl, we should force ssl to be used to
avoid fallback to tcp connection.
2022-06-16 10:04:07 +08:00
zhongwencool fbfed35371
Merge pull request #8214 from zmstone/0613-best-effort-json-for-hocon-type-check-errors
refactor: best-effort json for hocon type check errors
2022-06-15 22:45:05 +08:00
Zaiming (Stone) Shi 738a551550 refactor: best-effort json for hocon validation errors 2022-06-15 15:01:15 +02:00
JianBo He 57e57205a0
Merge pull request #8207 from HJianBo/gw-fixes
Gateway restart won't make built-in-database data lost
2022-06-15 17:41:13 +08:00
JianBo He 78dd9fe247 fix(gw): compatible with the max_connection is string type 2022-06-15 13:48:15 +08:00
JianBo He 14a26fc876 fix(redis): make sentinel required 2022-06-15 13:48:15 +08:00
Zaiming (Stone) Shi aea4180aa0 chore(connector): schema validator throw error messages directly 2022-06-14 16:20:10 +02:00
Zhongwen Deng c7cc2e85b2 fix: add default&example for schema 2022-06-06 09:52:10 +08:00
Shawn 88ca25c60c fix(resource): fast return when starting a unavailable resource 2022-06-01 08:24:53 +08:00
JianBo He efc0ca2b62
Merge branch 'master' into changes-enable_pipelining-type 2022-05-31 20:49:10 +08:00
Zhongwen Deng 9ec804ae03 feat: generate example.conf from schemas 2022-05-31 19:20:27 +08:00
JianBo He ebf4ed0270 feat(connector-http): using pos_integer() instead of boolean() for enable_pipelining 2022-05-31 17:14:59 +08:00
Shawn 1054c364ad refactor(resource): improve health check and alarm it if resource down 2022-05-31 01:40:40 +08:00
Shawn a157539710 refactor(rule): rename http_bridge -> webhook 2022-05-25 18:56:12 +08:00
Shawn 71de9616d3 refactor(rule): rename outputs -> actions 2022-05-25 17:02:37 +08:00
Xinyu Liu 54a6505984
Merge pull request #7982 from terry-xiaoyu/bridge_mode_in_mqtt_connector
fix: add bridge_mode field to the mqtt connector
2022-05-19 08:56:42 +08:00
Shawn 7e21ef6435 fix: add bridge_mode field to the mqtt connector 2022-05-18 18:28:43 +08:00
Shawn d89d692d3e refactor: move some APIs to new module emqx_bridge_resource 2022-05-18 15:12:34 +08:00
Zaiming (Stone) Shi 4013dd2f14 refactor: refine pmap implementation 2022-05-13 15:22:42 +02:00
DDDHuang 448f9b5c9f
Merge pull request #7940 from DDDHuang/mqtt_bridge_cs
fix: clean session when destroy mqtt bridge
2022-05-13 16:04:09 +08:00
DDDHuang 9744040795 fix: clean session when destroy mqtt bridge 2022-05-13 14:02:40 +08:00
EMQ-YangM 798a6dcfb9 fix: update comment 2022-05-13 13:55:17 +08:00
EMQ-YangM 637b81ec18 fix: mqtt connector disable bridge_mode 2022-05-13 13:08:04 +08:00
DDDHuang 2de69c97ba fix: ruleengine & connector & bridge api, better error message 2022-05-12 16:51:36 +08:00
Zhongwen Deng 703a8d3eff fix: clear cert crash when config is undefined 2022-05-10 15:47:16 +08:00
Chris Hicks 841acb7828
Merge pull request #7783 from emqx/EMQX-4199-introduce-a-new-emqx-resource-manager-module
feat: isolate resource manager processes
2022-05-09 18:33:01 +02:00
Chris 0b3e30e813 feat: isolate resource manager processes 2022-05-09 13:24:34 +02:00
Ilya Averyanov 4cff19985e
Merge pull request #7872 from savonarola/fix-mongo-conn-healthcheck
fix(mongodb): fix mongodb connection healthcheck
2022-05-06 11:47:26 +03:00
Ilya Averyanov 0dae3f43a9 fix(mongodb): fix mongodb connection healthcheck 2022-05-05 15:36:38 +03:00
Chris 6971a3109f fix: remove case clause where the error clause can never be reached 2022-05-05 14:26:04 +02:00
Shawn 5b5878f4a9 fix: don't drop input ssl options even if invalid 2022-05-05 14:53:42 +08:00
zhouzb afe526ba67
Merge pull request #7789 from tigercl/chore/authn-fields
chore: rename certificate in jwt and selector in mongodb, remove pool_size in jwks
2022-04-29 10:14:06 +08:00
DDDHuang 778973b2de
Merge pull request #7791 from DDDHuang/auto_failed_connector
fix: disconnected status in auto_reconnect = false
2022-04-29 09:30:16 +08:00
Xinyu Liu 8528d76371
Merge pull request #7815 from terry-xiaoyu/fix_bridge_ssl_dry_run_fail
fix: convert ssl certfiles for dry-run creating bridges
2022-04-28 22:41:50 +08:00
Shawn 4b7d9b7239 chore(erlfmt): make the erlfmt happy 2022-04-28 22:39:30 +08:00
Zaiming (Stone) Shi 33e9823317 style: reformat emqx_connector_mqtt_mod.erl 2022-04-28 15:07:07 +02:00
zhouzb 95e63fcadf fix: fix bad code 2022-04-28 16:51:36 +08:00
Shawn e18c2a6de2 fix: convert ssl certfiles for dry-run creating bridges 2022-04-28 16:17:55 +08:00
zhouzb bd0d0d9797
Merge branch 'master' into chore/authn-fields 2022-04-28 16:09:26 +08:00
DDDHuang 05cf8af14b fix: connector, remove bad include 2022-04-28 15:32:54 +08:00
DDDHuang 132b37813c refactor: code format emqx_connector emqx_resource 2022-04-28 15:32:47 +08:00
DDDHuang 2a2308bbf8 refactor: resource check & connector status 2022-04-28 15:32:35 +08:00
DDDHuang e3e7acbcde fix: remove match topic in mqtt bridge 2022-04-28 15:23:30 +08:00
Zaiming (Stone) Shi 02c3f87b31 style: reformat all remaining apps 2022-04-27 15:51:18 +02:00
Shawn 0635918d16 fix: define ssl SNI field as a non-empty-string 2022-04-27 14:17:22 +08:00
Shawn 318d0df419 fix: return value of post_config_update 2022-04-27 11:59:16 +08:00
Shawn 94e24c2621 refactor: move ssl file handling from resources to bridges 2022-04-27 11:59:15 +08:00
zhouzb fa9bd74595 chore: rename selector to filter and fix spellcheck 2022-04-27 11:29:18 +08:00
zhongwencool 4b763a3a12
Merge pull request #7742 from zhongwencool/schema-namespace
fix: add namespace to schema
2022-04-26 09:26:04 +08:00
JimMoen e9843aa225 fix(doc): html line break use `</br>` 2022-04-24 18:56:21 +08:00
Yang Miao 742e9eed43
Merge pull request #7743 from EMQ-YangM/work1
fix: avoid '/connectors' function pattern matching failures
2022-04-24 18:50:56 +08:00
EMQ-YangM 9311080f0b fix: avoid '/connectors' function pattern matching failures 2022-04-24 16:07:15 +08:00
Zhongwen Deng 7d9403d056 fix: add namespace to schema 2022-04-24 15:47:23 +08:00
Shawn 984a26945c docs: add i18n ids 2022-04-23 03:51:38 +08:00
Yang Miao 4a43a10296
Merge pull request #7701 from EMQ-YangM/improve_docs
fix: improve document quality
2022-04-22 16:52:09 +08:00
EMQ-YangM 28c8f661ab fix(emqx_connector_http): request field reply to previous value 2022-04-22 09:31:14 +08:00
DDDHuang 9088752afa fix: bad return in retry prepare 2022-04-22 00:20:13 +08:00
DDDHuang 4b7a5bbf53 fix: bad empty map 2022-04-21 21:26:35 +08:00
DDDHuang 7417e5070d fix: perpare sql when connector doing init; add prepare statement field; bad log path 2022-04-21 16:00:15 +08:00
DDDHuang 365ca67078 fix: auth mysql prepare query 2022-04-21 16:00:15 +08:00
DDDHuang 94795098c9 fix: bad function name & dialyzer 2022-04-21 16:00:15 +08:00
DDDHuang 039619ee23 feat: authz & authn mysql resource support prepare sql query 2022-04-21 16:00:15 +08:00
DDDHuang d2c4b862b6 fix: bad code & xref 2022-04-21 16:00:14 +08:00
DDDHuang 0faf1240f3 fix: mysql support prepare sql 2022-04-21 16:00:14 +08:00
EMQ-YangM cef16ac382 fix: improve document quality 2022-04-21 15:05:56 +08:00
Xinyu Liu f8d046b259
Merge pull request #7673 from terry-xiaoyu/mqtt_bridge_issues
fix: improve the error logs if update bridge failed
2022-04-20 18:37:48 +08:00
Shawn 3ce969fd79 refactor: always recreate resources no matter it is connected or not 2022-04-20 11:43:05 +08:00
Shawn 1b79b3f3c9 fix: improve the error logs if update bridge failed 2022-04-19 19:45:46 +08:00
JimMoen 88c98bb310 fix(authn): api required fields 2022-04-19 18:57:01 +08:00
EMQ-YangM 7fbabf5577 fix: connector docs 2022-04-19 10:43:53 +08:00
DDDHuang 92eaf403f1 fix: mongo connector bad schema 2022-04-16 16:37:10 +08:00
DDDHuang e22d01d25f fix: page & limit & pool_size pos_integer; some bad spelling 2022-04-16 16:37:10 +08:00
Zaiming (Stone) Shi 356fbf6ce1
Merge pull request #7519 from zmstone/0405-5.0-sync-ssl-lib-from-4.3
5.0 refactor: delete deprecated emqx_plugin_libs_ssl module
2022-04-05 22:57:56 +01:00
Zaiming (Stone) Shi ba51f03cb7 refactor: remove the deprecated save_files_return_opts 2022-04-05 17:38:33 +02:00
ieQu1 99b3f2f83e docs(schema): Document missing records and fields 2022-04-04 23:58:43 +02:00
ieQu1 edb91e2e4a docs(schema): Add descriptions for authZ and connector 2022-03-30 13:01:01 +02:00
ieQu1 80cd1350a5 docs(schema): Add documentation for connector records 2022-03-30 11:53:53 +02:00
Shawn 79222e694e fix: list connectors with default values 2022-03-25 15:15:27 +08:00
DDDHuang e517b41a4d fix(api): desc & remove useless api 2022-03-21 15:03:37 +08:00
DDDHuang 4d4348759d fix: generate connector & exhook & retainer api response code 2022-03-21 14:12:19 +08:00
DDDHuang 74e6fa5a94 fix: generate bridge api response code
fix: generate connector api response code SUITE

fix: generate exhook api response code
2022-03-21 14:12:19 +08:00
JimMoen ed021d125f fix(mongo): servers for mongo_sharded and mongo_rs 2022-03-18 17:58:27 +08:00