yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,454 Commits 393 Branches 669 Tags 74 MiB
71d3148544
Commit Graph

452 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi
0697c692ed refactor: mongo_type and redis_type are not mandatory 
the connector schemas are shared between authn, auth and bridges,
the difference is that authn and authz configs are unions like:

[
    {mongo_type = rs,
     ...
    }
    {another backend config}
]

however the brdige types are maps like, for example:

mongodb_rs.$name {
    mongo_type = rs
    ...
}

in which case, the mongo_type is not required.

in order to keep the schema static as much as possible,
this field is chanegd to 'required => false' with a default value.

However, for authn and authz, the union selector will still raise
exception if the there is no type provided.
 2023-01-09 14:26:16 +01:00
Zaiming (Stone) Shi
e52f9d5920 refactor: use union member type selector for authz sources 2023-01-09 14:26:16 +01:00
firest
f7b50c56da feat(bridges): integrate PostgreSQL into bridges 
support both simple and batch query
 2023-01-06 11:47:22 +08:00
Thales Macedo Garitezi
00b91d8a60
Merge pull request #9664 from thalesmg/fix-mongo-duration-field-descs-v50 
docs(mongodb_connector): fix duration field descriptions
 2023-01-04 10:03:38 -03:00
Erik Timan
f767db4d8f
Merge pull request #9660 from olcai/refactor-ingress-bridge-counter 
refactor(emqx_resource): ingress bridge counter
 2023-01-04 10:43:07 +01:00
Thales Macedo Garitezi
7bbdc20993 docs(mongodb_connector): fix duration field descriptions 
https://emqx.atlassian.net/browse/EMQX-8642

Adds different descriptions for the different fields of type duration
for MongoDB.

Based off: https://www.mongodb.com/docs/manual/reference/connection-string/
 2023-01-03 10:01:56 -03:00
Erik Timan
b9d012e072 refactor(emqx_resource): ingress bridge counter 
Unify code paths for resource metrics by removing
emqx_resource:inc_received/1 and adding
emqx_resource_metrics:received_inc/1 & friends.
 2023-01-02 15:11:52 +01:00
firest
bc37556963 fix(mysql): fix the problem of data loss and bad match when mysql is disconnected 2023-01-02 21:46:24 +08:00
Zaiming (Stone) Shi
dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
Zaiming (Stone) Shi
0ce1ca89b7 refactor: use string type for server and servers 2022-12-30 14:20:23 +01:00
Thales Macedo Garitezi
446a4c74d0 fix(gcp_pubsub): fix potential jwt accumulation and lack of refresh (v5.0) 
https://emqx.atlassian.net/browse/EMQX-8653
Related:
- https://emqx.atlassian.net/browse/EEC-737
- https://emqx.atlassian.net/browse/EMQX-8652

Since the rule resource testing mechanism creates a new resource to
test the configuration, a new JWT associated with an unique temporary
resource was being created and left in the JWT table, leaking it.

Also, a wrong case clause when setting the new refresh timer for the
JWT worker was preventing it from refreshing from the 2nd refresh
onward.
 2022-12-29 16:30:36 -03:00
zhongwencool
81b226a801
Merge pull request #9608 from zmstone/1123-pin-hocon-0.33.0 
1123 pin hocon 0.33.0
 2022-12-26 19:22:21 +08:00
Thales Macedo Garitezi
35dc75b7ed feat(mqtt): add option to customize clientid prefix for egress bridges 
https://emqx.atlassian.net/browse/EMQX-8445

Currently the bridge client’s client ID is prefixed with the resource
ID.

Sometimes it’s useful for users to have control of this prefix,
e.g. prefix based ACL rules in the target broker.
 2022-12-23 09:50:26 -03:00
Ilya Averyanov
fa7433628c fix(logs): set password field sensitive for connectors 2022-12-23 12:46:03 +01:00
Zaiming (Stone) Shi
6292d7ec68 refactor: remove dead code 2022-12-23 12:07:36 +01:00
Erik Timan
13942f5c49 refactor: rename error return in mysql connector 2022-12-22 10:29:12 +01:00
Erik Timan
7df24000a0 test: add more EE mysql bridge test cases 2022-12-22 10:27:38 +01:00
Erik Timan
23ac426608 fix(emqx_connector): check for key among prepared statements on query 
An infinite loop was triggered in the mysql connector when a query
used a prepared statement key that was not among the defined prepared
statements on start. We now check that the key is defined among the
prepared statements before recursing. It seems that this bug was never
triggered in any production code flow and simply found while writing
tests.

An error return spell fix is also included as well as a FIXME comment
regarding running mysql:prepare and not distinguishing between
transient failures and syntax errors. Syntax errors should not be
retried.
 2022-12-22 10:27:38 +01:00
Zaiming (Stone) Shi
516147ad00
Merge pull request #9581 from zmstone/1219-fix-mqtt-bridge-config 
1219 fix mqtt bridge config
 2022-12-20 14:47:02 +01:00
Zaiming (Stone) Shi
da51433dc3 refactor: add default value for eggress remote 'qos' and 'retain' 
otherwise when updating from dashboard, there is no way to set
'false' for 'retain' because it's a checkbox,
it's either 'true' or 'undefined'
 2022-12-20 00:00:31 +01:00
Zaiming (Stone) Shi
29f394aa70 fix(shared): handle unsubscribe for sticky strategy 
prior to this change, the message is dispatched to a shared subscriber
even after unsubscribed
 2022-12-19 19:12:38 +01:00
Andrew Mayorov
ed2d5aa48a
fix: handle status in format_status/1 callback 
Which is expected argument type for this callback. Also try to
make sure that random maps won't pass through this callback unnoticed.
 2022-12-16 13:45:06 +03:00
Zaiming (Stone) Shi
9e3da5b661 chore: bump app versions 2022-12-14 20:07:41 +01:00
Zaiming (Stone) Shi
42c58e2a91 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 15:29:13 +01:00
Thales Macedo Garitezi
1cd91a24e9 feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0) 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
34e9056779 refactor: fix typo in variable name 
Might confuse people to think it's related to `replayq`.
 2022-12-12 17:17:51 -03:00
Zaiming (Stone) Shi
40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
Ilya Averyanov
6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi
1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Erik Timan
485c4ddcab
Merge pull request #9393 from olcai/add-mysql-bridge-test 
test: add basic mysql bridge test (ee5.0)
 2022-12-02 09:43:22 +01:00
Zaiming (Stone) Shi
eb017ab034 test: add test case to cover mqtt bridge config upgrade 2022-12-01 16:50:10 +01:00
Erik Timan
5461311edc test: more EE mysql bridge tests 2022-12-01 10:27:51 +01:00
Erik Timan
96bff1d32e test: improve basic tests for EE mysql bridge 2022-12-01 10:27:51 +01:00
Zaiming (Stone) Shi
9a6901987f feat: add mqtt bridge config upgrade converter 2022-11-30 21:01:59 +01:00
Zaiming (Stone) Shi
983e904858 fix(connector): fix ssl clear 2022-11-30 17:03:41 +01:00
firest
81586a4c8d chore: bump connector version 2022-11-30 23:14:54 +08:00
firest
51ace31f30 fix(connector): fix previous incorrectly fixes 
There was an error in the try_clear_certs2/3 clause that the argument isn't the ssl field but including it.
 2022-11-30 22:14:44 +08:00
Zaiming (Stone) Shi
b398617614 chore: bump app versions 2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi
7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
firest
fc2793c300 fix: fix upside down function clause 2022-11-24 10:37:52 +08:00
Zaiming (Stone) Shi
67cef65fdb refactor(emqx_connector_ssl): simplify implementation 2022-11-23 23:49:43 +01:00
firest
6dc5078a96 fix: fix obsolete SSL files aren't deleted after the bridge configuration update 2022-11-23 23:49:43 +01:00
kraftwerk28
00c57de4c3 feat: do not drop MQTTv5 properties in rule/bridge 2022-11-22 20:09:59 +01:00
Zaiming (Stone) Shi
09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
firest
d0687c7aaa fix(connector): keep connector name type as binary don't convert it to atom 
After investigation, it was confirmed that there was no need to convert the connector name to atom
 2022-11-01 15:36:31 +08:00
Zaiming (Stone) Shi
c157392452 docs: fix self-closing html tag, change </br> to <br/> 2022-10-27 13:57:18 +02:00
JianBo He
834910b6f6 chore: update app.src 2022-09-27 17:24:44 +08:00
JianBo He
39ad172377 chore: replace ip_port() wuth host_port() for all connectors 2022-09-27 15:19:42 +08:00
Zaiming (Stone) Shi
00e4b4da5a fix(schema): support hostname.domain:port for mqtt bridge 2022-09-27 09:02:49 +02:00
Shawn
dfe14be8b2 chore: bump app vsns 2022-09-26 18:23:28 +08:00
Shawn
4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
JimMoen
a660004fa4 fix(bridge): mqtt bridge client info username & password 
for non configured `username` and `password`, default values `<<>>` should be
used. emqtt needs the values.
 2022-09-16 15:09:51 +08:00
Shawn
d5d3972ff5 chore: add test cases for MQTT Bridge reconnecting 2022-09-15 10:19:33 +08:00
Shawn
4e211c12d3 fix(mqtt_bridge): return value of sending messages was discarded 2022-09-15 08:57:01 +08:00
Shawn
1c03c236f5 fix(mqtt_bridge): handle send_to_remote in idle state 2022-09-14 15:19:30 +08:00
JimMoen
f018b8ab49 fix(bridge/authz/exhook): no need to drop invalid certs 2022-09-14 14:08:55 +08:00
JimMoen
b5bf5d3adb fix: mqtt bridge payload default value 2022-09-13 18:58:01 +08:00
Zaiming (Stone) Shi
81ec4c4403
Merge pull request #8906 from zmstone/merge-master-to-ee50-a 
Merge master to ee50
 2022-09-07 11:19:41 +02:00
Shawn
2b33ca6d49 fix: no error log print if insert bool values into mysql 2022-09-07 16:00:09 +08:00
Zaiming (Stone) Shi
befc4acced Merge remote-tracking branch 'origin/master' into merge-master-to-ee50-a 2022-09-06 20:31:38 +02:00
Xinyu Liu
23b6ff399d
Merge pull request #8844 from thalesmg/mongodb-bridge 
feat: add mongodb bridge (e5.0)
 2022-09-03 12:27:01 +08:00
Xinyu Liu
7fed4faac3
Merge pull request #8859 from terry-xiaoyu/refactor_counters_for_resources 
Some improvements and bug fixes to emqx_resources
 2022-09-03 12:16:49 +08:00
JimMoen
d533e35df0 chore: update app vsn 2022-09-02 22:26:21 +08:00
Shawn
26234d38b9 fix: mark the async msg 'queuing' not 'sent.inflight' on recoverable_error 2022-09-02 18:41:43 +08:00
JimMoen
16618131ad fix(bridge): mqtt source/sink bridge use empty payload template 2022-09-02 14:44:38 +08:00
JimMoen
2693fc263d fix(bridge): show file paths for SSL certs in dashboard for bridge 2022-09-02 14:12:03 +08:00
Shawn
b45f3de8db refactor(resource): rename metrics batched,queued -> batching,queuing 2022-09-02 12:41:14 +08:00
JimMoen
74659bb864 fix(bridge): rm mqtt bridge default username and password 2022-09-02 10:43:32 +08:00
Thales Macedo Garitezi
275171d217 feat: handle lists of servers in mongo servers config 2022-09-01 14:47:14 -03:00
Thales Macedo Garitezi
3d4afd65df feat: add mongodb bridge (e5.0) 2022-09-01 14:47:14 -03:00
Zaiming (Stone) Shi
8a5a9ca2bf
Merge pull request #8854 from zmstone/0831-chore-merge-main-to-dev-ee50 
0831 chore merge main to dev ee50
 2022-09-01 11:35:23 +01:00
JianBo He
a5ac5b6f3a
Merge branch 'dev/ee5.0' into refactor-mqtt-bridge 2022-09-01 09:30:37 +08:00
Zaiming (Stone) Shi
5a3fb4a011 Merge remote-tracking branch 'origin/master' into dev/ee5.0 2022-08-31 17:11:15 +02:00
Shawn
0ef0b68de4 refactor: change '{recoverable_error,Reason}' to '{error,{recoverable_error,Reason}}' 2022-08-31 18:25:00 +08:00
Shawn
ca52b8eb29 fix: start connector-mqtt failed when username/password not provided 2022-08-31 09:18:10 +08:00
Shawn
73e19d84ee feat: use the new metrics to bridge APIs 2022-08-30 23:47:58 +08:00
Shawn
65dfa63324 fix: update the counters for data bridges 2022-08-30 12:28:01 +08:00
Shawn
262e68f7d2 fix: return error when receive HTTP code other than 2xx 2022-08-30 10:16:02 +08:00
Shawn
b5ad5233a1 fix(mqtt-bridge): username and password defaults to undefined 2022-08-30 10:14:56 +08:00
JianBo He
0aa10702db feat(bridge): support async mode resource options 2022-08-30 09:34:56 +08:00
JianBo He
ca7ad9cc15 chore: refactor mqtt connector 2022-08-30 09:34:56 +08:00
Zaiming (Stone) Shi
2eb078ff3b chore: bump changed app versions 2022-08-26 14:32:28 +02:00
Shawn
1625b8eaeb fix(mysql_bridge): export the query_mode option to the APIs 2022-08-26 17:11:24 +08:00
Shawn
4ac6f12252 refactor: configs and APIs for mysql bridge 2022-08-24 10:08:03 +08:00
Shawn
45352206a3 fix(auth): remove emqx_connector from testcases of authz/authn 2022-08-22 20:20:45 +08:00
Shawn
55c18c0b5f fix(bridges): update the test cases for new config structure 2022-08-22 18:24:59 +08:00
Shawn
aea8c77b49 refactor: new config structure for mqtt bridge 2022-08-22 18:24:59 +08:00
lafirest
4ad04b646f
Merge pull request #8759 from lafirest/feat/mysql_batch_query 
feat(bridge): add `on_batch_query` on emqx_connector_mysql
 2022-08-22 17:22:39 +08:00
JimMoen
f0c2b53868 fix(bpapi): make bpapi static_checks happy 2022-08-22 10:51:44 +08:00
JimMoen
2648362c62 fix(bridge): password for bridge/db format as password for dashboard 2022-08-19 15:54:19 +08:00
JimMoen
06363e63d9 fix(influxdb): connector use a fallbacke pool_size for influxdb client 2022-08-19 15:54:19 +08:00
firest
8d8afd1688 feat(bridge): add on_batch_query on emqx_connector_mysql 2022-08-19 13:16:22 +08:00
ieQu1
b8a7ab9ad3 fix(connector_psql): Wrap password in a fun 2022-08-17 14:31:12 +02:00
Shawn
de3a325953 fix: revert the changes in connector mysql 2022-08-16 09:06:13 +08:00
Xinyu Liu
2898966439
Merge branch 'dev/ee5.0' into resource_opts 2022-08-15 21:43:22 +08:00
Shawn
19d85d485b refactor(resource): add resource_opts level into config structure 2022-08-15 21:40:10 +08:00
Shawn
b3162fe5ff fix: conflicts between master and ee5.0 2022-08-14 22:24:09 +08:00
JimMoen
dc7953c3e1 chore: refine async query variable name 2022-08-12 17:28:56 +08:00
lafirest
404e79b7d7
Merge pull request #8687 from lafirest/fix/bridge_mysql 
fix(bridge): replace prepare_statement by sql_template
 2022-08-12 10:12:34 +08:00
JimMoen
449239d0f4 fix: count all msg size of event_message for mqtt bridge 2022-08-12 01:29:56 +08:00