yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,504 Commits 393 Branches 669 Tags 74 MiB
85a130f68b
Commit Graph

540 Commits

Author SHA1 Message Date
Kjell Winblad
85a130f68b refactor: move MySQL open source connector to its own app 2023-06-27 16:15:18 +02:00
Kjell Winblad
dd6aa96c99 refactor: move Redis open source connector to its own app 2023-06-27 10:24:40 +02:00
Kjell Winblad
b78a75bb5c refactor: the MongoDB connector into its own application 2023-06-26 14:32:59 +02:00
Thales Macedo Garitezi
2f00cf7f84
Merge pull request #11107 from thalesmg/fix-mongo-health-check-reason-master 
fix(mongo): return health check failure reason
 2023-06-22 09:30:34 -03:00
Thales Macedo Garitezi
7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer 
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
 2023-06-22 09:17:45 -03:00
Zaiming (Stone) Shi
c58a98954b Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-22 11:05:51 +02:00
Thales Macedo Garitezi
18f0510353 fix(mongo): return health check failure reason 
Fixes https://emqx.atlassian.net/browse/EMQX-10335
 2023-06-21 15:09:37 -03:00
Paulo Zulato
8430ec673c feat(kafka): check whether target topic exists 
Fixes https://emqx.atlassian.net/browse/EMQX-9026
 2023-06-21 15:00:35 -03:00
Zaiming (Stone) Shi
7cf8a6c892 chore: bump app vsns 2023-06-21 16:36:51 +02:00
Thales Macedo Garitezi
dabefdb01b fix(gcp_pubsub): redact request when errors occur 2023-06-19 15:59:00 -03:00
Paulo Zulato
c9a2ddf98c feat(mysql): check whether target table exists 
Fixes https://emqx.atlassian.net/browse/EMQX-9026
 2023-06-19 15:22:27 -03:00
Paulo Zulato
9454af9a8b feat(postgresql): check whether target table exists 
Fixes https://emqx.atlassian.net/browse/EMQX-9026
 2023-06-19 11:12:10 -03:00
Stefan Strigler
0d6d441f4c test(emqx_connector): start/stop test for webhook bridge 2023-06-14 09:56:50 +02:00
Stefan Strigler
b2a5065641 fix(emqx_connector): report errors in on_start handler 2023-06-13 16:57:08 +02:00
Andrew Mayorov
e6fb0203b4
refactor(pluglib): move connectivity checks to emqx_connector_lib 2023-06-09 14:44:37 +03:00
Andrew Mayorov
8919a6ef93
refactor(pluglib): provide SQL related utils in emqx_utils_sql 2023-06-09 14:44:37 +03:00
Andrew Mayorov
d6c1ee183f
refactor(pluglib): move emqx_placeholder to utils app 
Also make user that existing code calls it directly.
 2023-06-09 14:44:36 +03:00
Zaiming (Stone) Shi
641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51 
0607 merge master to release 51
 2023-06-08 09:22:21 +02:00
JianBo He
0b95bc1c1b
Merge pull request #10907 from HJianBo/refactor-on-stop-simple 
feat: refactored some bridges to avoid leaking resources part.2
 2023-06-08 09:37:31 +08:00
Zaiming (Stone) Shi
ccd2589ff2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-07 21:43:29 +02:00
Thales Macedo Garitezi
260fae296b feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1) 
Fixes https://emqx.atlassian.net/browse/EMQX-9603

Rather than relying on a JWT worker to produce and refresh tokens, we
could just produce then on demand when pushing the messages to GCP
PubSub.  That can generate a bit of extra work (as multiple processes
might realize it’s time to refresh the JWT and do so), but that
shouldn’t be much.  In return, we avoid any possibility of not having
a fresh JWT when pushing messages.
 2023-06-06 13:19:24 -03:00
Andrew Mayorov
95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys 
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
 2023-06-05 22:41:07 +03:00
Thales Macedo Garitezi
46393343e2 chore: use timeout_duration types for timer fields 
Fixes https://emqx.atlassian.net/browse/EMQX-10020
 2023-06-05 11:46:38 -03:00
JianBo He
dbc0cdce67 chore: dont allocate resource for simple connectiors 2023-06-05 10:59:00 +08:00
Thales Macedo Garitezi
0072f4a598 fix(ecpool,bridge): treat {error, ecpool_empty} as a retriable error 
Example from flaky CI test:

```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
 2023-06-01 17:13:06 -03:00
firest
6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Andrew Mayorov
7e7b50c5ba
refactor(mqttbridge): move into separate application 2023-05-30 17:56:34 +03:00
Andrew Mayorov
ebd612b194
chore: bump applications versions 
* emqx_connector 0.1.25
* emqx_rule_engine 5.0.19
* emqx_ee_bridge 0.1.15
 2023-05-30 17:21:45 +03:00
Andrew Mayorov
eed9358abd
chore: bump ecpool to 0.5.4 
With fixed typings and empty pool handling.
 2023-05-30 17:21:45 +03:00
Andrew Mayorov
a5fc26736d
refactor(mqttconn): split ingress/egress into 2 separate pools 
Each with a more refined set of responsibilities, at the cost of slight
code duplication. Also provide two different config fields for each pool
size.
 2023-05-30 17:21:44 +03:00
Andrew Mayorov
c7528e9b35
feat(mqttconn): add pool_size config parameter 
That currently tunes the number of MQTT clients employed both for
subscriptions (if shared subscription is used) and for publishing to
a remote broker.
 2023-05-30 17:21:44 +03:00
Andrew Mayorov
6e97dffdb8
feat(mqttconn): deprecate mode config parameter 
It adds no value: the only mode was `cluster_shareload` and we just as
well can decide to "share" the load across cluster just by looking if
the remote topic is shared subcription filter or not.
 2023-05-30 17:21:44 +03:00
Andrew Mayorov
4e6269bedb
feat(mqttconn): subscribe each worker if shared subcription 
Also rename `subscriptions` -> `ingress` and `forwards` -> `egress` for
consistency with the config schema.
 2023-05-30 14:50:05 +03:00
Andrew Mayorov
81e78516aa
feat(mqttconn): employ ecpool instead of a single worker 2023-05-30 14:50:05 +03:00
Andrew Mayorov
6967f621d8
fix(mqttconn): unify error interpretation in sync/async modes 
Also move this logic to the mqtt connector itself, in order to avoid
dealing with extra callback layer.
 2023-05-30 14:50:05 +03:00
Andrew Mayorov
67d703f8c5
refactor(mqttconn): simplify mqtt connector 
Inline `emqx_connector_mqtt_msg` module code into
`emqx_connector_mqtt_worker` module, since it's not really used
anywhere else and does not provide any reusable abstractions.
 2023-05-30 14:50:05 +03:00
Andrew Mayorov
bd956d00b6
feat(mqttconn): stop using gproc in hot path 
Also drop fiddling with `mountpoint` since this option seems not to be
used anywhere.
 2023-05-30 14:50:05 +03:00
Andrew Mayorov
4da0d83faf
chore(mqttconn): remove dead code 2023-05-30 14:50:04 +03:00
Zaiming (Stone) Shi
36e268c933 chore: bump app versions 2023-05-26 16:05:37 +02:00
Zaiming (Stone) Shi
cc5b4d3748 Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test 2023-05-26 15:58:16 +02:00
Thales Macedo Garitezi
194faefc8a fix(webhook): keep resource_opts.request_timeout for webhook bridge (r5.0) 
Fixes https://emqx.atlassian.net/browse/EMQX-10001

Recently, we unified request_timeout in a single field located at the
webhook connector schema.  However, the correct fix would be to use
the resource_opts.request_timeout one, as that’s the only one that
allows infinity timeout.
 2023-05-25 13:19:27 -03:00
JimMoen
28015597ee
Merge remote-tracking branch 'emqx/release-50' into merge-release-50 2023-05-24 19:34:12 +08:00
Zaiming (Stone) Shi
256cb73e43
Merge pull request #10792 from thalesmg/fix-webhook-bridge-health-check-clause-r50 
fix(webhook): fix empty ehttpc worker list clause on health check (r5.0)
 2023-05-24 08:46:59 +02:00
Thales Macedo Garitezi
25437adf02 fix(webhook): fix empty ehttpc worker list clause on health check (r5.0) 
Fixes https://emqx.atlassian.net/browse/EMQX-9970
 2023-05-23 14:06:40 -03:00
Kjell Winblad
ea2fea77d7 fix: RabbitMQ field not marked as required 
This commit makes sure that the RabbitMQ password filed is marked as
required. This ensures that the user provides a password and that the
bridge does not throw a function clause exception if the password filed
is not set.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9974
 2023-05-23 15:50:18 +02:00
Zaiming (Stone) Shi
732a7be187 Merge remote-tracking branch 'origin/release-50' 2023-05-22 17:46:54 +02:00
Zaiming (Stone) Shi
be90c63c78 chore(mqtt-connector): refine logging level 
connect failure should be at warning level but not error,
the connecting state is visiable from dashbaord
also the resource manager logs connection failures in general
at warning level
 2023-05-19 18:55:16 +02:00
Zaiming (Stone) Shi
5bbcf4b712 fix(mqtt-connector): faster shutdown 2023-05-19 18:55:16 +02:00
Thales Macedo Garitezi
f83f9a2d79 chore: bump app vsns 2023-05-17 14:56:11 -03:00
Thales Macedo Garitezi
b2afe4e90c
Merge pull request #10690 from thalesmg/perf-webhook-retry-async-reply-v50 
perf(webhook): add async retries and evaluate reply callback in fresh process
 2023-05-17 11:06:48 -03:00