Thales Macedo Garitezi
a5d848515b
fix: project sub-fields in action/connector `resource_opts` for some bridges
...
Fixes https://emqx.atlassian.net/browse/EMQX-11589
2023-12-14 12:09:32 -03:00
Thales Macedo Garitezi
a7fe5da8d6
fix(connector_api): avoid calling resource process to get channels
...
Fixes https://emqx.atlassian.net/browse/EMQX-11586
2023-12-14 10:22:46 -03:00
Thales Macedo Garitezi
edec431a6c
Merge pull request #12144 from thalesmg/fix-conn-schema-query-mode-r54-20231211
...
fix({action,connector}_schema): fix `resource_opts` subfields for connectors and actions, and remove redundant subfields from actions, use query mode from actions
2023-12-13 09:19:46 -03:00
Thales Macedo Garitezi
7f0cbcc323
Merge pull request #12155 from thalesmg/fix-conn-inconsistent-status-r54-20231212
...
fix(connector_api): return status reason when status is inconsistent
2023-12-12 17:23:22 -03:00
Thales Macedo Garitezi
4a71aa58ce
fix(connector_api): return status reason when status is inconsistent
...
Fixes https://emqx.atlassian.net/browse/EMQX-11581
2023-12-12 11:29:34 -03:00
Thales Macedo Garitezi
6e49cce055
feat(connector_api): list disabled channels too
...
Fixes https://emqx.atlassian.net/browse/EMQX-11583
2023-12-12 11:20:33 -03:00
Thales Macedo Garitezi
593283df93
fix(redis_bridge_schema): use correct `resource_opts` subfields for connector and action
2023-12-11 16:58:43 -03:00
Thales Macedo Garitezi
fbbb55633d
fix(connector_schema): remove `query_mode` from `resource_opts`
...
The connector query mode is inferred during its creation, and later it must be overridden
by an action, anyway.
2023-12-11 14:13:40 -03:00
Stefan Strigler
5d7ae4b980
Merge pull request #12132 from sstrigler/EMQX-11154-bridge-v-2-my-sql-support
...
feat(emqx_bridge_mysql): port to shared connectors
2023-12-11 13:56:34 +01:00
Zaiming (Stone) Shi
ddbb8560fa
fix(dialyzer): batch 2
2023-12-08 17:59:55 +01:00
Stefan Strigler
71607aa2ad
feat(emqx_bridge_mysql): port to shared connectors
2023-12-08 16:15:18 +01:00
Thales Macedo Garitezi
c45ed7b888
fix(connectors_api): fill default raw config values before returning
...
Fixes https://emqx.atlassian.net/browse/EMQX-11562
2023-12-07 09:44:48 -03:00
Thales Macedo Garitezi
c274450c5e
Merge pull request #12118 from thalesmg/fix-missing-type-name-api-actions-r54-20231206
...
fix(actions_api): add missing fields to API spec and returned data
2023-12-07 09:34:29 -03:00
Zaiming (Stone) Shi
33a7282cdd
fix(dialyzer): only include eunit when TEST is defined
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi
423b586c56
fix(dialyzer): fix some dialyzer issues found on otp 26
2023-12-06 20:32:49 +01:00
Thales Macedo Garitezi
2757b95850
fix(actions_api): fill raw config defaults before returning
...
Fixes https://emqx.atlassian.net/browse/EMQX-11541
2023-12-06 11:13:09 -03:00
Thales Macedo Garitezi
d594b38ceb
Merge pull request #12083 from thalesmg/fix-connector-bws-m-20231201
...
fix(connector): don't start buffer workers for the connector itself
2023-12-04 16:14:15 -03:00
Thales Macedo Garitezi
10b1a8bbda
feat(connector): add some `resource_opts` schema facilities
2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
29ae45c39d
fix(connector): don't start buffer workers for the connector itself
...
Fixes https://emqx.atlassian.net/browse/EMQX-11448
2023-12-01 17:20:38 -03:00
Zaiming (Stone) Shi
1a78e7ae79
fix(connector): ensure webhook bridge convert to http connector
2023-12-01 11:56:37 +01:00
Zaiming (Stone) Shi
7f4d91d490
Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54
2023-12-01 08:32:22 +01:00
Ivan Dyachkov
9fd2fa95a8
chore: bump apps versions
2023-11-30 20:01:12 +01:00
Ivan Dyachkov
ec10c51073
Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53
2023-11-30 19:51:12 +01:00
zhongwencool
0aec2f7605
feat: redis bridge v2
2023-11-29 16:06:46 +08:00
JianBo He
cdb90ebe6b
feat: rename webhook bridge to http bridge
2023-11-28 09:53:46 +08:00
JianBo He
dc99651690
test(bridge): ensure almost test cases passed
2023-11-28 09:53:46 +08:00
JianBo He
96af7a74e8
feat: impl the http bridge v2
2023-11-28 09:53:46 +08:00
JianBo He
f9a1e747fd
chore(http): break the bridge confs to connector and action parts
2023-11-28 09:53:46 +08:00
Stefan Strigler
70978df5bb
fix: use parse_id
2023-11-27 16:32:15 +01:00
Stefan Strigler
048f4724a9
feat(emqx_connector): add field 'actions' in API response
...
Also unify schemas, use emqx_connector_schema for the generic parts.
2023-11-27 16:09:17 +01:00
Kjell Winblad
66945dcc5c
fix: address more comments from @thalesmg
2023-11-24 20:31:29 +01:00
Kjell Winblad
3dca83c854
fix: all missing descriptions
2023-11-24 17:33:39 +01:00
Kjell Winblad
d5b62eead0
feat: split pgsql, matrix and timescale into connector action
...
This commit splits the bridges pgsql, matrix and timescale into
connector and action.
Fixes:
https://emqx.atlassian.net/browse/EMQX-11155
2023-11-24 17:33:35 +01:00
Stefan Strigler
4e077c951b
feat(emqx_bridge_mongodb): port mongodb to shared connector and actions
2023-11-24 15:09:35 +01:00
Thales Macedo Garitezi
f3693e5dbc
fix(gcp_pubsub_producer): add missing references to api specs
2023-11-23 09:02:52 -03:00
Thales Macedo Garitezi
6c9417efe0
Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122
2023-11-22 12:02:34 -03:00
Zaiming (Stone) Shi
3261a12140
fix(emqx_resource): do not allow leading _ or - as resource name
2023-11-22 10:58:54 +01:00
Kjell Winblad
b0d670aaa9
Merge pull request #11992 from kjellwinblad/kjell/backport/action_upgrade_downgrade_hooks
...
fix(action): upgrade and downgrade strategy
2023-11-21 19:14:37 +01:00
Thales Macedo Garitezi
9e1796ec4f
feat(gcp_pubsub_producer): migrate GCP PubSub producer to actions
...
Fixes https://emqx.atlassian.net/browse/EMQX-11157
2023-11-21 14:22:42 -03:00
Thales Macedo Garitezi
6c9a8461f7
Merge pull request #11989 from thalesmg/confluent-action-m-20231120
...
feat: add confluent connector/action
2023-11-21 11:01:40 -03:00
Thales Macedo Garitezi
068d151b14
Merge pull request #11983 from thalesmg/sync-r53-to-m-20231120
...
sync `release-53` to `master`
2023-11-21 09:02:51 -03:00
Thales Macedo Garitezi
759468efcf
Merge pull request #11986 from thalesmg/split-resource-opts-actions-r53-20231120
...
refactor: split `resource_opts` fields between connector and actions
2023-11-21 09:01:29 -03:00
Thales Macedo Garitezi
75ee3ced68
feat: add confluent connector/action
...
Fixes https://emqx.atlassian.net/browse/EMQX-11044
See also: https://emqx.atlassian.net/wiki/spaces/EMQX/pages/712179934/Confluent+Integration
2023-11-21 08:59:42 -03:00
Kjell Winblad
6030bf6fa5
fix(action): upgrade and downgrade strategy
...
This commit adds upgrade and downgrade hooks that are called when
upgrading from a bridge V1 to connector and action or the other way
around. The automatic translation is used if the callback is not
defined.
NOTE: Backported from master
2023-11-21 11:18:21 +01:00
Kjell Winblad
d214ae8772
fix: problems found by @thalesmg in code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-20 18:59:57 +01:00
Kjell Winblad
edbfe090c9
feat: add custom callback for creating connector from bridge V1 config
2023-11-20 17:23:51 +01:00
Kjell Winblad
3aa8044475
fix(action): upgrade and downgrade strategy
...
This commit fixes the upgrade and downgrade strategy when upgrading
from a bridge V1 to connector and action or the other way around so that
the custom callbacks get the complete unchanged input instead of the
result of the automatic translation. The automatic translation is used
if the callback is not defined.
2023-11-20 16:40:27 +01:00
Thales Macedo Garitezi
f40f6bc5dd
refactor: split `resource_opts` fields between connector and actions
2023-11-20 12:37:36 -03:00
Stefan Strigler
1f1d9e58c6
fix(emqx_connector): don't crash in API on delete with active channels
2023-11-20 16:23:46 +01:00
Thales Macedo Garitezi
53e796bbd0
Merge remote-tracking branch 'origin/release-53' into sync-r53-to-m-20231120
2023-11-20 11:49:52 -03:00
Kjell Winblad
79a764f117
fix: bridge to action upgrade fix up hook should run after upgrade
...
This commit changes how the `emqx_action_info` callback
`bridge_v1_to_action_fixup/1` works. It is now called after the
automatic upgrade instead of before. Since the full Bridge V1 config
might be needed to do the fixup, it is provided in a special field
`<<"__bridge_v1_conf__">>`. The `<<"__bridge_v1_conf__">>` field is
removed after the callback is called and can thus be ignored if it is
not needed.
2023-11-20 12:06:21 +01:00
Thales Macedo Garitezi
eb3f54184e
refactor: address review comments and avoid transformations without schema knowledge
2023-11-17 16:16:34 -03:00
Kjell Winblad
86c126ffcd
feat: callbacks for fixup after automatic Bridge V1 upgrade/downgrade
...
This commit adds callbacks to the emqx_action_info module for
doing fixes (such as changing a field name) after the automatic
split of a Bridge V1 config or the merge of connector and action
configs for the compatibility layer.
2023-11-17 16:10:08 -03:00
Ivan Dyachkov
65ba381fd8
Merge remote-tracking branch 'upstream/master' into 1115-sync-master-to-r53
2023-11-15 09:25:03 +01:00
Ivan Dyachkov
7c0e345d3a
Merge remote-tracking branch 'upstream/release-54'
2023-11-14 19:38:21 +01:00
Zaiming (Stone) Shi
88637f81d1
Merge pull request #11936 from zmstone/1112-readable-types
...
refactor(schema): keep type converters close
2023-11-14 17:56:07 +01:00
Thales Macedo Garitezi
d6e9bbb95c
fix(connector): validate connector name before converting ssl certs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11336
See also: https://github.com/emqx/emqx/pull/11540
2023-11-14 09:29:59 -03:00
Ivan Dyachkov
28a577ad09
chore: bump apps versions
2023-11-14 11:02:26 +01:00
Andrew Mayorov
2b36b48f98
feat(connector): use `emqx_schema_secret` as password field schema
2023-11-14 16:05:23 +07:00
Zaiming (Stone) Shi
4c5d64abc2
refactor(schema): keep type converters close
2023-11-13 15:26:30 +01:00
firest
548e39e799
refactor(sysk): refactor the syskeeper to v2 style
2023-11-10 09:52:38 +08:00
Ivan Dyachkov
0c91bec98d
chore: merge 'upstream/release-53'
2023-11-08 09:24:38 +01:00
Stefan Strigler
fa740a1927
docs: fix inconsistencies for bridge and connector operations
...
It's only `start` allowed for now. Also fixed some wordings.
2023-11-07 13:58:23 +01:00
Zaiming (Stone) Shi
f19904d43c
Merge pull request #11877 from zmstone/1102-rename-kafka-to-config
...
Rename connector channel related configs in bridge_v2 to 'parameters'
2023-11-07 13:41:55 +01:00
Zaiming (Stone) Shi
9a26c03a5c
refactor: unify top level field names for bridge v2
2023-11-06 21:45:08 +01:00
Thales Macedo Garitezi
2b8cf50a1d
chore: rename `bridges_v2` -> `actions` in the public facing APIs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11330
After feedback from Product team, we should rename `bridges_v2` to `actions` everywhere.
We'll start with the public facing APIs.
- HTTP API
- Hocon schema root key
2023-11-06 15:37:07 -03:00
Zaiming (Stone) Shi
600747b7e5
fix(bridge): do not allow dot in bridge name
...
also validate name at the API entry
2023-11-03 20:44:57 +01:00
Stefan Strigler
45bfcea6ae
Merge pull request #11866 from sstrigler/EMQX-11294-e-5-3-1-swaggers-several-misdescriptions-of-ap-is
...
EMQX 11294 swaggers several misdescriptions of APIs
2023-11-03 14:55:46 +01:00
Zaiming (Stone) Shi
afe6f79a66
feat(bridge): add description field to bridge and connector
2023-11-03 06:53:42 +01:00
Andrew Mayorov
910e81bc41
Merge pull request #10442 from keynslug/ft/EMQX-9257/placeholder
...
feat(tpl): split `emqx_placeholder` into a couple of modules
2023-11-02 22:50:05 +07:00
Andrew Mayorov
8e4585d64f
chore: move template modules to `emqx_utils`
...
Even though most of the time these modules will be used by
connectors, there are exceptions (namely, `emqx_rule_engine`).
Besides, they are general enough to land there, more so given
that `emqx_placeholder` is already there.
2023-11-02 17:11:12 +07:00
Andrew Mayorov
343b679741
feat(tpl): make escaping mechanism more foolproof
...
Treat "${$}" as literal "$". This allows to template express
strings, for example, of the form "${some_var_value}" where
`some_var_value` is interpolated from bindings.
2023-11-02 17:11:11 +07:00
Andrew Mayorov
f689d6c233
fix(tpl): ensure backward compat in `emqx_rule_engine`
...
Missing bindings in string templates will be rendered as "undefined",
as before. Rendering still assumes that missing binding with implicit
default (`undefined`) is an error.
This will also restore complete backward compat in `emqx_prometheus`.
2023-11-02 17:11:11 +07:00
Andrew Mayorov
1fcdfe991c
fix(tpl): add few missing `@doc`s + correct typespecs
2023-11-02 17:11:11 +07:00
Andrew Mayorov
a9693eada7
fix(tpl): rename `trivial` -> `is_const`
...
This is clearer. Former naming was a bit misleading.
2023-11-02 17:11:11 +07:00
Andrew Mayorov
7bb995f0c6
feat(tpl): support `:n` SQL parameters
2023-11-02 17:11:11 +07:00
Andrew Mayorov
49f5325c67
feat(tpl): unify validations / errors var representations
2023-11-02 17:11:10 +07:00
Andrew Mayorov
b812f9af5a
feat(tpl): use `emqx_connector_template` in `emqx_rule_engine` app
2023-11-02 17:11:10 +07:00
Andrew Mayorov
0538a77700
feat(tpl): use `emqx_connector_template` in `emqx_authn`, `emqx_authz`
...
This slightly changes semantics: now the attempt to create authenticator
with illegal bindings in templates will fail, instead of treating them
as literals. The runtime behaviour on the other hand should be the same.
2023-11-02 17:11:10 +07:00
Andrew Mayorov
28d55d72ca
feat(tpl): split `emqx_placeholder` into a couple of modules
...
Located under `emqx_connector` for now. The APIs of the new modules
are slightly different from the old ones. The new ones are more
explicit in terms of error handling. Also copy the according testsuite
from `emqx_plugin_libs` and enrich it for better coverage.
2023-11-02 17:11:09 +07:00
Stefan Strigler
29683072a1
fix(emqx_connector): remove `stop` and `restart` operations
2023-11-01 09:18:50 +01:00
Thales Macedo Garitezi
45a39d97c6
fix(bridges_v1): avoid create dangling connectors when updating bridges via api v1
...
Fixes https://emqx.atlassian.net/browse/EMQX-11291
2023-10-31 16:39:47 -03:00
Thales Macedo Garitezi
8eb822d898
test(bridges): add bridge v1 compatibility layer test suite
2023-10-31 13:21:01 -03:00
Ivan Dyachkov
b1ab213081
chore: merge 'upstream/release-53' into 1031-sync-r53
2023-10-31 11:06:25 +01:00
Stefan Strigler
176bbe88bc
fix: use `_producer` for AEH bridge type
2023-10-31 08:34:03 +01:00
Serge Tupchii
d94193ac15
fix(emqx_connector_schema): add kafka alias for kafka_producer
2023-10-30 19:14:58 +02:00
Serge Tupchii
0935bb6225
fix(emqx_connector): fix badarity error
2023-10-30 17:52:39 +02:00
Kjell Winblad
9dc3a169b3
feat: split bridges into a connector part and a bridge part
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.
Issues:
https://emqx.atlassian.net/browse/EMQX-10805
2023-10-30 14:48:47 +01:00
Ilya Averyanov
4f14e8df22
chore(source dir): split out postgresql connector
2023-10-13 16:58:57 +03:00
Thales Macedo Garitezi
b07dddd49e
fix(postgres): format unicode error messages from driver
...
Fixes https://emqx.atlassian.net/browse/EMQX-11024
Sample error:
```
{error, error, <<"42501">>, insufficient_privilege,
<<229,175,185,232,161,168,32,109,113,116,116,95,117,115,101,114,32,230,157,131,233,153,144,228,184,141,229,164,159>>,
[]}
```
2023-10-11 17:56:12 -03:00
Ivan Dyachkov
dafd7c6085
chore: bump apps versions
2023-09-21 10:58:42 +02:00
zhongwencool
2f1fa2e961
chore: unified slog message formatting to improve logging consistency
2023-09-20 18:13:00 +08:00
Ivan Dyachkov
24230a64df
chore: bump app versions
2023-09-04 11:39:21 +02:00
Serge Tupchii
ed9afe3345
refactor(emqx_connector): use `hocon_schema:override/2` to make pgsql 'username' field required
2023-09-01 19:56:07 +03:00
firest
0b066fa20c
fix(ldap): fix dependency problem
2023-08-14 16:59:37 +08:00
firest
efff585b82
feat(ldap-authn): add test suite for the LDAP authenticator
2023-08-03 18:56:41 +08:00
Ivan Dyachkov
63adeabf72
chore: bump app versions
2023-07-27 15:29:03 +02:00
Thales Macedo Garitezi
7a16ff4f04
fix(postgres_bridge): fix table existence check and handle sync_required
...
Fixes https://emqx.atlassian.net/browse/EMQX-10629
During health checking, we check whether tables in the SQL statement
exist. Such check was done by asking the backend to parse the
statement using a named prepared statements. Concurrent health checks
could then result in the error:
```erlang
{error,{error,error,<<"42P05">>,duplicate_prepared_statement,<<"prepared statement \"get_status\" already exists">>,[{file,<<"prepare.c">>},{line,<<"451">>},{routine,<<"StorePreparedStatement">>},{severity,<<"ERROR">>}]}}
```
This could lead to an inconsistent state in the driver process, which
would crash later when a message from the backend (`READY_FOR_QUERY`, "idle"):
```
2023-07-24T13:05:58.892043+00:00 [error] Generic server <0.2134.0> terminating. Reason: {'module could not be loaded',[{undefined,handle_message,[90,<<"I">>,...
```
Added calls to `epgsql:sync/1` for functions that could return
`{error, sync_required}`.
Also, redundant calls to `parse2` were removed to reduce the number of requests.
2023-07-25 09:15:14 -03:00
Kjell Winblad
f28510b3ad
refactor: HTTP connector into emqx_bridge_http app
2023-07-12 14:46:43 +02:00
firest
d01eee7fe4
chore: update changes && app version
2023-07-07 18:22:54 +08:00
firest
8cee75139d
fix(bridge): ensure the username of pgsql must exists
2023-07-07 18:06:28 +08:00
Thales Macedo Garitezi
ca435975de
fix(webhook): treat http status code 429 as recoverable
2023-06-30 09:46:03 -03:00
Thales Macedo Garitezi
59b109eb5c
fix(webhook): treat 404 and other error replies as errors in async requests
...
Fixes https://emqx.atlassian.net/browse/EMQX-10405
The problem here was that, for async requests, ehttpc responses of the form `{ok, 4__, _,
_}` and similar were being treated as successes.
2023-06-29 15:45:23 -03:00
Kjell Winblad
85a130f68b
refactor: move MySQL open source connector to its own app
2023-06-27 16:15:18 +02:00
Kjell Winblad
dd6aa96c99
refactor: move Redis open source connector to its own app
2023-06-27 10:24:40 +02:00
Kjell Winblad
b78a75bb5c
refactor: the MongoDB connector into its own application
2023-06-26 14:32:59 +02:00
Thales Macedo Garitezi
2f00cf7f84
Merge pull request #11107 from thalesmg/fix-mongo-health-check-reason-master
...
fix(mongo): return health check failure reason
2023-06-22 09:30:34 -03:00
Thales Macedo Garitezi
7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
...
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
Zaiming (Stone) Shi
c58a98954b
Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master
2023-06-22 11:05:51 +02:00
Thales Macedo Garitezi
18f0510353
fix(mongo): return health check failure reason
...
Fixes https://emqx.atlassian.net/browse/EMQX-10335
2023-06-21 15:09:37 -03:00
Paulo Zulato
8430ec673c
feat(kafka): check whether target topic exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-21 15:00:35 -03:00
Zaiming (Stone) Shi
7cf8a6c892
chore: bump app vsns
2023-06-21 16:36:51 +02:00
Thales Macedo Garitezi
dabefdb01b
fix(gcp_pubsub): redact request when errors occur
2023-06-19 15:59:00 -03:00
Paulo Zulato
c9a2ddf98c
feat(mysql): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
Paulo Zulato
9454af9a8b
feat(postgresql): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 11:12:10 -03:00
Stefan Strigler
0d6d441f4c
test(emqx_connector): start/stop test for webhook bridge
2023-06-14 09:56:50 +02:00
Stefan Strigler
b2a5065641
fix(emqx_connector): report errors in on_start handler
2023-06-13 16:57:08 +02:00
Andrew Mayorov
e6fb0203b4
refactor(pluglib): move connectivity checks to `emqx_connector_lib`
2023-06-09 14:44:37 +03:00
Andrew Mayorov
8919a6ef93
refactor(pluglib): provide SQL related utils in `emqx_utils_sql`
2023-06-09 14:44:37 +03:00
Andrew Mayorov
d6c1ee183f
refactor(pluglib): move `emqx_placeholder` to utils app
...
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
Zaiming (Stone) Shi
641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51
...
0607 merge master to release 51
2023-06-08 09:22:21 +02:00
JianBo He
0b95bc1c1b
Merge pull request #10907 from HJianBo/refactor-on-stop-simple
...
feat: refactored some bridges to avoid leaking resources part.2
2023-06-08 09:37:31 +08:00
Zaiming (Stone) Shi
ccd2589ff2
Merge remote-tracking branch 'origin/master' into release-51
2023-06-07 21:43:29 +02:00
Thales Macedo Garitezi
260fae296b
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9603
Rather than relying on a JWT worker to produce and refresh tokens, we
could just produce then on demand when pushing the messages to GCP
PubSub. That can generate a bit of extra work (as multiple processes
might realize it’s time to refresh the JWT and do so), but that
shouldn’t be much. In return, we avoid any possibility of not having
a fresh JWT when pushing messages.
2023-06-06 13:19:24 -03:00
Andrew Mayorov
95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys
...
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
2023-06-05 22:41:07 +03:00
Thales Macedo Garitezi
46393343e2
chore: use `timeout_duration` types for timer fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
JianBo He
dbc0cdce67
chore: dont allocate resource for simple connectiors
2023-06-05 10:59:00 +08:00
Thales Macedo Garitezi
0072f4a598
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
...
Example from flaky CI test:
```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
2023-06-01 17:13:06 -03:00
firest
cb31032e6d
chore: make dialyzer && test happy
2023-06-01 11:15:51 +08:00
firest
6081ce8b00
feat: refactored some bridges to avoid leaking resources during crashes at creation
2023-05-31 16:48:13 +08:00
Andrew Mayorov
7e7b50c5ba
refactor(mqttbridge): move into separate application
2023-05-30 17:56:34 +03:00
Andrew Mayorov
ebd612b194
chore: bump applications versions
...
* emqx_connector 0.1.25
* emqx_rule_engine 5.0.19
* emqx_ee_bridge 0.1.15
2023-05-30 17:21:45 +03:00
Andrew Mayorov
eed9358abd
chore: bump `ecpool` to 0.5.4
...
With fixed typings and empty pool handling.
2023-05-30 17:21:45 +03:00
Andrew Mayorov
a5fc26736d
refactor(mqttconn): split ingress/egress into 2 separate pools
...
Each with a more refined set of responsibilities, at the cost of slight
code duplication. Also provide two different config fields for each pool
size.
2023-05-30 17:21:44 +03:00
Andrew Mayorov
c7528e9b35
feat(mqttconn): add `pool_size` config parameter
...
That currently tunes the number of MQTT clients employed both for
subscriptions (if shared subscription is used) and for publishing to
a remote broker.
2023-05-30 17:21:44 +03:00
Andrew Mayorov
6e97dffdb8
feat(mqttconn): deprecate `mode` config parameter
...
It adds no value: the only mode was `cluster_shareload` and we just as
well can decide to "share" the load across cluster just by looking if
the remote topic is shared subcription filter or not.
2023-05-30 17:21:44 +03:00
Andrew Mayorov
4e6269bedb
feat(mqttconn): subscribe each worker if shared subcription
...
Also rename `subscriptions` -> `ingress` and `forwards` -> `egress` for
consistency with the config schema.
2023-05-30 14:50:05 +03:00
Andrew Mayorov
81e78516aa
feat(mqttconn): employ ecpool instead of a single worker
2023-05-30 14:50:05 +03:00
Andrew Mayorov
6967f621d8
fix(mqttconn): unify error interpretation in sync/async modes
...
Also move this logic to the mqtt connector itself, in order to avoid
dealing with extra callback layer.
2023-05-30 14:50:05 +03:00
Andrew Mayorov
67d703f8c5
refactor(mqttconn): simplify mqtt connector
...
Inline `emqx_connector_mqtt_msg` module code into
`emqx_connector_mqtt_worker` module, since it's not really used
anywhere else and does not provide any reusable abstractions.
2023-05-30 14:50:05 +03:00
Andrew Mayorov
bd956d00b6
feat(mqttconn): stop using gproc in hot path
...
Also drop fiddling with `mountpoint` since this option seems not to be
used anywhere.
2023-05-30 14:50:05 +03:00
Andrew Mayorov
4da0d83faf
chore(mqttconn): remove dead code
2023-05-30 14:50:04 +03:00
Zaiming (Stone) Shi
36e268c933
chore: bump app versions
2023-05-26 16:05:37 +02:00
Zaiming (Stone) Shi
cc5b4d3748
Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test
2023-05-26 15:58:16 +02:00
Thales Macedo Garitezi
194faefc8a
fix(webhook): keep `resource_opts.request_timeout` for webhook bridge (r5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10001
Recently, we unified request_timeout in a single field located at the
webhook connector schema. However, the correct fix would be to use
the resource_opts.request_timeout one, as that’s the only one that
allows infinity timeout.
2023-05-25 13:19:27 -03:00
JimMoen
28015597ee
Merge remote-tracking branch 'emqx/release-50' into merge-release-50
2023-05-24 19:34:12 +08:00
Zaiming (Stone) Shi
256cb73e43
Merge pull request #10792 from thalesmg/fix-webhook-bridge-health-check-clause-r50
...
fix(webhook): fix empty ehttpc worker list clause on health check (r5.0)
2023-05-24 08:46:59 +02:00
Thales Macedo Garitezi
25437adf02
fix(webhook): fix empty ehttpc worker list clause on health check (r5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9970
2023-05-23 14:06:40 -03:00
Kjell Winblad
ea2fea77d7
fix: RabbitMQ field not marked as required
...
This commit makes sure that the RabbitMQ password filed is marked as
required. This ensures that the user provides a password and that the
bridge does not throw a function clause exception if the password filed
is not set.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9974
2023-05-23 15:50:18 +02:00
Zaiming (Stone) Shi
732a7be187
Merge remote-tracking branch 'origin/release-50'
2023-05-22 17:46:54 +02:00