a752119d05
fix(prom_di): use bridge_v2 metrics api
2024-01-25 09:22:22 +08:00
fa842736d2
feat: refactor kinesis bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11461
2024-01-24 18:31:03 +01:00
67c9e9c8fc
fix(actions_api): reset metrics on all nodes
...
Fixes https://emqx.atlassian.net/browse/EMQX-11796
2024-01-23 10:40:02 -03:00
aafb683ec7
Merge pull request #12330 from emqx/bridge-v2-cassandra
...
refactor: split cassandra bridges to actions and connectors
2024-01-23 14:45:32 +08:00
e337e1dc40
feat(opents): improve the OpentsDB bridge to v2 style
2024-01-22 20:45:10 +08:00
b32c0fb0d8
refactor: split cassandra bridges to actions and connectors
2024-01-22 10:16:46 +08:00
dc15d37dcc
perf(emqx_bridge/connector): load and unload bridges/connectors in parallel
...
This should reduce app start/stop time, when a large number of bridges/connectors are not healthy.
2024-01-19 21:44:31 +02:00
a6568dec75
perf(emqx_bridge/connector): apply post config bridge/connector changes in parallel
...
This can greatly improve the performance when many bridges/connectors are being changed,
e.g. when a backup file is being imported.
Fixes: EMQX-11751
2024-01-19 21:44:31 +02:00
4068258c78
fix(bridge_api): explicitly state bpapi version ranges
...
Fixes https://emqx.atlassian.net/browse/EMQX-11767
The currently supported version for a node may not be available at the time it's probed.
2024-01-19 10:20:57 -03:00
e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489
...
feat: refactor MQTT bridge to source, action, and connector
2024-01-18 09:28:14 -03:00
2a41cad54f
fix(sources): remote irrelevant `resource_opts` fields for sources
...
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
2024-01-16 11:29:35 -03:00
a8f9e5676f
docs(mqtt_bridge): add API examples
2024-01-16 10:42:52 -03:00
60fab6ee45
refactor: attempt to improve function names
2024-01-16 10:42:52 -03:00
c6cd3adccb
refactor: fix type upgrade calls and move compatiblitly logic
...
Some bridge V1 to V2 calls were wrong but did not seem to cause issues
(perhaps due to locking test coverage). This commit also move
compatibility logic from the API module to the emqx_bridge_v2 module
where most of the compatibility logic exists.
2024-01-16 10:42:52 -03:00
440a543a85
docs: fix typo
2024-01-16 10:42:52 -03:00
938429f351
chore(mqtt_bridge): change schema to remote `remote` sub-fields and hide `local`
...
`local` is still needed for backwards compatibility
2024-01-16 10:42:52 -03:00
007af20a30
test(bridge_v2_api): adapt more tests to sources
2024-01-16 10:42:52 -03:00
fc88a1ed1e
test(sources_api): add some tests to cover `/sources` HTTP API
...
Also fixes a bug with `DELETE /sources/:id`
2024-01-16 10:42:52 -03:00
8f304d3456
test(bridge_v2_api): refactor suite to use CT matrix
2024-01-16 10:42:52 -03:00
cc24fe6e93
feat(mqtt_consumer): add support for rule engine `FROM`
2024-01-16 10:42:52 -03:00
28de7c89c7
feat: add `/sources*` HTTP APIs
2024-01-16 10:42:52 -03:00
6511693b2e
refactor(action_api): prepare for `/sources` HTTP API
2024-01-16 10:42:52 -03:00
7fc069da46
test: fix another broken test
2024-01-16 10:42:52 -03:00
ab1b0dda67
refactor: fix typo
2024-01-16 10:42:52 -03:00
697c8f5ee1
test: fix broken tests
2024-01-16 10:42:52 -03:00
cc34660ab9
fix(actions): use backward-compatible ids
2024-01-16 10:42:52 -03:00
f199a0f24a
feat: refactor MQTT bridge to source, action, and connector
...
This commit:
* refactors the MQTT V1 bridge into connector, source and action
* Extends the compatibility layer so it works for sources
* Fixes the MQTT bridge test suite so that all test cases passes
We still need to add a HTTP API handling sources. Also, we still need to
add HTTP API example schemes and examples for the MQTT
connector/action/source.
We should also make sure that we handle the corner cases of the MQTT V1
bridge automatic upgrade downgrade in a sufficiently good way:
* An error is currently thrown when converting an MQTT V1 bridge without
egress or ingress config.
* If there is a source and action with the same name we will currently
throw an error in the compatibility layer.
* We will also throw an error when converting an MQTT V1 bridge with
both ingress and egress.
The above is probably the right thing to do but we have to make sure
that we return a reasonable error to the user when this happens.
(partly)
Fixes:
https://emqx.atlassian.net/browse/EMQX-11489
2024-01-16 10:42:52 -03:00
ab66986f16
feat: add 'tags' field for action and connector
2024-01-16 13:57:48 +01:00
58a54adbb4
Merge remote-tracking branch 'origin/master' into release-55
2024-01-15 17:58:48 +01:00
ace443fc18
refactor: refactor es's action
2024-01-15 11:31:13 +08:00
e49d3ca50c
feat: support elasticsearch bridge
2024-01-15 09:33:44 +08:00
a5e7db793b
fix(mongodb_action): make `batch_size` hidden and fixed in the API
...
Fixes https://emqx.atlassian.net/browse/ED-1171
The old bridge schema already had this override, but it had not been ported to actions.
2024-01-12 10:04:32 -03:00
79a4a041e4
fix(bridge_api): correctly deobfuscate secrets during dry run
...
Fixes https://emqx.atlassian.net/browse/EMQX-11733
2024-01-11 15:42:14 -03:00
abd4873091
fix(mysql_bridge): allow deleting bridge when sql contains undefined fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-11655
2024-01-09 17:25:34 -03:00
ec34b6f41d
refactor: remove redis_type from redis action
2024-01-08 15:09:37 +08:00
3404f39fd2
chore: remove redundant redis action swagger example
2024-01-08 10:45:54 +08:00
7f57ec47d5
Merge pull request #12261 from lafirest/fix/iotdb
...
feat(iotdb): improve the IoTDB bridge to v2 style
2024-01-08 09:46:07 +08:00
e20b024b6e
fix(iotdb): fix iotdb testcases
2024-01-06 13:01:43 +08:00
d16458ccd0
feat(iotdb): improve the IoTDB bridge to v2 style
2024-01-05 16:54:31 +08:00
19e2ec9748
fix: update influxdb testcases
2024-01-03 18:39:29 +08:00
5e100f52b8
style: erlfmt all `rebar.config` files and `bin/nodetool`
2023-12-29 09:08:03 +08:00
a16bce8c24
fix: deobfuscate ft's secret keys in api
2023-12-22 13:13:44 +01:00
37930f1d3c
fix: emqx conf ctl load failed with connectors/actions
2023-12-21 22:09:53 +08:00
b61ccf3373
fix(bridges_v1): don't list v2-only bridges in API
...
Fixes https://emqx.atlassian.net/browse/EMQX-11614
2023-12-20 10:46:35 -03:00
3f06ebcaf7
Merge pull request #12181 from thalesmg/fix-fill-default-v1-api-return-r54-20231215
...
fix(bridge_v1_api): fill defaults for v2 raw configs and fix redis connector schema
2023-12-18 11:34:07 -03:00
c29ada4666
fix(redis_bridge): fix connector schema and action info transformations
2023-12-15 17:33:39 -03:00
a5978aa39a
Merge pull request #12184 from sstrigler/EMQX-11587-use-common-functions-to-create-api-schemata-and-examples-in-action-implementations
...
refactor(emqx_bridge): common api_fields fn for actions
2023-12-15 19:02:14 +01:00
432ddc5a3b
fix(bridge_v1_schema): undo changing v1 schema to avoid confusion
2023-12-15 12:55:34 -03:00
ec74f77ad8
refactor(emqx_bridge): common api_fields fn for actions
2023-12-15 14:23:33 +01:00
f94b943ec2
fix(bridge_v1_api): fill defaults for v2 raw configs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11593
2023-12-15 10:18:42 -03:00
b703b671eb
Merge pull request #12170 from thalesmg/feat-bump-actions-failed-unk-r54-20231214
...
feat(rule_engine): also bump `unknown` counter for unrecoverable action errors
2023-12-15 09:40:03 -03:00
57a8f2351d
feat(rule_engine): also bump `unknown` counter for unrecoverable action errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-11494
2023-12-14 13:42:28 -03:00
a5d848515b
fix: project sub-fields in action/connector `resource_opts` for some bridges
...
Fixes https://emqx.atlassian.net/browse/EMQX-11589
2023-12-14 12:09:32 -03:00
edec431a6c
Merge pull request #12144 from thalesmg/fix-conn-schema-query-mode-r54-20231211
...
fix({action,connector}_schema): fix `resource_opts` subfields for connectors and actions, and remove redundant subfields from actions, use query mode from actions
2023-12-13 09:19:46 -03:00
3c43db2828
Merge pull request #12147 from zhongwencool/bridge-description
...
fix:add descriptions fields to bridge_v1
2023-12-13 09:13:17 +08:00
d7b96af893
Merge pull request #12145 from thalesmg/fix-postgres-res-opts-schema-r54-20231211
...
fix(postgres_schema): fix usages of `resource_opts` in connector and action schemas
2023-12-12 11:02:55 -03:00
71011ad084
Merge pull request #12142 from sstrigler/EMQX-11487-e-5-4-0-mongodb-test-connection-failed
...
mongodb test connection failed
2023-12-12 09:05:04 +01:00
363055a32e
fix: add descriptions fields to bridge_v1
2023-12-12 10:18:18 +08:00
b80c9b0863
fix(actions_schema): remove redundant `resource_opts` subfields for actions
...
Buffer workers don't use those fields.
2023-12-11 16:58:43 -03:00
593283df93
fix(redis_bridge_schema): use correct `resource_opts` subfields for connector and action
2023-12-11 16:58:43 -03:00
4fc14fb4ba
Merge pull request #12143 from thalesmg/fix-reply-to-async-r54-20231211
...
fix(actions): increment rule statistics even if channel is not installed
2023-12-11 15:35:35 -03:00
f7adaa8911
fix(postgres_schema): fix usages of `resource_opts` in connector and action schemas
...
Fixes https://emqx.atlassian.net/browse/EMQX-11569
2023-12-11 15:03:20 -03:00
fbbb55633d
fix(connector_schema): remove `query_mode` from `resource_opts`
...
The connector query mode is inferred during its creation, and later it must be overridden
by an action, anyway.
2023-12-11 14:13:40 -03:00
b4a5c141ad
fix(actions): use action query mode instead of connector's query mode
2023-12-11 14:10:51 -03:00
2495f59c91
fix(actions): increment rule statistics even if channel is not installed
...
Fixes new bug posted after https://emqx.atlassian.net/browse/EMQX-11494 was already fixed.
Also reduces the usage of error throwing for flow control a bit.
2023-12-11 14:01:30 -03:00
a7c344ba29
fix(emqx_bridge): remove unused
2023-12-11 14:58:09 +01:00
31a7301f6f
refactor(emqx_bridge): be more generic in to_bridge_v1_config
2023-12-11 14:50:38 +01:00
6143b3a2be
fix(emqx_bridge_mongodb): use correct config when calling downgrade_type
...
Also limit result in mongodb ..._to_bridge_v1_config to not include more fields
than what's in the schema (most notably `description`).
2023-12-11 14:08:45 +01:00
5d7ae4b980
Merge pull request #12132 from sstrigler/EMQX-11154-bridge-v-2-my-sql-support
...
feat(emqx_bridge_mysql): port to shared connectors
2023-12-11 13:56:34 +01:00
ddbb8560fa
fix(dialyzer): batch 2
2023-12-08 17:59:55 +01:00
71607aa2ad
feat(emqx_bridge_mysql): port to shared connectors
2023-12-08 16:15:18 +01:00
c274450c5e
Merge pull request #12118 from thalesmg/fix-missing-type-name-api-actions-r54-20231206
...
fix(actions_api): add missing fields to API spec and returned data
2023-12-07 09:34:29 -03:00
33a7282cdd
fix(dialyzer): only include eunit when TEST is defined
2023-12-06 20:39:26 +01:00
fdeedc360e
fix(actions_api): add missing fields to `POST` API schema
2023-12-06 11:13:09 -03:00
2757b95850
fix(actions_api): fill raw config defaults before returning
...
Fixes https://emqx.atlassian.net/browse/EMQX-11541
2023-12-06 11:13:09 -03:00
f489de8860
Merge pull request #12100 from thalesmg/fix-password-action-probe-m-20231205
...
fix(bridges/actions api): correctly deobfuscate passwords when probing
2023-12-05 14:25:32 -03:00
2579f8ea92
Merge pull request #12096 from thalesmg/fix-delete-deps-rules-m-20231204
...
fix(api): return list of dependent rule ids when trying to delete bridge/action
2023-12-05 14:24:50 -03:00
e42f4155d3
fix(bridges/actions api): correctly deobfuscate passwords when probing
...
Fixes https://emqx.atlassian.net/browse/EMQX-11533
2023-12-05 11:49:50 -03:00
1b68fbff05
fix(api): return list of dependent rule ids when trying to delete bridge/action
...
Fixes https://emqx.atlassian.net/browse/EMQX-11523
2023-12-05 11:47:26 -03:00
b2cd8aaab7
refactor: use single clause
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-12-04 10:02:05 -03:00
ac2c99f996
test(connector): add test to ensure connector schema has `resource_opts`
...
https://emqx.atlassian.net/browse/EMQX-11509
2023-12-04 10:02:05 -03:00
956b25cc4c
Merge pull request #12071 from zmstone/1201-sync-release-54
...
1201 sync release 54
2023-12-01 14:38:47 +01:00
cb60880bab
Merge pull request #12067 from thalesmg/fix-kafka-check-client-m-20231130
...
fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
2023-12-01 09:55:12 -03:00
7f4d91d490
Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54
2023-12-01 08:32:22 +01:00
0388e1c1c4
fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
...
Fixes https://emqx.atlassian.net/browse/EMQX-11494
2023-11-30 17:51:32 -03:00
9fd2fa95a8
chore: bump apps versions
2023-11-30 20:01:12 +01:00
684d637fed
test(bridge_api): workaround strange config syncing problem
...
For some unknown reason, this test has difficulties in syncing the config correctly
between the nodes, while the equivalent in bridge_v2_api_SUITE doesn't.
2023-11-30 19:51:35 +01:00
ec10c51073
Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53
2023-11-30 19:51:12 +01:00
dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26
...
1129 prepare for OTP 26
2023-11-30 19:38:40 +01:00
14644988e0
chore: change triple-quotes to single-quotes
2023-11-29 16:15:18 +01:00
0aec2f7605
feat: redis bridge v2
2023-11-29 16:06:46 +08:00
891ecc179d
chore: fix flaky tests
2023-11-28 10:34:44 +08:00
c8b5c51bbc
chore: fix failed test cases
2023-11-28 09:53:46 +08:00
cdb90ebe6b
feat: rename webhook bridge to http bridge
2023-11-28 09:53:46 +08:00
dc99651690
test(bridge): ensure almost test cases passed
2023-11-28 09:53:46 +08:00
96af7a74e8
feat: impl the http bridge v2
2023-11-28 09:53:46 +08:00
ac370fed61
Merge pull request #12035 from sstrigler/EMQX-11382-a-new-field-should-be-added-to-the-interface-of-the-connector-to-return-the-actions-associated-with-it
...
feat(emqx_connector): add field 'actions' in API response
2023-11-27 17:39:44 +01:00
048f4724a9
feat(emqx_connector): add field 'actions' in API response
...
Also unify schemas, use emqx_connector_schema for the generic parts.
2023-11-27 16:09:17 +01:00
6acdc6e432
feat: add `/actions/:id/metrics/`, `/actions/:id/metrics/reset` APIs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11381
2023-11-27 11:33:11 -03:00
66945dcc5c
fix: address more comments from @thalesmg
2023-11-24 20:31:29 +01:00
JimMoen