acb19a06cf
Merge remote-tracking branch 'origin/release-57'
2024-05-21 15:04:46 +02:00
5b2c2b336c
feat(s3-bridge): meld 2 separate actions into a single one
...
Discriminated through `mode` parameter. Also rename "simple" action to
a "direct" upload mode.
2024-05-17 09:34:21 +02:00
401f0fa84b
Merge branch 'release-57' into sync-r57-m-20240508
2024-05-09 09:13:30 -03:00
3b655f56cb
fix(auth,http): improve URI handling
2024-05-04 09:47:13 +03:00
e32745bca6
test(apply rule trace): add test case for republish and console print
2024-05-03 15:02:37 +02:00
dd6566f3c5
Merge pull request #12948 from thalesmg/fix-http-bridge-header-update-obfuscate-r57-20240429
...
fix(http connector): deobfuscate sensitive headers
2024-04-30 13:34:15 -03:00
ccbcc0c4e3
feat(s3-bridge): implement aggregated upload action
2024-04-30 10:48:15 +02:00
a847389159
fix(http connector): deobfuscate sensitive headers
...
Fixes https://emqx.atlassian.net/browse/EMQX-12213
2024-04-29 15:20:46 -03:00
69bdcd2f24
Merge pull request #12898 from emqx/iotdb-1.3.0
...
feat: support iotdb 1.3.0
2024-04-22 10:56:06 +08:00
a8b1224225
Merge pull request #12893 from zmstone/0416-add-is-template-flag-to-dashboard-schema
...
0416 add `is_template` flag to dashboard schema
2024-04-19 10:41:43 +02:00
c0521fd250
feat: support iotdb 1.3.0
2024-04-19 08:54:18 +08:00
2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
...
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
c96ae8dd23
fix: return 503 if bridge bpapi call timeout
2024-04-17 20:34:33 +02:00
cf56050759
feat: avoid mixing request with and without the stop_after_render flag
...
Previously a batch of requests that was sent to a connector could
contain both requests with the stop_after_rendering flag and requests
without this flag. When this happened a warning message was generated and
the stop_after_render flags for the batch would be ignored. This commit
fixes so that a mixed batch is never created so there is no longer any
need for a warning message or ignoring flags.
2024-04-17 16:21:21 +02:00
3ef160eed2
Merge remote-tracking branch 'upstream/release-56' into 0415-sync-release-56
2024-04-15 08:09:03 +02:00
9d1a69aaa9
fix: cannot import retained messages
2024-04-07 17:03:14 +08:00
60cad74286
feat(resource): non-blocking channel health checks
...
Fixes https://emqx.atlassian.net/browse/EMQX-12015
Continuation of https://github.com/emqx/emqx/pull/12812
2024-04-04 16:13:30 -03:00
319ec50c0d
fix: source bridges missing after restore the backup files
2024-04-03 18:26:51 +08:00
bade09b56e
feat(resource manager): perform non-blocking resource health checks
...
Fixes https://emqx.atlassian.net/browse/EMQX-12015
This introduces only _resource_ non-blocking health checks. _Channel_ non-blocking health
checks may be introduced later.
2024-04-01 14:46:15 -03:00
db9efb9317
chore: bump apps versions
2024-03-28 10:19:09 +01:00
94c6e5bb38
Merge pull request #12727 from kjellwinblad/kjell/emqx_connector_info/EMQX-11427
...
refactor: add emqx_connector_info behavior
2024-03-25 14:31:40 +01:00
2b684c7bcc
fix(emqx_connector_info): refactoring based on thalesmg's comments
2024-03-22 09:29:26 +01:00
89befa580e
refactor(emqx_connector_info): remove code that is no longer needed
2024-03-21 11:58:54 +01:00
f2dc940436
Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56
2024-03-19 15:20:08 +01:00
2a611e4508
chore: fix the error api desc
2024-03-18 18:27:28 +08:00
485d8dbbdb
feat: add the `get /source_types` endpoint to return all supported source
2024-03-18 18:11:06 +08:00
b587ebac0c
fix(bridges): fix default value for `enable` when attempting operations
...
Fixes https://emqx.atlassian.net/browse/EMQX-11999
Although the frontend doesn't send `enable` when creating a bridge/action/connector, the
default value in the schema is `true`, so when attempting to fetch it from the raw config
we should adhere to that default.
2024-03-13 11:25:22 -03:00
1edf284fed
test: trigger hocon validators when checking schema
2024-03-12 14:32:16 -03:00
79d7821222
Merge pull request #12655 from thalesmg/fix-kconsu-status-r56-20240306
...
fix(kafka_consumer): check client connectivity
2024-03-08 10:53:44 -03:00
963e0de0c3
fix(kafka_consumer): check client connectivity
...
Fixes https://emqx.atlassian.net/browse/EMQX-11945
2024-03-07 14:22:03 -03:00
f1e9da9048
Merge pull request #12604 from thalesmg/test-gcp-consu-tm-m-20240227
...
test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
2024-03-06 09:13:41 +08:00
676df7eb30
test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
...
Checks that, if a migrated bridge originally has a `topic_mapping` and is later updated
with V2 APIs (without topic mapping in the input), then the new V2 `topic` field prevails.
2024-03-05 09:12:24 -03:00
9143d5994d
feat: refactor MS SQL Server bridge to connector and action
2024-03-04 17:13:21 +08:00
d56fb22208
Merge pull request #12595 from thalesmg/kafka-consumer-source-m-20240223
...
feat: migrate kafka consumer bridge to source + connector
2024-03-01 10:11:16 +08:00
7f1b4cef27
feat: pulsar bridge v2
2024-02-29 16:23:37 +08:00
6b9844ae82
feat: migrate kafka consumer bridge to source + connector
...
Fixes https://emqx.atlassian.net/browse/EMQX-11848
2024-02-27 17:52:33 -03:00
54c542c795
chore: rename _probe_ to t_probe_
2024-02-27 17:57:15 +08:00
c67f2130f2
fix: check connector and bridge_v2 with the right schema
2024-02-27 17:57:10 +08:00
2adfefe297
docs: stop generating schema doc for 'bridges' root
2024-02-23 14:51:57 +01:00
46877e979b
chore: update copyright-year
2024-02-23 08:21:06 +01:00
205b97f732
fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config
...
Fixes https://emqx.atlassian.net/browse/EMQX-11891
2024-02-22 10:27:28 -03:00
7bb9d5d8f6
Merge pull request #12543 from kjellwinblad/kjell/refactor/dynamodb/EMQX-11456
...
feat: refactor DynamoDB bridge to connector and action
2024-02-22 09:47:57 +01:00
7feb444e31
Merge pull request #12557 from keynslug/fix/s3-bridge-startup
...
fix(s3-bridge): anticipate repeated connector start requests
2024-02-21 15:13:48 +01:00
6561d989d6
feat: refactor DynamoDB bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11456
2024-02-21 14:58:48 +01:00
6b97983303
fix(s3-bridge): anticipate repeated connector start requests
...
Starting disconnected connector manually implies that there's no
attempt to stop it first.
2024-02-21 14:23:29 +01:00
415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
...
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
...
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
333eb34300
test: call emqx_resource:remove_local
2024-02-21 10:53:22 +01:00
d8b6ecd185
fix(bridge-s3): expose connector-level `resource_opts` properly
2024-02-21 10:49:04 +01:00
ee02079661
Merge pull request #12548 from zhongwencool/fix-rabbitmq-source-crash
...
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 16:34:43 +08:00
2a7a662c03
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 15:30:05 +08:00
d469f4158e
chore: bump app vsns
2024-02-20 16:53:57 -03:00
86c85f3b45
Merge pull request #12512 from kjellwinblad/kjell/refactor/hstreamdb/EMQX-11458
...
feat: refactor HStreamDB bridge to connector and action
2024-02-20 13:30:29 +01:00
146d89bd89
feat: convert `gcp_pubsub_consumer` to connector/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 17:54:26 -03:00
11e4a295ed
fix: use connector type for constructing resource id
2024-02-19 14:42:35 -03:00
9a32895a1a
feat: convert `gcp_pubsub_consumer` to connector/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
8ae0e78786
refactor: split clickhouse bridges to actions and connectors
2024-02-18 10:01:53 +08:00
f57f617ba3
refactor(schema): ensure roots/0 and namespace/0 for all schema modules
2024-02-16 11:35:32 +01:00
a153d758c3
feat: refactor HStreamDB bridge to connector and action
...
This commit also upgrades the hstreamdb_erl driver library and change
the action/bridge to use the new hstreamdb_erl.
Much of the code for the new API is copied from:
be1a1604dd/lib-ee/emqx_rule_actions/src/emqx_backend_hstreamdb_actions.erlhttps://emqx.atlassian.net/browse/EMQX-11458
2024-02-14 17:34:35 +01:00
366827390e
Merge pull request #12488 from kjellwinblad/kjell/refactor/rocketmq_bridge/EMQX-11467
...
feat: refactor RocketMQ bridge to connector and action
2024-02-13 16:19:21 +01:00
a9fdf9f1f9
Merge pull request #12495 from keynslug/ft/EMQX-11830/s3-bridge-simple
...
feat(s3): introduce S3 connector and action
2024-02-12 16:05:22 +01:00
802c760406
feat(s3): introduce S3 connector and action
...
This is a trivial connector based on `emqx_s3` and simple action
that maps each incoming event into an S3 object upload. Due to
current `emqx_s3` limitation this bridge is compatible with
backends providing S3 API with path-style bucket access.
2024-02-09 14:16:32 +01:00
64367d834b
test(bridge_v2): actually check schema
2024-02-08 13:39:31 -03:00
e284a83f73
feat: refactor RocketMQ bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11467
2024-02-08 16:21:58 +01:00
b444c82a42
feat: rabbitmq bridge v2 integration
2024-02-06 07:57:12 +08:00
e5b14a38ed
Merge pull request #12472 from thalesmg/fix-bridgev2-api-multicall-m-20240202
...
fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
2024-02-05 10:28:28 -03:00
76d242df9b
fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
...
Fixes https://emqx.atlassian.net/browse/EMQX-11834
2024-02-02 17:36:08 -03:00
76401a302a
fix(config): apply config upgrade to deprecated configs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11845
Fixes https://github.com/emqx/emqx/issues/12452
2024-02-02 15:12:08 -03:00
fa0baf2f38
Merge pull request #12439 from kjellwinblad/kjell/refactor/oracle_bridge/EMQX-11464
...
feat: refactor Oracle bridge to connector and action
2024-02-02 10:53:15 +01:00
24d8a9ab00
Merge pull request #12449 from lafirest/feat/td
...
feat(tdengine): improve the TDengine bridge to v2 style
2024-02-02 16:17:01 +08:00
dfad020c49
feat(tdengine): improve the TDengine bridge to v2 style
2024-02-01 23:14:24 +08:00
a4272c71dc
feat: refactor Oracle bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11464
2024-02-01 16:03:05 +01:00
f9d16340e0
fix: don't add disable bridge to connector's channel
2024-02-01 09:12:43 +08:00
2d693402c5
refactor: split greptimedb bridge to actions and connectors
2024-01-26 11:01:53 +02:00
5bf37cbe0a
Merge pull request #12376 from kjellwinblad/kjell/kinesis
...
feat: refactor kinesis bridge to connector and action
2024-01-25 14:58:40 +01:00
6a21766ce3
refactor: split cassandra bridges to actions and connectors
2024-01-25 16:01:44 +08:00
dad8a32e0b
feat(opents): improve the OpentsDB bridge to v2 style
2024-01-25 16:01:43 +08:00
a752119d05
fix(prom_di): use bridge_v2 metrics api
2024-01-25 09:22:22 +08:00
fa842736d2
feat: refactor kinesis bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11461
2024-01-24 18:31:03 +01:00
67c9e9c8fc
fix(actions_api): reset metrics on all nodes
...
Fixes https://emqx.atlassian.net/browse/EMQX-11796
2024-01-23 10:40:02 -03:00
aafb683ec7
Merge pull request #12330 from emqx/bridge-v2-cassandra
...
refactor: split cassandra bridges to actions and connectors
2024-01-23 14:45:32 +08:00
e337e1dc40
feat(opents): improve the OpentsDB bridge to v2 style
2024-01-22 20:45:10 +08:00
b32c0fb0d8
refactor: split cassandra bridges to actions and connectors
2024-01-22 10:16:46 +08:00
dc15d37dcc
perf(emqx_bridge/connector): load and unload bridges/connectors in parallel
...
This should reduce app start/stop time, when a large number of bridges/connectors are not healthy.
2024-01-19 21:44:31 +02:00
a6568dec75
perf(emqx_bridge/connector): apply post config bridge/connector changes in parallel
...
This can greatly improve the performance when many bridges/connectors are being changed,
e.g. when a backup file is being imported.
Fixes: EMQX-11751
2024-01-19 21:44:31 +02:00
4068258c78
fix(bridge_api): explicitly state bpapi version ranges
...
Fixes https://emqx.atlassian.net/browse/EMQX-11767
The currently supported version for a node may not be available at the time it's probed.
2024-01-19 10:20:57 -03:00
e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489
...
feat: refactor MQTT bridge to source, action, and connector
2024-01-18 09:28:14 -03:00
2a41cad54f
fix(sources): remote irrelevant `resource_opts` fields for sources
...
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
2024-01-16 11:29:35 -03:00
a8f9e5676f
docs(mqtt_bridge): add API examples
2024-01-16 10:42:52 -03:00
60fab6ee45
refactor: attempt to improve function names
2024-01-16 10:42:52 -03:00
c6cd3adccb
refactor: fix type upgrade calls and move compatiblitly logic
...
Some bridge V1 to V2 calls were wrong but did not seem to cause issues
(perhaps due to locking test coverage). This commit also move
compatibility logic from the API module to the emqx_bridge_v2 module
where most of the compatibility logic exists.
2024-01-16 10:42:52 -03:00
440a543a85
docs: fix typo
2024-01-16 10:42:52 -03:00
938429f351
chore(mqtt_bridge): change schema to remote `remote` sub-fields and hide `local`
...
`local` is still needed for backwards compatibility
2024-01-16 10:42:52 -03:00
007af20a30
test(bridge_v2_api): adapt more tests to sources
2024-01-16 10:42:52 -03:00
fc88a1ed1e
test(sources_api): add some tests to cover `/sources` HTTP API
...
Also fixes a bug with `DELETE /sources/:id`
2024-01-16 10:42:52 -03:00
8f304d3456
test(bridge_v2_api): refactor suite to use CT matrix
2024-01-16 10:42:52 -03:00
cc24fe6e93
feat(mqtt_consumer): add support for rule engine `FROM`
2024-01-16 10:42:52 -03:00
28de7c89c7
feat: add `/sources*` HTTP APIs
2024-01-16 10:42:52 -03:00
6511693b2e
refactor(action_api): prepare for `/sources` HTTP API
2024-01-16 10:42:52 -03:00
7fc069da46
test: fix another broken test
2024-01-16 10:42:52 -03:00
ieQu1