Commit Graph

869 Commits

Author SHA1 Message Date
Kjell Winblad
26c988fe11
Merge pull request #13077 from kjellwinblad/kjell/fix_action_update_race_issue/EMQX-12376
fix: action config update would sometimes not be reflected in connector
2024-05-23 09:11:48 +02:00
Thales Macedo Garitezi
4094032649 fix(bridge v2 api): don't attempt to start disabled connector when starting action/source
Fixes https://emqx.atlassian.net/browse/EMQX-12435
2024-05-22 13:33:55 -03:00
Kjell Winblad
917474f694 fix: action config update would sometimes not be reflected in connector
Before this commit the following happened sometimes:

1. action status is connected
2. action config is updated to something that should change the status to
   disconnected
3. action status is still connected and the old config is being used by
   the connector even though the config has been correctly updated.

The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).

The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00
Andrew Mayorov
5b2c2b336c
feat(s3-bridge): meld 2 separate actions into a single one
Discriminated through `mode` parameter. Also rename "simple" action to
a "direct" upload mode.
2024-05-17 09:34:21 +02:00
Kjell Winblad
e32745bca6 test(apply rule trace): add test case for republish and console print 2024-05-03 15:02:37 +02:00
Thales Macedo Garitezi
dd6566f3c5
Merge pull request #12948 from thalesmg/fix-http-bridge-header-update-obfuscate-r57-20240429
fix(http connector): deobfuscate sensitive headers
2024-04-30 13:34:15 -03:00
Andrew Mayorov
ccbcc0c4e3
feat(s3-bridge): implement aggregated upload action 2024-04-30 10:48:15 +02:00
Thales Macedo Garitezi
a847389159 fix(http connector): deobfuscate sensitive headers
Fixes https://emqx.atlassian.net/browse/EMQX-12213
2024-04-29 15:20:46 -03:00
JianBo He
69bdcd2f24
Merge pull request #12898 from emqx/iotdb-1.3.0
feat: support iotdb 1.3.0
2024-04-22 10:56:06 +08:00
Zaiming (Stone) Shi
a8b1224225
Merge pull request #12893 from zmstone/0416-add-is-template-flag-to-dashboard-schema
0416 add `is_template` flag to dashboard schema
2024-04-19 10:41:43 +02:00
zhongwencool
c0521fd250 feat: support iotdb 1.3.0 2024-04-19 08:54:18 +08:00
Kjell Winblad
2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
zmstone
c96ae8dd23 fix: return 503 if bridge bpapi call timeout 2024-04-17 20:34:33 +02:00
Kjell Winblad
cf56050759 feat: avoid mixing request with and without the stop_after_render flag
Previously a batch of requests that was sent to a connector could
contain both requests with the stop_after_rendering flag and requests
without this flag. When this happened a warning message was generated and
the stop_after_render flags for the batch would be ignored. This commit
fixes so that a mixed batch is never created so there is no longer any
need for a warning message or ignoring flags.
2024-04-17 16:21:21 +02:00
Ivan Dyachkov
3ef160eed2 Merge remote-tracking branch 'upstream/release-56' into 0415-sync-release-56 2024-04-15 08:09:03 +02:00
Shawn
9d1a69aaa9 fix: cannot import retained messages 2024-04-07 17:03:14 +08:00
Thales Macedo Garitezi
60cad74286 feat(resource): non-blocking channel health checks
Fixes https://emqx.atlassian.net/browse/EMQX-12015

Continuation of https://github.com/emqx/emqx/pull/12812
2024-04-04 16:13:30 -03:00
Shawn
319ec50c0d fix: source bridges missing after restore the backup files 2024-04-03 18:26:51 +08:00
Thales Macedo Garitezi
bade09b56e feat(resource manager): perform non-blocking resource health checks
Fixes https://emqx.atlassian.net/browse/EMQX-12015

This introduces only _resource_ non-blocking health checks.  _Channel_ non-blocking health
checks may be introduced later.
2024-04-01 14:46:15 -03:00
Ivan Dyachkov
db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Kjell Winblad
94c6e5bb38
Merge pull request #12727 from kjellwinblad/kjell/emqx_connector_info/EMQX-11427
refactor: add emqx_connector_info behavior
2024-03-25 14:31:40 +01:00
Kjell Winblad
2b684c7bcc fix(emqx_connector_info): refactoring based on thalesmg's comments 2024-03-22 09:29:26 +01:00
Kjell Winblad
89befa580e refactor(emqx_connector_info): remove code that is no longer needed 2024-03-21 11:58:54 +01:00
Ivan Dyachkov
f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
JianBo He
2a611e4508 chore: fix the error api desc 2024-03-18 18:27:28 +08:00
JianBo He
485d8dbbdb feat: add the get /source_types endpoint to return all supported source 2024-03-18 18:11:06 +08:00
Thales Macedo Garitezi
b587ebac0c fix(bridges): fix default value for enable when attempting operations
Fixes https://emqx.atlassian.net/browse/EMQX-11999

Although the frontend doesn't send `enable` when creating a bridge/action/connector, the
default value in the schema is `true`, so when attempting to fetch it from the raw config
we should adhere to that default.
2024-03-13 11:25:22 -03:00
Thales Macedo Garitezi
1edf284fed test: trigger hocon validators when checking schema 2024-03-12 14:32:16 -03:00
Thales Macedo Garitezi
79d7821222
Merge pull request #12655 from thalesmg/fix-kconsu-status-r56-20240306
fix(kafka_consumer): check client connectivity
2024-03-08 10:53:44 -03:00
Thales Macedo Garitezi
963e0de0c3 fix(kafka_consumer): check client connectivity
Fixes https://emqx.atlassian.net/browse/EMQX-11945
2024-03-07 14:22:03 -03:00
JianBo He
f1e9da9048
Merge pull request #12604 from thalesmg/test-gcp-consu-tm-m-20240227
test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
2024-03-06 09:13:41 +08:00
Thales Macedo Garitezi
676df7eb30 test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
Checks that, if a migrated bridge originally has a `topic_mapping` and is later updated
with V2 APIs (without topic mapping in the input), then the new V2 `topic` field prevails.
2024-03-05 09:12:24 -03:00
JimMoen
9143d5994d
feat: refactor MS SQL Server bridge to connector and action 2024-03-04 17:13:21 +08:00
zhongwencool
d56fb22208
Merge pull request #12595 from thalesmg/kafka-consumer-source-m-20240223
feat: migrate kafka consumer bridge to source + connector
2024-03-01 10:11:16 +08:00
zhongwencool
7f1b4cef27 feat: pulsar bridge v2 2024-02-29 16:23:37 +08:00
Thales Macedo Garitezi
6b9844ae82 feat: migrate kafka consumer bridge to source + connector
Fixes https://emqx.atlassian.net/browse/EMQX-11848
2024-02-27 17:52:33 -03:00
zhongwencool
54c542c795 chore: rename _probe_ to t_probe_ 2024-02-27 17:57:15 +08:00
zhongwencool
c67f2130f2 fix: check connector and bridge_v2 with the right schema 2024-02-27 17:57:10 +08:00
Zaiming (Stone) Shi
2adfefe297 docs: stop generating schema doc for 'bridges' root 2024-02-23 14:51:57 +01:00
Zaiming (Stone) Shi
46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
Thales Macedo Garitezi
205b97f732 fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config
Fixes https://emqx.atlassian.net/browse/EMQX-11891
2024-02-22 10:27:28 -03:00
Kjell Winblad
7bb9d5d8f6
Merge pull request #12543 from kjellwinblad/kjell/refactor/dynamodb/EMQX-11456
feat: refactor DynamoDB bridge to connector and action
2024-02-22 09:47:57 +01:00
Andrew Mayorov
7feb444e31
Merge pull request #12557 from keynslug/fix/s3-bridge-startup
fix(s3-bridge): anticipate repeated connector start requests
2024-02-21 15:13:48 +01:00
Kjell Winblad
6561d989d6 feat: refactor DynamoDB bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11456
2024-02-21 14:58:48 +01:00
Andrew Mayorov
6b97983303
fix(s3-bridge): anticipate repeated connector start requests
Starting disconnected connector manually implies that there's no
attempt to stop it first.
2024-02-21 14:23:29 +01:00
Zaiming (Stone) Shi
415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
Thales Macedo Garitezi
4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
Zaiming (Stone) Shi
333eb34300 test: call emqx_resource:remove_local 2024-02-21 10:53:22 +01:00
Andrew Mayorov
d8b6ecd185
fix(bridge-s3): expose connector-level resource_opts properly 2024-02-21 10:49:04 +01:00
zhongwencool
ee02079661
Merge pull request #12548 from zhongwencool/fix-rabbitmq-source-crash
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 16:34:43 +08:00