yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,748 Commits 393 Branches 684 Tags 74 MiB
f47dacc5d0
Commit Graph

553 Commits

Author SHA1 Message Date
firest
51e8c955ff fix(connector): make the connector id unique when doing probe testing 2024-01-23 13:03:12 +08:00
firest
d221f34ce8 chore: bump version && update changes 2024-01-06 13:34:07 +08:00
firest
d16458ccd0 feat(iotdb): improve the IoTDB bridge to v2 style 2024-01-05 16:54:31 +08:00
Thales Macedo Garitezi
432ddc5a3b fix(bridge_v1_schema): undo changing v1 schema to avoid confusion 2023-12-15 12:55:34 -03:00
Thales Macedo Garitezi
a7fe5da8d6 fix(connector_api): avoid calling resource process to get channels 
Fixes https://emqx.atlassian.net/browse/EMQX-11586
 2023-12-14 10:22:46 -03:00
Thales Macedo Garitezi
b4a5c141ad fix(actions): use action query mode instead of connector's query mode 2023-12-11 14:10:51 -03:00
Thales Macedo Garitezi
2495f59c91 fix(actions): increment rule statistics even if channel is not installed 
Fixes new bug posted after https://emqx.atlassian.net/browse/EMQX-11494 was already fixed.

Also reduces the usage of error throwing for flow control a bit.
 2023-12-11 14:01:30 -03:00
Zaiming (Stone) Shi
423b586c56 fix(dialyzer): fix some dialyzer issues found on otp 26 2023-12-06 20:32:49 +01:00
Thales Macedo Garitezi
c1c7d1246f
Merge pull request #12084 from thalesmg/refactor-resource-status-state-m-20231201 
refactor(resource_manager): use macros and better differentiate status from state
 2023-12-04 17:55:27 -03:00
Thales Macedo Garitezi
d594b38ceb
Merge pull request #12083 from thalesmg/fix-connector-bws-m-20231201 
fix(connector): don't start buffer workers for the connector itself
 2023-12-04 16:14:15 -03:00
Thales Macedo Garitezi
10b1a8bbda feat(connector): add some resource_opts schema facilities 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
dc5e3b939c refactor(resource_manager): use macros and better differentiate status from state 
Internally in `emqx_resource_manager`, there seems to be many points where the
`gen_statem` states are conflated with resource status, since their names coincide.  While
that works for now, introducing a new `gen_statem` state, an internal state, shouldn't
necessarily imply a new, externally facing resource status.

Here we also introduce the usage of some macros to avoid the pitfalls of making a typo in
a state/status name.
 2023-12-01 18:23:05 -03:00
Thales Macedo Garitezi
29ae45c39d fix(connector): don't start buffer workers for the connector itself 
Fixes https://emqx.atlassian.net/browse/EMQX-11448
 2023-12-01 17:20:38 -03:00
Thales Macedo Garitezi
cb60880bab
Merge pull request #12067 from thalesmg/fix-kafka-check-client-m-20231130 
fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
 2023-12-01 09:55:12 -03:00
Thales Macedo Garitezi
0388e1c1c4 fix(kafka_producer): add resource_opts to connector schema, and check for client connectivity 
Fixes https://emqx.atlassian.net/browse/EMQX-11494
 2023-11-30 17:51:32 -03:00
Ivan Dyachkov
9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Ivan Dyachkov
ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
Zaiming (Stone) Shi
14644988e0 chore: change triple-quotes to single-quotes 2023-11-29 16:15:18 +01:00
JianBo He
c8b5c51bbc chore: fix failed test cases 2023-11-28 09:53:46 +08:00
Thales Macedo Garitezi
6c9417efe0 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122 2023-11-22 12:02:34 -03:00
Zaiming (Stone) Shi
3261a12140 fix(emqx_resource): do not allow leading _ or - as resource name 2023-11-22 10:58:54 +01:00
Thales Macedo Garitezi
9e1796ec4f feat(gcp_pubsub_producer): migrate GCP PubSub producer to actions 
Fixes https://emqx.atlassian.net/browse/EMQX-11157
 2023-11-21 14:22:42 -03:00
Thales Macedo Garitezi
b92821188b fix(kafka_producer): make status connecting while the client fails to connect 
Fixes https://emqx.atlassian.net/browse/EMQX-11408

To make it consistent with the previous bridge behavior.

Also, introduces macros for resource status to avoid problems with typos.
 2023-11-16 14:50:23 -03:00
Thales Macedo Garitezi
d6e9bbb95c fix(connector): validate connector name before converting ssl certs 
Fixes https://emqx.atlassian.net/browse/EMQX-11336

See also: https://github.com/emqx/emqx/pull/11540
 2023-11-14 09:29:59 -03:00
Ivan Dyachkov
a49aea3b56 chore: bump app versions 2023-11-14 09:27:04 +01:00
Kjell Winblad
cd5b1f9b96 docs(bridge_V2): type specs for operations 2023-11-14 09:20:46 +01:00
Thales Macedo Garitezi
b255836cbd
Merge pull request #11890 from thalesmg/fix-kafka-unhealthy-r53-20231106 
fix(resource): take error from action/connector before attempting query
 2023-11-07 12:38:57 -03:00
Thales Macedo Garitezi
7dcdbc9e51 fix(resource): take error from action/connector before attempting query 
Fixes https://emqx.atlassian.net/browse/EMQX-11284
Fixes https://emqx.atlassian.net/browse/EMQX-11298
 2023-11-07 10:04:04 -03:00
Thales Macedo Garitezi
2b8cf50a1d chore: rename bridges_v2 -> actions in the public facing APIs 
Fixes https://emqx.atlassian.net/browse/EMQX-11330

After feedback from Product team, we should rename `bridges_v2` to `actions` everywhere.
We'll start with the public facing APIs.

- HTTP API
- Hocon schema root key
 2023-11-06 15:37:07 -03:00
Zaiming (Stone) Shi
600747b7e5 fix(bridge): do not allow dot in bridge name 
also validate name at the API entry
 2023-11-03 20:44:57 +01:00
Kjell Winblad
357b664c8d fix(bridge_v2): more fixes thanks to PR comments from @thalesmg 2023-11-01 15:27:54 +01:00
Kjell Winblad
96d6c6db49 test(bridge_v2): emqx_bridge_v2_kafka_producer_SUITE fix after API change 2023-11-01 15:27:53 +01:00
Kjell Winblad
edb1d37e67 chore(bridge_v2): make fixes thanks to PR comments from @thalesmg 2023-11-01 15:27:53 +01:00
Kjell Winblad
95f3b94ac3 fix(bridge_v2): channels should not be removed when status is connecting 
This fixes so that channels are not removed from the resource state when
their status is connecting. This is needed for Kafka since Kafka's message
buffer is stored in the resource state.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11270
 2023-11-01 15:27:53 +01:00
Kjell Winblad
9dc3a169b3 feat: split bridges into a connector part and a bridge part 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>

Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.

Issues:
https://emqx.atlassian.net/browse/EMQX-10805
 2023-10-30 14:48:47 +01:00
Thales Macedo Garitezi
cf2075d7d8 chore: remove mention of is_buffer_supported from typespec 2023-10-10 09:49:18 -03:00
Thales Macedo Garitezi
d6781efee2 fix(resource): change how buffer workers are started 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
902b1d6ec5 fix(pulsar_producer): use simple_async_internal_buffer query mode for Pulsar 
Since it has internal buffering, it necessitates the same fix as Kafka producer.
 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
eebfb44f72 fix(resource): create simple_async_internal_buffer query mode for bridges with internal buffering 
Since authn/authz backends also use simple async/sync queries, we may want to avoid them
calling the connector when it's not connected.
 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
79cf0a2ced fix(kafka_producer): correctly handle metrics for connector that have internal buffers 
Fixes https://emqx.atlassian.net/browse/EMQX-11086

There’s currently a metric inconsistency due to the internal buffering nature of Kafka
Producer (wolff).

We use simple_sync_query to call the Kafka Producer bridge.  If that times out, the call
is accounted as failed, even though the message is buffered in wolff and later sent
successfully.
 2023-10-09 15:02:25 -03:00
Thales Macedo Garitezi
34186fcc74 fix(kafka_producer): send messages to wolff producer to buffer even when connector is in connecting state 
Fixes https://emqx.atlassian.net/browse/EMQX-11085

Messages would not be sent to wolff if the connection was down, so they were effectively lost.
 2023-10-06 11:43:29 -03:00
Zaiming (Stone) Shi
c2d750aa09 fix(resource): redact query args in exception log 2023-09-29 09:20:42 +02:00
firest
dca8fdb17f fix(resource): respect the start_timeout 2023-09-28 16:36:41 +08:00
Ivan Dyachkov
dafd7c6085 chore: bump apps versions 2023-09-21 10:58:42 +02:00
zhongwencool
123d31fa7d
Merge pull request #11640 from zhongwencool/ensure-destory-resource 
fix: always return ok when remove local resource
 2023-09-21 09:21:45 +08:00
zhongwencool
dd687d9582 fix: dialyzer warning 2023-09-20 22:41:26 +08:00
zhongwencool
2f1fa2e961 chore: unified slog message formatting to improve logging consistency 2023-09-20 18:13:00 +08:00
zhongwencool
c26a18e949 fix: always return ok when remove local resource 2023-09-20 18:02:42 +08:00
Paulo Zulato
dfcede8794 fix: increment matched counter when bridge is unhealthy 
Fixes https://emqx.atlassian.net/browse/EMQX-10767
 2023-08-30 10:52:53 -03:00
Paulo Zulato
cc3ba18734 fix: increment dropped message counter when bridge is unhealthy 
Fixes https://emqx.atlassian.net/browse/EMQX-10767
 2023-08-28 19:47:11 -03:00