Fixes https://emqx.atlassian.net/browse/EMQX-11412
- The wrong type was being used in a list lookup function, resulting in the automatic
transformation being called erroneously and mangling the config.
- There was a left-over workaround still around which could still mangle the config.
This commit adds upgrade and downgrade hooks that are called when
upgrading from a bridge V1 to connector and action or the other way
around. The automatic translation is used if the callback is not
defined.
NOTE: Backported from master
This commit fixes the upgrade and downgrade strategy when upgrading
from a bridge V1 to connector and action or the other way around so that
the custom callbacks get the complete unchanged input instead of the
result of the automatic translation. The automatic translation is used
if the callback is not defined.
This commit adds callbacks to the emqx_action_info module for
doing fixes (such as changing a field name) after the automatic
split of a Bridge V1 config or the merge of connector and action
configs for the compatibility layer.
Fixes https://emqx.atlassian.net/browse/EMQX-11330
After feedback from Product team, we should rename `bridges_v2` to `actions` everywhere.
We'll start with the public facing APIs.
- HTTP API
- Hocon schema root key
Examples would show `type` and `name` properties in the request body, which is
not accepted by the schema.
Also fixes some minor inconsistencies in the example names of connectors and
bridges.
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.
Issues:
https://emqx.atlassian.net/browse/EMQX-10805
Fixes https://emqx.atlassian.net/browse/EMQX-10847
Checking the whole Kafka message from AEH, it seems like the timestamp type is append,
which means that it’s the broker who controls the timestamp, and the timestamp defined by
the producer is ignored.
Ref: https://kafka.apache.org/documentation/#brokerconfigs_log.message.timestamp.type
Example message consumed from AEH:
```
%{
"headers" => %{},
"key" => "",
"offset" => 4,
"topic" => "test0",
"ts" => 1692879703006,
"ts_type" => "append",
"value" => "{\"username\":\"undefined\",\"topic\":\"t/aeh/produ\",\"timestamp\":1692879692189,\"qos\":0,\"publish_received_at\":1692879692189,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaa\",\"node\":\"emqx@127.0.0.1\",\"metadata\":{\"rule_id\":\"rule_aehp\"},\"id\":\"000603AA44B34E08F4AF000006E30003\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```
Note the ts_type above is append.
Example message from a Kafka broker whose ts type is create:
```
%{
"headers" => %{},
"key" => "",
"offset" => 4,
"topic" => "test-topic-three-partitions",
"ts" => 1692881883668,
"ts_type" => "create",
"value" => "{\"username\":\"undefined\",\"topic\":\"t/kafka/produ\",\"timestamp\":1692881883668,\"qos\":0,\"publish_received_at\":1692881883668,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaaaa\",\"node\":\"emqx@127.0.0.1\",\"id\":\"000603AAC7529FEEF4AC000007050000\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```
Unfortunately, I couldn’t find anywhere in AEH where that configuration could be changed.
Zaiming (Stone) Shi