Commit Graph

164 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi
322b7bb7d2 chore: bump app vsn 2023-12-22 13:00:37 +01:00
Thales Macedo Garitezi
cf9331a95f Merge branch 'release-54' into sync-r54-m-20231218 2023-12-18 17:21:08 -03:00
Zaiming (Stone) Shi
7338e394c8 perf: upgrade to wolff 1.9.1 for better performance
Upgrade from 1.8.0:
- wollf-1.9.0 has the global stats disabled by default
- wolff-1.9.1 improved client pid lookup performance
2023-12-18 18:54:21 +01:00
Thales Macedo Garitezi
432ddc5a3b fix(bridge_v1_schema): undo changing v1 schema to avoid confusion 2023-12-15 12:55:34 -03:00
Thales Macedo Garitezi
5bdfac52b8 fix(bridge_kafka_producer): add missing resource_opts sub-fields to connector schema 2023-12-04 10:02:05 -03:00
Zaiming (Stone) Shi
55621fbb86
Merge pull request #12074 from zmstone/1201-user-peer-module-for-all
test: replace 'slave' and 'ct_slave' with 'peer'
2023-12-01 21:35:40 +01:00
Thales Macedo Garitezi
cb60880bab
Merge pull request #12067 from thalesmg/fix-kafka-check-client-m-20231130
fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
2023-12-01 09:55:12 -03:00
Zaiming (Stone) Shi
22f7cc1622 test: replace 'slave' and 'ct_slave' with 'peer' 2023-12-01 08:07:09 +01:00
Thales Macedo Garitezi
0388e1c1c4 fix(kafka_producer): add resource_opts to connector schema, and check for client connectivity
Fixes https://emqx.atlassian.net/browse/EMQX-11494
2023-11-30 17:51:32 -03:00
Ivan Dyachkov
9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Zaiming (Stone) Shi
dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26
1129 prepare for OTP 26
2023-11-30 19:38:40 +01:00
Zaiming (Stone) Shi
e6eb97e104 chore: upgrade dependency brod_gssapi to work with OTP 26 2023-11-29 16:15:18 +01:00
Stefan Strigler
048f4724a9 feat(emqx_connector): add field 'actions' in API response
Also unify schemas, use emqx_connector_schema for the generic parts.
2023-11-27 16:09:17 +01:00
zhongwencool
b5a00ec6b2
Merge pull request #12023 from emqx/master
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
Stefan Strigler
c1ef773e77 fix: check for sane state after regular shutdown 2023-11-24 15:09:35 +01:00
Stefan Strigler
d2901afd1b fix(emqx_bridge_kafka): match example in api schema 2023-11-24 10:07:11 +01:00
Thales Macedo Garitezi
fdfa3213cc fix(bridge_api): don't mangle configs, use correct type as argument
Fixes https://emqx.atlassian.net/browse/EMQX-11412

- The wrong type was being used in a list lookup function, resulting in the automatic
transformation being called erroneously and mangling the config.
- There was a left-over workaround still around which could still mangle the config.
2023-11-23 11:41:26 -03:00
Thales Macedo Garitezi
6c9417efe0 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122 2023-11-22 12:02:34 -03:00
Thales Macedo Garitezi
bb549cdf8b feat(actions): allow multiple action info modules per application 2023-11-22 09:33:01 -03:00
Thales Macedo Garitezi
38d3a1d7d0 feat(actions): allow multiple action info modules per application 2023-11-22 09:30:37 -03:00
Thales Macedo Garitezi
39791511fc chore: remove obsolete workaround 2023-11-21 17:38:13 -03:00
Thales Macedo Garitezi
b3dffa4390 fix(kafka): don't return parameters from /bridges API
Fixes https://emqx.atlassian.net/browse/EMQX-11412
2023-11-21 16:00:15 -03:00
Kjell Winblad
b0d670aaa9
Merge pull request #11992 from kjellwinblad/kjell/backport/action_upgrade_downgrade_hooks
fix(action): upgrade and downgrade strategy
2023-11-21 19:14:37 +01:00
Thales Macedo Garitezi
068d151b14
Merge pull request #11983 from thalesmg/sync-r53-to-m-20231120
sync `release-53` to `master`
2023-11-21 09:02:51 -03:00
Kjell Winblad
6030bf6fa5 fix(action): upgrade and downgrade strategy
This commit adds upgrade and downgrade hooks that are called when
upgrading from a bridge V1 to connector and action or the other way
around. The automatic translation is used if the callback is not
defined.

NOTE: Backported from master
2023-11-21 11:18:21 +01:00
Thales Macedo Garitezi
6f8630304d
Merge pull request #11980 from kjellwinblad/kjell/fix_fixup_callbacks/EMQX-11428
fix: bridge to action upgrade fixup hook should run after upgrade
2023-11-20 16:19:43 -03:00
Kjell Winblad
7fb5ade832 fix(kafka_producer): hocon renames kafka field 2023-11-20 19:20:13 +01:00
Kjell Winblad
d214ae8772
fix: problems found by @thalesmg in code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-20 18:59:57 +01:00
Kjell Winblad
3aa8044475 fix(action): upgrade and downgrade strategy
This commit fixes the upgrade and downgrade strategy when upgrading
from a bridge V1 to connector and action or the other way around so that
the custom callbacks get the complete unchanged input instead of the
result of the automatic translation. The automatic translation is used
if the callback is not defined.
2023-11-20 16:40:27 +01:00
Thales Macedo Garitezi
f40f6bc5dd refactor: split resource_opts fields between connector and actions 2023-11-20 12:37:36 -03:00
Thales Macedo Garitezi
53e796bbd0 Merge remote-tracking branch 'origin/release-53' into sync-r53-to-m-20231120 2023-11-20 11:49:52 -03:00
Kjell Winblad
79a764f117 fix: bridge to action upgrade fix up hook should run after upgrade
This commit changes how the `emqx_action_info` callback
`bridge_v1_to_action_fixup/1` works. It is now called after the
automatic upgrade instead of before. Since the full Bridge V1 config
might be needed to do the fixup, it is provided in a special field
`<<"__bridge_v1_conf__">>`. The `<<"__bridge_v1_conf__">>` field is
removed after the callback is called and can thus be ignored if it is
not needed.
2023-11-20 12:06:21 +01:00
Stefan Strigler
84ff7b0b38 feat(emqx_bridge): action_info with dynamic lookup
This allows a n:1 relation between v1 bridge_types to action/connector types as
it's the case with mongodb for instance, where we had `mongodb_single`
`mongodb_sharded` etc and the new implementation will just have `mongodb`.
2023-11-20 10:22:37 +01:00
Thales Macedo Garitezi
eb3f54184e refactor: address review comments and avoid transformations without schema knowledge 2023-11-17 16:16:34 -03:00
Kjell Winblad
86c126ffcd feat: callbacks for fixup after automatic Bridge V1 upgrade/downgrade
This commit adds callbacks to the emqx_action_info module for
doing fixes (such as changing a field name) after the automatic
split of a Bridge V1 config or the merge of connector and action
configs for the compatibility layer.
2023-11-17 16:10:08 -03:00
Thales Macedo Garitezi
b92821188b fix(kafka_producer): make status connecting while the client fails to connect
Fixes https://emqx.atlassian.net/browse/EMQX-11408

To make it consistent with the previous bridge behavior.

Also, introduces macros for resource status to avoid problems with typos.
2023-11-16 14:50:23 -03:00
Thales Macedo Garitezi
36b5d58957 test: reorganize test suite a bit 2023-11-16 13:16:02 -03:00
Andrew Mayorov
d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
Andrew Mayorov
893e90b372
fix(kafka): use safe publish in consumer
Routing with v2 schema is actually more strict with respect to input to
`emqx_router` module routines. This causes Kafka consumer bridge to
crash when it tries to publish a message to a topic that looks like a
topic filter.
2023-11-15 22:37:00 +07:00
Ivan Dyachkov
7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Ivan Dyachkov
1c57993c91 Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53 2023-11-14 10:44:35 +01:00
Andrew Mayorov
fda395014c
test(kafka): verify file-based secrets work 2023-11-14 16:05:25 +07:00
Andrew Mayorov
aa458b65d6
test(kafka): simplify consumer testsuite matrix setup 2023-11-14 16:05:25 +07:00
Andrew Mayorov
18cd98def6
chore(test): fix formatting quirks 2023-11-14 16:05:25 +07:00
Andrew Mayorov
c300eb41a7
feat(kafka): accept wrapped secrets as passwords 2023-11-14 16:05:25 +07:00
Ivan Dyachkov
a49aea3b56 chore: bump app versions 2023-11-14 09:27:04 +01:00
Kjell Winblad
093c8b0c6e docs: add missing copyright headers 2023-11-14 09:20:46 +01:00
Kjell Winblad
e93b71d8d5 fix: problems found by @thalesmg in code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-14 09:20:46 +01:00
Kjell Winblad
5e8e407017 refactor: action schema retrival after PR feedback 2023-11-14 09:20:46 +01:00
Kjell Winblad
9eaee8f333 refactor(emqx_bridge_v2): make independent of Kafka
This removes the Kafka specific knowledge from emqx_bridge_v2 and
makes it possible to add new Bridge V2 bridges without modifying
the emqx_bridge application.
2023-11-14 09:20:46 +01:00