Commit Graph

18596 Commits

Author SHA1 Message Date
ieQu1
62542e5844 feat(ds): Metadata storage for the replication layer 2023-11-21 19:37:01 +01:00
Ilya Averyanov
c6fd1e4c75
Merge pull request #11971 from savonarola/1116-no-auth-rebalance-status
feat(rebalance): improve rebalance usability
2023-11-21 12:10:18 +02:00
Andrew Mayorov
cd48c283db
Merge pull request #11988 from keynslug/ft/EMQX-11222/basic-config
feat(schema): introduce separate root for new session persistence
2023-11-21 15:02:21 +07:00
Thales Macedo Garitezi
6f8630304d
Merge pull request #11980 from kjellwinblad/kjell/fix_fixup_callbacks/EMQX-11428
fix: bridge to action upgrade fixup hook should run after upgrade
2023-11-20 16:19:43 -03:00
Kjell Winblad
7fb5ade832 fix(kafka_producer): hocon renames kafka field 2023-11-20 19:20:13 +01:00
Ivan Dyachkov
fc6d08ca97
Merge pull request #11977 from id/2011-disable-copilot
chore: remove github copilot placeholder from pr template
2023-11-20 19:09:21 +01:00
Kjell Winblad
d214ae8772
fix: problems found by @thalesmg in code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-20 18:59:57 +01:00
Andrew Mayorov
110a5a4896
feat(schema): introduce separate root for new session persistence
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
2023-11-20 23:49:14 +07:00
Andrew Mayorov
8e107ffe45
Merge pull request #11958 from keynslug/ft/EMQX-9745/preserve-acks
feat(sessds): preserve acks / replays in session state
2023-11-20 23:40:22 +07:00
Kjell Winblad
edbfe090c9 feat: add custom callback for creating connector from bridge V1 config 2023-11-20 17:23:51 +01:00
Andrew Mayorov
9684e79ee0
fix(sessds): ensure dup flag is on for replayed messages 2023-11-20 22:54:40 +07:00
Kjell Winblad
3aa8044475 fix(action): upgrade and downgrade strategy
This commit fixes the upgrade and downgrade strategy when upgrading
from a bridge V1 to connector and action or the other way around so that
the custom callbacks get the complete unchanged input instead of the
result of the automatic translation. The automatic translation is used
if the callback is not defined.
2023-11-20 16:40:27 +01:00
Andrew Mayorov
ef7cfd0202
feat(sessds): add field to ds_pubrange for forward-compat 2023-11-20 19:56:55 +07:00
Stefan Strigler
d99f033d74
Merge pull request #11967 from sstrigler/bridge-v1-type-fun
feat(emqx_bridge): action_info with dynamic lookup
2023-11-20 12:35:22 +01:00
Kjell Winblad
79a764f117 fix: bridge to action upgrade fix up hook should run after upgrade
This commit changes how the `emqx_action_info` callback
`bridge_v1_to_action_fixup/1` works. It is now called after the
automatic upgrade instead of before. Since the full Bridge V1 config
might be needed to do the fixup, it is provided in a special field
`<<"__bridge_v1_conf__">>`. The `<<"__bridge_v1_conf__">>` field is
removed after the callback is called and can thus be ignored if it is
not needed.
2023-11-20 12:06:21 +01:00
Stefan Strigler
84ff7b0b38 feat(emqx_bridge): action_info with dynamic lookup
This allows a n:1 relation between v1 bridge_types to action/connector types as
it's the case with mongodb for instance, where we had `mongodb_single`
`mongodb_sharded` etc and the new implementation will just have `mongodb`.
2023-11-20 10:22:37 +01:00
Andrew Mayorov
7081f1951f
refactor(sessds): use ds_pubrange record as is
Instead of converting it into almost similar runtime representation.
2023-11-20 15:40:43 +07:00
Andrew Mayorov
a5ff4144fe
test(sessds): add complex testcase for session replay 2023-11-20 15:40:43 +07:00
Ivan Dyachkov
ba2133abb3 chore: remove github copilot placeholder from pr template
https://githubnext.com/copilot-for-prs-sunset
2023-11-20 09:12:38 +01:00
Andrew Mayorov
1246d714c5
feat(sessds): preserve acks / ranges in mnesia for replays 2023-11-20 13:33:48 +07:00
Andrew Mayorov
5b40304d1f
chore(sessds): simplify subscriptions handling
There's currently no point in storing parsed topic filters in the
subscriptions table.
2023-11-20 13:33:48 +07:00
Andrew Mayorov
648b6ac63e
chore(sessds): rename iterators -> subscriptions
Also try to make clearer the difference between 2 flavors of topic
filter representation in use.
2023-11-20 13:33:48 +07:00
Ilya Averyanov
e93e9ed108 feat(rebalance): improve rebalance usability
* make availability API endpoint public
* allow connections during wait_health_check interval
* make availability status calculation more consistent and lightweight
* refactor test to get rid of some mocks and to use cth
2023-11-18 21:29:16 +03:00
Thales Macedo Garitezi
6c19d1e78e
Merge pull request #11968 from kjellwinblad/kjell/bridge_v1_upgrade_downgrade_fixup_callbacks
Bridge V1 config upgrade downgrade fixup callbacks
2023-11-17 17:57:26 -03:00
Thales Macedo Garitezi
eb3f54184e refactor: address review comments and avoid transformations without schema knowledge 2023-11-17 16:16:34 -03:00
Kjell Winblad
86c126ffcd feat: callbacks for fixup after automatic Bridge V1 upgrade/downgrade
This commit adds callbacks to the emqx_action_info module for
doing fixes (such as changing a field name) after the automatic
split of a Bridge V1 config or the merge of connector and action
configs for the compatibility layer.
2023-11-17 16:10:08 -03:00
Kjell Winblad
9feba802e9 chore: add convenience function for creating action schemas 2023-11-17 16:10:08 -03:00
Ilya Averyanov
bdbadaa2ca
Merge pull request #11955 from savonarola/1115-fix-mongo-resource-stop
fix: fix deadlock while stopping mongodb resource
2023-11-17 11:31:24 +02:00
Ilya Averyanov
4c54efd8b1 fix(auth): fix deadlock while stopping mongodb resource 2023-11-16 19:29:08 +03:00
Ivan Dyachkov
1395f1c424
Merge pull request #11951 from id/1114-sync-r54-take2
sync release-54 to master
2023-11-15 09:45:08 +01:00
Thales Macedo Garitezi
f5545bbdf3 fix(schema_types): add missing secret() type 2023-11-14 16:08:13 -03:00
Thales Macedo Garitezi
5275a01727 fix(syskeeper): refactor action to new action info scheme 2023-11-14 19:49:28 +01:00
Thales Macedo Garitezi
d2cf29fae4 fix(schema): use hoconsc:union/1 2023-11-14 19:49:08 +01:00
Thales Macedo Garitezi
423fb32171 test: fix noise about undefined unofficial callbacks 2023-11-14 19:48:39 +01:00
Ivan Dyachkov
7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Zaiming (Stone) Shi
88637f81d1
Merge pull request #11936 from zmstone/1112-readable-types
refactor(schema): keep type converters close
2023-11-14 17:56:07 +01:00
Andrew Mayorov
e80600ca0f
Merge pull request #11896 from keynslug/ft/EMQX-10808/opt-file-secret-bridges
feat(bridge): accept wrapped secrets as passwords
2023-11-14 23:08:38 +07:00
ieQu1
8ddef21ac4
Merge pull request #11948 from ieQu1/dev/EMQX-10998-receive-maximum
fix(ds): Respect receive_maximum from the connection info
2023-11-14 16:45:36 +01:00
ieQu1
124c0e2dba fix(ds): Respect receive_maximum from the connection info 2023-11-14 16:06:19 +01:00
Ivan Dyachkov
bbf00c2f71
Merge pull request #11946 from id/1114-sync-r53
sync release-53 to master
2023-11-14 13:38:24 +01:00
Thales Macedo Garitezi
01a2a3b1c0
Merge pull request #11930 from thalesmg/ds-session-discard-m-20231110
feat(ds): implement session discard
2023-11-14 09:30:34 -03:00
Ivan Dyachkov
28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Ivan Dyachkov
1c57993c91 Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53 2023-11-14 10:44:35 +01:00
Andrew Mayorov
d1c3b1c659
fix(bridges): handle secrets more consistently
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-14 16:05:54 +07:00
Andrew Mayorov
11c7024d9d
chore: add changelog entry 2023-11-14 16:05:53 +07:00
Andrew Mayorov
d4dd4508c9
feat(greptimedb): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov
5d620465e2
feat(influx): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov
7d6c9d2daa
feat(iotdb): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov
7817502b8b
feat(rocketmq): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov
2449d54b1f
feat(pulsar): accept wrapped secrets as passwords 2023-11-14 16:05:52 +07:00