Commit Graph

22147 Commits

Author SHA1 Message Date
Andrew Mayorov
8db70b5bbc
test(dsrepl): add crash-restart-recover testcase
That verifies nothing is lost in the event of abrupt node failures.
2024-06-26 20:44:33 +02:00
Andrew Mayorov
ae89b61af0
feat(cth-cluster): make restart/1 more generic 2024-06-26 20:44:33 +02:00
Andrew Mayorov
5fd5fc76e5
fix(dsstore): ensure backward compatibility 2024-06-26 20:44:33 +02:00
Andrew Mayorov
8538a5a5b6
test(dsrepl): anticipate transitionless membership changes
E.g. when a membership change is applied twice in a row.
2024-06-26 20:44:33 +02:00
Andrew Mayorov
19072414cb
chore: bump erlang-rocksdb to 1.8.0-emqx-6 2024-06-26 20:44:33 +02:00
Andrew Mayorov
cd0663074e
test(dsrepl): add add_generation events into the mix
They usually cause storage layer to perform flushes, and thus enable
testing `handle_flush/1` codepath in different circumstances.
2024-06-26 20:44:33 +02:00
Andrew Mayorov
2180cc7c26
fix(dsstore): avoid storing cf_refs() in the RocksDB itself
This is both pointless and confusing.
2024-06-26 20:44:33 +02:00
Andrew Mayorov
80ea2e62f7
fix(stream): ensure that chain/1 preserves the order 2024-06-26 20:44:32 +02:00
Andrew Mayorov
7895e9cc45
feat(dsstore): make WAL-less mode optional
And make the upper layer choose when to use it.
2024-06-26 20:44:32 +02:00
Andrew Mayorov
0c0757b8c2
feat(dsrepl): enable WAL-less batch writes 2024-06-26 20:44:32 +02:00
Andrew Mayorov
2705226eb5
feat(dsrepl): release log entries occasionally
Also make tracepoints in `apply/3` callback implementation more
uniform.
2024-06-26 20:44:32 +02:00
Andrew Mayorov
b6894c18fa
chore(dsrepl): improve tracepoints usability a bit 2024-06-26 20:44:32 +02:00
Andrew Mayorov
10e9fed22b
Merge pull request #13326 from keynslug/fix/ct/api-clients-flaky
test(api-clients): refactor suite to isolate flakiness
2024-06-25 18:26:42 +02:00
ieQu1
9f30da334f
Merge pull request #13248 from ieQu1/dev/EMQX-12491-local-backend
replace builtin DS backend with builtin_local and builtin_raft
2024-06-24 23:07:57 +02:00
ieQu1
d349f84f04
fix(ds): Apply remarks 2024-06-24 20:20:40 +02:00
Andrew Mayorov
d12b985507
test(api-clients): refactor suite to isolate flakiness 2024-06-24 16:54:20 +02:00
Thales Macedo Garitezi
debf1e6cd5
Merge pull request #13316 from thalesmg/fix-mt-read-user-prop-m-20240621
fix(message transformation): correctly read from user properties in context
2024-06-24 09:31:48 -03:00
ieQu1
c0472a06f1
test(ds): Set initial conditions in repl. suite explicitly 2024-06-22 02:55:05 +02:00
Thales Macedo Garitezi
6682004dc8
Merge pull request #13319 from thalesmg/20240621-sync-r57-m
sync release-57 to master
2024-06-21 17:44:08 -03:00
Thales Macedo Garitezi
ec83d999bf Merge remote-tracking branch 'origin/release-57' into 20240621-sync-r57-m 2024-06-21 14:11:08 -03:00
ieQu1
9a58d71378
fix(ds): Move DS backend supervision trees to their own apps 2024-06-21 18:22:04 +02:00
ieQu1
3d69ec496a
fix(schema): Transform config of `singleton' discovery_strategy 2024-06-21 17:16:25 +02:00
ieQu1
d7d878fd43
test(ds): Fix emqx standalone test profile selection 2024-06-21 17:16:25 +02:00
ieQu1
8aa27488b6
test: Disable certain DS-related suites in CE 2024-06-21 17:16:24 +02:00
ieQu1
1d3b1868fb
test(ds): Use close_db API 2024-06-21 17:16:24 +02:00
ieQu1
b2f7815a7f
test(ds): Don't start raft explicitly 2024-06-21 17:16:24 +02:00
ieQu1
3851fc189f
fix(ds): Avoid reverse dependencies from storage to repl. layer 2024-06-21 17:16:24 +02:00
ieQu1
bc915216a0
feat(ds): Support metrics for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1
be6c5e172f
fix(ds): Disable DS management APIs for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1
99c9b56cf3
feat(ds_buffer): Add `ls' function to list all local buffers 2024-06-21 17:16:24 +02:00
ieQu1
5a8818edf3
feat(ds): Add schema for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1
4484f30021
feat(ds_schema): Add schema for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1
8990b1312b
refactor(ds_schema): Rename backend builtin -> builtin_raft 2024-06-21 17:16:24 +02:00
ieQu1
ea48b1265d
refactor(ds_schema): Extract common builtin fields 2024-06-21 17:16:24 +02:00
ieQu1
abe41de19b
refactor(ds_schema): builtin_local_buffer -> builtin_buffer 2024-06-21 17:16:24 +02:00
ieQu1
ecb172b07e
refactor(ds): Rename egress metrics to 'buffer' 2024-06-21 17:16:24 +02:00
ieQu1
09c3ae795d
refactor(ds_raft): Replace egress server with common emqx_ds_buffer 2024-06-21 17:16:24 +02:00
ieQu1
a0fbd37e58
refactor(emqx): Use emqx_ds_backends application 2024-06-21 17:16:24 +02:00
ieQu1
ef09cfcd71
feat(ds): Add emqx_ds_backends application 2024-06-21 17:16:24 +02:00
ieQu1
279619fc80
feat(ds): Add `builtin_local' backend 2024-06-21 17:16:24 +02:00
ieQu1
a8ea0ae4e5
refactor(ds): Extract DS replication layer to a separate application 2024-06-21 17:16:24 +02:00
ieQu1
63f1856a2c
feat(ds): Dynamic backend registration 2024-06-21 17:16:24 +02:00
ieQu1
83dc8f4d77
fix(ds): Fix return values of emqx_ds_storage_layer functions 2024-06-21 17:16:23 +02:00
ieQu1
a18d1987a2
test(ds): Add a helper function for diffing messages 2024-06-21 17:16:23 +02:00
Ilia Averianov
f1b8c356a6
Merge pull request #13299 from savonarola/0617-shared-sub-leader-poc
feat(queue): implement PoC version of session ↔️ shared group leader interaction
2024-06-21 16:45:17 +03:00
Thales Macedo Garitezi
28c7d94bd2 fix(message transformation): correctly read from user properties in context
Fixes https://emqx.atlassian.net/browse/EMQX-12582
2024-06-21 09:39:22 -03:00
Ilya Averyanov
db28a042d5 feat(queue): handle renew_lease_timeout 2024-06-21 15:18:37 +03:00
Ilya Averyanov
2096755ad6 feat(queue): reorganize group sm callbacks and methods 2024-06-21 13:01:38 +03:00
Ilya Averyanov
b9c5911883 feat(queue): implement find_leader_timeout event and so the connecting group_sm state 2024-06-21 12:22:17 +03:00
SergeTupchiy
fb266fbf8c
Merge pull request #13300 from SergeTupchiy/EMQX-12195-cluster-link-conf-backup
cluster link conf backup
2024-06-21 11:42:57 +03:00