5532f40d83
Merge branch 'release-57' into 20240702-m-sync-r57-mix-umbrella
2024-07-02 11:52:36 -03:00
19f3b030f9
chore: preparing to run common tests / eunit with mix
2024-07-02 09:45:45 -03:00
30efa1f57e
test(dsrepl): relax crash-recover testcase to tolerate message loss
...
Which is quite an expected occasion for this kind of stress test.
2024-06-26 20:44:34 +02:00
8db70b5bbc
test(dsrepl): add crash-restart-recover testcase
...
That verifies nothing is lost in the event of abrupt node failures.
2024-06-26 20:44:33 +02:00
8538a5a5b6
test(dsrepl): anticipate transitionless membership changes
...
E.g. when a membership change is applied twice in a row.
2024-06-26 20:44:33 +02:00
7895e9cc45
feat(dsstore): make WAL-less mode optional
...
And make the upper layer choose when to use it.
2024-06-26 20:44:32 +02:00
8aa27488b6
test: Disable certain DS-related suites in CE
2024-06-21 17:16:24 +02:00
1d3b1868fb
test(ds): Use close_db API
2024-06-21 17:16:24 +02:00
09c3ae795d
refactor(ds_raft): Replace egress server with common emqx_ds_buffer
2024-06-21 17:16:24 +02:00
ef09cfcd71
feat(ds): Add `emqx_ds_backends` application
2024-06-21 17:16:24 +02:00
279619fc80
feat(ds): Add `builtin_local' backend
2024-06-21 17:16:24 +02:00
a8ea0ae4e5
refactor(ds): Extract DS replication layer to a separate application
2024-06-21 17:16:24 +02:00
a18d1987a2
test(ds): Add a helper function for diffing messages
2024-06-21 17:16:23 +02:00
22009bcc58
test(dsstore): verify that inherited TLS trie is persisted
2024-06-17 17:53:38 +02:00
3d9837a0b8
test(ds): Add emqx_ds_replication_SUITE:t_drop_generation
2024-05-24 19:09:33 +02:00
29345aaa30
fix(ds): Fix idle event generation in bitfield_lts layout
2024-05-22 18:01:33 +02:00
eb7c43ee9d
fix(ds): Always store messages in the current generation
2024-05-22 18:01:33 +02:00
074d98a14a
test(ds): Refactor ds_SUITE
2024-05-22 18:01:33 +02:00
c6fc76e335
fix(ds): Perform read operations on the leader.
2024-05-22 18:01:32 +02:00
35e360fcbe
feat(api-ds): provide more information on nonexistent site leave
2024-05-14 15:57:41 +02:00
3da3a36863
test(ds): Add generation in the replication suite
2024-05-13 19:51:04 +02:00
9f7ef9f34f
fix(ds): Apply review remarks
2024-05-13 19:35:24 +02:00
07aa708894
test(ds): Refactor replication suite
2024-05-09 03:56:56 +02:00
63e51fca66
test(ds): Use streams to fill the storage
2024-05-09 02:46:57 +02:00
a0a3977043
feat(ds): Assign latest timestamp deterministically
2024-05-08 23:17:57 +02:00
2236af84ba
feat(ds): two-stage storage commit on the storage level
2024-05-08 23:17:57 +02:00
68ca891f41
test(ds): Use streams to create traffic
2024-05-08 23:17:57 +02:00
1ff2e02fd9
feat(ds): Pass current time to the storage layer via argument
2024-05-06 11:21:32 +02:00
c4d1360b96
fix(dsrepl): trigger election for new ra servers unconditionallly
...
Otherwise we might end up in a situation when there's no member online
yet at the time of the election trigger, and the election will never
happen.
2024-04-15 16:42:29 +02:00
1e95bd4da6
test(dsrepl): test unresponsive nodes removal / node restarts
2024-04-08 21:27:56 +02:00
dcde30c38a
test(dsrepl): add two more testcases for rebalancing
2024-04-08 13:22:31 +02:00
556ffc78c9
feat(dsrepl): implement membership changes and rebalancing
2024-04-05 18:57:28 +02:00
70396e9766
Merge pull request #12825 from keynslug/feat/EMQX-12110/repl-meta-api
...
feat(dsrepl): add APIs to manage DB replication sites
2024-04-04 22:32:03 +02:00
ad52f7838e
feat(dsrepl): add APIs to manage DB replication sites
2024-04-04 21:22:01 +02:00
f37ed3a40a
fix(ds): Limit the number of retries in egress to 0
2024-04-03 16:38:49 +02:00
ae5935e7f7
test(ds): Attempt to stabilize metrics_worker tests in CI
2024-04-02 19:14:10 +02:00
94ca7ad0f8
feat(ds): Report counters for LTS storage layout
2024-04-02 16:47:43 +02:00
75b092bf0e
fix(ds): Actually retry sending batch
2024-04-02 16:45:49 +02:00
c666c65c6a
test(ds): factor out storage iteration into helper module
2024-04-02 13:48:51 +02:00
e029b8f996
test(dsrepl): wait for whole cluster readiness
...
To minimize the chance of flaky tests due to the shards not being
completely online.
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-04-02 13:48:50 +02:00
2cd357a5bd
fix(ds): ensure store batch is idempotent wrt generations
2024-04-02 13:48:50 +02:00
77a022bd93
feat(dsrepl): transfer storage snapshot during ra snapshot recovery
2024-04-02 13:48:49 +02:00
74881e8706
feat(dsrepl): make storage layer unaware of granularity of time
...
Storage also becomes a bit more pure, depending on the upper layer to
provide the timestamps, which also makes it possible to handle more
operations idempotently.
2024-03-20 13:20:23 +01:00
5cc0246351
feat(dsrepl): allow to tune select ra options
2024-03-19 20:21:55 +01:00
e6c2c2fb07
feat(dsrepl): manage generations / db config through ra machine
2024-03-19 20:11:39 +01:00
f2dc940436
Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56
2024-03-19 15:20:08 +01:00
11ae04d810
test(ds): rm unused var warning
2024-03-14 17:16:24 -03:00
6af01b916e
feat(ds): implement `get_delete_streams`, `make_delete_iterator` and `delete_next` callbacks for builtin storage
...
Part of https://emqx.atlassian.net/browse/EMQX-11841
2024-03-08 09:56:46 -03:00
f7e3afde16
test(ds): avoid introducing new macros
2024-03-07 16:49:20 +01:00
69427dc42d
test(ds): add tests for error mapping and replay recovery
2024-03-07 12:59:58 +01:00
Thales Macedo Garitezi