yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,422 Commits 393 Branches 684 Tags 74 MiB
Commit Graph

352 Commits

Author SHA1 Message Date
ieQu1 3721be65ee
fix(ds): Improve comments 2024-07-09 13:15:15 +02:00
ieQu1 dc4ae82798
test(ds): Add message ID 2024-07-09 13:15:14 +02:00
ieQu1 661f79544b
fix(ds): Optimize hot loop of skipstream storage layout 2024-07-09 13:15:14 +02:00
ieQu1 23dafbb03b
feat(ds): Add a benchmarking tool for storage efficiency analysis 2024-07-09 13:15:14 +02:00
ieQu1 afeb2ab8aa
feat(ds): Add metrics for skipstream layout 2024-07-09 13:15:14 +02:00
ieQu1 8c5e4a2376
test(ds): Generalize storage layout test suite for different layouts 2024-07-09 13:15:14 +02:00
ieQu1 a4642d4d06
feat(ds): Add a new storage layout engine: "skipstream" 
This layout is based on LTS as well, but it uses separate index
streams for constrained replay of streams with learned wildcards
 2024-07-09 13:15:14 +02:00
ieQu1 de48077ac4
test(ds): Add new helper functions 
- Improve message comparison
- Add set operations
 2024-07-09 13:15:14 +02:00
ieQu1 210556e545
feat(ds): Generalize value serialization 
- Add a new utility module for message serialization
- Add experimental serializer based on ASN.1
 2024-07-09 13:15:14 +02:00
ieQu1 843973ef32
fix(ds): bitfield_lts: static_key_size -> static_key_bits 2024-07-09 13:15:14 +02:00
ieQu1 f84fb34692
feat(ds_lts): New APIs: info, reverse lookups and topic compression 2024-07-09 13:15:14 +02:00
ieQu1 eb80402ccb
fix(ds): Improve typespecs and descriptions in storage_layer 2024-07-09 13:15:14 +02:00
ieQu1 71dad0242e
docs(ds): Move Raft-related parts to emqx_ds_builtin_raft README 2024-07-09 13:15:14 +02:00
ieQu1 afe1c5617d
refactor(ds): Rename macros for bitfield_lts metrics 2024-07-09 13:15:14 +02:00
ieQu1 0f2c19b656
refactor(ds): Move end_of_stream detection logic for delete_next 2024-07-09 13:15:14 +02:00
ieQu1 b565976794
fix(ds): Fix hashing of empty wildcard topic levels in bitfield_lts 2024-07-09 13:15:13 +02:00
Thales Macedo Garitezi 32ace85e1c Merge branch 'release-57' into 20240703-m-sync-r57 2024-07-03 16:03:15 -03:00
Thales Macedo Garitezi afb65817c4 ci(mix): refactor to avoid conflicts 
This introduces a refactoring that will hopefully help avoid conflicts with ongoing work
to migrate our build / CI tooling to Mix.
 2024-07-03 14:37:25 -03:00
Thales Macedo Garitezi 5532f40d83 Merge branch 'release-57' into 20240702-m-sync-r57-mix-umbrella 2024-07-02 11:52:36 -03:00
Thales Macedo Garitezi 19f3b030f9 chore: preparing to run common tests / eunit with mix 2024-07-02 09:45:45 -03:00
Thales Macedo Garitezi b74828d7ea perf: "mixify" project to improve release speed 
Fixes https://emqx.atlassian.net/browse/EMQX-12527
 2024-07-02 09:39:48 -03:00
Andrew Mayorov 30efa1f57e
test(dsrepl): relax crash-recover testcase to tolerate message loss 
Which is quite an expected occasion for this kind of stress test.
 2024-06-26 20:44:34 +02:00
Andrew Mayorov 733751fadd
refactor(dsstore): keep passing `Options` to both prepare + commit 2024-06-26 20:44:34 +02:00
Andrew Mayorov 5b5f33c421
chore(dsstore): resurrect `prepare_batch` entry tracepoint 2024-06-26 20:44:34 +02:00
Andrew Mayorov 8db70b5bbc
test(dsrepl): add crash-restart-recover testcase 
That verifies nothing is lost in the event of abrupt node failures.
 2024-06-26 20:44:33 +02:00
Andrew Mayorov 5fd5fc76e5
fix(dsstore): ensure backward compatibility 2024-06-26 20:44:33 +02:00
Andrew Mayorov 8538a5a5b6
test(dsrepl): anticipate transitionless membership changes 
E.g. when a membership change is applied twice in a row.
 2024-06-26 20:44:33 +02:00
Andrew Mayorov 2180cc7c26
fix(dsstore): avoid storing `cf_refs()` in the RocksDB itself 
This is both pointless and confusing.
 2024-06-26 20:44:33 +02:00
Andrew Mayorov 7895e9cc45
feat(dsstore): make WAL-less mode optional 
And make the upper layer choose when to use it.
 2024-06-26 20:44:32 +02:00
Andrew Mayorov 0c0757b8c2
feat(dsrepl): enable WAL-less batch writes 2024-06-26 20:44:32 +02:00
Andrew Mayorov b6894c18fa
chore(dsrepl): improve tracepoints usability a bit 2024-06-26 20:44:32 +02:00
ieQu1 d349f84f04
fix(ds): Apply remarks 2024-06-24 20:20:40 +02:00
zmstone 5d1608f34b chore: minimize oss/platform diff 2024-06-24 15:30:03 +02:00
ieQu1 9a58d71378
fix(ds): Move DS backend supervision trees to their own apps 2024-06-21 18:22:04 +02:00
ieQu1 8aa27488b6
test: Disable certain DS-related suites in CE 2024-06-21 17:16:24 +02:00
ieQu1 1d3b1868fb
test(ds): Use close_db API 2024-06-21 17:16:24 +02:00
ieQu1 3851fc189f
fix(ds): Avoid reverse dependencies from storage to repl. layer 2024-06-21 17:16:24 +02:00
ieQu1 bc915216a0
feat(ds): Support metrics for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1 99c9b56cf3
feat(ds_buffer): Add `ls' function to list all local buffers 2024-06-21 17:16:24 +02:00
ieQu1 ecb172b07e
refactor(ds): Rename egress metrics to 'buffer' 2024-06-21 17:16:24 +02:00
ieQu1 09c3ae795d
refactor(ds_raft): Replace egress server with common emqx_ds_buffer 2024-06-21 17:16:24 +02:00
ieQu1 a0fbd37e58
refactor(emqx): Use emqx_ds_backends application 2024-06-21 17:16:24 +02:00
ieQu1 ef09cfcd71
feat(ds): Add `emqx_ds_backends` application 2024-06-21 17:16:24 +02:00
ieQu1 279619fc80
feat(ds): Add `builtin_local' backend 2024-06-21 17:16:24 +02:00
ieQu1 a8ea0ae4e5
refactor(ds): Extract DS replication layer to a separate application 2024-06-21 17:16:24 +02:00
ieQu1 63f1856a2c
feat(ds): Dynamic backend registration 2024-06-21 17:16:24 +02:00
ieQu1 83dc8f4d77
fix(ds): Fix return values of emqx_ds_storage_layer functions 2024-06-21 17:16:23 +02:00
ieQu1 a18d1987a2
test(ds): Add a helper function for diffing messages 2024-06-21 17:16:23 +02:00
zmstone 213e4785e7 Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master 2024-06-19 14:54:48 +02:00
Thales Macedo Garitezi 8ce16fd7d9 fix(dsreplmeta): check site status when fetching shard info 
Fixes https://emqx.atlassian.net/browse/EMQX-12356
 2024-06-18 10:40:09 -03:00