yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,422 Commits 393 Branches 669 Tags 74 MiB
dependabot/github_actions/actions-51fe3393f3
Commit Graph

352 Commits

Author SHA1 Message Date
ieQu1
3721be65ee
fix(ds): Improve comments 2024-07-09 13:15:15 +02:00
ieQu1
dc4ae82798
test(ds): Add message ID 2024-07-09 13:15:14 +02:00
ieQu1
661f79544b
fix(ds): Optimize hot loop of skipstream storage layout 2024-07-09 13:15:14 +02:00
ieQu1
23dafbb03b
feat(ds): Add a benchmarking tool for storage efficiency analysis 2024-07-09 13:15:14 +02:00
ieQu1
afeb2ab8aa
feat(ds): Add metrics for skipstream layout 2024-07-09 13:15:14 +02:00
ieQu1
8c5e4a2376
test(ds): Generalize storage layout test suite for different layouts 2024-07-09 13:15:14 +02:00
ieQu1
a4642d4d06
feat(ds): Add a new storage layout engine: "skipstream" 
This layout is based on LTS as well, but it uses separate index
streams for constrained replay of streams with learned wildcards
 2024-07-09 13:15:14 +02:00
ieQu1
de48077ac4
test(ds): Add new helper functions 
- Improve message comparison
- Add set operations
 2024-07-09 13:15:14 +02:00
ieQu1
210556e545
feat(ds): Generalize value serialization 
- Add a new utility module for message serialization
- Add experimental serializer based on ASN.1
 2024-07-09 13:15:14 +02:00
ieQu1
843973ef32
fix(ds): bitfield_lts: static_key_size -> static_key_bits 2024-07-09 13:15:14 +02:00
ieQu1
f84fb34692
feat(ds_lts): New APIs: info, reverse lookups and topic compression 2024-07-09 13:15:14 +02:00
ieQu1
eb80402ccb
fix(ds): Improve typespecs and descriptions in storage_layer 2024-07-09 13:15:14 +02:00
ieQu1
71dad0242e
docs(ds): Move Raft-related parts to emqx_ds_builtin_raft README 2024-07-09 13:15:14 +02:00
ieQu1
afe1c5617d
refactor(ds): Rename macros for bitfield_lts metrics 2024-07-09 13:15:14 +02:00
ieQu1
0f2c19b656
refactor(ds): Move end_of_stream detection logic for delete_next 2024-07-09 13:15:14 +02:00
ieQu1
b565976794
fix(ds): Fix hashing of empty wildcard topic levels in bitfield_lts 2024-07-09 13:15:13 +02:00
Thales Macedo Garitezi
32ace85e1c Merge branch 'release-57' into 20240703-m-sync-r57 2024-07-03 16:03:15 -03:00
Thales Macedo Garitezi
afb65817c4 ci(mix): refactor to avoid conflicts 
This introduces a refactoring that will hopefully help avoid conflicts with ongoing work
to migrate our build / CI tooling to Mix.
 2024-07-03 14:37:25 -03:00
Thales Macedo Garitezi
5532f40d83 Merge branch 'release-57' into 20240702-m-sync-r57-mix-umbrella 2024-07-02 11:52:36 -03:00
Thales Macedo Garitezi
19f3b030f9 chore: preparing to run common tests / eunit with mix 2024-07-02 09:45:45 -03:00
Thales Macedo Garitezi
b74828d7ea perf: "mixify" project to improve release speed 
Fixes https://emqx.atlassian.net/browse/EMQX-12527
 2024-07-02 09:39:48 -03:00
Andrew Mayorov
30efa1f57e
test(dsrepl): relax crash-recover testcase to tolerate message loss 
Which is quite an expected occasion for this kind of stress test.
 2024-06-26 20:44:34 +02:00
Andrew Mayorov
733751fadd
refactor(dsstore): keep passing Options to both prepare + commit 2024-06-26 20:44:34 +02:00
Andrew Mayorov
5b5f33c421
chore(dsstore): resurrect prepare_batch entry tracepoint 2024-06-26 20:44:34 +02:00
Andrew Mayorov
8db70b5bbc
test(dsrepl): add crash-restart-recover testcase 
That verifies nothing is lost in the event of abrupt node failures.
 2024-06-26 20:44:33 +02:00
Andrew Mayorov
5fd5fc76e5
fix(dsstore): ensure backward compatibility 2024-06-26 20:44:33 +02:00
Andrew Mayorov
8538a5a5b6
test(dsrepl): anticipate transitionless membership changes 
E.g. when a membership change is applied twice in a row.
 2024-06-26 20:44:33 +02:00
Andrew Mayorov
2180cc7c26
fix(dsstore): avoid storing cf_refs() in the RocksDB itself 
This is both pointless and confusing.
 2024-06-26 20:44:33 +02:00
Andrew Mayorov
7895e9cc45
feat(dsstore): make WAL-less mode optional 
And make the upper layer choose when to use it.
 2024-06-26 20:44:32 +02:00
Andrew Mayorov
0c0757b8c2
feat(dsrepl): enable WAL-less batch writes 2024-06-26 20:44:32 +02:00
Andrew Mayorov
b6894c18fa
chore(dsrepl): improve tracepoints usability a bit 2024-06-26 20:44:32 +02:00
ieQu1
d349f84f04
fix(ds): Apply remarks 2024-06-24 20:20:40 +02:00
zmstone
5d1608f34b chore: minimize oss/platform diff 2024-06-24 15:30:03 +02:00
ieQu1
9a58d71378
fix(ds): Move DS backend supervision trees to their own apps 2024-06-21 18:22:04 +02:00
ieQu1
8aa27488b6
test: Disable certain DS-related suites in CE 2024-06-21 17:16:24 +02:00
ieQu1
1d3b1868fb
test(ds): Use close_db API 2024-06-21 17:16:24 +02:00
ieQu1
3851fc189f
fix(ds): Avoid reverse dependencies from storage to repl. layer 2024-06-21 17:16:24 +02:00
ieQu1
bc915216a0
feat(ds): Support metrics for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1
99c9b56cf3
feat(ds_buffer): Add `ls' function to list all local buffers 2024-06-21 17:16:24 +02:00
ieQu1
ecb172b07e
refactor(ds): Rename egress metrics to 'buffer' 2024-06-21 17:16:24 +02:00
ieQu1
09c3ae795d
refactor(ds_raft): Replace egress server with common emqx_ds_buffer 2024-06-21 17:16:24 +02:00
ieQu1
a0fbd37e58
refactor(emqx): Use emqx_ds_backends application 2024-06-21 17:16:24 +02:00
ieQu1
ef09cfcd71
feat(ds): Add emqx_ds_backends application 2024-06-21 17:16:24 +02:00
ieQu1
279619fc80
feat(ds): Add `builtin_local' backend 2024-06-21 17:16:24 +02:00
ieQu1
a8ea0ae4e5
refactor(ds): Extract DS replication layer to a separate application 2024-06-21 17:16:24 +02:00
ieQu1
63f1856a2c
feat(ds): Dynamic backend registration 2024-06-21 17:16:24 +02:00
ieQu1
83dc8f4d77
fix(ds): Fix return values of emqx_ds_storage_layer functions 2024-06-21 17:16:23 +02:00
ieQu1
a18d1987a2
test(ds): Add a helper function for diffing messages 2024-06-21 17:16:23 +02:00
zmstone
213e4785e7 Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master 2024-06-19 14:54:48 +02:00
Thales Macedo Garitezi
8ce16fd7d9 fix(dsreplmeta): check site status when fetching shard info 
Fixes https://emqx.atlassian.net/browse/EMQX-12356
 2024-06-18 10:40:09 -03:00