yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,397 Commits 393 Branches 684 Tags 74 MiB
cfcd81e17b
Commit Graph

16397 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
ieQu1
cfcd81e17b fix(ds): Fix build 2023-06-22 11:07:52 +02:00
ieQu1
b29c5ad23c feat(emqx_ds): Add API draft for logic layer 2023-06-22 10:20:18 +02:00
ieQu1
a343cdb1d5 refactor(ds): message_storage -> message_storage_bitmask 2023-06-22 10:20:18 +02:00
ieQu1
a4219db163 refactor(ds): emqx_ds_local_store -> emqx_ds_storage_layer 2023-06-22 10:20:18 +02:00
ieQu1
1159f99432 refactor(ds): emqx_replay -> emqx_ds 2023-06-22 10:20:18 +02:00
ieQu1
8d6bcc1414 refactor(ds): emqx_replay -> emqx_durable_storage 2023-06-22 10:20:18 +02:00
ieQu1
c4f721f214 chore(ds): Add callback definitions to local_store 2023-06-22 10:20:18 +02:00
ieQu1
efed7cdc2f chore(ds): behavior -> behaviour 2023-06-22 10:20:18 +02:00
ieQu1
0f6c81e42d chore(ds): Change license to BSL 2023-06-22 10:20:18 +02:00
ieQu1
04adb65c09 refactor(ds): zone -> shard 
Also bump erlang-rocksdb version
 2023-06-22 10:20:18 +02:00
Andrew Mayorov
8ac0bba958 test(ds): verify preserve / restore works with stored iterators 2023-06-22 10:20:18 +02:00
Andrew Mayorov
d2065e0c1b feat(ds): allow to preserve / restore iterators in the db 
So that we could guarantee replay consistency / availability under
the assumption that nodes may be restarted or even lost occasionally.
 2023-06-22 10:20:18 +02:00
Andrew Mayorov
cbded9af39 feat(ds): Make create_generation safer against bad input 2023-06-22 10:20:18 +02:00
Andrew Mayorov
83d1606d00 feat(ds): Make iteration fully generation-aware 2023-06-22 10:20:18 +02:00
Andrew Mayorov
bf9d57f6a9 chore(ds): Simplify iteration-related typespecs 2023-06-22 10:20:18 +02:00
Andrew Mayorov
cf6a5e1643 feat(ds): Allow to create new storage generations 2023-06-22 10:20:18 +02:00
Andrew Mayorov
f80e906553 fix(ds): Correct typespec 2023-06-22 10:20:18 +02:00
Andrew Mayorov
f5a7b49f57 feat(ds): Wire iteration options up to the app config 2023-06-22 10:20:18 +02:00
Andrew Mayorov
2bf8a07b05 test(ds): Split unit tests off into a full-fledged suite 2023-06-22 10:20:18 +02:00
Andrew Mayorov
cf70184ebd test(ds): Provide more general keymapper_info/1 2023-06-22 10:20:18 +02:00
Andrew Mayorov
2f385022cd chore(ds): Rename testsuite to reflect test subject better 2023-06-22 10:20:18 +02:00
Andrew Mayorov
d875fa49d3 feat(ds): Enable periodic iterator refresh 
This might be helpful during replays taking multiple tens of seconds so
that underlying iterators won't hold onto in-memory / on-disk data
structures for too long, preventing rocksdb from recycling them.
 2023-06-22 10:20:18 +02:00
Andrew Mayorov
7f408da251 refactor(ds): Use inline functions instead of macros 2023-06-22 10:20:07 +02:00
Andrew Mayorov
a11e75d189 fix(ds): clear bitmask of topic filter tail containing wildcards 2023-06-22 10:16:55 +02:00
Andrew Mayorov
ac0935ef91 test(ds): Proptest that iteration is exhaustive 
Compare iteration results against what an extremely simplified model
produces.
 2023-06-22 10:16:45 +02:00
Andrew Mayorov
60e3070328 test(ds): Scale up number of messages per topic in proptests 2023-06-22 10:16:41 +02:00
Andrew Mayorov
82a7292851 test(ds): Use _build/test/proper as a scratch dir for testruns 2023-06-22 10:16:32 +02:00
Andrew Mayorov
440fecd268 test(ds): Add a proptest on iterator preserve / restore 
Which verifies that preservation and restoration of iterators does not
affect the outcome of an iteration (under the precondition that the
state of database is constant during an iteration).
 2023-06-22 10:16:23 +02:00
Andrew Mayorov
1f5425975f feat(ds): add an ability to preserve and restore iterators 
This will allow to persist iteration state and to periodically recreate
iterators during long replays.
 2023-06-22 10:16:20 +02:00
Andrew Mayorov
94e8dbfe7f test(ds): Move proptests into a separate module 
Following conventions. Also add few proptests on keyspace filters.
 2023-06-22 10:16:16 +02:00
Andrew Mayorov
4c643aa7d5 refactor(ds): Introduce keyspace filter concept 
So we could conveniently test it separately.
 2023-06-22 10:16:10 +02:00
Andrew Mayorov
c5826cee56 refactor(ds): rename compute_hash_* → compute_topic_* 2023-06-22 10:16:05 +02:00
Andrew Mayorov
862e17cd78 chore(ds): assign CODEOWNER 2023-06-22 10:16:01 +02:00
Andrew Mayorov
85d3783ff1 chore(ds): Drop few unused macrodefinitions 2023-06-22 10:15:56 +02:00
Andrew Mayorov
a0bcdb5104 chore(ds): Attempt to make compute_next_seek's logic clearer 2023-06-22 10:15:49 +02:00
ieQu1
f1e39d977a fix(ds): Fix dialyzer warnings 2023-06-22 10:15:47 +02:00
ieQu1
adcbf40d27 refactor(ds): tau -> epoch 2023-06-22 10:15:44 +02:00
ieQu1
120d4e66ae refactor(ds): Factor out configuration to a separate module 2023-06-22 10:15:42 +02:00
ieQu1
0bbc5ecb32 refactor(ds): Introduce bitwise_concat function 2023-06-22 10:15:40 +02:00
ieQu1
b5bb77dd58 feat(ds): Add metadata storage and supervision tree 2023-06-22 10:15:37 +02:00
Andrew Mayorov
5e30a5d3dd feat(ds): Implement keyspace partitioning across time 2023-06-22 10:15:32 +02:00
Andrew Mayorov
8c9c9cc669 feat(ds): Allow to specify message store options 
* Keymapper
* Column family name + DB options
* DB write / read options
 2023-06-22 10:15:25 +02:00
Andrew Mayorov
5b6c4d766b test(ds): Add basic property tests 2023-06-22 10:15:20 +02:00
Andrew Mayorov
a1e29e4e24 refactor(ds): Improve function naming 2023-06-22 10:15:03 +02:00
Andrew Mayorov
cd12338c3f feat(ds): Smoke tests for wildcard iterator scans 2023-06-22 10:14:46 +02:00
Andrew Mayorov
9c1cd4911d feat(ds): Implement iterator next 2023-06-22 10:14:43 +02:00
ieQu1
27b925405b test(ds): Add testcase 2023-06-22 10:14:41 +02:00
Andrew Mayorov
691a27883d feat(ds): Initial commit 2023-06-22 10:14:37 +02:00
Paulo Zulato
62d3766726
Merge pull request #10645 from paulozulato/data-bridge-target-unavailable 
Data bridge target unavailable
 2023-06-21 18:19:23 -03:00
Andrew Mayorov
62b832be45
Merge pull request #11118 from fix/EMQX-9964/bump-hocon 
chore: bump hocon to 0.39.10
 2023-06-21 23:13:35 +02:00