Ilya Averyanov
a1b9a14fa1
feat(ds): allow fdb implementation for durable storage
2023-11-22 14:49:38 +03:00
ieQu1
3165b4f645
fix(ds): Abort application startup when rocksdb is not avialable
2023-11-21 20:27:07 +01:00
ieQu1
3d823beb11
fix(ds): Apply review remarks
2023-11-21 19:37:01 +01:00
ieQu1
f5c71e8068
refactor(ds): Add a wrapper to the store batch API
2023-11-21 19:37:01 +01:00
ieQu1
2a1f7d946a
feat(ds): Shard messages by publisher client ID
2023-11-21 19:37:01 +01:00
ieQu1
62542e5844
feat(ds): Metadata storage for the replication layer
2023-11-21 19:37:01 +01:00
Ivan Dyachkov
28a577ad09
chore: bump apps versions
2023-11-14 11:02:26 +01:00
Thales Macedo Garitezi
088de9476c
fix(ds): use integer for tag values
...
Follow up to https://github.com/emqx/emqx/pull/11906#discussion_r1389115973
2023-11-10 11:26:32 -03:00
ieQu1
2ade6da7a6
fix(ds): Remove redundand type
2023-11-10 12:53:30 +01:00
ieQu1
feef23fc08
feat(ds): Pass store_batch through RPC
2023-11-10 12:16:39 +01:00
ieQu1
8dc8237331
feat(ds): Introduce emqx_ds behavior
2023-11-10 12:16:39 +01:00
Thales Macedo Garitezi
42536bb5f7
chore(ds): avoid using records in persistence / rpc
...
Fixes https://emqx.atlassian.net/browse/EMQX-11279
2023-11-09 12:00:25 -03:00
ieQu1
a1cdbaa76d
fix(ds): Address code review remarks
2023-11-08 04:31:27 +01:00
ieQu1
c030188eb7
chore(ds): Add rebar.config file to app/emqx_durable_storage
2023-11-08 03:04:19 +01:00
ieQu1
7cb0322856
fix(emqx): Move bpapi and emqx message record to emqx_utils app
2023-11-08 03:04:19 +01:00
ieQu1
74cb43f8b1
fix(ds): Add unique ID to the key
2023-11-08 03:04:19 +01:00
Andrew Mayorov
54951c273f
feat(ds): mix safe cutoff into keymapper filter during iteration
2023-11-08 03:04:19 +01:00
Andrew Mayorov
7a94db25c3
fix(ds): don't iterate over incomplete epoch in bitmask lts storage
2023-11-08 03:04:19 +01:00
ieQu1
38b7f516bc
fix(ds): Change the directory where message database is stored
2023-11-08 03:04:19 +01:00
ieQu1
8e5dda40be
fix(ds): Fix static checks
2023-11-08 03:04:19 +01:00
ieQu1
99329e1243
refactor(ds): Address review remarks
2023-11-08 03:04:19 +01:00
ieQu1
2de79dd9ac
feat(ds): Replay QoS1 messages
2023-11-08 03:04:19 +01:00
ieQu1
e745e42093
test(ds): Explore full range of keys when testing ratchet function
2023-11-08 03:04:19 +01:00
ieQu1
87689890ff
chore(ds): Fix linter and compilation warnings
2023-11-08 03:04:19 +01:00
ieQu1
ef46c09caf
feat(ds): Implement ratchet function for bitmask keymapper
2023-11-08 03:04:19 +01:00
ieQu1
164ae9e94a
feat(ds): LTS bitfield storage passes all tests
2023-11-08 03:04:19 +01:00
ieQu1
56b6b176c2
fix(ds): LTS shall keeps the concrete topic indexes
2023-11-08 03:04:19 +01:00
ieQu1
7428e7037b
feat(ds): Bitfield + Learned Topic Structure
2023-11-08 03:04:19 +01:00
ieQu1
ac91dbc58f
feat(ds): Restore LTS trie from a dump
2023-11-08 03:04:19 +01:00
ieQu1
f1ab7c8a7c
feat(ds): Add persist callback to LTS trie
2023-11-08 03:04:19 +01:00
ieQu1
c149e0e2df
fix(ds): Pass topic filter to emqx_ds:make_iterator call
2023-11-08 03:04:19 +01:00
ieQu1
51a6f623fd
refactor(ds): Split out bitfield keymapper to a different module
2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
903b3863d1
chore(ps_ds): make persistent session module use new emqx_ds APIs
2023-11-08 03:04:19 +01:00
ieQu1
2972bf14ee
refactor(ds): Implement create_generation gen_rpc storage layer call
2023-11-08 03:04:19 +01:00
ieQu1
6d65707d41
refactor(ds): Implement drop_db function
2023-11-08 03:04:19 +01:00
ieQu1
c6a721a7eb
refactor(ds): Passthrough open_db and get_channels to storage layer
2023-11-08 03:04:19 +01:00
ieQu1
59d01dc823
refactor(ds): Implement emqx_ds:open_db
2023-11-08 03:04:19 +01:00
ieQu1
7095cb8583
refactor(ds): Refactor storage layer
2023-11-08 03:04:19 +01:00
ieQu1
c91df2f5cd
refactor(ds): Create a prototype of replication layer
2023-11-08 03:04:19 +01:00
ieQu1
f1454bb57e
feat(ds): learned topic structure
2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
7ab57824dc
chore(ds): change return type of storage_layer:next/{1,2}
...
Part of https://emqx.atlassian.net/browse/EMQX-10942
The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.
From:
```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```
To:
```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.
-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
2023-11-08 03:04:19 +01:00
Zaiming (Stone) Shi
ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53
...
0928 sync release 53
2023-10-02 21:57:32 +02:00
Zaiming (Stone) Shi
3f6e0e890b
chore: bump emqx_durable_storage and emqx_license app vsn
2023-10-02 20:13:09 +02:00
Thales Macedo Garitezi
2358d67908
refactor: move session stuff from emqx_ds to emqx_persistent_session_ds
...
Part of https://emqx.atlassian.net/browse/EMQX-10942
2023-09-29 18:00:24 -03:00
Ivan Dyachkov
dafd7c6085
chore: bump apps versions
2023-09-21 10:58:42 +02:00
Ivan Dyachkov
105bebc250
chore: merge release-52 into master
2023-09-21 10:22:47 +02:00
Andrew Mayorov
a2ddd9d5f5
fix(session): respect existing session even if expiry interval = 0
...
If the original connection had Session-Expiry-Interval > 0, and the
new connection set Session-Expiry-Interval = 0, the MQTTv5 spec says
that (supposedly) we still have to continue with the existing session
(if it hasn't expired yet).
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:54 +04:00
Andrew Mayorov
c1583f7f9d
fix(ds): refine topic() type to describe parsed topics
...
And separate it from `topic_filter()` type, which describes parsed
topic filters.
2023-09-20 22:55:52 +04:00
Andrew Mayorov
7a9916c84d
fix(sessds): convert ds iterator topics upon opening ds session
2023-09-20 22:55:25 +04:00
Andrew Mayorov
045d8b7f10
refactor(ds): reorder functions to improve readability
2023-09-20 22:55:24 +04:00
