28a577ad09
chore: bump apps versions
2023-11-14 11:02:26 +01:00
088de9476c
fix(ds): use integer for tag values
...
Follow up to https://github.com/emqx/emqx/pull/11906#discussion_r1389115973
2023-11-10 11:26:32 -03:00
2ade6da7a6
fix(ds): Remove redundand type
2023-11-10 12:53:30 +01:00
feef23fc08
feat(ds): Pass store_batch through RPC
2023-11-10 12:16:39 +01:00
8dc8237331
feat(ds): Introduce emqx_ds behavior
2023-11-10 12:16:39 +01:00
42536bb5f7
chore(ds): avoid using records in persistence / rpc
...
Fixes https://emqx.atlassian.net/browse/EMQX-11279
2023-11-09 12:00:25 -03:00
a1cdbaa76d
fix(ds): Address code review remarks
2023-11-08 04:31:27 +01:00
c030188eb7
chore(ds): Add rebar.config file to app/emqx_durable_storage
2023-11-08 03:04:19 +01:00
7cb0322856
fix(emqx): Move bpapi and emqx message record to emqx_utils app
2023-11-08 03:04:19 +01:00
74cb43f8b1
fix(ds): Add unique ID to the key
2023-11-08 03:04:19 +01:00
54951c273f
feat(ds): mix safe cutoff into keymapper filter during iteration
2023-11-08 03:04:19 +01:00
7a94db25c3
fix(ds): don't iterate over incomplete epoch in bitmask lts storage
2023-11-08 03:04:19 +01:00
38b7f516bc
fix(ds): Change the directory where message database is stored
2023-11-08 03:04:19 +01:00
8e5dda40be
fix(ds): Fix static checks
2023-11-08 03:04:19 +01:00
99329e1243
refactor(ds): Address review remarks
2023-11-08 03:04:19 +01:00
2de79dd9ac
feat(ds): Replay QoS1 messages
2023-11-08 03:04:19 +01:00
e745e42093
test(ds): Explore full range of keys when testing ratchet function
2023-11-08 03:04:19 +01:00
87689890ff
chore(ds): Fix linter and compilation warnings
2023-11-08 03:04:19 +01:00
ef46c09caf
feat(ds): Implement ratchet function for bitmask keymapper
2023-11-08 03:04:19 +01:00
164ae9e94a
feat(ds): LTS bitfield storage passes all tests
2023-11-08 03:04:19 +01:00
56b6b176c2
fix(ds): LTS shall keeps the concrete topic indexes
2023-11-08 03:04:19 +01:00
7428e7037b
feat(ds): Bitfield + Learned Topic Structure
2023-11-08 03:04:19 +01:00
ac91dbc58f
feat(ds): Restore LTS trie from a dump
2023-11-08 03:04:19 +01:00
f1ab7c8a7c
feat(ds): Add persist callback to LTS trie
2023-11-08 03:04:19 +01:00
c149e0e2df
fix(ds): Pass topic filter to emqx_ds:make_iterator call
2023-11-08 03:04:19 +01:00
51a6f623fd
refactor(ds): Split out bitfield keymapper to a different module
2023-11-08 03:04:19 +01:00
903b3863d1
chore(ps_ds): make persistent session module use new `emqx_ds` APIs
2023-11-08 03:04:19 +01:00
2972bf14ee
refactor(ds): Implement create_generation gen_rpc storage layer call
2023-11-08 03:04:19 +01:00
6d65707d41
refactor(ds): Implement drop_db function
2023-11-08 03:04:19 +01:00
c6a721a7eb
refactor(ds): Passthrough open_db and get_channels to storage layer
2023-11-08 03:04:19 +01:00
59d01dc823
refactor(ds): Implement emqx_ds:open_db
2023-11-08 03:04:19 +01:00
7095cb8583
refactor(ds): Refactor storage layer
2023-11-08 03:04:19 +01:00
c91df2f5cd
refactor(ds): Create a prototype of replication layer
2023-11-08 03:04:19 +01:00
f1454bb57e
feat(ds): learned topic structure
2023-11-08 03:04:19 +01:00
7ab57824dc
chore(ds): change return type of `storage_layer:next/{1,2}`
...
Part of https://emqx.atlassian.net/browse/EMQX-10942
The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.
From:
```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```
To:
```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.
-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
2023-11-08 03:04:19 +01:00
ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53
...
0928 sync release 53
2023-10-02 21:57:32 +02:00
3f6e0e890b
chore: bump emqx_durable_storage and emqx_license app vsn
2023-10-02 20:13:09 +02:00
2358d67908
refactor: move session stuff from `emqx_ds` to `emqx_persistent_session_ds`
...
Part of https://emqx.atlassian.net/browse/EMQX-10942
2023-09-29 18:00:24 -03:00
dafd7c6085
chore: bump apps versions
2023-09-21 10:58:42 +02:00
105bebc250
chore: merge release-52 into master
2023-09-21 10:22:47 +02:00
a2ddd9d5f5
fix(session): respect existing session even if expiry interval = 0
...
If the original connection had Session-Expiry-Interval > 0, and the
new connection set Session-Expiry-Interval = 0, the MQTTv5 spec says
that (supposedly) we still have to continue with the existing session
(if it hasn't expired yet).
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:54 +04:00
c1583f7f9d
fix(ds): refine `topic()` type to describe parsed topics
...
And separate it from `topic_filter()` type, which describes parsed
topic filters.
2023-09-20 22:55:52 +04:00
7a9916c84d
fix(sessds): convert ds iterator topics upon opening ds session
2023-09-20 22:55:25 +04:00
045d8b7f10
refactor(ds): reorder functions to improve readability
2023-09-20 22:55:24 +04:00
7326ef550b
fix(sessds): make existing parts of persistent session impl work
2023-09-20 22:55:22 +04:00
5ed5ac48ee
refactor: combine shard id and keyspace into a single value
2023-09-19 12:31:32 -03:00
a511088fd4
refactor: address review comments
2023-09-18 17:47:56 -03:00
b30bcf32bd
feat(ds): introduce keyspace concept
...
Fixes https://emqx.atlassian.net/browse/EMQX-10579
This introduces the concept of "keyspaces" to our durable storage (DS) implementation, and
also refactors some places where "shard" and "keyspace" would be mixed up.
We might want to tune the storage options differently for distinct sets of topics, the
keyspaces. The keyspace is composed by one or more shards.
- Keyspaces are identified simply by binary strings.
- DS configuration is scoped by keyspaces instead of shards.
- Starting a new DS shard requires definining to which keyspace the shard belongs.
2023-09-14 16:39:00 -03:00
cae9ae1fab
fix(ps_router): use `disc_copies` for storing persistent session routes
2023-09-13 10:02:39 -03:00
ea73d674b9
chore: bump version to e5.2.1-alpha.2
2023-09-13 14:55:17 +02:00
Ivan Dyachkov