yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,577 Commits 393 Branches 669 Tags 74 MiB
d214ae8772
Commit Graph

80 Commits

Author SHA1 Message Date
Ivan Dyachkov
28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Thales Macedo Garitezi
088de9476c fix(ds): use integer for tag values 
Follow up to https://github.com/emqx/emqx/pull/11906#discussion_r1389115973
 2023-11-10 11:26:32 -03:00
ieQu1
2ade6da7a6 fix(ds): Remove redundand type 2023-11-10 12:53:30 +01:00
ieQu1
feef23fc08 feat(ds): Pass store_batch through RPC 2023-11-10 12:16:39 +01:00
ieQu1
8dc8237331 feat(ds): Introduce emqx_ds behavior 2023-11-10 12:16:39 +01:00
Thales Macedo Garitezi
42536bb5f7 chore(ds): avoid using records in persistence / rpc 
Fixes https://emqx.atlassian.net/browse/EMQX-11279
 2023-11-09 12:00:25 -03:00
ieQu1
a1cdbaa76d fix(ds): Address code review remarks 2023-11-08 04:31:27 +01:00
ieQu1
c030188eb7 chore(ds): Add rebar.config file to app/emqx_durable_storage 2023-11-08 03:04:19 +01:00
ieQu1
7cb0322856 fix(emqx): Move bpapi and emqx message record to emqx_utils app 2023-11-08 03:04:19 +01:00
ieQu1
74cb43f8b1 fix(ds): Add unique ID to the key 2023-11-08 03:04:19 +01:00
Andrew Mayorov
54951c273f feat(ds): mix safe cutoff into keymapper filter during iteration 2023-11-08 03:04:19 +01:00
Andrew Mayorov
7a94db25c3 fix(ds): don't iterate over incomplete epoch in bitmask lts storage 2023-11-08 03:04:19 +01:00
ieQu1
38b7f516bc fix(ds): Change the directory where message database is stored 2023-11-08 03:04:19 +01:00
ieQu1
8e5dda40be fix(ds): Fix static checks 2023-11-08 03:04:19 +01:00
ieQu1
99329e1243 refactor(ds): Address review remarks 2023-11-08 03:04:19 +01:00
ieQu1
2de79dd9ac feat(ds): Replay QoS1 messages 2023-11-08 03:04:19 +01:00
ieQu1
e745e42093 test(ds): Explore full range of keys when testing ratchet function 2023-11-08 03:04:19 +01:00
ieQu1
87689890ff chore(ds): Fix linter and compilation warnings 2023-11-08 03:04:19 +01:00
ieQu1
ef46c09caf feat(ds): Implement ratchet function for bitmask keymapper 2023-11-08 03:04:19 +01:00
ieQu1
164ae9e94a feat(ds): LTS bitfield storage passes all tests 2023-11-08 03:04:19 +01:00
ieQu1
56b6b176c2 fix(ds): LTS shall keeps the concrete topic indexes 2023-11-08 03:04:19 +01:00
ieQu1
7428e7037b feat(ds): Bitfield + Learned Topic Structure 2023-11-08 03:04:19 +01:00
ieQu1
ac91dbc58f feat(ds): Restore LTS trie from a dump 2023-11-08 03:04:19 +01:00
ieQu1
f1ab7c8a7c feat(ds): Add persist callback to LTS trie 2023-11-08 03:04:19 +01:00
ieQu1
c149e0e2df fix(ds): Pass topic filter to emqx_ds:make_iterator call 2023-11-08 03:04:19 +01:00
ieQu1
51a6f623fd refactor(ds): Split out bitfield keymapper to a different module 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
903b3863d1 chore(ps_ds): make persistent session module use new emqx_ds APIs 2023-11-08 03:04:19 +01:00
ieQu1
2972bf14ee refactor(ds): Implement create_generation gen_rpc storage layer call 2023-11-08 03:04:19 +01:00
ieQu1
6d65707d41 refactor(ds): Implement drop_db function 2023-11-08 03:04:19 +01:00
ieQu1
c6a721a7eb refactor(ds): Passthrough open_db and get_channels to storage layer 2023-11-08 03:04:19 +01:00
ieQu1
59d01dc823 refactor(ds): Implement emqx_ds:open_db 2023-11-08 03:04:19 +01:00
ieQu1
7095cb8583 refactor(ds): Refactor storage layer 2023-11-08 03:04:19 +01:00
ieQu1
c91df2f5cd refactor(ds): Create a prototype of replication layer 2023-11-08 03:04:19 +01:00
ieQu1
f1454bb57e feat(ds): learned topic structure 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
7ab57824dc chore(ds): change return type of storage_layer:next/{1,2} 
Part of https://emqx.atlassian.net/browse/EMQX-10942

The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.

From:

```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```

To:

```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.

-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
 2023-11-08 03:04:19 +01:00
Zaiming (Stone) Shi
ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53 
0928 sync release 53
 2023-10-02 21:57:32 +02:00
Zaiming (Stone) Shi
3f6e0e890b chore: bump emqx_durable_storage and emqx_license app vsn 2023-10-02 20:13:09 +02:00
Thales Macedo Garitezi
2358d67908 refactor: move session stuff from emqx_ds to emqx_persistent_session_ds 
Part of https://emqx.atlassian.net/browse/EMQX-10942
 2023-09-29 18:00:24 -03:00
Ivan Dyachkov
dafd7c6085 chore: bump apps versions 2023-09-21 10:58:42 +02:00
Ivan Dyachkov
105bebc250 chore: merge release-52 into master 2023-09-21 10:22:47 +02:00
Andrew Mayorov
a2ddd9d5f5
fix(session): respect existing session even if expiry interval = 0 
If the original connection had Session-Expiry-Interval > 0, and the
new connection set Session-Expiry-Interval = 0, the MQTTv5 spec says
that (supposedly) we still have to continue with the existing session
(if it hasn't expired yet).

Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-09-20 22:55:54 +04:00
Andrew Mayorov
c1583f7f9d
fix(ds): refine topic() type to describe parsed topics 
And separate it from `topic_filter()` type, which describes parsed
topic filters.
 2023-09-20 22:55:52 +04:00
Andrew Mayorov
7a9916c84d
fix(sessds): convert ds iterator topics upon opening ds session 2023-09-20 22:55:25 +04:00
Andrew Mayorov
045d8b7f10
refactor(ds): reorder functions to improve readability 2023-09-20 22:55:24 +04:00
Andrew Mayorov
7326ef550b
fix(sessds): make existing parts of persistent session impl work 2023-09-20 22:55:22 +04:00
Thales Macedo Garitezi
5ed5ac48ee refactor: combine shard id and keyspace into a single value 2023-09-19 12:31:32 -03:00
Thales Macedo Garitezi
a511088fd4 refactor: address review comments 2023-09-18 17:47:56 -03:00
Thales Macedo Garitezi
b30bcf32bd feat(ds): introduce keyspace concept 
Fixes https://emqx.atlassian.net/browse/EMQX-10579

This introduces the concept of "keyspaces" to our durable storage (DS) implementation, and
also refactors some places where "shard" and "keyspace" would be mixed up.

We might want to tune the storage options differently for distinct sets of topics, the
keyspaces.  The keyspace is composed by one or more shards.

- Keyspaces are identified simply by binary strings.
- DS configuration is scoped by keyspaces instead of shards.
- Starting a new DS shard requires definining to which keyspace the shard belongs.
 2023-09-14 16:39:00 -03:00
Thales Macedo Garitezi
cae9ae1fab fix(ps_router): use disc_copies for storing persistent session routes 2023-09-13 10:02:39 -03:00
William Yang
ea73d674b9 chore: bump version to e5.2.1-alpha.2 2023-09-13 14:55:17 +02:00