yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,719 Commits 393 Branches 684 Tags 74 MiB
c6c1d886f0
Commit Graph

92 Commits

Author SHA1 Message Date
ieQu1
e8d18b0e09
Merge pull request #12014 from ieQu1/ds-qos0 
Support QoS0 messages in persistent_session_ds
 2023-11-23 23:43:18 +01:00
ieQu1
449bafc27e fix(ds): LTS trie handles empty topic levels 2023-11-23 22:16:38 +01:00
ieQu1
c5bb86db67 feat(ds): Support QoS 0 2023-11-23 22:16:38 +01:00
ieQu1
1da9ec1d7d
Merge pull request #12015 from ieQu1/ds-lts-dont-store-message 
feat(ds): Don't store #message record in the DB
 2023-11-23 18:56:45 +01:00
ieQu1
c69b82455e feat(ds): Don't store #message record in the DB 2023-11-23 17:56:35 +01:00
Ilya Averyanov
29d42506d5 fix(ds): impose more flexible types for ds implementations 2023-11-23 17:16:11 +03:00
Ilya Averyanov
a1b9a14fa1 feat(ds): allow fdb implementation for durable storage 2023-11-22 14:49:38 +03:00
ieQu1
3165b4f645 fix(ds): Abort application startup when rocksdb is not avialable 2023-11-21 20:27:07 +01:00
ieQu1
3d823beb11 fix(ds): Apply review remarks 2023-11-21 19:37:01 +01:00
ieQu1
f5c71e8068 refactor(ds): Add a wrapper to the store batch API 2023-11-21 19:37:01 +01:00
ieQu1
2a1f7d946a feat(ds): Shard messages by publisher client ID 2023-11-21 19:37:01 +01:00
ieQu1
62542e5844 feat(ds): Metadata storage for the replication layer 2023-11-21 19:37:01 +01:00
Ivan Dyachkov
28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Thales Macedo Garitezi
088de9476c fix(ds): use integer for tag values 
Follow up to https://github.com/emqx/emqx/pull/11906#discussion_r1389115973
 2023-11-10 11:26:32 -03:00
ieQu1
2ade6da7a6 fix(ds): Remove redundand type 2023-11-10 12:53:30 +01:00
ieQu1
feef23fc08 feat(ds): Pass store_batch through RPC 2023-11-10 12:16:39 +01:00
ieQu1
8dc8237331 feat(ds): Introduce emqx_ds behavior 2023-11-10 12:16:39 +01:00
Thales Macedo Garitezi
42536bb5f7 chore(ds): avoid using records in persistence / rpc 
Fixes https://emqx.atlassian.net/browse/EMQX-11279
 2023-11-09 12:00:25 -03:00
ieQu1
a1cdbaa76d fix(ds): Address code review remarks 2023-11-08 04:31:27 +01:00
ieQu1
c030188eb7 chore(ds): Add rebar.config file to app/emqx_durable_storage 2023-11-08 03:04:19 +01:00
ieQu1
7cb0322856 fix(emqx): Move bpapi and emqx message record to emqx_utils app 2023-11-08 03:04:19 +01:00
ieQu1
74cb43f8b1 fix(ds): Add unique ID to the key 2023-11-08 03:04:19 +01:00
Andrew Mayorov
54951c273f feat(ds): mix safe cutoff into keymapper filter during iteration 2023-11-08 03:04:19 +01:00
Andrew Mayorov
7a94db25c3 fix(ds): don't iterate over incomplete epoch in bitmask lts storage 2023-11-08 03:04:19 +01:00
ieQu1
38b7f516bc fix(ds): Change the directory where message database is stored 2023-11-08 03:04:19 +01:00
ieQu1
8e5dda40be fix(ds): Fix static checks 2023-11-08 03:04:19 +01:00
ieQu1
99329e1243 refactor(ds): Address review remarks 2023-11-08 03:04:19 +01:00
ieQu1
2de79dd9ac feat(ds): Replay QoS1 messages 2023-11-08 03:04:19 +01:00
ieQu1
e745e42093 test(ds): Explore full range of keys when testing ratchet function 2023-11-08 03:04:19 +01:00
ieQu1
87689890ff chore(ds): Fix linter and compilation warnings 2023-11-08 03:04:19 +01:00
ieQu1
ef46c09caf feat(ds): Implement ratchet function for bitmask keymapper 2023-11-08 03:04:19 +01:00
ieQu1
164ae9e94a feat(ds): LTS bitfield storage passes all tests 2023-11-08 03:04:19 +01:00
ieQu1
56b6b176c2 fix(ds): LTS shall keeps the concrete topic indexes 2023-11-08 03:04:19 +01:00
ieQu1
7428e7037b feat(ds): Bitfield + Learned Topic Structure 2023-11-08 03:04:19 +01:00
ieQu1
ac91dbc58f feat(ds): Restore LTS trie from a dump 2023-11-08 03:04:19 +01:00
ieQu1
f1ab7c8a7c feat(ds): Add persist callback to LTS trie 2023-11-08 03:04:19 +01:00
ieQu1
c149e0e2df fix(ds): Pass topic filter to emqx_ds:make_iterator call 2023-11-08 03:04:19 +01:00
ieQu1
51a6f623fd refactor(ds): Split out bitfield keymapper to a different module 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
903b3863d1 chore(ps_ds): make persistent session module use new emqx_ds APIs 2023-11-08 03:04:19 +01:00
ieQu1
2972bf14ee refactor(ds): Implement create_generation gen_rpc storage layer call 2023-11-08 03:04:19 +01:00
ieQu1
6d65707d41 refactor(ds): Implement drop_db function 2023-11-08 03:04:19 +01:00
ieQu1
c6a721a7eb refactor(ds): Passthrough open_db and get_channels to storage layer 2023-11-08 03:04:19 +01:00
ieQu1
59d01dc823 refactor(ds): Implement emqx_ds:open_db 2023-11-08 03:04:19 +01:00
ieQu1
7095cb8583 refactor(ds): Refactor storage layer 2023-11-08 03:04:19 +01:00
ieQu1
c91df2f5cd refactor(ds): Create a prototype of replication layer 2023-11-08 03:04:19 +01:00
ieQu1
f1454bb57e feat(ds): learned topic structure 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
7ab57824dc chore(ds): change return type of storage_layer:next/{1,2} 
Part of https://emqx.atlassian.net/browse/EMQX-10942

The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.

From:

```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```

To:

```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.

-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
 2023-11-08 03:04:19 +01:00
Zaiming (Stone) Shi
ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53 
0928 sync release 53
 2023-10-02 21:57:32 +02:00
Zaiming (Stone) Shi
3f6e0e890b chore: bump emqx_durable_storage and emqx_license app vsn 2023-10-02 20:13:09 +02:00
Thales Macedo Garitezi
2358d67908 refactor: move session stuff from emqx_ds to emqx_persistent_session_ds 
Part of https://emqx.atlassian.net/browse/EMQX-10942
 2023-09-29 18:00:24 -03:00