yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,953 Commits 393 Branches 684 Tags 74 MiB
Commit Graph

11558 Commits

Author SHA1 Message Date
ieQu1 a62db08676
feat(ds): Add REST API for durable storage 2024-04-05 15:22:06 +02:00
ieQu1 d09787d1a6
fix(ds): Fix return types in replication_layer_meta 2024-04-05 15:22:06 +02:00
Ivan Dyachkov be47fe49ad chore: bump ecql version to 0.7.0 
PR: https://github.com/emqx/ecql/pull/13
No functional changes, just switch gen_fsm to gen_statem.
 2024-04-05 13:31:33 +02:00
Andrew Mayorov 70396e9766
Merge pull request #12825 from keynslug/feat/EMQX-12110/repl-meta-api 
feat(dsrepl): add APIs to manage DB replication sites
 2024-04-04 22:32:03 +02:00
Andrew Mayorov df6c5b35fe
feat(dsrepl): add more primitive operations to modify DB sites 2024-04-04 21:22:49 +02:00
Andrew Mayorov bb8ffee18c
feat(dsrepl): add API to get current DB replication sites 2024-04-04 21:22:02 +02:00
Andrew Mayorov ad52f7838e
feat(dsrepl): add APIs to manage DB replication sites 2024-04-04 21:22:01 +02:00
Thales Macedo Garitezi 60cad74286 feat(resource): non-blocking channel health checks 
Fixes https://emqx.atlassian.net/browse/EMQX-12015

Continuation of https://github.com/emqx/emqx/pull/12812
 2024-04-04 16:13:30 -03:00
Thales Macedo Garitezi 8d58b40f33
Merge pull request #12831 from thalesmg/ds-checkpoint-clean-m-20240404 
feat(ds): clear all checkpoints when (re)starting storage layer
 2024-04-04 15:41:50 -03:00
Thales Macedo Garitezi 217b35bce5
Merge pull request #12798 from thalesmg/ds-client-api-v2-m-20240327 
feat(client mgmt api): add cursor-based list API
 2024-04-04 15:10:49 -03:00
Thales Macedo Garitezi c57c36adb2 feat(ds): clear all checkpoints when (re)starting storage layer 
Fixes https://emqx.atlassian.net/browse/EMQX-12143
 2024-04-04 14:05:52 -03:00
Thales Macedo Garitezi 069cd4fbb4
Merge pull request #12812 from thalesmg/async-res-manager-health-check-m-20240328 
feat(resource manager): perform non-blocking health checks
 2024-04-04 09:07:02 -03:00
zmstone 0e79b543cf refactor: move variform to emqx_utils 2024-04-04 11:10:56 +02:00
ieQu1 f37ed3a40a fix(ds): Limit the number of retries in egress to 0 2024-04-03 16:38:49 +02:00
Shawn 319ec50c0d fix: source bridges missing after restore the backup files 2024-04-03 18:26:51 +08:00
ieQu1 2bbfada7af
fix(ds): Make async batches truly async 2024-04-03 11:57:47 +02:00
ieQu1 92ca90c0ca
fix(ds): Improve egress logging 2024-04-03 11:57:47 +02:00
ieQu1 ae5935e7f7
test(ds): Attempt to stabilize metrics_worker tests in CI 2024-04-02 19:14:10 +02:00
ieQu1 4382971443
fix(ds): Preserve errors in the egress 2024-04-02 16:47:43 +02:00
ieQu1 94ca7ad0f8
feat(ds): Report counters for LTS storage layout 2024-04-02 16:47:43 +02:00
ieQu1 f14c253dea
fix(prometheus): Don't add DS metrics when feature is disabled 2024-04-02 16:47:43 +02:00
ieQu1 b379f331de
fix(sessds): Handle errors when storing messages 2024-04-02 16:47:41 +02:00
ieQu1 f41e538526
feat(sessds): Observe next time 2024-04-02 16:45:52 +02:00
ieQu1 b9ad241658
feat(sessds): Add metrics for the number of persisted messages 2024-04-02 16:45:52 +02:00
ieQu1 75b092bf0e
fix(ds): Actually retry sending batch 2024-04-02 16:45:49 +02:00
ieQu1 0de255cac8
feat(ds): Report egress flush time 2024-04-02 16:25:04 +02:00
ieQu1 044f3d4ef5
fix(ds): Don't reverse entries in the atomic batch 2024-04-02 16:25:04 +02:00
ieQu1 606f2a88cd
feat(ds): Add egress metrics 2024-04-02 16:25:04 +02:00
ieQu1 c9de336234
feat(ds): Add metrics worker to the builtin db supervision tree 2024-04-02 16:25:04 +02:00
ieQu1 d8204021dc
refactor(metrics): Move metrics worker to emqx_utils application 2024-04-02 16:25:04 +02:00
Thales Macedo Garitezi 2097e854fc feat(client mgmt api): add cursor-based list API 
Fixes https://emqx.atlassian.net/browse/EMQX-12028
 2024-04-02 10:55:28 -03:00
Andrew Mayorov 778e897f1f
chore(dsrepl): describe snapshot ownership and few shortcomings 2024-04-02 13:48:51 +02:00
Andrew Mayorov c666c65c6a
test(ds): factor out storage iteration into helper module 2024-04-02 13:48:51 +02:00
Andrew Mayorov 7cebf598a8
chore(dsrepl): simplify snapshot transfer code a bit 
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
 2024-04-02 13:48:51 +02:00
Andrew Mayorov e029b8f996
test(dsrepl): wait for whole cluster readiness 
To minimize the chance of flaky tests due to the shards not being
completely online.

Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
 2024-04-02 13:48:50 +02:00
Andrew Mayorov e8b06a6a9f
chore(dsrepl): mark few more BPAPI targets as obsolete 2024-04-02 13:48:50 +02:00
Andrew Mayorov d31cd0c728
feat(ds): ensure LTS state ids are deterministic 2024-04-02 13:48:50 +02:00
Andrew Mayorov 2cd357a5bd
fix(ds): ensure store batch is idempotent wrt generations 2024-04-02 13:48:50 +02:00
Andrew Mayorov 77a022bd93
feat(dsrepl): transfer storage snapshot during ra snapshot recovery 2024-04-02 13:48:49 +02:00
Andrew Mayorov b8b9b7739b
chore(ds): slightly simplify working with storage generations 2024-04-02 13:48:08 +02:00
Andrew Mayorov 2d074df209
Merge pull request #12797 from keynslug/fix/dsrepl-error-handling 
fix(dsrepl): handle RPC errors gracefully when storage is down
 2024-04-02 13:40:31 +02:00
JimMoen 5759ba5162
chore: bump app version 2024-04-02 17:09:22 +08:00
JimMoen 50bceee9ab
fix(stats): `'subscribers.count'` contains shared-subscriber 2024-04-02 16:56:40 +08:00
JimMoen 0f4b148294
refactor: uniform shared_sub table macros 2024-04-02 16:56:39 +08:00
JimMoen 1a4cfc2a2d
fix(api_schema): removed metrics schema in api spec 
- Followup [PR#6622](https://github.com/emqx/emqx/pull/6622).
 2024-04-02 16:56:36 +08:00
Thales Macedo Garitezi bade09b56e feat(resource manager): perform non-blocking resource health checks 
Fixes https://emqx.atlassian.net/browse/EMQX-12015

This introduces only _resource_ non-blocking health checks.  _Channel_ non-blocking health
checks may be introduced later.
 2024-04-01 14:46:15 -03:00
Serge Tupchii c62410ff75 refactor: remove already bound variable 2024-04-01 17:03:50 +03:00
Serge Tupchii ceb04ba06d fix(emqx_mgmt): do not attempt to get a stacktrace of a remote client connection process 2024-04-01 16:42:12 +03:00
Serge Tupchii 42af1f9d63 fix: handle internal timeout errors in client Mqueue/Inflight APIs 2024-03-29 23:03:35 +02:00
Serge Tupchii f5a820cb10 fix(emqx_mgmt): catch OOM shutdown exits properly when calling a client conn process 
The exit reason is expected to include gen_server `Location`:
  `{{shutdown, OOMInfo}, Location}`.
 2024-03-29 13:09:08 +02:00
zmstone bfca3ebc71 feat(variform): support array syntax '[' and ']' 2024-03-28 19:34:57 +01:00
zmstone 5f26e4ed5e feat(variform): implement variform engine 2024-03-28 19:34:57 +01:00
SergeTupchiy 2e528d1dd8
Merge pull request #12802 from SergeTupchiy/EMQX-11826-prevent-left-node-from-rejoining-5.6.1 
prevent a left node from rejoining the same cluster
 2024-03-28 19:49:18 +02:00
zmstone ad95473aae refactor: move string functions to emqx_variform 2024-03-28 18:03:37 +01:00
Serge Tupchii 3eda182e9a fix: prevent a node from discovering and re-joining the same cluster after it has (manually) left it. 2024-03-28 18:09:27 +02:00
zmstone 9bf65a415b feat(variform): add a variable transformer 2024-03-28 16:11:26 +01:00
Andrew Mayorov 35c43eb8a0
feat(sessds): handle recoverable errors in stream scheduler 2024-03-28 15:17:01 +01:00
Andrew Mayorov fa66a640c3
fix(dsrepl): handle RPC errors gracefully when storage is down 2024-03-28 15:17:01 +01:00
SergeTupchiy 63c017a72f
Merge pull request #12804 from SergeTupchiy/EMQX-12058-improve-force-shutdown-error-reason-5.6.1 
chore: rename `message_queue_too_long` error reason to `mailbox_overflow` (5.6.1 port)
 2024-03-28 16:14:51 +02:00
Thales Macedo Garitezi 8fb4ef9fe3 test: fix flaky test 2024-03-28 10:53:44 -03:00
SergeTupchiy 93c87fcb25
Merge pull request #12803 from SergeTupchiy/EMQX-11808-remove-uploaded-invalid-backups-5.6.1 
fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid (5.6.1 port)
 2024-03-28 15:37:10 +02:00
Thales Macedo Garitezi 04bf763890 fix(kafka-based bridges): avoid trying to get raw config for replayq dir 
Fixes https://emqx.atlassian.net/browse/EMQX-12049
 2024-03-28 09:13:34 -03:00
ieQu1 b1855f95c1
fix(bpapi): Add exceptions for experimental features 2024-03-28 12:07:45 +01:00
ieQu1 8c6d8bdd12
fix(bpapi): Add exceptions for experimental features 2024-03-28 12:03:41 +01:00
Serge Tupchii 906a77d167 chore: rename `message_queue_too_long` error reason to `mailbox_overflow` 
`mailbox_overflow` is consistent with the corresponding config parameter:
 'force_shutdown.max_mailbox_size'
 2024-03-28 12:56:00 +02:00
Serge Tupchii 1d1f595e6f fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid 2024-03-28 12:54:51 +02:00
ieQu1 02b72d79fc
fix(bpapi): Fix comparison of releases during compatibility check 2024-03-28 11:21:30 +01:00
Ivan Dyachkov db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Ivan Dyachkov f4446ec680 Merge remote-tracking branch 'upstream/release-56' into 0328-sync-release-56 2024-03-28 09:59:54 +01:00
Ivan Dyachkov ea917d6b2b
Merge pull request #12795 from id/0327-cut-5.6.0 
chore: 5.6.0
 2024-03-28 09:30:08 +01:00
Ilya Averyanov 11bc1aeea1 fix: update hstreamdb driver 2024-03-28 10:30:48 +03:00
JimMoen cf29f97440
revert: "test: comment out hstreamdb:stop_client" 
This reverts commits:
3536b32ad2b126c27a1b0bd91e615069230931d7..
d63f10928d
 2024-03-28 14:32:18 +08:00
JimMoen 34b5cd7bfd
chore: reuse hstream_erl `0.5.18+v0.18.1` 
- change ezstd dep vsn for bulid
 2024-03-28 14:31:10 +08:00
JianBo He d63f10928d test: comment out hstreamdb:stop_client 2024-03-28 13:25:01 +08:00
JianBo He 3536b32ad2 ci: print the failed stacktrace 2024-03-28 11:17:26 +08:00
JianBo He d7917d05c6 chore: upgrade hstreamdb to 0.5.22+v0.18.1 2024-03-28 10:11:53 +08:00
ieQu1 a279a399db
ci: Override ezstd dependency 2024-03-27 20:54:46 +01:00
ieQu1 4b3c29ded8
fix(bpapi): Add 5.5 dump 2024-03-27 17:39:38 +01:00
Zaiming (Stone) Shi beb9152d50
Merge pull request #12750 from zmstone/0316-introduce-client-attrs 
0316 introduce client_attrs
 2024-03-27 16:19:47 +01:00
ieQu1 5ed20e15cd
fix(bpapi): Add 5.5 bpapi file 2024-03-27 13:45:00 +01:00
Thales Macedo Garitezi df0c67fe50
Merge pull request #12787 from thalesmg/mv-api-examples-m-20240326 
docs(message validation): add swagger API examples
 2024-03-27 09:06:58 -03:00
Ivan Dyachkov 33d9619271 chore: 5.6.0 2024-03-27 11:17:13 +01:00
Serge Tupchii a3b730ff10 fix(emqx_session_mem): keep msg inflight insert timestamp in Inflight after retry 2024-03-27 11:31:51 +02:00
JianBo He 4f9daa0d4a chore: bump version to 5.6.0-rc.6 2024-03-27 14:04:27 +08:00
JianBo He d82b6d216d chore: remove the duplicated time field 
Fixes https://emqx.atlassian.net/browse/EMQX-12104
 2024-03-27 12:09:21 +08:00
Ivan Dyachkov f127bf53b3 chore: 5.6.0-rc.5 2024-03-26 17:52:57 +01:00
ieQu1 291cf689e2
Merge pull request #12786 from ieQu1/dev/check-core-emqx-vsn 
fix(mria): Strictly check compatibility of the replicant with core
 2024-03-26 17:49:37 +01:00
ieQu1 ba43da1b8e
fix(mria): Strictly check compatibility of the replicant with core 2024-03-26 16:32:01 +01:00
Thales Macedo Garitezi da4a04c411
Merge pull request #12777 from thalesmg/test-flaky-conf-copy-m-20240325 
test: attempt to fix flaky test
 2024-03-26 11:52:12 -03:00
zmstone 78c5f8177c test: fix test case 2024-03-26 14:51:21 +01:00
Thales Macedo Garitezi 10a43db742 docs(message validation): add swagger API examples 
Fixes https://emqx.atlassian.net/browse/EMQX-12102
 2024-03-26 09:39:05 -03:00
zmstone 067f018337 test: fix test cases 2024-03-26 13:36:12 +01:00
zmstone b1c9bb63d7 chore: delete dead code 
hocon schema validator never validates 'undefined'
 2024-03-26 13:20:48 +01:00
zmstone c42e980442 feat(log): configurable time format 
now logs can be configured to use 'epoch' or 'rfc3339' time format
 2024-03-26 13:09:32 +01:00
Thales Macedo Garitezi cc37030265
Merge pull request #12770 from thalesmg/mv-metrics-m-20240321 
feat(message validation): implement metrics
 2024-03-26 08:59:05 -03:00
zhongwencool 56a40d61ae fix: bpapi undef abort on old node 2024-03-26 13:06:56 +08:00
Ivan Dyachkov 2ae973821f chore: 5.6.0-rc.4 2024-03-25 20:00:10 +01:00
Zaiming (Stone) Shi b06d3932af
Merge pull request #12768 from keynslug/fix/EMQX-12068/routing-schema-conflict 
fix(routing): add heuristic for routing schema in empty cluster
 2024-03-25 19:50:51 +01:00
Thales Macedo Garitezi 6259f95aab test: attempt to fix flaky test 
```
   expected: {ok,<<...>>}
       got: {error,enoent}
      line: 187
   comment: #{node => 'emqx_conf_app_SUITE9@127.0.0.1'}
```
 2024-03-25 14:36:22 -03:00
Thales Macedo Garitezi 0f426e6e77 feat(ds): make durable sessions handle will messages 
Fixes https://emqx.atlassian.net/browse/EMQX-10431
 2024-03-25 14:18:25 -03:00