Commit Graph

20677 Commits

Author SHA1 Message Date
Andrew Mayorov 393a1b1391
wip(sessds): handle recoverable errors in stream scheduler 2024-03-26 18:23:08 +01:00
Andrew Mayorov c6085a6ab0
wip(dsrepl): avoid contacting offline nodes prematurely 2024-03-26 18:13:30 +01:00
Andrew Mayorov 0b69c8b7f3
wip(dsrepl): print leadership info as part of status report 2024-03-26 17:56:35 +01:00
Andrew Mayorov 534e177e7c
wip(dsrepl): handle RPC errors gracefully gracefully 2024-03-26 17:13:38 +01:00
Andrew Mayorov 663ea69574
wip(sessds): taint streams temporarily during failures 2024-03-26 16:43:06 +01:00
Andrew Mayorov 0e36f7afa4
chore(dsrepl): mark few more BPAPI targets as obsolete 2024-03-25 20:14:28 +01:00
Andrew Mayorov 4dbf68716a
feat(ds): ensure LTS state ids are deterministic 2024-03-25 20:14:28 +01:00
Andrew Mayorov 5d6efa622c
fix(ds): ensure store batch is idempotent wrt generations 2024-03-25 20:14:28 +01:00
Andrew Mayorov cc9926f159
feat(dsrepl): transfer storage snapshot during ra snapshot recovery 2024-03-25 20:14:28 +01:00
Andrew Mayorov 9e34fcd1d7
chore(ds): slightly simplify working with storage generations 2024-03-25 20:14:27 +01:00
Thales Macedo Garitezi 4af1a8cc56
Merge pull request #12742 from thalesmg/ds-will-msg-m-20240318
feat(ds): make durable sessions handle will messages
2024-03-25 15:33:57 -03:00
Thales Macedo Garitezi 0f426e6e77 feat(ds): make durable sessions handle will messages
Fixes https://emqx.atlassian.net/browse/EMQX-10431
2024-03-25 14:18:25 -03:00
Thales Macedo Garitezi 8c1a1d21a7 feat(session): pass will message down to session when creating/opening 2024-03-25 14:18:23 -03:00
ieQu1 8b963d5960
Merge pull request #12739 from ieQu1/dev/zone-durability
feat(sessds): Add zone overrides for session durability settings
2024-03-25 17:32:53 +01:00
Zaiming (Stone) Shi c8cb03f0d9
Merge pull request #12773 from zmstone/0323-upgrade-http-clients
chore: upgrade http client libs gun-1.3.11 and ehttpc-0.4.13
2024-03-25 17:29:44 +01:00
Kjell Winblad 94c6e5bb38
Merge pull request #12727 from kjellwinblad/kjell/emqx_connector_info/EMQX-11427
refactor: add emqx_connector_info behavior
2024-03-25 14:31:40 +01:00
Thales Macedo Garitezi f393d75a1e
Merge pull request #12771 from thalesmg/test-another-flaky-monitor-m-20240322
test: attempt to fix flaky test
2024-03-25 09:19:20 -03:00
Kjell Winblad e39fa5eaa9 docs(es_connector_info): add copyright header 2024-03-25 10:44:33 +01:00
JimMoen 3e97e5528e
Merge pull request #12765 from JimMoen/EMQX-12043/subscribers-count
fix(stats): `'subscribers.count'` contains shared-subscriber
2024-03-25 17:19:13 +08:00
JimMoen 5f7f9e43f9
fix(stats): `'subscribers.count'` contains shared-subscriber 2024-03-25 15:12:57 +08:00
zmstone d7b54b3a53 docs: add changelog for pr #12773 2024-03-23 10:41:46 +01:00
zmstone 1d9ec85441 chore: upgrade http client libs gun-1.3.11 and ehttpc-0.4.13 2024-03-23 10:29:05 +01:00
Thales Macedo Garitezi 303d0ed81b test: attempt to fix flaky test 2024-03-22 16:29:00 -03:00
SergeTupchiy 8e979d511c
Merge pull request #12766 from SergeTupchiy/EMQX-12058-improve-force-shutdown-error-reason
chore: rename `message_queue_too_long` to `mailbox_overflow`
2024-03-22 14:32:06 +02:00
Serge Tupchii d2a1a7f7cf chore: rename `message_queue_too_long` error reason to `mailbox_overflow`
`mailbox_overflow` is consistent with the corresponding config parameter:
 'force_shutdown.max_mailbox_size'
2024-03-22 12:20:20 +02:00
JimMoen 42faffc320
refactor: uniform shared_sub table macros 2024-03-22 18:03:13 +08:00
Kjell Winblad 2b684c7bcc fix(emqx_connector_info): refactoring based on thalesmg's comments 2024-03-22 09:29:26 +01:00
JimMoen 9b085b5797
fix(api_schema): removed metrics schema in api spec
- Followup [PR#6622](https://github.com/emqx/emqx/pull/6622).
2024-03-22 10:57:14 +08:00
Thales Macedo Garitezi 23ad37f566
Merge pull request #12762 from thalesmg/ds-fix-sync-egress-reply-m-20240321
fix(ds): add caller to pending replies before flushing and reply failures to sync callers
2024-03-21 16:42:42 -03:00
Thales Macedo Garitezi cfd4a1297f
Merge pull request #12761 from thalesmg/test-fix-flaky-dash-mon-m-20240321
test: fix flaky test
2024-03-21 16:04:08 -03:00
Thales Macedo Garitezi e17f663fa5
Merge pull request #12749 from thalesmg/mv-followup1-m-20240320
Follow up features and fixes for message validation - part 1
2024-03-21 16:03:49 -03:00
Thales Macedo Garitezi 796c04e7a8 test: fix flaky test
We should emit the trace event before replying to callers.

Example failure:

https://github.com/emqx/emqx/actions/runs/8378977952/job/22946318696#step:6:182

```
 =CRITICAL REPORT==== 21-Mar-2024::17:45:37.676024 ===
"check stage" failed: error
{assertMatch,[{module,emqx_ds_storage_bitfield_lts_SUITE},
              {line,270},
              {expression,"? of_kind ( emqx_ds_replication_layer_egress_flush , Trace )"},
              {pattern,"[ # { batch := [ _ , _ , _ ] } ]"},
              {value,[]}]}
Stacktrace: [{emqx_ds_storage_bitfield_lts_SUITE,
                 '-t_atomic_store_batch/1-fun-1-',1,
                 [{file,
                      "/__w/emqx/emqx/apps/emqx_durable_storage/test/emqx_ds_storage_bitfield_lts_SUITE.erl"},
                  {line,270}]},
             {emqx_ds_storage_bitfield_lts_SUITE,t_atomic_store_batch,1,
                 [{file,
                      "/__w/emqx/emqx/apps/emqx_durable_storage/test/emqx_ds_storage_bitfield_lts_SUITE.erl"},
                  {line,249}]}]
```
2024-03-21 15:47:29 -03:00
Thales Macedo Garitezi 68af211130 fix(ds): reply sync callers after raft store failure 2024-03-21 15:40:21 -03:00
Thales Macedo Garitezi 70737a437a fix(ds): add caller to pending replies before flushing 2024-03-21 14:39:21 -03:00
Thales Macedo Garitezi fa7ec231e3 test: fix flaky test 2024-03-21 14:07:09 -03:00
SergeTupchiy e2a2295c99
Merge pull request #12759 from SergeTupchiy/EMQX-11808-remove-uploaded-invalid-backups
fix(emqx_mgmt_data_backup): remove invalid uploaded backup files
2024-03-21 18:58:48 +02:00
Thales Macedo Garitezi 00aa7b5621 feat: create new `message.validation_failed` hookpoint and rule engine event 2024-03-21 13:46:27 -03:00
Thales Macedo Garitezi 7d7c6685d4
Merge pull request #12753 from thalesmg/test-flaky-retry-m-20240321
test: attempt to stabilize flaky test
2024-03-21 13:43:51 -03:00
Thales Macedo Garitezi 8ffd1304a5
Merge pull request #12755 from thalesmg/fix-kafka-probe-buffer-disk-m-20240321
fix(kafka-based bridges): avoid trying to get raw config for replayq dir
2024-03-21 13:39:33 -03:00
Serge Tupchii 40eccb10d6 fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid 2024-03-21 17:36:39 +02:00
Kjell Winblad 4bfc51c052 refactor(emqx_connector_info): improve documentation and type specs 2024-03-21 16:08:31 +01:00
Thales Macedo Garitezi e837791f94 fix(kafka-based bridges): avoid trying to get raw config for replayq dir
Fixes https://emqx.atlassian.net/browse/EMQX-12049
2024-03-21 11:49:00 -03:00
ieQu1 34c3c4e892
Merge pull request #12474 from ieQu1/dev/ds-rest-api
feat(sessds): Expose relevant durable session info in the REST API
2024-03-21 15:28:07 +01:00
Thales Macedo Garitezi 64399b6861 test: attempt to stabilize flaky test 2024-03-21 11:04:14 -03:00
Kjell Winblad 89befa580e refactor(emqx_connector_info): remove code that is no longer needed 2024-03-21 11:58:54 +01:00
ieQu1 cada944350
feat(sessds): Expose relevant durable session info in the REST API 2024-03-21 10:37:04 +01:00
Kjell Winblad a3b75eb39d refactor(remaining connectors): to use emqx_connector_info
This commit refactors the sqlserver, iotdb, elasticsearch, opents,
greptimedb, pulsar, tdengine, rabbitmq and s3 connectors to use the
`emqx_connector_info` behavior.
2024-03-21 09:47:57 +01:00
Andrew Mayorov e10d43cdce
Merge pull request #12361 from keynslug/ft/EMQX-11756/emqx-ds-replication
feat(ds): implement raft-based replication
2024-03-20 21:32:42 +01:00
Thales Macedo Garitezi 62030e8942 feat(message validation): forbid repeated schema checks
Fixes https://emqx.atlassian.net/browse/EMQX-12054
2024-03-20 15:26:16 -03:00
Thales Macedo Garitezi b8cd1c9020 feat(message validation api): add enable/disable HTTP API 2024-03-20 14:59:32 -03:00