Commit Graph

20003 Commits

Author SHA1 Message Date
Andrew Mayorov b94ed120ed
wip: do not store sessions in rocksdb 2024-02-29 21:15:01 +01:00
Andrew Mayorov 19cc7c34a5
wip: disable rocksdb wal 2024-02-29 21:06:08 +01:00
Andrew Mayorov 475efc93b3
wip: allow to tune select ra options 2024-02-29 17:19:38 +01:00
Andrew Mayorov da6844957e
wip: allocate shards predictably 2024-02-27 21:38:42 +01:00
Andrew Mayorov 41f1fd8ebc
feat(utils-stream): add a few more stream combinators 2024-02-27 21:38:42 +01:00
Andrew Mayorov 574746b190
wip: tune egress draining strategy 2024-02-22 18:41:10 +01:00
Andrew Mayorov 1895b250e9
wip: provide persist feedback to broker + alternative egress impl 2024-02-20 20:33:57 +01:00
Andrew Mayorov 83dd6a2896
wip: handle errors gracefully in shard egress process
Also add cooldown on timeout / unavailability.
2024-02-19 19:09:22 +01:00
Andrew Mayorov 77c65266d0
wip: clarify how to perform leadership transfer in runtime 2024-02-19 19:07:59 +01:00
Andrew Mayorov 0e832db1d4
wip: catch and mask badrpcs in the read path 2024-02-15 16:51:52 +01:00
Andrew Mayorov 58bd42bfc1
wip: prefer local replica in read path 2024-02-12 19:24:42 +01:00
Andrew Mayorov 3a6b4b57d7
wip: require majority for replication-related tables 2024-02-12 19:24:42 +01:00
Andrew Mayorov 5ab036a5d2
wip: tolerate trigger election timeouts for existing servers 2024-02-12 19:24:42 +01:00
Andrew Mayorov 114efb9818
fix(boot): add `emqx_durable_storage` to the list of restarted apps 2024-02-12 19:24:42 +01:00
Andrew Mayorov 9d94ee73de
wip: move shard allocation to separate process
That starts shard and egress processes only when shards are fully
allocated.
2024-02-12 19:24:42 +01:00
Andrew Mayorov 1b107d578c
wip: make db + shard part of machine state
It doesn't feel right, but right now is the easiest way to have it
in the scope of `apply/3`, because `init/1` doesn't actually invoked
for ra machines recovered from the existing log / snapshot.
2024-02-12 19:24:42 +01:00
Andrew Mayorov 5e32fe4355
wip: cache shard metadata in persistent terms 2024-02-12 19:24:41 +01:00
Andrew Mayorov 7fa3bbf176
wip: manage generations / db config through ra machine 2024-02-12 19:24:41 +01:00
Andrew Mayorov 46e8118e1c
wip: allocate shards only when predefined number of sites online 2024-02-12 19:24:41 +01:00
Andrew Mayorov fcb5ed346f
wip: reassign timestamp at the time of submission 2024-02-12 19:24:41 +01:00
Andrew Mayorov bb0cf62879
wip: implement raft-based replication 2024-02-12 19:24:41 +01:00
Ilya Averyanov edd28be090
Merge pull request #12499 from savonarola/0209-emqx-banned-extended-rules
feat(banned): allow ban by clientid/username regexps, peerhost cidrs
2024-02-12 17:12:47 +02:00
Andrew Mayorov a9fdf9f1f9
Merge pull request #12495 from keynslug/ft/EMQX-11830/s3-bridge-simple
feat(s3): introduce S3 connector and action
2024-02-12 16:05:22 +01:00
Zaiming (Stone) Shi 953492df88
Merge pull request #12505 from zmstone/0212-upgrade-wolff-1.10.2
chore: upgrade to wolff 1.10.2
2024-02-12 14:36:06 +01:00
Andrew Mayorov 2b1231b980
fix(s3-bridge): ensure bridge and connector names are identical 2024-02-12 13:46:54 +01:00
Ilya Averyanov 8a4c66dc75 feat(banned): add warning about performance impact of matching rules 2024-02-12 12:56:03 +03:00
Andrew Mayorov 0edc5b5992
docs(s3): fix README example 2024-02-12 10:44:18 +01:00
Andrew Mayorov 5b7bb89f37
docs(s3): add missing labels and fix wording
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-02-12 10:39:42 +01:00
Zaiming (Stone) Shi d4fb3e83f6 chore: upgrade to wolff 1.10.2
Wolff 1.10.2 added an enhancement which makes Kafka resource health
checks more lightweight and lowers the chance of false resource alerts.
2024-02-12 09:05:44 +01:00
Ilya Averyanov 90fd2b26d3 feat(banned): allow ban by clientid/username regexps, peerhost cidrs 2024-02-10 17:59:22 +03:00
Andrew Mayorov c108262771
chore(s3-bridge): use `emqx_connector_schema` helper for brewity 2024-02-09 16:08:33 +01:00
Andrew Mayorov f476f4343b
fix(release): add `emqx_bridge_s3` to applications list 2024-02-09 14:16:32 +01:00
Andrew Mayorov 2a4e37869e
fix(dashboard): provide full context in startup errors 2024-02-09 14:16:32 +01:00
Andrew Mayorov 802c760406
feat(s3): introduce S3 connector and action
This is a trivial connector based on `emqx_s3` and simple action
that maps each incoming event into an S3 object upload. Due to
current `emqx_s3` limitation this bridge is compatible with
backends providing S3 API with path-style bucket access.
2024-02-09 14:16:32 +01:00
SergeTupchiy 755b59b7fe
Merge pull request #12494 from SergeTupchiy/EMQX-11825-mongodb-connector-perf
Improve mongodb connector performance
2024-02-09 13:12:12 +02:00
Zaiming (Stone) Shi cbebc59115
Merge pull request #12492 from zmstone/0208-fix-return-receive-maximum-in-connack
fix(emqx_channel): return Receive-Maximum in CONNACK when no error
2024-02-09 12:08:46 +01:00
Andrew Mayorov 8f66bd9ddf
fix(s3-client): make log levels saner 2024-02-09 12:03:41 +01:00
Andrew Mayorov 4ff04ab1f3
feat(s3): separate concepts to make app reusable in bridges 2024-02-09 12:03:41 +01:00
Zaiming (Stone) Shi cd3cc41f90 docs: add change log for PR #12492 2024-02-09 10:49:51 +01:00
Zaiming (Stone) Shi a74c828e19 test: add test case to cover Receive-Maximum in CONNACK 2024-02-09 10:42:48 +01:00
Andrew Mayorov a5266f68ec
feat(s3): switch schema to use secrets with loader support
This will make applications using `emqx_s3` follow the same
conventions as bridges and support loading secrets from files
at runtime.
2024-02-08 22:43:20 +01:00
Thales Macedo Garitezi 30855e76e5
Merge pull request #12493 from thalesmg/test-fix-schema-checks-m-20240208
test(bridge_v2): actually check schema
2024-02-08 18:03:13 -03:00
Serge Tupchii 1d9ce709f4 perf: un-deprecate mongodb topology pool_size and set it to 10 by default
Benchmarks proved that it effects performance:
10 ecpool workers / 10 topology pool workers outperform
100 ecpool / 1 topology worker.

Fixes: EMQX-11825
2024-02-08 20:39:19 +02:00
Serge Tupchii c9ee5addb4 perf: upgrade mongodb-erlang to v3.0.23
Fixes: EMQX-11825
2024-02-08 20:00:26 +02:00
Thales Macedo Garitezi 64367d834b test(bridge_v2): actually check schema 2024-02-08 13:39:31 -03:00
Zaiming (Stone) Shi cf22692c74 fix(emqx_channel): return Receive-Maximum in CONNACK when no error 2024-02-08 17:05:16 +01:00
ieQu1 811edb32a2
Merge pull request #12485 from ieQu1/dev/ds-fix-unsubscribe
fix(ds): Fix unsubscribe logic related to the replay
2024-02-07 21:38:25 +01:00
ieQu1 19c6d1127f
refactor(sessds): Extract subscription mgmt logic to separate module 2024-02-07 15:27:11 +01:00
ieQu1 3000a8f286
fix(sessds): Postpone deletion of the subscription until fully acked 2024-02-07 14:51:02 +01:00
zhongwencool 82850b5894
Merge pull request #12476 from Hecate2/patch-1
chore: fix typo in Windows.md
2024-02-06 15:28:07 +08:00