Commit Graph

11278 Commits

Author SHA1 Message Date
Andrew Mayorov
d30c99512a
feat(utils-stream): add a few more stream combinators 2024-03-19 20:21:54 +01:00
Andrew Mayorov
887e151be5
fix(dsrepl): handle errors gracefully in shard egress process
Also add cooldown on timeout / unavailability.
2024-03-19 20:21:53 +01:00
Andrew Mayorov
e16aee99b4
chore(dsrepl): clarify how to perform leadership transfer in runtime 2024-03-19 20:21:18 +01:00
Andrew Mayorov
00d509f27b
feat(dsrepl): prefer local replica in read path
To optimize out any unnecessary RPCs. Given the load should be
smoothed evenly across the cluster, choosing non-leader node is
not a priority.
2024-03-19 20:11:42 +01:00
Andrew Mayorov
19305c223c
fix(dsrepl): require majority for replication-related tables 2024-03-19 20:11:42 +01:00
Andrew Mayorov
f89909f60c
fix(dsrepl): tolerate trigger election timeouts for existing servers 2024-03-19 20:11:42 +01:00
Andrew Mayorov
6c6ea50e42
fix(boot): add emqx_durable_storage to the list of restarted apps 2024-03-19 20:11:42 +01:00
Andrew Mayorov
3b59cf2ebf
feat(dsrepl): move shard allocation to separate process
That starts shard and egress processes only when shards are fully
allocated.
2024-03-19 20:11:41 +01:00
Andrew Mayorov
4dafbf21f6
fix(dsrepl): make db + shard part of machine state
It doesn't feel right, but right now is the easiest way to have it
in the scope of `apply/3`, because `init/1` doesn't actually invoked
for ra machines recovered from the existing log / snapshot.
2024-03-19 20:11:41 +01:00
Andrew Mayorov
d19128ed65
feat(dsrepl): cache shard metadata in persistent terms 2024-03-19 20:11:41 +01:00
Andrew Mayorov
e6c2c2fb07
feat(dsrepl): manage generations / db config through ra machine 2024-03-19 20:11:39 +01:00
Andrew Mayorov
5e94bdb932
feat(dsrepl): allocate shards once predefined number of sites online
Before this commit the most likely shard allocation outcome was
that all shard are allocated to just one node.
2024-03-19 20:11:03 +01:00
Andrew Mayorov
be793e4735
fix(dsrepl): reassign timestamp at the time of submission
This is needed to ensure total message order for a shard, and
guarantee that no messages will be written "in the past". which
may break replay consistency.
2024-03-19 20:11:01 +01:00
Andrew Mayorov
146f082fdc
feat(dsrepl): implement raft-based replication
Still very rough but mostly working.
2024-03-19 20:09:44 +01:00
Ivan Dyachkov
923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
sync release-56
2024-03-19 19:09:41 +01:00
Ivan Dyachkov
f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
Thales Macedo Garitezi
a689ae72e3
Merge pull request #12711 from thalesmg/data-validation-m-20240311
feat: implement message validation
2024-03-19 11:12:28 -03:00
Thales Macedo Garitezi
c478e375f6 refactor: change failure log configuration schema 2024-03-19 10:04:18 -03:00
JianBo He
c7db157ecd
Merge pull request #12735 from zhongwencool/fix-ban-crash
fix: don't crash when ban ip error
2024-03-19 20:57:54 +08:00
Thales Macedo Garitezi
2eac54bf54 chore: rm move API
Only `reorder` API will be supported, after discussions with product and frontend teams.
2024-03-19 09:55:55 -03:00
SergeTupchiy
92fd0e453a
Merge pull request #12719 from SergeTupchiy/EMQX-11900-multiple-clientid-username-clients-API
Support multiple clientid/username Qs params in clients API
2024-03-19 14:44:39 +02:00
JianBo He
24c04e715d
Merge pull request #12715 from zhongwencool/fix-source-replace-error
fix: cant replace source conf
2024-03-19 19:32:47 +08:00
zhongwencool
111cbbbe3c fix: don't crash when ban ip error 2024-03-19 19:10:36 +08:00
zhongwencool
d7c01a4fe0 fix: cant replace source conf 2024-03-19 18:53:51 +08:00
zhongwencool
9a44c6712a fix: otel metrics create view crash 2024-03-19 12:20:49 +08:00
Thales Macedo Garitezi
e8a8416e44 feat(message_validation): impose restrictions on validation name 2024-03-18 15:57:39 -03:00
Thales Macedo Garitezi
74c03377f2 feat(message_validation_api): implement reorder API 2024-03-18 14:30:52 -03:00
Thales Macedo Garitezi
bcb7fe96d5 refactor: remove ambiguity from api
In preparation for the "reorder" API
2024-03-18 13:20:05 -03:00
Thales Macedo Garitezi
96ffc9b174 refactor: export schema_check fn from schema registry 2024-03-18 13:11:39 -03:00
Thales Macedo Garitezi
cde87bce66 chore: add missing traces 2024-03-18 13:11:39 -03:00
Thales Macedo Garitezi
f84a996671 feat: implement message validation
Fixes https://emqx.atlassian.net/browse/EMQX-11980
2024-03-18 13:11:39 -03:00
Serge Tupchii
8f8b023429 refactor: simplify match spec holder construction 2024-03-18 18:00:26 +02:00
Serge Tupchii
b3d44125f6 feat: support multiple clientid / username Qs params in "/clients" API
Additionally, add an option to specify which Client info fields to return in the response.
2024-03-18 18:00:26 +02:00
JianBo He
2a611e4508 chore: fix the error api desc 2024-03-18 18:27:28 +08:00
JianBo He
485d8dbbdb feat: add the get /source_types endpoint to return all supported source 2024-03-18 18:11:06 +08:00
JimMoen
889eb48368
fix(shared-sub): update stats when unsub shared topic filter 2024-03-16 20:43:05 +08:00
JimMoen
59e280dc54
test: cluster consistented metrics and label for nodes running/stopped
- `emqx_cluster_sessions_count`
- `emqx_cluster_sessions_max`
- `emqx_subscriptions_shared_count`
- `emqx_subscriptions_shared_max`
- `emqx_cluster_nodes_running`
- `emqx_cluster_nodes_stopped`
2024-03-16 20:43:04 +08:00
JimMoen
55a8488a4f
test: fields in /monitor_current
- `retained_msg_count`
- `shared_subscriptions`
2024-03-16 20:43:03 +08:00
JimMoen
02b3292025
fix: cluster nodes running/stopped with node name as label 2024-03-16 20:39:34 +08:00
JimMoen
0e25eb287b
fix(prom): cluster session count/max is cluster consistented 2024-03-16 20:39:33 +08:00
JimMoen
62f8f889e1
refactor: func rename to fix typo 2024-03-16 20:39:32 +08:00
JimMoen
e56bee618b
fix: shared_subscriptions is cluster consistented 2024-03-16 20:39:31 +08:00
Ivan Dyachkov
3f93780d8c
Merge pull request #12717 from id/0315-prep-5.6.0-rc.1
chore: 5.6.0-rc.1
2024-03-15 21:28:31 +01:00
Ivan Dyachkov
e28992d586 chore: 5.6.0-rc.1 2024-03-15 17:08:00 +01:00
Ivan Dyachkov
ea76dac760 chore: fix xref 2024-03-15 17:08:00 +01:00
Kjell Winblad
1fecd01405
Merge pull request #12708 from kjellwinblad/kjell/dynamo_bridge/update_error/EMQX-11984
fix: DynamoDB connector status check takes too long
2024-03-15 16:20:19 +01:00
ieQu1
c22735b3f5
Merge pull request #12707 from ieQu1/dev/store-ds-session-ip
Store peer name in the durable session metadata.
2024-03-15 16:15:29 +01:00
ieQu1
a93f747afa fix(sessds): Return peername of the disconnected client in the REST 2024-03-15 13:48:53 +01:00
ieQu1
b6cc9177a6 feat(sessds): Store peername in the persistent session state 2024-03-15 13:48:53 +01:00
SergeTupchiy
e35e8847b9
Merge pull request #12685 from SergeTupchiy/refactor-emqx-mgmt-call-client
Refactor emqx mgmt call client
2024-03-15 10:01:49 +02:00