611b3f0e07
feat(dsrepl): use more straightforward way to drop ra shards
2024-03-20 13:20:23 +01:00
74881e8706
feat(dsrepl): make storage layer unaware of granularity of time
...
Storage also becomes a bit more pure, depending on the upper layer to
provide the timestamps, which also makes it possible to handle more
operations idempotently.
2024-03-20 13:20:23 +01:00
783f769a98
fix(mgmt): Fix bulk kickout of durable sessions
2024-03-20 12:52:44 +01:00
223bbc05fe
refactor(mysql connector): to use emqx_connector_info
...
This commit refactors the mysql connector to use the
`emqx_connector_info` behavior.
2024-03-20 11:12:38 +01:00
c6d7c57bbd
refactor(clickhouse connector): to use emqx_connector_info
...
This commit refactors the clickhouse connector to use the
`emqx_connector_info` behavior.
2024-03-20 10:59:58 +01:00
5390203184
Merge pull request #12744 from JimMoen/EMQX-12046/fix-otel-cpu-sup
...
fix(otel): cpu usage/idle metrics for opentelemetry
2024-03-20 17:52:29 +08:00
afa3da6d0d
refactor(cassandra connector): to use emqx_connector_info
...
This commit refactors the cassandra connector to use the
`emqx_connector_info` behavior.
2024-03-20 10:50:44 +01:00
159ab81904
refactor(influxdb connector): to use emqx_connector_info
...
This commit refactors the influxdb connector to use the
`emqx_connector_info` behavior.
2024-03-20 10:39:16 +01:00
c1e6092013
refactor(oracle connector): to use emqx_connector_info
...
This commit refactors the oracle connector to use the
`emqx_connector_info` behavior.
2024-03-20 10:12:55 +01:00
808efe0f64
chore: rm outdated comment
2024-03-20 16:49:36 +08:00
b8b4c5a34a
fix(otel): cpu usage/idle metrics for opentelemetry
2024-03-20 16:48:21 +08:00
0547b32727
feat(sessds): Add zone overrides for session durability settings
2024-03-19 22:51:06 +01:00
611e4f6710
Merge pull request #12740 from ieQu1/dev/kick-durable-sessions
...
fix(sessds): Channel must destroy sessions that are kicked
2024-03-19 21:30:40 +01:00
e2db038e7c
Merge pull request #12734 from keynslug/fix/EMQX-12030/topics-api
...
fix(api-topics): expose persistent session topics in APIs
2024-03-19 21:08:23 +01:00
3cb36a5619
feat(ds-lts): extract timestamp from storage key itself
...
1. This avoids the need to deserialize the message to get the timestamp.
2. It also makes possible to decouple the storage key timestamp from the
message timestamp, which might be useful for replication purposes.
2024-03-19 20:21:56 +01:00
5cc0246351
feat(dsrepl): allow to tune select ra options
2024-03-19 20:21:55 +01:00
54b5adf868
feat(dsrepl): allocate shards predictably
...
To ensure strictly optimal and fair shard allocation across
cluster. Before this commit it was quite easy to end up with
an allocation significantly skewed towards some node, because
of the nature of randomness and relatively small number of
shards.
2024-03-19 20:21:55 +01:00
d30c99512a
feat(utils-stream): add a few more stream combinators
2024-03-19 20:21:54 +01:00
887e151be5
fix(dsrepl): handle errors gracefully in shard egress process
...
Also add cooldown on timeout / unavailability.
2024-03-19 20:21:53 +01:00
e16aee99b4
chore(dsrepl): clarify how to perform leadership transfer in runtime
2024-03-19 20:21:18 +01:00
749ad73819
Merge pull request #12721 from emqx/dependabot/github_actions/actions-d8a5788e1a
...
chore(deps): bump the actions group with 3 updates
2024-03-19 20:16:03 +01:00
00d509f27b
feat(dsrepl): prefer local replica in read path
...
To optimize out any unnecessary RPCs. Given the load should be
smoothed evenly across the cluster, choosing non-leader node is
not a priority.
2024-03-19 20:11:42 +01:00
19305c223c
fix(dsrepl): require majority for replication-related tables
2024-03-19 20:11:42 +01:00
f89909f60c
fix(dsrepl): tolerate trigger election timeouts for existing servers
2024-03-19 20:11:42 +01:00
6c6ea50e42
fix(boot): add `emqx_durable_storage` to the list of restarted apps
2024-03-19 20:11:42 +01:00
3b59cf2ebf
feat(dsrepl): move shard allocation to separate process
...
That starts shard and egress processes only when shards are fully
allocated.
2024-03-19 20:11:41 +01:00
4dafbf21f6
fix(dsrepl): make db + shard part of machine state
...
It doesn't feel right, but right now is the easiest way to have it
in the scope of `apply/3`, because `init/1` doesn't actually invoked
for ra machines recovered from the existing log / snapshot.
2024-03-19 20:11:41 +01:00
d19128ed65
feat(dsrepl): cache shard metadata in persistent terms
2024-03-19 20:11:41 +01:00
e6c2c2fb07
feat(dsrepl): manage generations / db config through ra machine
2024-03-19 20:11:39 +01:00
5e94bdb932
feat(dsrepl): allocate shards once predefined number of sites online
...
Before this commit the most likely shard allocation outcome was
that all shard are allocated to just one node.
2024-03-19 20:11:03 +01:00
be793e4735
fix(dsrepl): reassign timestamp at the time of submission
...
This is needed to ensure total message order for a shard, and
guarantee that no messages will be written "in the past". which
may break replay consistency.
2024-03-19 20:11:01 +01:00
146f082fdc
feat(dsrepl): implement raft-based replication
...
Still very rough but mostly working.
2024-03-19 20:09:44 +01:00
76d14de480
Merge pull request #12722 from emqx/dependabot/github_actions/dot-github/actions/prepare-jmeter/actions-prepare-jmeter-0101c7941f
...
chore(deps): bump the actions-prepare-jmeter group in /.github/actions/prepare-jmeter with 2 updates
2024-03-19 19:14:36 +01:00
923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
...
sync release-56
2024-03-19 19:09:41 +01:00
9d7d8edcc3
Merge pull request #12741 from keynslug/fix/flaky-ds-replay-testcase
...
test(ds): stabilize `t_session_replay_retry` testcase
2024-03-19 18:59:54 +01:00
f453bdf055
chore(topic-api): describe `emqx_persistent_session_ds_router:stream/1`
2024-03-19 18:51:06 +01:00
027b237f07
test(ds): stabilize `t_session_replay_retry` testcase
2024-03-19 18:33:38 +01:00
73121f3ccc
Merge pull request #12730 from zmstone/0318-docs-fix-escape-in-i18n
...
docs: there is no need to escape quotes in triple-quotes
2024-03-19 18:30:14 +01:00
236685f714
fix(sessds): Channel must destroy sessions that are kicked
2024-03-19 17:49:04 +01:00
df8c7152fc
refactor(mongodb connector): to use emqx_connector_info
...
This commit refactors the mongodb connector to use the
`emqx_connector_info` behavior.
2024-03-19 16:09:30 +01:00
c3d7d68cfc
refactor(kinesis,pgsql,timescale,matrix connectors): emqx_connector_info
...
This commit refactors the kinesis, pgsql, timescale and matrix
connectors to use the `emqx_connector_info` behavior.
2024-03-19 15:57:23 +01:00
f2dc940436
Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56
2024-03-19 15:20:08 +01:00
a689ae72e3
Merge pull request #12711 from thalesmg/data-validation-m-20240311
...
feat: implement message validation
2024-03-19 11:12:28 -03:00
10ea367030
test(client-api): attempt to get rid of inter-testcase side effects
2024-03-19 14:56:50 +01:00
3129a8fa42
test(client-api): fix snabbkaffe crash noise in the CT logs
2024-03-19 14:55:28 +01:00
b8505ebd54
chore: upgrade to jesse 1.8.0 for draft-06 as default version
2024-03-19 14:47:11 +01:00
a3e631cda2
refactor(kafka connectors): to use emqx_connector_info
...
This commit refactors the `emqx_bridge_kafka` to use the
`emqx_connector_info` behavior. The `emqx_bridge_kafka` related
information can thus be removed from `emqx_connector_chema` and
`emqx_connector_resource`.
2024-03-19 14:41:23 +01:00
b8ef357fef
refactor(http connector): to use emqx_connector_info
...
This commit refactors the `emqx_bridge_http` to use the
`emqx_connector_info` behavior. The `emqx_bridge_http` related
information can thus be removed from `emqx_connector_chema` and
`emqx_connector_resource`.
2024-03-19 14:41:23 +01:00
dc08fab0e8
refactor(hstreamdb connector): to use emqx_connector_info
...
This commit refactors the `emqx_bridge_hstreamdb` to use the
`emqx_connector_info` behavior. The `emqx_bridge_hstreamdb` related
information can thus be removed from `emqx_connector_ee_schema` and
`emqx_connector_schema`.
2024-03-19 14:41:20 +01:00
d0bebe8b93
refactor(gcp_pubsub connectors): to use emqx_connector_info
...
This commit refactors the emqx_bridge_gcp_pubsub app to use the
emqx_connector_info behavior. The emqx_bridge_gcp_pubsub
related information can thus be removed from emqx_connector_ee_schema
and emqx_connector_schema.
2024-03-19 14:40:33 +01:00
Andrew Mayorov