Commit Graph

20997 Commits

Author SHA1 Message Date
Andrew Mayorov 3b59cf2ebf
feat(dsrepl): move shard allocation to separate process
That starts shard and egress processes only when shards are fully
allocated.
2024-03-19 20:11:41 +01:00
Andrew Mayorov 4dafbf21f6
fix(dsrepl): make db + shard part of machine state
It doesn't feel right, but right now is the easiest way to have it
in the scope of `apply/3`, because `init/1` doesn't actually invoked
for ra machines recovered from the existing log / snapshot.
2024-03-19 20:11:41 +01:00
Andrew Mayorov d19128ed65
feat(dsrepl): cache shard metadata in persistent terms 2024-03-19 20:11:41 +01:00
Andrew Mayorov e6c2c2fb07
feat(dsrepl): manage generations / db config through ra machine 2024-03-19 20:11:39 +01:00
Andrew Mayorov 5e94bdb932
feat(dsrepl): allocate shards once predefined number of sites online
Before this commit the most likely shard allocation outcome was
that all shard are allocated to just one node.
2024-03-19 20:11:03 +01:00
Andrew Mayorov be793e4735
fix(dsrepl): reassign timestamp at the time of submission
This is needed to ensure total message order for a shard, and
guarantee that no messages will be written "in the past". which
may break replay consistency.
2024-03-19 20:11:01 +01:00
Andrew Mayorov 146f082fdc
feat(dsrepl): implement raft-based replication
Still very rough but mostly working.
2024-03-19 20:09:44 +01:00
Ivan Dyachkov 76d14de480
Merge pull request #12722 from emqx/dependabot/github_actions/dot-github/actions/prepare-jmeter/actions-prepare-jmeter-0101c7941f
chore(deps): bump the actions-prepare-jmeter group in /.github/actions/prepare-jmeter with 2 updates
2024-03-19 19:14:36 +01:00
Ivan Dyachkov 923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
sync release-56
2024-03-19 19:09:41 +01:00
Andrew Mayorov 9d7d8edcc3
Merge pull request #12741 from keynslug/fix/flaky-ds-replay-testcase
test(ds): stabilize `t_session_replay_retry` testcase
2024-03-19 18:59:54 +01:00
Andrew Mayorov f453bdf055
chore(topic-api): describe `emqx_persistent_session_ds_router:stream/1` 2024-03-19 18:51:06 +01:00
Andrew Mayorov 027b237f07
test(ds): stabilize `t_session_replay_retry` testcase 2024-03-19 18:33:38 +01:00
Zaiming (Stone) Shi 73121f3ccc
Merge pull request #12730 from zmstone/0318-docs-fix-escape-in-i18n
docs: there is no need to escape quotes in triple-quotes
2024-03-19 18:30:14 +01:00
ieQu1 236685f714 fix(sessds): Channel must destroy sessions that are kicked 2024-03-19 17:49:04 +01:00
Kjell Winblad df8c7152fc refactor(mongodb connector): to use emqx_connector_info
This commit refactors the mongodb connector to use the
`emqx_connector_info` behavior.
2024-03-19 16:09:30 +01:00
Kjell Winblad c3d7d68cfc refactor(kinesis,pgsql,timescale,matrix connectors): emqx_connector_info
This commit refactors the kinesis, pgsql, timescale and matrix
connectors to use the `emqx_connector_info` behavior.
2024-03-19 15:57:23 +01:00
Ivan Dyachkov f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
Thales Macedo Garitezi a689ae72e3
Merge pull request #12711 from thalesmg/data-validation-m-20240311
feat: implement message validation
2024-03-19 11:12:28 -03:00
Andrew Mayorov 10ea367030
test(client-api): attempt to get rid of inter-testcase side effects 2024-03-19 14:56:50 +01:00
Andrew Mayorov 3129a8fa42
test(client-api): fix snabbkaffe crash noise in the CT logs 2024-03-19 14:55:28 +01:00
zmstone b8505ebd54 chore: upgrade to jesse 1.8.0 for draft-06 as default version 2024-03-19 14:47:11 +01:00
Kjell Winblad a3e631cda2 refactor(kafka connectors): to use emqx_connector_info
This commit refactors the `emqx_bridge_kafka` to use the
`emqx_connector_info` behavior. The `emqx_bridge_kafka` related
information can thus be removed from `emqx_connector_chema` and
`emqx_connector_resource`.
2024-03-19 14:41:23 +01:00
Kjell Winblad b8ef357fef refactor(http connector): to use emqx_connector_info
This commit refactors the `emqx_bridge_http` to use the
`emqx_connector_info` behavior. The `emqx_bridge_http` related
information can thus be removed from `emqx_connector_chema` and
`emqx_connector_resource`.
2024-03-19 14:41:23 +01:00
Kjell Winblad dc08fab0e8 refactor(hstreamdb connector): to use emqx_connector_info
This commit refactors the `emqx_bridge_hstreamdb` to use the
`emqx_connector_info` behavior. The `emqx_bridge_hstreamdb` related
information can thus be removed from `emqx_connector_ee_schema` and
`emqx_connector_schema`.
2024-03-19 14:41:20 +01:00
Kjell Winblad d0bebe8b93 refactor(gcp_pubsub connectors): to use emqx_connector_info
This commit refactors the emqx_bridge_gcp_pubsub app to use the
emqx_connector_info behavior. The emqx_bridge_gcp_pubsub
related information can thus be removed from emqx_connector_ee_schema
and emqx_connector_schema.
2024-03-19 14:40:33 +01:00
Kjell Winblad 76d32b6e97 refactor(confluent connector): to use emqx_connector_info
This commit refactors the emqx_bridge_confluent to use the
emqx_connector_info behavior. The emqx_bridge_confluent related
information can thus be removed from emqx_connector_ee_schema and
emqx_connector_schema.
2024-03-19 14:39:43 +01:00
Kjell Winblad 795b668c4f refactor(azure_event_hub connector): to use emqx_connector_info
This commit refactors the emqx_bridge_azure_event_hub to use the
emqx_connector_info behavior. The emqx_bridge_azure_event_hub related
information can thus be removed from emqx_connector_ee_schema and
emqx_connector_schema.
2024-03-19 14:39:17 +01:00
Thales Macedo Garitezi c478e375f6 refactor: change failure log configuration schema 2024-03-19 10:04:18 -03:00
JianBo He c7db157ecd
Merge pull request #12735 from zhongwencool/fix-ban-crash
fix: don't crash when ban ip error
2024-03-19 20:57:54 +08:00
Thales Macedo Garitezi 2eac54bf54 chore: rm `move` API
Only `reorder` API will be supported, after discussions with product and frontend teams.
2024-03-19 09:55:55 -03:00
SergeTupchiy 92fd0e453a
Merge pull request #12719 from SergeTupchiy/EMQX-11900-multiple-clientid-username-clients-API
Support multiple clientid/username Qs params in clients API
2024-03-19 14:44:39 +02:00
Andrew Mayorov e1b3263177
fix(sessds): make dialyzer a tad happier 2024-03-19 13:11:32 +01:00
JianBo He 24c04e715d
Merge pull request #12715 from zhongwencool/fix-source-replace-error
fix: cant replace source conf
2024-03-19 19:32:47 +08:00
zhongwencool 111cbbbe3c fix: don't crash when ban ip error 2024-03-19 19:10:36 +08:00
zhongwencool d7c01a4fe0 fix: cant replace source conf 2024-03-19 18:53:51 +08:00
Andrew Mayorov 985f1c4879
fix(api-topics): expose persistent session topics in APIs 2024-03-19 11:01:26 +01:00
JianBo He 92b53f23da
Merge pull request #12733 from zhongwencool/fix-otel-metrics-create-view-crash
fix: otel metrics create view crash
2024-03-19 16:10:58 +08:00
Ivan Dyachkov 0ce56fc397
Merge pull request #12728 from id/0318-optimize-docker-image-layers
build(docker): optimize docker image layers
2024-03-19 07:53:02 +01:00
zhongwencool 9a44c6712a fix: otel metrics create view crash 2024-03-19 12:20:49 +08:00
JianBo He 8227e567a1
Merge pull request #12725 from HJianBo/export_the_source_types
Support to return the supported source types via `GET /source_types` endpoint
2024-03-19 09:07:51 +08:00
Thales Macedo Garitezi e8a8416e44 feat(message_validation): impose restrictions on validation name 2024-03-18 15:57:39 -03:00
Thales Macedo Garitezi 74c03377f2 feat(message_validation_api): implement `reorder` API 2024-03-18 14:30:52 -03:00
Ivan Dyachkov 255d9e7d1e ci(docker): return condition on when to push images 2024-03-18 17:23:21 +01:00
Ivan Dyachkov 58d0f04056 build(docker): optimize docker image layers
chown only mutable state directories
2024-03-18 17:23:19 +01:00
Thales Macedo Garitezi bcb7fe96d5 refactor: remove ambiguity from api
In preparation for the "reorder" API
2024-03-18 13:20:05 -03:00
Thales Macedo Garitezi 96ffc9b174 refactor: export `schema_check` fn from schema registry 2024-03-18 13:11:39 -03:00
Thales Macedo Garitezi cde87bce66 chore: add missing traces 2024-03-18 13:11:39 -03:00
Thales Macedo Garitezi f84a996671 feat: implement message validation
Fixes https://emqx.atlassian.net/browse/EMQX-11980
2024-03-18 13:11:39 -03:00
Serge Tupchii 8f8b023429 refactor: simplify match spec holder construction 2024-03-18 18:00:26 +02:00
Serge Tupchii b3d44125f6 feat: support multiple clientid / username Qs params in "/clients" API
Additionally, add an option to specify which Client info fields to return in the response.
2024-03-18 18:00:26 +02:00