Commit Graph

20630 Commits

Author SHA1 Message Date
Andrew Mayorov 19305c223c
fix(dsrepl): require majority for replication-related tables 2024-03-19 20:11:42 +01:00
Andrew Mayorov f89909f60c
fix(dsrepl): tolerate trigger election timeouts for existing servers 2024-03-19 20:11:42 +01:00
Andrew Mayorov 6c6ea50e42
fix(boot): add `emqx_durable_storage` to the list of restarted apps 2024-03-19 20:11:42 +01:00
Andrew Mayorov 3b59cf2ebf
feat(dsrepl): move shard allocation to separate process
That starts shard and egress processes only when shards are fully
allocated.
2024-03-19 20:11:41 +01:00
Andrew Mayorov 4dafbf21f6
fix(dsrepl): make db + shard part of machine state
It doesn't feel right, but right now is the easiest way to have it
in the scope of `apply/3`, because `init/1` doesn't actually invoked
for ra machines recovered from the existing log / snapshot.
2024-03-19 20:11:41 +01:00
Andrew Mayorov d19128ed65
feat(dsrepl): cache shard metadata in persistent terms 2024-03-19 20:11:41 +01:00
Andrew Mayorov e6c2c2fb07
feat(dsrepl): manage generations / db config through ra machine 2024-03-19 20:11:39 +01:00
Andrew Mayorov 5e94bdb932
feat(dsrepl): allocate shards once predefined number of sites online
Before this commit the most likely shard allocation outcome was
that all shard are allocated to just one node.
2024-03-19 20:11:03 +01:00
Andrew Mayorov be793e4735
fix(dsrepl): reassign timestamp at the time of submission
This is needed to ensure total message order for a shard, and
guarantee that no messages will be written "in the past". which
may break replay consistency.
2024-03-19 20:11:01 +01:00
Andrew Mayorov 146f082fdc
feat(dsrepl): implement raft-based replication
Still very rough but mostly working.
2024-03-19 20:09:44 +01:00
Ivan Dyachkov 76d14de480
Merge pull request #12722 from emqx/dependabot/github_actions/dot-github/actions/prepare-jmeter/actions-prepare-jmeter-0101c7941f
chore(deps): bump the actions-prepare-jmeter group in /.github/actions/prepare-jmeter with 2 updates
2024-03-19 19:14:36 +01:00
Ivan Dyachkov 923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
sync release-56
2024-03-19 19:09:41 +01:00
Ivan Dyachkov f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
Thales Macedo Garitezi a689ae72e3
Merge pull request #12711 from thalesmg/data-validation-m-20240311
feat: implement message validation
2024-03-19 11:12:28 -03:00
Thales Macedo Garitezi c478e375f6 refactor: change failure log configuration schema 2024-03-19 10:04:18 -03:00
JianBo He c7db157ecd
Merge pull request #12735 from zhongwencool/fix-ban-crash
fix: don't crash when ban ip error
2024-03-19 20:57:54 +08:00
Thales Macedo Garitezi 2eac54bf54 chore: rm `move` API
Only `reorder` API will be supported, after discussions with product and frontend teams.
2024-03-19 09:55:55 -03:00
SergeTupchiy 92fd0e453a
Merge pull request #12719 from SergeTupchiy/EMQX-11900-multiple-clientid-username-clients-API
Support multiple clientid/username Qs params in clients API
2024-03-19 14:44:39 +02:00
JianBo He 24c04e715d
Merge pull request #12715 from zhongwencool/fix-source-replace-error
fix: cant replace source conf
2024-03-19 19:32:47 +08:00
zhongwencool 111cbbbe3c fix: don't crash when ban ip error 2024-03-19 19:10:36 +08:00
zhongwencool d7c01a4fe0 fix: cant replace source conf 2024-03-19 18:53:51 +08:00
JianBo He 92b53f23da
Merge pull request #12733 from zhongwencool/fix-otel-metrics-create-view-crash
fix: otel metrics create view crash
2024-03-19 16:10:58 +08:00
Ivan Dyachkov 0ce56fc397
Merge pull request #12728 from id/0318-optimize-docker-image-layers
build(docker): optimize docker image layers
2024-03-19 07:53:02 +01:00
zhongwencool 9a44c6712a fix: otel metrics create view crash 2024-03-19 12:20:49 +08:00
JianBo He 8227e567a1
Merge pull request #12725 from HJianBo/export_the_source_types
Support to return the supported source types via `GET /source_types` endpoint
2024-03-19 09:07:51 +08:00
Thales Macedo Garitezi e8a8416e44 feat(message_validation): impose restrictions on validation name 2024-03-18 15:57:39 -03:00
Thales Macedo Garitezi 74c03377f2 feat(message_validation_api): implement `reorder` API 2024-03-18 14:30:52 -03:00
Ivan Dyachkov 255d9e7d1e ci(docker): return condition on when to push images 2024-03-18 17:23:21 +01:00
Ivan Dyachkov 58d0f04056 build(docker): optimize docker image layers
chown only mutable state directories
2024-03-18 17:23:19 +01:00
Thales Macedo Garitezi bcb7fe96d5 refactor: remove ambiguity from api
In preparation for the "reorder" API
2024-03-18 13:20:05 -03:00
Thales Macedo Garitezi 96ffc9b174 refactor: export `schema_check` fn from schema registry 2024-03-18 13:11:39 -03:00
Thales Macedo Garitezi cde87bce66 chore: add missing traces 2024-03-18 13:11:39 -03:00
Thales Macedo Garitezi f84a996671 feat: implement message validation
Fixes https://emqx.atlassian.net/browse/EMQX-11980
2024-03-18 13:11:39 -03:00
Serge Tupchii 8f8b023429 refactor: simplify match spec holder construction 2024-03-18 18:00:26 +02:00
Serge Tupchii b3d44125f6 feat: support multiple clientid / username Qs params in "/clients" API
Additionally, add an option to specify which Client info fields to return in the response.
2024-03-18 18:00:26 +02:00
JianBo He 2a611e4508 chore: fix the error api desc 2024-03-18 18:27:28 +08:00
JianBo He 485d8dbbdb feat: add the `get /source_types` endpoint to return all supported source 2024-03-18 18:11:06 +08:00
Ivan Dyachkov f4e20f8fc9
Merge pull request #12724 from emqx/0318-fix-multiarch-docker-image-push
ci(docker): invoke buildx again to push multiarch images
2024-03-18 10:47:38 +01:00
Ivan Dyachkov 507e6cf770 ci(docker): invoke buildx again to push multiarch images 2024-03-18 09:08:12 +01:00
dependabot[bot] db249454b9
chore(deps): bump the actions-prepare-jmeter group
Bumps the actions-prepare-jmeter group in /.github/actions/prepare-jmeter with 2 updates: [actions/checkout](https://github.com/actions/checkout) and [actions/setup-java](https://github.com/actions/setup-java).


Updates `actions/checkout` from 4.1.1 to 4.1.2
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](b4ffde65f4...9bb56186c3)

Updates `actions/setup-java` from 4.1.0 to 4.2.1
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](9704b39bf2...99b8673ff6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions-prepare-jmeter
- dependency-name: actions/setup-java
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions-prepare-jmeter
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-18 03:52:39 +00:00
dependabot[bot] c7ad4f0f68
chore(deps): bump the actions group with 3 updates
Bumps the actions group with 3 updates: [actions/checkout](https://github.com/actions/checkout), [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) and [docker/login-action](https://github.com/docker/login-action).


Updates `actions/checkout` from 4.1.1 to 4.1.2
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](b4ffde65f4...9bb56186c3)

Updates `docker/setup-buildx-action` from 3.1.0 to 3.2.0
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](0d103c3126...2b51285047)

Updates `docker/login-action` from 3.0.0 to 3.1.0
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](343f7c4344...e92390c5fb)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-18 03:11:41 +00:00
JianBo He 59ef14ed9c
Merge pull request #12714 from JimMoen/fix-shared-sub-count-in-api
0315 Fix `shared_subscriptions` counting error. followup #12670
2024-03-18 09:37:44 +08:00
JimMoen 871c8e7790
docs: add changelog for PR 12714 2024-03-16 20:43:06 +08:00
JimMoen 889eb48368
fix(shared-sub): update stats when unsub shared topic filter 2024-03-16 20:43:05 +08:00
JimMoen 59e280dc54
test: cluster consistented metrics and label for nodes running/stopped
- `emqx_cluster_sessions_count`
- `emqx_cluster_sessions_max`
- `emqx_subscriptions_shared_count`
- `emqx_subscriptions_shared_max`
- `emqx_cluster_nodes_running`
- `emqx_cluster_nodes_stopped`
2024-03-16 20:43:04 +08:00
JimMoen 55a8488a4f
test: fields in `/monitor_current`
- `retained_msg_count`
- `shared_subscriptions`
2024-03-16 20:43:03 +08:00
JimMoen 02b3292025
fix: cluster nodes running/stopped with node name as label 2024-03-16 20:39:34 +08:00
JimMoen 0e25eb287b
fix(prom): cluster session count/max is cluster consistented 2024-03-16 20:39:33 +08:00
JimMoen 62f8f889e1
refactor: func rename to fix typo 2024-03-16 20:39:32 +08:00
JimMoen e56bee618b
fix: `shared_subscriptions` is cluster consistented 2024-03-16 20:39:31 +08:00