91dd1183ad
feat(queue): fix dialyzer issues
2024-07-30 13:01:14 +03:00
65ab81ff74
feat(queue): fix quick resubscription
2024-07-30 13:01:14 +03:00
53d4cd3174
feat(queue): rename leader' stream_progresses to stream_states
2024-07-30 13:01:14 +03:00
7d004b37da
feat(queue): implement stream finalization
2024-07-30 13:01:13 +03:00
e5547005eb
feat(queue): implement resubscribe test
2024-07-30 13:01:13 +03:00
fada2a3fea
feat(queue): reorganize and document shared subs module
2024-07-30 13:01:13 +03:00
b4a010d63b
feat(queue): implement unsubscribe
2024-07-30 13:01:13 +03:00
9bde981c44
feat(queue): fix static check issues
2024-07-30 13:01:12 +03:00
7658e081c5
feat(queue): move design docs to the EIP
2024-07-30 13:01:12 +03:00
8dce530d15
feat(queue): fix progress reporting and more tests
...
We test reassignment during the intensive replay
2024-07-30 13:01:12 +03:00
a20d262327
feat(queue): send progress before fetching new messages
2024-07-30 13:01:11 +03:00
d32f282feb
feat(queue): add graceful disconnect
2024-07-30 13:01:11 +03:00
1d728a05b2
feat(queue): send metadata with agent when connecting to leader
...
It will be used to attach agent taints to improve stream assignment.
2024-07-30 13:01:11 +03:00
49bff5c08a
feat(queue): wrap remote calls in a proto
2024-07-30 13:01:10 +03:00
61eda0ff31
feat(queue): identify agents by SessionId in tests
2024-07-30 13:01:10 +03:00
8f0d807c00
feat(queue): add new test scenarios
2024-07-30 13:01:10 +03:00
bceb5d43ed
feat(queue): fix stream rebalancing issues, update tests
2024-07-30 13:01:10 +03:00
03fea34962
feat(queue): document protocol between agent and leader
...
Document leader's states
2024-07-30 13:01:09 +03:00
082514f557
feat(queue): implement full protocol between agent and leader
2024-07-30 13:01:09 +03:00
c831f0772f
feat(queue): handle renew_lease_timeout
2024-07-30 13:01:09 +03:00
ca455ad992
Merge pull request #13532 from emqx/sync-release-57-20240729-021938
...
Sync release-57
2024-07-30 10:49:09 +02:00
a49cd78aae
refactor: force getenv to access only OS env with prefix EMQXVAR_
2024-07-29 23:54:00 +02:00
18721d05bc
Merge pull request #13526 from ieQu1/replicant-ee
...
fix(mria): Reserve replicant role for EE only
2024-07-29 22:10:06 +02:00
7f7d0741d2
Merge pull request #13528 from zmstone/0726-unrecoverable-error-limit
...
0726 unrecoverable error limit
2024-07-29 21:32:14 +02:00
2e39c4ad5e
Merge pull request #13495 from thalesmg/20240719-m-hide-enable-fields-mkII
...
chore: hide enable flags from schema and config examples
2024-07-29 21:31:20 +02:00
e08425e67d
refactor(log-throttler): remove unnecessary code
...
there is no need to reset counters before erasing
2024-07-29 20:41:27 +02:00
f6f1d32da0
feat: throttle with resource_id
2024-07-29 20:41:27 +02:00
2924ec582a
feat: add unrecoverable_resource_error throttle
2024-07-29 20:41:27 +02:00
8dc1d1424a
chore: add resource tag for log
2024-07-29 20:41:27 +02:00
693d5dd394
feat: attempt to automatically decode `payload` similar to key and message templates
2024-07-29 13:01:06 -03:00
60aefd1065
Merge pull request #13520 from lafirest/feat/scram-rest-acl
...
feat(scram): supports ACL rules in `scram_restapi` backend
2024-07-29 22:46:50 +08:00
c637422302
Merge pull request #13518 from thalesmg/20240724-r57-dynamic-kprodu-action-mkIII
...
feat(kafka producer): allow dynamic topics (mkIII)
2024-07-29 22:43:20 +08:00
e80d43d14d
test(fix): use ebin path without plugins
...
Without the filtering that already exists in cth:ebin_path, the rebar3 plugins path may
take priority over normal dependencies. Since we just updated hocon, and there seems to
be an older hocon among the rebar3 plugins, it started to break the test because older
hocon was getting loaded in the peer.
2024-07-29 09:50:25 -03:00
b3074144cc
chore: temporarily revert `NO_DOC` changes to fields with default value = false
...
These will be dealt with in follow up PRs, by allowing the parent struct to be set to a
special `disabled` value in such cases.
2024-07-29 09:50:25 -03:00
6786c9b517
refactor: improve descriptions and identifiers
...
Co-authored-by: zmstone <zmstone@gmail.com>
2024-07-29 09:45:52 -03:00
8913de10c0
Merge pull request #13527 from thalesmg/20240726-r57-multiple-froms-sql-test
...
fix(rule engine tester): fix message publish with bridge source in from clause
2024-07-29 09:37:17 -03:00
5ddd7d7a6a
test: superuser skipped all authz check
2024-07-29 17:27:36 +08:00
d7cac74bed
feat: add authz skipped trace
2024-07-29 17:27:36 +08:00
c1e2801f41
Merge remote-tracking branch 'origin/release-57' into sync-release-57-20240729-021938
2024-07-29 02:19:38 +00:00
268f887700
test(mgmt): Disable certain tests on OSS
2024-07-26 20:24:53 +02:00
1d56ac6e5e
refactor: change topic schema type
2024-07-26 14:26:21 -03:00
4e0742c66f
feat: make kafka producer freely dynamic
2024-07-26 14:25:20 -03:00
8c1302f455
test(conf_app): Remove redundand config
2024-07-26 17:25:09 +02:00
b8a2a8ea18
test(router): Skip certain tests on OSS
2024-07-26 17:25:09 +02:00
b7c424a13d
test(persmsg): Remove redundant config
2024-07-26 17:17:06 +02:00
1b6494ab9a
test(mgmt): Remove redundant config
2024-07-26 17:17:06 +02:00
41bf5cd6ca
test(otel): Remove redundant config
2024-07-26 17:17:06 +02:00
548bcceab7
test(auth): Remove redundant config
2024-07-26 17:17:06 +02:00
1beda1cd11
test(mria): Remove role from the example config
2024-07-26 17:17:06 +02:00
9da744c423
fix(mria): Reserve replicant role for EE only
2024-07-26 17:17:06 +02:00
b2f2af6871
Merge pull request #13523 from lafirest/fix/oidc
...
fix(oidc): fixed update and callback errors for OIDC
2024-07-26 21:09:11 +08:00
3fae704903
fix(rule engine tester): fix message publish with bridge source in from clause
...
Fixes https://emqx.atlassian.net/browse/EMQX-12762
2024-07-26 09:27:16 -03:00
397c104a85
feat(exclusive): added CLI interface for exclusive topics
2024-07-26 16:56:47 +08:00
49b24a3049
fix(oidc): fixed update and callback errors for OIDC
2024-07-26 15:41:22 +08:00
7bf70aaab6
feat(scram): supports ACL rules in `scram_restapi` backend
2024-07-26 14:30:28 +08:00
df1f4fad70
feat(kafka producer): allow dynamic topics from pre-configured topics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12656
2024-07-25 15:33:12 -03:00
39b8cb1789
Merge pull request #13487 from thalesmg/20240715-m-refactor-cluster-link-api
...
feat(cluster link): refactor http api, add status and metrics
2024-07-25 14:51:36 -03:00
33eccb35da
chore: update wolff -> 3.0.2
2024-07-25 14:30:50 -03:00
7631420eef
test: add test case to cover ldap search timeout
2024-07-25 18:43:37 +02:00
8f94e9684c
fix: handle ldap seqrch error
2024-07-25 18:42:09 +02:00
03821c7b49
fix(cluster link metrics): route count metric is cluster-wide
2024-07-25 13:12:08 -03:00
6da71200f3
refactor: improve bookkeeping api
2024-07-25 13:12:08 -03:00
6dbf015c93
refactor: demote hidden config to hardcoded value
2024-07-25 13:12:08 -03:00
30259284d1
chore: namespace metrics by type
2024-07-25 13:12:08 -03:00
87e4e2340d
refactor: better metric and error fold
2024-07-25 13:12:08 -03:00
1925ed2f55
Merge pull request #13507 from lafirest/feat/env_func
...
feat(variform): add a builtin function to get env vars
2024-07-25 22:46:50 +08:00
a45f817f0e
Merge pull request #13515 from lafirest/fix/exclusive
...
fix(exclusive): allow the same client to resubscribe to an existing exclusive topic
2024-07-25 21:26:08 +08:00
79020b2436
feat(variform): add a builtin function to get env vars
2024-07-25 18:50:53 +08:00
141d8144e4
fix(scram): change the name from `scram_http` to `scram_restapi`
2024-07-25 17:01:49 +08:00
117c8197d7
fix(exclusive): allow the same client to resubscribe to an existing exclusive topic
2024-07-25 09:40:15 +08:00
b283a8c1ff
Merge pull request #13505 from thalesmg/20240722-m-rule-conn-deps-part-2
...
feat(rule engine api): add filters options for action and source ids
2024-07-24 16:52:47 -03:00
c728b98e79
Merge pull request #13510 from thalesmg/20240723-r57-fix-jwt-about-to-expire-check
...
fix(jwt): fix grace period for renewal check
2024-07-24 16:52:35 -03:00
dda73651c5
fix(cluster link metrics): use periodic full table scan and gauge to count routes
2024-07-24 16:46:04 -03:00
c31e28153f
Merge pull request #13513 from id/20240724-sync-release-57
...
sync release-57
2024-07-24 20:12:06 +02:00
7829838dc5
feat(cluster link api): add forwarding resource metrics to response
2024-07-24 14:53:57 -03:00
80e035f115
feat(rule engine api): add filters options for action and source ids
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654 (requirement 2)
2024-07-24 13:32:50 -03:00
34f5a886ce
refactor(cluster link api): return erpc errors in status and metrics responses
2024-07-24 12:07:34 -03:00
79db2e6d7f
test: fix flaky test
2024-07-24 11:17:00 -03:00
3e4eeddb78
fix: add missing `resource_type` callback implementations
2024-07-24 10:53:33 -03:00
d2da311416
fix(resource): create undocumented callback
...
Created by https://github.com/emqx/emqx/pull/13449 but not added as a callback.
2024-07-24 10:53:33 -03:00
76e51fa532
fix: correctly use maybe match clause
2024-07-24 10:17:45 -03:00
82bb876de0
docs: improve descriptions
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2024-07-24 10:15:01 -03:00
2d507146ab
refactor: change style of case clause
2024-07-24 10:13:48 -03:00
216a6abed9
refactor: rename CRUD functions
2024-07-24 10:11:03 -03:00
ca2d4ad2a0
refactor: move metrics logic to separate module
2024-07-24 10:04:27 -03:00
9a950571d8
Merge pull request #13492 from thalesmg/20240718-m-rules-conn-deps
...
feat: return dependent entities in connectors/actions/sources API
2024-07-24 09:16:00 -03:00
9e65e0d048
Merge pull request #13503 from thalesmg/20240722-r57-resource-manager-hc-interval-startup
...
fix(connector resource): use configuration `resource_opts` for health check interval when starting up
2024-07-24 09:15:47 -03:00
b010efb647
fix(ds): Improve logic of skipstream LTS layout
...
Iterators:
Previously it used timestamp of the next message as a reference. This
won't work well for the upcoming beamformer/beamsplitter feature. This
commit changes the logic so iterators store timestamp of the last seen
message.
Cooked batches:
Cooked batches no longer store index entries. Creation of indexes has
been delegated to commit callback.
2024-07-24 10:32:06 +02:00
606d829256
Merge remote-tracking branch 'upstream/release-57' into 20240724-sync-release-57
2024-07-24 10:28:00 +02:00
c7a7658c7a
Merge pull request #13449 from zhongwencool/resource-log
...
feat: add group/type to resource slog
2024-07-24 14:34:25 +08:00
4d7535df2d
chore: use pgsqsl replace postgresql
2024-07-24 13:49:31 +08:00
8a344a8646
Merge pull request #13504 from lafirest/feat/scram-http
...
feat(authn): added a HTTP backend for the authentication mechanism scram
2024-07-24 10:28:57 +08:00
7374123c5c
fix(jwt): fix grace period for renewal check
2024-07-23 17:25:29 -03:00
9c0f1df8a3
Merge pull request #13506 from thalesmg/20240722-m-peername-sys-events
...
feat: add `peername` to rule events that already have `peerhost`
2024-07-23 09:38:57 -03:00
e7d07ea17c
feat: add resource_type to emqx_resource behaviour
2024-07-23 18:24:28 +08:00
878b218692
feat(authn): added a HTTP backend for the authentication mechanism scram
2024-07-23 16:07:32 +08:00
e74a921d33
chore: compile error
2024-07-23 15:14:42 +08:00
2a58a36e37
chore: add resource tag for log
2024-07-23 15:14:42 +08:00
2bb062d3a3
chore: create_local/5 for emqx_resource_proto_v1
2024-07-23 15:14:42 +08:00
f29988ed8e
chore: add tag to resouce log
2024-07-23 15:14:42 +08:00
e148d903e8
feat: log resource_id
2024-07-23 15:14:42 +08:00
0a04b1ad6e
feat: add group/type to resource slog
2024-07-23 15:14:41 +08:00
cba3f532f8
feat: don't record dry_run log
2024-07-23 15:14:41 +08:00
7bb7b10a31
Merge pull request #13114 from emqx/emqx-relup-gen
...
feat: generate relup tarball, add relup APIs
2024-07-23 15:00:52 +08:00
439abe430b
refactor: remove relup revert callback functions
2024-07-23 11:45:55 +08:00
99e6613713
test(rule events): add test cases for `schema.validation_failed` and `message.transformation_failed` events
2024-07-22 16:31:48 -03:00
d9832252d8
refactor: add namespace to avoid clashes with operations or other resources
2024-07-22 16:04:19 -03:00
6a5849488c
feat(cluster link): add metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
07cb147d38
fix(cluster link schema): username is not required
2024-07-22 16:04:19 -03:00
ba3cbe02e3
feat(cluster link api): add status to responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
0b1f0db73c
chore(cluster link): refactor HTTP API for CRUD operations
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
7ca5205f3f
feat: add `peername` to rule events that already have `peerhost`
...
Fixes https://emqx.atlassian.net/browse/EMQX-12342
2024-07-22 16:01:30 -03:00
69f5b6fa6c
chore: hide `enable` fields from docgen
...
Fixes https://emqx.atlassian.net/browse/EMQX-12730
2024-07-22 13:26:53 -03:00
8ae54ac325
fix(connector resource): use configuration `resource_opts` for health check interval when starting up
...
Fixes https://emqx.atlassian.net/browse/EMQX-12738
2024-07-22 11:34:10 -03:00
220fbe8a0a
test: fix flaky test
2024-07-22 09:44:51 -03:00
a2bed1efb8
Merge pull request #13480 from zhongwencool/rule-engine-log-tag
...
feat: Rule engine log tag
2024-07-22 16:47:13 +08:00
57b67ebb37
Merge pull request #13481 from zhongwencool/match_rule_error
...
chore: add authz tag to match_rule_error log
2024-07-22 16:46:49 +08:00
862336a2cb
feat: hide relup plugins from APIs and CLIs
2024-07-22 16:07:50 +08:00
ed2fab51e9
Merge remote-tracking branch 'origin/release-57' into sync-release-57-20240722-022026
2024-07-22 02:20:27 +00:00
65544f34ec
chore: bump hocon -> 0.43.2
2024-07-19 17:25:18 -03:00
8d535bbd24
Merge pull request #13464 from thalesmg/20240712-m-res-manager-shutdown-logs
...
chore: attempt to reduce race condition supervisor noproc shutdown error logs
2024-07-19 14:57:56 -03:00
4d174b8678
feat(sources & actions api): add dependent rules to response
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654
2024-07-19 14:43:55 -03:00
eb2d3a3b7e
chore: attempt to reduce race condition supervisor shutdown errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-12442
e.g.:
```
2024-05-23T08:52:39.811845+00:00 [error] Supervisor: {local,emqx_resource_manager_sup}. Context: shutdown_error. Reason: noproc. Offender: id=<<99, 101, 110, 115, 111, 114, 101, 100>>,pid=<0.7752.1030>.
```
It could be just a race condition, as it seems to be the case for resource manager: i) a call is made to the process to stop it; ii) the call times out; iii) the after clause ends up calling supervisor:terminate_child; iv) while the supervisor is finding the child to terminate, the process actually finishes terminating, and the supervisor receives a noproc reason back.
2024-07-19 10:57:00 -03:00
ae828e8cfb
feat(connectors api): add dependent actions and sources to response
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654
2024-07-19 10:33:48 -03:00
464e202742
Merge pull request #13488 from thalesmg/20240718-m-mix-fix-machine-dep
...
chore(new mix build): fix app dependency for release
2024-07-19 09:21:02 -03:00
b7200656a5
Merge pull request #13486 from keynslug/fix/ci/ds-raft-flaky-next
...
test(dsraft): attempt to stabilize flaky testcases
2024-07-19 12:15:36 +02:00
fc3405fe4c
fix: bp_api for relup
2024-07-19 17:35:36 +08:00
f11dfce292
ci: suppress dialyzer checks for quicer and odbc types
2024-07-19 17:11:49 +08:00
c61828460a
chore: emqx_utils_api:with_node/2 support simple http-code
2024-07-19 15:02:14 +08:00
4d25f28bb2
fix: dialyzer checks
2024-07-19 14:19:50 +08:00
3ad7dc262b
fix: some sanity-checks
2024-07-19 11:38:44 +08:00
3c8ef35b18
fix: show relup status even if no packages installed
2024-07-19 11:20:37 +08:00
01883e9759
chore(new mix build): fix app dependency for release
2024-07-18 11:33:02 -03:00
ca47e4768d
Merge pull request #13485 from thalesmg/20240717-r57-rm-dead-code-purge-proto
...
chore: remove dead code
2024-07-18 09:22:27 -03:00
3a893626b8
Merge pull request #13474 from keynslug/ft/EMQX-12309/ds-cas-api
...
feat(ds): allow isolated batches with preconditions
2024-07-18 12:52:13 +02:00
2008130071
feat: add HTTP APIs for relup
2024-07-18 18:48:38 +08:00
4a04ffdca1
Merge pull request #13483 from id/20240717-sync-release-57
...
sync release-57
2024-07-18 10:41:49 +02:00
c2d49ff34f
chore(rmq_tests): fix rabbitmq tests
...
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2024-07-18 08:34:00 +02:00
ac52bf39ce
Merge pull request #13443 from zhongwencool/cluster-link-cli-load
...
fix: update cluster.links via cli
2024-07-18 09:13:59 +08:00
0e545ffcec
feat(ds): add dedicated `#message_matcher{}` for preconditions
2024-07-17 21:27:17 +02:00
2e89656a90
test(dsraft): start `t_replication_transfers_snapshots` from stable state
2024-07-17 20:01:55 +02:00
466fa41ec3
fix(dsraft): rely on last resort timeout with unresponsive replicas
...
This simplifies the shard transition scheduling logic and makes it less
prone to races.
2024-07-17 19:24:38 +02:00
93c725732c
chore: remove dead code
2024-07-17 12:56:45 -03:00
4edbcc55e7
Merge pull request #13463 from thalesmg/20240712-m-gprodu-backoff-retry
...
feat(gcp pubsub producer): retry on 502 and 503 http status code responses
2024-07-17 12:49:27 -03:00
cd8bf2725a
Merge pull request #13453 from thalesmg/20240711-r57-mt-fixes
...
batch of message transformation fixes
2024-07-17 12:45:19 -03:00
0c05b3f019
fix(ds): make conditionals less confusing
2024-07-17 16:23:41 +02:00
78fe9304be
Merge pull request #13462 from keynslug/fix/ci/flaky-ds-raft
...
fix(dsraft): preserve pending replica set transitions
2024-07-17 16:11:59 +02:00
14022aded1
feat(ds): allow isolated batches with preconditions
...
Namely, single message deletions and preconditions that can be used to
build complex "compare-and-swap"-style operations. Also allow user to
declare that atomic batches support is needed for a DB.
2024-07-17 15:57:17 +02:00
02e1007a16
feat(dslocal): implement `force_monotonic_timestamps => false`
2024-07-17 15:49:50 +02:00
3381eecd6f
chore: apply code review
2024-07-17 21:27:07 +08:00
Ilya Averyanov