Ilya Averyanov
91dd1183ad
feat(queue): fix dialyzer issues
2024-07-30 13:01:14 +03:00
Ilya Averyanov
65ab81ff74
feat(queue): fix quick resubscription
2024-07-30 13:01:14 +03:00
Ilya Averyanov
53d4cd3174
feat(queue): rename leader' stream_progresses to stream_states
2024-07-30 13:01:14 +03:00
Ilya Averyanov
7d004b37da
feat(queue): implement stream finalization
2024-07-30 13:01:13 +03:00
Ilya Averyanov
e5547005eb
feat(queue): implement resubscribe test
2024-07-30 13:01:13 +03:00
Ilya Averyanov
fada2a3fea
feat(queue): reorganize and document shared subs module
2024-07-30 13:01:13 +03:00
Ilya Averyanov
b4a010d63b
feat(queue): implement unsubscribe
2024-07-30 13:01:13 +03:00
Ilya Averyanov
9bde981c44
feat(queue): fix static check issues
2024-07-30 13:01:12 +03:00
Ilya Averyanov
7658e081c5
feat(queue): move design docs to the EIP
2024-07-30 13:01:12 +03:00
Ilya Averyanov
8dce530d15
feat(queue): fix progress reporting and more tests
...
We test reassignment during the intensive replay
2024-07-30 13:01:12 +03:00
Ilya Averyanov
a20d262327
feat(queue): send progress before fetching new messages
2024-07-30 13:01:11 +03:00
Ilya Averyanov
d32f282feb
feat(queue): add graceful disconnect
2024-07-30 13:01:11 +03:00
Ilya Averyanov
1d728a05b2
feat(queue): send metadata with agent when connecting to leader
...
It will be used to attach agent taints to improve stream assignment.
2024-07-30 13:01:11 +03:00
Ilya Averyanov
49bff5c08a
feat(queue): wrap remote calls in a proto
2024-07-30 13:01:10 +03:00
Ilya Averyanov
61eda0ff31
feat(queue): identify agents by SessionId in tests
2024-07-30 13:01:10 +03:00
Ilya Averyanov
8f0d807c00
feat(queue): add new test scenarios
2024-07-30 13:01:10 +03:00
Ilya Averyanov
bceb5d43ed
feat(queue): fix stream rebalancing issues, update tests
2024-07-30 13:01:10 +03:00
Ilya Averyanov
03fea34962
feat(queue): document protocol between agent and leader
...
Document leader's states
2024-07-30 13:01:09 +03:00
Ilya Averyanov
082514f557
feat(queue): implement full protocol between agent and leader
2024-07-30 13:01:09 +03:00
Ilya Averyanov
c831f0772f
feat(queue): handle renew_lease_timeout
2024-07-30 13:01:09 +03:00
Ivan Dyachkov
ca455ad992
Merge pull request #13532 from emqx/sync-release-57-20240729-021938
...
Sync release-57
2024-07-30 10:49:09 +02:00
zmstone
a49cd78aae
refactor: force getenv to access only OS env with prefix EMQXVAR_
2024-07-29 23:54:00 +02:00
ieQu1
18721d05bc
Merge pull request #13526 from ieQu1/replicant-ee
...
fix(mria): Reserve replicant role for EE only
2024-07-29 22:10:06 +02:00
zmstone
7f7d0741d2
Merge pull request #13528 from zmstone/0726-unrecoverable-error-limit
...
0726 unrecoverable error limit
2024-07-29 21:32:14 +02:00
zmstone
2e39c4ad5e
Merge pull request #13495 from thalesmg/20240719-m-hide-enable-fields-mkII
...
chore: hide enable flags from schema and config examples
2024-07-29 21:31:20 +02:00
zmstone
e08425e67d
refactor(log-throttler): remove unnecessary code
...
there is no need to reset counters before erasing
2024-07-29 20:41:27 +02:00
zhongwencool
f6f1d32da0
feat: throttle with resource_id
2024-07-29 20:41:27 +02:00
zhongwencool
2924ec582a
feat: add unrecoverable_resource_error throttle
2024-07-29 20:41:27 +02:00
zhongwencool
8dc1d1424a
chore: add resource tag for log
2024-07-29 20:41:27 +02:00
Thales Macedo Garitezi
693d5dd394
feat: attempt to automatically decode payload similar to key and message templates
2024-07-29 13:01:06 -03:00
lafirest
60aefd1065
Merge pull request #13520 from lafirest/feat/scram-rest-acl
...
feat(scram): supports ACL rules in `scram_restapi` backend
2024-07-29 22:46:50 +08:00
JianBo He
c637422302
Merge pull request #13518 from thalesmg/20240724-r57-dynamic-kprodu-action-mkIII
...
feat(kafka producer): allow dynamic topics (mkIII)
2024-07-29 22:43:20 +08:00
Thales Macedo Garitezi
e80d43d14d
test(fix): use ebin path without plugins
...
Without the filtering that already exists in cth:ebin_path, the rebar3 plugins path may
take priority over normal dependencies. Since we just updated hocon, and there seems to
be an older hocon among the rebar3 plugins, it started to break the test because older
hocon was getting loaded in the peer.
2024-07-29 09:50:25 -03:00
Thales Macedo Garitezi
b3074144cc
chore: temporarily revert NO_DOC changes to fields with default value = false
...
These will be dealt with in follow up PRs, by allowing the parent struct to be set to a
special `disabled` value in such cases.
2024-07-29 09:50:25 -03:00
Thales Macedo Garitezi
6786c9b517
refactor: improve descriptions and identifiers
...
Co-authored-by: zmstone <zmstone@gmail.com>
2024-07-29 09:45:52 -03:00
Thales Macedo Garitezi
8913de10c0
Merge pull request #13527 from thalesmg/20240726-r57-multiple-froms-sql-test
...
fix(rule engine tester): fix message publish with bridge source in from clause
2024-07-29 09:37:17 -03:00
JimMoen
5ddd7d7a6a
test: superuser skipped all authz check
2024-07-29 17:27:36 +08:00
JimMoen
d7cac74bed
feat: add authz skipped trace
2024-07-29 17:27:36 +08:00
id
c1e2801f41
Merge remote-tracking branch 'origin/release-57' into sync-release-57-20240729-021938
2024-07-29 02:19:38 +00:00
ieQu1
268f887700
test(mgmt): Disable certain tests on OSS
2024-07-26 20:24:53 +02:00
Thales Macedo Garitezi
1d56ac6e5e
refactor: change topic schema type
2024-07-26 14:26:21 -03:00
Thales Macedo Garitezi
4e0742c66f
feat: make kafka producer freely dynamic
2024-07-26 14:25:20 -03:00
ieQu1
8c1302f455
test(conf_app): Remove redundand config
2024-07-26 17:25:09 +02:00
ieQu1
b8a2a8ea18
test(router): Skip certain tests on OSS
2024-07-26 17:25:09 +02:00
ieQu1
b7c424a13d
test(persmsg): Remove redundant config
2024-07-26 17:17:06 +02:00
ieQu1
1b6494ab9a
test(mgmt): Remove redundant config
2024-07-26 17:17:06 +02:00
ieQu1
41bf5cd6ca
test(otel): Remove redundant config
2024-07-26 17:17:06 +02:00
ieQu1
548bcceab7
test(auth): Remove redundant config
2024-07-26 17:17:06 +02:00
ieQu1
1beda1cd11
test(mria): Remove role from the example config
2024-07-26 17:17:06 +02:00
ieQu1
9da744c423
fix(mria): Reserve replicant role for EE only
2024-07-26 17:17:06 +02:00
lafirest
b2f2af6871
Merge pull request #13523 from lafirest/fix/oidc
...
fix(oidc): fixed update and callback errors for OIDC
2024-07-26 21:09:11 +08:00
Thales Macedo Garitezi
3fae704903
fix(rule engine tester): fix message publish with bridge source in from clause
...
Fixes https://emqx.atlassian.net/browse/EMQX-12762
2024-07-26 09:27:16 -03:00
firest
397c104a85
feat(exclusive): added CLI interface for exclusive topics
2024-07-26 16:56:47 +08:00
firest
49b24a3049
fix(oidc): fixed update and callback errors for OIDC
2024-07-26 15:41:22 +08:00
firest
7bf70aaab6
feat(scram): supports ACL rules in scram_restapi backend
2024-07-26 14:30:28 +08:00
Thales Macedo Garitezi
df1f4fad70
feat(kafka producer): allow dynamic topics from pre-configured topics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12656
2024-07-25 15:33:12 -03:00
Thales Macedo Garitezi
39b8cb1789
Merge pull request #13487 from thalesmg/20240715-m-refactor-cluster-link-api
...
feat(cluster link): refactor http api, add status and metrics
2024-07-25 14:51:36 -03:00
Thales Macedo Garitezi
33eccb35da
chore: update wolff -> 3.0.2
2024-07-25 14:30:50 -03:00
zmstone
7631420eef
test: add test case to cover ldap search timeout
2024-07-25 18:43:37 +02:00
zmstone
8f94e9684c
fix: handle ldap seqrch error
2024-07-25 18:42:09 +02:00
Thales Macedo Garitezi
03821c7b49
fix(cluster link metrics): route count metric is cluster-wide
2024-07-25 13:12:08 -03:00
Thales Macedo Garitezi
6da71200f3
refactor: improve bookkeeping api
2024-07-25 13:12:08 -03:00
Thales Macedo Garitezi
6dbf015c93
refactor: demote hidden config to hardcoded value
2024-07-25 13:12:08 -03:00
Thales Macedo Garitezi
30259284d1
chore: namespace metrics by type
2024-07-25 13:12:08 -03:00
Thales Macedo Garitezi
87e4e2340d
refactor: better metric and error fold
2024-07-25 13:12:08 -03:00
lafirest
1925ed2f55
Merge pull request #13507 from lafirest/feat/env_func
...
feat(variform): add a builtin function to get env vars
2024-07-25 22:46:50 +08:00
lafirest
a45f817f0e
Merge pull request #13515 from lafirest/fix/exclusive
...
fix(exclusive): allow the same client to resubscribe to an existing exclusive topic
2024-07-25 21:26:08 +08:00
firest
79020b2436
feat(variform): add a builtin function to get env vars
2024-07-25 18:50:53 +08:00
firest
141d8144e4
fix(scram): change the name from scram_http to scram_restapi
2024-07-25 17:01:49 +08:00
firest
117c8197d7
fix(exclusive): allow the same client to resubscribe to an existing exclusive topic
2024-07-25 09:40:15 +08:00
Thales Macedo Garitezi
b283a8c1ff
Merge pull request #13505 from thalesmg/20240722-m-rule-conn-deps-part-2
...
feat(rule engine api): add filters options for action and source ids
2024-07-24 16:52:47 -03:00
Thales Macedo Garitezi
c728b98e79
Merge pull request #13510 from thalesmg/20240723-r57-fix-jwt-about-to-expire-check
...
fix(jwt): fix grace period for renewal check
2024-07-24 16:52:35 -03:00
Thales Macedo Garitezi
dda73651c5
fix(cluster link metrics): use periodic full table scan and gauge to count routes
2024-07-24 16:46:04 -03:00
Ivan Dyachkov
c31e28153f
Merge pull request #13513 from id/20240724-sync-release-57
...
sync release-57
2024-07-24 20:12:06 +02:00
Thales Macedo Garitezi
7829838dc5
feat(cluster link api): add forwarding resource metrics to response
2024-07-24 14:53:57 -03:00
Thales Macedo Garitezi
80e035f115
feat(rule engine api): add filters options for action and source ids
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654 (requirement 2)
2024-07-24 13:32:50 -03:00
Thales Macedo Garitezi
34f5a886ce
refactor(cluster link api): return erpc errors in status and metrics responses
2024-07-24 12:07:34 -03:00
Thales Macedo Garitezi
79db2e6d7f
test: fix flaky test
2024-07-24 11:17:00 -03:00
Thales Macedo Garitezi
3e4eeddb78
fix: add missing resource_type callback implementations
2024-07-24 10:53:33 -03:00
Thales Macedo Garitezi
d2da311416
fix(resource): create undocumented callback
...
Created by https://github.com/emqx/emqx/pull/13449 but not added as a callback.
2024-07-24 10:53:33 -03:00
Thales Macedo Garitezi
76e51fa532
fix: correctly use maybe match clause
2024-07-24 10:17:45 -03:00
Thales Macedo Garitezi
82bb876de0
docs: improve descriptions
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2024-07-24 10:15:01 -03:00
Thales Macedo Garitezi
2d507146ab
refactor: change style of case clause
2024-07-24 10:13:48 -03:00
Thales Macedo Garitezi
216a6abed9
refactor: rename CRUD functions
2024-07-24 10:11:03 -03:00
Thales Macedo Garitezi
ca2d4ad2a0
refactor: move metrics logic to separate module
2024-07-24 10:04:27 -03:00
Thales Macedo Garitezi
9a950571d8
Merge pull request #13492 from thalesmg/20240718-m-rules-conn-deps
...
feat: return dependent entities in connectors/actions/sources API
2024-07-24 09:16:00 -03:00
Thales Macedo Garitezi
9e65e0d048
Merge pull request #13503 from thalesmg/20240722-r57-resource-manager-hc-interval-startup
...
fix(connector resource): use configuration `resource_opts` for health check interval when starting up
2024-07-24 09:15:47 -03:00
ieQu1
b010efb647
fix(ds): Improve logic of skipstream LTS layout
...
Iterators:
Previously it used timestamp of the next message as a reference. This
won't work well for the upcoming beamformer/beamsplitter feature. This
commit changes the logic so iterators store timestamp of the last seen
message.
Cooked batches:
Cooked batches no longer store index entries. Creation of indexes has
been delegated to commit callback.
2024-07-24 10:32:06 +02:00
Ivan Dyachkov
606d829256
Merge remote-tracking branch 'upstream/release-57' into 20240724-sync-release-57
2024-07-24 10:28:00 +02:00
zhongwencool
c7a7658c7a
Merge pull request #13449 from zhongwencool/resource-log
...
feat: add group/type to resource slog
2024-07-24 14:34:25 +08:00
zhongwencool
4d7535df2d
chore: use pgsqsl replace postgresql
2024-07-24 13:49:31 +08:00
lafirest
8a344a8646
Merge pull request #13504 from lafirest/feat/scram-http
...
feat(authn): added a HTTP backend for the authentication mechanism scram
2024-07-24 10:28:57 +08:00
Thales Macedo Garitezi
7374123c5c
fix(jwt): fix grace period for renewal check
2024-07-23 17:25:29 -03:00
Thales Macedo Garitezi
9c0f1df8a3
Merge pull request #13506 from thalesmg/20240722-m-peername-sys-events
...
feat: add `peername` to rule events that already have `peerhost`
2024-07-23 09:38:57 -03:00
zhongwencool
e7d07ea17c
feat: add resource_type to emqx_resource behaviour
2024-07-23 18:24:28 +08:00
firest
878b218692
feat(authn): added a HTTP backend for the authentication mechanism scram
2024-07-23 16:07:32 +08:00
zhongwencool
e74a921d33
chore: compile error
2024-07-23 15:14:42 +08:00
zhongwencool
2a58a36e37
chore: add resource tag for log
2024-07-23 15:14:42 +08:00
zhongwencool
2bb062d3a3
chore: create_local/5 for emqx_resource_proto_v1
2024-07-23 15:14:42 +08:00
zhongwencool
f29988ed8e
chore: add tag to resouce log
2024-07-23 15:14:42 +08:00
zhongwencool
e148d903e8
feat: log resource_id
2024-07-23 15:14:42 +08:00
zhongwencool
0a04b1ad6e
feat: add group/type to resource slog
2024-07-23 15:14:41 +08:00
zhongwencool
cba3f532f8
feat: don't record dry_run log
2024-07-23 15:14:41 +08:00
Xinyu Liu
7bb7b10a31
Merge pull request #13114 from emqx/emqx-relup-gen
...
feat: generate relup tarball, add relup APIs
2024-07-23 15:00:52 +08:00
Shawn
439abe430b
refactor: remove relup revert callback functions
2024-07-23 11:45:55 +08:00
Thales Macedo Garitezi
99e6613713
test(rule events): add test cases for schema.validation_failed and message.transformation_failed events
2024-07-22 16:31:48 -03:00
Thales Macedo Garitezi
d9832252d8
refactor: add namespace to avoid clashes with operations or other resources
2024-07-22 16:04:19 -03:00
Thales Macedo Garitezi
6a5849488c
feat(cluster link): add metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
Thales Macedo Garitezi
07cb147d38
fix(cluster link schema): username is not required
2024-07-22 16:04:19 -03:00
Thales Macedo Garitezi
ba3cbe02e3
feat(cluster link api): add status to responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
Thales Macedo Garitezi
0b1f0db73c
chore(cluster link): refactor HTTP API for CRUD operations
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
Thales Macedo Garitezi
7ca5205f3f
feat: add peername to rule events that already have peerhost
...
Fixes https://emqx.atlassian.net/browse/EMQX-12342
2024-07-22 16:01:30 -03:00
Thales Macedo Garitezi
69f5b6fa6c
chore: hide enable fields from docgen
...
Fixes https://emqx.atlassian.net/browse/EMQX-12730
2024-07-22 13:26:53 -03:00
Thales Macedo Garitezi
8ae54ac325
fix(connector resource): use configuration resource_opts for health check interval when starting up
...
Fixes https://emqx.atlassian.net/browse/EMQX-12738
2024-07-22 11:34:10 -03:00
Thales Macedo Garitezi
220fbe8a0a
test: fix flaky test
2024-07-22 09:44:51 -03:00
zhongwencool
a2bed1efb8
Merge pull request #13480 from zhongwencool/rule-engine-log-tag
...
feat: Rule engine log tag
2024-07-22 16:47:13 +08:00
zhongwencool
57b67ebb37
Merge pull request #13481 from zhongwencool/match_rule_error
...
chore: add authz tag to match_rule_error log
2024-07-22 16:46:49 +08:00
Shawn
862336a2cb
feat: hide relup plugins from APIs and CLIs
2024-07-22 16:07:50 +08:00
id
ed2fab51e9
Merge remote-tracking branch 'origin/release-57' into sync-release-57-20240722-022026
2024-07-22 02:20:27 +00:00
Thales Macedo Garitezi
65544f34ec
chore: bump hocon -> 0.43.2
2024-07-19 17:25:18 -03:00
Thales Macedo Garitezi
8d535bbd24
Merge pull request #13464 from thalesmg/20240712-m-res-manager-shutdown-logs
...
chore: attempt to reduce race condition supervisor noproc shutdown error logs
2024-07-19 14:57:56 -03:00
Thales Macedo Garitezi
4d174b8678
feat(sources & actions api): add dependent rules to response
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654
2024-07-19 14:43:55 -03:00
Thales Macedo Garitezi
eb2d3a3b7e
chore: attempt to reduce race condition supervisor shutdown errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-12442
e.g.:
```
2024-05-23T08:52:39.811845+00:00 [error] Supervisor: {local,emqx_resource_manager_sup}. Context: shutdown_error. Reason: noproc. Offender: id=<<99, 101, 110, 115, 111, 114, 101, 100>>,pid=<0.7752.1030>.
```
It could be just a race condition, as it seems to be the case for resource manager: i) a call is made to the process to stop it; ii) the call times out; iii) the after clause ends up calling supervisor:terminate_child; iv) while the supervisor is finding the child to terminate, the process actually finishes terminating, and the supervisor receives a noproc reason back.
2024-07-19 10:57:00 -03:00
Thales Macedo Garitezi
ae828e8cfb
feat(connectors api): add dependent actions and sources to response
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654
2024-07-19 10:33:48 -03:00
Thales Macedo Garitezi
464e202742
Merge pull request #13488 from thalesmg/20240718-m-mix-fix-machine-dep
...
chore(new mix build): fix app dependency for release
2024-07-19 09:21:02 -03:00
Andrew Mayorov
b7200656a5
Merge pull request #13486 from keynslug/fix/ci/ds-raft-flaky-next
...
test(dsraft): attempt to stabilize flaky testcases
2024-07-19 12:15:36 +02:00
Shawn
fc3405fe4c
fix: bp_api for relup
2024-07-19 17:35:36 +08:00
Shawn
f11dfce292
ci: suppress dialyzer checks for quicer and odbc types
2024-07-19 17:11:49 +08:00
Shawn
c61828460a
chore: emqx_utils_api:with_node/2 support simple http-code
2024-07-19 15:02:14 +08:00
Shawn
4d25f28bb2
fix: dialyzer checks
2024-07-19 14:19:50 +08:00
Shawn
3ad7dc262b
fix: some sanity-checks
2024-07-19 11:38:44 +08:00
Shawn
3c8ef35b18
fix: show relup status even if no packages installed
2024-07-19 11:20:37 +08:00
Thales Macedo Garitezi
01883e9759
chore(new mix build): fix app dependency for release
2024-07-18 11:33:02 -03:00
Thales Macedo Garitezi
ca47e4768d
Merge pull request #13485 from thalesmg/20240717-r57-rm-dead-code-purge-proto
...
chore: remove dead code
2024-07-18 09:22:27 -03:00
Andrew Mayorov
3a893626b8
Merge pull request #13474 from keynslug/ft/EMQX-12309/ds-cas-api
...
feat(ds): allow isolated batches with preconditions
2024-07-18 12:52:13 +02:00
Shawn
2008130071
feat: add HTTP APIs for relup
2024-07-18 18:48:38 +08:00
Ivan Dyachkov
4a04ffdca1
Merge pull request #13483 from id/20240717-sync-release-57
...
sync release-57
2024-07-18 10:41:49 +02:00
Ivan Dyachkov
c2d49ff34f
chore(rmq_tests): fix rabbitmq tests
...
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2024-07-18 08:34:00 +02:00
zhongwencool
ac52bf39ce
Merge pull request #13443 from zhongwencool/cluster-link-cli-load
...
fix: update cluster.links via cli
2024-07-18 09:13:59 +08:00
Andrew Mayorov
0e545ffcec
feat(ds): add dedicated #message_matcher{} for preconditions
2024-07-17 21:27:17 +02:00
Andrew Mayorov
2e89656a90
test(dsraft): start t_replication_transfers_snapshots from stable state
2024-07-17 20:01:55 +02:00
Andrew Mayorov
466fa41ec3
fix(dsraft): rely on last resort timeout with unresponsive replicas
...
This simplifies the shard transition scheduling logic and makes it less
prone to races.
2024-07-17 19:24:38 +02:00
Thales Macedo Garitezi
93c725732c
chore: remove dead code
2024-07-17 12:56:45 -03:00
Thales Macedo Garitezi
4edbcc55e7
Merge pull request #13463 from thalesmg/20240712-m-gprodu-backoff-retry
...
feat(gcp pubsub producer): retry on 502 and 503 http status code responses
2024-07-17 12:49:27 -03:00
Thales Macedo Garitezi
cd8bf2725a
Merge pull request #13453 from thalesmg/20240711-r57-mt-fixes
...
batch of message transformation fixes
2024-07-17 12:45:19 -03:00
Andrew Mayorov
0c05b3f019
fix(ds): make conditionals less confusing
2024-07-17 16:23:41 +02:00
Andrew Mayorov
78fe9304be
Merge pull request #13462 from keynslug/fix/ci/flaky-ds-raft
...
fix(dsraft): preserve pending replica set transitions
2024-07-17 16:11:59 +02:00
Andrew Mayorov
14022aded1
feat(ds): allow isolated batches with preconditions
...
Namely, single message deletions and preconditions that can be used to
build complex "compare-and-swap"-style operations. Also allow user to
declare that atomic batches support is needed for a DB.
2024-07-17 15:57:17 +02:00
Andrew Mayorov
02e1007a16
feat(dslocal): implement force_monotonic_timestamps => false
2024-07-17 15:49:50 +02:00
zhongwencool
3381eecd6f
chore: apply code review
2024-07-17 21:27:07 +08:00
