03821c7b49
fix(cluster link metrics): route count metric is cluster-wide
2024-07-25 13:12:08 -03:00
6da71200f3
refactor: improve bookkeeping api
2024-07-25 13:12:08 -03:00
6dbf015c93
refactor: demote hidden config to hardcoded value
2024-07-25 13:12:08 -03:00
30259284d1
chore: namespace metrics by type
2024-07-25 13:12:08 -03:00
87e4e2340d
refactor: better metric and error fold
2024-07-25 13:12:08 -03:00
1925ed2f55
Merge pull request #13507 from lafirest/feat/env_func
...
feat(variform): add a builtin function to get env vars
2024-07-25 22:46:50 +08:00
a45f817f0e
Merge pull request #13515 from lafirest/fix/exclusive
...
fix(exclusive): allow the same client to resubscribe to an existing exclusive topic
2024-07-25 21:26:08 +08:00
79020b2436
feat(variform): add a builtin function to get env vars
2024-07-25 18:50:53 +08:00
141d8144e4
fix(scram): change the name from `scram_http` to `scram_restapi`
2024-07-25 17:01:49 +08:00
117c8197d7
fix(exclusive): allow the same client to resubscribe to an existing exclusive topic
2024-07-25 09:40:15 +08:00
b283a8c1ff
Merge pull request #13505 from thalesmg/20240722-m-rule-conn-deps-part-2
...
feat(rule engine api): add filters options for action and source ids
2024-07-24 16:52:47 -03:00
c728b98e79
Merge pull request #13510 from thalesmg/20240723-r57-fix-jwt-about-to-expire-check
...
fix(jwt): fix grace period for renewal check
2024-07-24 16:52:35 -03:00
dda73651c5
fix(cluster link metrics): use periodic full table scan and gauge to count routes
2024-07-24 16:46:04 -03:00
c31e28153f
Merge pull request #13513 from id/20240724-sync-release-57
...
sync release-57
2024-07-24 20:12:06 +02:00
7829838dc5
feat(cluster link api): add forwarding resource metrics to response
2024-07-24 14:53:57 -03:00
80e035f115
feat(rule engine api): add filters options for action and source ids
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654 (requirement 2)
2024-07-24 13:32:50 -03:00
34f5a886ce
refactor(cluster link api): return erpc errors in status and metrics responses
2024-07-24 12:07:34 -03:00
79db2e6d7f
test: fix flaky test
2024-07-24 11:17:00 -03:00
3e4eeddb78
fix: add missing `resource_type` callback implementations
2024-07-24 10:53:33 -03:00
d2da311416
fix(resource): create undocumented callback
...
Created by https://github.com/emqx/emqx/pull/13449 but not added as a callback.
2024-07-24 10:53:33 -03:00
76e51fa532
fix: correctly use maybe match clause
2024-07-24 10:17:45 -03:00
82bb876de0
docs: improve descriptions
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2024-07-24 10:15:01 -03:00
2d507146ab
refactor: change style of case clause
2024-07-24 10:13:48 -03:00
216a6abed9
refactor: rename CRUD functions
2024-07-24 10:11:03 -03:00
ca2d4ad2a0
refactor: move metrics logic to separate module
2024-07-24 10:04:27 -03:00
9a950571d8
Merge pull request #13492 from thalesmg/20240718-m-rules-conn-deps
...
feat: return dependent entities in connectors/actions/sources API
2024-07-24 09:16:00 -03:00
9e65e0d048
Merge pull request #13503 from thalesmg/20240722-r57-resource-manager-hc-interval-startup
...
fix(connector resource): use configuration `resource_opts` for health check interval when starting up
2024-07-24 09:15:47 -03:00
b010efb647
fix(ds): Improve logic of skipstream LTS layout
...
Iterators:
Previously it used timestamp of the next message as a reference. This
won't work well for the upcoming beamformer/beamsplitter feature. This
commit changes the logic so iterators store timestamp of the last seen
message.
Cooked batches:
Cooked batches no longer store index entries. Creation of indexes has
been delegated to commit callback.
2024-07-24 10:32:06 +02:00
606d829256
Merge remote-tracking branch 'upstream/release-57' into 20240724-sync-release-57
2024-07-24 10:28:00 +02:00
c7a7658c7a
Merge pull request #13449 from zhongwencool/resource-log
...
feat: add group/type to resource slog
2024-07-24 14:34:25 +08:00
4d7535df2d
chore: use pgsqsl replace postgresql
2024-07-24 13:49:31 +08:00
8a344a8646
Merge pull request #13504 from lafirest/feat/scram-http
...
feat(authn): added a HTTP backend for the authentication mechanism scram
2024-07-24 10:28:57 +08:00
7374123c5c
fix(jwt): fix grace period for renewal check
2024-07-23 17:25:29 -03:00
9c0f1df8a3
Merge pull request #13506 from thalesmg/20240722-m-peername-sys-events
...
feat: add `peername` to rule events that already have `peerhost`
2024-07-23 09:38:57 -03:00
e7d07ea17c
feat: add resource_type to emqx_resource behaviour
2024-07-23 18:24:28 +08:00
878b218692
feat(authn): added a HTTP backend for the authentication mechanism scram
2024-07-23 16:07:32 +08:00
e74a921d33
chore: compile error
2024-07-23 15:14:42 +08:00
2a58a36e37
chore: add resource tag for log
2024-07-23 15:14:42 +08:00
2bb062d3a3
chore: create_local/5 for emqx_resource_proto_v1
2024-07-23 15:14:42 +08:00
f29988ed8e
chore: add tag to resouce log
2024-07-23 15:14:42 +08:00
e148d903e8
feat: log resource_id
2024-07-23 15:14:42 +08:00
0a04b1ad6e
feat: add group/type to resource slog
2024-07-23 15:14:41 +08:00
cba3f532f8
feat: don't record dry_run log
2024-07-23 15:14:41 +08:00
7bb7b10a31
Merge pull request #13114 from emqx/emqx-relup-gen
...
feat: generate relup tarball, add relup APIs
2024-07-23 15:00:52 +08:00
439abe430b
refactor: remove relup revert callback functions
2024-07-23 11:45:55 +08:00
99e6613713
test(rule events): add test cases for `schema.validation_failed` and `message.transformation_failed` events
2024-07-22 16:31:48 -03:00
d9832252d8
refactor: add namespace to avoid clashes with operations or other resources
2024-07-22 16:04:19 -03:00
6a5849488c
feat(cluster link): add metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
07cb147d38
fix(cluster link schema): username is not required
2024-07-22 16:04:19 -03:00
ba3cbe02e3
feat(cluster link api): add status to responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
0b1f0db73c
chore(cluster link): refactor HTTP API for CRUD operations
...
Fixes https://emqx.atlassian.net/browse/EMQX-12627
2024-07-22 16:04:19 -03:00
7ca5205f3f
feat: add `peername` to rule events that already have `peerhost`
...
Fixes https://emqx.atlassian.net/browse/EMQX-12342
2024-07-22 16:01:30 -03:00
69f5b6fa6c
chore: hide `enable` fields from docgen
...
Fixes https://emqx.atlassian.net/browse/EMQX-12730
2024-07-22 13:26:53 -03:00
8ae54ac325
fix(connector resource): use configuration `resource_opts` for health check interval when starting up
...
Fixes https://emqx.atlassian.net/browse/EMQX-12738
2024-07-22 11:34:10 -03:00
220fbe8a0a
test: fix flaky test
2024-07-22 09:44:51 -03:00
a2bed1efb8
Merge pull request #13480 from zhongwencool/rule-engine-log-tag
...
feat: Rule engine log tag
2024-07-22 16:47:13 +08:00
57b67ebb37
Merge pull request #13481 from zhongwencool/match_rule_error
...
chore: add authz tag to match_rule_error log
2024-07-22 16:46:49 +08:00
862336a2cb
feat: hide relup plugins from APIs and CLIs
2024-07-22 16:07:50 +08:00
ed2fab51e9
Merge remote-tracking branch 'origin/release-57' into sync-release-57-20240722-022026
2024-07-22 02:20:27 +00:00
65544f34ec
chore: bump hocon -> 0.43.2
2024-07-19 17:25:18 -03:00
8d535bbd24
Merge pull request #13464 from thalesmg/20240712-m-res-manager-shutdown-logs
...
chore: attempt to reduce race condition supervisor noproc shutdown error logs
2024-07-19 14:57:56 -03:00
4d174b8678
feat(sources & actions api): add dependent rules to response
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654
2024-07-19 14:43:55 -03:00
eb2d3a3b7e
chore: attempt to reduce race condition supervisor shutdown errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-12442
e.g.:
```
2024-05-23T08:52:39.811845+00:00 [error] Supervisor: {local,emqx_resource_manager_sup}. Context: shutdown_error. Reason: noproc. Offender: id=<<99, 101, 110, 115, 111, 114, 101, 100>>,pid=<0.7752.1030>.
```
It could be just a race condition, as it seems to be the case for resource manager: i) a call is made to the process to stop it; ii) the call times out; iii) the after clause ends up calling supervisor:terminate_child; iv) while the supervisor is finding the child to terminate, the process actually finishes terminating, and the supervisor receives a noproc reason back.
2024-07-19 10:57:00 -03:00
ae828e8cfb
feat(connectors api): add dependent actions and sources to response
...
Fixes https://emqx.atlassian.net/browse/EMQX-12654
2024-07-19 10:33:48 -03:00
464e202742
Merge pull request #13488 from thalesmg/20240718-m-mix-fix-machine-dep
...
chore(new mix build): fix app dependency for release
2024-07-19 09:21:02 -03:00
b7200656a5
Merge pull request #13486 from keynslug/fix/ci/ds-raft-flaky-next
...
test(dsraft): attempt to stabilize flaky testcases
2024-07-19 12:15:36 +02:00
fc3405fe4c
fix: bp_api for relup
2024-07-19 17:35:36 +08:00
f11dfce292
ci: suppress dialyzer checks for quicer and odbc types
2024-07-19 17:11:49 +08:00
c61828460a
chore: emqx_utils_api:with_node/2 support simple http-code
2024-07-19 15:02:14 +08:00
4d25f28bb2
fix: dialyzer checks
2024-07-19 14:19:50 +08:00
3ad7dc262b
fix: some sanity-checks
2024-07-19 11:38:44 +08:00
3c8ef35b18
fix: show relup status even if no packages installed
2024-07-19 11:20:37 +08:00
01883e9759
chore(new mix build): fix app dependency for release
2024-07-18 11:33:02 -03:00
ca47e4768d
Merge pull request #13485 from thalesmg/20240717-r57-rm-dead-code-purge-proto
...
chore: remove dead code
2024-07-18 09:22:27 -03:00
3a893626b8
Merge pull request #13474 from keynslug/ft/EMQX-12309/ds-cas-api
...
feat(ds): allow isolated batches with preconditions
2024-07-18 12:52:13 +02:00
2008130071
feat: add HTTP APIs for relup
2024-07-18 18:48:38 +08:00
4a04ffdca1
Merge pull request #13483 from id/20240717-sync-release-57
...
sync release-57
2024-07-18 10:41:49 +02:00
c2d49ff34f
chore(rmq_tests): fix rabbitmq tests
...
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2024-07-18 08:34:00 +02:00
ac52bf39ce
Merge pull request #13443 from zhongwencool/cluster-link-cli-load
...
fix: update cluster.links via cli
2024-07-18 09:13:59 +08:00
0e545ffcec
feat(ds): add dedicated `#message_matcher{}` for preconditions
2024-07-17 21:27:17 +02:00
2e89656a90
test(dsraft): start `t_replication_transfers_snapshots` from stable state
2024-07-17 20:01:55 +02:00
466fa41ec3
fix(dsraft): rely on last resort timeout with unresponsive replicas
...
This simplifies the shard transition scheduling logic and makes it less
prone to races.
2024-07-17 19:24:38 +02:00
93c725732c
chore: remove dead code
2024-07-17 12:56:45 -03:00
4edbcc55e7
Merge pull request #13463 from thalesmg/20240712-m-gprodu-backoff-retry
...
feat(gcp pubsub producer): retry on 502 and 503 http status code responses
2024-07-17 12:49:27 -03:00
cd8bf2725a
Merge pull request #13453 from thalesmg/20240711-r57-mt-fixes
...
batch of message transformation fixes
2024-07-17 12:45:19 -03:00
0c05b3f019
fix(ds): make conditionals less confusing
2024-07-17 16:23:41 +02:00
78fe9304be
Merge pull request #13462 from keynslug/fix/ci/flaky-ds-raft
...
fix(dsraft): preserve pending replica set transitions
2024-07-17 16:11:59 +02:00
14022aded1
feat(ds): allow isolated batches with preconditions
...
Namely, single message deletions and preconditions that can be used to
build complex "compare-and-swap"-style operations. Also allow user to
declare that atomic batches support is needed for a DB.
2024-07-17 15:57:17 +02:00
02e1007a16
feat(dslocal): implement `force_monotonic_timestamps => false`
2024-07-17 15:49:50 +02:00
3381eecd6f
chore: apply code review
2024-07-17 21:27:07 +08:00
3c832db13d
test: test cluster.links reloaded
2024-07-17 21:06:48 +08:00
937fb153c2
fix: fill_default/3 should populate default values for all parameters except the 'ds'
2024-07-17 21:06:48 +08:00
7b6b9580c8
test: add test for updating cluster.links
2024-07-17 21:06:48 +08:00
2783192f77
fix: update cluster.links via cli
2024-07-17 21:06:48 +08:00
083537daa3
fix: retry not_found if conf file not exist
2024-07-17 21:06:48 +08:00
ae3812da85
feat(ds): allow to turn monotonic timestamps off for DB
...
That tells implementation how to assign timestamps to messages. Current
implicit default is now `force_monotonic_timestamps => true`.
2024-07-17 14:40:23 +02:00
6b130c6422
fix(dsraft): preserve pending replica set transitions
...
Otherwise, information about pending replica set transitions taking a
long time to complete could be lost on subsequent target set changes and
node crashes.
2024-07-17 12:17:07 +02:00
292b331064
Merge remote-tracking branch 'upstream/release-57' into 20240717-sync-release-57
2024-07-17 11:29:25 +02:00
52031441cf
chore: add authz tag to match_rule_error log
2024-07-17 12:21:32 +08:00
604cff4887
feat: add rule tag to rule_engine log
2024-07-17 12:15:57 +08:00
6697035812
Merge pull request #13441 from lafirest/feat/coap
...
feat(coap): use content-sensitive udp proxy for coap
2024-07-17 10:01:51 +08:00
1ad02a11e2
Merge pull request #13455 from thalesmg/20240711-m-mix-umbrella-part-III-no-ci
...
sync new mix build work to master
2024-07-16 14:41:10 -03:00
d04915d6a6
test(dsraft): increase `ra_server` logging level for debugging
2024-07-16 15:54:49 +02:00
78bb102311
test(dsraft): attempt to start select testcases from stable state
2024-07-16 15:54:49 +02:00
4a08bfc93f
feat(mix ct): improve failure logging
2024-07-15 09:26:04 -03:00
02a0ccfdd1
ci: preparations for new mix build
2024-07-15 09:26:04 -03:00
70786d6aca
test: fix suite apps
2024-07-15 09:26:04 -03:00
066fd0481b
feat(mix): compile asn1 files
2024-07-15 09:26:04 -03:00
9e4a84cf76
Merge pull request #13442 from thalesmg/20240709-r57-decouple-connector-action-hc
...
fix(resource manager): disentangle connector and channel health check frequencies
2024-07-15 09:12:07 -03:00
ec183f1d4c
test(coap): fix ci errors
2024-07-15 10:52:37 +08:00
46c2c75b7b
fix(connection): Make process_msg function tail-recursive
2024-07-14 06:00:00 +02:00
0e57b39cf2
feat(gcp pubsub producer): retry on 502 and 503 http status code responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-12625
2024-07-12 15:29:59 -03:00
2401a2fb80
test(dsraft): run `t_join_leave_errors` case in tracing context
2024-07-12 18:28:24 +02:00
96c9020727
chore: improve protobuf decoding error messages
...
Fixes https://emqx.atlassian.net/browse/EMQX-12677
2024-07-12 13:27:30 -03:00
af81800aec
chore(dsraft): log a bit more informative messages in shard allocator
2024-07-12 18:24:58 +02:00
8e8b382ec0
chore(dsraft): provide more details when replica is unready
2024-07-12 18:23:23 +02:00
70a760850f
chore(dsraft): correct comment spelling errors
2024-07-12 15:27:29 +02:00
205ad507ea
test(dsraft): attempt to ensure testcases start from stable state
...
Where "stable state" is currently defined as "everyone knows and agrees
on the current leader".
2024-07-12 15:26:00 +02:00
9ca8aeb155
chore(mgmt): reduce test flakyness
2024-07-12 12:10:49 +03:00
854754eb60
feat(coap): use content-sensitive udp proxy for coap
2024-07-12 16:23:46 +08:00
83cc3ffeb0
fix(banned): let the bootfile of banned be optional
2024-07-12 13:58:14 +08:00
1b7d23cef4
Merge pull request #13451 from lafirest/fix/def_banned_file
...
fix: do not convert a empty file name to a empty list
2024-07-12 13:50:32 +08:00
2816170e9d
chore: add `$events.message_transformation_failed` to rule engine tester
...
Fixes https://emqx.atlassian.net/browse/EMQX-12679
2024-07-11 17:52:08 -03:00
5f595966d8
chore(message transformation): allow empty operation list
...
Fixes https://emqx.atlassian.net/browse/EMQX-12682
2024-07-11 17:52:08 -03:00
04b547d6f5
fix(schema registry): handle large names during lookup
...
Fixes https://emqx.atlassian.net/browse/EMQX-12692
2024-07-11 14:35:31 -03:00
f3c6d10f76
fix(mix): fix compile paths and deps
2024-07-11 14:19:23 -03:00
01d89be743
feat(message transformation): add timestamp and pub_props fields to read context
...
Fixes https://emqx.atlassian.net/browse/EMQX-12684
Fixes https://emqx.atlassian.net/browse/EMQX-12678
2024-07-11 12:15:31 -03:00
44e4b3616d
feat(variform): allow hyphens in identifiers
...
Fixes https://emqx.atlassian.net/browse/EMQX-12683
2024-07-11 12:15:23 -03:00
a4cc3ba9e8
Merge pull request #13375 from kjellwinblad/kjell/fix_connector_lister_speed_limit_clearing/EMQX-12514
...
fix: default value for max_conn_rate etc should be set to infinity
2024-07-11 16:36:01 +02:00
d9b5c5863b
fix: do not convert a empty file name to a empty list
2024-07-11 18:12:38 +08:00
c9e12f30cd
Merge pull request #13444 from lafirest/fix/oidc
...
fix(oidc): Avoid crashes and avoid deleting jwks on update
2024-07-11 17:55:50 +08:00
7664b06e98
Merge pull request #13434 from zmstone/0704-refine-rpc-config
...
0704 refine rpc config
2024-07-11 10:25:45 +02:00
4825079964
chore(ds): Support platform profile
2024-07-10 12:03:23 +02:00
b0e3e405cf
fix(oidc): Avoid crashes and avoid deleting jwks on update
2024-07-10 15:22:43 +08:00
44d533fe6d
Merge pull request #13432 from JimMoen/0705-fix-jwt-pem-check
...
fix: create authn jwt with bad public key
2024-07-10 10:33:54 +08:00
7a23ae7b4d
refactor: expose only server_port for rpc
...
previously, there were 4 port configs:
- tcp_server_port
- ssl_server_port
- tcp_client_port
- ssl_client_port
2024-07-09 22:26:29 +02:00
ee13773496
refactor: rename rpc.tcp_client_num to rpc.client_num
...
tcp_client_num is kept as an alias
2024-07-09 22:26:29 +02:00
48e604bda8
fix(mix grpc): include default erlc options
2024-07-09 15:56:30 -03:00
818070ad44
test(mix): add integration test path
2024-07-09 15:56:30 -03:00
5279ad76be
fix(grpc compiler): unload apps to avoid side effects
2024-07-09 15:56:30 -03:00
b91515b131
fix(schema registry mix): gpb is a runtime dep
2024-07-09 15:56:30 -03:00
6d94809950
Merge pull request #13415 from thalesmg/20240703-m-couchbase-action
...
feat: implement couchbase connector and action
2024-07-09 15:53:11 -03:00
3c370a90aa
Merge pull request #13436 from thalesmg/20240708-r57-custom-headers-jwks
...
feat(jwks): allow specifying custom request headers
2024-07-09 15:52:44 -03:00
9f8a1885a7
fix(resource manager): disentangle connector and channel health check frequencies
...
Fixes https://emqx.atlassian.net/browse/EMQX-12674
2024-07-09 14:53:39 -03:00
d25c4ba06f
Merge pull request #13421 from thalesmg/20240705-r57-docs-mt-api-examples
...
docs(message transformation): add api examples
2024-07-09 10:16:53 -03:00
3721be65ee
fix(ds): Improve comments
2024-07-09 13:15:15 +02:00
d7732a6aac
test(ds): Attempt to stabilize a flaky test
2024-07-09 13:15:15 +02:00
e70c1cfea3
test(ds): Improve stability of replication test suite
2024-07-09 13:15:15 +02:00
dc4ae82798
test(ds): Add message ID
2024-07-09 13:15:14 +02:00
d1b574a67e
perf(dslocal): Run heavy-duty operations in a temporary process
2024-07-09 13:15:14 +02:00
Thales Macedo Garitezi