074d98a14a
test(ds): Refactor ds_SUITE
2024-05-22 18:01:33 +02:00
e4a73f003a
feat(ds): Implement format_status callback
...
Reduce volume of logs and crash reports from DS
2024-05-22 18:01:32 +02:00
1526c527d0
fix(ds): Log generation operations
2024-05-22 18:01:32 +02:00
aca2d9586c
fix(ds): Fix return type of drop_generation
2024-05-22 18:01:32 +02:00
c6fc76e335
fix(ds): Perform read operations on the leader.
2024-05-22 18:01:32 +02:00
4580906405
fix(ds): Use erpc instead of gen_rpc for `delete_next'
2024-05-22 18:01:32 +02:00
8016e9adf4
fix(s3-bridge): restore backward config compatibility
2024-05-22 17:59:52 +02:00
4b540e3bd0
fix: do not leak action configurations in alarm messages
2024-05-22 17:43:07 +02:00
e6c5c1b598
chore(dsrepl): provide more information in rebalancing log messages
2024-05-22 17:24:08 +02:00
c355c9ad50
fix(dsrepl): properly handle transaction abort during forget site
2024-05-22 17:22:55 +02:00
cff8b97e8a
fix: handle channel updated during health check
...
This commit fixes an issue found by CI test case
emqx_bridge_influxdb_SUITE:t_start_stop and others. While the channel
health check process is running, the channel could be removed or updated
which could cause a crash in the resource manager or non up-to-date
alarms being triggered.
2024-05-22 12:15:04 +02:00
39d758c4d6
fix: do not return configs for channels from emqx_resource_manager
2024-05-22 12:15:04 +02:00
917474f694
fix: action config update would sometimes not be reflected in connector
...
Before this commit the following happened sometimes:
1. action status is connected
2. action config is updated to something that should change the status to
disconnected
3. action status is still connected and the old config is being used by
the connector even though the config has been correctly updated.
The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).
The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00
f3e8037e0f
fix(rocketmq): fix namespace error for RocketMQ
2024-05-22 16:29:38 +08:00
a3cd3e31b1
Merge pull request #13084 from keynslug/fix/EMQX-12422/part-size
...
fix(s3-bridge): report inconsistent `min/max_part_size` option
2024-05-22 09:48:09 +02:00
e5f7aa9817
refactor: plguin functions and types rename
2024-05-22 11:51:54 +08:00
33aa61daea
fix: use hocon format as plugin config
2024-05-22 06:37:58 +08:00
14f2a68799
fix: bpapi spec type
2024-05-22 06:01:57 +08:00
31a35f2a15
Merge pull request #13076 from thalesmg/fix-ds-session-count-client-api-r57-20240520
...
fix(client mgmt api): cache disconnected durable session count for `/clients` api
2024-05-21 15:34:21 -03:00
5abd23af5a
test: plugin refactor
2024-05-22 02:09:37 +08:00
87b3b214b9
fix: make static_check happy
2024-05-22 02:09:36 +08:00
3ce091e9d7
fix: ensure plugin tarball and extracted
2024-05-22 02:09:35 +08:00
1abc8cf502
fix: ensure plugin's config on boot and join cluster
2024-05-22 02:09:34 +08:00
677352e498
fix: ensure installed and plugin state on boot
2024-05-22 02:09:33 +08:00
2076681e76
fix: put plugin config with binary namevsn key
2024-05-22 02:09:32 +08:00
140b7ce51e
fix(plugin): schema content only provided in enterprise edition
2024-05-22 02:09:31 +08:00
a7f2f95318
fix: ensure avro file
2024-05-22 02:09:30 +08:00
df7dcb2764
fix: do not let plugin start failed lead emqx start failed
2024-05-22 02:09:29 +08:00
e0e4517d9e
fix: ensure plugin config on boot
2024-05-22 02:09:28 +08:00
e8d4e88118
fix(plugin): serde not found on new-joined nodes
2024-05-22 02:09:26 +08:00
28f6eb0200
fix(s3-bridge): report inconsistent `min/max_part_size` option
2024-05-21 18:49:01 +02:00
7c5cb1acc5
fix(monitor current api): expose `disconnected_durable_sessions` count
...
Fixes https://emqx.atlassian.net/browse/EMQX-12423
2024-05-21 10:03:09 -03:00
8b4a1c3d75
fix(client mgmt api): cache disconnected durable session count for `/clients` api
...
Fixes https://emqx.atlassian.net/browse/EMQX-12396
2024-05-20 11:48:08 -03:00
8279d8c787
Merge pull request #13053 from kjellwinblad/kjell/fix_rule_trace_issues/EMQX-12327/EMQX-12335/EMQX-12336
...
Fix small action trace issues
2024-05-20 16:25:38 +02:00
a5d973b681
docs: hide `/clients_v2` from swagger api docs
...
Since it's not yet ready for production, we'll hide it from the API docs.
2024-05-20 10:00:43 -03:00
3c87bcde46
fix(rule trace): restore logger metadata to its previous value
2024-05-20 10:57:15 +02:00
6940930c3e
chore: bump emqx_auth_http app vsn
2024-05-19 09:30:39 +02:00
c6ee37bacd
Merge pull request #13068 from zmstone/0517-fix-dynamic-license-usage-for-dashboard
...
fix(license): show dynamic max_connections for bc
2024-05-18 09:49:18 +02:00
fc3f525a67
Merge pull request #13066 from keynslug/fix/s3-bridge/desc-html
...
chore(s3-bridge): decrease default aggregation settings
2024-05-17 19:43:19 +02:00
b3afff9638
fix(license): show dynamic max_connections for bc
2024-05-17 17:44:21 +02:00
388e6c8262
fix(dssubs): introduce separate gauge for subscriptions from durable sessions
...
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-17 10:17:16 -03:00
5ce095f30e
fix(rule trace): only include stop_after_rendering when value is true
2024-05-17 14:53:14 +02:00
385ac268d7
chore(s3-bridge): decrease default aggreg interval to 30 minutes
2024-05-17 14:49:18 +02:00
c348f759af
chore(s3-bridge): decrease default aggreg limit to 100K records
2024-05-17 14:19:59 +02:00
2e6db85578
fix(rule trace): rename rule_trigger_time(s) and cleaups
...
This commit renames trace fields rule_trigger_time and
rule_trigger_times to rule_trigger_ts and makes sure that the value for
rule_trigger_ts will always be a list of timestamps.
2024-05-17 14:13:36 +02:00
4d1db9f847
Merge pull request #13059 from keynslug/feat/EMQX-12204/single-action-schema
...
feat(s3-bridge): meld 2 separate actions into a single one
2024-05-17 13:27:40 +02:00
2209b26fa5
fix(rule trace): do not leak trace meta data when tracing action result
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12391
2024-05-17 10:05:33 +02:00
5ba71ae6f7
Merge pull request #13062 from ieQu1/dev/ds-better-naming
...
fix(sessds): Rename last_alive_update_interval -> heartbeat_interval
2024-05-17 09:55:40 +02:00
39c2433afb
fix(s3-bridge): update config typespecs
2024-05-17 09:47:16 +02:00
86f99959b0
Merge pull request #13054 from keynslug/fix/EMQX-12365/node-leave
...
fix(dsrepl): anticipate and handle nodes leaving the cluster
2024-05-17 09:43:15 +02:00
5b2c2b336c
feat(s3-bridge): meld 2 separate actions into a single one
...
Discriminated through `mode` parameter. Also rename "simple" action to
a "direct" upload mode.
2024-05-17 09:34:21 +02:00
1ac7aa151e
Merge pull request #13048 from thalesmg/sv-backup-r57-20240514
...
feat(schema validation): implement backup restore
2024-05-17 09:47:04 +08:00
6f30c1c541
fix(sessds): Rename last_alive_update_interval -> heartbeat_interval
2024-05-17 01:20:10 +02:00
73f17249e9
fix(sessds): Rename session_persistence to durable_sessions
2024-05-16 21:31:32 +02:00
ee6e7174cf
fix(sessds): Rename the durable messages DB to `messages`
2024-05-16 21:31:32 +02:00
5157e61418
fix(dsrepl): verify if shards already allocated first
2024-05-16 18:56:54 +02:00
0119728d45
feat(dsrepl): also reflect pending transitions in ds status
2024-05-16 18:56:21 +02:00
26c4a4f597
feat(dsrepl): reflect conflicts and inconsistencies in ds status
2024-05-16 18:32:08 +02:00
7e86e3e61c
fix(dsrepl): anticipate and handle nodes leaving the cluster
...
Also make `claim_site/2` safer by refusing to claim a site for a node
that is already there.
2024-05-16 18:32:07 +02:00
ed16e678be
fix(rule apply test): add default values to context
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12378
2024-05-16 17:31:44 +02:00
520e91c8fd
fix(greptime trace): remove async info as it is confusing
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12385
2024-05-16 16:50:09 +02:00
b7c2f4a6d7
fix(best_effor_json): make tuple list check more efficient
2024-05-16 15:40:38 +02:00
ba74135079
Merge pull request #13052 from kjellwinblad/kjell/evaluate_lazy_trace_values_in_normal_log_formatters/EMQX-12380
...
fix: evaluate lazy values in normal log formatters when level is debug
2024-05-16 15:02:53 +02:00
34a29e6363
feat(schema validation): implement support for `ctl conf load`
2024-05-16 09:55:42 -03:00
0be1249bbe
Merge pull request #13041 from zmstone/0513-http-authn-header-missing-should-result-in-meaningful-error-message
...
0513 http authn header missing should result in meaningful error message
2024-05-16 13:17:20 +02:00
71b393be89
Merge pull request #13057 from ieQu1/dev/default-n-shards
...
fix(ds): Change the default number of shards to 12
2024-05-16 13:05:34 +02:00
413ad60bdb
fix(cassandra): format rendered trace in a better way
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12393
2024-05-16 11:35:12 +02:00
9fd8e930be
fix(best_effort_json): only do tuple list to map without losing pairs
2024-05-16 11:19:48 +02:00
72b0c63ed2
Merge pull request #13043 from kjellwinblad/kjell/kafka_incorrect_query_type/EMQX-12363
...
fix(kafka trace): do not include query type in trace entry
2024-05-16 10:08:16 +02:00
deffb01544
fix(ds): Change the default number of shards to 12
2024-05-16 08:06:07 +02:00
2eb1bb2b13
Merge pull request #13032 from kjellwinblad/kjell/no_ctx_message_validation_failed/EMQX-12354
...
fix: add handling of message_validation_failed ctx for rule testing
2024-05-15 11:20:18 -03:00
246bce85ec
fix: make mysql action rendered trace include parameters
...
When doing non-batch inserts with the MySQL action, prepared statements
are used. We therefore need to include the parameters to the prepared
statement when tracing the rendered templates.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12335
2024-05-15 14:49:33 +02:00
c72682d81b
fix: remove query mode from redis action trace as it only supports sync
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12336
2024-05-15 14:32:15 +02:00
8cbfc4c0cd
fix: evaluate lazy values in normal log formatters when level is debug
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12380
2024-05-15 11:47:21 +02:00
faf3492e58
Merge pull request #13039 from lafirest/fix/rocket_key
...
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-15 15:28:35 +08:00
3e1e4bab17
Merge pull request #13047 from keynslug/fix/EMQX-12366/api-ds-leave
...
feat(api-ds): provide more information on nonexistent site leave
2024-05-14 23:21:07 +02:00
fd949240c0
feat(schema validation): implement backup restore
...
Fixes https://emqx.atlassian.net/browse/EMQX-12346
2024-05-14 15:58:19 -03:00
bc5071590b
Merge pull request #13033 from thalesmg/fix-ds-more-client-api-fields-r57-20240513
...
fix(ds clients mgmt api): add more fields to disconnected durable sessions
2024-05-14 15:29:40 -03:00
7023e6ad96
Merge pull request #13017 from thalesmg/fix-ds-subs-pages-r57-20240510
...
fix(subs mgmt api): attempt to return mixed clients ds/non-ds in the same page
2024-05-14 15:29:29 -03:00
e811f4a392
fix: add handling of message_validation_failed ctx for rule testing
...
Before this commit, the backend could not handle the test context for
message_validation_failed events that could be sent from the dashboard
since no handling code for this contexts existed in the backend. This
has now been fixed by adding such handling code.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12354
2024-05-14 13:07:32 -03:00
1730a41337
Merge pull request #13046 from thalesmg/refactor-mv-rename-sv-r57-20240514
...
refactor: rename `message_validation` to `schema_validation`
2024-05-14 13:03:12 -03:00
adbc392265
Merge pull request #13027 from kjellwinblad/kjell/fix_trace_json_report_formatting/EMQX-12344
...
fix: JSON trace formatter should handle report style log entries
2024-05-14 18:02:59 +02:00
437a0c0d55
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-14 23:01:31 +08:00
2f90d15ce6
fix(ds-api): adapt CLI to API changes
2024-05-14 16:55:17 +02:00
35e360fcbe
feat(api-ds): provide more information on nonexistent site leave
2024-05-14 15:57:41 +02:00
257dae5211
refactor: rename `message_validation` to `schema_validation`
...
Request from Product team.
Fixes https://emqx.atlassian.net/browse/EMQX-12383
2024-05-14 10:32:27 -03:00
15acd86b31
Merge pull request #13034 from thalesmg/test-flaky-mq-r57-20240513
...
test(client mgmt api): attempt to fix flaky test
2024-05-14 09:01:29 -03:00
e8d2c48d3c
Merge pull request #13018 from thalesmg/fix-postgres-log-spam-r57-20240510
...
fix(postgres): reduce log spamming when connection goes down
2024-05-14 09:00:36 -03:00
f75c7a5cea
fix(trace log entry error): better structure and naming
2024-05-14 13:38:41 +02:00
74ed0f578a
Merge pull request #13042 from keynslug/fix/EMQX-12204/api-ns
...
fix(s3-aggreg): disambiguate action schema namespace
2024-05-14 13:12:34 +02:00
121084b2ff
fix(kafka trace): do not include query type in trace entry
...
For Kafka that has its internal buffering the on_query_async callback is
used both for sync and async queries (for sync queries the caller waits
for a response directly after calling on_query_async). Therefore, before
this commit, the trace included incorrect query type information. This
has been fixed by removing the query type information from the trace.
This should be okay since the query type is not essential information
and can be derived from the configuration.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12363
2024-05-14 12:24:30 +02:00
525e4dac95
Merge pull request #13036 from ieQu1/dev/reduce-log-spam
...
tests: Reduce log spam in the failed test suites
2024-05-14 10:53:30 +02:00
c10a48f4be
fix(s3-aggreg): disambiguate action schema namespace
...
Otherwise schemas for `s3` and `s3_aggregated_upload` collide when
projected into OpenAPI schema.
2024-05-14 10:49:21 +02:00
93232d4253
fix(authn/http): log meaningful error message if http header is missing
2024-05-14 10:22:07 +02:00
98a93662c1
Merge pull request #13035 from keynslug/fix/EMQX-12291/pers-sys-msg
...
fix(sessds): persist $SYS messages as well
2024-05-14 10:05:49 +02:00
ac3f5a083d
test: Reduce log spam in the failed test suites
2024-05-13 22:00:33 +02:00
8506ca7919
Merge pull request #12998 from ieQu1/dev/improve-latency
...
Use leader's clock when calculating LTS cutoff timestamp
2024-05-13 21:54:06 +02:00
66b7ac4c45
fix(sessds): persist $SYS messages as well
...
Otherwise, persistent sessions will not be able to receive $SYS messages
whatsoever.
2024-05-13 21:27:55 +02:00
a2a5fd1f8e
test(client mgmt api): attempt to fix flaky test
...
```
%%% emqx_mgmt_api_clients_SUITE ==> msgs_plain_encoding.t_mqueue_messages: FAILED
%%% emqx_mgmt_api_clients_SUITE ==>
Failure/Error: ?assertEqual(79, Count)
expected: 79
got: 100
line: 1303
```
2024-05-13 15:53:21 -03:00
3da3a36863
test(ds): Add generation in the replication suite
2024-05-13 19:51:04 +02:00
ieQu1