Commit Graph

12104 Commits

Author SHA1 Message Date
ieQu1 aca2d9586c
fix(ds): Fix return type of drop_generation 2024-05-22 18:01:32 +02:00
ieQu1 c6fc76e335
fix(ds): Perform read operations on the leader. 2024-05-22 18:01:32 +02:00
ieQu1 4580906405
fix(ds): Use erpc instead of gen_rpc for `delete_next' 2024-05-22 18:01:32 +02:00
Andrew Mayorov 8016e9adf4
fix(s3-bridge): restore backward config compatibility 2024-05-22 17:59:52 +02:00
Kjell Winblad 4b540e3bd0 fix: do not leak action configurations in alarm messages 2024-05-22 17:43:07 +02:00
Andrew Mayorov e6c5c1b598
chore(dsrepl): provide more information in rebalancing log messages 2024-05-22 17:24:08 +02:00
Andrew Mayorov c355c9ad50
fix(dsrepl): properly handle transaction abort during forget site 2024-05-22 17:22:55 +02:00
Kjell Winblad cff8b97e8a fix: handle channel updated during health check
This commit fixes an issue found by CI test case
emqx_bridge_influxdb_SUITE:t_start_stop and others. While the channel
health check process is running, the channel could be removed or updated
which could cause a crash in the resource manager or non up-to-date
alarms being triggered.
2024-05-22 12:15:04 +02:00
Kjell Winblad 39d758c4d6 fix: do not return configs for channels from emqx_resource_manager 2024-05-22 12:15:04 +02:00
Kjell Winblad 917474f694 fix: action config update would sometimes not be reflected in connector
Before this commit the following happened sometimes:

1. action status is connected
2. action config is updated to something that should change the status to
   disconnected
3. action status is still connected and the old config is being used by
   the connector even though the config has been correctly updated.

The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).

The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00
firest f3e8037e0f fix(rocketmq): fix namespace error for RocketMQ 2024-05-22 16:29:38 +08:00
Andrew Mayorov a3cd3e31b1
Merge pull request #13084 from keynslug/fix/EMQX-12422/part-size
fix(s3-bridge): report inconsistent `min/max_part_size` option
2024-05-22 09:48:09 +02:00
JimMoen e5f7aa9817
refactor: plguin functions and types rename 2024-05-22 11:51:54 +08:00
JimMoen 33aa61daea
fix: use hocon format as plugin config 2024-05-22 06:37:58 +08:00
JimMoen 14f2a68799
fix: bpapi spec type 2024-05-22 06:01:57 +08:00
Thales Macedo Garitezi 31a35f2a15
Merge pull request #13076 from thalesmg/fix-ds-session-count-client-api-r57-20240520
fix(client mgmt api): cache disconnected durable session count for `/clients` api
2024-05-21 15:34:21 -03:00
JimMoen 5abd23af5a
test: plugin refactor 2024-05-22 02:09:37 +08:00
JimMoen 87b3b214b9
fix: make static_check happy 2024-05-22 02:09:36 +08:00
JimMoen 3ce091e9d7
fix: ensure plugin tarball and extracted 2024-05-22 02:09:35 +08:00
JimMoen 1abc8cf502
fix: ensure plugin's config on boot and join cluster 2024-05-22 02:09:34 +08:00
JimMoen 677352e498
fix: ensure installed and plugin state on boot 2024-05-22 02:09:33 +08:00
JimMoen 2076681e76
fix: put plugin config with binary namevsn key 2024-05-22 02:09:32 +08:00
JimMoen 140b7ce51e
fix(plugin): schema content only provided in enterprise edition 2024-05-22 02:09:31 +08:00
JimMoen a7f2f95318
fix: ensure avro file 2024-05-22 02:09:30 +08:00
JimMoen df7dcb2764
fix: do not let plugin start failed lead emqx start failed 2024-05-22 02:09:29 +08:00
JimMoen e0e4517d9e
fix: ensure plugin config on boot 2024-05-22 02:09:28 +08:00
JimMoen e8d4e88118
fix(plugin): serde not found on new-joined nodes 2024-05-22 02:09:26 +08:00
JimMoen 4ad0743f61
Merge pull request #13081 from JimMoen/fix-typo
chore: fix typos
2024-05-22 02:05:20 +08:00
Andrew Mayorov 28f6eb0200
fix(s3-bridge): report inconsistent `min/max_part_size` option 2024-05-21 18:49:01 +02:00
SergeTupchiy 528d6b7ef1
Merge pull request #13082 from SergeTupchiy/routes-fixes
Minor routes fixes
2024-05-21 19:11:47 +03:00
Serge Tupchii 38e57e321f fix(ds_router): return correct type from lookup_routes/1, fix fun specs 2024-05-21 16:31:31 +03:00
Serge Tupchii 0db6b8dc54 fix(emqx_router): return correct type from lookup_routes/1 (v2 schema) 2024-05-21 16:28:59 +03:00
ieQu1 acb19a06cf Merge remote-tracking branch 'origin/release-57' 2024-05-21 15:04:46 +02:00
Thales Macedo Garitezi 7c5cb1acc5 fix(monitor current api): expose `disconnected_durable_sessions` count
Fixes https://emqx.atlassian.net/browse/EMQX-12423
2024-05-21 10:03:09 -03:00
JimMoen bb3c66638c
chore: fix typos 2024-05-21 17:45:20 +08:00
JianBo He 4f341bef8b
Merge pull request #12920 from cisiqo/patch-6
fix: STOMP heartbeat
2024-05-21 16:57:58 +08:00
Thales Macedo Garitezi 8b4a1c3d75 fix(client mgmt api): cache disconnected durable session count for `/clients` api
Fixes https://emqx.atlassian.net/browse/EMQX-12396
2024-05-20 11:48:08 -03:00
Kjell Winblad 8279d8c787
Merge pull request #13053 from kjellwinblad/kjell/fix_rule_trace_issues/EMQX-12327/EMQX-12335/EMQX-12336
Fix small action trace issues
2024-05-20 16:25:38 +02:00
Thales Macedo Garitezi a5d973b681 docs: hide `/clients_v2` from swagger api docs
Since it's not yet ready for production, we'll hide it from the API docs.
2024-05-20 10:00:43 -03:00
Kjell Winblad 3c87bcde46 fix(rule trace): restore logger metadata to its previous value 2024-05-20 10:57:15 +02:00
zmstone 6940930c3e chore: bump emqx_auth_http app vsn 2024-05-19 09:30:39 +02:00
Zaiming (Stone) Shi c6ee37bacd
Merge pull request #13068 from zmstone/0517-fix-dynamic-license-usage-for-dashboard
fix(license): show dynamic max_connections for bc
2024-05-18 09:49:18 +02:00
Andrew Mayorov fc3f525a67
Merge pull request #13066 from keynslug/fix/s3-bridge/desc-html
chore(s3-bridge): decrease default aggregation settings
2024-05-17 19:43:19 +02:00
zmstone b3afff9638 fix(license): show dynamic max_connections for bc 2024-05-17 17:44:21 +02:00
Thales Macedo Garitezi 388e6c8262 fix(dssubs): introduce separate gauge for subscriptions from durable sessions
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-17 10:17:16 -03:00
Kjell Winblad 5ce095f30e fix(rule trace): only include stop_after_rendering when value is true 2024-05-17 14:53:14 +02:00
Andrew Mayorov 385ac268d7
chore(s3-bridge): decrease default aggreg interval to 30 minutes 2024-05-17 14:49:18 +02:00
Andrew Mayorov c348f759af
chore(s3-bridge): decrease default aggreg limit to 100K records 2024-05-17 14:19:59 +02:00
Kjell Winblad 2e6db85578 fix(rule trace): rename rule_trigger_time(s) and cleaups
This commit renames trace fields rule_trigger_time and
rule_trigger_times to rule_trigger_ts and makes sure that the value for
rule_trigger_ts will always be a list of timestamps.
2024-05-17 14:13:36 +02:00
Andrew Mayorov 4d1db9f847
Merge pull request #13059 from keynslug/feat/EMQX-12204/single-action-schema
feat(s3-bridge): meld 2 separate actions into a single one
2024-05-17 13:27:40 +02:00
Kjell Winblad 2209b26fa5 fix(rule trace): do not leak trace meta data when tracing action result
Fixes:
https://emqx.atlassian.net/browse/EMQX-12391
2024-05-17 10:05:33 +02:00
ieQu1 5ba71ae6f7
Merge pull request #13062 from ieQu1/dev/ds-better-naming
fix(sessds): Rename last_alive_update_interval -> heartbeat_interval
2024-05-17 09:55:40 +02:00
Andrew Mayorov 39c2433afb
fix(s3-bridge): update config typespecs 2024-05-17 09:47:16 +02:00
Andrew Mayorov 86f99959b0
Merge pull request #13054 from keynslug/fix/EMQX-12365/node-leave
fix(dsrepl): anticipate and handle nodes leaving the cluster
2024-05-17 09:43:15 +02:00
Andrew Mayorov 5b2c2b336c
feat(s3-bridge): meld 2 separate actions into a single one
Discriminated through `mode` parameter. Also rename "simple" action to
a "direct" upload mode.
2024-05-17 09:34:21 +02:00
zhongwencool b4eb910cb1 chore: improved InfluxDB connection error messaging 2024-05-17 10:38:11 +08:00
zhongwencool 1ac7aa151e
Merge pull request #13048 from thalesmg/sv-backup-r57-20240514
feat(schema validation): implement backup restore
2024-05-17 09:47:04 +08:00
ieQu1 6f30c1c541
fix(sessds): Rename last_alive_update_interval -> heartbeat_interval 2024-05-17 01:20:10 +02:00
ieQu1 73f17249e9
fix(sessds): Rename session_persistence to durable_sessions 2024-05-16 21:31:32 +02:00
ieQu1 ee6e7174cf
fix(sessds): Rename the durable messages DB to `messages` 2024-05-16 21:31:32 +02:00
Andrew Mayorov 5157e61418
fix(dsrepl): verify if shards already allocated first 2024-05-16 18:56:54 +02:00
Andrew Mayorov 0119728d45
feat(dsrepl): also reflect pending transitions in ds status 2024-05-16 18:56:21 +02:00
Andrew Mayorov 26c4a4f597
feat(dsrepl): reflect conflicts and inconsistencies in ds status 2024-05-16 18:32:08 +02:00
Andrew Mayorov 7e86e3e61c
fix(dsrepl): anticipate and handle nodes leaving the cluster
Also make `claim_site/2` safer by refusing to claim a site for a node
that is already there.
2024-05-16 18:32:07 +02:00
Kjell Winblad ed16e678be fix(rule apply test): add default values to context
Fixes:
https://emqx.atlassian.net/browse/EMQX-12378
2024-05-16 17:31:44 +02:00
Kjell Winblad 520e91c8fd fix(greptime trace): remove async info as it is confusing
Fixes:
https://emqx.atlassian.net/browse/EMQX-12385
2024-05-16 16:50:09 +02:00
Ilia Averianov 22d5f17de8
Merge pull request #13025 from savonarola/0513-fix-cursor-usage
Get rid of qlc in retainer
2024-05-16 17:06:53 +03:00
Kjell Winblad b7c2f4a6d7 fix(best_effor_json): make tuple list check more efficient 2024-05-16 15:40:38 +02:00
Ilya Averyanov 1a664c941b chore(retainer): scan table in batches, improve stream usage
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-05-16 16:06:11 +03:00
Kjell Winblad ba74135079
Merge pull request #13052 from kjellwinblad/kjell/evaluate_lazy_trace_values_in_normal_log_formatters/EMQX-12380
fix: evaluate lazy values in normal log formatters when level is debug
2024-05-16 15:02:53 +02:00
Thales Macedo Garitezi 34a29e6363 feat(schema validation): implement support for `ctl conf load` 2024-05-16 09:55:42 -03:00
Ilia Averianov 322989c83f
Merge pull request #13040 from savonarola/0513-fix-http-authn-error-handling
fix(auth_http): fix query encoding
2024-05-16 15:12:57 +03:00
Zaiming (Stone) Shi 0be1249bbe
Merge pull request #13041 from zmstone/0513-http-authn-header-missing-should-result-in-meaningful-error-message
0513 http authn header missing should result in meaningful error message
2024-05-16 13:17:20 +02:00
ieQu1 71b393be89
Merge pull request #13057 from ieQu1/dev/default-n-shards
fix(ds): Change the default number of shards to 12
2024-05-16 13:05:34 +02:00
Kjell Winblad 413ad60bdb fix(cassandra): format rendered trace in a better way
Fixes:
https://emqx.atlassian.net/browse/EMQX-12393
2024-05-16 11:35:12 +02:00
Kjell Winblad 9fd8e930be fix(best_effort_json): only do tuple list to map without losing pairs 2024-05-16 11:19:48 +02:00
Ilya Averyanov e1ce6377f3 chore(streams): add stream methods, optimize streams 2024-05-16 12:04:08 +03:00
Ilia Averianov 1f7e358e1e
Merge pull request #13006 from savonarola/0509-fix-retainer-banned
fix(banned): verify delayed/taken over/retained messages against clientid_re ban rules
2024-05-16 11:37:15 +03:00
zmstone 2acde5a4e4 fix(authn/http): log meaningful error message if http header is missing 2024-05-16 11:36:52 +03:00
zmstone 4fb484d4cf chore(emqx_schema): change atom array to enum array for alarm.actions 2024-05-16 11:36:52 +03:00
Ilya Averyanov daf2e5a444 chore(auth_http): unify http request generation
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-05-16 11:36:52 +03:00
Kjell Winblad cf5b464777
Merge pull request #12937 from kjellwinblad/kjell/add_use_legacy_protocol_option_to_mongodb_auth/EMQX-12245
test(mongodb authn, authz): add test cases for use_legacy_protocol
2024-05-16 10:10:17 +02:00
Kjell Winblad 72b0c63ed2
Merge pull request #13043 from kjellwinblad/kjell/kafka_incorrect_query_type/EMQX-12363
fix(kafka trace): do not include query type in trace entry
2024-05-16 10:08:16 +02:00
ieQu1 deffb01544
fix(ds): Change the default number of shards to 12 2024-05-16 08:06:07 +02:00
Ilya Averyanov 552b62236c chore(retainer): move filter/foreach to emqx_utils_stream 2024-05-15 19:09:06 +03:00
Ilya Averyanov 0b39aaadbd chore(retainer): get rid of qlc usage 2024-05-15 19:09:06 +03:00
Thales Macedo Garitezi 2eb1bb2b13
Merge pull request #13032 from kjellwinblad/kjell/no_ctx_message_validation_failed/EMQX-12354
fix: add handling of message_validation_failed ctx for rule testing
2024-05-15 11:20:18 -03:00
Kjell Winblad 246bce85ec fix: make mysql action rendered trace include parameters
When doing non-batch inserts with the MySQL action, prepared statements
are used. We therefore need to include the parameters to the prepared
statement when tracing the rendered templates.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12335
2024-05-15 14:49:33 +02:00
Kjell Winblad c72682d81b fix: remove query mode from redis action trace as it only supports sync
Fixes:
https://emqx.atlassian.net/browse/EMQX-12336
2024-05-15 14:32:15 +02:00
Ilya Averyanov 78a87ab5a6 fix(banned): verify delayed/taken over/retained messages against clientid_re rules 2024-05-15 14:05:38 +03:00
Kjell Winblad 8cbfc4c0cd fix: evaluate lazy values in normal log formatters when level is debug
Fixes:
https://emqx.atlassian.net/browse/EMQX-12380
2024-05-15 11:47:21 +02:00
lafirest faf3492e58
Merge pull request #13039 from lafirest/fix/rocket_key
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-15 15:28:35 +08:00
Andrew Mayorov 3e1e4bab17
Merge pull request #13047 from keynslug/fix/EMQX-12366/api-ds-leave
feat(api-ds): provide more information on nonexistent site leave
2024-05-14 23:21:07 +02:00
Thales Macedo Garitezi bcbeadd4af test: stabilize flaky test 2024-05-14 17:53:29 -03:00
Thales Macedo Garitezi f062517ccb test: stabilize flaky test 2024-05-14 17:53:29 -03:00
Thales Macedo Garitezi b0aa7f25aa test: add more context to help debug flaky test
Ex: https://github.com/emqx/emqx/actions/runs/9084293110/job/24967912514?pr=13050#step:6:18330
2024-05-14 17:53:29 -03:00
Thales Macedo Garitezi fd949240c0 feat(schema validation): implement backup restore
Fixes https://emqx.atlassian.net/browse/EMQX-12346
2024-05-14 15:58:19 -03:00
Thales Macedo Garitezi bc5071590b
Merge pull request #13033 from thalesmg/fix-ds-more-client-api-fields-r57-20240513
fix(ds clients mgmt api): add more fields to disconnected durable sessions
2024-05-14 15:29:40 -03:00
Thales Macedo Garitezi 7023e6ad96
Merge pull request #13017 from thalesmg/fix-ds-subs-pages-r57-20240510
fix(subs mgmt api): attempt to return mixed clients ds/non-ds in the same page
2024-05-14 15:29:29 -03:00
Thales Macedo Garitezi 2a0ac34656 Merge branch 'release-57' into sync-r57-m-20240514 2024-05-14 15:03:04 -03:00