Kjell Winblad
9b089a4e5a
fix: use explain instead of msg as error explanation and add test
2024-05-29 10:20:15 +02:00
Kjell Winblad
479889f162
Merge pull request #13136 from kjellwinblad/kjell/fix_oracle_trace_format/EMQX-12433
...
fix(oracle action trace): parameters should not be rendered as IO Data
2024-05-29 09:12:38 +02:00
zhongwencool
7d9333a9d7
chore: bump influxdb app version
2024-05-29 11:13:17 +08:00
zhongwencool
00d74826ec
chore: improved InfluxDB connection error messaging
2024-05-29 11:13:17 +08:00
Thales Macedo Garitezi
802361dbd0
fix(connector api): handle `timeout` when waiting for connector status
...
Fixes https://emqx.atlassian.net/browse/EMQX-12251
2024-05-28 14:19:32 -03:00
Kjell Winblad
82800faadf
fix: make protobuf schema decode error reported to user less cryptic
...
Before this commit the user would see a cryptic warning log including
function_clause when decoding a message failed. This commit improves
this by removing function_caluse as well as some other cryptic words
from the message and adding a description describing what went wrong.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12453
2024-05-28 18:07:14 +02:00
Kjell Winblad
3151a3123e
chore: bump version of emqx_redis_bridge app
2024-05-28 17:23:35 +02:00
Kjell Winblad
3f1fe413ee
refactor: simplify redis trace formatting
2024-05-28 17:22:58 +02:00
Kjell Winblad
cbdde7165e
fix(redis trace): add separators in redis batch action trace
...
Logger will transform data that looks like IO data into a string which
made redis batch traces look like the spaces had been removed
from the strings. To prevent this, we render the batched commands into
binary string separated by spaces and semicolon. The components of a
single command is separated by spaces and the commands in a batch are
separated by semicolons.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12428
2024-05-28 17:22:58 +02:00
Kjell Winblad
5e3de1a4e3
fix(emqx_logger_jsonfmt:best_effort_json_test): test cases
2024-05-28 17:15:32 +02:00
zmstone
40080f5e7d
Merge pull request #13141 from zmstone/0528-test-fix-dashboard-config-test-with-stable-base-config
...
test: fix test case to have deterministic base config
2024-05-28 14:32:55 +02:00
Andrew Mayorov
9ae4de10ed
Merge pull request #13127 from keynslug/fix/EMQX-12439/update-leaner-error
...
fix(bridge-v2): report descriptive error on invalid update request
2024-05-28 13:11:41 +03:00
zmstone
e9cc88cb95
test: fix test case to have deterministic base config
2024-05-28 12:03:01 +02:00
Kjell Winblad
84bc6ba28c
fix: trace formatter should not crash when given a non-binary payload
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12474
2024-05-28 11:13:51 +02:00
Kjell Winblad
e55bda6fe6
chore: bump app versions
...
Bump app version as required to pass the check which is done by the
following script:
./scripts/apps-version-check.sh
2024-05-28 11:01:00 +02:00
Kjell Winblad
f65168982b
fix(oracle action trace): parameters should not be rendered as IO Data
...
This forces the parameters to the database statement to be rendered as
a JSON array in JSON traces instead of being rendered as a string when
the parameters are interpreted as IO data.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12433
2024-05-28 10:59:44 +02:00
Andrew Mayorov
c04aaad0a4
chore(bridge-v2): leave comment describing need for `nowarn_function`
2024-05-28 10:31:14 +02:00
ieQu1
8fbeca4321
chore: Version bumps
2024-05-28 00:14:01 +02:00
ieQu1
04305d004f
refactor(sessds): Extract metadata keys to a header
2024-05-27 23:36:20 +02:00
ieQu1
16d1425982
refactor(emqx): Move files related to durable session to a subdir
2024-05-27 23:31:15 +02:00
zmstone
2771a10d39
test: fix a flaky one
2024-05-27 20:59:50 +02:00
Andrew Mayorov
663f4fd39f
chore: bump application versions
2024-05-27 17:49:16 +02:00
Andrew Mayorov
08d88ea814
feat(bridge-api): improve error messages for Update Source API
2024-05-27 17:49:16 +02:00
Andrew Mayorov
ed7c29ec26
fix(api): mention spec override in "request metadata" type
2024-05-27 16:57:45 +02:00
Andrew Mayorov
5c2a68076f
fix(bridge-v2): report descriptive error on invalid update request
...
Before this commit, generic validation errors were reported as union
mismatches of _all_ of the bridge schemas. After this commit, specific
schema is chosen before validation.
2024-05-27 14:37:10 +02:00
Ivan Dyachkov
f393fd5304
chore: release 5.7.0
2024-05-27 14:32:23 +02:00
zmstone
b2f5e50f16
Merge pull request #13120 from ieQu1/dev/ds-api-site-status
...
fix(dsapi): Check status of the node via Mria
2024-05-25 05:42:05 +02:00
zmstone
0763e29786
chore: prepare for 5.7.0-rc.2 release
2024-05-24 21:10:37 +02:00
ieQu1
73df01e934
fix(dsapi): Check status of the node via Mria
2024-05-24 20:59:07 +02:00
ieQu1
7f356aa3a8
Merge pull request #13115 from ieQu1/dev/fix-drop-generations2
...
Mitigate transient errors when dropping generations
2024-05-24 20:13:39 +02:00
ieQu1
830b62d899
fix(dsrepl): Retry sending ra commands to the leader
2024-05-24 19:09:34 +02:00
ieQu1
1ffc7d5d9e
fix(dsrepl): Treat all exceptions from storage layer as recoverable
2024-05-24 19:09:34 +02:00
ieQu1
25721bceba
fix(dsstor): Delete generation metadata before dropping it
2024-05-24 19:09:33 +02:00
ieQu1
3d9837a0b8
test(ds): Add emqx_ds_replication_SUITE:t_drop_generation
2024-05-24 19:09:33 +02:00
Ilya Averyanov
327080e70e
perf(emqx_placeholder): do not try to decode maps as json
...
fix
2024-05-24 20:08:31 +03:00
Andrew Mayorov
c30b881239
test(monitor): verify historical and current metrics are consistent
2024-05-24 14:56:10 +02:00
Andrew Mayorov
fd9655bc35
fix(monitor): unify metric names for consistency and simplicity
2024-05-24 14:55:48 +02:00
Andrew Mayorov
7b137c0aaf
fix(monitor): reflect durable subscriptions in historical metrics
...
Essentially in the same manner as they are currently reflected in the
current metrics.
2024-05-24 14:08:35 +02:00
zmstone
6ec8ae5a5f
Merge pull request #13112 from HJianBo/change-the-default-batch-size-s3-uploader
...
Fix(s3): change the batch_size to 1 for direct mode
2024-05-24 10:56:53 +02:00
Andrew Mayorov
29d203ae33
Merge pull request #13108 from keynslug/fix/ct-flaky/mgmt
...
test(mgmt): avoid inter-suite contamination by switching to cth tooling
2024-05-24 11:14:16 +03:00
JianBo He
adaf238131
fix(s3): fix the convert function if the mode is missing
2024-05-24 16:02:06 +08:00
JimMoen
5b60f6a5b2
fix: make dialyzer happy
2024-05-24 14:29:02 +08:00
JimMoen
43506495ae
fix: ensure plugin configured states after join cluster
2024-05-24 13:51:56 +08:00
JimMoen
874bcd9a8b
fix: match decode plugin config map failed case
2024-05-24 10:20:50 +08:00
zmstone
c3c6da7ec4
chore: prepare for release cut 5.7.0-rc.1
2024-05-23 21:39:15 +02:00
Zaiming (Stone) Shi
3a4b8e6c24
Merge pull request #13106 from zmstone/0523-add-customer-type
...
0523 add customer type (4=BYOC) to license schema
2024-05-23 21:30:02 +02:00
Zaiming (Stone) Shi
40288b99b2
Merge pull request #13101 from JimMoen/fix-plugin-config-map
...
fix: allow put plugin config without schema
2024-05-23 21:18:43 +02:00
Thales Macedo Garitezi
823a6bc521
Merge pull request #13107 from thalesmg/hide-clientv2-response-type-r57-20240523
...
fix(client mgmt api): hide unused return type
2024-05-23 16:03:51 -03:00
Andrew Mayorov
7878d106f6
test(mgmt): avoid inter-suite contamination by switching to cth tooling
2024-05-23 20:17:34 +02:00
Andrew Mayorov
b4c6968f8c
Merge pull request #13104 from keynslug/fix/EMQX-12415/last-resort-poll
...
fix(dsrepl): trigger "last-resort" pending transitions handler when idle
2024-05-23 21:17:03 +03:00
ieQu1
0dc507441a
Merge pull request #13105 from ieQu1/dev/sessds-renew-streams-interval
...
fix(sessds): Change the defaults to renew streams every 1 second
2024-05-23 18:23:20 +02:00
Thales Macedo Garitezi
b1e3c941fb
fix(client mgmt api): hide unused return type
...
Fixes https://emqx.atlassian.net/browse/EMQX-12427
2024-05-23 13:17:43 -03:00
Andrew Mayorov
d47bf8076a
test(mgmt): better isolate testsuite running environments
2024-05-23 17:53:13 +02:00
zmstone
c737045dff
chore: add customer type 4 (BYOC)
2024-05-23 17:30:06 +02:00
ieQu1
5584b658c9
fix(sessds): Change the defaults to renew streams every 1 second
2024-05-23 17:17:29 +02:00
Zaiming (Stone) Shi
1d99ba2a8a
Merge pull request #13102 from JimMoen/fix-ds-prom-badkey
...
fix: add ds metrics into prometheus zipper init acc
2024-05-23 16:57:06 +02:00
Thales Macedo Garitezi
118c1d2806
Merge pull request #13103 from thalesmg/fix-another-ds-counter-nit-r57-20240523
...
fix(monitor api): fix cluster metric aggregation
2024-05-23 10:57:53 -03:00
Andrew Mayorov
ba6382adae
fix(dsrepl): trigger "last-resort" pending transitions handler when idle
...
This is a hack to work around the unintended issues causing shard
allocator to become idle even when there are pending transitions.
2024-05-23 14:54:01 +02:00
Andrew Mayorov
839b143fc4
fix(sessds): consider message durability disabled by default
2024-05-23 14:48:39 +02:00
Thales Macedo Garitezi
c0e3a81c61
fix(monitor api): fix cluster metric aggregation
...
Fixes https://emqx.atlassian.net/browse/EMQX-12423
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-23 09:38:24 -03:00
Andrew Mayorov
bf326acd7b
fix(dsrepl): handle stopping non-yet-running shard supervisor
2024-05-23 14:34:27 +02:00
Kjell Winblad
0e5b02c373
Merge pull request #13100 from kjellwinblad/kjell/s3_aggregate_only_has_result_trace/EMQX-12429
...
fix: add action rendered trace for s3 in aggregated mode
2024-05-23 14:29:18 +02:00
JimMoen
a40c7d646a
fix: apply suggestions from code review, thanks @zmstone
2024-05-23 20:28:14 +08:00
JimMoen
b91ecac168
refactor(prom): unify function name to avoid multiple levels call
2024-05-23 20:11:22 +08:00
JimMoen
065411c69c
fix: add ds metrics into prometheus zipper init acc
2024-05-23 20:11:21 +08:00
Kjell Winblad
89b47e8ffc
fix(s3 tracing): do not format records as IO data in aggregate mode
2024-05-23 12:41:30 +02:00
Kjell Winblad
fb7688ab94
fix(trace): make sure that the payload encode works with nested payloads
...
This commit makes sure that the trace setting for payload encode works
even when the payload is in a nested structure or when the payload key
is a binary instead of an atom.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12424
2024-05-23 12:30:56 +02:00
JimMoen
5dc96a37e2
fix: ensure plugin avsc serde before config
2024-05-23 17:46:58 +08:00
JimMoen
25a9aa1797
fix: allow put plugin config without schema
2024-05-23 17:33:34 +08:00
JimMoen
99869821fb
fix: ensure plugin config map after fresh install
2024-05-23 17:16:16 +08:00
Kjell Winblad
e10c87b825
fix: add action rendered trace for s3 in aggregated mode
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12429
2024-05-23 10:25:27 +02:00
Andrew Mayorov
398dc97ed6
Merge pull request #13092 from keynslug/fix/dsrepl/site-autoleave
...
fix(dsrepl): properly handle transaction abort during forget site
2024-05-23 10:56:23 +03:00
ieQu1
950fa82be3
Merge pull request #13097 from ieQu1/dev/ds-replay-unrecoverable-no-test
...
feat(sessds): Handle unrecoverable errors
2024-05-23 09:49:21 +02:00
Kjell Winblad
26c988fe11
Merge pull request #13077 from kjellwinblad/kjell/fix_action_update_race_issue/EMQX-12376
...
fix: action config update would sometimes not be reflected in connector
2024-05-23 09:11:48 +02:00
Andrew Mayorov
2f6729d203
Merge pull request #13089 from keynslug/fix/EMQX-12437/restore
...
fix(s3-bridge): restore backward config compatibility
2024-05-23 10:10:01 +03:00
JianBo He
4053356597
Merge pull request #13063 from JimMoen/improve-plugin-config-behavior
...
fix(plugin): serde not found on new-joined nodes
2024-05-23 11:24:44 +08:00
lafirest
b1076221b0
Merge pull request #13085 from lafirest/fix/rocket_namespace
...
fix(rocketmq): fix namespace error for RocketMQ
2024-05-23 10:39:30 +08:00
ieQu1
07774ab060
feat(sessds): Handle unrecoverable errors
2024-05-22 22:45:37 +02:00
Thales Macedo Garitezi
63cb12d7c2
Merge pull request #13095 from thalesmg/fix-monitor-ds-counters-again-r57-20240522
...
fix(monitor api): add ds connections and subscriptions to old counters
2024-05-22 16:43:15 -03:00
ieQu1
bfd01c425d
Merge pull request #13072 from ieQu1/dev/fix-drop-generation
...
Idempotent drop_generation
2024-05-22 21:13:30 +02:00
ieQu1
6eb04f90a3
fix(ds): Allow to write batches to older generations
2024-05-22 20:28:16 +02:00
Thales Macedo Garitezi
5cad4497de
fix(monitor api): add ds connections and subscriptions to old counters
...
Fixes https://emqx.atlassian.net/browse/EMQX-12423
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-22 15:19:46 -03:00
Thales Macedo Garitezi
4094032649
fix(bridge v2 api): don't attempt to start disabled connector when starting action/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-12435
2024-05-22 13:33:55 -03:00
Kjell Winblad
88c96e26de
refactor: simplify the code with maps:map/2
...
Thanks @thalesmg for the suggestion
2024-05-22 18:03:43 +02:00
ieQu1
59a09fb86f
fix(ds): Apply review remarks
2024-05-22 18:01:33 +02:00
ieQu1
5c78ecba40
docs(ds): Update documentation for the storage layouts
2024-05-22 18:01:33 +02:00
ieQu1
29345aaa30
fix(ds): Fix idle event generation in bitfield_lts layout
2024-05-22 18:01:33 +02:00
ieQu1
b3ded7edce
fix(ds): Fix code review remark
2024-05-22 18:01:33 +02:00
ieQu1
60edf5e9b8
fix(ds): Move responsibility of returning end_of_stream to the CBM
2024-05-22 18:01:33 +02:00
ieQu1
0ff307e789
fix(ds): Include generation ID in the storage events
...
Make sure storage events originating from generation X are handled in
the context of the same generation.
2024-05-22 18:01:33 +02:00
ieQu1
acdae4fba3
fix(ds): Workaround for the idempotency error when dropping gens
2024-05-22 18:01:33 +02:00
ieQu1
eb7c43ee9d
fix(ds): Always store messages in the current generation
2024-05-22 18:01:33 +02:00
ieQu1
074d98a14a
test(ds): Refactor ds_SUITE
2024-05-22 18:01:33 +02:00
ieQu1
e4a73f003a
feat(ds): Implement format_status callback
...
Reduce volume of logs and crash reports from DS
2024-05-22 18:01:32 +02:00
ieQu1
1526c527d0
fix(ds): Log generation operations
2024-05-22 18:01:32 +02:00
ieQu1
aca2d9586c
fix(ds): Fix return type of drop_generation
2024-05-22 18:01:32 +02:00
ieQu1
c6fc76e335
fix(ds): Perform read operations on the leader.
2024-05-22 18:01:32 +02:00
ieQu1
4580906405
fix(ds): Use erpc instead of gen_rpc for `delete_next'
2024-05-22 18:01:32 +02:00
Andrew Mayorov
8016e9adf4
fix(s3-bridge): restore backward config compatibility
2024-05-22 17:59:52 +02:00
Kjell Winblad
4b540e3bd0
fix: do not leak action configurations in alarm messages
2024-05-22 17:43:07 +02:00
Andrew Mayorov
e6c5c1b598
chore(dsrepl): provide more information in rebalancing log messages
2024-05-22 17:24:08 +02:00
Andrew Mayorov
c355c9ad50
fix(dsrepl): properly handle transaction abort during forget site
2024-05-22 17:22:55 +02:00
Kjell Winblad
cff8b97e8a
fix: handle channel updated during health check
...
This commit fixes an issue found by CI test case
emqx_bridge_influxdb_SUITE:t_start_stop and others. While the channel
health check process is running, the channel could be removed or updated
which could cause a crash in the resource manager or non up-to-date
alarms being triggered.
2024-05-22 12:15:04 +02:00
Kjell Winblad
39d758c4d6
fix: do not return configs for channels from emqx_resource_manager
2024-05-22 12:15:04 +02:00
Kjell Winblad
917474f694
fix: action config update would sometimes not be reflected in connector
...
Before this commit the following happened sometimes:
1. action status is connected
2. action config is updated to something that should change the status to
disconnected
3. action status is still connected and the old config is being used by
the connector even though the config has been correctly updated.
The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).
The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00
firest
f3e8037e0f
fix(rocketmq): fix namespace error for RocketMQ
2024-05-22 16:29:38 +08:00
Andrew Mayorov
a3cd3e31b1
Merge pull request #13084 from keynslug/fix/EMQX-12422/part-size
...
fix(s3-bridge): report inconsistent `min/max_part_size` option
2024-05-22 09:48:09 +02:00
JimMoen
e5f7aa9817
refactor: plguin functions and types rename
2024-05-22 11:51:54 +08:00
JimMoen
33aa61daea
fix: use hocon format as plugin config
2024-05-22 06:37:58 +08:00
JimMoen
14f2a68799
fix: bpapi spec type
2024-05-22 06:01:57 +08:00
Thales Macedo Garitezi
31a35f2a15
Merge pull request #13076 from thalesmg/fix-ds-session-count-client-api-r57-20240520
...
fix(client mgmt api): cache disconnected durable session count for `/clients` api
2024-05-21 15:34:21 -03:00
JimMoen
5abd23af5a
test: plugin refactor
2024-05-22 02:09:37 +08:00
JimMoen
87b3b214b9
fix: make static_check happy
2024-05-22 02:09:36 +08:00
JimMoen
3ce091e9d7
fix: ensure plugin tarball and extracted
2024-05-22 02:09:35 +08:00
JimMoen
1abc8cf502
fix: ensure plugin's config on boot and join cluster
2024-05-22 02:09:34 +08:00
JimMoen
677352e498
fix: ensure installed and plugin state on boot
2024-05-22 02:09:33 +08:00
JimMoen
2076681e76
fix: put plugin config with binary namevsn key
2024-05-22 02:09:32 +08:00
JimMoen
140b7ce51e
fix(plugin): schema content only provided in enterprise edition
2024-05-22 02:09:31 +08:00
JimMoen
a7f2f95318
fix: ensure avro file
2024-05-22 02:09:30 +08:00
JimMoen
df7dcb2764
fix: do not let plugin start failed lead emqx start failed
2024-05-22 02:09:29 +08:00
JimMoen
e0e4517d9e
fix: ensure plugin config on boot
2024-05-22 02:09:28 +08:00
JimMoen
e8d4e88118
fix(plugin): serde not found on new-joined nodes
2024-05-22 02:09:26 +08:00
Andrew Mayorov
28f6eb0200
fix(s3-bridge): report inconsistent `min/max_part_size` option
2024-05-21 18:49:01 +02:00
Thales Macedo Garitezi
7c5cb1acc5
fix(monitor current api): expose `disconnected_durable_sessions` count
...
Fixes https://emqx.atlassian.net/browse/EMQX-12423
2024-05-21 10:03:09 -03:00
Thales Macedo Garitezi
8b4a1c3d75
fix(client mgmt api): cache disconnected durable session count for `/clients` api
...
Fixes https://emqx.atlassian.net/browse/EMQX-12396
2024-05-20 11:48:08 -03:00
Kjell Winblad
8279d8c787
Merge pull request #13053 from kjellwinblad/kjell/fix_rule_trace_issues/EMQX-12327/EMQX-12335/EMQX-12336
...
Fix small action trace issues
2024-05-20 16:25:38 +02:00
Thales Macedo Garitezi
a5d973b681
docs: hide `/clients_v2` from swagger api docs
...
Since it's not yet ready for production, we'll hide it from the API docs.
2024-05-20 10:00:43 -03:00
Kjell Winblad
3c87bcde46
fix(rule trace): restore logger metadata to its previous value
2024-05-20 10:57:15 +02:00
zmstone
6940930c3e
chore: bump emqx_auth_http app vsn
2024-05-19 09:30:39 +02:00
zmstone
331d44a78a
chore: improve Kafka producer health-check error logs
...
The Kafka producer lib wolff returns detailed error logs
about which host:port had issue, EMQX should log them to help
troubleshooting
2024-05-18 19:12:51 +02:00
Zaiming (Stone) Shi
c6ee37bacd
Merge pull request #13068 from zmstone/0517-fix-dynamic-license-usage-for-dashboard
...
fix(license): show dynamic max_connections for bc
2024-05-18 09:49:18 +02:00
Andrew Mayorov
fc3f525a67
Merge pull request #13066 from keynslug/fix/s3-bridge/desc-html
...
chore(s3-bridge): decrease default aggregation settings
2024-05-17 19:43:19 +02:00
zmstone
b3afff9638
fix(license): show dynamic max_connections for bc
2024-05-17 17:44:21 +02:00
Thales Macedo Garitezi
388e6c8262
fix(dssubs): introduce separate gauge for subscriptions from durable sessions
...
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-17 10:17:16 -03:00
Kjell Winblad
5ce095f30e
fix(rule trace): only include stop_after_rendering when value is true
2024-05-17 14:53:14 +02:00
Andrew Mayorov
385ac268d7
chore(s3-bridge): decrease default aggreg interval to 30 minutes
2024-05-17 14:49:18 +02:00
Andrew Mayorov
c348f759af
chore(s3-bridge): decrease default aggreg limit to 100K records
2024-05-17 14:19:59 +02:00
Kjell Winblad
2e6db85578
fix(rule trace): rename rule_trigger_time(s) and cleaups
...
This commit renames trace fields rule_trigger_time and
rule_trigger_times to rule_trigger_ts and makes sure that the value for
rule_trigger_ts will always be a list of timestamps.
2024-05-17 14:13:36 +02:00
Andrew Mayorov
4d1db9f847
Merge pull request #13059 from keynslug/feat/EMQX-12204/single-action-schema
...
feat(s3-bridge): meld 2 separate actions into a single one
2024-05-17 13:27:40 +02:00
Kjell Winblad
2209b26fa5
fix(rule trace): do not leak trace meta data when tracing action result
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12391
2024-05-17 10:05:33 +02:00
ieQu1
5ba71ae6f7
Merge pull request #13062 from ieQu1/dev/ds-better-naming
...
fix(sessds): Rename last_alive_update_interval -> heartbeat_interval
2024-05-17 09:55:40 +02:00
Andrew Mayorov
39c2433afb
fix(s3-bridge): update config typespecs
2024-05-17 09:47:16 +02:00
Andrew Mayorov
86f99959b0
Merge pull request #13054 from keynslug/fix/EMQX-12365/node-leave
...
fix(dsrepl): anticipate and handle nodes leaving the cluster
2024-05-17 09:43:15 +02:00
Andrew Mayorov
5b2c2b336c
feat(s3-bridge): meld 2 separate actions into a single one
...
Discriminated through `mode` parameter. Also rename "simple" action to
a "direct" upload mode.
2024-05-17 09:34:21 +02:00
zhongwencool
1ac7aa151e
Merge pull request #13048 from thalesmg/sv-backup-r57-20240514
...
feat(schema validation): implement backup restore
2024-05-17 09:47:04 +08:00
ieQu1
6f30c1c541
fix(sessds): Rename last_alive_update_interval -> heartbeat_interval
2024-05-17 01:20:10 +02:00
ieQu1
73f17249e9
fix(sessds): Rename session_persistence to durable_sessions
2024-05-16 21:31:32 +02:00
ieQu1
ee6e7174cf
fix(sessds): Rename the durable messages DB to `messages`
2024-05-16 21:31:32 +02:00
Andrew Mayorov
5157e61418
fix(dsrepl): verify if shards already allocated first
2024-05-16 18:56:54 +02:00
Andrew Mayorov
0119728d45
feat(dsrepl): also reflect pending transitions in ds status
2024-05-16 18:56:21 +02:00
Andrew Mayorov
26c4a4f597
feat(dsrepl): reflect conflicts and inconsistencies in ds status
2024-05-16 18:32:08 +02:00
Andrew Mayorov
7e86e3e61c
fix(dsrepl): anticipate and handle nodes leaving the cluster
...
Also make `claim_site/2` safer by refusing to claim a site for a node
that is already there.
2024-05-16 18:32:07 +02:00
Kjell Winblad
ed16e678be
fix(rule apply test): add default values to context
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12378
2024-05-16 17:31:44 +02:00
Kjell Winblad
520e91c8fd
fix(greptime trace): remove async info as it is confusing
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12385
2024-05-16 16:50:09 +02:00
Kjell Winblad
b7c2f4a6d7
fix(best_effor_json): make tuple list check more efficient
2024-05-16 15:40:38 +02:00
Kjell Winblad
ba74135079
Merge pull request #13052 from kjellwinblad/kjell/evaluate_lazy_trace_values_in_normal_log_formatters/EMQX-12380
...
fix: evaluate lazy values in normal log formatters when level is debug
2024-05-16 15:02:53 +02:00
Thales Macedo Garitezi
34a29e6363
feat(schema validation): implement support for `ctl conf load`
2024-05-16 09:55:42 -03:00
Zaiming (Stone) Shi
0be1249bbe
Merge pull request #13041 from zmstone/0513-http-authn-header-missing-should-result-in-meaningful-error-message
...
0513 http authn header missing should result in meaningful error message
2024-05-16 13:17:20 +02:00
ieQu1
71b393be89
Merge pull request #13057 from ieQu1/dev/default-n-shards
...
fix(ds): Change the default number of shards to 12
2024-05-16 13:05:34 +02:00
Kjell Winblad
413ad60bdb
fix(cassandra): format rendered trace in a better way
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12393
2024-05-16 11:35:12 +02:00
Kjell Winblad
9fd8e930be
fix(best_effort_json): only do tuple list to map without losing pairs
2024-05-16 11:19:48 +02:00
Kjell Winblad
72b0c63ed2
Merge pull request #13043 from kjellwinblad/kjell/kafka_incorrect_query_type/EMQX-12363
...
fix(kafka trace): do not include query type in trace entry
2024-05-16 10:08:16 +02:00
ieQu1
deffb01544
fix(ds): Change the default number of shards to 12
2024-05-16 08:06:07 +02:00
Thales Macedo Garitezi
2eb1bb2b13
Merge pull request #13032 from kjellwinblad/kjell/no_ctx_message_validation_failed/EMQX-12354
...
fix: add handling of message_validation_failed ctx for rule testing
2024-05-15 11:20:18 -03:00
Kjell Winblad
246bce85ec
fix: make mysql action rendered trace include parameters
...
When doing non-batch inserts with the MySQL action, prepared statements
are used. We therefore need to include the parameters to the prepared
statement when tracing the rendered templates.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12335
2024-05-15 14:49:33 +02:00
Kjell Winblad
c72682d81b
fix: remove query mode from redis action trace as it only supports sync
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12336
2024-05-15 14:32:15 +02:00
Kjell Winblad
8cbfc4c0cd
fix: evaluate lazy values in normal log formatters when level is debug
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12380
2024-05-15 11:47:21 +02:00
lafirest
faf3492e58
Merge pull request #13039 from lafirest/fix/rocket_key
...
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-15 15:28:35 +08:00
Andrew Mayorov
3e1e4bab17
Merge pull request #13047 from keynslug/fix/EMQX-12366/api-ds-leave
...
feat(api-ds): provide more information on nonexistent site leave
2024-05-14 23:21:07 +02:00
Thales Macedo Garitezi
fd949240c0
feat(schema validation): implement backup restore
...
Fixes https://emqx.atlassian.net/browse/EMQX-12346
2024-05-14 15:58:19 -03:00
Thales Macedo Garitezi
bc5071590b
Merge pull request #13033 from thalesmg/fix-ds-more-client-api-fields-r57-20240513
...
fix(ds clients mgmt api): add more fields to disconnected durable sessions
2024-05-14 15:29:40 -03:00
Thales Macedo Garitezi
7023e6ad96
Merge pull request #13017 from thalesmg/fix-ds-subs-pages-r57-20240510
...
fix(subs mgmt api): attempt to return mixed clients ds/non-ds in the same page
2024-05-14 15:29:29 -03:00
Kjell Winblad
e811f4a392
fix: add handling of message_validation_failed ctx for rule testing
...
Before this commit, the backend could not handle the test context for
message_validation_failed events that could be sent from the dashboard
since no handling code for this contexts existed in the backend. This
has now been fixed by adding such handling code.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12354
2024-05-14 13:07:32 -03:00
Thales Macedo Garitezi
1730a41337
Merge pull request #13046 from thalesmg/refactor-mv-rename-sv-r57-20240514
...
refactor: rename `message_validation` to `schema_validation`
2024-05-14 13:03:12 -03:00
Kjell Winblad
adbc392265
Merge pull request #13027 from kjellwinblad/kjell/fix_trace_json_report_formatting/EMQX-12344
...
fix: JSON trace formatter should handle report style log entries
2024-05-14 18:02:59 +02:00
firest
437a0c0d55
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-14 23:01:31 +08:00
Andrew Mayorov
2f90d15ce6
fix(ds-api): adapt CLI to API changes
2024-05-14 16:55:17 +02:00
Andrew Mayorov
35e360fcbe
feat(api-ds): provide more information on nonexistent site leave
2024-05-14 15:57:41 +02:00
Thales Macedo Garitezi
257dae5211
refactor: rename `message_validation` to `schema_validation`
...
Request from Product team.
Fixes https://emqx.atlassian.net/browse/EMQX-12383
2024-05-14 10:32:27 -03:00
Thales Macedo Garitezi
15acd86b31
Merge pull request #13034 from thalesmg/test-flaky-mq-r57-20240513
...
test(client mgmt api): attempt to fix flaky test
2024-05-14 09:01:29 -03:00
Thales Macedo Garitezi
e8d2c48d3c
Merge pull request #13018 from thalesmg/fix-postgres-log-spam-r57-20240510
...
fix(postgres): reduce log spamming when connection goes down
2024-05-14 09:00:36 -03:00
Kjell Winblad
f75c7a5cea
fix(trace log entry error): better structure and naming
2024-05-14 13:38:41 +02:00
Andrew Mayorov
74ed0f578a
Merge pull request #13042 from keynslug/fix/EMQX-12204/api-ns
...
fix(s3-aggreg): disambiguate action schema namespace
2024-05-14 13:12:34 +02:00
Kjell Winblad
121084b2ff
fix(kafka trace): do not include query type in trace entry
...
For Kafka that has its internal buffering the on_query_async callback is
used both for sync and async queries (for sync queries the caller waits
for a response directly after calling on_query_async). Therefore, before
this commit, the trace included incorrect query type information. This
has been fixed by removing the query type information from the trace.
This should be okay since the query type is not essential information
and can be derived from the configuration.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12363
2024-05-14 12:24:30 +02:00
ieQu1
525e4dac95
Merge pull request #13036 from ieQu1/dev/reduce-log-spam
...
tests: Reduce log spam in the failed test suites
2024-05-14 10:53:30 +02:00
Andrew Mayorov
c10a48f4be
fix(s3-aggreg): disambiguate action schema namespace
...
Otherwise schemas for `s3` and `s3_aggregated_upload` collide when
projected into OpenAPI schema.
2024-05-14 10:49:21 +02:00
zmstone
93232d4253
fix(authn/http): log meaningful error message if http header is missing
2024-05-14 10:22:07 +02:00
Andrew Mayorov
98a93662c1
Merge pull request #13035 from keynslug/fix/EMQX-12291/pers-sys-msg
...
fix(sessds): persist $SYS messages as well
2024-05-14 10:05:49 +02:00
ieQu1
ac3f5a083d
test: Reduce log spam in the failed test suites
2024-05-13 22:00:33 +02:00
ieQu1
8506ca7919
Merge pull request #12998 from ieQu1/dev/improve-latency
...
Use leader's clock when calculating LTS cutoff timestamp
2024-05-13 21:54:06 +02:00
Andrew Mayorov
66b7ac4c45
fix(sessds): persist $SYS messages as well
...
Otherwise, persistent sessions will not be able to receive $SYS messages
whatsoever.
2024-05-13 21:27:55 +02:00
Thales Macedo Garitezi
a2a5fd1f8e
test(client mgmt api): attempt to fix flaky test
...
```
%%% emqx_mgmt_api_clients_SUITE ==> msgs_plain_encoding.t_mqueue_messages: FAILED
%%% emqx_mgmt_api_clients_SUITE ==>
Failure/Error: ?assertEqual(79, Count)
expected: 79
got: 100
line: 1303
```
2024-05-13 15:53:21 -03:00
ieQu1
3da3a36863
test(ds): Add generation in the replication suite
2024-05-13 19:51:04 +02:00
ieQu1
9f7ef9f34f
fix(ds): Apply review remarks
2024-05-13 19:35:24 +02:00
Thales Macedo Garitezi
e7c47f4ec0
fix(ds clients mgmt api): add more fields to disconnected durable sessions
...
Fixes https://emqx.atlassian.net/browse/EMQX-12369
2024-05-13 14:25:02 -03:00
Kjell Winblad
1c34a84dd2
Merge pull request #13028 from kjellwinblad/kjell/swagger_example_missing_ruleid_paramter/EMQX-12325
...
fix: include ruleid parameter in swagger example for apply rule endpoint
2024-05-13 17:35:31 +02:00
Thales Macedo Garitezi
81473519bd
Merge pull request #13031 from thalesmg/fix-clientsv2-sec-r57-20240513
...
fix(client mgmt api): require authn for endpoint
2024-05-13 12:17:31 -03:00
Thales Macedo Garitezi
39899b917d
fix(client mgmt api): require authn for endpoint
...
Fixes https://emqx.atlassian.net/browse/EMQX-12361
2024-05-13 10:54:21 -03:00
Thales Macedo Garitezi
6b032faebc
fix(client mgmt api): return expired durable sessions with `is_expired: true`
...
Fixes https://emqx.atlassian.net/browse/EMQX-12274
2024-05-13 10:51:24 -03:00
Thales Macedo Garitezi
0dee2d67b1
fix: always check if first query has more rows
2024-05-13 09:10:18 -03:00
Kjell Winblad
c8690a8273
fix: include ruleid parameter in swagger example for apply rule endpoint
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12325
2024-05-13 12:28:17 +02:00
Kjell Winblad
921d82f083
fix: JSON trace formatter should handle report style log entries
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12344
2024-05-13 12:11:11 +02:00
zmstone
7faf73e0f4
chore(emqx_schema): change atom array to enum array for alarm.actions
2024-05-13 10:54:33 +02:00
JianBo He
54f6bd181b
Merge pull request #13010 from JimMoen/fix-jt808-reg-failed
...
fix(gw): jt808 REG_ACK failed due to faulty clientinfo
2024-05-13 14:19:18 +08:00
JimMoen
2f212f34a3
Merge pull request #12989 from JimMoen/fix-plugin-config-file-backup
...
fix(plugin): add a backup for the plugin config file
2024-05-11 16:38:36 +08:00
JimMoen
a5ecceb755
chore: apply suggestions from code review, thanks @thalesmg
2024-05-11 11:36:40 +08:00
Thales Macedo Garitezi
e25fe62cbc
fix(subs mgmt api): attempt to return mixed clients ds/non-ds in the same page
...
Fixes https://emqx.atlassian.net/browse/EMQX-12294
2024-05-10 14:48:46 -03:00
Kjell Winblad
c6df069c5a
Merge pull request #13012 from kjellwinblad/kjell/fix_incorrect_config_crash/EMQX-12315
...
fix: listener crash if access_rules config option is incorrect
2024-05-10 17:46:40 +02:00
Thales Macedo Garitezi
f690db9849
fix(postgres): reduce log spamming when connection goes down
...
Fixes https://emqx.atlassian.net/browse/EMQX-12334
See also: https://github.com/emqx/epgsql/pull/10
2024-05-10 12:15:01 -03:00
JimMoen
3c7e8a3bd5
test(gw): jt808 register failed
2024-05-10 23:09:22 +08:00
JimMoen
f8fd4d15be
fix(gw): process jt808 client deregister packet
2024-05-10 23:00:59 +08:00
Zaiming (Stone) Shi
c491b83857
Merge pull request #13014 from zmstone/0510-add-is_template-for-kinesis-partition-key
...
fix(kinesis): partition_key should be of 'template' type
2024-05-10 16:11:35 +02:00
Kjell Winblad
b13fa37771
fix: dialyzer warning
2024-05-10 15:51:28 +02:00
Thales Macedo Garitezi
0eaef18391
Merge pull request #13005 from thalesmg/fix-aggreg-sup-tree-r57-20240509
...
fix(aggregator): refactor supervision tree
2024-05-10 10:22:33 -03:00
Kjell Winblad
2bed5894e3
test: add test cases for listeners access_rules validation and split
2024-05-10 14:21:43 +02:00
Thales Macedo Garitezi
3a29696a48
fix(aggregator): refactor supervision tree
...
Instead of using a aggregator supervisor with a fixed local name, we should allow
specifying different names so each action app will spawn its own aggregator supervisor.
2024-05-10 09:06:12 -03:00
zmstone
47df968d4a
fix(kinesis): partition_key should be of 'template' type
2024-05-10 13:57:38 +02:00
Kjell Winblad
0aeb2cd77f
fix: listener crash if access_rules config option is incorrect
...
Previously when changing the access_rules configuration option of a
listener to something that was not a valid rule, the listener would
crash. This has now been fixed by the addition of a configuration
validator that checks the access_rules field.
Additionally, a configuration option converter has been added to the
access_rules field so that one can specify several rules in a single
string by using "," (comma) as separator.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12315
2024-05-10 12:27:04 +02:00
JimMoen
1a6f0c9234
chore: bump gw_jt808 app vsn
2024-05-10 17:13:23 +08:00
JimMoen
b07242515f
fix(gw): jt808 REG_ACK failed due to faulty clientinfo
2024-05-10 17:12:28 +08:00
Kjell Winblad
8c0b55a89e
Merge pull request #12992 from kjellwinblad/kjell/fix_rule_unescape_function/EMQX-12313
...
fix: make rule engine unescape convert \a to the terminal alarm char
2024-05-10 09:48:33 +02:00
lafirest
da7b11020a
Merge pull request #13001 from lafirest/fix/sysk_reconn
...
fix(sysk): fix that the syskeeper forwarding never reconnecting
2024-05-10 11:24:28 +08:00
firest
1f3b640a3d
chore: update change
2024-05-10 07:29:14 +08:00
Thales Macedo Garitezi
7b80a9aa44
fix(aggregator): namespace aggregator ids with action type
...
Otherwise, actions of different types but same names will clash when starting the
aggregator supervision tree.
2024-05-09 16:02:11 -03:00
Andrew Mayorov
57dda70d2f
Merge pull request #12994 from keynslug/fix/sessds/log-levels
...
fix(sessds): use milder log level for regular error conditions
2024-05-09 18:34:54 +02:00
firest
e3a59c4037
fix(sysk): fix that the syskeeper forwarding never reconnecting
2024-05-09 16:27:58 +08:00
Ilia Averianov
a1aedee446
Merge pull request #12996 from savonarola/0508-fix-cursor-usage
...
fix(retainer): fix qlc cursor cleanup
2024-05-09 08:04:10 +03:00
ieQu1
07aa708894
test(ds): Refactor replication suite
2024-05-09 03:56:56 +02:00
ieQu1
63e51fca66
test(ds): Use streams to fill the storage
2024-05-09 02:46:57 +02:00
ieQu1
a0a3977043
feat(ds): Assign latest timestamp deterministically
2024-05-08 23:17:57 +02:00
ieQu1
2236af84ba
feat(ds): two-stage storage commit on the storage level
2024-05-08 23:17:57 +02:00
ieQu1
f250f00f3f
chore: Bump snabbkaffe version to 1.0.10
2024-05-08 23:17:57 +02:00
ieQu1
1ddbbca90e
feat(ds): Allow incremental update of the LTS trie
2024-05-08 23:17:57 +02:00
ieQu1
68ca891f41
test(ds): Use streams to create traffic
2024-05-08 23:17:57 +02:00
Zaiming (Stone) Shi
018d79b498
Merge pull request #12993 from zmstone/0508-fix-update-listener-zone-from-dashboard
...
0508 fix update listener zone from dashboard
2024-05-08 22:03:10 +02:00
Thales Macedo Garitezi
55f27281b3
Merge pull request #12991 from thalesmg/refactor-mv-message-type-r57-20240508
...
refactor(message validation): rename `message_name` to `message_type`
2024-05-08 16:58:28 -03:00
Thales Macedo Garitezi
d3905b6336
Merge pull request #12990 from thalesmg/fix-mv-conf-update-r57-20240508
...
fix(message validation): don't call `emqx_conf:update` during boot
2024-05-08 16:58:15 -03:00
Ilya Averyanov
57287f0722
fix(retainer): fix qlc cursor cleanup
2024-05-08 21:38:37 +03:00
Thales Macedo Garitezi
8d0574abf0
Merge pull request #12979 from thalesmg/fix-ps-monitor-topic-count-r57-20240506
...
fix(monitor api): count persistent routes and subscriptions
2024-05-08 15:09:16 -03:00
Andrew Mayorov
d84c180ccb
feat(dsrepl): avoid contacting unreachable ra servers
...
Assuming estabilished Erlang distribution channel is a reliable way to
tell whether a remote node is reachable.
2024-05-08 18:12:13 +02:00
Andrew Mayorov
3e956e8711
fix(sessds): use milder log level for regular error conditions
...
Especially when such events are emitted in (potentially) tight loops,
i.e. pulling messages from iterators pointing to remote shards.
2024-05-08 17:40:08 +02:00
zmstone
9edbad5459
fix(listener_api): do not allow update listener with unknown zone name
2024-05-08 16:37:55 +02:00
Thales Macedo Garitezi
d3bc32dc35
fix(message validation): don't call `emqx_conf:update` during boot
...
`load/0` and `unload/0` shouldn't call cluster operations.
2024-05-08 10:18:29 -03:00
Kjell Winblad
916168c755
fix: make rule engine unescape convert \a to the terminal alarm char
...
The rule engine unescape function should convert the escape sequence \a
to the alarm bell symbol (ASCII 7). This bug was created as it was
assumed that Erlang convert $\a to 7 but Erlang does not handle the \a
escape sequence and instead convert it to 97 (ASCII code for a).
Fixes:
https://emqx.atlassian.net/browse/EMQX-12313
2024-05-08 15:18:13 +02:00
Thales Macedo Garitezi
5172606032
refactor(message validation): rename `message_name` to `message_type`
...
On Product's behest, to make naming consistent with protobuf nomenclature.
https://protobuf.dev/programming-guides/proto3/#adding-types
2024-05-08 09:13:40 -03:00
zmstone
3a2ff34433
chore: add zone in listener config example
2024-05-08 11:43:10 +02:00
JimMoen
25be9c2890
fix(plugin): use binary `NameVsn` to re-build plugin avsc serde
2024-05-08 15:43:54 +08:00
JimMoen
68c601ad72
fix(plugin): add a backup for the plugin config file
2024-05-08 15:43:53 +08:00
JianBo He
4403b4f5ce
Merge pull request #12976 from HJianBo/fix-duplicated-disconnect
...
Fix duplicated disconnect event
2024-05-08 14:43:19 +08:00
Kjell Winblad
72f68afbca
Merge pull request #12981 from kjellwinblad/kjell/fixup_trace_all_http_and_error
...
Make formatting of action result trace entries better
2024-05-08 06:10:03 +02:00