Commit Graph

11930 Commits

Author SHA1 Message Date
ieQu1 8506ca7919
Merge pull request #12998 from ieQu1/dev/improve-latency
Use leader's clock when calculating LTS cutoff timestamp
2024-05-13 21:54:06 +02:00
Andrew Mayorov 66b7ac4c45
fix(sessds): persist $SYS messages as well
Otherwise, persistent sessions will not be able to receive $SYS messages
whatsoever.
2024-05-13 21:27:55 +02:00
Thales Macedo Garitezi a2a5fd1f8e test(client mgmt api): attempt to fix flaky test
```
%%% emqx_mgmt_api_clients_SUITE ==> msgs_plain_encoding.t_mqueue_messages: FAILED
%%% emqx_mgmt_api_clients_SUITE ==>
Failure/Error: ?assertEqual(79, Count)
  expected: 79
       got: 100
      line: 1303
```
2024-05-13 15:53:21 -03:00
ieQu1 3da3a36863
test(ds): Add generation in the replication suite 2024-05-13 19:51:04 +02:00
ieQu1 9f7ef9f34f
fix(ds): Apply review remarks 2024-05-13 19:35:24 +02:00
Thales Macedo Garitezi e7c47f4ec0 fix(ds clients mgmt api): add more fields to disconnected durable sessions
Fixes https://emqx.atlassian.net/browse/EMQX-12369
2024-05-13 14:25:02 -03:00
Kjell Winblad 1c34a84dd2
Merge pull request #13028 from kjellwinblad/kjell/swagger_example_missing_ruleid_paramter/EMQX-12325
fix: include ruleid parameter in swagger example for apply rule endpoint
2024-05-13 17:35:31 +02:00
Thales Macedo Garitezi 81473519bd
Merge pull request #13031 from thalesmg/fix-clientsv2-sec-r57-20240513
fix(client mgmt api): require authn for endpoint
2024-05-13 12:17:31 -03:00
Thales Macedo Garitezi 39899b917d fix(client mgmt api): require authn for endpoint
Fixes https://emqx.atlassian.net/browse/EMQX-12361
2024-05-13 10:54:21 -03:00
Thales Macedo Garitezi 6b032faebc fix(client mgmt api): return expired durable sessions with `is_expired: true`
Fixes https://emqx.atlassian.net/browse/EMQX-12274
2024-05-13 10:51:24 -03:00
Thales Macedo Garitezi 0dee2d67b1 fix: always check if first query has more rows 2024-05-13 09:10:18 -03:00
Kjell Winblad c8690a8273 fix: include ruleid parameter in swagger example for apply rule endpoint
Fixes:
https://emqx.atlassian.net/browse/EMQX-12325
2024-05-13 12:28:17 +02:00
Kjell Winblad 921d82f083 fix: JSON trace formatter should handle report style log entries
Fixes:
https://emqx.atlassian.net/browse/EMQX-12344
2024-05-13 12:11:11 +02:00
zmstone 7faf73e0f4 chore(emqx_schema): change atom array to enum array for alarm.actions 2024-05-13 10:54:33 +02:00
JianBo He 54f6bd181b
Merge pull request #13010 from JimMoen/fix-jt808-reg-failed
fix(gw): jt808 REG_ACK failed due to faulty clientinfo
2024-05-13 14:19:18 +08:00
JimMoen 2f212f34a3
Merge pull request #12989 from JimMoen/fix-plugin-config-file-backup
fix(plugin): add a backup for the plugin config file
2024-05-11 16:38:36 +08:00
JimMoen a5ecceb755
chore: apply suggestions from code review, thanks @thalesmg 2024-05-11 11:36:40 +08:00
Thales Macedo Garitezi e25fe62cbc fix(subs mgmt api): attempt to return mixed clients ds/non-ds in the same page
Fixes https://emqx.atlassian.net/browse/EMQX-12294
2024-05-10 14:48:46 -03:00
Kjell Winblad c6df069c5a
Merge pull request #13012 from kjellwinblad/kjell/fix_incorrect_config_crash/EMQX-12315
fix: listener crash if access_rules config option is incorrect
2024-05-10 17:46:40 +02:00
Thales Macedo Garitezi f690db9849 fix(postgres): reduce log spamming when connection goes down
Fixes https://emqx.atlassian.net/browse/EMQX-12334

See also: https://github.com/emqx/epgsql/pull/10
2024-05-10 12:15:01 -03:00
JimMoen 3c7e8a3bd5
test(gw): jt808 register failed 2024-05-10 23:09:22 +08:00
JimMoen f8fd4d15be
fix(gw): process jt808 client deregister packet 2024-05-10 23:00:59 +08:00
Zaiming (Stone) Shi c491b83857
Merge pull request #13014 from zmstone/0510-add-is_template-for-kinesis-partition-key
fix(kinesis): partition_key should be of 'template' type
2024-05-10 16:11:35 +02:00
Kjell Winblad b13fa37771 fix: dialyzer warning 2024-05-10 15:51:28 +02:00
Thales Macedo Garitezi 0eaef18391
Merge pull request #13005 from thalesmg/fix-aggreg-sup-tree-r57-20240509
fix(aggregator): refactor supervision tree
2024-05-10 10:22:33 -03:00
Kjell Winblad 2bed5894e3 test: add test cases for listeners access_rules validation and split 2024-05-10 14:21:43 +02:00
Thales Macedo Garitezi 3a29696a48 fix(aggregator): refactor supervision tree
Instead of using a aggregator supervisor with a fixed local name, we should allow
specifying different names so each action app will spawn its own aggregator supervisor.
2024-05-10 09:06:12 -03:00
zmstone 47df968d4a fix(kinesis): partition_key should be of 'template' type 2024-05-10 13:57:38 +02:00
Kjell Winblad 0aeb2cd77f fix: listener crash if access_rules config option is incorrect
Previously when changing the access_rules configuration option of a
listener to something that was not a valid rule, the listener would
crash. This has now been fixed by the addition of a configuration
validator that checks the access_rules field.

Additionally, a configuration option converter has been added to the
access_rules field so that one can specify several rules in a single
string by using "," (comma) as separator.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12315
2024-05-10 12:27:04 +02:00
JimMoen 1a6f0c9234
chore: bump gw_jt808 app vsn 2024-05-10 17:13:23 +08:00
JimMoen b07242515f
fix(gw): jt808 REG_ACK failed due to faulty clientinfo 2024-05-10 17:12:28 +08:00
Kjell Winblad 8c0b55a89e
Merge pull request #12992 from kjellwinblad/kjell/fix_rule_unescape_function/EMQX-12313
fix: make rule engine unescape convert \a to the terminal alarm char
2024-05-10 09:48:33 +02:00
lafirest da7b11020a
Merge pull request #13001 from lafirest/fix/sysk_reconn
fix(sysk): fix that the syskeeper forwarding never reconnecting
2024-05-10 11:24:28 +08:00
firest 1f3b640a3d chore: update change 2024-05-10 07:29:14 +08:00
Thales Macedo Garitezi 7b80a9aa44 fix(aggregator): namespace aggregator ids with action type
Otherwise, actions of different types but same names will clash when starting the
aggregator supervision tree.
2024-05-09 16:02:11 -03:00
Andrew Mayorov 57dda70d2f
Merge pull request #12994 from keynslug/fix/sessds/log-levels
fix(sessds): use milder log level for regular error conditions
2024-05-09 18:34:54 +02:00
firest e3a59c4037 fix(sysk): fix that the syskeeper forwarding never reconnecting 2024-05-09 16:27:58 +08:00
Ilia Averianov a1aedee446
Merge pull request #12996 from savonarola/0508-fix-cursor-usage
fix(retainer): fix qlc cursor cleanup
2024-05-09 08:04:10 +03:00
ieQu1 07aa708894
test(ds): Refactor replication suite 2024-05-09 03:56:56 +02:00
ieQu1 63e51fca66
test(ds): Use streams to fill the storage 2024-05-09 02:46:57 +02:00
ieQu1 a0a3977043
feat(ds): Assign latest timestamp deterministically 2024-05-08 23:17:57 +02:00
ieQu1 2236af84ba
feat(ds): two-stage storage commit on the storage level 2024-05-08 23:17:57 +02:00
ieQu1 f250f00f3f
chore: Bump snabbkaffe version to 1.0.10 2024-05-08 23:17:57 +02:00
ieQu1 1ddbbca90e
feat(ds): Allow incremental update of the LTS trie 2024-05-08 23:17:57 +02:00
ieQu1 68ca891f41
test(ds): Use streams to create traffic 2024-05-08 23:17:57 +02:00
Zaiming (Stone) Shi 018d79b498
Merge pull request #12993 from zmstone/0508-fix-update-listener-zone-from-dashboard
0508 fix update listener zone from dashboard
2024-05-08 22:03:10 +02:00
Thales Macedo Garitezi 55f27281b3
Merge pull request #12991 from thalesmg/refactor-mv-message-type-r57-20240508
refactor(message validation): rename `message_name` to `message_type`
2024-05-08 16:58:28 -03:00
Thales Macedo Garitezi d3905b6336
Merge pull request #12990 from thalesmg/fix-mv-conf-update-r57-20240508
fix(message validation): don't call `emqx_conf:update` during boot
2024-05-08 16:58:15 -03:00
Ilya Averyanov 57287f0722 fix(retainer): fix qlc cursor cleanup 2024-05-08 21:38:37 +03:00
Thales Macedo Garitezi 8d0574abf0
Merge pull request #12979 from thalesmg/fix-ps-monitor-topic-count-r57-20240506
fix(monitor api): count persistent routes and subscriptions
2024-05-08 15:09:16 -03:00
Andrew Mayorov d84c180ccb
feat(dsrepl): avoid contacting unreachable ra servers
Assuming estabilished Erlang distribution channel is a reliable way to
tell whether a remote node is reachable.
2024-05-08 18:12:13 +02:00
Andrew Mayorov 3e956e8711
fix(sessds): use milder log level for regular error conditions
Especially when such events are emitted in (potentially) tight loops,
i.e. pulling messages from iterators pointing to remote shards.
2024-05-08 17:40:08 +02:00
zmstone 9edbad5459 fix(listener_api): do not allow update listener with unknown zone name 2024-05-08 16:37:55 +02:00
Thales Macedo Garitezi d3bc32dc35 fix(message validation): don't call `emqx_conf:update` during boot
`load/0` and `unload/0` shouldn't call cluster operations.
2024-05-08 10:18:29 -03:00
Kjell Winblad 916168c755 fix: make rule engine unescape convert \a to the terminal alarm char
The rule engine unescape function should convert the escape sequence \a
to the alarm bell symbol (ASCII 7). This bug was created as it was
assumed that Erlang convert $\a to 7 but Erlang does not handle the \a
escape sequence and instead convert it to 97 (ASCII code for a).

Fixes:
https://emqx.atlassian.net/browse/EMQX-12313
2024-05-08 15:18:13 +02:00
Thales Macedo Garitezi 5172606032 refactor(message validation): rename `message_name` to `message_type`
On Product's behest, to make naming consistent with protobuf nomenclature.

https://protobuf.dev/programming-guides/proto3/#adding-types
2024-05-08 09:13:40 -03:00
zmstone 3a2ff34433 chore: add zone in listener config example 2024-05-08 11:43:10 +02:00
JimMoen 25be9c2890
fix(plugin): use binary `NameVsn` to re-build plugin avsc serde 2024-05-08 15:43:54 +08:00
JimMoen 68c601ad72
fix(plugin): add a backup for the plugin config file 2024-05-08 15:43:53 +08:00
JianBo He 4403b4f5ce
Merge pull request #12976 from HJianBo/fix-duplicated-disconnect
Fix duplicated disconnect event
2024-05-08 14:43:19 +08:00
Kjell Winblad 72f68afbca
Merge pull request #12981 from kjellwinblad/kjell/fixup_trace_all_http_and_error
Make formatting of action result trace entries better
2024-05-08 06:10:03 +02:00
Thales Macedo Garitezi 0c7b221c42 fix(clients mgmt api): set `durable` flag for DS sessions and subscriptions 2024-05-07 14:55:50 -03:00
Kjell Winblad d5324e295f test: do cleanup in emqx_common_test_helpers:on_exit function
Thanks @thalesmg for the suggestion.
2024-05-07 18:18:18 +02:00
Thales Macedo Garitezi 02c58b67f0
Merge pull request #12986 from thalesmg/fix-mv-name-length-r57-20240507
fix(message validation): limit the length of validation name
2024-05-07 11:39:17 -03:00
Thales Macedo Garitezi 2989793f4c
Merge pull request #12980 from thalesmg/fix-ds-get-offline-client-stuff-r57-20240506
fix(mgmt clients api): hold channel info after client disconnects for display in API
2024-05-07 11:18:00 -03:00
Thales Macedo Garitezi d4d2b4c460
Merge pull request #12985 from thalesmg/fix-mv-cluster-metrics-r57-20240507
fix(message validation): apply config CRUD operations on whole cluster
2024-05-07 11:10:58 -03:00
Kjell Winblad 09ee7ec0e2 fix(rule tracing): make sure that recoverable errors are traced 2024-05-07 15:49:38 +02:00
Thales Macedo Garitezi 96bb6e7d88 fix(message validation): limit the length of validation name
Fixes https://emqx.atlassian.net/browse/EMQX-12299
2024-05-07 10:04:56 -03:00
Thales Macedo Garitezi 06f16d4ee0 fix(message validation): apply config CRUD operations on whole cluster
Fixes https://emqx.atlassian.net/browse/EMQX-12298
2024-05-07 09:45:50 -03:00
Thales Macedo Garitezi 6d6eb42fa3 fix(mgmt clients api): hold channel info after client disconnects for display in api
Fixes https://emqx.atlassian.net/browse/EMQX-12266
2024-05-07 09:22:33 -03:00
Serge Tupchii b869488d35 fix(emqx_license): don't crash if an imported backup file has no license config 2024-05-07 13:48:04 +03:00
Kjell Winblad feecc36607 fix(rule tracing): clean up error tuple in the action_failed trace 2024-05-07 09:13:55 +02:00
JianBo He 1642b06bf9 test: add tests 2024-05-07 10:30:53 +08:00
Thales Macedo Garitezi 28355a4cfc fix(monitor api): count persistent routes and subscriptions
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-06 16:51:31 -03:00
Zaiming (Stone) Shi 93e31d088a
Merge pull request #12978 from JimMoen/fix-typo
chore: fix typo
2024-05-06 21:28:51 +02:00
Kjell Winblad ca88f5731b fix(rule tracing): format result traces in a more structured way 2024-05-06 17:33:59 +02:00
Thales Macedo Garitezi cd4aaf663b docs: briefly describe delivery callbacks 2024-05-06 10:00:10 -03:00
Thales Macedo Garitezi e6b1dfb095 refactor: improve error 2024-05-06 09:53:49 -03:00
Thales Macedo Garitezi 793cc4df2c fix: add fully qualified calls to enable hot beam reloads 2024-05-06 09:52:31 -03:00
Thales Macedo Garitezi edcb750d9f fix: remove unused dependency 2024-05-06 09:51:32 -03:00
Thales Macedo Garitezi 36542a4b1d docs: bump copyright years 2024-05-06 09:50:12 -03:00
JimMoen 1df2d837ee
chore: fix typo 2024-05-06 17:37:32 +08:00
ieQu1 3642bcd1b6
docs(ds): Fix comment for the builtin DS metrics 2024-05-06 11:21:32 +02:00
ieQu1 b2a633aca1
fix(ds): Use leader's clock for computing LTS safe cutoff time 2024-05-06 11:21:32 +02:00
ieQu1 1ff2e02fd9
feat(ds): Pass current time to the storage layer via argument 2024-05-06 11:21:32 +02:00
ieQu1 8ac9700aab
feat(ds): Add an API for DB-global variables 2024-05-06 11:21:32 +02:00
ieQu1 86d45522e3
fix(dsrepl): Don't reverse elements of batches 2024-05-06 11:21:32 +02:00
ieQu1 bcfa7b2209
fix(ds): Destroy LTS tries when the generation is dropped 2024-05-06 11:21:32 +02:00
ieQu1 9999ccd36c
feat(ds): Ignore safe cutoff time for streams without varying levels 2024-05-06 11:21:32 +02:00
JianBo He b320b20c7b fix(mqtt): disconnected event should not be sent twice 2024-05-06 16:17:02 +08:00
Thales Macedo Garitezi 32493e395c refactor: avoid passing record down to template rendering function 2024-05-03 17:43:30 -03:00
Thales Macedo Garitezi 157e2c2535 refactor: make api more generic 2024-05-03 17:25:02 -03:00
Thales Macedo Garitezi a6508e2fca refactor: use better names
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2024-05-03 17:04:09 -03:00
Thales Macedo Garitezi d61f4078e2 refactor: extract blob aggregator logic to separate application
This will allow other bridges to share the same aggregation logic as S3 aggregated action.
2024-05-03 14:35:58 -03:00
Kjell Winblad 004dc80fb2
Merge pull request #12958 from kjellwinblad/kjell/fixup_trace
fix: rule trace formatting, republish and console stop after rendering
2024-05-03 16:24:17 +02:00
Zaiming (Stone) Shi 52357742aa
Merge pull request #12962 from zmstone/0502-fix-ssl-client-opts-to-allow-wildcard-cert-by-default
fix(ssl-clients): allow wildcard certificates by default
2024-05-03 15:19:37 +02:00
Kjell Winblad 5bfe31b691 fix: issues found during PR review (thanks @thalesmg and @zmstone)
* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
  maintainability
* Added callback to format return value from connector
  * Extended test case to check that the format return value callback
    works
* Added handling of "lazy" trace entry data to the text formatter. Do we
  need to handle this data in the normal log formatters as well?
2024-05-03 15:04:57 +02:00
Kjell Winblad e32745bca6 test(apply rule trace): add test case for republish and console print 2024-05-03 15:02:37 +02:00
Kjell Winblad b9cd75eef0 test(emqx_mgmt_api_trace_SUITE): fix bad test cases 2024-05-03 15:02:37 +02:00
Kjell Winblad 76ccef7ce3 fix(trace api): trace should be ready after create
Fixes:
https://emqx.atlassian.net/browse/EMQX-12276
2024-05-03 15:02:37 +02:00