0763e29786
chore: prepare for 5.7.0-rc.2 release
2024-05-24 21:10:37 +02:00
c3c6da7ec4
chore: prepare for release cut 5.7.0-rc.1
2024-05-23 21:39:15 +02:00
b4c6968f8c
Merge pull request #13104 from keynslug/fix/EMQX-12415/last-resort-poll
...
fix(dsrepl): trigger "last-resort" pending transitions handler when idle
2024-05-23 21:17:03 +03:00
5584b658c9
fix(sessds): Change the defaults to renew streams every 1 second
2024-05-23 17:17:29 +02:00
118c1d2806
Merge pull request #13103 from thalesmg/fix-another-ds-counter-nit-r57-20240523
...
fix(monitor api): fix cluster metric aggregation
2024-05-23 10:57:53 -03:00
839b143fc4
fix(sessds): consider message durability disabled by default
2024-05-23 14:48:39 +02:00
c0e3a81c61
fix(monitor api): fix cluster metric aggregation
...
Fixes https://emqx.atlassian.net/browse/EMQX-12423
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-23 09:38:24 -03:00
0e5b02c373
Merge pull request #13100 from kjellwinblad/kjell/s3_aggregate_only_has_result_trace/EMQX-12429
...
fix: add action rendered trace for s3 in aggregated mode
2024-05-23 14:29:18 +02:00
fb7688ab94
fix(trace): make sure that the payload encode works with nested payloads
...
This commit makes sure that the trace setting for payload encode works
even when the payload is in a nested structure or when the payload key
is a binary instead of an atom.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12424
2024-05-23 12:30:56 +02:00
950fa82be3
Merge pull request #13097 from ieQu1/dev/ds-replay-unrecoverable-no-test
...
feat(sessds): Handle unrecoverable errors
2024-05-23 09:49:21 +02:00
4053356597
Merge pull request #13063 from JimMoen/improve-plugin-config-behavior
...
fix(plugin): serde not found on new-joined nodes
2024-05-23 11:24:44 +08:00
07774ab060
feat(sessds): Handle unrecoverable errors
2024-05-22 22:45:37 +02:00
5c78ecba40
docs(ds): Update documentation for the storage layouts
2024-05-22 18:01:33 +02:00
29345aaa30
fix(ds): Fix idle event generation in bitfield_lts layout
2024-05-22 18:01:33 +02:00
31a35f2a15
Merge pull request #13076 from thalesmg/fix-ds-session-count-client-api-r57-20240520
...
fix(client mgmt api): cache disconnected durable session count for `/clients` api
2024-05-21 15:34:21 -03:00
87b3b214b9
fix: make static_check happy
2024-05-22 02:09:36 +08:00
8b4a1c3d75
fix(client mgmt api): cache disconnected durable session count for `/clients` api
...
Fixes https://emqx.atlassian.net/browse/EMQX-12396
2024-05-20 11:48:08 -03:00
8279d8c787
Merge pull request #13053 from kjellwinblad/kjell/fix_rule_trace_issues/EMQX-12327/EMQX-12335/EMQX-12336
...
Fix small action trace issues
2024-05-20 16:25:38 +02:00
388e6c8262
fix(dssubs): introduce separate gauge for subscriptions from durable sessions
...
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-17 10:17:16 -03:00
6f30c1c541
fix(sessds): Rename last_alive_update_interval -> heartbeat_interval
2024-05-17 01:20:10 +02:00
73f17249e9
fix(sessds): Rename session_persistence to durable_sessions
2024-05-16 21:31:32 +02:00
ee6e7174cf
fix(sessds): Rename the durable messages DB to `messages`
2024-05-16 21:31:32 +02:00
b7c2f4a6d7
fix(best_effor_json): make tuple list check more efficient
2024-05-16 15:40:38 +02:00
ba74135079
Merge pull request #13052 from kjellwinblad/kjell/evaluate_lazy_trace_values_in_normal_log_formatters/EMQX-12380
...
fix: evaluate lazy values in normal log formatters when level is debug
2024-05-16 15:02:53 +02:00
0be1249bbe
Merge pull request #13041 from zmstone/0513-http-authn-header-missing-should-result-in-meaningful-error-message
...
0513 http authn header missing should result in meaningful error message
2024-05-16 13:17:20 +02:00
71b393be89
Merge pull request #13057 from ieQu1/dev/default-n-shards
...
fix(ds): Change the default number of shards to 12
2024-05-16 13:05:34 +02:00
9fd8e930be
fix(best_effort_json): only do tuple list to map without losing pairs
2024-05-16 11:19:48 +02:00
deffb01544
fix(ds): Change the default number of shards to 12
2024-05-16 08:06:07 +02:00
8cbfc4c0cd
fix: evaluate lazy values in normal log formatters when level is debug
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12380
2024-05-15 11:47:21 +02:00
faf3492e58
Merge pull request #13039 from lafirest/fix/rocket_key
...
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-15 15:28:35 +08:00
bc5071590b
Merge pull request #13033 from thalesmg/fix-ds-more-client-api-fields-r57-20240513
...
fix(ds clients mgmt api): add more fields to disconnected durable sessions
2024-05-14 15:29:40 -03:00
1730a41337
Merge pull request #13046 from thalesmg/refactor-mv-rename-sv-r57-20240514
...
refactor: rename `message_validation` to `schema_validation`
2024-05-14 13:03:12 -03:00
adbc392265
Merge pull request #13027 from kjellwinblad/kjell/fix_trace_json_report_formatting/EMQX-12344
...
fix: JSON trace formatter should handle report style log entries
2024-05-14 18:02:59 +02:00
437a0c0d55
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-14 23:01:31 +08:00
257dae5211
refactor: rename `message_validation` to `schema_validation`
...
Request from Product team.
Fixes https://emqx.atlassian.net/browse/EMQX-12383
2024-05-14 10:32:27 -03:00
f75c7a5cea
fix(trace log entry error): better structure and naming
2024-05-14 13:38:41 +02:00
525e4dac95
Merge pull request #13036 from ieQu1/dev/reduce-log-spam
...
tests: Reduce log spam in the failed test suites
2024-05-14 10:53:30 +02:00
98a93662c1
Merge pull request #13035 from keynslug/fix/EMQX-12291/pers-sys-msg
...
fix(sessds): persist $SYS messages as well
2024-05-14 10:05:49 +02:00
ac3f5a083d
test: Reduce log spam in the failed test suites
2024-05-13 22:00:33 +02:00
8506ca7919
Merge pull request #12998 from ieQu1/dev/improve-latency
...
Use leader's clock when calculating LTS cutoff timestamp
2024-05-13 21:54:06 +02:00
66b7ac4c45
fix(sessds): persist $SYS messages as well
...
Otherwise, persistent sessions will not be able to receive $SYS messages
whatsoever.
2024-05-13 21:27:55 +02:00
e7c47f4ec0
fix(ds clients mgmt api): add more fields to disconnected durable sessions
...
Fixes https://emqx.atlassian.net/browse/EMQX-12369
2024-05-13 14:25:02 -03:00
921d82f083
fix: JSON trace formatter should handle report style log entries
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12344
2024-05-13 12:11:11 +02:00
7faf73e0f4
chore(emqx_schema): change atom array to enum array for alarm.actions
2024-05-13 10:54:33 +02:00
c6df069c5a
Merge pull request #13012 from kjellwinblad/kjell/fix_incorrect_config_crash/EMQX-12315
...
fix: listener crash if access_rules config option is incorrect
2024-05-10 17:46:40 +02:00
b13fa37771
fix: dialyzer warning
2024-05-10 15:51:28 +02:00
2bed5894e3
test: add test cases for listeners access_rules validation and split
2024-05-10 14:21:43 +02:00
0aeb2cd77f
fix: listener crash if access_rules config option is incorrect
...
Previously when changing the access_rules configuration option of a
listener to something that was not a valid rule, the listener would
crash. This has now been fixed by the addition of a configuration
validator that checks the access_rules field.
Additionally, a configuration option converter has been added to the
access_rules field so that one can specify several rules in a single
string by using "," (comma) as separator.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12315
2024-05-10 12:27:04 +02:00
57dda70d2f
Merge pull request #12994 from keynslug/fix/sessds/log-levels
...
fix(sessds): use milder log level for regular error conditions
2024-05-09 18:34:54 +02:00
f250f00f3f
chore: Bump snabbkaffe version to 1.0.10
2024-05-08 23:17:57 +02:00
018d79b498
Merge pull request #12993 from zmstone/0508-fix-update-listener-zone-from-dashboard
...
0508 fix update listener zone from dashboard
2024-05-08 22:03:10 +02:00
8d0574abf0
Merge pull request #12979 from thalesmg/fix-ps-monitor-topic-count-r57-20240506
...
fix(monitor api): count persistent routes and subscriptions
2024-05-08 15:09:16 -03:00
3e956e8711
fix(sessds): use milder log level for regular error conditions
...
Especially when such events are emitted in (potentially) tight loops,
i.e. pulling messages from iterators pointing to remote shards.
2024-05-08 17:40:08 +02:00
9edbad5459
fix(listener_api): do not allow update listener with unknown zone name
2024-05-08 16:37:55 +02:00
4403b4f5ce
Merge pull request #12976 from HJianBo/fix-duplicated-disconnect
...
Fix duplicated disconnect event
2024-05-08 14:43:19 +08:00
72f68afbca
Merge pull request #12981 from kjellwinblad/kjell/fixup_trace_all_http_and_error
...
Make formatting of action result trace entries better
2024-05-08 06:10:03 +02:00
0c7b221c42
fix(clients mgmt api): set `durable` flag for DS sessions and subscriptions
2024-05-07 14:55:50 -03:00
2989793f4c
Merge pull request #12980 from thalesmg/fix-ds-get-offline-client-stuff-r57-20240506
...
fix(mgmt clients api): hold channel info after client disconnects for display in API
2024-05-07 11:18:00 -03:00
09ee7ec0e2
fix(rule tracing): make sure that recoverable errors are traced
2024-05-07 15:49:38 +02:00
6d6eb42fa3
fix(mgmt clients api): hold channel info after client disconnects for display in api
...
Fixes https://emqx.atlassian.net/browse/EMQX-12266
2024-05-07 09:22:33 -03:00
28355a4cfc
fix(monitor api): count persistent routes and subscriptions
...
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-06 16:51:31 -03:00
1df2d837ee
chore: fix typo
2024-05-06 17:37:32 +08:00
b320b20c7b
fix(mqtt): disconnected event should not be sent twice
2024-05-06 16:17:02 +08:00
004dc80fb2
Merge pull request #12958 from kjellwinblad/kjell/fixup_trace
...
fix: rule trace formatting, republish and console stop after rendering
2024-05-03 16:24:17 +02:00
52357742aa
Merge pull request #12962 from zmstone/0502-fix-ssl-client-opts-to-allow-wildcard-cert-by-default
...
fix(ssl-clients): allow wildcard certificates by default
2024-05-03 15:19:37 +02:00
5bfe31b691
fix: issues found during PR review (thanks @thalesmg and @zmstone)
...
* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
maintainability
* Added callback to format return value from connector
* Extended test case to check that the format return value callback
works
* Added handling of "lazy" trace entry data to the text formatter. Do we
need to handle this data in the normal log formatters as well?
2024-05-03 15:04:57 +02:00
76ccef7ce3
fix(trace api): trace should be ready after create
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12276
2024-05-03 15:02:37 +02:00
ea7633c484
fix: rule trace formatting, republish and console stop after rendering
...
* Better rule trace formatting for many trace entries
* The republish and console actions have got working stop after
rendering functionality
2024-05-03 15:02:36 +02:00
37d66e90fb
fix(ssl-clients): allow wildcard certificates by default
2024-05-03 12:52:42 +02:00
80d724c504
feat(authn): add connection expire based on authn data
2024-04-30 17:04:55 +03:00
1974ec15ec
fix(client_attrs): fix client_attrs extraction loop
2024-04-30 12:12:35 +02:00
9917293fd0
fix(schema): description should be in binary() type
2024-04-29 21:44:55 +02:00
7a05a4754f
docs: expose zone config in schema doc
2024-04-29 13:57:18 +02:00
10625eacac
chore: upgrade to hocon-0.42.2
...
hocon pretty-print quotes more string values
if a string has '.' or '-', or if it starts with a digit 0-9,
then it's quoted.
see details here: https://github.com/emqx/hocon/pull/293
2024-04-28 14:05:30 +02:00
5f4215b333
fix(mgmt): avoid 500 error when hocon syntax error
2024-04-27 21:12:42 +02:00
8db5e51592
feat: plugin config with avro schema and apis
2024-04-26 10:25:04 +08:00
6af651cf2f
Merge pull request #12931 from zmstone/0425-fix-acl.conf-topic-template-render-fialure-handling
...
0425 fix acl.conf topic template render fialure handling
2024-04-25 20:30:02 +02:00
01923147a2
fix(variform and authz): do not initialize empty client_attrs field
...
when client_attrs_init expression renders to empty string,
do not initialize the attribute.
also fixed an ACL error: a template render failure for a topic
would stop the ACL checks for the following topics if more
than one topic is configured.
2024-04-25 17:32:07 +02:00
ef9884cf47
refactor(rule trace): templates rendered trace to increase code reuse
...
* The code for passing the trace context to a sub process has been
improved to increase code reuse. This code is used when the action
templates are rendered in a sub process.
* A macro has also been added for the error term that is thrown when the
action shall be stopped after the templates has been rendered. This is
also done to reduce code duplication and to reduce the risk of
introducing bugs due to typos.
* Fix incorrect type spec
Thanks to @zmstone for suggesting these improvements in comments to a PR
(https://github.com/emqx/emqx/pull/12916 ).
2024-04-25 13:15:36 +02:00
a2dd8f5aee
feat: add stop after render and after render trace to cassandra action
2024-04-25 11:51:46 +02:00
b02ed4e6ec
feat: add stop after render and after render trace to pgsql action
2024-04-25 11:51:46 +02:00
5ca90ccced
fix: improve structure of log trace entries for HTTP action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12025
2024-04-25 11:51:46 +02:00
f2fa35efdf
Merge remote-tracking branch 'origin/master' into 0424-merge-latest-master-to-release-57
2024-04-24 22:02:56 +02:00
02a0faa1d6
refactor: avoid evaluating Data more than once in SLOG_THROTTE macro
2024-04-24 16:26:34 +03:00
b66d7a688e
Merge remote-tracking branch 'origin/release-57' into release-57
2024-04-24 13:09:06 +02:00
6049b4966f
fix: bump escokd to 5.11.2 to handle udp_error/closed message
2024-04-22 16:41:09 +08:00
b723e87e54
Merge remote-tracking branch 'upstream/release-56' into 0422-sync-release-56
2024-04-22 08:12:05 +02:00
6e99f01ecd
Merge pull request #12863 from kjellwinblad/kjell/new_trace_log_formatter/EMQX-12025
...
feat(emqx_trace): add JSON trace log entry formatter
2024-04-22 06:25:18 +02:00
bac5100635
Merge pull request #12874 from ieQu1/dev/EMQX-12030-subscriptions-api
...
API for durable subscriptions
2024-04-19 17:11:53 +02:00
2890bc2619
fix(tracing): remove internal extra field from the trace config
...
This commit removes the internal extra field from the trace config
structure exposed to the user via the HTTP API.
2024-04-19 17:05:34 +02:00
9e46c18443
fix(emqx_trace_SUITE:t_base_create_delete): broken test case
2024-04-19 15:41:50 +02:00
ede7246882
fix(sessds): Avoid double-enriching transient messages
2024-04-19 13:39:04 +02:00
a8b1224225
Merge pull request #12893 from zmstone/0416-add-is-template-flag-to-dashboard-schema
...
0416 add `is_template` flag to dashboard schema
2024-04-19 10:41:43 +02:00
c163956d08
fix(trace formatter): remove record field to enable rolling upgrade
2024-04-19 09:52:20 +02:00
f1e6565ddd
refactor(sessds): Move all subscription logic to the subs module
2024-04-19 01:20:29 +02:00
d12966db5b
test: Avoid dumping raw snabbkaffe traces to the console
2024-04-19 01:20:29 +02:00
285bfa9367
fix: improve rendering of action_template_rendered trace
2024-04-18 17:00:47 +02:00
f5b0439724
fix(emqx_rule_engine_api_rule_apply_SUITE): flaky test case
2024-04-18 14:28:28 +02:00
aa388adba9
fix(json trace format): format client_ids and rule_ids as lists
2024-04-18 11:27:39 +02:00
10957e7d79
fix: change name of the default trace log format from plain to text
2024-04-18 10:42:51 +02:00
ef9f8a8fdf
fix: unreachable clause found by dialyzer
2024-04-18 10:28:37 +02:00
6c0ee8bb01
test(emqx_trace_SUITE): fix failure due to new field
2024-04-18 10:28:37 +02:00
968dc2ccda
fix(json trace format): use best_effort_json instead of new encoder
2024-04-18 10:28:37 +02:00
aa39835242
feat(emqx_cli): add parameter for trace format (plain or json)
2024-04-18 10:28:29 +02:00
6fd7a06c5d
fix: problems reported by dialyzer
2024-04-18 10:20:18 +02:00
a6558740e8
feat(emqx_trace): add JSON trace log entry formatter
...
This commit makes it possible to select the JSON trace log entry
formatter when crating a trace pattern. This will make it easier for the
dashboard and automatic tests to parse the log entries.
Fixes: EMQX-12025 (partly)
2024-04-18 10:20:11 +02:00
2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
...
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
e93bd314bc
chore: 5.6.1
2024-04-18 08:03:56 +02:00
5a4bfff9e5
refactor: add template_str type
2024-04-17 20:34:33 +02:00
6cf29ba688
fix: clean up traces to make them easier to parse and understand
2024-04-17 18:20:17 +02:00
51c8173174
feat(bridge): add is_template flag to bridge config fields
2024-04-17 13:10:55 +02:00
124c5047d0
feat(sessds): Add API for getting session data from the cold storage
2024-04-17 12:31:54 +02:00
e439a2e0f2
fix(sessds): Save protocol name and version in the session metadata
2024-04-17 12:31:54 +02:00
197a4c30be
fix(sessds): Strip unneccessary data from the durable session state
2024-04-17 01:21:52 +02:00
87ffaf89e5
refactor(sessds_state): Use macros for map keys
2024-04-17 01:21:52 +02:00
113a990482
feat(sessds): Support max subscriptions
2024-04-17 01:21:51 +02:00
6c897c26ae
fix(sessds): Commit session on unsubscribe
2024-04-17 01:21:51 +02:00
b30ddc206e
fix(sessds): Immutable subscriptions
...
This commit fixes two issues:
- Behavior of overlapping subscriptions has been aligned with the
in-memory session.
- Fixed handling of replays when subscription changes (either by
client or EMQX configuration)
2024-04-17 01:21:51 +02:00
3e0c649e8e
feat(sessds): Store awaiting rel
2024-04-17 01:21:51 +02:00
4d38a8fd44
Merge pull request #12872 from zmstone/0412-support-variform-for-client_attrs
...
0412 use variform for client_attrs
2024-04-16 13:29:56 +02:00
c645cfa5d6
fix(sessds): Graceful handling of shared subscription error
2024-04-16 12:37:01 +02:00
f3bb28c6af
chore: 5.6.1-rc.2
2024-04-16 09:47:32 +02:00
a0ffe5e7ae
chore: 5.6.1-rc.1
2024-04-15 22:05:05 +02:00
3ef160eed2
Merge remote-tracking branch 'upstream/release-56' into 0415-sync-release-56
2024-04-15 08:09:03 +02:00
b76b6fbe63
feat(variform): initialize client_attrs with variform
...
Moved regular expression extraction as a variform function.
2024-04-14 10:13:24 +02:00
da5b01aa46
refactor(client_attr): allow more than one initial extraction
2024-04-13 01:00:25 +02:00
6777f04780
fix: iolist_to_binrary -> unicode:characters_to_binary
...
It is not always safe to use iolist_to_binrary on the output of an
io_lib:format call with the ~ts placeholder.
2024-04-12 15:21:25 +02:00
9998940aa2
fix(trace): several improvements thanks to comments from @zmstone
2024-04-11 16:48:43 +02:00
3b7cade671
chore: 5.6.1-beta.1
2024-04-11 13:49:40 +02:00
55179ccfed
chore: update ekka to 0.19.3
...
Included updates:
- https://github.com/emqx/mria/pull/178
2024-04-09 19:04:38 +03:00
7bcd553786
test: move test case with rule_engine dep from emqx to emqx_rule_engine
2024-04-09 14:08:46 +02:00
3898950017
test: fix test by adding dependency
2024-04-09 11:38:35 +02:00
03a9c46ca7
fix(sys_topic): format shared topics
2024-04-09 17:06:36 +08:00
47e0f3bb1f
fix(mgmt): $queue shared topics format in mgmt topics api
2024-04-09 17:06:32 +08:00
e11c4a9c83
Merge pull request #12826 from emqx/import-source-bridges
...
fix: source bridges missing after restore the backup files
2024-04-08 16:02:15 +08:00
5479932190
feat(apply rule test): make option to stop action after render work
...
This commit makes the apply rule HTTP API option to stop an action work
for the HTTP action, and adds infrastructure that makes it easy to add
this functionality to other actions.
2024-04-06 17:21:12 +02:00
ef705c2285
feat: add apply rule API, clientid/ruleid tracing for rule and connector
...
This commit adds:
* Support for forwarding the rule id and client id to the connector so
that events such as template rendered successfully can be traced.
* HTTP API for for applying/activating a rule with the given context
2024-04-06 17:20:47 +02:00
70396e9766
Merge pull request #12825 from keynslug/feat/EMQX-12110/repl-meta-api
...
feat(dsrepl): add APIs to manage DB replication sites
2024-04-04 22:32:03 +02:00
ad52f7838e
feat(dsrepl): add APIs to manage DB replication sites
2024-04-04 21:22:01 +02:00
59a442cdb5
feat(rule trace): add support for ruleid as a trace type
2024-04-04 14:55:32 +02:00
319ec50c0d
fix: source bridges missing after restore the backup files
2024-04-03 18:26:51 +08:00
b379f331de
fix(sessds): Handle errors when storing messages
2024-04-02 16:47:41 +02:00
b9ad241658
feat(sessds): Add metrics for the number of persisted messages
2024-04-02 16:45:52 +02:00
d8204021dc
refactor(metrics): Move metrics worker to emqx_utils application
2024-04-02 16:25:04 +02:00
2d074df209
Merge pull request #12797 from keynslug/fix/dsrepl-error-handling
...
fix(dsrepl): handle RPC errors gracefully when storage is down
2024-04-02 13:40:31 +02:00
5759ba5162
chore: bump app version
2024-04-02 17:09:22 +08:00
50bceee9ab
fix(stats): `'subscribers.count'` contains shared-subscriber
2024-04-02 16:56:40 +08:00
0f4b148294
refactor: uniform shared_sub table macros
2024-04-02 16:56:39 +08:00
3eda182e9a
fix: prevent a node from discovering and re-joining the same cluster after it has (manually) left it.
2024-03-28 18:09:27 +02:00
35c43eb8a0
feat(sessds): handle recoverable errors in stream scheduler
2024-03-28 15:17:01 +01:00
zmstone