yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,526 Commits 393 Branches 669 Tags 74 MiB
528d6b7ef1
Commit Graph

12022 Commits

Author SHA1 Message Date
Kjell Winblad
72b0c63ed2
Merge pull request #13043 from kjellwinblad/kjell/kafka_incorrect_query_type/EMQX-12363 
fix(kafka trace): do not include query type in trace entry
 2024-05-16 10:08:16 +02:00
ieQu1
deffb01544
fix(ds): Change the default number of shards to 12 2024-05-16 08:06:07 +02:00
Ilya Averyanov
552b62236c chore(retainer): move filter/foreach to emqx_utils_stream 2024-05-15 19:09:06 +03:00
Ilya Averyanov
0b39aaadbd chore(retainer): get rid of qlc usage 2024-05-15 19:09:06 +03:00
Thales Macedo Garitezi
2eb1bb2b13
Merge pull request #13032 from kjellwinblad/kjell/no_ctx_message_validation_failed/EMQX-12354 
fix: add handling of message_validation_failed ctx for rule testing
 2024-05-15 11:20:18 -03:00
Kjell Winblad
246bce85ec fix: make mysql action rendered trace include parameters 
When doing non-batch inserts with the MySQL action, prepared statements
are used. We therefore need to include the parameters to the prepared
statement when tracing the rendered templates.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12335
 2024-05-15 14:49:33 +02:00
Kjell Winblad
c72682d81b fix: remove query mode from redis action trace as it only supports sync 
Fixes:
https://emqx.atlassian.net/browse/EMQX-12336
 2024-05-15 14:32:15 +02:00
Ilya Averyanov
78a87ab5a6 fix(banned): verify delayed/taken over/retained messages against clientid_re rules 2024-05-15 14:05:38 +03:00
Kjell Winblad
8cbfc4c0cd fix: evaluate lazy values in normal log formatters when level is debug 
Fixes:
https://emqx.atlassian.net/browse/EMQX-12380
 2024-05-15 11:47:21 +02:00
lafirest
faf3492e58
Merge pull request #13039 from lafirest/fix/rocket_key 
fix(rocketmq): fix a typo and a RocketMQ encoding error
 2024-05-15 15:28:35 +08:00
Andrew Mayorov
3e1e4bab17
Merge pull request #13047 from keynslug/fix/EMQX-12366/api-ds-leave 
feat(api-ds): provide more information on nonexistent site leave
 2024-05-14 23:21:07 +02:00
Thales Macedo Garitezi
bcbeadd4af test: stabilize flaky test 2024-05-14 17:53:29 -03:00
Thales Macedo Garitezi
f062517ccb test: stabilize flaky test 2024-05-14 17:53:29 -03:00
Thales Macedo Garitezi
b0aa7f25aa test: add more context to help debug flaky test 
Ex: https://github.com/emqx/emqx/actions/runs/9084293110/job/24967912514?pr=13050#step:6:18330
 2024-05-14 17:53:29 -03:00
Thales Macedo Garitezi
fd949240c0 feat(schema validation): implement backup restore 
Fixes https://emqx.atlassian.net/browse/EMQX-12346
 2024-05-14 15:58:19 -03:00
Thales Macedo Garitezi
bc5071590b
Merge pull request #13033 from thalesmg/fix-ds-more-client-api-fields-r57-20240513 
fix(ds clients mgmt api): add more fields to disconnected durable sessions
 2024-05-14 15:29:40 -03:00
Thales Macedo Garitezi
7023e6ad96
Merge pull request #13017 from thalesmg/fix-ds-subs-pages-r57-20240510 
fix(subs mgmt api): attempt to return mixed clients ds/non-ds in the same page
 2024-05-14 15:29:29 -03:00
Thales Macedo Garitezi
2a0ac34656 Merge branch 'release-57' into sync-r57-m-20240514 2024-05-14 15:03:04 -03:00
Kjell Winblad
e811f4a392 fix: add handling of message_validation_failed ctx for rule testing 
Before this commit, the backend could not handle the test context for
message_validation_failed events that could be sent from the dashboard
since no handling code for this contexts existed in the backend. This
has now been fixed by adding such handling code.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12354
 2024-05-14 13:07:32 -03:00
Thales Macedo Garitezi
1730a41337
Merge pull request #13046 from thalesmg/refactor-mv-rename-sv-r57-20240514 
refactor: rename `message_validation` to `schema_validation`
 2024-05-14 13:03:12 -03:00
Kjell Winblad
adbc392265
Merge pull request #13027 from kjellwinblad/kjell/fix_trace_json_report_formatting/EMQX-12344 
fix: JSON trace formatter should handle report style log entries
 2024-05-14 18:02:59 +02:00
firest
437a0c0d55 fix(rocketmq): fix a typo and a RocketMQ encoding error 2024-05-14 23:01:31 +08:00
Andrew Mayorov
2f90d15ce6
fix(ds-api): adapt CLI to API changes 2024-05-14 16:55:17 +02:00
Andrew Mayorov
35e360fcbe
feat(api-ds): provide more information on nonexistent site leave 2024-05-14 15:57:41 +02:00
Thales Macedo Garitezi
257dae5211 refactor: rename message_validation to schema_validation 
Request from Product team.

Fixes https://emqx.atlassian.net/browse/EMQX-12383
 2024-05-14 10:32:27 -03:00
lafirest
dc391d2403
Merge pull request #13009 from lafirest/fix/rate_log 
feat(limiter): lift the log level of rate limiter to `warning`
 2024-05-14 21:11:26 +08:00
Thales Macedo Garitezi
15acd86b31
Merge pull request #13034 from thalesmg/test-flaky-mq-r57-20240513 
test(client mgmt api): attempt to fix flaky test
 2024-05-14 09:01:29 -03:00
Thales Macedo Garitezi
e8d2c48d3c
Merge pull request #13018 from thalesmg/fix-postgres-log-spam-r57-20240510 
fix(postgres): reduce log spamming when connection goes down
 2024-05-14 09:00:36 -03:00
Kjell Winblad
f75c7a5cea fix(trace log entry error): better structure and naming 2024-05-14 13:38:41 +02:00
Andrew Mayorov
74ed0f578a
Merge pull request #13042 from keynslug/fix/EMQX-12204/api-ns 
fix(s3-aggreg): disambiguate action schema namespace
 2024-05-14 13:12:34 +02:00
Kjell Winblad
121084b2ff fix(kafka trace): do not include query type in trace entry 
For Kafka that has its internal buffering the on_query_async callback is
used both for sync and async queries (for sync queries the caller waits
for a response directly after calling on_query_async). Therefore, before
this commit, the trace included incorrect query type information. This
has been fixed by removing the query type information from the trace.
This should be okay since the query type is not essential information
and can be derived from the configuration.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12363
 2024-05-14 12:24:30 +02:00
ieQu1
525e4dac95
Merge pull request #13036 from ieQu1/dev/reduce-log-spam 
tests: Reduce log spam in the failed test suites
 2024-05-14 10:53:30 +02:00
ieQu1
f663373c57
Merge pull request #13037 from ieQu1/dev/merge-release57-240513 
Sync release-57 to the master
 2024-05-14 10:50:30 +02:00
Andrew Mayorov
c10a48f4be
fix(s3-aggreg): disambiguate action schema namespace 
Otherwise schemas for `s3` and `s3_aggregated_upload` collide when
projected into OpenAPI schema.
 2024-05-14 10:49:21 +02:00
zmstone
93232d4253 fix(authn/http): log meaningful error message if http header is missing 2024-05-14 10:22:07 +02:00
Andrew Mayorov
98a93662c1
Merge pull request #13035 from keynslug/fix/EMQX-12291/pers-sys-msg 
fix(sessds): persist $SYS messages as well
 2024-05-14 10:05:49 +02:00
firest
7a7bd84cba fix(limiter): improved log content 2024-05-14 15:34:13 +08:00
Ilya Averyanov
bca3782d73 fix(auth_http): fix query encoding 
* ignore authenticator if JSON format is set up for requests, but non-utf8 data is going to be sent
* use application/json format by default
* fix encoding of query part of the requests
 2024-05-14 10:32:53 +03:00
ieQu1
25c6ac3d16
Merge remote-tracking branch 'origin/release-57' 2024-05-13 22:05:29 +02:00
ieQu1
ac3f5a083d
test: Reduce log spam in the failed test suites 2024-05-13 22:00:33 +02:00
ieQu1
8506ca7919
Merge pull request #12998 from ieQu1/dev/improve-latency 
Use leader's clock when calculating LTS cutoff timestamp
 2024-05-13 21:54:06 +02:00
Andrew Mayorov
66b7ac4c45
fix(sessds): persist $SYS messages as well 
Otherwise, persistent sessions will not be able to receive $SYS messages
whatsoever.
 2024-05-13 21:27:55 +02:00
Thales Macedo Garitezi
a2a5fd1f8e test(client mgmt api): attempt to fix flaky test 
```
%%% emqx_mgmt_api_clients_SUITE ==> msgs_plain_encoding.t_mqueue_messages: FAILED
%%% emqx_mgmt_api_clients_SUITE ==>
Failure/Error: ?assertEqual(79, Count)
  expected: 79
       got: 100
      line: 1303
```
 2024-05-13 15:53:21 -03:00
ieQu1
3da3a36863
test(ds): Add generation in the replication suite 2024-05-13 19:51:04 +02:00
ieQu1
9f7ef9f34f
fix(ds): Apply review remarks 2024-05-13 19:35:24 +02:00
Thales Macedo Garitezi
e7c47f4ec0 fix(ds clients mgmt api): add more fields to disconnected durable sessions 
Fixes https://emqx.atlassian.net/browse/EMQX-12369
 2024-05-13 14:25:02 -03:00
Kjell Winblad
1c34a84dd2
Merge pull request #13028 from kjellwinblad/kjell/swagger_example_missing_ruleid_paramter/EMQX-12325 
fix: include ruleid parameter in swagger example for apply rule endpoint
 2024-05-13 17:35:31 +02:00
Thales Macedo Garitezi
81473519bd
Merge pull request #13031 from thalesmg/fix-clientsv2-sec-r57-20240513 
fix(client mgmt api): require authn for endpoint
 2024-05-13 12:17:31 -03:00
Thales Macedo Garitezi
39899b917d fix(client mgmt api): require authn for endpoint 
Fixes https://emqx.atlassian.net/browse/EMQX-12361
 2024-05-13 10:54:21 -03:00
Thales Macedo Garitezi
6b032faebc fix(client mgmt api): return expired durable sessions with is_expired: true 
Fixes https://emqx.atlassian.net/browse/EMQX-12274
 2024-05-13 10:51:24 -03:00
Thales Macedo Garitezi
0dee2d67b1 fix: always check if first query has more rows 2024-05-13 09:10:18 -03:00
Kjell Winblad
c8690a8273 fix: include ruleid parameter in swagger example for apply rule endpoint 
Fixes:
https://emqx.atlassian.net/browse/EMQX-12325
 2024-05-13 12:28:17 +02:00
Kjell Winblad
921d82f083 fix: JSON trace formatter should handle report style log entries 
Fixes:
https://emqx.atlassian.net/browse/EMQX-12344
 2024-05-13 12:11:11 +02:00
zmstone
7faf73e0f4 chore(emqx_schema): change atom array to enum array for alarm.actions 2024-05-13 10:54:33 +02:00
zmstone
290ebe2fc5 fix: deny subscribing to +/# by default ACL 
Prior to this change, EMQX default ACL has a deny rule to reject
subscribing to `#`.
For completeness, the default ACL should also deny `+/#` because
they are essentially equivalent.
 2024-05-13 09:26:42 +02:00
JianBo He
54f6bd181b
Merge pull request #13010 from JimMoen/fix-jt808-reg-failed 
fix(gw): jt808 REG_ACK failed due to faulty clientinfo
 2024-05-13 14:19:18 +08:00
JimMoen
2f212f34a3
Merge pull request #12989 from JimMoen/fix-plugin-config-file-backup 
fix(plugin): add a backup for the plugin config file
 2024-05-11 16:38:36 +08:00
JimMoen
a5ecceb755
chore: apply suggestions from code review, thanks @thalesmg 2024-05-11 11:36:40 +08:00
Thales Macedo Garitezi
e25fe62cbc fix(subs mgmt api): attempt to return mixed clients ds/non-ds in the same page 
Fixes https://emqx.atlassian.net/browse/EMQX-12294
 2024-05-10 14:48:46 -03:00
Kjell Winblad
c6df069c5a
Merge pull request #13012 from kjellwinblad/kjell/fix_incorrect_config_crash/EMQX-12315 
fix: listener crash if access_rules config option is incorrect
 2024-05-10 17:46:40 +02:00
Thales Macedo Garitezi
f690db9849 fix(postgres): reduce log spamming when connection goes down 
Fixes https://emqx.atlassian.net/browse/EMQX-12334

See also: https://github.com/emqx/epgsql/pull/10
 2024-05-10 12:15:01 -03:00
JimMoen
3c7e8a3bd5
test(gw): jt808 register failed 2024-05-10 23:09:22 +08:00
JimMoen
f8fd4d15be
fix(gw): process jt808 client deregister packet 2024-05-10 23:00:59 +08:00
Zaiming (Stone) Shi
c491b83857
Merge pull request #13014 from zmstone/0510-add-is_template-for-kinesis-partition-key 
fix(kinesis): partition_key should be of 'template' type
 2024-05-10 16:11:35 +02:00
Kjell Winblad
b13fa37771 fix: dialyzer warning 2024-05-10 15:51:28 +02:00
Thales Macedo Garitezi
6be4e6f631 Merge branch 'release-57' into sync-r57-m-20240510 2024-05-10 10:23:28 -03:00
Thales Macedo Garitezi
0eaef18391
Merge pull request #13005 from thalesmg/fix-aggreg-sup-tree-r57-20240509 
fix(aggregator): refactor supervision tree
 2024-05-10 10:22:33 -03:00
Kjell Winblad
2bed5894e3 test: add test cases for listeners access_rules validation and split 2024-05-10 14:21:43 +02:00
Thales Macedo Garitezi
3a29696a48 fix(aggregator): refactor supervision tree 
Instead of using a aggregator supervisor with a fixed local name, we should allow
specifying different names so each action app will spawn its own aggregator supervisor.
 2024-05-10 09:06:12 -03:00
zmstone
47df968d4a fix(kinesis): partition_key should be of 'template' type 2024-05-10 13:57:38 +02:00
firest
1fefb421bc feat(limiter): lift the log level of rate limiter to warning 2024-05-10 18:47:55 +08:00
Kjell Winblad
0aeb2cd77f fix: listener crash if access_rules config option is incorrect 
Previously when changing the access_rules configuration option of a
listener to something that was not a valid rule, the listener would
crash. This has now been fixed by the addition of a configuration
validator that checks the access_rules field.

Additionally, a configuration option converter has been added to the
access_rules field so that one can specify several rules in a single
string by using "," (comma) as separator.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12315
 2024-05-10 12:27:04 +02:00
JimMoen
1a6f0c9234
chore: bump gw_jt808 app vsn 2024-05-10 17:13:23 +08:00
JimMoen
b07242515f
fix(gw): jt808 REG_ACK failed due to faulty clientinfo 2024-05-10 17:12:28 +08:00
Ivan Dyachkov
900536712d
Merge pull request #12997 from thalesmg/sync-r57-m-20240508 
sync `release-57` to `master`
 2024-05-10 10:41:30 +02:00
Kjell Winblad
8c0b55a89e
Merge pull request #12992 from kjellwinblad/kjell/fix_rule_unescape_function/EMQX-12313 
fix: make rule engine unescape convert \a to the terminal alarm char
 2024-05-10 09:48:33 +02:00
lafirest
da7b11020a
Merge pull request #13001 from lafirest/fix/sysk_reconn 
fix(sysk): fix that the syskeeper forwarding never reconnecting
 2024-05-10 11:24:28 +08:00
firest
1f3b640a3d chore: update change 2024-05-10 07:29:14 +08:00
Thales Macedo Garitezi
7b80a9aa44 fix(aggregator): namespace aggregator ids with action type 
Otherwise, actions of different types but same names will clash when starting the
aggregator supervision tree.
 2024-05-09 16:02:11 -03:00
Andrew Mayorov
57dda70d2f
Merge pull request #12994 from keynslug/fix/sessds/log-levels 
fix(sessds): use milder log level for regular error conditions
 2024-05-09 18:34:54 +02:00
lafirest
23a1b18459
Merge pull request #12983 from emqx/feat/authn_hook 
feat(events): add new hook && event `client.check_authn_complete`
 2024-05-09 22:36:52 +08:00
Thales Macedo Garitezi
401f0fa84b Merge branch 'release-57' into sync-r57-m-20240508 2024-05-09 09:13:30 -03:00
firest
2bc014db69 fix(events): call client.check_authn_complete even if authentication fails 2024-05-09 18:15:32 +08:00
firest
e3a59c4037 fix(sysk): fix that the syskeeper forwarding never reconnecting 2024-05-09 16:27:58 +08:00
Ilia Averianov
a1aedee446
Merge pull request #12996 from savonarola/0508-fix-cursor-usage 
fix(retainer): fix qlc cursor cleanup
 2024-05-09 08:04:10 +03:00
firest
d2ca4e9f11 chore: update change 2024-05-09 11:04:56 +08:00
ieQu1
07aa708894
test(ds): Refactor replication suite 2024-05-09 03:56:56 +02:00
zhongwencool
107d00adee
Merge pull request #12944 from zhongwencool/mqtt-utf8-strict-mode 
fix: non-utf8 clientid connect event case crash if strict_mod=false
 2024-05-09 09:16:10 +08:00
ieQu1
63e51fca66
test(ds): Use streams to fill the storage 2024-05-09 02:46:57 +02:00
ieQu1
a0a3977043
feat(ds): Assign latest timestamp deterministically 2024-05-08 23:17:57 +02:00
ieQu1
2236af84ba
feat(ds): two-stage storage commit on the storage level 2024-05-08 23:17:57 +02:00
ieQu1
f250f00f3f
chore: Bump snabbkaffe version to 1.0.10 2024-05-08 23:17:57 +02:00
ieQu1
1ddbbca90e
feat(ds): Allow incremental update of the LTS trie 2024-05-08 23:17:57 +02:00
ieQu1
68ca891f41
test(ds): Use streams to create traffic 2024-05-08 23:17:57 +02:00
Zaiming (Stone) Shi
018d79b498
Merge pull request #12993 from zmstone/0508-fix-update-listener-zone-from-dashboard 
0508 fix update listener zone from dashboard
 2024-05-08 22:03:10 +02:00
Thales Macedo Garitezi
55f27281b3
Merge pull request #12991 from thalesmg/refactor-mv-message-type-r57-20240508 
refactor(message validation): rename `message_name` to `message_type`
 2024-05-08 16:58:28 -03:00
Thales Macedo Garitezi
d3905b6336
Merge pull request #12990 from thalesmg/fix-mv-conf-update-r57-20240508 
fix(message validation): don't call `emqx_conf:update` during boot
 2024-05-08 16:58:15 -03:00
Ilya Averyanov
57287f0722 fix(retainer): fix qlc cursor cleanup 2024-05-08 21:38:37 +03:00
Thales Macedo Garitezi
8d0574abf0
Merge pull request #12979 from thalesmg/fix-ps-monitor-topic-count-r57-20240506 
fix(monitor api): count persistent routes and subscriptions
 2024-05-08 15:09:16 -03:00
Andrew Mayorov
d84c180ccb
feat(dsrepl): avoid contacting unreachable ra servers 
Assuming estabilished Erlang distribution channel is a reliable way to
tell whether a remote node is reachable.
 2024-05-08 18:12:13 +02:00