yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,759 Commits 393 Branches 684 Tags 74 MiB
6bfddd9952
Commit Graph

244 Commits

Author SHA1 Message Date
JimMoen
dff31b293c
perf(utils_sql): pre-build regexp 2024-06-06 11:01:38 +08:00
JimMoen
144264e0d8
test: add utils_sql test cases 2024-06-06 00:30:18 +08:00
JimMoen
627b8c45d2
fix(utils_sql): improve insert sql regular expression 2024-06-06 00:30:17 +08:00
Thales Macedo Garitezi
ae0379f974
Merge pull request #13181 from thalesmg/force-stop-connector-r57-20240604 
fix(resource manager): force kill process if stuck when stopping/removing
 2024-06-05 09:08:30 -03:00
Thales Macedo Garitezi
3013189cd7 fix(resource manager): force kill process if stuck when stopping/removing 
Fixes https://emqx.atlassian.net/browse/EMQX-12357
 2024-06-04 11:38:24 -03:00
zmstone
e3ed7b59dd feat(redis): add a rule function to help formatting redis args 
The new function named 'map_to_redis_hset_args' can be used to format
a map's key-value pairs into redis HSET (or HMSET) arg list.

This new function is dedicated for redis to avoid abuse for other
data integrations.
 2024-06-03 14:09:15 +02:00
zmstone
238c207b09 chore: bump app versions 2024-05-27 20:26:52 +02:00
zmstone
41cbfcfaa1 Merge remote-tracking branch 'origin/release-57' into 0527-sync-5.7.0-to-master 2024-05-27 20:10:44 +02:00
Ilya Averyanov
327080e70e perf(emqx_placeholder): do not try to decode maps as json 
fix
 2024-05-24 20:08:31 +03:00
ieQu1
acb19a06cf Merge remote-tracking branch 'origin/release-57' 2024-05-21 15:04:46 +02:00
Ilia Averianov
22d5f17de8
Merge pull request #13025 from savonarola/0513-fix-cursor-usage 
Get rid of qlc in retainer
 2024-05-16 17:06:53 +03:00
Ilya Averyanov
1a664c941b chore(retainer): scan table in batches, improve stream usage 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2024-05-16 16:06:11 +03:00
Thales Macedo Garitezi
34a29e6363 feat(schema validation): implement support for ctl conf load 2024-05-16 09:55:42 -03:00
Ilia Averianov
322989c83f
Merge pull request #13040 from savonarola/0513-fix-http-authn-error-handling 
fix(auth_http): fix query encoding
 2024-05-16 15:12:57 +03:00
Ilya Averyanov
e1ce6377f3 chore(streams): add stream methods, optimize streams 2024-05-16 12:04:08 +03:00
Ilya Averyanov
daf2e5a444 chore(auth_http): unify http request generation 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2024-05-16 11:36:52 +03:00
Ilya Averyanov
552b62236c chore(retainer): move filter/foreach to emqx_utils_stream 2024-05-15 19:09:06 +03:00
Ilya Averyanov
bca3782d73 fix(auth_http): fix query encoding 
* ignore authenticator if JSON format is set up for requests, but non-utf8 data is going to be sent
* use application/json format by default
* fix encoding of query part of the requests
 2024-05-14 10:32:53 +03:00
ieQu1
25c6ac3d16
Merge remote-tracking branch 'origin/release-57' 2024-05-13 22:05:29 +02:00
ieQu1
8506ca7919
Merge pull request #12998 from ieQu1/dev/improve-latency 
Use leader's clock when calculating LTS cutoff timestamp
 2024-05-13 21:54:06 +02:00
ieQu1
9f7ef9f34f
fix(ds): Apply review remarks 2024-05-13 19:35:24 +02:00
Thales Macedo Garitezi
6be4e6f631 Merge branch 'release-57' into sync-r57-m-20240510 2024-05-10 10:23:28 -03:00
ieQu1
63e51fca66
test(ds): Use streams to fill the storage 2024-05-09 02:46:57 +02:00
ieQu1
a0a3977043
feat(ds): Assign latest timestamp deterministically 2024-05-08 23:17:57 +02:00
ieQu1
68ca891f41
test(ds): Use streams to create traffic 2024-05-08 23:17:57 +02:00
Kjell Winblad
916168c755 fix: make rule engine unescape convert \a to the terminal alarm char 
The rule engine unescape function should convert the escape sequence \a
to the alarm bell symbol (ASCII 7). This bug was created as it was
assumed that Erlang convert $\a to 7 but Erlang does not handle the \a
escape sequence and instead convert it to 97 (ASCII code for a).

Fixes:
https://emqx.atlassian.net/browse/EMQX-12313
 2024-05-08 15:18:13 +02:00
Ilya Averyanov
31026d51f7 chore(auth,http): cache REs for parsing URIs 2024-05-06 11:46:18 +03:00
Ilya Averyanov
3b655f56cb fix(auth,http): improve URI handling 2024-05-04 09:47:13 +03:00
Thales Macedo Garitezi
dd6566f3c5
Merge pull request #12948 from thalesmg/fix-http-bridge-header-update-obfuscate-r57-20240429 
fix(http connector): deobfuscate sensitive headers
 2024-04-30 13:34:15 -03:00
zmstone
1974ec15ec fix(client_attrs): fix client_attrs extraction loop 2024-04-30 12:12:35 +02:00
zmstone
c8d6976b14 feat: add conditions to variform expressions 
- refactored `coalesce` function to allow lazy evaluation
- added `iif(Cond, IfExpr, EleseExpr)` to allow simple conditions
 2024-04-30 07:28:27 +02:00
Thales Macedo Garitezi
a847389159 fix(http connector): deobfuscate sensitive headers 
Fixes https://emqx.atlassian.net/browse/EMQX-12213
 2024-04-29 15:20:46 -03:00
zmstone
f80d078de3 feat(variform): Add more functions 2024-04-15 16:56:52 +02:00
zmstone
b76b6fbe63 feat(variform): initialize client_attrs with variform 
Moved regular expression extraction as a variform function.
 2024-04-14 10:13:24 +02:00
zmstone
e6330dddec fix(variform): allow numbers to be numbers 2024-04-10 11:57:45 +02:00
Zaiming (Stone) Shi
2fea651d1d
Merge pull request #12851 from zmstone/0327-feat-add-emqx_variform 
emqx_variform for string substitution and transform
 2024-04-10 10:18:40 +02:00
zmstone
53b78086ed chore: fix xref checks 2024-04-09 09:34:05 +02:00
zmstone
41677eb785 refactor: make elvis happy 2024-04-08 21:25:58 +02:00
zmstone
bf12efac6d fix(variform): add basic tests 2024-04-08 21:08:43 +02:00
Thales Macedo Garitezi
069cd4fbb4
Merge pull request #12812 from thalesmg/async-res-manager-health-check-m-20240328 
feat(resource manager): perform non-blocking health checks
 2024-04-04 09:07:02 -03:00
zmstone
0e79b543cf refactor: move variform to emqx_utils 2024-04-04 11:10:56 +02:00
ieQu1
ae5935e7f7
test(ds): Attempt to stabilize metrics_worker tests in CI 2024-04-02 19:14:10 +02:00
ieQu1
d8204021dc
refactor(metrics): Move metrics worker to emqx_utils application 2024-04-02 16:25:04 +02:00
Thales Macedo Garitezi
bade09b56e feat(resource manager): perform non-blocking resource health checks 
Fixes https://emqx.atlassian.net/browse/EMQX-12015

This introduces only _resource_ non-blocking health checks.  _Channel_ non-blocking health
checks may be introduced later.
 2024-04-01 14:46:15 -03:00
Ivan Dyachkov
db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Ivan Dyachkov
f4446ec680 Merge remote-tracking branch 'upstream/release-56' into 0328-sync-release-56 2024-03-28 09:59:54 +01:00
zmstone
22838f027a fix: mountpoint template render should not replace unknown as undefined 
For backward compatibility, the unknown vars used in mountpoint
is kept unchanged.
e.g. '${unknown}/foo/bar' should be rendered as '${unknown}/foo/bar'
but not 'undefined/foo/bar'
 2024-03-23 10:16:05 +01:00
zmstone
5e9814d171 fix: add debug level logging for invalid client attributes 2024-03-23 10:16:05 +01:00
zmstone
c75840306b fix: restrict client_attr key and value string format 
The keys and values are used to render templates for
authz rules, such as topic names, and SQL statements etc.
 2024-03-23 10:16:02 +01:00
zmstone
9ec99fef4a feat: allow client_attr used in authz rules 2024-03-23 10:16:02 +01:00
Serge Tupchii
d2a1a7f7cf chore: rename message_queue_too_long error reason to mailbox_overflow 
`mailbox_overflow` is consistent with the corresponding config parameter:
 'force_shutdown.max_mailbox_size'
 2024-03-22 12:20:20 +02:00
Serge Tupchii
cb5fdb3c79 fix: rework In-flight / Mqueue API 
- use timestamp 'position' to find the next chunk of data
- add 'start' position to response meta
- sort In-flight messages by insertion time
- sort Mqueue messages by priority
 2024-03-20 15:25:33 +02:00
Andrew Mayorov
d30c99512a
feat(utils-stream): add a few more stream combinators 2024-03-19 20:21:54 +01:00
Ivan Dyachkov
923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56 
sync release-56
 2024-03-19 19:09:41 +01:00
Thales Macedo Garitezi
f84a996671 feat: implement message validation 
Fixes https://emqx.atlassian.net/browse/EMQX-11980
 2024-03-18 13:11:39 -03:00
Shawn
29111a2192 fix: dialyzer problems 2024-03-08 14:25:19 +08:00
Shawn
163d095dca fix: port the changes for date_to_unix_ts SQL fun from 4.4 2024-03-08 10:48:08 +08:00
zmstone
f57f2fa1b7 chore: bump app version numbers 2024-03-06 19:37:06 +01:00
zmstone
9929025820 fix(rule_func): time zone shift at wrong precision 2024-03-04 21:25:49 +01:00
Thales Macedo Garitezi
577ef41e45 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 12:48:06 -03:00
lafirest
79f0720d14
Merge pull request #12635 from lafirest/fix/r551 
fix(http): fix that sensitive headers may be printed in log when querying
 2024-03-04 22:32:14 +08:00
Thales Macedo Garitezi
0dd9990f24 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 09:44:30 -03:00
firest
c5eb09a72f fix(http): fix that sensitive headers may be printed in log when querying 2024-03-04 20:36:01 +08:00
JimMoen
0a33f9d027
fix(calendar): leap year time to unix timestamp 2024-03-04 15:37:49 +08:00
firest
1c1c4e497d chore: bump version && update change 2024-03-01 12:42:19 +08:00
firest
5a3a34cce7 fix(redact): enhanced the redact for sensitive headers 2024-03-01 12:30:26 +08:00
firest
0670272188 fix: Revert "fix: redact all headers from logs" 
This reverts commit d8032f47ca.
 2024-03-01 10:32:57 +08:00
Andrew Mayorov
367ffa8e80
feat(utils-fs): add function to compute relpaths naively 2024-02-27 23:41:50 +01:00
Thales Macedo Garitezi
d8032f47ca fix: redact all headers from logs 
Fixes https://emqx.atlassian.net/browse/EMQX-11904

Since headers are usually used for authentication and the headers used for that are very
flexible, we redact all headers from logs to avoid leaking anything.
 2024-02-26 18:04:12 -03:00
Zaiming (Stone) Shi
00c2aeb1ab refactor: delete mfa from log metadata 2024-02-26 12:15:00 +01:00
Zaiming (Stone) Shi
46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
Andrew Mayorov
ae8f59979d
fix(utils): handle improper lists as well in redact/1 2024-02-22 13:35:11 +01:00
Thales Macedo Garitezi
d469f4158e chore: bump app vsns 2024-02-20 16:53:57 -03:00
zhongwencool
e9c8446d57 feat: upgrade erlfmt to support maybe syntax 2024-01-31 20:06:49 +08:00
JianBo He
d12335c4c6 chore: add tests 2024-01-29 18:10:42 +08:00
JianBo He
ab99a17c99 chore(utils_stream): simplfy the csv reader implementation 2024-01-29 11:10:50 +08:00
JianBo He
2e35024df1 test: update eunit tests 2024-01-29 10:49:07 +08:00
JianBo He
9915c85b0b chore(authn_mnesia): use emqx_utils_stream module to imporve reusability 2024-01-29 10:49:07 +08:00
Serge Tupchii
d3a6870097 feat(emqx_utils): add pforeach/2,3 2024-01-19 21:44:31 +02:00
Serge Tupchii
54457b7093 feat(emqx_utils): allow infinity timeout in pmap/3 2024-01-19 19:34:04 +02:00
Andrew Mayorov
eca2f972bc
Merge pull request #12237 from keynslug/fix/EMQX-11483/broker 
feat(routing): add route sync process pool
 2024-01-12 14:44:31 +01:00
Andrew Mayorov
f92b5b3f32
feat(stream): add simple stream over process message queue 2024-01-12 11:57:23 +01:00
Thales Macedo Garitezi
79a4a041e4 fix(bridge_api): correctly deobfuscate secrets during dry run 
Fixes https://emqx.atlassian.net/browse/EMQX-11733
 2024-01-11 15:42:14 -03:00
Ivan Dyachkov
06117c3a33 Merge remote-tracking branch 'upstream/release-54' into 0105-sync-r54 2024-01-05 14:20:38 +01:00
zhongwencool
90404b62a4 fix: incorrectly attempt to update the field value when the previous field don't have value 2024-01-04 10:12:34 +08:00
Zaiming (Stone) Shi
23ded313ec chore: update app versions 2023-12-22 15:29:22 +01:00
Zaiming (Stone) Shi
891fd972bd Merge remote-tracking branch 'origin/release-54' into 1221-prepare-bpapi-on-otp-26 2023-12-22 15:19:25 +01:00
zhongwencool
a16bce8c24 fix: deobfuscate ft's secret keys in api 2023-12-22 13:13:44 +01:00
Zaiming (Stone) Shi
20543d55ef chore: bump app vsn 2023-12-22 13:13:30 +01:00
Zaiming (Stone) Shi
322b7bb7d2 chore: bump app vsn 2023-12-22 13:00:37 +01:00
Thales Macedo Garitezi
cf9331a95f Merge branch 'release-54' into sync-r54-m-20231218 2023-12-18 17:21:08 -03:00
Thales Macedo Garitezi
2c61b2bfbb fix(mysql_bridge): forbid update statements with batch operations 
Fixes https://emqx.atlassian.net/browse/EMQX-11605
 2023-12-18 15:05:14 -03:00
Zaiming (Stone) Shi
35504bd323 refactor: move ntoa (ip address formatting) code to emqx_utils 2023-12-17 21:03:16 +01:00
zhongwencool
d9e179b364 fix: use microsecond precision as the primary key for audit logs to prevent duplication 2023-12-15 09:07:04 +08:00
Stefan Strigler
be31486983 fix(emqx_utils): use deep_merge in unindent 2023-12-11 14:40:24 +01:00
Stefan Strigler
5d7ae4b980
Merge pull request #12132 from sstrigler/EMQX-11154-bridge-v-2-my-sql-support 
feat(emqx_bridge_mysql): port to shared connectors
 2023-12-11 13:56:34 +01:00
Zaiming (Stone) Shi
ddbb8560fa fix(dialyzer): batch 2 2023-12-08 17:59:55 +01:00
Stefan Strigler
71607aa2ad feat(emqx_bridge_mysql): port to shared connectors 2023-12-08 16:15:18 +01:00
Zaiming (Stone) Shi
423b586c56 fix(dialyzer): fix some dialyzer issues found on otp 26 2023-12-06 20:32:49 +01:00
Zaiming (Stone) Shi
de61d9d609 chore: upgrade to jiffy from 1.0.5 to 1.0.6 
1.0.5 has an unexported type which causes dialyzer to fail on otp 26
 2023-12-06 19:18:15 +01:00