Commit Graph

237 Commits

Author SHA1 Message Date
Thales Macedo Garitezi cea857cb8e fix(rule_api): fix sql test when testing against events
Fixes https://emqx.atlassian.net/browse/EMQX-11449
2023-12-04 15:24:09 -03:00
Ivan Dyachkov ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
Zaiming (Stone) Shi 14644988e0 chore: change triple-quotes to single-quotes 2023-11-29 16:15:18 +01:00
JianBo He c8b5c51bbc chore: fix failed test cases 2023-11-28 09:53:46 +08:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Ivan Dyachkov 1c57993c91 Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53 2023-11-14 10:44:35 +01:00
Kjell Winblad d26a1b9afb fix(bridge_v1): no hard coded downgrade and upgrade type functions 2023-11-14 09:20:46 +01:00
Ivan Dyachkov 0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
Ivan Dyachkov 0c91bec98d chore: merge 'upstream/release-53' 2023-11-08 09:24:38 +01:00
Thales Macedo Garitezi 0ee74cd8ae fix(rules_api): downgrade bridge id
Fixes https://emqx.atlassian.net/browse/EMQX-11312
2023-11-03 15:57:13 -03:00
Thales Macedo Garitezi c84e4a4187 fix(rule_engine): don't enable a rule that references non-existent bridge 2023-11-03 09:24:53 -03:00
Andrew Mayorov f689d6c233
fix(tpl): ensure backward compat in `emqx_rule_engine`
Missing bindings in string templates will be rendered as "undefined",
as before. Rendering still assumes that missing binding with implicit
default (`undefined`) is an error.

This will also restore complete backward compat in `emqx_prometheus`.
2023-11-02 17:11:11 +07:00
Andrew Mayorov b812f9af5a
feat(tpl): use `emqx_connector_template` in `emqx_rule_engine` app 2023-11-02 17:11:10 +07:00
JimMoen d563121284
refactor: move ?REDISPATCH_TO macro to emqx_mqtt.hrl 2023-10-27 17:54:14 +08:00
Thales Macedo Garitezi 8c4beec6f0 test(rule_engine): add test to exemplify `foreach` with json array payload
Relates to https://emqx.atlassian.net/browse/EMQX-11174
2023-10-19 18:19:42 -03:00
Ilya Averyanov 1eb75b43c4 chore(auth): split emqx_authn and emqx_authz apps 2023-10-05 13:41:50 +03:00
Ivan Dyachkov 105bebc250 chore: merge release-52 into master 2023-09-21 10:22:47 +02:00
zhongwencool 8b6d5d4ca9 test: add test for build_console's args 2023-09-18 16:48:38 +08:00
Zaiming (Stone) Shi ac41f497b4 feat: add bytesize rule sql function 2023-09-15 16:04:21 +02:00
Thales Macedo Garitezi e089fda260
Merge pull request #11568 from thalesmg/republish-props-m-20230905
feat(republish): allow templating mqtt properties
2023-09-07 16:43:00 -03:00
Thales Macedo Garitezi 68293231b8 test: add cases for more complex placeholders 2023-09-07 13:40:03 -03:00
Thales Macedo Garitezi 02b8bbf76a chore: drop templating support for `Subscription-Identifier`
This is highly-dependent on the session state, as is `Topic-Alias`.
2023-09-07 13:40:03 -03:00
Thales Macedo Garitezi 78c5a779d7 feat(republish): allow templating mqtt properties
Fixes https://emqx.atlassian.net/browse/EMQX-10912
2023-09-06 09:54:02 -03:00
Zaiming (Stone) Shi e794143ae1 Merge remote-tracking branch 'origin/release-52' into 0906-sync-release-52-to-master 2023-09-06 09:08:22 +02:00
Ivan Dyachkov 999988cab4
Merge pull request #11401 from kjellwinblad/kjell/fix/mongo_date_format/EMQX-10727
fix: rule SQL mongo_date function should return a string in test mode
2023-08-21 16:20:59 +02:00
Thales Macedo Garitezi 66dec69d09 fix(rule_engine): capture function_clause errors
Fixes https://emqx.atlassian.net/browse/EMQX-10798
2023-08-18 13:51:19 -03:00
lafirest bef8eddad3
Merge pull request #11405 from lafirest/fix/kal_kialo
fix(calendar): make date parse error reason more sense
2023-08-09 14:42:08 +08:00
firest 9f38f5f26a fix(calendar): make date parse error reason more sense 2023-08-08 14:28:38 +08:00
Kjell Winblad 3ed031db70 fix: rule SQL mongo_date function should return a string in test mode
The rule SQL mongo_date function should return a string with the format
ISODate(*), where * is an ISO date string when running the rule in test
mode.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10727
2023-08-07 16:10:30 +02:00
JianBo He 951a96457b
Revert "feat(index): add topic index facility " 2023-07-27 13:42:43 +08:00
JianBo He 5e4855334e
Revert "Fix(topicidx): allow to return matches unique by record id" 2023-07-27 13:39:37 +08:00
zhongwencool 2c193f26e6
Merge pull request #11332 from HJianBo/fix-topic-index-unique-bugs 2023-07-25 19:07:07 +08:00
JianBo He d05a5cfe0f fix(rule): fix the `matches/2` for some edge cases 2023-07-25 14:35:18 +08:00
JianBo He e630331de1 fix(rule): fix a quering problem when 'a/b' and 'a/b/#' exist at the same time.
When using `ets:next` to query the next level of topic words, we should prioritize the next level
of '#', '+'.
2023-07-24 23:04:53 +08:00
JianBo He dcf4819c04 test(rule): add tests to ensure the rules ordering 2023-07-24 19:30:34 +08:00
Andrew Mayorov 6432c9c8fc fix(topicidx): allow to return matches unique by record id 2023-07-24 17:43:11 +08:00
firest af6405fa25 fix(nested_put): fix a data loss bug introduced by #11172 2023-07-24 16:39:37 +08:00
JimMoen c393c2e091
test: ets table cleanup after cases 2023-07-24 09:33:28 +08:00
JimMoen 4e4b1ac115
refactor: module move to app emqx_rule_engine
- Rename to emqx_rule_index.erl
- Remove test funcs from src -> test dir
2023-07-24 09:33:28 +08:00
Thales Macedo Garitezi f9452241bd test(rules): add a few more tests to assert our rule evaluation behavior 2023-07-14 10:23:20 -03:00
Thales Macedo Garitezi 4034bcbb26 fix(rule_runtime): avoid rewriting select clause sources (by @keynslug)
Thanks to @keynslug for the insight and the patch.

With this, we avoid rewriting the parsed select clause sources, and
attempt to reuse the result context when reading values during
evaluation.
2023-07-13 14:46:08 -03:00
Thales Macedo Garitezi bffef386c1 fix(rule_maps): avoid losing data when using `emqx_rule_maps:nested_put`
Fixes https://emqx.atlassian.net/browse/EMQX-10541
2023-07-12 14:28:18 -03:00
firest 902603780b test(ruleengine): add test for the fix of duplicate payload 2023-07-03 18:22:35 +08:00
Stefan Strigler 321fd53132 fix: use ReplyTo in QUERY for async 2023-06-29 16:09:45 +02:00
Stefan Strigler 422c79583a test(emqx_rule_engine): fix simple sync 2023-06-29 16:09:45 +02:00
Stefan Strigler 837acd4234 test(emqx_rule_engine): test rule metrics 2023-06-29 16:09:45 +02:00
Kjell Winblad cf31b65076 fix: rule engine different behavior for div and mod
Previously, the div operation could only be used as an infix operation
while mod could only be used as a function call. After this commit, one
can use both div and mod using function call syntax and infix syntax.

Fixes: https://emqx.atlassian.net/browse/EMQX-10216
2023-06-12 17:20:52 +02:00
Andrew Mayorov a51baaa206
refactor(pluglib): move conversion utils to `emqx_utils_conv` 2023-06-09 14:44:37 +03:00
某文 367a0e312a chore: remove config_path() function 2023-06-02 07:09:08 +08:00
Andrew Mayorov 29fe201676
fix(ruleeng): parse bridge id on create / update
Instead of parsing it every time in the `send_message/2` hot path.
2023-05-31 17:08:42 +03:00
Thales Macedo Garitezi 57aacb471c fix(rule_engine_api): don't crash when formatting empty metrics
Fixes https://emqx.atlassian.net/browse/EMQX-10073
Fixes https://github.com/emqx/emqx/issues/10714#issuecomment-1567987664

Similar issue to https://github.com/emqx/emqx/pull/10743, but on the
rule engine API.
2023-05-30 11:54:27 -03:00
Kjell Winblad 354603da01 test: helper function was interpreted as a test function 2023-05-29 17:20:37 +02:00
Kjell Winblad 995025b572 feat: add timezone_to_second/1 function to rule engine
This commit adds the functions timezone_to_offset_seconds and its alias
timezone_to_second to the rule engine. As the names suggests, these
functions convert a timzone offset string such as "+02:00", "Z", "local"
to an integer representing how many seconds that the given timezone
differs from UTC. timezone_to_offset_seconds is the one of the two
functions that should be advertised while timezone_to_second in kept for
backwards compatibility with 4.X.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10058
2023-05-29 14:30:43 +02:00
Zaiming (Stone) Shi 732a7be187 Merge remote-tracking branch 'origin/release-50' 2023-05-22 17:46:54 +02:00
lafirest 88e4078885
Merge pull request #10747 from lafirest/fix/rule_funs_time_offset
fix: port the `emqx_calendar` from v4.4
2023-05-22 20:52:21 +08:00
firest 087dc59115 test: add test cases for the `rule_test` API 2023-05-22 15:25:35 +08:00
firest 56a6b699ac fix: port the `emqx_calendar` from v4.4 2023-05-22 09:55:31 +08:00
Thales Macedo Garitezi 447b76464b Merge branch 'release-50' into merge-r50-into-v50-a 2023-05-17 14:50:18 -03:00
firest 18043150be fix: cannot access columns exported by FOREACH in DO clause 2023-05-17 10:55:47 +08:00
Ilya Averyanov c113a8ac6a
Merge pull request #10677 from savonarola/0511-fix-rule-api
fix(api): respond 404 on the deletion of nonexistent rule
2023-05-16 16:50:30 +03:00
Zaiming (Stone) Shi 3c64735b87 feat(rule_engine): support external SQL function provider 2023-05-15 08:58:40 +02:00
Ilya Averyanov 49e9ace1c1 fix(api): respond 404 on the deletion of nonexistent rule 2023-05-11 23:12:59 +05:00
Kjell Winblad 0c727fc9b2
Merge pull request #10392 from kjellwinblad/kjell/rule_engine/add_missing_datetime_function/EMQX-9245
freat: add date_to_unix_ts/3 function to the rule engine
2023-04-17 18:09:44 +02:00
Zaiming (Stone) Shi 18974a8e11 refactor: make schema dump and swagger spec work with split desc files 2023-04-17 13:58:11 +02:00
Kjell Winblad 37f42a486c test: improve proper test so it generates more cases 2023-04-17 12:03:13 +02:00
zhongwencool 494ffe86ad
Merge branch 'master' into conf-refactor 2023-04-15 06:40:38 +08:00
Zhongwen Deng 8facd130f6 chore: update cluster_hocon_file/0 function 2023-04-14 23:19:38 +08:00
Stefan Strigler f8e9e54393 refactor: move emqx_json to emqx_utils_json 2023-04-14 13:31:27 +02:00
Kjell Winblad d3ccd8a65d feat: add date_to_unix_ts/3 function to the rule engine
Fixes:
https://emqx.atlassian.net/browse/EMQX-9245
2023-04-13 16:14:03 +02:00
Stefan Strigler f668ad7b9e
Merge pull request #10336 from sstrigler/EMQX-8507-rule-engine-need-a-new-api-to-crud-rule-engines-setting-configs
feat: add `/rule_engine` API endpoint
2023-04-12 11:44:42 +02:00
Stefan Strigler e6f8682c47 fix: ensure we don't return 'rules' in rule_engine 2023-04-11 15:42:56 +02:00
Kjell Winblad 1938882f16
Merge pull request #10318 from kjellwinblad/kjell/feat/rule_engine_from_clause_support_both_string_types
feat(rule engine sql): enable both ' and " strings in FROM clause
2023-04-06 09:19:44 +02:00
Stefan Strigler 9cfe9cc709 fix: don't allow `rules` to be set from `/rule_engine` 2023-04-05 16:28:54 +02:00
Stefan Strigler aea870f319 feat: add `/rule_engine` API endpoint 2023-04-05 15:19:31 +02:00
Kjell Winblad 0e66eb5f3f feat(rule engine sql): enable both ' and " strings in FROM clause
This commit upgrades the rulesql dependency to version 1.5 instead of
1.4. The difference between these two versions is that strings surrounded
by ' and " are supported in FROM clauses in version 1.5, but in version
1.4, only strings surrounded by " are supported.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9243
2023-04-04 10:36:48 +02:00
Thales Macedo Garitezi f3ffc02bff feat(bridges): enable async query mode for all bridges with buffer workers
Fixes https://emqx.atlassian.net/browse/EMQX-9130

Since buffer workers always support async calls ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.

For bridges that currently only allow sync query modes, we should
allow them to be configured with async.  That means basically all
bridge types except Kafka Producer.
2023-04-03 14:49:51 -03:00
Zaiming (Stone) Shi 80eb9d7542
Merge pull request #10252 from emqx/release-50
0327 merge release-50 to master
2023-03-29 12:33:17 +02:00
Thales Macedo Garitezi 1824e7efcc fix(rule_engine): count referenced bridges in `from` clauses as dependencies (rv5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9325

Currently, ingress bridges referenced in the `FROM` clause of rules
are not being accounted as dependencies.

When we try to delete an ingress bridge that's referenced in a rule
like `select * from "$bridges/mqtt:ingress"`, that bridge does not
trigger an UI warning about dependent actions.
2023-03-28 10:31:10 -03:00
Zaiming (Stone) Shi f5e9d3eab2 test: use valid config for jq function timeout 2023-03-28 14:29:24 +02:00
Stefan Strigler 6ebd3dc747 feat(emqx_rule_engine): decompose error tuples
sqltester for instance returns sth like {"...sytnax error...", OrigSql}
2023-03-03 09:59:24 +01:00
Stefan Strigler 64b5e9585e feat(emqx_rule_engine): API format errors in a human readable way 2023-03-02 16:49:11 +01:00
Thales Macedo Garitezi eb7dca3691 chore(rule_engine): reset metrics when disabling a rule
https://emqx.atlassian.net/browse/EMQX-8502

When a bridge is disabled, its metrics are reset.  With this change,
we make rule actions behave like that: disabling a rule will reset its
metrics.
2023-01-04 09:30:47 -03:00
Zaiming (Stone) Shi dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
JimMoen 4906da9796
test(rule-funcs): test compression funcs 2022-12-19 15:43:50 +08:00
Zaiming (Stone) Shi bd65e8aad6 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:42:50 +01:00
Zaiming (Stone) Shi 1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Stefan Strigler 0b324da7cb refactor: move metrics out of /rules(/:id) to /rules/:id/metrics 2022-12-06 15:23:28 +01:00
Zaiming (Stone) Shi cca3421308 refactor: use static function references 2022-12-06 09:40:03 +01:00
Zaiming (Stone) Shi 7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
JianBo He 8a0c468b01 test: refine tests for lots of List HTTP API 2022-11-24 20:14:33 +01:00
Zaiming (Stone) Shi f3df2c80d8 feat: add user_properties arg for republish action 2022-11-22 20:09:59 +01:00
kraftwerk28 00c57de4c3 feat: do not drop MQTTv5 properties in rule/bridge 2022-11-22 20:09:59 +01:00
Zaiming (Stone) Shi 09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
JimMoen 4f00206693
test: different resons for disconnect event 2022-11-08 23:22:58 +08:00
Zaiming (Stone) Shi d704de4a84 fix(emqx_rule_events): drop tuple-value from message headers
the message headers are fed to a JSON ecnoder which
may crash if the header value is a tuple
2022-10-27 20:31:33 +02:00
Shawn 60a90858f8 feat: check dependent actions before removing the bridges 2022-09-06 15:34:23 +08:00
Zhongwen Deng 56417a3130 feat: list rules support for pagination and fuzzy filtering 2022-07-25 09:36:08 +08:00
Shawn 6685a3c5a8 fix: remove the 'headers' field from the rule events 2022-06-17 15:57:10 +08:00
Shawn 55fce33477 fix: add telemetry testcases in all/0 2022-05-25 21:55:28 +08:00
Shawn a157539710 refactor(rule): rename http_bridge -> webhook 2022-05-25 18:56:12 +08:00