Commit Graph

553 Commits

Author SHA1 Message Date
firest 8cd21da94b chore: update apps version 2023-08-16 10:55:54 +08:00
firest b08102269a refactor(calendar): refactor datetime-related code and remove redundant 2023-08-16 10:55:46 +08:00
Andrew Mayorov fe9477f92e
chore: bump applications versions
* emqx_rule_engine 5.0.23
2023-08-14 15:36:58 +04:00
Andrew Mayorov 47dfba4341
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching 2023-08-14 15:36:58 +04:00
lafirest bef8eddad3
Merge pull request #11405 from lafirest/fix/kal_kialo
fix(calendar): make date parse error reason more sense
2023-08-09 14:42:08 +08:00
Kjell Winblad b38461e50a fix: mongo_date/2 shall give user friendly value in the test environment 2023-08-08 14:47:07 +02:00
firest 9f38f5f26a fix(calendar): make date parse error reason more sense 2023-08-08 14:28:38 +08:00
Kjell Winblad 3ed031db70 fix: rule SQL mongo_date function should return a string in test mode
The rule SQL mongo_date function should return a string with the format
ISODate(*), where * is an ISO date string when running the rule in test
mode.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10727
2023-08-07 16:10:30 +02:00
Ivan Dyachkov 63adeabf72 chore: bump app versions 2023-07-27 15:29:03 +02:00
JianBo He 951a96457b
Revert "feat(index): add topic index facility " 2023-07-27 13:42:43 +08:00
JianBo He 5e4855334e
Revert "Fix(topicidx): allow to return matches unique by record id" 2023-07-27 13:39:37 +08:00
zhongwencool 2c193f26e6
Merge pull request #11332 from HJianBo/fix-topic-index-unique-bugs 2023-07-25 19:07:07 +08:00
JianBo He d05a5cfe0f fix(rule): fix the `matches/2` for some edge cases 2023-07-25 14:35:18 +08:00
JianBo He e630331de1 fix(rule): fix a quering problem when 'a/b' and 'a/b/#' exist at the same time.
When using `ets:next` to query the next level of topic words, we should prioritize the next level
of '#', '+'.
2023-07-24 23:04:53 +08:00
JianBo He dcf4819c04 test(rule): add tests to ensure the rules ordering 2023-07-24 19:30:34 +08:00
Andrew Mayorov 0496038361 fix(ruleeng): ensure topic index matched rules evalauted once 2023-07-24 17:44:12 +08:00
Andrew Mayorov 6432c9c8fc fix(topicidx): allow to return matches unique by record id 2023-07-24 17:43:11 +08:00
firest af6405fa25 fix(nested_put): fix a data loss bug introduced by #11172 2023-07-24 16:39:37 +08:00
JimMoen c393c2e091
test: ets table cleanup after cases 2023-07-24 09:33:28 +08:00
JimMoen 4e4b1ac115
refactor: module move to app emqx_rule_engine
- Rename to emqx_rule_index.erl
- Remove test funcs from src -> test dir
2023-07-24 09:33:28 +08:00
Andrew Mayorov b821bdee00
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching 2023-07-24 09:27:31 +08:00
Thales Macedo Garitezi 2531c3e7d1
Merge pull request #11306 from thalesmg/rule-actions-reply-dropped-r51-20230719
fix(rule_metrics): notify rule metrics of late replies and expired requests
2023-07-21 16:51:14 -03:00
Serge Tupchii a1a7c4fab6 fix(emqx_rule_engine): fix typo in `behaviour` module attribute
The type resulted in missing `rule_engine` config after importing data from a backup file.

Fixes: EMQX-10590
2023-07-21 10:33:56 +03:00
Thales Macedo Garitezi eb41b77de4 fix(rule_metrics): notify rule metrics of late replies and expired requests
Fixes https://emqx.atlassian.net/browse/EMQX-10600
2023-07-19 11:39:28 -03:00
Thales Macedo Garitezi ab5fd1e5c3
Merge pull request #11260 from thalesmg/fix-rule-maps-nested-put-r51
fix(rule_maps): avoid losing data when using `emqx_rule_maps:nested_put`
2023-07-14 15:02:24 -03:00
Thales Macedo Garitezi f9452241bd test(rules): add a few more tests to assert our rule evaluation behavior 2023-07-14 10:23:20 -03:00
Thales Macedo Garitezi 4034bcbb26 fix(rule_runtime): avoid rewriting select clause sources (by @keynslug)
Thanks to @keynslug for the insight and the patch.

With this, we avoid rewriting the parsed select clause sources, and
attempt to reuse the result context when reading values during
evaluation.
2023-07-13 14:46:08 -03:00
Thales Macedo Garitezi bffef386c1 fix(rule_maps): avoid losing data when using `emqx_rule_maps:nested_put`
Fixes https://emqx.atlassian.net/browse/EMQX-10541
2023-07-12 14:28:18 -03:00
Thales Macedo Garitezi 541d03a0ba refactor(schema_registry): refactor schema registry app and modules
Fixes https://emqx.atlassian.net/browse/EMQX-10361

- Moves `lib-ee/emqx_ee_schema_registry` to `apps/emqx_schema_registry`.
- Removes the `_ee_` segment from module names.
  - Exceptions are the table names which are kept to avoid backwards incompatibilities.
2023-07-12 09:24:15 -03:00
Ilya Averyanov 19f9fc5089 feat(authz): bump app versions 2023-07-07 19:41:34 +03:00
Ilya Averyanov 7de26a1776 feat(authz): use extensible map format for actions in authz rules
* support authorization on retain, qos fields
* refactored authz tests heavily
2023-07-07 19:40:34 +03:00
lafirest bf5167a8bf
Merge pull request #11172 from lafirest/fix/dup_payload
fix(ruleengine): fix duplicate `payload` problems
2023-07-05 14:20:26 +08:00
Kjell Winblad 33cb29efdb
Merge pull request #11169 from kjellwinblad/kjell/improved_sparkplugb/EMQX-10351
feat: add Sparkplug encode and decode functions to the rule engine
2023-07-04 07:24:06 +02:00
firest 902603780b test(ruleengine): add test for the fix of duplicate payload 2023-07-03 18:22:35 +08:00
firest bd4f207365 fix(ruleengine): fix duplicate `payload` problems 2023-07-03 14:21:56 +08:00
Stefan Strigler 07cf250093
Merge pull request #11126 from sstrigler/EMQX-8842-fix-rule-metrics
fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun
2023-06-30 20:07:23 +02:00
Kjell Winblad 714363bd01 fix: decouple emqx_rule_engine application from emqx_ee_schema_registry
This commit decouples the emqx_rule_engine application from the
emqx_ee_schema_registry application by making it possible to register a
callback module that defines extra rule engine SQL functions instead of
calling a module in emqx_ee_schema_registry directly from the
emqx_rule_engine application.
2023-06-30 16:56:16 +02:00
Kjell Winblad 039e27a153 feat: add Sparkplug encode and decode functions to the rule engine
Fixes:
https://emqx.atlassian.net/browse/EMQX-10429
2023-06-30 10:06:56 +02:00
firest 872698d19e feat(ruleengine): port random && uuid_v4 functions from v4 2023-06-30 14:38:27 +08:00
Stefan Strigler 14da61194e fix: handle resource errors 2023-06-29 16:09:45 +02:00
Stefan Strigler 321fd53132 fix: use ReplyTo in QUERY for async 2023-06-29 16:09:45 +02:00
Stefan Strigler d71d70c889 fix: allow to pass in Opts rather than just a ReplyTo 2023-06-29 16:09:45 +02:00
Stefan Strigler 422c79583a test(emqx_rule_engine): fix simple sync 2023-06-29 16:09:45 +02:00
Stefan Strigler 837acd4234 test(emqx_rule_engine): test rule metrics 2023-06-29 16:09:45 +02:00
Stefan Strigler ae636a52d7 fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun 2023-06-29 16:09:45 +02:00
Zaiming (Stone) Shi 7cf8a6c892 chore: bump app vsns 2023-06-21 16:36:51 +02:00
Kjell Winblad 59ed8798c3
Merge pull request #11026 from kjellwinblad/kjell/fix/rule_engine/div_mod_strange/EMQX-10216
fix: rule engine different behavior for div and mod
2023-06-17 07:20:37 +02:00
zhongwencool 3331af4757 feat: make gateway and slow_subs low level and hide rule_engine 2023-06-13 11:47:38 +08:00
Kjell Winblad cf31b65076 fix: rule engine different behavior for div and mod
Previously, the div operation could only be used as an infix operation
while mod could only be used as a function call. After this commit, one
can use both div and mod using function call syntax and infix syntax.

Fixes: https://emqx.atlassian.net/browse/EMQX-10216
2023-06-12 17:20:52 +02:00
Zaiming (Stone) Shi 97850de524 Merge remote-tracking branch 'origin/release-51' into 0610-merge-release-51-to-master 2023-06-10 12:23:55 +02:00
Andrew Mayorov a51baaa206
refactor(pluglib): move conversion utils to `emqx_utils_conv` 2023-06-09 14:44:37 +03:00
Andrew Mayorov 7d0abb6146
feat(emqx): add `emqx_topic:match_any/2` utility 2023-06-09 14:44:37 +03:00
Andrew Mayorov d6c1ee183f
refactor(pluglib): move `emqx_placeholder` to utils app
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
Serge Tupchii e4d09d4ad4 feat: implement configuration and user data export/import CLI
Closes: EMQX-9203
2023-06-09 14:11:47 +03:00
Ivan Dyachkov 711cedc282 fix: missing 404 response in rule engine api schema 2023-06-08 11:46:16 +02:00
Andrew Mayorov 79d430cf03
refactor(pluglib): move `emqx_plugin_libs_proto_v1` into emqx app
And rename it to `emqx_metrics_proto_v1` in the process.
2023-06-07 18:30:12 +03:00
Thales Macedo Garitezi 46393343e2 chore: use `timeout_duration` types for timer fields
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
某文 367a0e312a chore: remove config_path() function 2023-06-02 07:09:08 +08:00
某文 2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
某文 c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
Andrew Mayorov 29fe201676
fix(ruleeng): parse bridge id on create / update
Instead of parsing it every time in the `send_message/2` hot path.
2023-05-31 17:08:42 +03:00
Thales Macedo Garitezi 57aacb471c fix(rule_engine_api): don't crash when formatting empty metrics
Fixes https://emqx.atlassian.net/browse/EMQX-10073
Fixes https://github.com/emqx/emqx/issues/10714#issuecomment-1567987664

Similar issue to https://github.com/emqx/emqx/pull/10743, but on the
rule engine API.
2023-05-30 11:54:27 -03:00
Thales Macedo Garitezi 04bd39861d chore: bump app vsns 2023-05-30 11:08:20 -03:00
Kjell Winblad 354603da01 test: helper function was interpreted as a test function 2023-05-29 17:20:37 +02:00
Kjell Winblad 995025b572 feat: add timezone_to_second/1 function to rule engine
This commit adds the functions timezone_to_offset_seconds and its alias
timezone_to_second to the rule engine. As the names suggests, these
functions convert a timzone offset string such as "+02:00", "Z", "local"
to an integer representing how many seconds that the given timezone
differs from UTC. timezone_to_offset_seconds is the one of the two
functions that should be advertised while timezone_to_second in kept for
backwards compatibility with 4.X.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10058
2023-05-29 14:30:43 +02:00
Zaiming (Stone) Shi 36e268c933 chore: bump app versions 2023-05-26 16:05:37 +02:00
Zaiming (Stone) Shi cc5b4d3748 Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test 2023-05-26 15:58:16 +02:00
Paulo Zulato ea86f4442b fix: avoid error 500 when node is re-joining cluster
Fixes https://emqx.atlassian.net/browse/EMQX-9899
2023-05-25 13:32:28 -03:00
Thales Macedo Garitezi 324459990f Merge branch 'release-50' into merge-r50-into-v50-20230524 2023-05-24 12:54:15 -03:00
Thales Macedo Garitezi 6c414ab991 fix(schema_registry): ensure `schema_encode` output in rule engine is a binary (r5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9981
2023-05-24 09:13:38 -03:00
Zaiming (Stone) Shi 732a7be187 Merge remote-tracking branch 'origin/release-50' 2023-05-22 17:46:54 +02:00
lafirest 88e4078885
Merge pull request #10747 from lafirest/fix/rule_funs_time_offset
fix: port the `emqx_calendar` from v4.4
2023-05-22 20:52:21 +08:00
firest 087dc59115 test: add test cases for the `rule_test` API 2023-05-22 15:25:35 +08:00
firest 56a6b699ac fix: port the `emqx_calendar` from v4.4 2023-05-22 09:55:31 +08:00
firest fa799e95a4 fix: supports test the `$events/delivery_dropped` event by API 2023-05-19 14:20:26 +08:00
Thales Macedo Garitezi 447b76464b Merge branch 'release-50' into merge-r50-into-v50-a 2023-05-17 14:50:18 -03:00
firest 1f7ede90a4 chore: update app version && changes 2023-05-17 11:01:31 +08:00
firest 18043150be fix: cannot access columns exported by FOREACH in DO clause 2023-05-17 10:55:47 +08:00
Ilya Averyanov c113a8ac6a
Merge pull request #10677 from savonarola/0511-fix-rule-api
fix(api): respond 404 on the deletion of nonexistent rule
2023-05-16 16:50:30 +03:00
Zaiming (Stone) Shi a0454b9417
Merge pull request #10674 from zmstone/0511-feat-support-rule-engine-eval
Add an experimental feature: support external rule-engine SQL functions
2023-05-16 10:55:00 +02:00
Zaiming (Stone) Shi 3c64735b87 feat(rule_engine): support external SQL function provider 2023-05-15 08:58:40 +02:00
JianBo He 383fec9dfc docs: fix invalid links 2023-05-15 11:18:09 +08:00
Ilya Averyanov 49e9ace1c1 fix(api): respond 404 on the deletion of nonexistent rule 2023-05-11 23:12:59 +05:00
某文 6056b0e7a8 fix: ensure atom key for emqx_config:get 2023-05-11 17:40:03 +08:00
Zhongwen Deng bcc8f4313b chore: make plugins config to low level 2023-04-20 18:00:19 +08:00
Kjell Winblad 7d3367467a
Merge pull request #10408 from kjellwinblad/kjell/rule_engine/add_missing_mongo_date_functions/EMQX-9244
feat: add mongo_date functions to the rule engine
2023-04-20 09:46:45 +02:00
Ivan Dyachkov dc78ecb41c chore: merge upstream/master 2023-04-18 17:33:32 +02:00
某文 fbadfc06e4 feat: change exhook, rule_engine, bridge to low importance level instead of hidden 2023-04-18 20:15:19 +08:00
Kjell Winblad 29584ca721 feat: add mongo_date functions to the rule engine
This commit adds mong_date built-in functions to the rule engine
SQL-like language. Corresponding functions already existed in EMQX 4.4
and this commit makes sure that EMQX 5.X also has these functions.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9244
2023-04-18 10:01:35 +02:00
Kjell Winblad 0c727fc9b2
Merge pull request #10392 from kjellwinblad/kjell/rule_engine/add_missing_datetime_function/EMQX-9245
freat: add date_to_unix_ts/3 function to the rule engine
2023-04-17 18:09:44 +02:00
Zaiming (Stone) Shi 18974a8e11 refactor: make schema dump and swagger spec work with split desc files 2023-04-17 13:58:11 +02:00
Kjell Winblad 37f42a486c test: improve proper test so it generates more cases 2023-04-17 12:03:13 +02:00
Ivan Dyachkov 9fc8a498f8 chore: bump apps versions 2023-04-17 09:09:08 +02:00
zhongwencool 494ffe86ad
Merge branch 'master' into conf-refactor 2023-04-15 06:40:38 +08:00
Zhongwen Deng 8facd130f6 chore: update cluster_hocon_file/0 function 2023-04-14 23:19:38 +08:00
Stefan Strigler 062ce5f819 refactor: rename emqx_map_lib to emqx_utils_maps 2023-04-14 13:41:34 +02:00
Stefan Strigler 9c11bfce80 refactor: rename emqx_misc to emqx_utils 2023-04-14 13:41:27 +02:00
Stefan Strigler f8e9e54393 refactor: move emqx_json to emqx_utils_json 2023-04-14 13:31:27 +02:00
Kjell Winblad a66d01d6f0
style: remove code duplication
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-14 12:48:03 +02:00
Kjell Winblad d3ccd8a65d feat: add date_to_unix_ts/3 function to the rule engine
Fixes:
https://emqx.atlassian.net/browse/EMQX-9245
2023-04-13 16:14:03 +02:00