Commit Graph

48 Commits

Author SHA1 Message Date
Kjell Winblad
995025b572 feat: add timezone_to_second/1 function to rule engine
This commit adds the functions timezone_to_offset_seconds and its alias
timezone_to_second to the rule engine. As the names suggests, these
functions convert a timzone offset string such as "+02:00", "Z", "local"
to an integer representing how many seconds that the given timezone
differs from UTC. timezone_to_offset_seconds is the one of the two
functions that should be advertised while timezone_to_second in kept for
backwards compatibility with 4.X.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10058
2023-05-29 14:30:43 +02:00
Thales Macedo Garitezi
6c414ab991 fix(schema_registry): ensure schema_encode output in rule engine is a binary (r5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9981
2023-05-24 09:13:38 -03:00
firest
56a6b699ac fix: port the emqx_calendar from v4.4 2023-05-22 09:55:31 +08:00
Kjell Winblad
7d3367467a
Merge pull request #10408 from kjellwinblad/kjell/rule_engine/add_missing_mongo_date_functions/EMQX-9244
feat: add mongo_date functions to the rule engine
2023-04-20 09:46:45 +02:00
Kjell Winblad
29584ca721 feat: add mongo_date functions to the rule engine
This commit adds mong_date built-in functions to the rule engine
SQL-like language. Corresponding functions already existed in EMQX 4.4
and this commit makes sure that EMQX 5.X also has these functions.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9244
2023-04-18 10:01:35 +02:00
Kjell Winblad
0c727fc9b2
Merge pull request #10392 from kjellwinblad/kjell/rule_engine/add_missing_datetime_function/EMQX-9245
freat: add date_to_unix_ts/3 function to the rule engine
2023-04-17 18:09:44 +02:00
Stefan Strigler
9c11bfce80 refactor: rename emqx_misc to emqx_utils 2023-04-14 13:41:27 +02:00
Stefan Strigler
f8e9e54393 refactor: move emqx_json to emqx_utils_json 2023-04-14 13:31:27 +02:00
Kjell Winblad
a66d01d6f0
style: remove code duplication
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-14 12:48:03 +02:00
Kjell Winblad
d3ccd8a65d feat: add date_to_unix_ts/3 function to the rule engine
Fixes:
https://emqx.atlassian.net/browse/EMQX-9245
2023-04-13 16:14:03 +02:00
Thales Macedo Garitezi
33100ecca6 feat: implement schema registry for 5.0 (avro)
Part of https://emqx.atlassian.net/browse/EMQX-9251

This ports part of the Schema Registry app from 4.x to 5.0.  Here,
only support for Avro is added.  Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
Zaiming (Stone) Shi
dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
JimMoen
5151242583
feat(rule-engine): zip, gzip, zip_compress funcs in rule-sql 2022-12-19 15:43:50 +08:00
Xinyu Liu
866810cea6
Merge pull request #8006 from emqx/copy-of-kjell/jq/timeout
feat(rule_engine): default timeout for jq/2 and jq/3 with timeout
2022-05-23 11:58:40 +08:00
EMQ-YangM
005fce63d1 feat: add rule engine function float2str/2 2022-05-19 16:16:05 +08:00
Kjell Winblad
05032467bd feat(rule_engine): default timeout for jq/2 and jq/3 with timeout
This commit adds a default timeout of 10 seconds to the rule engine's
`jq/2` function, and adds a new function `jq/3` (where the last parameter is
a timeout value). The default timeout can be configured with the setting
"rule_engine.jq_function_default_timeout".

Having a timeout when executing jq code in the rule engine is important
as jq code can potentially run forever. Also, the Erlang jq library
limits the number of jq programs that can execute concurrently so a jq
program that loops forever could potentially also prevent a "non-buggy"
jq program from ever starting.
2022-05-17 19:36:32 +02:00
Shawn
82da4d82be chore: update jq to v0.2.0 2022-05-06 22:20:35 +08:00
Kjell Winblad
74c33cd4e5 feat(rule_engine): add jq function to the rule engine
This commit adds a function to the rule engine that alows users
to transform text or JSON objects using [jq filter programs][1].

[jq][1] is a command line tool that can be used to transform
and filter JSON text using jq's built-in language. The rule engine
function that is added with this commit uses the
[Erlang jq NIF library][2] that wraps the jq C library in an
Erlang NIF function.

[1]: https://stedolan.github.io/jq/
[2]: https://github.com/emqx/jq
2022-05-04 17:09:16 +02:00
Zaiming (Stone) Shi
6b44d96c0d refactor: rename functions to please elvis 2022-04-29 10:52:08 +02:00
Zaiming (Stone) Shi
02c3f87b31 style: reformat all remaining apps 2022-04-27 15:51:18 +02:00
EMQ-YangM
e4b62f3a5f feat: add rule-engine functions 2022-04-15 18:02:24 +08:00
Kian-Meng Ang
fc92e4c8bf docs: fix typos 2022-01-27 12:29:38 +01:00
Zaiming (Stone) Shi
63167cea70 chore: update copyright 2022-01-05 20:55:00 +01:00
Zaiming Shi
6edd862dd5 refactor: no more EMQX_ENTERPRISE compile flag
The compile flag was introduced in EQM X 4.3 series
where CE and EE code was diverged large enough which made
non-practicle to determin edition at runtime.

such approach made testing quite challenging as we'll have to
build with different compile flags inorder to run per-edition
test cases

In this commit, we try to retrieve edition info from EMQX's
description text, (put to PT for fast access) at runtime
so we can test ALL editions from a super-set edition (EE).
2021-11-20 20:41:06 +01:00
lafirest
3524942766 fix: change all ~s in io format to ~ts
we should use unicode to replace latin1
2021-10-12 15:09:24 +08:00
Shawn
420ccf0f51 refactor(rules): change republish as an output 2021-09-26 23:00:08 +08:00
Shawn
f33e28af6d fix(rules): update test cases for emqx_rule_engine_SUITE 2021-09-26 23:00:08 +08:00
Shawn
af295a9b71 refactor(rules): remove resources and actions 2021-09-26 23:00:08 +08:00
JimMoen
9c95557bfc fix(emqx_types): type spec refs does not exist file. 2021-09-17 19:28:22 +08:00
Shawn
1dae970cd3 refactor(rule): move emqx_rule_utils.erl -> emqx_plugin_libs_rule.erl 2021-09-10 11:43:03 +08:00
DDDHuang
8125ec7d08
feat: topic metrics api (#5520) 2021-08-24 10:52:18 +08:00
Turtle
80ef076cee feat(rule_engine): rule sql add null function 2021-05-08 17:15:07 +08:00
Zaiming Shi
6354e75626 chore: update copyrights 2021-04-23 20:43:00 +02:00
Shawn
19a9bab3a4 feat(rule_engine): add rule sql functions for datetime rfc3339 2021-04-22 10:32:44 +08:00
Zaiming Shi
d8e410e770 fix(emqx_rule_engine): missing calls
some calls are for enterprise only
2021-04-11 22:55:39 +02:00
JianBo He
496e5381fb chore(rule-engine): compatible with the lower case hash string 2021-03-25 09:10:05 +08:00
JianBo He
277cab86d3 refactor: improve the speed of hexstr and binary conversions 2021-03-23 18:25:19 +08:00
Shawn
e201484163 fix(rule): incorrect merge for map_get/2,3 and map_put/3 2021-02-22 18:16:24 +08:00
Shawn
c2e1bc039b
feat(rule): add more sql functions #4143 (#4144) 2021-02-20 17:10:10 +08:00
Zaiming Shi
4a877a3a69 chore(apps): relocate some apps
emqx_telemetry is for opensource onlyh
emqx_rule_engiune is common
2021-02-19 11:24:41 +01:00
Zaiming Shi
8f477e8411 refactor(proj): Move opensource apps to lib-opensource
One step closer for better code abstraction for opensorce vs enterprise
So far this relocation is to make opensource to enterprise merges
conflict free.
2021-02-11 16:45:05 +01:00
Shawn
c3159420bb feat(funcs): add bitsize/1 and subbits/2,3,6 functions 2020-12-22 11:18:16 +08:00
Zaiming Shi
7bdd093ef5 Merge remote-tracking branch 'origin/dev/v4.3.0' into umbrella-fix-build 2020-12-11 08:55:57 +01:00
Zaiming Shi
3a91ff8a7b chore(proj): resync apps 2020-12-11 08:19:26 +01:00
Zaiming Shi
3dc11bc9e1 fix(emqx_rule_funcs): Fix strng:pad calls 2020-12-10 11:22:38 +01:00
Shawn
15d048d8ec feat(funcs): add bitsize/1 and subbits/2,3,6 functions
Given following Rule SQL:

```
select
  subbits(payload, 1, 8) as firstByte,
  case
    when firstByte = 123 then 't/{'
    when firstByte = 125 then 't/}'
    else 't/' + str(firstByte)
  end as tp
from "t/#"
```

Input Payload: `{x}`
Output: `{"tp": "t/{", "firstByte": 123}`

Input Payload: `}x`
Output: `{"tp": "t/}", "firstByte": 125}`

Input Payload: `a`
Output: `{"tp": "t/97", "firstByte": 97}`
2020-12-10 11:26:07 +08:00
Zaiming Shi
2fda0a1565 chore(apps): Sync again all apps 2020-12-07 21:30:21 +01:00
Zaiming Shi
686c006d6e refactor(proj): Add apps 2020-12-04 22:11:53 +01:00