Commit Graph

345 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
eb7dca3691 chore(rule_engine): reset metrics when disabling a rule
https://emqx.atlassian.net/browse/EMQX-8502

When a bridge is disabled, its metrics are reset.  With this change,
we make rule actions behave like that: disabling a rule will reset its
metrics.
2023-01-04 09:30:47 -03:00
Zaiming (Stone) Shi
dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
JimMoen
5151242583
feat(rule-engine): zip, gzip, zip_compress funcs in rule-sql 2022-12-19 15:43:50 +08:00
Zaiming (Stone) Shi
bd65e8aad6 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:42:50 +01:00
Zaiming (Stone) Shi
1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Stefan Strigler
0b324da7cb refactor: move metrics out of /rules(/:id) to /rules/:id/metrics 2022-12-06 15:23:28 +01:00
Zaiming (Stone) Shi
cca3421308 refactor: use static function references 2022-12-06 09:40:03 +01:00
Zaiming (Stone) Shi
b398617614 chore: bump app versions 2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi
7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
JianBo He
edb35c08a8 chore: refactor ms2qs function type 2022-11-24 20:14:33 +01:00
JianBo He
9786a6c267 refactor(mgmt): convert fuzzy filter func to named func 2022-11-24 20:14:33 +01:00
JianBo He
b9c5a5f822 fix(delayed): return correct node name 2022-11-24 20:14:33 +01:00
JianBo He
8a0c468b01 test: refine tests for lots of List HTTP API 2022-11-24 20:14:33 +01:00
JianBo He
1fe9c105aa refactor(mgmt): smplify the node_query/cluster_query implementation 2022-11-24 20:14:33 +01:00
JianBo He
08121e7df6 fix(mgmt): optimize the speed of query tail pages
In the previous, when you query the tail pages, all the front of rows
will be queried out and formatted. It greatly hurts the speed of query.

Currently, we only format the final result rows. i.e, the query for the
last page of data will be 10x faster.
2022-11-24 20:14:33 +01:00
Zaiming (Stone) Shi
f3df2c80d8 feat: add user_properties arg for republish action 2022-11-22 20:09:59 +01:00
kraftwerk28
00c57de4c3 feat: do not drop MQTTv5 properties in rule/bridge 2022-11-22 20:09:59 +01:00
Zaiming (Stone) Shi
c940b901f5 chore: fix app versions 2022-11-16 16:26:43 +01:00
Zaiming (Stone) Shi
09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
Zaiming (Stone) Shi
d704de4a84 fix(emqx_rule_events): drop tuple-value from message headers
the message headers are fed to a JSON ecnoder which
may crash if the header value is a tuple
2022-10-27 20:31:33 +02:00
Zaiming (Stone) Shi
a314950be9 docs: fix zh punctuations 2022-10-26 16:19:38 +02:00
Shawn
4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
Kjell Winblad
9ea0147a8c feat(rule engine jq function): config for changing implementation
This commit adds a rule engine configuration option for changing the
implementation module used for the rule engine function jq. The two
options are `jq_port` (uses Erlang port programs to interact with jq)
and `jq_nif` (uses an Erlang NIF library to interact with jq).

Thanks to @terry-xiaoyu (Xinyu Liu <506895667@qq.com>) for Chinese
translations
2022-09-22 09:02:30 +02:00
Shawn
60a90858f8 feat: check dependent actions before removing the bridges 2022-09-06 15:34:23 +08:00
Shawn
0ef0b68de4 refactor: change '{recoverable_error,Reason}' to '{error,{recoverable_error,Reason}}' 2022-08-31 18:25:00 +08:00
Shawn
14633eaac8 fix: please the elvis 2022-08-31 09:40:22 +08:00
Shawn
ca52b8eb29 fix: start connector-mqtt failed when username/password not provided 2022-08-31 09:18:10 +08:00
Shawn
73e19d84ee feat: use the new metrics to bridge APIs 2022-08-30 23:47:58 +08:00
Shawn
9e50866cd0 fix: rename queue_max_bytes -> max_queue_bytes 2022-08-30 17:18:54 +08:00
Shawn
6b0ccfbc43 refactor: rename the error return resource_down -> recoverable_error 2022-08-26 17:11:12 +08:00
Shawn
a896aa8b27 fix: incorrect replayq dir for the emqx_resource 2022-08-25 16:06:18 +08:00
Shawn
9327c0f51b fix(mysql_bridge): export the query_mode option to the APIs 2022-08-25 11:38:50 +08:00
Shawn
de3a325953 fix: revert the changes in connector mysql 2022-08-16 09:06:13 +08:00
Shawn
d1de262f31 fix: inc 'actions.failed' if bridge query failed 2022-08-15 17:21:14 +08:00
Zhongwen Deng
56417a3130 feat: list rules support for pagination and fuzzy filtering 2022-07-25 09:36:08 +08:00
Shawn
b55048f705 chore: update app versions 2022-07-08 16:29:07 +08:00
Shawn
4da106882c fix(rule): create_at lost when save to config files 2022-07-08 09:17:23 +08:00
Shawn
f132b6f6ae fix: format the outputs for rule engine CLIs 2022-06-17 18:09:22 +08:00
Shawn
6685a3c5a8 fix: remove the 'headers' field from the rule events 2022-06-17 15:57:10 +08:00
Xinyu Liu
3b00b16abe
Merge pull request #8221 from terry-xiaoyu/fix_delayed_module_disbled_after_emqx_stop
feat: fix the hook priorities
2022-06-16 09:42:42 +08:00
Zaiming (Stone) Shi
738a551550 refactor: best-effort json for hocon validation errors 2022-06-15 15:01:15 +02:00
Shawn
39b1b20506 feat: fix the hook priorities 2022-06-15 19:03:40 +08:00
ieQu1
7817aec0d0 feat(rule_engine): CLI to list rules 2022-06-13 15:16:54 +02:00
Zaiming (Stone) Shi
2065be569e fix(emqx_cluster_rpc): fail fast on stale state
Due to:

* Cluster RPC MFA is not idempotent!
* There is a lack of rollback for callback's side-effects

For instance, when two nodes try to add a cluster-singleton
concurrently, one of them will have to wait for the table lock
then try to catch-up, then try to apply MFA.
The catch-up will have the singleton created, but the initiated
initiated multicall apply will fail causing the commit to rollback,
but not to 'undo' the singleton creation.
Later, the retries will fail indefinitely.
2022-06-12 20:18:48 +02:00
Zhongwen Deng
c7cc2e85b2 fix: add default&example for schema 2022-06-06 09:52:10 +08:00
Zhongwen Deng
9ec804ae03 feat: generate example.conf from schemas 2022-05-31 19:20:27 +08:00
Shawn
eacc7768d2 fix: make 'name' field optional 2022-05-27 18:54:25 +08:00
Shawn
a157539710 refactor(rule): rename http_bridge -> webhook 2022-05-25 18:56:12 +08:00
Shawn
71de9616d3 refactor(rule): rename outputs -> actions 2022-05-25 17:02:37 +08:00
Xinyu Liu
866810cea6
Merge pull request #8006 from emqx/copy-of-kjell/jq/timeout
feat(rule_engine): default timeout for jq/2 and jq/3 with timeout
2022-05-23 11:58:40 +08:00
EMQ-YangM
005fce63d1 feat: add rule engine function float2str/2 2022-05-19 16:16:05 +08:00
Shawn
d89d692d3e refactor: move some APIs to new module emqx_bridge_resource 2022-05-18 15:12:34 +08:00
Kjell Winblad
402ab7e759 chore: remove unnecessary repeated code
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-05-18 07:21:16 +02:00
Kjell Winblad
05032467bd feat(rule_engine): default timeout for jq/2 and jq/3 with timeout
This commit adds a default timeout of 10 seconds to the rule engine's
`jq/2` function, and adds a new function `jq/3` (where the last parameter is
a timeout value). The default timeout can be configured with the setting
"rule_engine.jq_function_default_timeout".

Having a timeout when executing jq code in the rule engine is important
as jq code can potentially run forever. Also, the Erlang jq library
limits the number of jq programs that can execute concurrently so a jq
program that loops forever could potentially also prevent a "non-buggy"
jq program from ever starting.
2022-05-17 19:36:32 +02:00
Shawn
c98fd04f8f fix: rename the rule metrics 2022-05-17 17:17:07 +08:00
JianBo He
d8ec51a0c2 feat(rule-engine): add proto_name/proto_ver for $event/client_disconnected event 2022-05-16 15:12:41 +08:00
DDDHuang
8b0adf69d2
Merge pull request #7926 from DDDHuang/rule_api_errormsg
fix: better error message for rule engine
2022-05-12 19:07:47 +08:00
DDDHuang
2de69c97ba fix: ruleengine & connector & bridge api, better error message 2022-05-12 16:51:36 +08:00
DDDHuang
a284ab9cf8 fix: better error message for rule engine 2022-05-12 11:06:07 +08:00
Thales Macedo Garitezi
47ed9cbc76
fix(telemetry): count bridges referenced in from rule clauses 2022-05-11 15:15:17 -03:00
Shawn
82da4d82be chore: update jq to v0.2.0 2022-05-06 22:20:35 +08:00
EMQ-YangM
685b06eeed fix: avoid 'rule_test' api crash 2022-05-05 17:28:59 +08:00
Kjell Winblad
74c33cd4e5 feat(rule_engine): add jq function to the rule engine
This commit adds a function to the rule engine that alows users
to transform text or JSON objects using [jq filter programs][1].

[jq][1] is a command line tool that can be used to transform
and filter JSON text using jq's built-in language. The rule engine
function that is added with this commit uses the
[Erlang jq NIF library][2] that wraps the jq C library in an
Erlang NIF function.

[1]: https://stedolan.github.io/jq/
[2]: https://github.com/emqx/jq
2022-05-04 17:09:16 +02:00
Zaiming (Stone) Shi
6b44d96c0d refactor: rename functions to please elvis 2022-04-29 10:52:08 +02:00
Zaiming (Stone) Shi
4e65322667 refactor: move emqx_plugin_libs_metrics to emqx app
because it can not depend on other apps
2022-04-29 12:41:36 +08:00
Zaiming (Stone) Shi
02c3f87b31 style: reformat all remaining apps 2022-04-27 15:51:18 +02:00
Xinyu Liu
46e993fa81
Merge pull request #7786 from terry-xiaoyu/save_tls_files_for_bridges
Save tls files for bridges
2022-04-27 21:48:18 +08:00
Shawn
ba800d853d fix(rule): republish all available fields if payload template empty 2022-04-27 11:59:15 +08:00
EMQ-YangM
1fac70d2bb fix: remove error field 2022-04-27 09:48:57 +08:00
EMQ-YangM
cef16ac382 fix: improve document quality 2022-04-21 15:05:56 +08:00
EMQ-YangM
51972af472 feat: add chinese to rule engine description 2022-04-20 10:23:56 +08:00
Xinyu Liu
21fe7f01ee
Merge pull request #7624 from DDDHuang/fix_api_params
Fix api params
2022-04-18 09:33:22 +08:00
DDDHuang
ea6b3c7b51 fix: format code 2022-04-16 16:37:10 +08:00
DDDHuang
82ce6ae9f9 fix: bad integer, bad qos; TODO: gateway api * 2 2022-04-16 16:37:10 +08:00
EMQ-YangM
e4b62f3a5f feat: add rule-engine functions 2022-04-15 18:02:24 +08:00
EMQ-YangM
ee6ac90d47 fix: behavior of rule-engine when selection field does not exist 2022-04-13 18:19:42 +08:00
EMQ-YangM
21dfd5dd47 fix: export rule reset_metrics api 2022-04-11 15:21:14 +08:00
EMQ-YangM
f91c0e140d fix: fix static_checks error 2022-04-11 10:25:48 +08:00
EMQ-YangM
fa0c4d17ee feat: impl rule_engine reset_metrics api 2022-04-11 10:25:48 +08:00
Thales Macedo Garitezi
1aea6b3ea7
Merge pull request #7522 from thalesmg/telemetry-revamp-part5
feat: add basic rule engine and bridge usage info to telemetry
2022-04-07 10:19:55 -03:00
Thales Macedo Garitezi
8354095e4b
feat(rules): export basic usage info for telemetry 2022-04-06 10:20:23 -03:00
EMQ-YangM
a31891e8e6 feat: enhanced rule engine error handling when json parsing error 2022-04-06 09:56:55 +08:00
ieQu1
99b3f2f83e docs(schema): Document missing records and fields 2022-04-04 23:58:43 +02:00
ieQu1
d23a88d0a9 docs(schema): Minor improvements to spelling and formatting. 2022-03-30 13:05:56 +02:00
ieQu1
ce450b7da7 docs(schema): Document the records 2022-03-30 11:53:53 +02:00
EMQ-YangM
3b43cca5bf fix: use masp:get/3 to avoid crash after match failure 2022-03-29 18:51:10 +08:00
EMQ-YangM
ccc134d061 fix(emqx_rule_api_schema): add rule events schema 2022-03-29 14:03:15 +08:00
EMQ-YangM
4791c64b73 feat: add rule event 'client.check_authz_complete' 2022-03-29 11:24:36 +08:00
EMQ-YangM
65164fb046 feat: add rule event 'client.connack' 2022-03-28 11:32:07 +08:00
JianBo He
83e16a5bc5
Merge pull request #7343 from HJianBo/fix-gw-bugs 2022-03-22 17:49:02 +08:00
EMQ-YangM
6cff53ad67 fix(rule): replace sql crlf with spaces
get around the hocon bug: error handling escape character \n
2022-03-21 19:26:06 +08:00
JianBo He
6da4592bb0 chore: fix elivs warnings 2022-03-21 18:52:41 +08:00
JianBo He
5682dcb72e fix(gw): enrich conninfo for coap&lwm2m 2022-03-21 18:17:24 +08:00
JianBo He
edb1460b56 chore(rule-engine): more safe generate even_msg content 2022-03-21 18:17:24 +08:00
Shawn
50ee6ad2e2 fix(rule): update rules without reset the metrics 2022-03-21 15:06:34 +08:00
Shawn
93c5fa60b4 fix(rules): store rule configs to cluster config files 2022-03-16 00:39:01 +08:00
Shawn
869dc1a081 fix(rules): ensure create_at unchanged after rule is updated 2022-03-16 00:38:42 +08:00
Shawn
0b4b3a7cf0 fix(dialyzer): some bad type specs 2022-03-11 18:33:08 +08:00
Shawn
590fa1b375 fix(rule): check request body for /rule_test crashes 2022-03-11 18:13:12 +08:00
Shawn
f924b9389d fix(rule): add swagger schema for events from bridge sources 2022-03-11 14:13:14 +08:00