Commit Graph

420 Commits

Author SHA1 Message Date
JimMoen 842cb7fadc
fix: debug log for rulesql parse failed
See also: PR#10059.
The other error logs will be printed in `emqx_rule_runtime:apply_rule/3`
2023-03-09 11:28:27 +08:00
Ivan Dyachkov c01f62a1c1 chore: bump apps versions 2023-03-03 13:58:32 +01:00
Stefan Strigler 6ebd3dc747 feat(emqx_rule_engine): decompose error tuples
sqltester for instance returns sth like {"...sytnax error...", OrigSql}
2023-03-03 09:59:24 +01:00
Stefan Strigler 44eca1fa72 fix(emqx_rule_engine): don't crash if we can't encode json 2023-03-03 09:36:13 +01:00
Stefan Strigler 64b5e9585e feat(emqx_rule_engine): API format errors in a human readable way 2023-03-02 16:49:11 +01:00
Zaiming (Stone) Shi 9316690c29 fix(schema): binary string for default values
A lot of the string value fields had default value defined in
schema as list-string rather than binary-string.
This caused the generated schema dump (in JSON format)
to have raw_default field as an integer array.
2023-02-21 09:09:51 +01:00
ieQu1 d9554c36ac chore: Bump umbrealla app versions 2023-02-17 09:50:50 +01:00
ieQu1 794bedef9f fix(emqx_ctl): Start CLI before ekka 2023-02-17 09:50:50 +01:00
Zaiming (Stone) Shi 157c919ba1 ci: add i18n style check script 2023-02-09 11:41:52 +01:00
Andrew Mayorov ce2dba15b4
feat: turn tables queried with search APIs into ordered sets
This is needed to ensure more or less consistent client experience
for the new planned cursor-based search APIs.
2023-01-16 15:39:10 +03:00
Zaiming (Stone) Shi 7073c62dd9 feat: add a 'this' for rule engine put env
now a dot is allowed in front of a var (path)
e.g. ${.clientid} is equivalent to ${clientid}

${.} however, means everything.
the parsed var is interally represented as $_THIS_
to make it easier to read/search.
2023-01-14 11:03:58 +01:00
Zhongwen Deng f15b29b1ef chore: upgrade app version 2023-01-13 18:22:29 +08:00
zhongwencool 0049b4a294
Merge branch 'master' into release-50 2023-01-13 16:39:35 +08:00
Thales Macedo Garitezi 48e1ba4832 feat(docs): add tags to schemas
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
2023-01-11 09:10:03 -03:00
Zhongwen Deng fdc0682c3f fix: elvis warning 2023-01-11 17:30:45 +08:00
Thales Macedo Garitezi eb7dca3691 chore(rule_engine): reset metrics when disabling a rule
https://emqx.atlassian.net/browse/EMQX-8502

When a bridge is disabled, its metrics are reset.  With this change,
we make rule actions behave like that: disabling a rule will reset its
metrics.
2023-01-04 09:30:47 -03:00
Zaiming (Stone) Shi dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
JimMoen 4906da9796
test(rule-funcs): test compression funcs 2022-12-19 15:43:50 +08:00
JimMoen 5151242583
feat(rule-engine): `zip`, `gzip`, `zip_compress` funcs in rule-sql 2022-12-19 15:43:50 +08:00
Zaiming (Stone) Shi bd65e8aad6 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:42:50 +01:00
Zaiming (Stone) Shi 1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Stefan Strigler 0b324da7cb refactor: move metrics out of /rules(/:id) to /rules/:id/metrics 2022-12-06 15:23:28 +01:00
Zaiming (Stone) Shi cca3421308 refactor: use static function references 2022-12-06 09:40:03 +01:00
Zaiming (Stone) Shi b398617614 chore: bump app versions 2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi 7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
JianBo He edb35c08a8 chore: refactor ms2qs function type 2022-11-24 20:14:33 +01:00
JianBo He 9786a6c267 refactor(mgmt): convert fuzzy filter func to named func 2022-11-24 20:14:33 +01:00
JianBo He b9c5a5f822 fix(delayed): return correct node name 2022-11-24 20:14:33 +01:00
JianBo He 8a0c468b01 test: refine tests for lots of List HTTP API 2022-11-24 20:14:33 +01:00
JianBo He 1fe9c105aa refactor(mgmt): smplify the node_query/cluster_query implementation 2022-11-24 20:14:33 +01:00
JianBo He 08121e7df6 fix(mgmt): optimize the speed of query tail pages
In the previous, when you query the tail pages, all the front of rows
will be queried out and formatted. It greatly hurts the speed of query.

Currently, we only format the final result rows. i.e, the query for the
last page of data will be 10x faster.
2022-11-24 20:14:33 +01:00
Zaiming (Stone) Shi f3df2c80d8 feat: add user_properties arg for republish action 2022-11-22 20:09:59 +01:00
kraftwerk28 00c57de4c3 feat: do not drop MQTTv5 properties in rule/bridge 2022-11-22 20:09:59 +01:00
Zaiming (Stone) Shi c940b901f5 chore: fix app versions 2022-11-16 16:26:43 +01:00
Zaiming (Stone) Shi 09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
JimMoen 4f00206693
test: different resons for disconnect event 2022-11-08 23:22:58 +08:00
Zaiming (Stone) Shi d704de4a84 fix(emqx_rule_events): drop tuple-value from message headers
the message headers are fed to a JSON ecnoder which
may crash if the header value is a tuple
2022-10-27 20:31:33 +02:00
Zaiming (Stone) Shi a314950be9 docs: fix zh punctuations 2022-10-26 16:19:38 +02:00
Shawn 4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
Kjell Winblad 9ea0147a8c feat(rule engine jq function): config for changing implementation
This commit adds a rule engine configuration option for changing the
implementation module used for the rule engine function jq. The two
options are `jq_port` (uses Erlang port programs to interact with jq)
and `jq_nif` (uses an Erlang NIF library to interact with jq).

Thanks to @terry-xiaoyu (Xinyu Liu <506895667@qq.com>) for Chinese
translations
2022-09-22 09:02:30 +02:00
Shawn 60a90858f8 feat: check dependent actions before removing the bridges 2022-09-06 15:34:23 +08:00
Shawn 0ef0b68de4 refactor: change '{recoverable_error,Reason}' to '{error,{recoverable_error,Reason}}' 2022-08-31 18:25:00 +08:00
Shawn 14633eaac8 fix: please the elvis 2022-08-31 09:40:22 +08:00
Shawn ca52b8eb29 fix: start connector-mqtt failed when username/password not provided 2022-08-31 09:18:10 +08:00
Shawn 73e19d84ee feat: use the new metrics to bridge APIs 2022-08-30 23:47:58 +08:00
Shawn 9e50866cd0 fix: rename queue_max_bytes -> max_queue_bytes 2022-08-30 17:18:54 +08:00
Shawn 6b0ccfbc43 refactor: rename the error return resource_down -> recoverable_error 2022-08-26 17:11:12 +08:00
Shawn a896aa8b27 fix: incorrect replayq dir for the emqx_resource 2022-08-25 16:06:18 +08:00
Shawn 9327c0f51b fix(mysql_bridge): export the query_mode option to the APIs 2022-08-25 11:38:50 +08:00
Shawn de3a325953 fix: revert the changes in connector mysql 2022-08-16 09:06:13 +08:00
Shawn d1de262f31 fix: inc 'actions.failed' if bridge query failed 2022-08-15 17:21:14 +08:00
Zhongwen Deng 56417a3130 feat: list rules support for pagination and fuzzy filtering 2022-07-25 09:36:08 +08:00
Shawn b55048f705 chore: update app versions 2022-07-08 16:29:07 +08:00
Shawn 4da106882c fix(rule): create_at lost when save to config files 2022-07-08 09:17:23 +08:00
Shawn f132b6f6ae fix: format the outputs for rule engine CLIs 2022-06-17 18:09:22 +08:00
Shawn 6685a3c5a8 fix: remove the 'headers' field from the rule events 2022-06-17 15:57:10 +08:00
Xinyu Liu 3b00b16abe
Merge pull request #8221 from terry-xiaoyu/fix_delayed_module_disbled_after_emqx_stop
feat: fix the hook priorities
2022-06-16 09:42:42 +08:00
Zaiming (Stone) Shi 738a551550 refactor: best-effort json for hocon validation errors 2022-06-15 15:01:15 +02:00
Shawn 39b1b20506 feat: fix the hook priorities 2022-06-15 19:03:40 +08:00
ieQu1 7817aec0d0 feat(rule_engine): CLI to list rules 2022-06-13 15:16:54 +02:00
Zaiming (Stone) Shi 2065be569e fix(emqx_cluster_rpc): fail fast on stale state
Due to:

* Cluster RPC MFA is not idempotent!
* There is a lack of rollback for callback's side-effects

For instance, when two nodes try to add a cluster-singleton
concurrently, one of them will have to wait for the table lock
then try to catch-up, then try to apply MFA.
The catch-up will have the singleton created, but the initiated
initiated multicall apply will fail causing the commit to rollback,
but not to 'undo' the singleton creation.
Later, the retries will fail indefinitely.
2022-06-12 20:18:48 +02:00
Zhongwen Deng c7cc2e85b2 fix: add default&example for schema 2022-06-06 09:52:10 +08:00
Zhongwen Deng 9ec804ae03 feat: generate example.conf from schemas 2022-05-31 19:20:27 +08:00
Zhongwen Deng 8aa60cc0a5 feat: generate a minimized emqx.conf 2022-05-31 19:20:27 +08:00
Shawn eacc7768d2 fix: make 'name' field optional 2022-05-27 18:54:25 +08:00
Zaiming (Stone) Shi ebe4d7c3fb refactor: stop releasing edge edition
Prior to EMQX 5.0, the edge edition's main difference comaring
to standard edition are:
* Less number of plugins in the release (smaller package size)
* Smaller number is vm.args (for lower memory usage)

Starting from 5.0 most of the plugins are included as features,
the tuned vm.args arguments does not justify a special release edition.

Also as nanomq is getting mature,
EMQ's recommendtation for MQTT broker in edge is https://nanomq.io/
2022-05-26 20:05:46 +02:00
Shawn 55fce33477 fix: add telemetry testcases in all/0 2022-05-25 21:55:28 +08:00
Shawn a157539710 refactor(rule): rename http_bridge -> webhook 2022-05-25 18:56:12 +08:00
Shawn 71de9616d3 refactor(rule): rename outputs -> actions 2022-05-25 17:02:37 +08:00
Xinyu Liu 866810cea6
Merge pull request #8006 from emqx/copy-of-kjell/jq/timeout
feat(rule_engine): default timeout for jq/2 and jq/3 with timeout
2022-05-23 11:58:40 +08:00
Shawn 7bc2d9edbb fix: remove bison from the required tools when building emqx 2022-05-21 14:17:12 +08:00
Kjell Winblad efd6461e15 docs: chinese documentation for jq timeout configuration
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-05-21 07:07:29 +02:00
EMQ-YangM 005fce63d1 feat: add rule engine function float2str/2 2022-05-19 16:16:05 +08:00
Xinyu Liu 406bc3502e
Merge pull request #7981 from terry-xiaoyu/improve_bridge_apis
refactor: move some APIs to new module emqx_bridge_resource
2022-05-18 18:31:48 +08:00
Yang Miao bf552f2c4b
Merge pull request #7980 from EMQ-YangM/fix_zero_padding
fix: zero padding should be to the MSB side
2022-05-18 17:22:23 +08:00
EMQ-YangM 66c62f432b fix: zero padding should be to the MSB side 2022-05-18 15:28:01 +08:00
Shawn d89d692d3e refactor: move some APIs to new module emqx_bridge_resource 2022-05-18 15:12:34 +08:00
Kjell Winblad 402ab7e759 chore: remove unnecessary repeated code
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-05-18 07:21:16 +02:00
Yang Miao 2b58733571
Merge pull request #7971 from EMQ-YangM/fix_hexstring_to_binary
fix: hexstr_to_bin support half byte
2022-05-18 10:14:25 +08:00
Kjell Winblad 05032467bd feat(rule_engine): default timeout for jq/2 and jq/3 with timeout
This commit adds a default timeout of 10 seconds to the rule engine's
`jq/2` function, and adds a new function `jq/3` (where the last parameter is
a timeout value). The default timeout can be configured with the setting
"rule_engine.jq_function_default_timeout".

Having a timeout when executing jq code in the rule engine is important
as jq code can potentially run forever. Also, the Erlang jq library
limits the number of jq programs that can execute concurrently so a jq
program that loops forever could potentially also prevent a "non-buggy"
jq program from ever starting.
2022-05-17 19:36:32 +02:00
Shawn c98fd04f8f fix: rename the rule metrics 2022-05-17 17:17:07 +08:00
EMQ-YangM d8da8f1686 fix: hexstr_to_bin support half byte 2022-05-17 16:47:59 +08:00
JianBo He d8ec51a0c2 feat(rule-engine): add proto_name/proto_ver for $event/client_disconnected event 2022-05-16 15:12:41 +08:00
DDDHuang 8b0adf69d2
Merge pull request #7926 from DDDHuang/rule_api_errormsg
fix: better error message for rule engine
2022-05-12 19:07:47 +08:00
DDDHuang 5210cd6e8d fix(test): rule engine api SUITE , bad rule name 2022-05-12 18:01:37 +08:00
DDDHuang 2de69c97ba fix: ruleengine & connector & bridge api, better error message 2022-05-12 16:51:36 +08:00
DDDHuang a284ab9cf8 fix: better error message for rule engine 2022-05-12 11:06:07 +08:00
Thales Macedo Garitezi 47ed9cbc76
fix(telemetry): count bridges referenced in `from` rule clauses 2022-05-11 15:15:17 -03:00
Shawn 82da4d82be chore: update jq to v0.2.0 2022-05-06 22:20:35 +08:00
EMQ-YangM 685b06eeed fix: avoid 'rule_test' api crash 2022-05-05 17:28:59 +08:00
Kjell Winblad 74c33cd4e5 feat(rule_engine): add jq function to the rule engine
This commit adds a function to the rule engine that alows users
to transform text or JSON objects using [jq filter programs][1].

[jq][1] is a command line tool that can be used to transform
and filter JSON text using jq's built-in language. The rule engine
function that is added with this commit uses the
[Erlang jq NIF library][2] that wraps the jq C library in an
Erlang NIF function.

[1]: https://stedolan.github.io/jq/
[2]: https://github.com/emqx/jq
2022-05-04 17:09:16 +02:00
Zaiming (Stone) Shi 6b44d96c0d refactor: rename functions to please elvis 2022-04-29 10:52:08 +02:00
Zaiming (Stone) Shi 4e65322667 refactor: move emqx_plugin_libs_metrics to emqx app
because it can not depend on other apps
2022-04-29 12:41:36 +08:00
Zaiming (Stone) Shi 02c3f87b31 style: reformat all remaining apps 2022-04-27 15:51:18 +02:00
Xinyu Liu 46e993fa81
Merge pull request #7786 from terry-xiaoyu/save_tls_files_for_bridges
Save tls files for bridges
2022-04-27 21:48:18 +08:00
Shawn ba800d853d fix(rule): republish all available fields if payload template empty 2022-04-27 11:59:15 +08:00
EMQ-YangM 1fac70d2bb fix: remove error field 2022-04-27 09:48:57 +08:00
JianBo He a58b88aeec chore(i18n): fix more and more typos 2022-04-26 09:28:05 +08:00
JimMoen e9843aa225 fix(doc): html line break use `</br>` 2022-04-24 18:56:21 +08:00
EMQ-YangM cef16ac382 fix: improve document quality 2022-04-21 15:05:56 +08:00