Commit Graph

530 Commits

Author SHA1 Message Date
Kjell Winblad 8279d8c787
Merge pull request #13053 from kjellwinblad/kjell/fix_rule_trace_issues/EMQX-12327/EMQX-12335/EMQX-12336
Fix small action trace issues
2024-05-20 16:25:38 +02:00
Kjell Winblad 3c87bcde46 fix(rule trace): restore logger metadata to its previous value 2024-05-20 10:57:15 +02:00
Kjell Winblad 5ce095f30e fix(rule trace): only include stop_after_rendering when value is true 2024-05-17 14:53:14 +02:00
Kjell Winblad 2e6db85578 fix(rule trace): rename rule_trigger_time(s) and cleaups
This commit renames trace fields rule_trigger_time and
rule_trigger_times to rule_trigger_ts and makes sure that the value for
rule_trigger_ts will always be a list of timestamps.
2024-05-17 14:13:36 +02:00
Kjell Winblad 2209b26fa5 fix(rule trace): do not leak trace meta data when tracing action result
Fixes:
https://emqx.atlassian.net/browse/EMQX-12391
2024-05-17 10:05:33 +02:00
Kjell Winblad ed16e678be fix(rule apply test): add default values to context
Fixes:
https://emqx.atlassian.net/browse/EMQX-12378
2024-05-16 17:31:44 +02:00
Thales Macedo Garitezi 2eb1bb2b13
Merge pull request #13032 from kjellwinblad/kjell/no_ctx_message_validation_failed/EMQX-12354
fix: add handling of message_validation_failed ctx for rule testing
2024-05-15 11:20:18 -03:00
lafirest faf3492e58
Merge pull request #13039 from lafirest/fix/rocket_key
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-15 15:28:35 +08:00
Kjell Winblad e811f4a392 fix: add handling of message_validation_failed ctx for rule testing
Before this commit, the backend could not handle the test context for
message_validation_failed events that could be sent from the dashboard
since no handling code for this contexts existed in the backend. This
has now been fixed by adding such handling code.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12354
2024-05-14 13:07:32 -03:00
firest 437a0c0d55 fix(rocketmq): fix a typo and a RocketMQ encoding error 2024-05-14 23:01:31 +08:00
Thales Macedo Garitezi 257dae5211 refactor: rename `message_validation` to `schema_validation`
Request from Product team.

Fixes https://emqx.atlassian.net/browse/EMQX-12383
2024-05-14 10:32:27 -03:00
Kjell Winblad c8690a8273 fix: include ruleid parameter in swagger example for apply rule endpoint
Fixes:
https://emqx.atlassian.net/browse/EMQX-12325
2024-05-13 12:28:17 +02:00
Kjell Winblad feecc36607 fix(rule tracing): clean up error tuple in the action_failed trace 2024-05-07 09:13:55 +02:00
Kjell Winblad 5bfe31b691 fix: issues found during PR review (thanks @thalesmg and @zmstone)
* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
  maintainability
* Added callback to format return value from connector
  * Extended test case to check that the format return value callback
    works
* Added handling of "lazy" trace entry data to the text formatter. Do we
  need to handle this data in the normal log formatters as well?
2024-05-03 15:04:57 +02:00
Kjell Winblad 9576efb716 fix(rule apply test): do not crash when the rule is not found 2024-05-03 15:02:37 +02:00
Kjell Winblad 3ba5cb7858 fix(apply rule): make sure we return something JSON compatible
Fixes:
https://emqx.atlassian.net/browse/EMQX-12275
2024-05-03 15:02:37 +02:00
Kjell Winblad ea7633c484 fix: rule trace formatting, republish and console stop after rendering
* Better rule trace formatting for many trace entries
* The republish and console actions have got working stop after
  rendering functionality
2024-05-03 15:02:36 +02:00
Thales Macedo Garitezi 607bf0fe44
Merge pull request #12952 from thalesmg/fix-dont-list-mv-rule-event-r57-20240429
fix(rule events): don't list validation failure event in CE API
2024-05-02 09:20:53 -03:00
Thales Macedo Garitezi 7a44746492 fix(rule events): don't list validation failures in CE
Fixes https://emqx.atlassian.net/browse/EMQX-12284
2024-04-30 14:41:09 -03:00
zmstone c8d6976b14 feat: add conditions to variform expressions
- refactored `coalesce` function to allow lazy evaluation
- added `iif(Cond, IfExpr, EleseExpr)` to allow simple conditions
2024-04-30 07:28:27 +02:00
Kjell Winblad ef9884cf47 refactor(rule trace): templates rendered trace to increase code reuse
* The code for passing the trace context to a sub process has been
  improved to increase code reuse. This code is used when the action
  templates are rendered in a sub process.
* A macro has also been added for the error term that is thrown when the
  action shall be stopped after the templates has been rendered. This is
  also done to reduce code duplication and to reduce the risk of
  introducing bugs due to typos.
* Fix incorrect type spec

Thanks to @zmstone for suggesting these improvements in comments to a PR
(https://github.com/emqx/emqx/pull/12916).
2024-04-25 13:15:36 +02:00
Kjell Winblad b2811f96b2 refactor(rule trace): simplify function for setting trace meta data
This commit simplifies a function to set trace meta data in line with a
suggestion from  @zmstone:

https://github.com/emqx/emqx/pull/12912#discussion_r1576053856
2024-04-25 11:51:46 +02:00
Kjell Winblad e9d498dde2 feat(rule tracing): add rule trigger time meta data field
Fixes:
https://emqx.atlassian.net/browse/EMQX-12025
2024-04-22 13:51:48 +02:00
Kjell Winblad 8f1486f6d3 fix: clean up trace messages to make it easier to interpret
This commit removes some redundant trace messages and renames some to
make it easier to interpret what is happening for the user.
2024-04-18 14:28:01 +02:00
Kjell Winblad 2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
Kjell Winblad 6cf29ba688 fix: clean up traces to make them easier to parse and understand 2024-04-17 18:20:17 +02:00
zmstone f80d078de3 feat(variform): Add more functions 2024-04-15 16:56:52 +02:00
zmstone b76b6fbe63 feat(variform): initialize client_attrs with variform
Moved regular expression extraction as a variform function.
2024-04-14 10:13:24 +02:00
Kjell Winblad f444c6fc32 fix: pass stop_action_after_render=true in trace meta data
Even if there is no trace we still need to pass
stop_action_after_render=true in the trace meta data so that the action
will be stopped.
2024-04-12 11:38:01 +02:00
Kjell Winblad 9998940aa2 fix(trace): several improvements thanks to comments from @zmstone 2024-04-11 16:48:43 +02:00
Kjell Winblad 31142df5cf fix: default value of stop_action_after_template_rendering to true
This commit changes the default value for the
stop_action_after_template_rendering option of the apply rule HTTP API
endpoint so that it is true instead of false.
2024-04-09 16:20:37 +02:00
Kjell Winblad 9628a00a82 docs(emqx_rule_api apply rule): fix doc strings 2024-04-08 15:34:29 +02:00
Kjell Winblad 02ee873094 docs(emqx_rule_api_schema): fix type spec 2024-04-08 13:42:15 +02:00
Kjell Winblad 79440064fe style: fix problems reported by elvis 2024-04-08 11:03:55 +02:00
Kjell Winblad 5479932190 feat(apply rule test): make option to stop action after render work
This commit makes the apply rule HTTP API option to stop an action work
for the HTTP action, and adds infrastructure that makes it easy to add
this functionality to other actions.
2024-04-06 17:21:12 +02:00
Kjell Winblad ef705c2285 feat: add apply rule API, clientid/ruleid tracing for rule and connector
This commit adds:

* Support for forwarding the rule id and client id to the connector so
  that events such as template rendered successfully can be traced.
* HTTP API for for applying/activating a rule with the given context
2024-04-06 17:20:47 +02:00
Kjell Winblad 59a442cdb5 feat(rule trace): add support for ruleid as a trace type 2024-04-04 14:55:32 +02:00
zmstone 5f26e4ed5e feat(variform): implement variform engine 2024-03-28 19:34:57 +01:00
zmstone ad95473aae refactor: move string functions to emqx_variform 2024-03-28 18:03:37 +01:00
Ivan Dyachkov db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Thales Macedo Garitezi 59ff43ec87 feat(message validation): implement metrics
Fixes https://emqx.atlassian.net/browse/EMQX-12071
2024-03-25 10:55:01 -03:00
Thales Macedo Garitezi 00aa7b5621 feat: create new `message.validation_failed` hookpoint and rule engine event 2024-03-21 13:46:27 -03:00
Ivan Dyachkov 923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
sync release-56
2024-03-19 19:09:41 +01:00
Ivan Dyachkov f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
Thales Macedo Garitezi f84a996671 feat: implement message validation
Fixes https://emqx.atlassian.net/browse/EMQX-11980
2024-03-18 13:11:39 -03:00
Serge Tupchii 8f8b023429 refactor: simplify match spec holder construction 2024-03-18 18:00:26 +02:00
Kjell Winblad 5a6f96212d feat(rule engine SQL): add an `unescape` function
The added `unescape` function unescapes escape sequences, transforming
them back to their represented characters. The following escape
sequences are supported:

- Standard C escape sequences:
  - `\n` for newline (LF)
  - `\t` for horizontal tab (HT)
  - `\r` for carriage return (CR)
  - `\b` for backspace (BS)
  - `\f` for formfeed (FF)
  - `\v` for vertical tab (VT)
  - `\'` for single quote (')
  - `\"` for double quote (")
  - `\\` for backslash (\)
  - `\?` for question mark (?)
  - `\a` for alert (bell, BEL)

- Hexadecimal escape codes:
  - `\xH...` where `H...` is one or more hexadecimal digits (0-9, A-F,
    a-f), allowing for the encoding of arbitrary utf32 characters.

If an escape sequence is not recognized, or if the hexadecimal escape
does not form a valid Unicode character, the function generates an
exception.

Fixes:
https://github.com/emqx/emqx/issues/12460
https://emqx.atlassian.net/browse/EMQX-11847
2024-03-08 12:05:06 +01:00
Shawn 163d095dca fix: port the changes for date_to_unix_ts SQL fun from 4.4 2024-03-08 10:48:08 +08:00
Kjell Winblad 69114bc6c2 fix(rule_engine): support non-byte sized input to bin2hexstr
The rule engine subbits function can return a bitstring which size is
not divisible by 8. Therefore, it makes sense that the rule engine
function bin2hexstr can handle such bitstrings as well. This is fixed by
this commit.

Fixes:
https://github.com/emqx/emqx/issues/12586
https://emqx.atlassian.net/browse/EMQX-11943
2024-03-06 15:26:49 +01:00
Kjell Winblad 365d054e01 fix: add subbits/4 and subits/5 rule_engine functions
The documentation for the family of subbits functions says that the
fifth and sixth parameters are optional (since they only make sense when
the forth parameter is 'integer'). However, before this commit
`subbits/4` and `subbits/5` did not exist.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11942
https://github.com/emqx/emqx/issues/12587
2024-03-05 13:27:07 +01:00