Commit Graph

530 Commits

Author SHA1 Message Date
Kjell Winblad
8279d8c787
Merge pull request #13053 from kjellwinblad/kjell/fix_rule_trace_issues/EMQX-12327/EMQX-12335/EMQX-12336
Fix small action trace issues
2024-05-20 16:25:38 +02:00
Kjell Winblad
3c87bcde46 fix(rule trace): restore logger metadata to its previous value 2024-05-20 10:57:15 +02:00
Kjell Winblad
5ce095f30e fix(rule trace): only include stop_after_rendering when value is true 2024-05-17 14:53:14 +02:00
Kjell Winblad
2e6db85578 fix(rule trace): rename rule_trigger_time(s) and cleaups
This commit renames trace fields rule_trigger_time and
rule_trigger_times to rule_trigger_ts and makes sure that the value for
rule_trigger_ts will always be a list of timestamps.
2024-05-17 14:13:36 +02:00
Kjell Winblad
2209b26fa5 fix(rule trace): do not leak trace meta data when tracing action result
Fixes:
https://emqx.atlassian.net/browse/EMQX-12391
2024-05-17 10:05:33 +02:00
Kjell Winblad
ed16e678be fix(rule apply test): add default values to context
Fixes:
https://emqx.atlassian.net/browse/EMQX-12378
2024-05-16 17:31:44 +02:00
Thales Macedo Garitezi
2eb1bb2b13
Merge pull request #13032 from kjellwinblad/kjell/no_ctx_message_validation_failed/EMQX-12354
fix: add handling of message_validation_failed ctx for rule testing
2024-05-15 11:20:18 -03:00
lafirest
faf3492e58
Merge pull request #13039 from lafirest/fix/rocket_key
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-15 15:28:35 +08:00
Kjell Winblad
e811f4a392 fix: add handling of message_validation_failed ctx for rule testing
Before this commit, the backend could not handle the test context for
message_validation_failed events that could be sent from the dashboard
since no handling code for this contexts existed in the backend. This
has now been fixed by adding such handling code.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12354
2024-05-14 13:07:32 -03:00
firest
437a0c0d55 fix(rocketmq): fix a typo and a RocketMQ encoding error 2024-05-14 23:01:31 +08:00
Thales Macedo Garitezi
257dae5211 refactor: rename message_validation to schema_validation
Request from Product team.

Fixes https://emqx.atlassian.net/browse/EMQX-12383
2024-05-14 10:32:27 -03:00
Kjell Winblad
c8690a8273 fix: include ruleid parameter in swagger example for apply rule endpoint
Fixes:
https://emqx.atlassian.net/browse/EMQX-12325
2024-05-13 12:28:17 +02:00
Kjell Winblad
feecc36607 fix(rule tracing): clean up error tuple in the action_failed trace 2024-05-07 09:13:55 +02:00
Kjell Winblad
5bfe31b691 fix: issues found during PR review (thanks @thalesmg and @zmstone)
* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
  maintainability
* Added callback to format return value from connector
  * Extended test case to check that the format return value callback
    works
* Added handling of "lazy" trace entry data to the text formatter. Do we
  need to handle this data in the normal log formatters as well?
2024-05-03 15:04:57 +02:00
Kjell Winblad
9576efb716 fix(rule apply test): do not crash when the rule is not found 2024-05-03 15:02:37 +02:00
Kjell Winblad
3ba5cb7858 fix(apply rule): make sure we return something JSON compatible
Fixes:
https://emqx.atlassian.net/browse/EMQX-12275
2024-05-03 15:02:37 +02:00
Kjell Winblad
ea7633c484 fix: rule trace formatting, republish and console stop after rendering
* Better rule trace formatting for many trace entries
* The republish and console actions have got working stop after
  rendering functionality
2024-05-03 15:02:36 +02:00
Thales Macedo Garitezi
607bf0fe44
Merge pull request #12952 from thalesmg/fix-dont-list-mv-rule-event-r57-20240429
fix(rule events): don't list validation failure event in CE API
2024-05-02 09:20:53 -03:00
Thales Macedo Garitezi
7a44746492 fix(rule events): don't list validation failures in CE
Fixes https://emqx.atlassian.net/browse/EMQX-12284
2024-04-30 14:41:09 -03:00
zmstone
c8d6976b14 feat: add conditions to variform expressions
- refactored `coalesce` function to allow lazy evaluation
- added `iif(Cond, IfExpr, EleseExpr)` to allow simple conditions
2024-04-30 07:28:27 +02:00
Kjell Winblad
ef9884cf47 refactor(rule trace): templates rendered trace to increase code reuse
* The code for passing the trace context to a sub process has been
  improved to increase code reuse. This code is used when the action
  templates are rendered in a sub process.
* A macro has also been added for the error term that is thrown when the
  action shall be stopped after the templates has been rendered. This is
  also done to reduce code duplication and to reduce the risk of
  introducing bugs due to typos.
* Fix incorrect type spec

Thanks to @zmstone for suggesting these improvements in comments to a PR
(https://github.com/emqx/emqx/pull/12916).
2024-04-25 13:15:36 +02:00
Kjell Winblad
b2811f96b2 refactor(rule trace): simplify function for setting trace meta data
This commit simplifies a function to set trace meta data in line with a
suggestion from  @zmstone:

https://github.com/emqx/emqx/pull/12912#discussion_r1576053856
2024-04-25 11:51:46 +02:00
Kjell Winblad
e9d498dde2 feat(rule tracing): add rule trigger time meta data field
Fixes:
https://emqx.atlassian.net/browse/EMQX-12025
2024-04-22 13:51:48 +02:00
Kjell Winblad
8f1486f6d3 fix: clean up trace messages to make it easier to interpret
This commit removes some redundant trace messages and renames some to
make it easier to interpret what is happening for the user.
2024-04-18 14:28:01 +02:00
Kjell Winblad
2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
Kjell Winblad
6cf29ba688 fix: clean up traces to make them easier to parse and understand 2024-04-17 18:20:17 +02:00
zmstone
f80d078de3 feat(variform): Add more functions 2024-04-15 16:56:52 +02:00
zmstone
b76b6fbe63 feat(variform): initialize client_attrs with variform
Moved regular expression extraction as a variform function.
2024-04-14 10:13:24 +02:00
Kjell Winblad
f444c6fc32 fix: pass stop_action_after_render=true in trace meta data
Even if there is no trace we still need to pass
stop_action_after_render=true in the trace meta data so that the action
will be stopped.
2024-04-12 11:38:01 +02:00
Kjell Winblad
9998940aa2 fix(trace): several improvements thanks to comments from @zmstone 2024-04-11 16:48:43 +02:00
Kjell Winblad
31142df5cf fix: default value of stop_action_after_template_rendering to true
This commit changes the default value for the
stop_action_after_template_rendering option of the apply rule HTTP API
endpoint so that it is true instead of false.
2024-04-09 16:20:37 +02:00
Kjell Winblad
9628a00a82 docs(emqx_rule_api apply rule): fix doc strings 2024-04-08 15:34:29 +02:00
Kjell Winblad
02ee873094 docs(emqx_rule_api_schema): fix type spec 2024-04-08 13:42:15 +02:00
Kjell Winblad
79440064fe style: fix problems reported by elvis 2024-04-08 11:03:55 +02:00
Kjell Winblad
5479932190 feat(apply rule test): make option to stop action after render work
This commit makes the apply rule HTTP API option to stop an action work
for the HTTP action, and adds infrastructure that makes it easy to add
this functionality to other actions.
2024-04-06 17:21:12 +02:00
Kjell Winblad
ef705c2285 feat: add apply rule API, clientid/ruleid tracing for rule and connector
This commit adds:

* Support for forwarding the rule id and client id to the connector so
  that events such as template rendered successfully can be traced.
* HTTP API for for applying/activating a rule with the given context
2024-04-06 17:20:47 +02:00
Kjell Winblad
59a442cdb5 feat(rule trace): add support for ruleid as a trace type 2024-04-04 14:55:32 +02:00
zmstone
5f26e4ed5e feat(variform): implement variform engine 2024-03-28 19:34:57 +01:00
zmstone
ad95473aae refactor: move string functions to emqx_variform 2024-03-28 18:03:37 +01:00
Ivan Dyachkov
db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Thales Macedo Garitezi
59ff43ec87 feat(message validation): implement metrics
Fixes https://emqx.atlassian.net/browse/EMQX-12071
2024-03-25 10:55:01 -03:00
Thales Macedo Garitezi
00aa7b5621 feat: create new message.validation_failed hookpoint and rule engine event 2024-03-21 13:46:27 -03:00
Ivan Dyachkov
923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
sync release-56
2024-03-19 19:09:41 +01:00
Ivan Dyachkov
f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
Thales Macedo Garitezi
f84a996671 feat: implement message validation
Fixes https://emqx.atlassian.net/browse/EMQX-11980
2024-03-18 13:11:39 -03:00
Serge Tupchii
8f8b023429 refactor: simplify match spec holder construction 2024-03-18 18:00:26 +02:00
Kjell Winblad
5a6f96212d feat(rule engine SQL): add an unescape function
The added `unescape` function unescapes escape sequences, transforming
them back to their represented characters. The following escape
sequences are supported:

- Standard C escape sequences:
  - `\n` for newline (LF)
  - `\t` for horizontal tab (HT)
  - `\r` for carriage return (CR)
  - `\b` for backspace (BS)
  - `\f` for formfeed (FF)
  - `\v` for vertical tab (VT)
  - `\'` for single quote (')
  - `\"` for double quote (")
  - `\\` for backslash (\)
  - `\?` for question mark (?)
  - `\a` for alert (bell, BEL)

- Hexadecimal escape codes:
  - `\xH...` where `H...` is one or more hexadecimal digits (0-9, A-F,
    a-f), allowing for the encoding of arbitrary utf32 characters.

If an escape sequence is not recognized, or if the hexadecimal escape
does not form a valid Unicode character, the function generates an
exception.

Fixes:
https://github.com/emqx/emqx/issues/12460
https://emqx.atlassian.net/browse/EMQX-11847
2024-03-08 12:05:06 +01:00
Shawn
163d095dca fix: port the changes for date_to_unix_ts SQL fun from 4.4 2024-03-08 10:48:08 +08:00
Kjell Winblad
69114bc6c2 fix(rule_engine): support non-byte sized input to bin2hexstr
The rule engine subbits function can return a bitstring which size is
not divisible by 8. Therefore, it makes sense that the rule engine
function bin2hexstr can handle such bitstrings as well. This is fixed by
this commit.

Fixes:
https://github.com/emqx/emqx/issues/12586
https://emqx.atlassian.net/browse/EMQX-11943
2024-03-06 15:26:49 +01:00
Kjell Winblad
365d054e01 fix: add subbits/4 and subits/5 rule_engine functions
The documentation for the family of subbits functions says that the
fifth and sixth parameters are optional (since they only make sense when
the forth parameter is 'integer'). However, before this commit
`subbits/4` and `subbits/5` did not exist.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11942
https://github.com/emqx/emqx/issues/12587
2024-03-05 13:27:07 +01:00