faf3492e58
Merge pull request #13039 from lafirest/fix/rocket_key
...
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-15 15:28:35 +08:00
2a0ac34656
Merge branch 'release-57' into sync-r57-m-20240514
2024-05-14 15:03:04 -03:00
e811f4a392
fix: add handling of message_validation_failed ctx for rule testing
...
Before this commit, the backend could not handle the test context for
message_validation_failed events that could be sent from the dashboard
since no handling code for this contexts existed in the backend. This
has now been fixed by adding such handling code.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12354
2024-05-14 13:07:32 -03:00
437a0c0d55
fix(rocketmq): fix a typo and a RocketMQ encoding error
2024-05-14 23:01:31 +08:00
257dae5211
refactor: rename `message_validation` to `schema_validation`
...
Request from Product team.
Fixes https://emqx.atlassian.net/browse/EMQX-12383
2024-05-14 10:32:27 -03:00
25c6ac3d16
Merge remote-tracking branch 'origin/release-57'
2024-05-13 22:05:29 +02:00
c8690a8273
fix: include ruleid parameter in swagger example for apply rule endpoint
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12325
2024-05-13 12:28:17 +02:00
900536712d
Merge pull request #12997 from thalesmg/sync-r57-m-20240508
...
sync `release-57` to `master`
2024-05-10 10:41:30 +02:00
2bc014db69
fix(events): call `client.check_authn_complete` even if authentication fails
2024-05-09 18:15:32 +08:00
d2ca4e9f11
chore: update change
2024-05-09 11:04:56 +08:00
f641d0b2b7
feat(events): add new hook && event `client.check_authn_complete`
2024-05-07 16:24:31 +08:00
feecc36607
fix(rule tracing): clean up error tuple in the action_failed trace
2024-05-07 09:13:55 +02:00
5bfe31b691
fix: issues found during PR review (thanks @thalesmg and @zmstone)
...
* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
maintainability
* Added callback to format return value from connector
* Extended test case to check that the format return value callback
works
* Added handling of "lazy" trace entry data to the text formatter. Do we
need to handle this data in the normal log formatters as well?
2024-05-03 15:04:57 +02:00
9576efb716
fix(rule apply test): do not crash when the rule is not found
2024-05-03 15:02:37 +02:00
3ba5cb7858
fix(apply rule): make sure we return something JSON compatible
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12275
2024-05-03 15:02:37 +02:00
ea7633c484
fix: rule trace formatting, republish and console stop after rendering
...
* Better rule trace formatting for many trace entries
* The republish and console actions have got working stop after
rendering functionality
2024-05-03 15:02:36 +02:00
607bf0fe44
Merge pull request #12952 from thalesmg/fix-dont-list-mv-rule-event-r57-20240429
...
fix(rule events): don't list validation failure event in CE API
2024-05-02 09:20:53 -03:00
7a44746492
fix(rule events): don't list validation failures in CE
...
Fixes https://emqx.atlassian.net/browse/EMQX-12284
2024-04-30 14:41:09 -03:00
c8d6976b14
feat: add conditions to variform expressions
...
- refactored `coalesce` function to allow lazy evaluation
- added `iif(Cond, IfExpr, EleseExpr)` to allow simple conditions
2024-04-30 07:28:27 +02:00
ef9884cf47
refactor(rule trace): templates rendered trace to increase code reuse
...
* The code for passing the trace context to a sub process has been
improved to increase code reuse. This code is used when the action
templates are rendered in a sub process.
* A macro has also been added for the error term that is thrown when the
action shall be stopped after the templates has been rendered. This is
also done to reduce code duplication and to reduce the risk of
introducing bugs due to typos.
* Fix incorrect type spec
Thanks to @zmstone for suggesting these improvements in comments to a PR
(https://github.com/emqx/emqx/pull/12916 ).
2024-04-25 13:15:36 +02:00
b2811f96b2
refactor(rule trace): simplify function for setting trace meta data
...
This commit simplifies a function to set trace meta data in line with a
suggestion from @zmstone:
https://github.com/emqx/emqx/pull/12912#discussion_r1576053856
2024-04-25 11:51:46 +02:00
e9d498dde2
feat(rule tracing): add rule trigger time meta data field
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12025
2024-04-22 13:51:48 +02:00
8f1486f6d3
fix: clean up trace messages to make it easier to interpret
...
This commit removes some redundant trace messages and renames some to
make it easier to interpret what is happening for the user.
2024-04-18 14:28:01 +02:00
2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
...
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
6cf29ba688
fix: clean up traces to make them easier to parse and understand
2024-04-17 18:20:17 +02:00
f80d078de3
feat(variform): Add more functions
2024-04-15 16:56:52 +02:00
b76b6fbe63
feat(variform): initialize client_attrs with variform
...
Moved regular expression extraction as a variform function.
2024-04-14 10:13:24 +02:00
f444c6fc32
fix: pass stop_action_after_render=true in trace meta data
...
Even if there is no trace we still need to pass
stop_action_after_render=true in the trace meta data so that the action
will be stopped.
2024-04-12 11:38:01 +02:00
9998940aa2
fix(trace): several improvements thanks to comments from @zmstone
2024-04-11 16:48:43 +02:00
31142df5cf
fix: default value of stop_action_after_template_rendering to true
...
This commit changes the default value for the
stop_action_after_template_rendering option of the apply rule HTTP API
endpoint so that it is true instead of false.
2024-04-09 16:20:37 +02:00
9628a00a82
docs(emqx_rule_api apply rule): fix doc strings
2024-04-08 15:34:29 +02:00
02ee873094
docs(emqx_rule_api_schema): fix type spec
2024-04-08 13:42:15 +02:00
79440064fe
style: fix problems reported by elvis
2024-04-08 11:03:55 +02:00
5479932190
feat(apply rule test): make option to stop action after render work
...
This commit makes the apply rule HTTP API option to stop an action work
for the HTTP action, and adds infrastructure that makes it easy to add
this functionality to other actions.
2024-04-06 17:21:12 +02:00
ef705c2285
feat: add apply rule API, clientid/ruleid tracing for rule and connector
...
This commit adds:
* Support for forwarding the rule id and client id to the connector so
that events such as template rendered successfully can be traced.
* HTTP API for for applying/activating a rule with the given context
2024-04-06 17:20:47 +02:00
59a442cdb5
feat(rule trace): add support for ruleid as a trace type
2024-04-04 14:55:32 +02:00
5f26e4ed5e
feat(variform): implement variform engine
2024-03-28 19:34:57 +01:00
ad95473aae
refactor: move string functions to emqx_variform
2024-03-28 18:03:37 +01:00
db9efb9317
chore: bump apps versions
2024-03-28 10:19:09 +01:00
59ff43ec87
feat(message validation): implement metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12071
2024-03-25 10:55:01 -03:00
00aa7b5621
feat: create new `message.validation_failed` hookpoint and rule engine event
2024-03-21 13:46:27 -03:00
923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
...
sync release-56
2024-03-19 19:09:41 +01:00
f2dc940436
Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56
2024-03-19 15:20:08 +01:00
f84a996671
feat: implement message validation
...
Fixes https://emqx.atlassian.net/browse/EMQX-11980
2024-03-18 13:11:39 -03:00
8f8b023429
refactor: simplify match spec holder construction
2024-03-18 18:00:26 +02:00
5a6f96212d
feat(rule engine SQL): add an `unescape` function
...
The added `unescape` function unescapes escape sequences, transforming
them back to their represented characters. The following escape
sequences are supported:
- Standard C escape sequences:
- `\n` for newline (LF)
- `\t` for horizontal tab (HT)
- `\r` for carriage return (CR)
- `\b` for backspace (BS)
- `\f` for formfeed (FF)
- `\v` for vertical tab (VT)
- `\'` for single quote (')
- `\"` for double quote (")
- `\\` for backslash (\)
- `\?` for question mark (?)
- `\a` for alert (bell, BEL)
- Hexadecimal escape codes:
- `\xH...` where `H...` is one or more hexadecimal digits (0-9, A-F,
a-f), allowing for the encoding of arbitrary utf32 characters.
If an escape sequence is not recognized, or if the hexadecimal escape
does not form a valid Unicode character, the function generates an
exception.
Fixes:
https://github.com/emqx/emqx/issues/12460
https://emqx.atlassian.net/browse/EMQX-11847
2024-03-08 12:05:06 +01:00
163d095dca
fix: port the changes for date_to_unix_ts SQL fun from 4.4
2024-03-08 10:48:08 +08:00
69114bc6c2
fix(rule_engine): support non-byte sized input to bin2hexstr
...
The rule engine subbits function can return a bitstring which size is
not divisible by 8. Therefore, it makes sense that the rule engine
function bin2hexstr can handle such bitstrings as well. This is fixed by
this commit.
Fixes:
https://github.com/emqx/emqx/issues/12586
https://emqx.atlassian.net/browse/EMQX-11943
2024-03-06 15:26:49 +01:00
365d054e01
fix: add subbits/4 and subits/5 rule_engine functions
...
The documentation for the family of subbits functions says that the
fifth and sixth parameters are optional (since they only make sense when
the forth parameter is 'integer'). However, before this commit
`subbits/4` and `subbits/5` did not exist.
Fixes:
https://emqx.atlassian.net/browse/EMQX-11942
https://github.com/emqx/emqx/issues/12587
2024-03-05 13:27:07 +01:00
46877e979b
chore: update copyright-year
2024-02-23 08:21:06 +01:00
d469f4158e
chore: bump app vsns
2024-02-20 16:53:57 -03:00
4afba8eb94
feat: port emqx/emqx-enterprise#1892 , add some SQL functions
2024-01-24 18:50:57 +08:00
cc24fe6e93
feat(mqtt_consumer): add support for rule engine `FROM`
2024-01-16 10:42:52 -03:00
23ded313ec
chore: update app versions
2023-12-22 15:29:22 +01:00
322b7bb7d2
chore: bump app vsn
2023-12-22 13:00:37 +01:00
18b69809da
Merge branch 'release-54' into sync-r54-m-20231221
2023-12-21 10:05:13 -03:00
b61ccf3373
fix(bridges_v1): don't list v2-only bridges in API
...
Fixes https://emqx.atlassian.net/browse/EMQX-11614
2023-12-20 10:46:35 -03:00
cf9331a95f
Merge branch 'release-54' into sync-r54-m-20231218
2023-12-18 17:21:08 -03:00
35504bd323
refactor: move ntoa (ip address formatting) code to emqx_utils
2023-12-17 21:03:16 +01:00
57a8f2351d
feat(rule_engine): also bump `unknown` counter for unrecoverable action errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-11494
2023-12-14 13:42:28 -03:00
99399c6e95
Merge pull request #12098 from thalesmg/revert-downgrade-bridge-id-m-20231205
...
chore: stop downgrading action id in rule API responses
2023-12-05 14:26:23 -03:00
e5a4be11f5
chore: stop downgrading action id in rule API responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-11405
2023-12-05 11:48:46 -03:00
cea857cb8e
fix(rule_api): fix sql test when testing against events
...
Fixes https://emqx.atlassian.net/browse/EMQX-11449
2023-12-04 15:24:09 -03:00
7f4d91d490
Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54
2023-12-01 08:32:22 +01:00
9fd2fa95a8
chore: bump apps versions
2023-11-30 20:01:12 +01:00
ec10c51073
Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53
2023-11-30 19:51:12 +01:00
0aec2f7605
feat: redis bridge v2
2023-11-29 16:06:46 +08:00
dc99651690
test(bridge): ensure almost test cases passed
2023-11-28 09:53:46 +08:00
84ff7b0b38
feat(emqx_bridge): action_info with dynamic lookup
...
This allows a n:1 relation between v1 bridge_types to action/connector types as
it's the case with mongodb for instance, where we had `mongodb_single`
`mongodb_sharded` etc and the new implementation will just have `mongodb`.
2023-11-20 10:22:37 +01:00
7c0e345d3a
Merge remote-tracking branch 'upstream/release-54'
2023-11-14 19:38:21 +01:00
28a577ad09
chore: bump apps versions
2023-11-14 11:02:26 +01:00
86110824eb
feat: upgrade hocon to 0.40.0 which supports union type display name
2023-11-10 13:41:51 +01:00
d603de10e6
refactor(schema): add namespace/0 to all schema modules
2023-11-10 13:41:51 +01:00
0bf5deaed9
chore: merge master into release-54
2023-11-09 08:32:26 +01:00
0c91bec98d
chore: merge 'upstream/release-53'
2023-11-08 09:24:38 +01:00
f17b762596
chore: don't disable rule that references non-existent bridge
...
After feedback from QA, we decided to rollback enforcing the rule to be disabled.
2023-11-06 10:09:14 -03:00
0ee74cd8ae
fix(rules_api): downgrade bridge id
...
Fixes https://emqx.atlassian.net/browse/EMQX-11312
2023-11-03 15:57:13 -03:00
c84e4a4187
fix(rule_engine): don't enable a rule that references non-existent bridge
2023-11-03 09:24:53 -03:00
910e81bc41
Merge pull request #10442 from keynslug/ft/EMQX-9257/placeholder
...
feat(tpl): split `emqx_placeholder` into a couple of modules
2023-11-02 22:50:05 +07:00
729c6edff6
chore(repub): simplify error handling in MQTT props templating
2023-11-02 20:13:32 +07:00
02c1bd70b6
feat(tpl): factor out loose json concept into a separate module
...
Which is called `emqx_jsonish`. Also introduce an _access module_
abstraction to extract information from such data during rendering.
2023-11-02 20:13:31 +07:00
69cfa740ea
fix(ruleeng): ensure full backward compatibility
2023-11-02 17:11:12 +07:00
75cc663786
chore(ruleeng): streamline application dependencies
2023-11-02 17:11:12 +07:00
8e4585d64f
chore: move template modules to `emqx_utils`
...
Even though most of the time these modules will be used by
connectors, there are exceptions (namely, `emqx_rule_engine`).
Besides, they are general enough to land there, more so given
that `emqx_placeholder` is already there.
2023-11-02 17:11:12 +07:00
f689d6c233
fix(tpl): ensure backward compat in `emqx_rule_engine`
...
Missing bindings in string templates will be rendered as "undefined",
as before. Rendering still assumes that missing binding with implicit
default (`undefined`) is an error.
This will also restore complete backward compat in `emqx_prometheus`.
2023-11-02 17:11:11 +07:00
49f5325c67
feat(tpl): unify validations / errors var representations
2023-11-02 17:11:10 +07:00
b812f9af5a
feat(tpl): use `emqx_connector_template` in `emqx_rule_engine` app
2023-11-02 17:11:10 +07:00
9dc3a169b3
feat: split bridges into a connector part and a bridge part
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.
Issues:
https://emqx.atlassian.net/browse/EMQX-10805
2023-10-30 14:48:47 +01:00
b5411da770
refactor: subscribe process to fix shared-sub
2023-10-27 09:23:43 +08:00
03d8e06ff7
chore(emqx_rule_funcs): regroup export functions
2023-10-09 20:03:21 +02:00
dc147fd310
fix(rule-engine): console action has no args field
2023-09-29 10:33:33 +02:00
dafd7c6085
chore: bump apps versions
2023-09-21 10:58:42 +02:00
105bebc250
chore: merge release-52 into master
2023-09-21 10:22:47 +02:00
5490807b20
chore: update apps/emqx_rule_engine/src/emqx_rule_engine_schema.erl
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-18 16:11:39 +08:00
ac41f497b4
feat: add bytesize rule sql function
2023-09-15 16:04:21 +02:00
32d838c85b
fix: format console action crash
2023-09-15 17:40:56 +08:00
d72348c8ed
fix: keep builtin_console actions not complain args
2023-09-15 16:38:04 +08:00
d0f1c75651
chore: bump apps vsns
2023-09-11 16:06:53 -03:00
e089fda260
Merge pull request #11568 from thalesmg/republish-props-m-20230905
...
feat(republish): allow templating mqtt properties
2023-09-07 16:43:00 -03:00
014bc64d3b
chore: ensure general prop is a binary
2023-09-07 13:40:03 -03:00
lafirest