Commit Graph

605 Commits

Author SHA1 Message Date
Kjell Winblad 88c96e26de refactor: simplify the code with maps:map/2
Thanks @thalesmg for the suggestion
2024-05-22 18:03:43 +02:00
Kjell Winblad 4b540e3bd0 fix: do not leak action configurations in alarm messages 2024-05-22 17:43:07 +02:00
Kjell Winblad cff8b97e8a fix: handle channel updated during health check
This commit fixes an issue found by CI test case
emqx_bridge_influxdb_SUITE:t_start_stop and others. While the channel
health check process is running, the channel could be removed or updated
which could cause a crash in the resource manager or non up-to-date
alarms being triggered.
2024-05-22 12:15:04 +02:00
Kjell Winblad 39d758c4d6 fix: do not return configs for channels from emqx_resource_manager 2024-05-22 12:15:04 +02:00
Kjell Winblad 917474f694 fix: action config update would sometimes not be reflected in connector
Before this commit the following happened sometimes:

1. action status is connected
2. action config is updated to something that should change the status to
   disconnected
3. action status is still connected and the old config is being used by
   the connector even though the config has been correctly updated.

The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).

The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00
Kjell Winblad 3c87bcde46 fix(rule trace): restore logger metadata to its previous value 2024-05-20 10:57:15 +02:00
Kjell Winblad 5ce095f30e fix(rule trace): only include stop_after_rendering when value is true 2024-05-17 14:53:14 +02:00
Kjell Winblad 2e6db85578 fix(rule trace): rename rule_trigger_time(s) and cleaups
This commit renames trace fields rule_trigger_time and
rule_trigger_times to rule_trigger_ts and makes sure that the value for
rule_trigger_ts will always be a list of timestamps.
2024-05-17 14:13:36 +02:00
firest 437a0c0d55 fix(rocketmq): fix a typo and a RocketMQ encoding error 2024-05-14 23:01:31 +08:00
Kjell Winblad 09ee7ec0e2 fix(rule tracing): make sure that recoverable errors are traced 2024-05-07 15:49:38 +02:00
Kjell Winblad 5bfe31b691 fix: issues found during PR review (thanks @thalesmg and @zmstone)
* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
  maintainability
* Added callback to format return value from connector
  * Extended test case to check that the format return value callback
    works
* Added handling of "lazy" trace entry data to the text formatter. Do we
  need to handle this data in the normal log formatters as well?
2024-05-03 15:04:57 +02:00
Thales Macedo Garitezi 5cf92dcb73 refactor: use `spawn_link` instead of `spawn_monitor`
This should cover the case when the resource manager is brutally killed.
2024-04-29 15:20:32 -03:00
Thales Macedo Garitezi 475077c798 fix(resource): account for ongoing channel health checks, update data and reply immediately when receiving an update 2024-04-29 15:20:32 -03:00
Thales Macedo Garitezi 79526d539a fix(resource manager): clean up any running health checks when terminating
Fixes https://github.com/emqx/emqx/pull/12812#discussion_r1555564254
2024-04-29 15:20:32 -03:00
Kjell Winblad e9d498dde2 feat(rule tracing): add rule trigger time meta data field
Fixes:
https://emqx.atlassian.net/browse/EMQX-12025
2024-04-22 13:51:48 +02:00
Kjell Winblad 6e99f01ecd
Merge pull request #12863 from kjellwinblad/kjell/new_trace_log_formatter/EMQX-12025
feat(emqx_trace): add JSON trace log entry formatter
2024-04-22 06:25:18 +02:00
JianBo He 69bdcd2f24
Merge pull request #12898 from emqx/iotdb-1.3.0
feat: support iotdb 1.3.0
2024-04-22 10:56:06 +08:00
ieQu1 bac5100635
Merge pull request #12874 from ieQu1/dev/EMQX-12030-subscriptions-api
API for durable subscriptions
2024-04-19 17:11:53 +02:00
Zaiming (Stone) Shi a8b1224225
Merge pull request #12893 from zmstone/0416-add-is-template-flag-to-dashboard-schema
0416 add `is_template` flag to dashboard schema
2024-04-19 10:41:43 +02:00
zhongwencool b163a87386 feat: support batch_size on iotdb 2024-04-19 14:52:56 +08:00
ieQu1 d12966db5b
test: Avoid dumping raw snabbkaffe traces to the console 2024-04-19 01:20:29 +02:00
Kjell Winblad 3232ab5ea3 fix(rule tracing): unset trace meta data in try-after-end
We wrap the reset of the process trace meta data in the after clause of
a try-after-end expression to be sure we never get any lingering
incorrect meta data.
2024-04-18 11:28:50 +02:00
Kjell Winblad 2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
Kjell Winblad b16b9d8fcc refactor: made code more readable thanks to suggestion from @zmstone 2024-04-18 06:51:30 +02:00
zmstone 6ab2b004ed fix(resource_manager): update cache after channel add 2024-04-17 20:34:33 +02:00
Kjell Winblad cf56050759 feat: avoid mixing request with and without the stop_after_render flag
Previously a batch of requests that was sent to a connector could
contain both requests with the stop_after_rendering flag and requests
without this flag. When this happened a warning message was generated and
the stop_after_render flags for the batch would be ignored. This commit
fixes so that a mixed batch is never created so there is no longer any
need for a warning message or ignoring flags.
2024-04-17 16:21:21 +02:00
Kjell Winblad b57725f996 test(resource_SUITE): do test case fixes needed due to rule tracing work 2024-04-08 12:41:35 +02:00
Kjell Winblad ef705c2285 feat: add apply rule API, clientid/ruleid tracing for rule and connector
This commit adds:

* Support for forwarding the rule id and client id to the connector so
  that events such as template rendered successfully can be traced.
* HTTP API for for applying/activating a rule with the given context
2024-04-06 17:20:47 +02:00
Thales Macedo Garitezi 60cad74286 feat(resource): non-blocking channel health checks
Fixes https://emqx.atlassian.net/browse/EMQX-12015

Continuation of https://github.com/emqx/emqx/pull/12812
2024-04-04 16:13:30 -03:00
Thales Macedo Garitezi bade09b56e feat(resource manager): perform non-blocking resource health checks
Fixes https://emqx.atlassian.net/browse/EMQX-12015

This introduces only _resource_ non-blocking health checks.  _Channel_ non-blocking health
checks may be introduced later.
2024-04-01 14:46:15 -03:00
JimMoen 9143d5994d
feat: refactor MS SQL Server bridge to connector and action 2024-03-04 17:13:21 +08:00
zhongwencool 54c542c795 chore: rename _probe_ to t_probe_ 2024-02-27 17:57:15 +08:00
zhongwencool c67f2130f2 fix: check connector and bridge_v2 with the right schema 2024-02-27 17:57:10 +08:00
Thales Macedo Garitezi 15f919e60f
Merge pull request #12564 from thalesmg/bw-support-batch-list-resp-m-20240221
feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
2024-02-23 09:37:42 -03:00
Zaiming (Stone) Shi 46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
Thales Macedo Garitezi d003f77021 feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
Fixes https://emqx.atlassian.net/browse/EMQX-11892

This allows callers of batching resources to receive results specific to their requests,
rather than a broad success or failure for the whole batch.
2024-02-22 16:26:02 -03:00
Zaiming (Stone) Shi 415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
Thales Macedo Garitezi 4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
Zaiming (Stone) Shi b14e51d7c4 docs: delete useless comments 2024-02-21 13:33:29 +01:00
Zaiming (Stone) Shi c8f9e12de1 refactor: deprecate emqx_resource_proto_v1 with v2 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi 06e0a7f2ed refactor: delete emqx_resource:remove api
this api is only used in tests,
changed to call the _local flavor instead
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi 8541d2efc1 refactor: delete emqx_resource:recreate
this api is only used in tests, changed to call the _local flavor
instead
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi 1878fded5a refactor: delete emqx_resource:create/4 and /5
the functions are only used in tests,
now the tests are rewritten to call create_local/4 and /5 instead
2024-02-21 10:53:22 +01:00
Thales Macedo Garitezi d469f4158e chore: bump app vsns 2024-02-20 16:53:57 -03:00
Thales Macedo Garitezi 9a32895a1a feat: convert `gcp_pubsub_consumer` to connector/source
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
zhongwencool 42a4bc3382
Merge pull request #12435 from zhongwencool/dont-add-disable-resource-to-connector
fix: don't add disable bridge to connector's channel
2024-02-01 10:39:47 +08:00
zhongwencool f9d16340e0 fix: don't add disable bridge to connector's channel 2024-02-01 09:12:43 +08:00
zhongwencool e9c8446d57 feat: upgrade erlfmt to support maybe syntax 2024-01-31 20:06:49 +08:00
Zaiming (Stone) Shi 55fdc83aa5 Merge remote-tracking branch 'origin/release-55' into 0130-syn-release-55-to-master 2024-01-30 12:06:43 +01:00
Thales Macedo Garitezi 6410f5a717 fix(resource_metrics): avoid detaching handler on crashes
Fixes https://emqx.atlassian.net/browse/EMQX-11821
2024-01-26 13:05:41 -03:00