Commit Graph

632 Commits

Author SHA1 Message Date
zhongwencool f6f1d32da0 feat: throttle with resource_id 2024-07-29 20:41:27 +02:00
zhongwencool 2924ec582a feat: add unrecoverable_resource_error throttle 2024-07-29 20:41:27 +02:00
zhongwencool 8dc1d1424a chore: add resource tag for log 2024-07-29 20:41:27 +02:00
zhongwencool e7d07ea17c feat: add resource_type to emqx_resource behaviour 2024-07-23 18:24:28 +08:00
zhongwencool e74a921d33 chore: compile error 2024-07-23 15:14:42 +08:00
zhongwencool 2a58a36e37 chore: add resource tag for log 2024-07-23 15:14:42 +08:00
zhongwencool 2bb062d3a3 chore: create_local/5 for emqx_resource_proto_v1 2024-07-23 15:14:42 +08:00
zhongwencool f29988ed8e chore: add tag to resouce log 2024-07-23 15:14:42 +08:00
zhongwencool e148d903e8 feat: log resource_id 2024-07-23 15:14:42 +08:00
zhongwencool 0a04b1ad6e feat: add group/type to resource slog 2024-07-23 15:14:41 +08:00
zhongwencool cba3f532f8 feat: don't record dry_run log 2024-07-23 15:14:41 +08:00
Thales Macedo Garitezi 9f8a1885a7 fix(resource manager): disentangle connector and channel health check frequencies
Fixes https://emqx.atlassian.net/browse/EMQX-12674
2024-07-09 14:53:39 -03:00
Thales Macedo Garitezi afb65817c4 ci(mix): refactor to avoid conflicts
This introduces a refactoring that will hopefully help avoid conflicts with ongoing work
to migrate our build / CI tooling to Mix.
2024-07-03 14:37:25 -03:00
zhongwencool 0dfa3e8c86 chore: ensure the module is loaded 2024-07-03 10:11:33 +08:00
zhongwencool e1c3b7587d feat: do not fail on other nodes when the RPC succeeds on the first node 2024-07-03 09:49:13 +08:00
Thales Macedo Garitezi 19f3b030f9 chore: preparing to run common tests / eunit with mix 2024-07-02 09:45:45 -03:00
Thales Macedo Garitezi b74828d7ea perf: "mixify" project to improve release speed
Fixes https://emqx.atlassian.net/browse/EMQX-12527
2024-07-02 09:39:48 -03:00
Thales Macedo Garitezi 4bd0abc93f chore: bump app vsns 2024-06-27 09:22:06 -03:00
Thales Macedo Garitezi ed5e6599d9 fix(buffer worker, kafka): send reply when async call fails immediately
Fixes https://emqx.atlassian.net/browse/EMQX-12585
2024-06-27 09:20:00 -03:00
zmstone 98f70ea8d8 Merge remote-tracking branch 'origin/release-572' into release-57 2024-06-26 22:28:08 +02:00
Thales Macedo Garitezi 954adc71c4 test: attempt to fix flaky test
https://github.com/emqx/emqx/actions/runs/9662725303/job/26653594859?pr=13328#step:6:186

```
%%% emqx_resource_SUITE ==> t_expiration_retry: FAILED
%%% emqx_resource_SUITE ==> {{panic,
     #{msg => "Unexpected result",
       result =>
           {run_stage_failed,error,
               {badmatch,{ok,timeout}},
               [{emqx_resource_SUITE,'-do_t_expiration_retry/0-fun-12-',0,
                    [{file,
                         "/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
                     {line,2569}]},
                {emqx_resource_SUITE,do_t_expiration_retry,0,
                    [{file,
                         "/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
                     {line,2518}]}]}}},
 [{emqx_resource_SUITE,do_t_expiration_retry,0,
      [{file,"/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
       {line,2594}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1302}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1234}]}]}
```
2024-06-26 13:40:49 -03:00
zmstone 5d1608f34b chore: minimize oss/platform diff 2024-06-24 15:30:03 +02:00
Thales Macedo Garitezi 9dd419d822 test(greptime): attempt to fix flaky tests
```
=WARNING REPORT==== 18-Jun-2024::12:42:04.739109 ===
    reason: no_endpoints
    msg: failed_to_start_greptimedb_connector
    client: #{protocol => http,
              pool => <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>,
              cli_opts =>
                  [{endpoints,[{http,"toxiproxy",4001}]},
                   {pool_size,2},
                   {pool_type,random},
                   {auto_reconnect,1},
                   {gprc_options,#{sync_start => true,
                                   connect_timeout => 5000}},
                   {dbname,"public"},
                   {auth,{basic,#{password => <<"******">>,
                                  username => "greptime_user"}}},
                   {https_enabled,false}]}
    connector: <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>
```
2024-06-18 18:23:22 -03:00
Thales Macedo Garitezi 3c501e4f2a fix: remove another redundant health check
Fixes https://emqx.atlassian.net/browse/EMQX-12521
2024-06-10 15:13:34 -03:00
Thales Macedo Garitezi 28a3c77e88 chore: remove redundant health check call
Since `emqx_resource_manager:channel_health_check` already does a base resource health
check, the call was repeated.
2024-06-07 11:07:12 -03:00
Thales Macedo Garitezi 3013189cd7 fix(resource manager): force kill process if stuck when stopping/removing
Fixes https://emqx.atlassian.net/browse/EMQX-12357
2024-06-04 11:38:24 -03:00
Thales Macedo Garitezi 802361dbd0 fix(connector api): handle `timeout` when waiting for connector status
Fixes https://emqx.atlassian.net/browse/EMQX-12251
2024-05-28 14:19:32 -03:00
Kjell Winblad 88c96e26de refactor: simplify the code with maps:map/2
Thanks @thalesmg for the suggestion
2024-05-22 18:03:43 +02:00
Kjell Winblad 4b540e3bd0 fix: do not leak action configurations in alarm messages 2024-05-22 17:43:07 +02:00
Kjell Winblad cff8b97e8a fix: handle channel updated during health check
This commit fixes an issue found by CI test case
emqx_bridge_influxdb_SUITE:t_start_stop and others. While the channel
health check process is running, the channel could be removed or updated
which could cause a crash in the resource manager or non up-to-date
alarms being triggered.
2024-05-22 12:15:04 +02:00
Kjell Winblad 39d758c4d6 fix: do not return configs for channels from emqx_resource_manager 2024-05-22 12:15:04 +02:00
Kjell Winblad 917474f694 fix: action config update would sometimes not be reflected in connector
Before this commit the following happened sometimes:

1. action status is connected
2. action config is updated to something that should change the status to
   disconnected
3. action status is still connected and the old config is being used by
   the connector even though the config has been correctly updated.

The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).

The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00
Kjell Winblad 3c87bcde46 fix(rule trace): restore logger metadata to its previous value 2024-05-20 10:57:15 +02:00
Kjell Winblad 5ce095f30e fix(rule trace): only include stop_after_rendering when value is true 2024-05-17 14:53:14 +02:00
Kjell Winblad 2e6db85578 fix(rule trace): rename rule_trigger_time(s) and cleaups
This commit renames trace fields rule_trigger_time and
rule_trigger_times to rule_trigger_ts and makes sure that the value for
rule_trigger_ts will always be a list of timestamps.
2024-05-17 14:13:36 +02:00
firest 437a0c0d55 fix(rocketmq): fix a typo and a RocketMQ encoding error 2024-05-14 23:01:31 +08:00
Kjell Winblad 09ee7ec0e2 fix(rule tracing): make sure that recoverable errors are traced 2024-05-07 15:49:38 +02:00
Kjell Winblad 5bfe31b691 fix: issues found during PR review (thanks @thalesmg and @zmstone)
* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
  maintainability
* Added callback to format return value from connector
  * Extended test case to check that the format return value callback
    works
* Added handling of "lazy" trace entry data to the text formatter. Do we
  need to handle this data in the normal log formatters as well?
2024-05-03 15:04:57 +02:00
Thales Macedo Garitezi 5cf92dcb73 refactor: use `spawn_link` instead of `spawn_monitor`
This should cover the case when the resource manager is brutally killed.
2024-04-29 15:20:32 -03:00
Thales Macedo Garitezi 475077c798 fix(resource): account for ongoing channel health checks, update data and reply immediately when receiving an update 2024-04-29 15:20:32 -03:00
Thales Macedo Garitezi 79526d539a fix(resource manager): clean up any running health checks when terminating
Fixes https://github.com/emqx/emqx/pull/12812#discussion_r1555564254
2024-04-29 15:20:32 -03:00
Kjell Winblad e9d498dde2 feat(rule tracing): add rule trigger time meta data field
Fixes:
https://emqx.atlassian.net/browse/EMQX-12025
2024-04-22 13:51:48 +02:00
Kjell Winblad 6e99f01ecd
Merge pull request #12863 from kjellwinblad/kjell/new_trace_log_formatter/EMQX-12025
feat(emqx_trace): add JSON trace log entry formatter
2024-04-22 06:25:18 +02:00
JianBo He 69bdcd2f24
Merge pull request #12898 from emqx/iotdb-1.3.0
feat: support iotdb 1.3.0
2024-04-22 10:56:06 +08:00
ieQu1 bac5100635
Merge pull request #12874 from ieQu1/dev/EMQX-12030-subscriptions-api
API for durable subscriptions
2024-04-19 17:11:53 +02:00
Zaiming (Stone) Shi a8b1224225
Merge pull request #12893 from zmstone/0416-add-is-template-flag-to-dashboard-schema
0416 add `is_template` flag to dashboard schema
2024-04-19 10:41:43 +02:00
zhongwencool b163a87386 feat: support batch_size on iotdb 2024-04-19 14:52:56 +08:00
ieQu1 d12966db5b
test: Avoid dumping raw snabbkaffe traces to the console 2024-04-19 01:20:29 +02:00
Kjell Winblad 3232ab5ea3 fix(rule tracing): unset trace meta data in try-after-end
We wrap the reset of the process trace meta data in the after clause of
a try-after-end expression to be sure we never get any lingering
incorrect meta data.
2024-04-18 11:28:50 +02:00
Kjell Winblad 2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
Add rule ID tracing support
2024-04-18 09:06:31 +02:00