Commit Graph

650 Commits

Author SHA1 Message Date
Ivan Dyachkov cc3b26a3ac Merge remote-tracking branch 'upstream/release-58' into 20240807-sync-release-branches 2024-08-07 09:48:38 +02:00
Ivan Dyachkov 4865999606 Merge remote-tracking branch 'upstream/master' into release-58 2024-08-05 10:59:59 +02:00
Thales Macedo Garitezi 9f97bff7d0 feat: expose `resource_opts.query_mode` for pulsar action
Fixes https://emqx.atlassian.net/browse/EMQX-12782
2024-07-31 11:13:11 -03:00
Ivan Dyachkov 74cef7937d Merge remote-tracking branch 'upstream/release-57' into 20240731-sync-release-57 2024-07-31 11:31:29 +02:00
zhongwencool f6f1d32da0 feat: throttle with resource_id 2024-07-29 20:41:27 +02:00
zhongwencool 2924ec582a feat: add unrecoverable_resource_error throttle 2024-07-29 20:41:27 +02:00
zhongwencool 8dc1d1424a chore: add resource tag for log 2024-07-29 20:41:27 +02:00
Thales Macedo Garitezi d2da311416 fix(resource): create undocumented callback
Created by https://github.com/emqx/emqx/pull/13449 but not added as a callback.
2024-07-24 10:53:33 -03:00
Ivan Dyachkov 606d829256 Merge remote-tracking branch 'upstream/release-57' into 20240724-sync-release-57 2024-07-24 10:28:00 +02:00
zhongwencool e7d07ea17c feat: add resource_type to emqx_resource behaviour 2024-07-23 18:24:28 +08:00
zhongwencool e74a921d33 chore: compile error 2024-07-23 15:14:42 +08:00
zhongwencool 2a58a36e37 chore: add resource tag for log 2024-07-23 15:14:42 +08:00
zhongwencool 2bb062d3a3 chore: create_local/5 for emqx_resource_proto_v1 2024-07-23 15:14:42 +08:00
zhongwencool f29988ed8e chore: add tag to resouce log 2024-07-23 15:14:42 +08:00
zhongwencool e148d903e8 feat: log resource_id 2024-07-23 15:14:42 +08:00
zhongwencool 0a04b1ad6e feat: add group/type to resource slog 2024-07-23 15:14:41 +08:00
zhongwencool cba3f532f8 feat: don't record dry_run log 2024-07-23 15:14:41 +08:00
Thales Macedo Garitezi 220fbe8a0a test: fix flaky test 2024-07-22 09:44:51 -03:00
Thales Macedo Garitezi 8d535bbd24
Merge pull request #13464 from thalesmg/20240712-m-res-manager-shutdown-logs
chore: attempt to reduce race condition supervisor noproc shutdown error logs
2024-07-19 14:57:56 -03:00
Thales Macedo Garitezi eb2d3a3b7e chore: attempt to reduce race condition supervisor shutdown errors
Fixes https://emqx.atlassian.net/browse/EMQX-12442

e.g.:
```
2024-05-23T08:52:39.811845+00:00 [error] Supervisor: {local,emqx_resource_manager_sup}. Context: shutdown_error. Reason: noproc. Offender: id=<<99, 101, 110, 115, 111, 114, 101, 100>>,pid=<0.7752.1030>.
```

It could be just a race condition, as it seems to be the case for resource manager: i) a call is made to the process to stop it; ii) the call times out; iii) the after clause ends up calling supervisor:terminate_child; iv) while the supervisor is finding the child to terminate, the process actually finishes terminating, and the supervisor receives a noproc reason back.
2024-07-19 10:57:00 -03:00
Ivan Dyachkov 292b331064 Merge remote-tracking branch 'upstream/release-57' into 20240717-sync-release-57 2024-07-17 11:29:25 +02:00
Thales Macedo Garitezi 9f8a1885a7 fix(resource manager): disentangle connector and channel health check frequencies
Fixes https://emqx.atlassian.net/browse/EMQX-12674
2024-07-09 14:53:39 -03:00
Thales Macedo Garitezi 32ace85e1c Merge branch 'release-57' into 20240703-m-sync-r57 2024-07-03 16:03:15 -03:00
Thales Macedo Garitezi afb65817c4 ci(mix): refactor to avoid conflicts
This introduces a refactoring that will hopefully help avoid conflicts with ongoing work
to migrate our build / CI tooling to Mix.
2024-07-03 14:37:25 -03:00
zhongwencool cfa7c3bf04 Merge remote-tracking branch 'upstream/release-57' into 20240702-m-sync-r57-mix-umbrella 2024-07-03 15:15:58 +08:00
zhongwencool 0dfa3e8c86 chore: ensure the module is loaded 2024-07-03 10:11:33 +08:00
zhongwencool e1c3b7587d feat: do not fail on other nodes when the RPC succeeds on the first node 2024-07-03 09:49:13 +08:00
Thales Macedo Garitezi 5532f40d83 Merge branch 'release-57' into 20240702-m-sync-r57-mix-umbrella 2024-07-02 11:52:36 -03:00
Thales Macedo Garitezi 19f3b030f9 chore: preparing to run common tests / eunit with mix 2024-07-02 09:45:45 -03:00
Thales Macedo Garitezi b74828d7ea perf: "mixify" project to improve release speed
Fixes https://emqx.atlassian.net/browse/EMQX-12527
2024-07-02 09:39:48 -03:00
Ivan Dyachkov e28750b522 Merge remote-tracking branch 'upstream/release-57' into 0701-sync-release-57 2024-07-01 13:52:14 +02:00
Thales Macedo Garitezi 4bd0abc93f chore: bump app vsns 2024-06-27 09:22:06 -03:00
Thales Macedo Garitezi ed5e6599d9 fix(buffer worker, kafka): send reply when async call fails immediately
Fixes https://emqx.atlassian.net/browse/EMQX-12585
2024-06-27 09:20:00 -03:00
zmstone 98f70ea8d8 Merge remote-tracking branch 'origin/release-572' into release-57 2024-06-26 22:28:08 +02:00
Thales Macedo Garitezi 954adc71c4 test: attempt to fix flaky test
https://github.com/emqx/emqx/actions/runs/9662725303/job/26653594859?pr=13328#step:6:186

```
%%% emqx_resource_SUITE ==> t_expiration_retry: FAILED
%%% emqx_resource_SUITE ==> {{panic,
     #{msg => "Unexpected result",
       result =>
           {run_stage_failed,error,
               {badmatch,{ok,timeout}},
               [{emqx_resource_SUITE,'-do_t_expiration_retry/0-fun-12-',0,
                    [{file,
                         "/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
                     {line,2569}]},
                {emqx_resource_SUITE,do_t_expiration_retry,0,
                    [{file,
                         "/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
                     {line,2518}]}]}}},
 [{emqx_resource_SUITE,do_t_expiration_retry,0,
      [{file,"/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
       {line,2594}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1302}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1234}]}]}
```
2024-06-26 13:40:49 -03:00
zmstone 5d1608f34b chore: minimize oss/platform diff 2024-06-24 15:30:03 +02:00
Thales Macedo Garitezi ec83d999bf Merge remote-tracking branch 'origin/release-57' into 20240621-sync-r57-m 2024-06-21 14:11:08 -03:00
Thales Macedo Garitezi 9dd419d822 test(greptime): attempt to fix flaky tests
```
=WARNING REPORT==== 18-Jun-2024::12:42:04.739109 ===
    reason: no_endpoints
    msg: failed_to_start_greptimedb_connector
    client: #{protocol => http,
              pool => <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>,
              cli_opts =>
                  [{endpoints,[{http,"toxiproxy",4001}]},
                   {pool_size,2},
                   {pool_type,random},
                   {auto_reconnect,1},
                   {gprc_options,#{sync_start => true,
                                   connect_timeout => 5000}},
                   {dbname,"public"},
                   {auth,{basic,#{password => <<"******">>,
                                  username => "greptime_user"}}},
                   {https_enabled,false}]}
    connector: <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>
```
2024-06-18 18:23:22 -03:00
Thales Macedo Garitezi db572d35a7 Merge remote-tracking branch 'origin/release-57' into sync-r57-m-20240611 2024-06-11 15:34:54 -03:00
Thales Macedo Garitezi 3c501e4f2a fix: remove another redundant health check
Fixes https://emqx.atlassian.net/browse/EMQX-12521
2024-06-10 15:13:34 -03:00
Thales Macedo Garitezi 28a3c77e88 chore: remove redundant health check call
Since `emqx_resource_manager:channel_health_check` already does a base resource health
check, the call was repeated.
2024-06-07 11:07:12 -03:00
zmstone ebf17c8143 Merge remote-tracking branch 'origin/release-57' into 0606-merge-release-57-to-master 2024-06-06 17:43:36 +02:00
Thales Macedo Garitezi 3013189cd7 fix(resource manager): force kill process if stuck when stopping/removing
Fixes https://emqx.atlassian.net/browse/EMQX-12357
2024-06-04 11:38:24 -03:00
Ilya Averyanov db713cb656 chore(bridges): increase log level for bridge buffer overflow reports
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2024-05-31 16:35:18 +03:00
Thales Macedo Garitezi 802361dbd0 fix(connector api): handle `timeout` when waiting for connector status
Fixes https://emqx.atlassian.net/browse/EMQX-12251
2024-05-28 14:19:32 -03:00
Kjell Winblad 88c96e26de refactor: simplify the code with maps:map/2
Thanks @thalesmg for the suggestion
2024-05-22 18:03:43 +02:00
Kjell Winblad 4b540e3bd0 fix: do not leak action configurations in alarm messages 2024-05-22 17:43:07 +02:00
Kjell Winblad cff8b97e8a fix: handle channel updated during health check
This commit fixes an issue found by CI test case
emqx_bridge_influxdb_SUITE:t_start_stop and others. While the channel
health check process is running, the channel could be removed or updated
which could cause a crash in the resource manager or non up-to-date
alarms being triggered.
2024-05-22 12:15:04 +02:00
Kjell Winblad 39d758c4d6 fix: do not return configs for channels from emqx_resource_manager 2024-05-22 12:15:04 +02:00
Kjell Winblad 917474f694 fix: action config update would sometimes not be reflected in connector
Before this commit the following happened sometimes:

1. action status is connected
2. action config is updated to something that should change the status to
   disconnected
3. action status is still connected and the old config is being used by
   the connector even though the config has been correctly updated.

The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).

The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00