Commit Graph

237 Commits

Author SHA1 Message Date
Shawn 6fde37791c refactor: new metrics for resources 2022-08-30 10:14:10 +08:00
Shawn 1625b8eaeb fix(mysql_bridge): export the query_mode option to the APIs 2022-08-26 17:11:24 +08:00
Shawn 6b0ccfbc43 refactor: rename the error return resource_down -> recoverable_error 2022-08-26 17:11:12 +08:00
Shawn a896aa8b27 fix: incorrect replayq dir for the emqx_resource 2022-08-25 16:06:18 +08:00
Shawn 86577365e4 fix: use gen_statem:cast/3 for async query 2022-08-23 22:41:45 +08:00
JimMoen f0c2b53868 fix(bpapi): make bpapi static_checks happy 2022-08-22 10:51:44 +08:00
JimMoen 62ecf6f545 fix(resource): keep `auto_retry` in `disconnected` state
Automatic retries should be maintained even in `disconnected` state without any state transition.
2022-08-22 02:52:06 +08:00
JimMoen 7c4ea38c06 fix(resource): make some resource opts internal
Resource options `start_after_created` and `start_timeout` are internal opts.
Not provided to users anymore.
2022-08-22 02:22:57 +08:00
JimMoen 06363e63d9 fix(influxdb): connector use a fallbacke `pool_size` for influxdb client 2022-08-19 15:54:19 +08:00
Shawn 9e35032d78 fix: make resume_interval defaults to health_check_interval 2022-08-16 10:09:02 +08:00
Shawn de3a325953 fix: revert the changes in connector mysql 2022-08-16 09:06:13 +08:00
Xinyu Liu 2898966439
Merge branch 'dev/ee5.0' into resource_opts 2022-08-15 21:43:22 +08:00
Shawn 19d85d485b refactor(resource): add resource_opts level into config structure 2022-08-15 21:40:10 +08:00
Shawn d1de262f31 fix: inc 'actions.failed' if bridge query failed 2022-08-15 17:21:14 +08:00
Shawn 665ef4142d fix: unify the health check interval 2022-08-15 17:21:14 +08:00
JimMoen 68946f1f6c feat: influxdb support `async`/`batch_async` query 2022-08-15 14:02:17 +08:00
JimMoen b01ae8ece6 chore: refine influxdb bridge/connector i18n 2022-08-15 14:00:14 +08:00
JimMoen 594d071c05 feat(influxdb): add async callback 2022-08-12 18:26:47 +08:00
JimMoen fa5e8f1422 chore: refine i18n label 2022-08-12 16:39:03 +08:00
JimMoen 3678673124 fix: schema default value using raw type before convert 2022-08-12 16:38:46 +08:00
Shawn 0cdf4b47f1 feat: add more resource creation opts 2022-08-12 13:47:45 +08:00
Shawn c3c4ed02b4 fix: bump emqx_dashboard to 5.0.4 2022-08-12 00:24:58 +08:00
JimMoen 3a76a50382 fix: syntax error and compile error 2022-08-11 20:58:43 +08:00
Shawn 2872f0b668 fix(bridges): support create resources with options 2022-08-11 19:11:44 +08:00
JimMoen 0f6c371760 feat(influxdb): influxdb connector add `on_batch_query/3` callback 2022-08-11 18:12:41 +08:00
JimMoen 22a4ca311c feat(resource): resource batch/async/queue config schema 2022-08-11 16:59:18 +08:00
Shawn 6203a01320 feat: add inflight window to emqx_resource 2022-08-11 08:36:35 +08:00
Shawn 82550a585a fix: add test cases for query async 2022-08-10 00:45:34 +08:00
Shawn efd6c56dd9 fix: test cases for batch query sync 2022-08-10 00:45:34 +08:00
Shawn 145ff66a9a fix: issues found by dialyzer and elvis 2022-08-10 00:45:26 +08:00
Shawn 35fe70b887 feat: support aysnc callback to connector modules 2022-08-10 00:34:35 +08:00
Shawn f1419d52f1 fix(resource): remove resource at the end of each test 2022-08-10 00:34:35 +08:00
Shawn a2afdeeb48 feat: add test cases for batching query 2022-08-10 00:34:35 +08:00
Shawn 75adba0781 fix: increase resource metrics using the resource id 2022-08-10 00:34:35 +08:00
Shawn d3950b9534 fix(resource): make option 'queue_enabled' disabled by default 2022-08-10 00:34:35 +08:00
Shawn 0377d3cf61 fix: update existing testcases for new emqx_resource 2022-08-10 00:34:35 +08:00
Shawn 2fb42e4d37 refactor: create emqx_resource_worker_sup for resource workers 2022-08-10 00:34:35 +08:00
Shawn 0087b7c960 fix: remove the extra file replay.erl 2022-08-10 00:34:35 +08:00
Shawn d8d8d674e4 feat(resource): start emqx_resource_worker in pools 2022-08-10 00:34:35 +08:00
Shawn 12904d797f feat(resource): first commit for batching/async/caching mechanism 2022-08-10 00:34:35 +08:00
DDDHuang 98b36c4681 fix: hstream db connector , TODO: start apps 2022-07-27 11:38:45 +08:00
JianBo He a78a389206 chore: using standard log format 2022-07-01 12:06:35 +08:00
Shawn d6ef2f7502 refactor: graceful recreate resources 2022-06-17 05:29:18 +08:00
Shawn cc25f92273 feat: add start_after_created option to resource:create/4 2022-06-16 23:34:52 +08:00
Zaiming (Stone) Shi 2065be569e fix(emqx_cluster_rpc): fail fast on stale state
Due to:

* Cluster RPC MFA is not idempotent!
* There is a lack of rollback for callback's side-effects

For instance, when two nodes try to add a cluster-singleton
concurrently, one of them will have to wait for the table lock
then try to catch-up, then try to apply MFA.
The catch-up will have the singleton created, but the initiated
initiated multicall apply will fail causing the commit to rollback,
but not to 'undo' the singleton creation.
Later, the retries will fail indefinitely.
2022-06-12 20:18:48 +02:00
Shawn b7f27157e5 fix: also alarm resource down when start resource failed 2022-06-01 15:41:55 +08:00
Shawn 88ca25c60c fix(resource): fast return when starting a unavailable resource 2022-06-01 08:24:53 +08:00
Shawn 9f69e3cad6 fix(resource): discard dry_run resource down alarm 2022-06-01 08:24:53 +08:00
Shawn d37a66e9b8 fix(test): update test cases for emqx_resource:health_check/1 2022-05-31 10:14:37 +08:00
Shawn 1054c364ad refactor(resource): improve health check and alarm it if resource down 2022-05-31 01:40:40 +08:00
EMQ-YangM 574a40b327 fix: wait for test_resource stop 2022-05-16 17:00:42 +08:00
EMQ-YangM b5addf7e05 fix: log all ignore events 2022-05-16 15:08:03 +08:00
EMQ-YangM bbbfea1b5b fix: ignore all other events 2022-05-16 15:08:03 +08:00
EMQ-YangM 1a1c82932a fix: when connecting health check failed, update status. 2022-05-16 10:47:20 +08:00
Chris 93799e3ac6 refactor: delete now unused emqx_resource modules 2022-05-16 09:54:26 +08:00
Xinyu Liu c4fd31ae25
Merge pull request #7916 from emqx/EMQX-4204-auto-timer-based-retry-when-in-disconnected-state
feat: add auto_retry for disconnected state in resource manager
2022-05-16 09:34:08 +08:00
JianBo He 3f59650e4b
Merge pull request #7944 from EMQ-YangM/fix_bridge_status
fix: restart resource should not clear metrics
2022-05-16 09:16:12 +08:00
Zaiming (Stone) Shi c355c40ea8 refactor: call emqx_alarm:ensure_deactivated everywhere 2022-05-13 16:02:55 +02:00
Chris 6574c33797 feat: add auto_retry for disconnected state in resource manager 2022-05-13 11:19:39 +02:00
EMQ-YangM d5c416736b fix: restart resource should not clear metrics 2022-05-13 16:05:41 +08:00
JimMoen a5ddc5390f refactor(resource): add resource recreate fun with empty opts 2022-05-12 14:19:56 +08:00
Chris 0b3e30e813 feat: isolate resource manager processes 2022-05-09 13:24:34 +02:00
EMQ-YangM c52b464b3c fix: check process alive before health check 2022-04-29 17:34:26 +08:00
EMQ-YangM 1bf33f75cc fix: set resource status disconnected 2022-04-29 17:05:12 +08:00
Zaiming (Stone) Shi 4e65322667 refactor: move emqx_plugin_libs_metrics to emqx app
because it can not depend on other apps
2022-04-29 12:41:36 +08:00
DDDHuang 132b37813c refactor: code format emqx_connector emqx_resource 2022-04-28 15:32:47 +08:00
DDDHuang 667da90e52 refactor: resource instance do_create_dry_run 2022-04-28 15:32:41 +08:00
DDDHuang 2a2308bbf8 refactor: resource check & connector status 2022-04-28 15:32:35 +08:00
DDDHuang a50980c496 fix: disconnected status in auto_reconnect = false 2022-04-28 09:47:36 +08:00
Zaiming (Stone) Shi 02c3f87b31 style: reformat all remaining apps 2022-04-27 15:51:18 +02:00
Shawn 94e24c2621 refactor: move ssl file handling from resources to bridges 2022-04-27 11:59:15 +08:00
Zaiming (Stone) Shi f42a5b90df Revert "feat: isolate resource manager processes"
This reverts commit 40cca58d4f.
2022-04-26 16:13:38 +02:00
Chris 40cca58d4f feat: isolate resource manager processes 2022-04-26 13:28:29 +02:00
Shawn 19630e9a99 feat: save ssl cert files for data bridges 2022-04-21 09:00:06 +08:00
Shawn 3ce969fd79 refactor: always recreate resources no matter it is connected or not 2022-04-20 11:43:05 +08:00
Shawn 278e9145b0 fix: go to different resource instance when health check 2022-04-19 23:00:34 +08:00
Ilya Averyanov e5f04f3bf7 chore(emqx_authn_jwt): wrap JWKS connector into emqx_resourse 2022-04-18 15:47:33 +03:00
EMQ-YangM 8f06a9ec62 feat: impl resource reset_metrics 2022-04-11 10:25:48 +08:00
EMQ-YangM 9a2d70f98e fix(emqx_resource): remove extra space 2022-03-25 18:26:18 +08:00
EMQ-YangM 6b662d87ba fix(emqx_resource): fix dialyzer warning 2022-03-25 18:15:23 +08:00
EMQ-YangM bb12378806 fix(emqx_resource_instance): improve the pattern match of the function call_health_check 2022-03-25 17:35:55 +08:00
zhongwencool 3414e0b601 feat(plugin): http api 2022-03-11 15:55:02 +08:00
Shawn 1d023b541f refactor(connnector): rename waiting_connect_complete -> wait_for_resource_ready
Rename the option to wait_for_resource_ready and defaults to 5s.
2022-03-10 10:46:57 +08:00
EMQ-YangM 583624fb8d fix(emqx_authn): fix test suite 2022-03-09 13:53:05 +08:00
EMQ-YangM db0e9e3358 fix(emqx_resource_instance): fix dialyzer warning 2022-03-08 14:09:39 +08:00
EMQ-YangM f29877bb6a fix(emqx_resource): remove create_opts async_create 2022-03-08 14:09:39 +08:00
EMQ-YangM bf57bf717c feat(emqx_resource_instance): add wait_connected Opts 2022-03-08 14:09:14 +08:00
Xinyu Liu 47a4fa5732
Merge pull request #7140 from EMQ-YangM/tmp_change_status
refactor(emqx_resource): change the status of emqx_resource to 'conne…
2022-02-28 11:13:47 +08:00
EMQ-YangM 376c9ee261 refactor(emqx_resource): change the status of emqx_resource to 'connected/connecting/disconnecting' 2022-02-25 15:02:41 +08:00
Zhongwen Deng db584f79d6 feat: upgrade hocon to 0.25.0 to replace nullable with required. 2022-02-24 22:39:03 +08:00
EMQ-YangM 48942f9c93 refactor(emqx_resource): move unused macro to test 2022-02-14 17:40:39 +08:00
EMQ-YangM df57daaabb refactor(emqx_resource): improve grouping strategy for emqx_resource_instance 2022-02-11 18:36:55 +08:00
Chris c2772fdd09 refactor: remove unused/old functions 2022-02-09 10:14:13 +01:00
EMQ-YangM 3b3f3fed76 refactor(emqx_resource_instance): add function create_default_checker 2022-02-07 17:49:51 +08:00
EMQ-YangM 8976e25b7a fix(emqx_resource_instance): create health_check process when create resources synchronously 2022-02-07 17:49:51 +08:00
Zaiming (Stone) Shi f7703c906e refactor: move plain_check impl to emqx_hocon 2022-01-29 10:16:55 +01:00
Zaiming (Stone) Shi d6f7ffec3c
Merge pull request #6881 from zmstone/refactor-upgrade-hocon-0.23.0
refactor: upgrade to hocon 0.23.0
2022-01-27 15:48:23 +01:00
Kian-Meng Ang fc92e4c8bf docs: fix typos 2022-01-27 12:29:38 +01:00
Zaiming (Stone) Shi cf1f19258e refactor(emqx_resource): catch only hocon throw exceptions 2022-01-27 00:23:08 +01:00
Zaiming (Stone) Shi 252d7e85d9 refactor: call new hocon apis 2022-01-27 00:23:08 +01:00