Commit Graph

39 Commits

Author SHA1 Message Date
firest a58ee801b2 fix(tdengine): enhanced health check result, make it more sense 2024-03-13 21:30:30 +08:00
Zaiming (Stone) Shi 46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
firest b4a6be0e7c fix(tdengine): remove redundant codes 2024-02-05 18:50:31 +08:00
firest 4a2c5a3161 fix(tdengine): keep the bridge schema still compatible with v1 2024-02-05 13:22:17 +08:00
firest 2241461acb chore: update change & bump version 2024-02-02 07:48:44 +08:00
firest dfad020c49 feat(tdengine): improve the TDengine bridge to v2 style 2024-02-01 23:14:24 +08:00
Zaiming (Stone) Shi 82403167c2 chore: update BSL license change date 2024-01-29 16:47:31 +01:00
JimMoen 5e100f52b8
style: erlfmt all `rebar.config` files and `bin/nodetool` 2023-12-29 09:08:03 +08:00
Andrew Mayorov 4385b2f020
feat(pgsql): accept wrapped secrets as passwords
That are coming from `emqx_schema_secret`. Also adapt pgsql-related
connectors.
2023-11-14 16:05:23 +07:00
zhongwencool 2f1fa2e961 chore: unified slog message formatting to improve logging consistency 2023-09-20 18:13:00 +08:00
Thales Macedo Garitezi 6cd503865b fix(machine_boot): ensure `emqx_bridge` starts after its companion apps
We need to reverse the dependency of `emqx_bridge` and `emqx_bridge_*`, because the former
loads and starts bridges during its application startup.  If the individual bridge
application being loaded has not started with its dependencies, the supervision tree will
not be ready for that.
2023-07-20 13:11:44 -03:00
firest 4f4a813f04 fix(tdengine): update the default template and the changelog 2023-07-18 10:31:18 +00:00
firest 749b8cd2ed test(tdengine): cover multi-table insert && subtable name generation 2023-07-13 07:39:01 +00:00
firest fef5087c41 fix(tdengine): fix SQL template errors
1. supports multi-table insert
2. supports constructing subtable names by mixed prefixes and placeholders
2023-07-13 04:44:48 +00:00
JimMoen b089fba100
refactor: rm ee_bridge and ee_connector application 2023-07-07 12:25:37 +08:00
firest 18dec53d8b feat: ensure data backends don't leak sensitive data 2023-07-07 11:04:13 +08:00
zhongwencool 07172e42f0 test: integer CI check failed 2023-06-20 08:39:23 +08:00
Andrew Mayorov b930f4cc73
Merge pull request #10987 from keynslug/fix/EMQX-9257/sep-placeholder
refactor: tear `emqx_plugin_libs` application apart
2023-06-09 17:02:14 +02:00
Andrew Mayorov 8919a6ef93
refactor(pluglib): provide SQL related utils in `emqx_utils_sql` 2023-06-09 14:44:37 +03:00
Andrew Mayorov d6c1ee183f
refactor(pluglib): move `emqx_placeholder` to utils app
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
Kjell Winblad ed9e29e769 refactor: refacor query_mode detection code
This commit refactor the query_mode resource detection code according to
a suggestion from @zmstone. This commit should not contain any
functional change except for a change of the Kafka producer bridge
config.
2023-06-08 16:26:55 +02:00
JianBo He dbc0cdce67 chore: dont allocate resource for simple connectiors 2023-06-05 10:59:00 +08:00
Thales Macedo Garitezi 33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
2023-06-02 16:20:36 -03:00
Thales Macedo Garitezi 0072f4a598 fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
Example from flaky CI test:

```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
2023-06-01 17:13:06 -03:00
Thales Macedo Garitezi 99796224d8 refactor(resource): rename `request_timeout` -> `request_ttl`
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 13:01:53 -03:00
Thales Macedo Garitezi 10425eb925 feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
firest ce32526c96 chore: bump version && update changes 2023-05-31 17:31:56 +08:00
firest 6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Zaiming (Stone) Shi 732a7be187 Merge remote-tracking branch 'origin/release-50' 2023-05-22 17:46:54 +02:00
Zaiming (Stone) Shi 7eef86363a test: make erlfmt happy 2023-05-19 17:58:14 +02:00
firest 5a08a7b9de fix(tdengine): minor improvement of code and changes 2023-05-19 11:14:31 +08:00
firest f1a3e5965e chore: update apps version && changes 2023-05-18 17:34:28 +08:00
firest 142125b9e4 test(tdengine): add test cases to cover the super table feature 2023-05-18 17:01:27 +08:00
firest 6ff77b221b fix(tdengine): add supports for the `automatically create` feature in the SQL template 2023-05-18 15:13:08 +08:00
JianBo He 383fec9dfc docs: fix invalid links 2023-05-15 11:18:09 +08:00
firest 0f979b3d24 refactor(tdengine): move tdengine bridge into its own app 2023-05-10 14:12:57 +08:00
JianBo He c53ccfea61 chore: fix typos 2023-04-18 13:48:55 +08:00
JianBo He ec8d8b805f chore: add ee bridge apps 2023-04-18 09:36:41 +08:00
JianBo He 7eacbffae9 chore: create dirs and README template for all ee bridges 2023-04-18 09:36:41 +08:00