Commit Graph

63 Commits

Author SHA1 Message Date
firest
8cee75139d fix(bridge): ensure the username of pgsql must exists 2023-07-07 18:06:28 +08:00
Paulo Zulato
8430ec673c feat(kafka): check whether target topic exists
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-21 15:00:35 -03:00
Paulo Zulato
9454af9a8b feat(postgresql): check whether target table exists
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 11:12:10 -03:00
Andrew Mayorov
d6c1ee183f
refactor(pluglib): move emqx_placeholder to utils app
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
JianBo He
dbc0cdce67 chore: dont allocate resource for simple connectiors 2023-06-05 10:59:00 +08:00
Thales Macedo Garitezi
0072f4a598 fix(ecpool,bridge): treat {error, ecpool_empty} as a retriable error
Example from flaky CI test:

```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
2023-06-01 17:13:06 -03:00
firest
6081ce8b00 feat: refactored some bridges to avoid leaking resources during crashes at creation 2023-05-31 16:48:13 +08:00
Thales Macedo Garitezi
377b143325 refactor: split parse_server into smaller functions, improve return type to use map 2023-04-24 14:17:29 -03:00
Andrew Mayorov
21e19a33ce
feat(respool): switch to emqx_resource_pool
Which was previously known as `emqx_plugin_libs_pool`. This is part
of the effort to get rid of `emqx_plugin_libs` application.
2023-04-18 12:51:14 +03:00
Stefan Strigler
9c11bfce80 refactor: rename emqx_misc to emqx_utils 2023-04-14 13:41:27 +02:00
Thales Macedo Garitezi
f3ffc02bff feat(bridges): enable async query mode for all bridges with buffer workers
Fixes https://emqx.atlassian.net/browse/EMQX-9130

Since buffer workers always support async calls ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.

For bridges that currently only allow sync query modes, we should
allow them to be configured with async.  That means basically all
bridge types except Kafka Producer.
2023-04-03 14:49:51 -03:00
Thales Macedo Garitezi
dbda504f2c fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0)
Fixes https://github.com/emqx/emqx/issues/9907

At v5.0.14, we changed the `ssl` option for the Postgres connector
from `true` to `required`, but there was another transformation in
`conn_opts/2` that led to an incorrect configuration.  This change
ended up preventing users from connecting to Postgres with their
previous configurations after upgrading EMQX.
2023-02-15 14:06:33 -03:00
Thales Macedo Garitezi
ca4a262b75 refactor: re-organize dealing with unrecoverable errors 2023-01-20 12:00:17 -03:00
Thales Macedo Garitezi
087b667263 fix(buffer_worker): allow signalling unrecoverable errors 2023-01-17 19:50:30 -03:00
Kjell Winblad
82e80e5610 fix: remove password from info logs
Information log messages for several connectors contained sensitive
password information. This commit fixes this by hiding the sensitive
information from the log messages.

Fixes: https://emqx.atlassian.net/browse/EMQX-8718
2023-01-12 09:28:01 +01:00
Shawn
3e9c4f444f refactor: remove the auto_reconnect field 2023-01-11 21:47:06 +08:00
firest
f7b50c56da feat(bridges): integrate PostgreSQL into bridges
support both simple and batch query
2023-01-06 11:47:22 +08:00
Zaiming (Stone) Shi
dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
Zaiming (Stone) Shi
0ce1ca89b7 refactor: use string type for server and servers 2022-12-30 14:20:23 +01:00
Zaiming (Stone) Shi
09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
JianBo He
39ad172377 chore: replace ip_port() wuth host_port() for all connectors 2022-09-27 15:19:42 +08:00
Zaiming (Stone) Shi
5a3fb4a011 Merge remote-tracking branch 'origin/master' into dev/ee5.0 2022-08-31 17:11:15 +02:00
ieQu1
b8a7ab9ad3 fix(connector_psql): Wrap password in a fun 2022-08-17 14:31:12 +02:00
Shawn
35fe70b887 feat: support aysnc callback to connector modules 2022-08-10 00:34:35 +08:00
Shawn
2fb42e4d37 refactor: create emqx_resource_worker_sup for resource workers 2022-08-10 00:34:35 +08:00
Shawn
841f816e39 refactor: improve the health_check for webhook bridges 2022-07-08 09:17:23 +08:00
JianBo He
8194427cb3 chore(pgsql): mapping ssl option from 'true' to 'required'
Once the pgsql client is configured with ssl, we should force ssl to be used to
avoid fallback to tcp connection.
2022-06-16 10:04:07 +08:00
DDDHuang
132b37813c refactor: code format emqx_connector emqx_resource 2022-04-28 15:32:47 +08:00
DDDHuang
2a2308bbf8 refactor: resource check & connector status 2022-04-28 15:32:35 +08:00
Zaiming (Stone) Shi
02c3f87b31 style: reformat all remaining apps 2022-04-27 15:51:18 +02:00
DDDHuang
7417e5070d fix: perpare sql when connector doing init; add prepare statement field; bad log path 2022-04-21 16:00:15 +08:00
EMQ-YangM
7fbabf5577 fix: connector docs 2022-04-19 10:43:53 +08:00
Zaiming (Stone) Shi
356fbf6ce1
Merge pull request #7519 from zmstone/0405-5.0-sync-ssl-lib-from-4.3
5.0 refactor: delete deprecated emqx_plugin_libs_ssl module
2022-04-05 22:57:56 +01:00
Zaiming (Stone) Shi
ba51f03cb7 refactor: remove the deprecated save_files_return_opts 2022-04-05 17:38:33 +02:00
ieQu1
99b3f2f83e docs(schema): Document missing records and fields 2022-04-04 23:58:43 +02:00
Zhongwen Deng
db584f79d6 feat: upgrade hocon to 0.25.0 to replace nullable with required. 2022-02-24 22:39:03 +08:00
JimMoen
ab7c2b72e3 refactor(connector): db connector provide default port 2022-02-15 00:51:48 +08:00
Chris
c2772fdd09 refactor: remove unused/old functions 2022-02-09 10:14:13 +01:00
EMQ-YangM
e9f3fa7b6b fix(emqx_connector): when start_pool fails, return error and reason 2022-01-26 16:07:46 +08:00
zhouzb
452f6e766a fix: return error when parse failed 2022-01-13 11:15:08 +08:00
zhouzb
2791052135 fix: fix errors caused by concurrent parsing of sql 2022-01-11 09:39:56 +08:00
Zaiming (Stone) Shi
63167cea70 chore: update copyright 2022-01-05 20:55:00 +01:00
Zaiming (Stone) Shi
2898fa76e1 Merge remote-tracking branch 'origin/release-5.0-beta.3' into merge-5.0-beta.3-to-master 2022-01-03 11:39:06 +01:00
zhongwencool
489fb7f806 fix(trace): copy binary:encode_hex/2 from binary.erl 2021-12-30 21:38:23 +08:00
zhongwencool
43141dffee
Merge branch 'release-5.0-beta.3' into trace-formatter 2021-12-29 22:37:59 +08:00
zhongwencool
121d906992 chore(log): add SLOG/3 to add meta info 2021-12-29 14:48:48 +08:00
zhongwencool
4b6bba11eb feat(trace): struct log for trace 2021-12-29 00:26:54 +08:00
zhouzb
1a22d5ca46 fix: fix undef function call 2021-12-28 11:21:41 +08:00
zhouzb
d26042703e chore: provide prepared_query and equery 2021-12-28 09:52:31 +08:00
zhouzb
fc89fb0f8a chore: using prepared query in postgresql connector 2021-12-28 09:16:00 +08:00