Commit Graph

498 Commits

Author SHA1 Message Date
Ivan Dyachkov f01e2f358b
Merge pull request #10367 from id/0411-sync-release-50-back-to-master
0411 sync release 50 back to master
2023-04-12 17:23:17 +02:00
Ivan Dyachkov bdffa925db chore: merge upstream/master release-50 2023-04-12 15:30:20 +02:00
JianBo He d16b5c40d6
Merge pull request #10356 from HJianBo/async-batch-cassa
feat: support async and batch callback for cassandra connector
2023-04-12 20:59:36 +08:00
JianBo He 9560fdc5a2 chore: typo fixes 2023-04-12 14:16:40 +08:00
Thales Macedo Garitezi 914184697e
Merge pull request #10337 from thalesmg/schema-registry-v50
feat: implement schema registry for 5.0 (avro)
2023-04-11 16:46:27 -03:00
Thales Macedo Garitezi 97b94b8882 feat: implement schema registry for 5.0 (avro)
Part of https://emqx.atlassian.net/browse/EMQX-9251

This ports part of the Schema Registry app from 4.x to 5.0.  Here,
only support for Avro is added.  Subsequent PRs will follow to add
support for other formats.
2023-04-11 14:50:45 -03:00
JianBo He c50326360f chore: update ecql version 2023-04-10 18:03:58 +08:00
JianBo He 30bdffe318 feat: support async and batch callback for cassandra connector 2023-04-10 15:08:10 +08:00
Thales Macedo Garitezi 33100ecca6 feat: implement schema registry for 5.0 (avro)
Part of https://emqx.atlassian.net/browse/EMQX-9251

This ports part of the Schema Registry app from 4.x to 5.0.  Here,
only support for Avro is added.  Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
Thales Macedo Garitezi 4c24b08244 fix(rule_action): fix metrics for bridges returning `async_return`
Kafka Producer, when called asynchronously, will return
`{async_return, {ok, pid()}}`, which currently counts as an unknown failure.
2023-04-06 16:00:01 -03:00
Zaiming (Stone) Shi cf71f9148e
Merge pull request #10287 from thalesmg/fix-kprod-zh-desc-mqtt-topic-rv50
docs(kafka_producer): update ZH translation of field (rv5.0)
2023-04-06 16:13:41 +02:00
Thales Macedo Garitezi 5d5b7ea215
Merge pull request #10306 from thalesmg/enable-async-buffer-workers-all-bridges-v50
feat(bridges): enable async query mode for all bridges with buffer workers
2023-04-04 17:10:46 -03:00
Thales Macedo Garitezi 0b6fd7fe14 fix(buffer_worker): check request timeout and health check interval
Port of https://github.com/emqx/emqx/pull/10154 for `release-50`

Fixes https://emqx.atlassian.net/browse/EMQX-9099

Originally, the `resume_interval`, which is what defines how often a
buffer worker will attempt to retry its inflight window, was set to
the same as the `health_check_interval`.  This had the problem that,
with default values, `health_check_interval = request_timeout`.  This
meant that, if a buffer worker with those configs were ever blocked,
all requests would have timed out by the time it retried them.

Here we change the default `resume_interval` to a reasonable value
dependent on `health_check_interval` and `request_timeout`, and also
expose that as a hidden parameter for fine tuning if necessary.
2023-04-04 08:58:36 -03:00
firest 11b3264251 fix: redact the password to `******` in API examples
fix #10222
2023-04-04 16:02:45 +08:00
lafirest ed31b735a8
Merge pull request #10220 from lafirest/bump/hackney
chore: bump hackney version
2023-04-04 10:35:09 +08:00
Thales Macedo Garitezi a8f8228a12
Merge pull request #10308 from thalesmg/test-increase-peer-timeout-v50
test(peer): increase init and startup timeout for peer nodes
2023-04-03 15:46:13 -03:00
Thales Macedo Garitezi f3ffc02bff feat(bridges): enable async query mode for all bridges with buffer workers
Fixes https://emqx.atlassian.net/browse/EMQX-9130

Since buffer workers always support async calls ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.

For bridges that currently only allow sync query modes, we should
allow them to be configured with async.  That means basically all
bridge types except Kafka Producer.
2023-04-03 14:49:51 -03:00
Thales Macedo Garitezi 8b5a717a1f test(peer): increase init and startup timeout for peer nodes
Attempt to stabilize tests that use cluster nodes.
2023-04-03 13:20:22 -03:00
Thales Macedo Garitezi ec1871ffde test(janitor): catch each callback invocation 2023-04-03 10:20:19 -03:00
Thales Macedo Garitezi 51eec32d6b
docs: improve descriptions
Co-authored-by: LenaLenaPan <120552185+LenaLenaPan@users.noreply.github.com>
2023-04-03 09:11:14 -03:00
Kjell Winblad 7c60a1e276
Merge pull request #10255 from emqx/Meggielqk-patch-1
Doc: Refined Chinese UI
2023-04-03 13:31:20 +02:00
Zaiming (Stone) Shi 677b76afce refactor: replace hidden => true with importance => hidden 2023-04-03 13:13:43 +02:00
Zaiming (Stone) Shi 36000abf51 refactor: relocate i18n files for apps/emqx 2023-04-03 13:12:24 +02:00
firest c8dca74b18 chore: bump hackney version 2023-04-03 18:41:49 +08:00
lafirest 9c7c49e9bf
Merge pull request #10201 from lafirest/fix/tdengine
fix(tdengine): remove the redundant table name in SQL template
2023-04-03 18:13:41 +08:00
zhongwencool d63680cf25
Merge pull request #10307 from emqx/release-50
Sync release-50 back to master
2023-04-02 11:36:41 +08:00
Thales Macedo Garitezi 246a792965
Merge pull request #10273 from thalesmg/refactor-kprod-start-error-msg-rv50
fix: return friendly message when kafka producer and consumer fails to start (rv5.0)
2023-03-31 16:25:26 -03:00
Kjell Winblad 58898ea11d
Merge pull request #10294 from kjellwinblad/kjell/feat/collection_var_syntax_mongodb/EMQX-9246
feat: (MongoDB bridge) use ${var} syntax in MongoDB collection field
2023-03-31 17:18:27 +02:00
Thales Macedo Garitezi 5011486b18 fix(kafka_consumer): return better error messages when probing kafka consumer bridge
Fixes https://emqx.atlassian.net/browse/EMQX-9422
2023-03-31 11:33:15 -03:00
Kjell Winblad fe400c7f8d
Merge pull request #10272 from kjellwinblad/kjell/better_label
docs: make clickhouse config label for Server URL more concise
2023-03-31 16:17:26 +02:00
Zaiming (Stone) Shi 945c26d027
Merge pull request #10291 from zmstone/0330-fix-flaky-test-influxdb
test: fix flaky influxdb test
2023-03-31 15:36:12 +02:00
Zaiming (Stone) Shi 9bbca07943
Merge pull request #10298 from emqx/release-50
Merge release-50 back to master
2023-03-31 15:35:51 +02:00
Thales Macedo Garitezi aabbbf5deb
docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-31 09:07:00 -03:00
Kjell Winblad 6f6d12f930
docs: better Chinese labels
Co-authored-by: LenaLenaPan <120552185+LenaLenaPan@users.noreply.github.com>
2023-03-31 09:39:04 +02:00
Kjell Winblad e808fef1e4 feat: (MongoDB bridge) use ${var} syntax for MongoDB collection
This commit makes it possible to use the ${var} syntax to refer to
variables in the payload of the message in the collection field.
This makes it possible to select which collection to insert into
dynamically.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9246
2023-03-30 17:49:56 +02:00
Thales Macedo Garitezi 632bffd451 fix: return friendly message when kafka producer fails to start (rv5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9392

The returned information does not allow to diagnose the issue (i.e.: a
connection issue due to the wrong host and port, the wrong password
failing authn).  However, such information is printed to the logs.

This changes the returned error to the API so that the user is hinted
at looking at the logs for further investigation of the error.
2023-03-30 11:51:36 -03:00
Zaiming (Stone) Shi 81a104690d test: fix flaky influxdb test 2023-03-30 16:19:22 +02:00
Kjell Winblad 8c4d211b54 fix: on_get_status clickhouse just reporting connecting
The on_get_status callback for clickhouse just returned `connecting`
without error information when the status check was unsuccessful. This is
fixed by letting the callback return error information similarly to how
the HTTP connector does it.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9374
https://emqx.atlassian.net/browse/EMQX-9278
2023-03-30 15:10:03 +02:00
Kjell Winblad 91a784134c docs: update Chinese version of batch value separator label 2023-03-30 14:55:58 +02:00
Kjell Winblad aefa0e5ffb docs: make clickhouse config label for Server URL more concise 2023-03-30 14:55:22 +02:00
Meggielqk dfc9141da6 docs: Refined Chinese UI
Also updated the English hints according to the refined Chinese according to Guowei's comments.

Co-authored-by: Kjell Winblad <kjellwinblad@gmail.com>
2023-03-30 14:50:24 +02:00
Thales Macedo Garitezi b3c6ae99d2
Merge pull request #10269 from thalesmg/tdengine-required-password-rv50
fix(tdengine): make password a required field (rv5.0)
2023-03-30 09:25:38 -03:00
Thales Macedo Garitezi 07e1f2e440 docs(kafka_producer): update ZH translation of field (rv5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9411
2023-03-30 09:18:16 -03:00
Kjell Winblad 808bf1d232
Merge pull request #10270 from kjellwinblad/kjell/clickhouse/on_get_status
fix: on_get_status clickhouse just reporting connecting
2023-03-30 10:57:48 +02:00
firest ee5b9d7e0a fix(tdengine): remove the redundant table name in SQL template 2023-03-30 10:34:10 +08:00
Zaiming (Stone) Shi 94726eb7f1 docs: refine zh translations 2023-03-29 15:07:19 +02:00
Thales Macedo Garitezi b2b9732edb fix(tdengine): make password a required field (rv5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9395

```
2023-03-29T09:59:02.655495+00:00 [warning] msg: start_resource_failed, mfa: emqx_resource_manager:start_resource/2, line: 524, id: <<"bridge:tdengine:jimmoen-test-trigger">>, reason: {error,function_clause,[{emqx_ee_connector_tdengine,on_start,[<<"bridge:tdengine:jimmoen-test-trigger:1076">>,#{database => <<"db">>,enable => true,pool_size => 8,resource_opts => #{auto_restart_interval => 60000,batch_size => 1,batch_time => 0,health_check_interval => 15000,max_queue_bytes => 104857600,query_mode => sync,request_timeout => 15000,start_after_created => true,start_timeout => 5000,worker_pool_size => 16},server => "127.0.0.1",sql => <<"insert into mqtt.t_mqtt_msg(ts, msgid, mqtt_topic, qos, payload, arrived) values (${ts}, ${id}, ${topic}, ${qos}, ${payload}, ${timestamp})">>,username => <<"root">>}],[{file,"emqx_ee_connector_tdengine.erl"},{line,76}]},{emqx_resource,call_start,3,[{file,"emqx_resource.erl"},{line,359}]},{emqx_resource_manager,start_resource,2,[{file,"emqx_resource_manager.erl"},{line,513}]},{gen_statem,loop_state_callback,11,[{file,"gen_statem.erl"},{line,1205}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
```
2023-03-29 10:07:02 -03:00
Kjell Winblad c693851740 fix: on_get_status clickhouse just reporting connecting
The on_get_status callback for clickhouse just returned `connecting`
without error information when the status check was unsuccessful. This is
fixed by letting the callback return error information similarly to how
the HTTP connector does it.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9374
https://emqx.atlassian.net/browse/EMQX-9278
2023-03-29 14:26:08 +02:00
Zaiming (Stone) Shi 80eb9d7542
Merge pull request #10252 from emqx/release-50
0327 merge release-50 to master
2023-03-29 12:33:17 +02:00
Zaiming (Stone) Shi 624c168dde fix: hide nodelay Kafka client socket option
also fix a few descriptions
2023-03-29 11:42:17 +02:00