firest
0b46acda87
test(opents): add test cases for OpenTSDB
2023-04-23 11:03:20 +08:00
firest
d826b0921d
fix(dynamo): separate the implementation of connector and client of Dynamo bridge
2023-04-23 10:57:36 +08:00
firest
5074825075
feat(opents): OpenTSDB integration
2023-04-23 09:56:24 +08:00
Ilya Averyanov
ebb75b275e
feat(ft): update app versions
2023-04-21 17:52:13 +03:00
Ilya Averyanov
0211bcf030
Merge branch 'master' into file-transfer
...
* master: (279 commits)
chore: shorten ct/run.sh script
chore: rename cassandra_impl to cassandra_connector
chore: fix mix.exs checking
refactor(cassandra): move cassandra bridge into its own app
chore: apply review suggestions
chore: update changes/ce/fix-10449.en.md
test: add a test for authn {}
chore: add changlog for authn_http validation
fix: always check authn_http's header and ssl_option
chore: apply suggestions from code review
fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
ci: simplify find-apps.sh for ee apps
perf(emqx_resource): don't reactivate alarms on reoccurring errors
ci: check if Elixir files are formatted in pre-commit hook
fix(dynamo): fix field name errors
chore: remove *_collector for prometheus api's example
chore: make plugins config to low level
chore: re-split dynamo i18n file
...
2023-04-21 17:37:17 +03:00
JianBo He
b270623c46
chore: rename cassandra_impl to cassandra_connector
2023-04-21 16:49:25 +08:00
firest
7704995279
fix(rocketmq): expose the driver parameter `sync_timeout` into the RocketMQ bridge configuration
2023-04-21 15:10:25 +08:00
JianBo He
5cc28a7b45
chore: fix mix.exs checking
2023-04-21 14:46:09 +08:00
JianBo He
bdce32e713
refactor(cassandra): move cassandra bridge into its own app
2023-04-21 14:45:38 +08:00
JianBo He
6532cfb337
Merge pull request #10452 from thalesmg/refactor-gcp-pubsub-bridge-v50
...
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
2023-04-21 14:42:00 +08:00
lafirest
48402d0476
Merge pull request #10461 from lafirest/fix/dynamo_db_svrs
...
fix(rocketmq): allow setting multiple addresses in RocketMQ bridge
2023-04-21 14:37:37 +08:00
firest
f602900a53
fix(rocketmq): fix that the status check of RocketMQ bridge may not accurate
2023-04-21 13:35:29 +08:00
firest
d865998a63
fix(rocketmq): fix test cases
2023-04-21 11:02:14 +08:00
firest
21473e7ca5
fix(dynamo): fix field name errors
2023-04-20 18:16:04 +08:00
firest
6e12abff39
fix(rocketmq): allow setting multiple addresses in RocketMQ bridge
2023-04-20 17:58:12 +08:00
Zaiming (Stone) Shi
415830a0a2
Merge remote-tracking branch 'origin/release-50' into 0420-sync-release-50-to-masteer
2023-04-20 11:03:39 +02:00
Kjell Winblad
7d3367467a
Merge pull request #10408 from kjellwinblad/kjell/rule_engine/add_missing_mongo_date_functions/EMQX-9244
...
feat: add mongo_date functions to the rule engine
2023-04-20 09:46:45 +02:00
Thales Macedo Garitezi
a9bd91fcff
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
...
Fixes https://emqx.atlassian.net/browse/EMQX-9536
Note: since GCP PubSub is not shared by any authn/authz backend,
there's no need to separate its connector into another app.
2023-04-19 13:24:32 -03:00
firest
e89f4d4565
fix(dynamo): fix terminology erros
...
- Changed `username` to `aws_access_key_id`
- Changed `password` to `aws_secret_access_key`
2023-04-19 16:26:42 +08:00
firest
bc353b0a06
fix(dynamo): change `database` to `table` in the schema of the DynamoDB bridge
...
there is no term like `database` in DynamoDB, the correct concept should be `table`
2023-04-19 15:56:29 +08:00
Zaiming (Stone) Shi
9efbb71ae8
Merge pull request #10441 from zmstone/0418-fix-flakyness
...
test: pick random port number for gcp pubsub mock server
2023-04-18 19:11:30 +02:00
Ivan Dyachkov
dc78ecb41c
chore: merge upstream/master
2023-04-18 17:33:32 +02:00
JianBo He
ad4b7d59eb
Merge pull request #10440 from HJianBo/fix-cassa-flaky-test
...
chore: fix flaky test in emqx_ee_bridge_cassa_SUITE
2023-04-18 22:44:54 +08:00
Andrew Mayorov
75813683be
Merge pull request #10371 from ft/EMQX-9257/resource-pool-switch
...
feat(respool): switch to `emqx_resource_pool`
2023-04-18 16:58:17 +03:00
JianBo He
87c5170696
chore: fix flaky test
2023-04-18 21:33:41 +08:00
Zaiming (Stone) Shi
97b8f00508
test: pick random port number for gcp pubsub mock server
2023-04-18 15:07:57 +02:00
Andrew Mayorov
a08c000222
test(mongo): do not rely on internals to clean database
2023-04-18 12:51:17 +03:00
Andrew Mayorov
21e19a33ce
feat(respool): switch to `emqx_resource_pool`
...
Which was previously known as `emqx_plugin_libs_pool`. This is part
of the effort to get rid of `emqx_plugin_libs` application.
2023-04-18 12:51:14 +03:00
Kjell Winblad
8a3fccb330
style: fix variable name style
2023-04-18 10:32:37 +02:00
Kjell Winblad
29584ca721
feat: add mongo_date functions to the rule engine
...
This commit adds mong_date built-in functions to the rule engine
SQL-like language. Corresponding functions already existed in EMQX 4.4
and this commit makes sure that EMQX 5.X also has these functions.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9244
2023-04-18 10:01:35 +02:00
Zaiming (Stone) Shi
cd1197925f
test: fix unit tests after type names changed
2023-04-18 09:48:28 +02:00
Thales Macedo Garitezi
9d15247dd5
feat(schema_registry): add support for protobuf schemas
...
Fixes https://emqx.atlassian.net/browse/EMQX-9470
2023-04-17 13:04:54 -03:00
Ivan Dyachkov
9fc8a498f8
chore: bump apps versions
2023-04-17 09:09:08 +02:00
Stefan Strigler
7df0493312
Merge pull request #10390 from sstrigler/EMQX-9549-new-emqx-utils-app-to-collect-utility-modules
...
New emqx_utils app to collect utility modules
2023-04-14 20:33:11 +02:00
Stefan Strigler
f0c13e0134
fix: stale ref to emqx_map_lib
2023-04-14 18:43:53 +02:00
Thales Macedo Garitezi
e073bc90bc
refactor(buffer_worker): rename `s/queue/buffer/g`
2023-04-14 11:37:19 -03:00
Stefan Strigler
edd1bc579f
fix: stale ref to emqx_json after rebase
2023-04-14 16:32:42 +02:00
Stefan Strigler
4f2f0ebb6a
fix: bad ref to emqx_misc after rebase
2023-04-14 14:58:19 +02:00
Stefan Strigler
062ce5f819
refactor: rename emqx_map_lib to emqx_utils_maps
2023-04-14 13:41:34 +02:00
Stefan Strigler
6e8665365b
refactor: rename emqx_tables to emqx_utils_ets
2023-04-14 13:41:33 +02:00
Stefan Strigler
d0df086c80
refactor: rename emqx_api_lib to emqx_utils_api
2023-04-14 13:41:33 +02:00
Stefan Strigler
9c11bfce80
refactor: rename emqx_misc to emqx_utils
2023-04-14 13:41:27 +02:00
Stefan Strigler
f8e9e54393
refactor: move emqx_json to emqx_utils_json
2023-04-14 13:31:27 +02:00
JimMoen
9ce0dbae7b
test: fix batch_size parameter
2023-04-14 13:11:45 +08:00
JimMoen
2484a79c7a
test: create bridge with invalid password
2023-04-14 12:52:08 +08:00
JimMoen
8cec62bd87
fix: async reply arg list
2023-04-14 12:43:36 +08:00
JimMoen
65c4234829
refactor: rename instance_id, which is also PoolName
2023-04-14 10:02:48 +08:00
JimMoen
2d1b94f7f7
fix: file mode 755 magic number
2023-04-14 10:02:48 +08:00
JimMoen
675f9894ab
fix: fix single quotes not applied for single query
...
- alwasy use `emqx_plugin_libs_rule:proc_batch_sql/3` to add single quotes for string
2023-04-14 10:02:47 +08:00
JimMoen
f8c0aa49a2
style: make elvis and shellcheck happy
2023-04-14 10:02:47 +08:00
JimMoen
437096985b
test: fix SUITE case `t_create_disconnected/1`
2023-04-14 10:02:47 +08:00
JimMoen
27fdf644aa
fix(sqlserver): let a single query be a length 1 batch
...
Need to be fixed:
`emqx_plugin_libs_rule:proc_tmpl/2` won't add single quotes for string
2023-04-14 10:02:46 +08:00
JimMoen
a379d909bf
test: use type VARCHAR to use utf8 encoding in sqlserver
2023-04-14 10:02:46 +08:00
JimMoen
c366267b0f
test: MS SQL Server data bridge
2023-04-14 10:02:46 +08:00
JimMoen
5841969877
feat: implement Microsoft SQL Server bridge (e5.0)
2023-04-14 10:02:45 +08:00
JimMoen
8e62afecdd
chore: suite module rename
2023-04-14 09:57:00 +08:00
JimMoen
8ceeafb0de
chore: fix file license in lib-ee
2023-04-14 09:56:59 +08:00
Thales Macedo Garitezi
9acfe00498
Merge pull request #10347 from thalesmg/refactor-kafka-bridge-dirs-v50
...
refactor(kafka_bridge): move kafka bridge into its own app
2023-04-13 13:26:36 -03:00
Kjell Winblad
601564fa0a
Merge pull request #10324 from kjellwinblad/kjell/fix/EMQX-9374
...
fix: check Clickhouse connection after creation
2023-04-13 16:31:35 +02:00
Kjell Winblad
0cec52d5e4
fix: check Clickhouse connection after creation
...
When pressing reconnect on a badly configured Clickhouse bridge in the
dashboard, no error message was shown. This commit fixes this issue by
testing the connection after creation and returning an error tuple if
the connection is not working.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9374
2023-04-13 10:09:35 +02:00
firest
a4d9234b24
test(dynamo): remove the flaky test case
2023-04-13 12:02:22 +08:00
Thales Macedo Garitezi
871ee90b3e
refactor(kafka_bridge): move kafka bridge into its own app
...
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-12 13:54:45 -03:00
Ivan Dyachkov
f01e2f358b
Merge pull request #10367 from id/0411-sync-release-50-back-to-master
...
0411 sync release 50 back to master
2023-04-12 17:23:17 +02:00
Ivan Dyachkov
bdffa925db
chore: merge upstream/master release-50
2023-04-12 15:30:20 +02:00
JianBo He
d16b5c40d6
Merge pull request #10356 from HJianBo/async-batch-cassa
...
feat: support async and batch callback for cassandra connector
2023-04-12 20:59:36 +08:00
JianBo He
9560fdc5a2
chore: typo fixes
2023-04-12 14:16:40 +08:00
Thales Macedo Garitezi
914184697e
Merge pull request #10337 from thalesmg/schema-registry-v50
...
feat: implement schema registry for 5.0 (avro)
2023-04-11 16:46:27 -03:00
Thales Macedo Garitezi
97b94b8882
feat: implement schema registry for 5.0 (avro)
...
Part of https://emqx.atlassian.net/browse/EMQX-9251
This ports part of the Schema Registry app from 4.x to 5.0. Here,
only support for Avro is added. Subsequent PRs will follow to add
support for other formats.
2023-04-11 14:50:45 -03:00
JianBo He
c50326360f
chore: update ecql version
2023-04-10 18:03:58 +08:00
JianBo He
30bdffe318
feat: support async and batch callback for cassandra connector
2023-04-10 15:08:10 +08:00
Andrew Mayorov
0d86ef8d3a
fix(ft-s3): use AWS4 signed urls for S3 export URIs
...
This will ensure that the S3 export URIs are valid in all AWS regions.
2023-04-07 22:15:01 +03:00
Ilya Averyanov
6fbc7dc822
fix(ft): remove s3 roots from ee schema
2023-04-07 17:36:47 +03:00
Ilya Averyanov
d21daae7ae
fix(ft): fix dynamic function calls
2023-04-07 17:36:47 +03:00
Ilya Averyanov
eae2478678
fix(ft): add descriptions for missing ft schema fields
2023-04-07 17:36:47 +03:00
Ilya Averyanov
2f1970adbc
fix(ft): make ee schema aware of ee fields
2023-04-07 17:36:47 +03:00
Ilya Averyanov
18aa39f892
fix(ft): add emqx_ft_schema to ee schemas
2023-04-07 17:36:47 +03:00
Ilya Averyanov
c7865e5eae
fix(ft-s3): fix dependency dependencies
2023-04-07 17:36:45 +03:00
Ilya Averyanov
71965d90e4
fix(ft-s3): synchronize erlcloud version across apps
2023-04-07 17:26:53 +03:00
Thales Macedo Garitezi
33100ecca6
feat: implement schema registry for 5.0 (avro)
...
Part of https://emqx.atlassian.net/browse/EMQX-9251
This ports part of the Schema Registry app from 4.x to 5.0. Here,
only support for Avro is added. Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
Thales Macedo Garitezi
4c24b08244
fix(rule_action): fix metrics for bridges returning `async_return`
...
Kafka Producer, when called asynchronously, will return
`{async_return, {ok, pid()}}`, which currently counts as an unknown failure.
2023-04-06 16:00:01 -03:00
Zaiming (Stone) Shi
cf71f9148e
Merge pull request #10287 from thalesmg/fix-kprod-zh-desc-mqtt-topic-rv50
...
docs(kafka_producer): update ZH translation of field (rv5.0)
2023-04-06 16:13:41 +02:00
Thales Macedo Garitezi
5d5b7ea215
Merge pull request #10306 from thalesmg/enable-async-buffer-workers-all-bridges-v50
...
feat(bridges): enable async query mode for all bridges with buffer workers
2023-04-04 17:10:46 -03:00
Thales Macedo Garitezi
0b6fd7fe14
fix(buffer_worker): check request timeout and health check interval
...
Port of https://github.com/emqx/emqx/pull/10154 for `release-50`
Fixes https://emqx.atlassian.net/browse/EMQX-9099
Originally, the `resume_interval`, which is what defines how often a
buffer worker will attempt to retry its inflight window, was set to
the same as the `health_check_interval`. This had the problem that,
with default values, `health_check_interval = request_timeout`. This
meant that, if a buffer worker with those configs were ever blocked,
all requests would have timed out by the time it retried them.
Here we change the default `resume_interval` to a reasonable value
dependent on `health_check_interval` and `request_timeout`, and also
expose that as a hidden parameter for fine tuning if necessary.
2023-04-04 08:58:36 -03:00
firest
11b3264251
fix: redact the password to `******` in API examples
...
fix #10222
2023-04-04 16:02:45 +08:00
lafirest
ed31b735a8
Merge pull request #10220 from lafirest/bump/hackney
...
chore: bump hackney version
2023-04-04 10:35:09 +08:00
Thales Macedo Garitezi
a8f8228a12
Merge pull request #10308 from thalesmg/test-increase-peer-timeout-v50
...
test(peer): increase init and startup timeout for peer nodes
2023-04-03 15:46:13 -03:00
Thales Macedo Garitezi
f3ffc02bff
feat(bridges): enable async query mode for all bridges with buffer workers
...
Fixes https://emqx.atlassian.net/browse/EMQX-9130
Since buffer workers always support async calls ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.
For bridges that currently only allow sync query modes, we should
allow them to be configured with async. That means basically all
bridge types except Kafka Producer.
2023-04-03 14:49:51 -03:00
Thales Macedo Garitezi
8b5a717a1f
test(peer): increase init and startup timeout for peer nodes
...
Attempt to stabilize tests that use cluster nodes.
2023-04-03 13:20:22 -03:00
Thales Macedo Garitezi
ec1871ffde
test(janitor): catch each callback invocation
2023-04-03 10:20:19 -03:00
Thales Macedo Garitezi
51eec32d6b
docs: improve descriptions
...
Co-authored-by: LenaLenaPan <120552185+LenaLenaPan@users.noreply.github.com>
2023-04-03 09:11:14 -03:00
Kjell Winblad
7c60a1e276
Merge pull request #10255 from emqx/Meggielqk-patch-1
...
Doc: Refined Chinese UI
2023-04-03 13:31:20 +02:00
Zaiming (Stone) Shi
677b76afce
refactor: replace hidden => true with importance => hidden
2023-04-03 13:13:43 +02:00
Zaiming (Stone) Shi
36000abf51
refactor: relocate i18n files for apps/emqx
2023-04-03 13:12:24 +02:00
firest
c8dca74b18
chore: bump hackney version
2023-04-03 18:41:49 +08:00
lafirest
9c7c49e9bf
Merge pull request #10201 from lafirest/fix/tdengine
...
fix(tdengine): remove the redundant table name in SQL template
2023-04-03 18:13:41 +08:00
zhongwencool
d63680cf25
Merge pull request #10307 from emqx/release-50
...
Sync release-50 back to master
2023-04-02 11:36:41 +08:00
Thales Macedo Garitezi
246a792965
Merge pull request #10273 from thalesmg/refactor-kprod-start-error-msg-rv50
...
fix: return friendly message when kafka producer and consumer fails to start (rv5.0)
2023-03-31 16:25:26 -03:00
Kjell Winblad
58898ea11d
Merge pull request #10294 from kjellwinblad/kjell/feat/collection_var_syntax_mongodb/EMQX-9246
...
feat: (MongoDB bridge) use ${var} syntax in MongoDB collection field
2023-03-31 17:18:27 +02:00
Thales Macedo Garitezi
5011486b18
fix(kafka_consumer): return better error messages when probing kafka consumer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-9422
2023-03-31 11:33:15 -03:00
Kjell Winblad
fe400c7f8d
Merge pull request #10272 from kjellwinblad/kjell/better_label
...
docs: make clickhouse config label for Server URL more concise
2023-03-31 16:17:26 +02:00
Zaiming (Stone) Shi
945c26d027
Merge pull request #10291 from zmstone/0330-fix-flaky-test-influxdb
...
test: fix flaky influxdb test
2023-03-31 15:36:12 +02:00
Zaiming (Stone) Shi
9bbca07943
Merge pull request #10298 from emqx/release-50
...
Merge release-50 back to master
2023-03-31 15:35:51 +02:00
Thales Macedo Garitezi
aabbbf5deb
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-31 09:07:00 -03:00
Kjell Winblad
6f6d12f930
docs: better Chinese labels
...
Co-authored-by: LenaLenaPan <120552185+LenaLenaPan@users.noreply.github.com>
2023-03-31 09:39:04 +02:00
Kjell Winblad
e808fef1e4
feat: (MongoDB bridge) use ${var} syntax for MongoDB collection
...
This commit makes it possible to use the ${var} syntax to refer to
variables in the payload of the message in the collection field.
This makes it possible to select which collection to insert into
dynamically.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9246
2023-03-30 17:49:56 +02:00
Thales Macedo Garitezi
632bffd451
fix: return friendly message when kafka producer fails to start (rv5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9392
The returned information does not allow to diagnose the issue (i.e.: a
connection issue due to the wrong host and port, the wrong password
failing authn). However, such information is printed to the logs.
This changes the returned error to the API so that the user is hinted
at looking at the logs for further investigation of the error.
2023-03-30 11:51:36 -03:00
Zaiming (Stone) Shi
81a104690d
test: fix flaky influxdb test
2023-03-30 16:19:22 +02:00
Kjell Winblad
8c4d211b54
fix: on_get_status clickhouse just reporting connecting
...
The on_get_status callback for clickhouse just returned `connecting`
without error information when the status check was unsuccessful. This is
fixed by letting the callback return error information similarly to how
the HTTP connector does it.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9374
https://emqx.atlassian.net/browse/EMQX-9278
2023-03-30 15:10:03 +02:00
Kjell Winblad
91a784134c
docs: update Chinese version of batch value separator label
2023-03-30 14:55:58 +02:00
Kjell Winblad
aefa0e5ffb
docs: make clickhouse config label for Server URL more concise
2023-03-30 14:55:22 +02:00
Meggielqk
dfc9141da6
docs: Refined Chinese UI
...
Also updated the English hints according to the refined Chinese according to Guowei's comments.
Co-authored-by: Kjell Winblad <kjellwinblad@gmail.com>
2023-03-30 14:50:24 +02:00
Thales Macedo Garitezi
b3c6ae99d2
Merge pull request #10269 from thalesmg/tdengine-required-password-rv50
...
fix(tdengine): make password a required field (rv5.0)
2023-03-30 09:25:38 -03:00
Thales Macedo Garitezi
07e1f2e440
docs(kafka_producer): update ZH translation of field (rv5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9411
2023-03-30 09:18:16 -03:00
Kjell Winblad
808bf1d232
Merge pull request #10270 from kjellwinblad/kjell/clickhouse/on_get_status
...
fix: on_get_status clickhouse just reporting connecting
2023-03-30 10:57:48 +02:00
firest
ee5b9d7e0a
fix(tdengine): remove the redundant table name in SQL template
2023-03-30 10:34:10 +08:00
Zaiming (Stone) Shi
94726eb7f1
docs: refine zh translations
2023-03-29 15:07:19 +02:00
Thales Macedo Garitezi
b2b9732edb
fix(tdengine): make password a required field (rv5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9395
```
2023-03-29T09:59:02.655495+00:00 [warning] msg: start_resource_failed, mfa: emqx_resource_manager:start_resource/2, line: 524, id: <<"bridge:tdengine:jimmoen-test-trigger">>, reason: {error,function_clause,[{emqx_ee_connector_tdengine,on_start,[<<"bridge:tdengine:jimmoen-test-trigger:1076">>,#{database => <<"db">>,enable => true,pool_size => 8,resource_opts => #{auto_restart_interval => 60000,batch_size => 1,batch_time => 0,health_check_interval => 15000,max_queue_bytes => 104857600,query_mode => sync,request_timeout => 15000,start_after_created => true,start_timeout => 5000,worker_pool_size => 16},server => "127.0.0.1",sql => <<"insert into mqtt.t_mqtt_msg(ts, msgid, mqtt_topic, qos, payload, arrived) values (${ts}, ${id}, ${topic}, ${qos}, ${payload}, ${timestamp})">>,username => <<"root">>}],[{file,"emqx_ee_connector_tdengine.erl"},{line,76}]},{emqx_resource,call_start,3,[{file,"emqx_resource.erl"},{line,359}]},{emqx_resource_manager,start_resource,2,[{file,"emqx_resource_manager.erl"},{line,513}]},{gen_statem,loop_state_callback,11,[{file,"gen_statem.erl"},{line,1205}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
```
2023-03-29 10:07:02 -03:00
Kjell Winblad
c693851740
fix: on_get_status clickhouse just reporting connecting
...
The on_get_status callback for clickhouse just returned `connecting`
without error information when the status check was unsuccessful. This is
fixed by letting the callback return error information similarly to how
the HTTP connector does it.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9374
https://emqx.atlassian.net/browse/EMQX-9278
2023-03-29 14:26:08 +02:00
Zaiming (Stone) Shi
80eb9d7542
Merge pull request #10252 from emqx/release-50
...
0327 merge release-50 to master
2023-03-29 12:33:17 +02:00
Zaiming (Stone) Shi
624c168dde
fix: hide nodelay Kafka client socket option
...
also fix a few descriptions
2023-03-29 11:42:17 +02:00
Kjell Winblad
8e0d315b7b
Merge pull request #10197 from kjellwinblad/0321-fix-inflight-window-hand-over-to-kjell
...
fix: add inflight window setting to the clickhouse bridge
2023-03-29 09:38:24 +02:00
Thales Macedo Garitezi
64faccf50b
test: fix flaky kafka consumer test
2023-03-28 14:50:55 -03:00
Thales Macedo Garitezi
1a7ca7235e
Merge pull request #10249 from thalesmg/fix-kafka-offset-doc-rv50
...
feat(kafka_consumer): tie `offset_reset_policy` and `begin_offset` together (rv5.0)
2023-03-28 11:37:46 -03:00
Thales Macedo Garitezi
1c8333030c
fix(kafka_producer): add back `is_buffer_supported` callback
...
Fixes https://emqx.atlassian.net/browse/EMQX-9366
This callback was accidentally removed while adding another feature,
which made the buffer workers to be used for this bridge while they
shouldn’t be.
2023-03-28 09:33:18 -03:00
Zaiming (Stone) Shi
d07987288a
chore: add some example annotations for config importance level
2023-03-28 14:29:24 +02:00
Thales Macedo Garitezi
69fc1123ee
refactor: change enum constructors and improve docs
2023-03-27 17:30:17 -03:00
Thales Macedo Garitezi
5cf09209cd
feat: tie `offset_reset_policy` and `begin_offset` together
...
To make the configuration more intuitive and avoid exposing more
parameters to the user, we should:
1) Remove reset_by_subscriber as an enum constructor for
`offset_reset_policy`, as that might make the consumer hang
indefinitely without manual action.
2) Set the `begin_offset` `brod_consumer` parameter to `earliest` or
`latest` depending on the value of `offset_reset_policy`, as that’s
probably the user’s intention.
2023-03-27 14:20:31 -03:00
Zaiming (Stone) Shi
da5e6f3d0a
test: test with only one Kafka partition for bad config recover test
2023-03-27 17:38:34 +02:00
Thales Macedo Garitezi
0d36b179c0
docs: fix kafka offset reset policy config description
2023-03-27 11:22:01 -03:00
Zaiming (Stone) Shi
7a6494cda6
Merge pull request #10242 from zmstone/0326-fix-logging-avoid-using-pid-as-log-data-field
...
fix: avoid uisng 'pid' as log data field
2023-03-27 16:13:50 +02:00
JianBo He
bfa5922209
Merge pull request #10140 from HJianBo/cassa
...
feat: support cassandra data bridge
2023-03-27 10:23:02 +08:00
JimMoen
798597c834
Merge pull request #10217 from JimMoen/conn-health-check-with-worker
...
fix: for connection used ecpool, let worker do health check fun
2023-03-27 09:51:52 +08:00
JianBo He
7085a2d6d7
chore: hidden the password field
...
ref: https://github.com/emqx/emqx/issues/10222
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-03-27 09:47:51 +08:00
Zaiming (Stone) Shi
19e136bb66
fix: avoid uisng 'pid' as log data field
...
Since it clashes with log metad data field
2023-03-26 10:33:18 +02:00
Zaiming (Stone) Shi
dd996ad1dc
chore: bump app vsns
2023-03-24 21:47:15 +01:00
Zaiming (Stone) Shi
b37f186142
Merge remote-tracking branch 'origin/master' into 0324-merge-release-50-back-to-master
2023-03-24 21:47:15 +01:00
Thales Macedo Garitezi
ff272a2071
Merge pull request #10206 from thalesmg/decouple-buffer-worker-query-call-mode-v50
...
feat(buffer_worker): decouple query mode from underlying connector call mode
2023-03-24 13:49:00 -03:00
Thales Macedo Garitezi
c92af4f763
docs: fix quote
2023-03-24 09:48:35 -03:00
Thales Macedo Garitezi
a239f0523a
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-24 09:23:06 -03:00
Thales Macedo Garitezi
ca820dfefc
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-24 09:21:08 -03:00
Thales Macedo Garitezi
5bdc71af6f
docs(kafka_consumer): make specific labels for key/value encoding modes
...
https://emqx.atlassian.net/browse/EMQX-9312
2023-03-24 09:08:27 -03:00
JimMoen
d0f0a93888
fix: ecpool worker exec dynamo health check fun
2023-03-24 11:15:52 +08:00
Thales Macedo Garitezi
f8d5d53908
feat(buffer_worker): decouple query mode from underlying connector call mode
...
Fixes https://emqx.atlassian.net/browse/EMQX-9129
Currently, if an user configures a bridge with query mode sync, then
all calls to the underlying driver/connector ("inner calls") will
always be synchronous, regardless of its support for async calls.
Since buffer workers always support async queries ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.
There are two situations when we want to force synchronous calls to
the underlying connector even if it supports async:
1) When using `simple_sync_query`, since we are bypassing the buffer
workers;
2) When retrying the inflight window, to avoid overwhelming the
driver.
2023-03-23 13:40:31 -03:00
Kinplemelon
28544e62df
docs: correct the chinese label for the `socket_nodelay` field
2023-03-23 16:27:09 +08:00
Kinplemelon
1cb975a811
Merge pull request #10148 from Kinplemelon/kinple/kafka-nodelay-label
...
docs: correct the chinese label for the `socket_nodelay` field
2023-03-23 16:24:24 +08:00
JianBo He
1a466fda0b
chore: use binary type InstanceId as PoolName
2023-03-23 16:06:51 +08:00
JianBo He
9b63bdc1e0
chore: apply review suggestions
...
- Rename sql to cql
- Add tests for `bridges_probe` API
2023-03-23 15:27:34 +08:00
JianBo He
ac41c7e653
chore: format codes
2023-03-23 11:53:26 +08:00
JianBo He
8cbbc9f271
Merge remote-tracking branch 'upstream/master' into cassa
2023-03-23 11:53:17 +08:00
Thales Macedo Garitezi
ddffba0355
Merge pull request #10154 from thalesmg/fix-buffer-worker-default-req-timeout
...
fix(buffer_worker): calculate default `resume_interval` based on `request_timeout` and `health_check_interval`
2023-03-22 20:21:04 -03:00