yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,431 Commits 393 Branches 684 Tags 74 MiB
Commit Graph

132 Commits

Author SHA1 Message Date
Andrew Mayorov 0d86ef8d3a
fix(ft-s3): use AWS4 signed urls for S3 export URIs 
This will ensure that the S3 export URIs are valid in all AWS regions.
 2023-04-07 22:15:01 +03:00
Ilya Averyanov c7865e5eae fix(ft-s3): fix dependency dependencies 2023-04-07 17:36:45 +03:00
Ilya Averyanov 71965d90e4 fix(ft-s3): synchronize erlcloud version across apps 2023-04-07 17:26:53 +03:00
Thales Macedo Garitezi 5d5b7ea215
Merge pull request #10306 from thalesmg/enable-async-buffer-workers-all-bridges-v50 
feat(bridges): enable async query mode for all bridges with buffer workers
 2023-04-04 17:10:46 -03:00
lafirest ed31b735a8
Merge pull request #10220 from lafirest/bump/hackney 
chore: bump hackney version
 2023-04-04 10:35:09 +08:00
Thales Macedo Garitezi f3ffc02bff feat(bridges): enable async query mode for all bridges with buffer workers 
Fixes https://emqx.atlassian.net/browse/EMQX-9130

Since buffer workers always support async calls ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.

For bridges that currently only allow sync query modes, we should
allow them to be configured with async.  That means basically all
bridge types except Kafka Producer.
 2023-04-03 14:49:51 -03:00
Zaiming (Stone) Shi 36000abf51 refactor: relocate i18n files for apps/emqx 2023-04-03 13:12:24 +02:00
firest c8dca74b18 chore: bump hackney version 2023-04-03 18:41:49 +08:00
zhongwencool d63680cf25
Merge pull request #10307 from emqx/release-50 
Sync release-50 back to master
 2023-04-02 11:36:41 +08:00
Kjell Winblad 58898ea11d
Merge pull request #10294 from kjellwinblad/kjell/feat/collection_var_syntax_mongodb/EMQX-9246 
feat: (MongoDB bridge) use ${var} syntax in MongoDB collection field
 2023-03-31 17:18:27 +02:00
Kjell Winblad fe400c7f8d
Merge pull request #10272 from kjellwinblad/kjell/better_label 
docs: make clickhouse config label for Server URL more concise
 2023-03-31 16:17:26 +02:00
Zaiming (Stone) Shi 945c26d027
Merge pull request #10291 from zmstone/0330-fix-flaky-test-influxdb 
test: fix flaky influxdb test
 2023-03-31 15:36:12 +02:00
Zaiming (Stone) Shi 9bbca07943
Merge pull request #10298 from emqx/release-50 
Merge release-50 back to master
 2023-03-31 15:35:51 +02:00
Kjell Winblad e808fef1e4 feat: (MongoDB bridge) use ${var} syntax for MongoDB collection 
This commit makes it possible to use the ${var} syntax to refer to
variables in the payload of the message in the collection field.
This makes it possible to select which collection to insert into
dynamically.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9246
 2023-03-30 17:49:56 +02:00
Zaiming (Stone) Shi 81a104690d test: fix flaky influxdb test 2023-03-30 16:19:22 +02:00
Kjell Winblad 8c4d211b54 fix: on_get_status clickhouse just reporting connecting 
The on_get_status callback for clickhouse just returned `connecting`
without error information when the status check was unsuccessful. This is
fixed by letting the callback return error information similarly to how
the HTTP connector does it.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9374
https://emqx.atlassian.net/browse/EMQX-9278
 2023-03-30 15:10:03 +02:00
Kjell Winblad aefa0e5ffb docs: make clickhouse config label for Server URL more concise 2023-03-30 14:55:22 +02:00
Thales Macedo Garitezi b2b9732edb fix(tdengine): make password a required field (rv5.0) 
Fixes https://emqx.atlassian.net/browse/EMQX-9395

```
2023-03-29T09:59:02.655495+00:00 [warning] msg: start_resource_failed, mfa: emqx_resource_manager:start_resource/2, line: 524, id: <<"bridge:tdengine:jimmoen-test-trigger">>, reason: {error,function_clause,[{emqx_ee_connector_tdengine,on_start,[<<"bridge:tdengine:jimmoen-test-trigger:1076">>,#{database => <<"db">>,enable => true,pool_size => 8,resource_opts => #{auto_restart_interval => 60000,batch_size => 1,batch_time => 0,health_check_interval => 15000,max_queue_bytes => 104857600,query_mode => sync,request_timeout => 15000,start_after_created => true,start_timeout => 5000,worker_pool_size => 16},server => "127.0.0.1",sql => <<"insert into mqtt.t_mqtt_msg(ts, msgid, mqtt_topic, qos, payload, arrived) values (${ts}, ${id}, ${topic}, ${qos}, ${payload}, ${timestamp})">>,username => <<"root">>}],[{file,"emqx_ee_connector_tdengine.erl"},{line,76}]},{emqx_resource,call_start,3,[{file,"emqx_resource.erl"},{line,359}]},{emqx_resource_manager,start_resource,2,[{file,"emqx_resource_manager.erl"},{line,513}]},{gen_statem,loop_state_callback,11,[{file,"gen_statem.erl"},{line,1205}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
```
 2023-03-29 10:07:02 -03:00
Kjell Winblad c693851740 fix: on_get_status clickhouse just reporting connecting 
The on_get_status callback for clickhouse just returned `connecting`
without error information when the status check was unsuccessful. This is
fixed by letting the callback return error information similarly to how
the HTTP connector does it.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9374
https://emqx.atlassian.net/browse/EMQX-9278
 2023-03-29 14:26:08 +02:00
JianBo He bfa5922209
Merge pull request #10140 from HJianBo/cassa 
feat: support cassandra data bridge
 2023-03-27 10:23:02 +08:00
JimMoen 798597c834
Merge pull request #10217 from JimMoen/conn-health-check-with-worker 
fix: for connection used ecpool, let worker do health check fun
 2023-03-27 09:51:52 +08:00
Zaiming (Stone) Shi dd996ad1dc chore: bump app vsns 2023-03-24 21:47:15 +01:00
JimMoen d0f0a93888
fix: ecpool worker exec dynamo health check fun 2023-03-24 11:15:52 +08:00
Thales Macedo Garitezi f8d5d53908 feat(buffer_worker): decouple query mode from underlying connector call mode 
Fixes https://emqx.atlassian.net/browse/EMQX-9129

Currently, if an user configures a bridge with query mode sync, then
all calls to the underlying driver/connector ("inner calls") will
always be synchronous, regardless of its support for async calls.

Since buffer workers always support async queries ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.

There are two situations when we want to force synchronous calls to
the underlying connector even if it supports async:

1) When using `simple_sync_query`, since we are bypassing the buffer
workers;
2) When retrying the inflight window, to avoid overwhelming the
driver.
 2023-03-23 13:40:31 -03:00
JianBo He 1a466fda0b chore: use binary type InstanceId as PoolName 2023-03-23 16:06:51 +08:00
JianBo He 9b63bdc1e0 chore: apply review suggestions 
- Rename sql to cql
- Add tests for `bridges_probe` API
 2023-03-23 15:27:34 +08:00
JianBo He 8cbbc9f271 Merge remote-tracking branch 'upstream/master' into cassa 2023-03-23 11:53:17 +08:00
firest 4ad3579966 test(bridges): add test suite for RocketMQ 2023-03-22 10:36:58 +08:00
firest 9853d00cad feat(bridges): integrate RocketMQ into data bridges 2023-03-22 10:36:52 +08:00
JianBo He 539ec2f774 chore(bridge): cover username/password auth for cassandra bridges 2023-03-21 13:55:53 +08:00
JianBo He c2c9abd568 chore: add running ct 2023-03-20 09:47:58 +08:00
JianBo He d8e6e2a1be test: refine CASSANDRA_HOST name 2023-03-17 18:52:52 +08:00
JianBo He 678cc937c0 test(bridge): cover ssl testing for cassandra bridge 2023-03-17 18:25:05 +08:00
JianBo He a20c39c83a test: correct CASSANDRA_HOST 2023-03-17 13:55:26 +08:00
JianBo He 75c1592e7c chore: make static_checks happy 2023-03-17 11:34:52 +08:00
JianBo He c0a216a740 feat(bridge): support cassandra bridge 2023-03-17 11:34:48 +08:00
Andrew Mayorov a86d06f043
chore: bump app versions following last merge-back 2023-03-10 16:44:15 +03:00
Zaiming (Stone) Shi fe27604010 Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master 2023-03-08 16:46:45 +01:00
Serge Tupchii 97e71c54d4 fix: use default template if timestamp is empty (undefined) in InfluxDB bridge 
Closes EMQX-8926
 2023-03-08 11:58:23 +01:00
firest b8b1a78025 feat(bridges): integrate the DynamoDB into data bridges 2023-03-08 11:13:38 +08:00
Thales Macedo Garitezi 9b087a21f5 fix(gcp_pubsub): remove conflicting `request_timeout` option 
Since the buffer worker schema already contains that configuration,
having it two places can lead to quite confusing behavior.
 2023-03-06 10:12:38 -03:00
Ivan Dyachkov c01f62a1c1 chore: bump apps versions 2023-03-03 13:58:32 +01:00
Kjell Winblad 67acdf0888 feat: add clickhouse database bridge 
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
 2023-03-02 12:22:11 +01:00
Zaiming (Stone) Shi 083330ad80 Merge remote-tracking branch 'origin/master' into 0301-merge-release-50-to-master 2023-03-01 08:53:03 +01:00
Zaiming (Stone) Shi 24f476e35f test: add README to influxdb test script 2023-02-28 19:38:43 +01:00
Erik Timan 44c0aae16b fix(emqx_ee_connector): improve log message on influxdb start failures 2023-02-24 11:31:57 +01:00
Erik Timan da42c91fb2 test(emqx_ee_bridge): check influxdb:is_alive/2 return 2023-02-24 09:03:34 +01:00
Erik Timan 8314c07147 chore: update influxdb-client-erl dep 2023-02-24 09:03:34 +01:00
Erik Timan 603532dacd fix(emqx_ee_connector): log reason for failure when starting influxdb connector 2023-02-24 09:03:34 +01:00
Erik Timan 291755f81f
Merge pull request #9895 from olcai/add-test-for-influxdb-ssl-opts 
test(emqx_ee_connector): add basic tests for influxdb incl. SSL opts
 2023-02-20 10:05:47 +01:00