Commit Graph

609 Commits

Author SHA1 Message Date
firest 6497bcae46 refactor(influxdb): move influxdb bridge into its own app 2023-05-12 11:48:49 +08:00
JimMoen fbd516d740
Merge pull request #10622 from JimMoen/refactor-dynamo-bridge
refactor(dynamo): move dynamo bridge into its own app
2023-05-11 11:00:50 +08:00
JimMoen e32ab10d01
chore: rename dynamo template files 2023-05-10 18:49:21 +08:00
JimMoen 8914e006c2
refactor(dynamo): move dynamo bridge into its own app 2023-05-10 18:49:21 +08:00
firest 066ed5c6ec refactor(pgsql): move pgsql && matrix && timescale bridges into their own app 2023-05-10 16:51:33 +08:00
firest 0f979b3d24 refactor(tdengine): move tdengine bridge into its own app 2023-05-10 14:12:57 +08:00
Kjell Winblad 70cf1533db feat: add RabbitMQ bridge 2023-05-09 14:32:26 +02:00
firest a119100a0e refactor(rocketmq): move rocketmq bridge into its own app 2023-05-09 15:57:56 +08:00
Thales Macedo Garitezi 270059f0c2
Merge pull request #10642 from thalesmg/fix-flaky-test-rule-engine-stop-v50
test: fix inter-suite flakiness
2023-05-08 15:37:26 -03:00
Stefan Strigler 1ade7ce9a3
Merge pull request #10560 from sstrigler/EMQX-8390-5-0-x-io-tdb-bridge-port-conf-and-api-to-5-0
IoTDB Bridge
2023-05-08 20:27:01 +02:00
Thales Macedo Garitezi 2667f06820 test: fix inter-suite flakiness 2023-05-08 13:33:46 -03:00
Stefan Strigler d231e708cb feat: add IotDB bridge 2023-05-08 16:19:01 +02:00
Zhongwen Deng 4f396a36a9 Merge remote-tracking branch 'upstream/master' into release-50 2023-05-08 14:58:03 +08:00
JimMoen d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app 2023-05-06 10:22:05 +08:00
firest 518b6e1211 fix(tdengine): redact the password for the TDEngine example API 2023-05-05 13:51:38 +08:00
Zhongwen Deng a4bb865e6a feat: add validations for emqx_ee_conf_schema 2023-05-04 16:53:03 +08:00
firest 277deee616 fix: add new sensitive keywords to redact checklist 2023-05-04 16:32:32 +08:00
Thales Macedo Garitezi 8aa7c014e7 perf(buffer_worker): avoid calling `ets:info/2`
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637

During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.

Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full.  Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60).

By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
Zaiming (Stone) Shi a4bc15cfa1 Merge remote-tracking branch 'origin/master' into 0502-merge-release-50-back-to-master 2023-05-02 18:59:31 +02:00
Zaiming (Stone) Shi d5f5f35787 Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master 2023-05-02 18:08:04 +02:00
Andrew Mayorov 90cf1ade74
chore: bump application versions
* emqx_connector 0.1.22
* emqx_bridge_gcp_pubsub 0.1.1
* emqx_bridge_kafka 0.1.2
* emqx_bridge_pulsar 0.1.1
* emqx_ee_connector 0.1.12
2023-05-02 17:29:22 +03:00
Andrew Mayorov 4575167607
feat(resource): drop `manager_id()` type 2023-05-02 17:29:20 +03:00
Andrew Mayorov aaef95b1da
feat(resman): stop adding uniqueness to manager ids
Before this change, a separate `manager_id` / `instance_id` was used
as resource manager id, which made connector interface somewhat
inconsistent: part of function calls to connector implementation
used instance id as first argument while the rest used resource id
itself.
2023-05-02 17:28:26 +03:00
Zaiming (Stone) Shi 57cc854a4a test(bridge): fix bridge map type filed converters
now the converters on map type fields only work at the wrapping map
level but not the values
2023-04-30 10:45:11 +02:00
Zaiming (Stone) Shi b58d3e8f94
Merge pull request #10529 from zmstone/0426-ensure-buffer-worker-monitors-cassandra-conn-pid
0426 ensure buffer worker monitors cassandra conn pid
2023-04-27 22:54:30 +02:00
Paulo Zulato dd90b2f498 feat(oracle): Oracle Database integration 2023-04-27 13:07:20 -03:00
Thales Macedo Garitezi 567413389c
Merge pull request #10519 from thalesmg/fix-flaky-res-test-v50
test(resource): fix flaky test
2023-04-27 09:33:40 -03:00
zhongwencool 1f0148a819
Merge pull request #10524 from lafirest/fix/rocketmq_sensitive
fix(rocketmq): keep sensitive data safe in rocketmq logs and state
2023-04-27 10:16:05 +08:00
Zaiming (Stone) Shi a8b000f062 refactor(sqlserver): support only sync mode at connector level 2023-04-26 20:44:56 +02:00
Ivan Dyachkov b76025f541 chore: merge upstream/release-50 2023-04-26 18:01:33 +02:00
Zaiming (Stone) Shi c83d630c97 fix(cassandra): ensure async calls return connection pid
so the buffer worker can monitor it and perform retries
if the connection restarted
2023-04-26 14:33:37 +02:00
firest 9eccfa5cdf fix(dynamo): fix test case errors 2023-04-26 17:03:01 +08:00
firest 6706fd90e1 fix(rocketmq): keep sensitive data safe in rocketmq logs and state 2023-04-26 16:10:35 +08:00
firest e467e082f0 fix(dynamo): remove all async callbacks of the Dynamo connector 2023-04-26 14:47:22 +08:00
Zhongwen Deng bc1bdae55d chore: reslove confilt for sync release-50 to master 2023-04-26 11:50:44 +08:00
zhongwencool 9d893b49eb
Merge branch 'master' into sync-release-50-to-master 2023-04-26 10:54:46 +08:00
Thales Macedo Garitezi 79cf5cad19
Merge pull request #10378 from thalesmg/pulsar-producer-e50
feat: implement Pulsar Producer bridge (e5.0)
2023-04-25 18:01:43 -03:00
Thales Macedo Garitezi 687509886e test: rm unused var warning 2023-04-25 17:37:20 -03:00
lafirest dc452deaf0
Merge pull request #10506 from lafirest/fix/dynamo_default_port
fix(dynamo): use correct default port for different schemas
2023-04-25 18:26:44 +08:00
firest 3bb50a5751 fix(rocketmq): fix that the update of ACL info not working 2023-04-25 16:55:03 +08:00
firest 33c27ac2ac fix(dynamo): use correct default port for different schemas 2023-04-25 10:50:44 +08:00
Thales Macedo Garitezi 99f3965f4e feat(schema_registry): use rocksdb as table type for protobuf cache 2023-04-24 14:24:42 -03:00
Thales Macedo Garitezi 377b143325 refactor: split `parse_server` into smaller functions, improve return type to use map 2023-04-24 14:17:29 -03:00
Thales Macedo Garitezi ad4be08bb2 feat: implement Pulsar Producer bridge (e5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-8398
2023-04-24 10:28:26 -03:00
firest 7ce04358c4 fix(Dynamo): fix DynamoDB bridge status check error 2023-04-24 17:28:33 +08:00
JianBo He 2b0106e047
Merge pull request #10485 from JimMoen/fix-mssql-bridge-default
fix: use default health check timeout for sqlserver
2023-04-23 18:20:10 +08:00
JimMoen d505b65ba8
fix: use default health check timeout for sqlserver 2023-04-23 15:47:19 +08:00
JimMoen 4e7472090b
fix: refine default sql and driver name for mssql bridge 2023-04-23 15:47:18 +08:00
firest 5ad5d7ee8d fix(opents): adjust code structure 2023-04-23 14:28:14 +08:00
lafirest 6ef032026f
Merge pull request #10471 from lafirest/fix/dynamo_bridge
fix(dynamo): separate the implementation of connector and client of Dynamo bridge
2023-04-23 14:25:23 +08:00