Commit Graph

66 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi
0ce1ca89b7 refactor: use string type for server and servers 2022-12-30 14:20:23 +01:00
Zaiming (Stone) Shi
a5e3f6f05b fix(kafka): Memory OLP is only applicable in linux systems 2022-12-28 15:31:53 +01:00
Thales Macedo Garitezi
7242ce426b fix(kafka_producer): cleanup client after failing to start producers
https://emqx.atlassian.net/browse/EMQX-8547

If a Kafka Producer bridge is given bad configuration (e.g.: bad authn
credentials), the Wolff client process is started successfully, as it
does not attempt to connect, but when the producer process is
attempted to be started, it fails (only then the client tries to
connect to Kafka).  At this point, an error was thrown, but the
supervised client process remained running.

If the configuration was later fixed and the bridge updated, which
prompted its removal and recreation, the Wolff client would report to
be "already started", so it would never pick up the new (fixed)
configuration, and the producers would perpetually fail to start until
the node would be restarted.

We simply ensure the client is stopped before throwing the error,
unrolling the start-up procedure.
2022-12-19 17:48:40 -03:00
Zaiming (Stone) Shi
9e3da5b661 chore: bump app versions 2022-12-14 20:07:41 +01:00
Zaiming (Stone) Shi
42c58e2a91 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 15:29:13 +01:00
Thales Macedo Garitezi
2932aceaf9 refactor: rename and move emqx_ee_bridge_gcp_pubsub_resource to emqx_ee_connector_gcp_pubsub 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
b9bc82f87a feat(gcp_pubsub): add local_topic config
Given the implicit convention that an egress bridge containing the
`local_topic` config will forward messages without the need for a rule
action, this was added to avoid needing a rule action.
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
1cd91a24e9 feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0) 2022-12-12 17:18:19 -03:00
Ilya Averyanov
6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi
9ea22d062d refactor: make all bridges optional (required = false) 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi
fa90638de2 chore: add description to apps 2022-11-30 17:03:41 +01:00
Zaiming (Stone) Shi
7305923d12 fix: bridge name parser should not leak atom 2022-11-16 18:25:42 +01:00
Zaiming (Stone) Shi
c940b901f5 chore: fix app versions 2022-11-16 16:26:43 +01:00
Thales Macedo Garitezi
ee4c723fcb refactor: simplify wolff telemetry handler id 2022-10-17 16:26:04 -03:00
Thales Macedo Garitezi
1b2b629cdd feat: emit telemetry events for all resource worker metrics 2022-10-13 15:32:04 -03:00
Thales Macedo Garitezi
1ad3b5df17 fix: uninstall telemetry handler on resource stop, use unique id 2022-10-13 10:29:53 -03:00
Thales Macedo Garitezi
98500313eb fix(kafka): some fixes for kafka producer
- MQTT topic should be a binary
- use correct gauge functions from `wolff_metrics`.
- don't double increment success counter for kafka action
- adds a few more metrics assertions
2022-10-10 17:11:29 -03:00
Kjell Winblad
57270fb8fc feat: add support for counters and gauges to the Kafka Bridge
This commit adds support for counters and gauges to the Kafka Brige.
The Kafka bridge uses [Wolff](https://github.com/kafka4beam/wolff) for
the  Kafka connection. Wolff does its own batching and does not use the
batching functionality in `emqx_resource_worker` that is used by other
bridge types. Therefore, the counter events have to be generated by
Wolff. We have added
[telemetry](https://github.com/beam-telemetry/telemetry) events to Wolff
that we hook into to change counters and gauges for the Kafka bridge. The
counter called `matched` does not depend on specific functionality of
any bridge type so the updates of this counter is moved higher up in the
call chain then previously so that it also gets updated for Kafka
bridges.
2022-10-10 14:40:57 -03:00
Shawn
dfe14be8b2 chore: bump app vsns 2022-09-26 18:23:28 +08:00
Zaiming (Stone) Shi
f6ac4c3a76
Merge pull request #8798 from zmstone/0815-feat-add-kafka-connector
feat: Add Kafka connector
2022-09-24 22:57:50 +02:00
JimMoen
6788881642 fix: hide influxdb udp config 2022-09-20 14:46:59 +08:00
Shawn
f41adb0997 refactor: change some default values of resource_opts 2022-09-14 15:18:07 +08:00
Zaiming (Stone) Shi
e45c99bf79 fix: kafka bridge schema 2022-09-13 20:02:48 +02:00
Zaiming (Stone) Shi
0c1595be02 feat: Add Kafka connector 2022-09-13 19:46:56 +02:00
Xinyu Liu
23b6ff399d
Merge pull request #8844 from thalesmg/mongodb-bridge
feat: add mongodb bridge (e5.0)
2022-09-03 12:27:01 +08:00
Shawn
b45f3de8db refactor(resource): rename metrics batched,queued -> batching,queuing 2022-09-02 12:41:14 +08:00
Thales Macedo Garitezi
3d4afd65df feat: add mongodb bridge (e5.0) 2022-09-01 14:47:14 -03:00
Shawn
ba1f5eecd3 fix: update the swagger for new resource metrics 2022-08-31 11:14:36 +08:00
Shawn
a896aa8b27 fix: incorrect replayq dir for the emqx_resource 2022-08-25 16:06:18 +08:00
Shawn
9327c0f51b fix(mysql_bridge): export the query_mode option to the APIs 2022-08-25 11:38:50 +08:00
JimMoen
a481e8e2ee fix(influxdb): refine influxdb bridge example 2022-08-24 14:44:53 +08:00
JimMoen
eb21a37145
Merge pull request #8785 from JimMoen/feat-influxdb-config
Remove influxdb connector config layer. See #8773
2022-08-24 10:56:26 +08:00
Shawn
4ac6f12252 refactor: configs and APIs for mysql bridge 2022-08-24 10:08:03 +08:00
JimMoen
ca6533395c feat: influxdb bridge structure fits new style 2022-08-24 09:59:22 +08:00
Shawn
aea8c77b49 refactor: new config structure for mqtt bridge 2022-08-22 18:24:59 +08:00
firest
2471580c47 fix(bridge): make spellcheck happy 2022-08-22 15:58:11 +08:00
firest
dce47aac17 fix(bridge): don't export internal fields 2022-08-19 15:31:37 +08:00
firest
8d8afd1688 feat(bridge): add on_batch_query on emqx_connector_mysql 2022-08-19 13:16:22 +08:00
Shawn
19d85d485b refactor(resource): add resource_opts level into config structure 2022-08-15 21:40:10 +08:00
lafirest
404e79b7d7
Merge pull request #8687 from lafirest/fix/bridge_mysql
fix(bridge): replace prepare_statement by sql_template
2022-08-12 10:12:34 +08:00
JimMoen
0c1285cdab fix: bridge creation opts refs 2022-08-12 10:07:54 +08:00
JimMoen
3a76a50382 fix: syntax error and compile error 2022-08-11 20:58:43 +08:00
Shawn
2872f0b668 fix(bridges): support create resources with options 2022-08-11 19:11:44 +08:00
JimMoen
0f6c371760 feat(influxdb): influxdb connector add on_batch_query/3 callback 2022-08-11 18:12:41 +08:00
firest
dc80499399 fix(bridge): add sql_template field format 2022-08-11 15:58:48 +08:00
firest
7581082fcb fix(bridge): replace prepare_statement by sql_template 2022-08-11 13:22:16 +08:00
JimMoen
223b84017e fix(influxdb): api schema write_syntax using raw type string() 2022-08-11 11:36:08 +08:00
JimMoen
0090a3ee93 fix(influxdb): fix illegal wirte_syntax example 2022-08-11 11:26:53 +08:00
JimMoen
9d17bf31f8 fix: InfluxDB api v1 not support uint type 2022-08-09 14:28:54 +08:00
JimMoen
d51ca2672a fix: spellcheck and update checker repo version 2022-08-09 11:50:05 +08:00