Commit Graph

282 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 7abba17b25
Merge pull request #9765 from zmstone/0115-add-password-converter
fix(schema): add password converter to ensure its binary() type
2023-01-18 15:09:05 +01:00
Zaiming (Stone) Shi 2d01e604a5
Merge pull request #9799 from zmstone/0118-fix-key_dispatch-kafka-produce-strategy
0118 fix key dispatch kafka produce strategy
2023-01-18 13:52:49 +01:00
Zaiming (Stone) Shi 8f275a66d0 test: add coverage for key_dispatch partition strategy 2023-01-18 11:47:37 +01:00
Zaiming (Stone) Shi d4f3b4c8c2 Merge remote-tracking branch 'origin/master' into fix-buffer-clear-replayq-on-delete-v50 2023-01-18 11:39:47 +01:00
Zaiming (Stone) Shi 3872c4451f fix(kafka): olp fix was accidentally deleted in 0fd8880d0a 2023-01-18 10:52:29 +01:00
Thales Macedo Garitezi 087b667263 fix(buffer_worker): allow signalling unrecoverable errors 2023-01-17 19:50:30 -03:00
Thales Macedo Garitezi fa01deb3eb chore: retry as much as possible, don't reply to caller too soon 2023-01-17 16:49:15 -03:00
Thales Macedo Garitezi 006b4bda97 feat(buffer_worker): monitor async workers and cancel their inflight requests upon death 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 32a9e60313 feat(buffer_worker): also use the inflight table for sync requests
Related: https://emqx.atlassian.net/browse/EMQX-8692

This should also correctly account for `retried.*` metrics for sync
requests.

Also fixes cases where race conditions for retrying async requests
could potentially lead to inconsistent metrics.

Fixes more cases where a stale reference to `replayq` was being held
accidentally after a `pop`.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 4c25be8a2c chore(influxdb): remove deprecated value from example 2023-01-17 16:48:48 -03:00
Stefan Strigler f37b3e4bc4 test: test against `bridges_probe` API 2023-01-17 15:29:19 +01:00
Stefan Strigler 5fc31dcd9e fix: use atom keys in example to make sure they exist 2023-01-17 15:29:19 +01:00
Zaiming (Stone) Shi 4a7e74f5d6 fix(schema): add password converter to ensure its binary() type 2023-01-17 11:12:17 +01:00
Shawn 1c7fe91a91 fix: typos in emqx_ee_bridge_mongodb.conf 2023-01-17 16:33:54 +08:00
JimMoen 40948f3c0b
fix(influxdb): authentication is not required for influxdb v1 api 2023-01-17 09:29:22 +08:00
Zaiming (Stone) Shi a7fc5e8fe1
Merge pull request #9761 from zmstone/0114-fix-kafka-value-template-and-docs
feat: introduce 'this' concept for placeholder, and use it in Kafka bridge
2023-01-16 13:37:29 +01:00
Zaiming (Stone) Shi 47414e0d53 docs: improve kafka key and value field description 2023-01-16 11:32:09 +01:00
Zaiming (Stone) Shi 376dab7cd8
Merge pull request #9743 from zmstone/0112-docs-fix-some-descriptions
0112 docs fix some descriptions
2023-01-16 09:56:56 +01:00
Zaiming (Stone) Shi 91c5a89985 test: wait for redis connected state
the case is sometimes flaky because the health check sometimes
return connecting
2023-01-14 18:33:55 +01:00
Stefan Strigler e08c1d2229 Merge remote-tracking branch 'olcai/refactor-bridges-api' into dev/api-refactor 2023-01-13 15:49:52 +01:00
Erik Timan 7a17fb7308 test(emqx_ee_bridge): fix bridge enable/disable in kafka producer suite 2023-01-13 14:40:54 +01:00
Thales Macedo Garitezi f25bd288ad
Merge pull request #9742 from thalesmg/expose-resource-opts-mongo-v50
feat(mongo): expose buffer worker opts to the bridge frontend (5.0)
2023-01-13 10:23:49 -03:00
Zhongwen Deng f15b29b1ef chore: upgrade app version 2023-01-13 18:22:29 +08:00
Zaiming (Stone) Shi 522e091dd2 docs: chane cache to buffer for Kafka bridge description 2023-01-13 07:46:23 +01:00
Erik Timan 860e21d40f feat(emqx_bridge): move metrics to own endpoint, rename reset_metrics
In order for the /bridges APIs to be consistent with other APIs, we move
out metrics from GET /bridges/{id} to its own endpoint,
/bridges/{id}/metrics. We also rename /bridges/reset_metrics to
/bridges/metrics/reset.
2023-01-12 19:39:41 +01:00
Thales Macedo Garitezi ed2be4d086 feat(mongo): expose buffer worker opts to the bridge frontend (5.0) 2023-01-12 15:23:41 -03:00
Shawn 1711823487 refactor: remove default value of timestamp field of influxdb 2023-01-12 22:25:44 +08:00
Xinyu Liu bae811e8b4
Merge pull request #9725 from terry-xiaoyu/remove_the_auto_reconnect_field
refactor: remove the auto_reconnect field
2023-01-12 11:11:00 +08:00
Thales Macedo Garitezi 2d5208fc29
Merge pull request #9485 from thalesmg/refactor-doc-gen-50-mkIII
feat(docs): add tags to schemas
2023-01-11 16:36:38 -03:00
Thales Macedo Garitezi 9d99d180f9
Merge pull request #9709 from thalesmg/mongodb-bridge-payload-template-v50
feat(mongodb): add `payload_template` field for bridge (e5.0)
2023-01-11 11:14:46 -03:00
Shawn 3e9c4f444f refactor: remove the auto_reconnect field 2023-01-11 21:47:06 +08:00
Thales Macedo Garitezi a9d0143d5f chore(license): change `key` type to `binary()`
So that the `raw_default` field in the schema JSON renders properly.
2023-01-11 09:10:04 -03:00
Thales Macedo Garitezi 48e1ba4832 feat(docs): add tags to schemas
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
2023-01-11 09:10:03 -03:00
Thales Macedo Garitezi f3cc722172
docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-10 09:06:12 -03:00
Zaiming (Stone) Shi 67f2159a27
Merge pull request #9653 from zmstone/0101-authz-schema-union-member-selection
0101 authz schema union member selection
2023-01-09 22:17:51 +01:00
Thales Macedo Garitezi b9f258b737 feat(mongodb): add `payload_template` field for bridge (e5.0)
https://emqx.atlassian.net/browse/EMQX-8705

Adds a `payload_template` fields that allows users to customize the
payload to publish to MongoDB.
2023-01-09 13:38:11 -03:00
lafirest 20f49b903f
Merge pull request #9702 from lafirest/feat/timescale_matrix
feat(bridges): add timescale && matrix bridges
2023-01-10 00:06:17 +08:00
firest 062f14bd65 test(bridges): add timescale && matrix test cases 2023-01-09 23:04:36 +08:00
firest e08d6dbc9b feat(bridges): add timescale && matrix bridges 2023-01-09 23:04:29 +08:00
Zaiming (Stone) Shi c3635f537a ci: wait for redis in emqx_ee_bridge_redis_SUITE 2023-01-09 14:39:56 +01:00
Zaiming (Stone) Shi 0697c692ed refactor: mongo_type and redis_type are not mandatory
the connector schemas are shared between authn, auth and bridges,
the difference is that authn and authz configs are unions like:

[
    {mongo_type = rs,
     ...
    }
    {another backend config}
]

however the brdige types are maps like, for example:

mongodb_rs.$name {
    mongo_type = rs
    ...
}

in which case, the mongo_type is not required.

in order to keep the schema static as much as possible,
this field is chanegd to 'required => false' with a default value.

However, for authn and authz, the union selector will still raise
exception if the there is no type provided.
2023-01-09 14:26:16 +01:00
Zaiming (Stone) Shi ffb09f0c4d test: rename a test option name to avoid clashing with prod config name 2023-01-09 14:26:16 +01:00
Zaiming (Stone) Shi e52f9d5920 refactor: use union member type selector for authz sources 2023-01-09 14:26:16 +01:00
Zaiming (Stone) Shi e498010f23 chore: update default license (100 conns) 2023-01-09 09:11:09 +01:00
Zaiming (Stone) Shi 3859878985
Merge pull request #9684 from zmstone/0105-do-not-use-testcase-skip
test: do not use tc_user_skip for test cases
2023-01-09 09:02:06 +01:00
firest ea405fe55d test(bridges): add test case for the PostgreSQL backend 2023-01-06 11:56:28 +08:00
firest f7b50c56da feat(bridges): integrate PostgreSQL into bridges
support both simple and batch query
2023-01-06 11:47:22 +08:00
Zaiming (Stone) Shi 5f12cdff6c ci: should not skip tests in github actions 2023-01-05 22:47:08 +01:00
Zaiming (Stone) Shi fc12a8c4c8 test: do not use tc_user_skip for test cases 2023-01-05 22:47:06 +01:00
Thales Macedo Garitezi af31ed4264 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-05 10:16:00 -03:00