e08c1d2229
Merge remote-tracking branch 'olcai/refactor-bridges-api' into dev/api-refactor
2023-01-13 15:49:52 +01:00
7a17fb7308
test(emqx_ee_bridge): fix bridge enable/disable in kafka producer suite
2023-01-13 14:40:54 +01:00
f25bd288ad
Merge pull request #9742 from thalesmg/expose-resource-opts-mongo-v50
...
feat(mongo): expose buffer worker opts to the bridge frontend (5.0)
2023-01-13 10:23:49 -03:00
f15b29b1ef
chore: upgrade app version
2023-01-13 18:22:29 +08:00
860e21d40f
feat(emqx_bridge): move metrics to own endpoint, rename reset_metrics
...
In order for the /bridges APIs to be consistent with other APIs, we move
out metrics from GET /bridges/{id} to its own endpoint,
/bridges/{id}/metrics. We also rename /bridges/reset_metrics to
/bridges/metrics/reset.
2023-01-12 19:39:41 +01:00
ed2be4d086
feat(mongo): expose buffer worker opts to the bridge frontend (5.0)
2023-01-12 15:23:41 -03:00
1711823487
refactor: remove default value of timestamp field of influxdb
2023-01-12 22:25:44 +08:00
bae811e8b4
Merge pull request #9725 from terry-xiaoyu/remove_the_auto_reconnect_field
...
refactor: remove the auto_reconnect field
2023-01-12 11:11:00 +08:00
2d5208fc29
Merge pull request #9485 from thalesmg/refactor-doc-gen-50-mkIII
...
feat(docs): add tags to schemas
2023-01-11 16:36:38 -03:00
9d99d180f9
Merge pull request #9709 from thalesmg/mongodb-bridge-payload-template-v50
...
feat(mongodb): add `payload_template` field for bridge (e5.0)
2023-01-11 11:14:46 -03:00
3e9c4f444f
refactor: remove the auto_reconnect field
2023-01-11 21:47:06 +08:00
a9d0143d5f
chore(license): change `key` type to `binary()`
...
So that the `raw_default` field in the schema JSON renders properly.
2023-01-11 09:10:04 -03:00
48e1ba4832
feat(docs): add tags to schemas
...
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
2023-01-11 09:10:03 -03:00
f3cc722172
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-10 09:06:12 -03:00
67f2159a27
Merge pull request #9653 from zmstone/0101-authz-schema-union-member-selection
...
0101 authz schema union member selection
2023-01-09 22:17:51 +01:00
b9f258b737
feat(mongodb): add `payload_template` field for bridge (e5.0)
...
https://emqx.atlassian.net/browse/EMQX-8705
Adds a `payload_template` fields that allows users to customize the
payload to publish to MongoDB.
2023-01-09 13:38:11 -03:00
20f49b903f
Merge pull request #9702 from lafirest/feat/timescale_matrix
...
feat(bridges): add timescale && matrix bridges
2023-01-10 00:06:17 +08:00
062f14bd65
test(bridges): add timescale && matrix test cases
2023-01-09 23:04:36 +08:00
e08d6dbc9b
feat(bridges): add timescale && matrix bridges
2023-01-09 23:04:29 +08:00
c3635f537a
ci: wait for redis in emqx_ee_bridge_redis_SUITE
2023-01-09 14:39:56 +01:00
0697c692ed
refactor: mongo_type and redis_type are not mandatory
...
the connector schemas are shared between authn, auth and bridges,
the difference is that authn and authz configs are unions like:
[
{mongo_type = rs,
...
}
{another backend config}
]
however the brdige types are maps like, for example:
mongodb_rs.$name {
mongo_type = rs
...
}
in which case, the mongo_type is not required.
in order to keep the schema static as much as possible,
this field is chanegd to 'required => false' with a default value.
However, for authn and authz, the union selector will still raise
exception if the there is no type provided.
2023-01-09 14:26:16 +01:00
ffb09f0c4d
test: rename a test option name to avoid clashing with prod config name
2023-01-09 14:26:16 +01:00
e52f9d5920
refactor: use union member type selector for authz sources
2023-01-09 14:26:16 +01:00
e498010f23
chore: update default license (100 conns)
2023-01-09 09:11:09 +01:00
3859878985
Merge pull request #9684 from zmstone/0105-do-not-use-testcase-skip
...
test: do not use tc_user_skip for test cases
2023-01-09 09:02:06 +01:00
ea405fe55d
test(bridges): add test case for the PostgreSQL backend
2023-01-06 11:56:28 +08:00
f7b50c56da
feat(bridges): integrate PostgreSQL into bridges
...
support both simple and batch query
2023-01-06 11:47:22 +08:00
5f12cdff6c
ci: should not skip tests in github actions
2023-01-05 22:47:08 +01:00
fc12a8c4c8
test: do not use tc_user_skip for test cases
2023-01-05 22:47:06 +01:00
af31ed4264
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-05 10:16:00 -03:00
fd360ac6c0
feat(buffer_worker): refactor buffer/resource workers to always use queue
...
This makes the buffer/resource workers always use `replayq` for
queuing, along with collecting multiple requests in a single call.
This is done to avoid long message queues for the buffer workers and
rely on `replayq`'s capabilities of offloading to disk and detecting
overflow.
Also, this deprecates the `enable_batch` and `enable_queue` resource
creation options, as: i) queuing is now always enables; ii) batch_size
> 1 <=> batch_enabled. The corresponding metric
`dropped.queue_not_enabled` is dropped, along with `batching`. The
batching is too ephemeral, especially considering a default batch time
of 20 ms, and is not shown in the dashboard, so it was removed.
2023-01-05 10:15:09 -03:00
0fd8880d0a
fix(kafka_producer): avoid multiplication of metrics when bridge is recreated
2023-01-05 10:11:59 -03:00
8e59319bfe
fix(kafka_producer): fix message loss when kafka connection is down
2023-01-05 10:11:59 -03:00
5bd9f110d6
test: attempt to reduce flakiness
2023-01-05 10:11:59 -03:00
3304025a9a
Merge pull request #9677 from thalesmg/test-other-gcp-flaky-v50
...
test: attempt to fix flaky test
2023-01-04 13:30:53 -03:00
5df485df17
test: attempt to fix flaky test
2023-01-04 10:38:21 -03:00
4feaf5916f
fix(mongodb): add `type` and `name` fields to the `get`/`post` API docs
...
https://emqx.atlassian.net/browse/EMQX-8643
Adds thoses fields to MongoDB API docs so the frontend may use them to
render properly.
2023-01-03 11:46:58 -03:00
d6c8a106da
test(gcp_pubsub): fix flaky test
2023-01-03 08:52:37 -03:00
7985cd3536
Merge pull request #9638 from lafirest/fix/mysql_dup
...
fix(mysql): fix the problem of data loss and bad match when mysql is disconnected
2023-01-02 23:15:44 +08:00
7e02eac3bc
Merge pull request #9619 from thalesmg/refactor-gauges-v50
...
refactor(metrics): use absolute gauge values rather than deltas (v5.0)
2023-01-02 10:56:47 -03:00
c77717b1f1
test(mysql): fix test case error
2023-01-02 21:55:22 +08:00
69ac6b9e0d
fix(kafka): fix handling of `dropped.queue_full` event from wolff
...
https://emqx.atlassian.net/browse/EMQX-8530
cd20a37e65/src/wolff_producer.erl (L772-L773)
2023-01-02 09:23:38 -03:00
fc6d9d0890
chore: issue new Enterprise evaluation license
2023-01-02 11:50:57 +01:00
b5d2fc0ede
chore: update BSL license year
2023-01-02 11:23:14 +01:00
dbc10c2eed
chore: update copyright year 2023
2023-01-02 09:22:27 +01:00
231d6da4dc
Merge pull request #9644 from id/fix-kafka-bridge-connectivity-detection
...
fix(kafka): detect connectivity in on_get_status
2022-12-30 21:54:56 +01:00
61246c43c4
fix(kakfa_producer): prevent multiple producers from multiplying each other's metrics
2022-12-30 16:51:24 -03:00
8b060a75f1
refactor(metrics): use absolute gauge values rather than deltas
...
https://emqx.atlassian.net/browse/EMQX-8548
Currently, we face several issues trying to keep resource metrics
reasonable. For example, when a resource is re-created and has its
metrics reset, but then its durable queue resumes its previous work
and leads to strange (often negative) metrics.
Instead using `counters` that are shared by more than one worker to
manage gauges, we introduce an ETS table whose key is not only scoped
by the Resource ID as before, but also by the worker ID. This way,
when a worker starts/terminates, they should set their own gauges to
their values (often 0 or `replayq:count` when resuming off a queue).
With this scoping and initialization procedure, we'll hopefully avoid
hitting those strange metrics scenarios and have better control over
the gauges.
2022-12-30 16:51:24 -03:00
0ce1ca89b7
refactor: use string type for server and servers
2022-12-30 14:20:23 +01:00
a612eacf3a
Merge pull request #9641 from thalesmg/fix-gcp-pubsub-jwt-refresh-v50
...
fix(gcp_pubsub): fix potential jwt accumulation and lack of refresh (v5.0)
2022-12-30 09:33:41 -03:00
Stefan Strigler