9d91ebe266
Merge pull request #9842 from savonarola/fix-redis-cluster-recover
...
fix: fix redis cluster resource recovery
2023-02-01 10:38:52 +02:00
fce1e74c3d
fix(connector): fix redis cluster resource recovery
2023-01-31 16:55:05 +02:00
b3e486041b
Merge pull request #9853 from zmstone/0127-refactor-buffer-worker-no-need-to-keep-request-for-reply-callback
...
0127 refactor buffer worker no need to keep request for reply callback
2023-01-31 08:44:01 +01:00
d47941601d
refactor(buffer_worker): rename trace points
2023-01-28 11:52:11 +01:00
7f66c6a9e2
Merge pull request #9840 from olcai/redact-influxdb-tokens
...
fix: redact influxdb tokens in logs and reduce log level
2023-01-28 11:47:36 +01:00
52b75ada04
Merge pull request #9832 from sstrigler/EMQX-8774-failure-to-handle-timeout-error-in-resource-worker
...
EMQX 8774 failure to handle timeout error in resource worker
2023-01-27 14:36:44 +01:00
514609bcf7
Merge pull request #9850 from zmstone/0127-fix-influxdb-bridge-atom-leak
...
0127 fix influxdb bridge atom leak
2023-01-27 14:30:20 +01:00
30a8a436b4
fix(influxdb_bridge): no need to create atom for pool name
2023-01-27 12:39:05 +01:00
d35e46b2d5
Merge pull request #9838 from keynslug/fix/redis-cluster-batching
...
feat(redis): disable batching in redis_cluster bridges
2023-01-27 15:27:57 +04:00
7d18128ba9
test: async write can return noproc
2023-01-27 11:43:51 +01:00
2d62de5188
test: fix expected result from timeout error
2023-01-27 11:43:48 +01:00
f6b3b930b0
chore: improve a error log
2023-01-26 14:21:27 +01:00
1f235ffee9
refactor(emqx_ee_connector): redo readact key function
2023-01-26 10:30:13 +01:00
bafc430184
Merge pull request #9837 from zmstone/0123-fix-resource-worker-leak-influxdb
...
0123 fix resource worker leak influxdb
2023-01-26 09:14:38 +01:00
26fcaecad7
fix(redis): disable batching in `redis_cluster` bridges
...
Through configuration subsystem.
2023-01-25 17:28:11 +03:00
805d08e823
fix: reduce log level from error to warning in several places
...
This reduces the log level from error to warning in places that are
connected to the influxdb bridge. Transient errors for external
resources should not render an error log.
2023-01-25 14:49:50 +01:00
8836494542
fix: redact influxdb tokens in a few logs
2023-01-25 14:48:32 +01:00
903a77b471
test(redis): ensure batch query hit different cluster shards
...
This will inevitably fail: it's not generally possible to update
different keys through the same cluster connection, one or more
update will fail with `MOVED` status. This testcase should serve
as a regression test later.
2023-01-25 15:33:05 +03:00
5fdf7fd24c
fix(kafka): use async callback to bump success counters
...
some telemetry events from wolff are discarded:
* dropped:
this is double counted in wolff,
we now only subscribe to the dropped_queue_full event
* retried_failed:
it has different meanings in wolff,
in wolff, it means it's the 2nd (or onward) produce attempt
in EMQX, it means it's eventually failed after some retries
* retried_success
since we are going to handle the success counters in callbac
this having this reported from wolff will only make things
harder to understand
* failed
wolff never fails (unelss drop which is a different counter)
2023-01-24 21:12:36 +01:00
feb0238911
fix(influxdb): ensure client is stopped before returning error
2023-01-24 20:35:02 +01:00
7603ab490b
fix(emqx_ee_connector): fix passing of influxdb ssl opts
2023-01-24 14:30:40 +01:00
28718edbfd
chore: bump application VSNs
2023-01-24 14:12:34 +01:00
8fde169abb
Merge pull request #9821 from thalesmg/buffer-worker-expiry-v50
...
feat(buffer_worker): add expiration time to requests
2023-01-24 13:54:04 +01:00
ca4a262b75
refactor: re-organize dealing with unrecoverable errors
2023-01-20 12:00:17 -03:00
6fa6c679bb
feat(buffer_worker): add expiration time to requests
...
With this, we avoid performing work or replying to callers that are no
longer waiting on a result.
Also introduces two new counters:
- `dropped.expired` :: happens when a request expires before being
sent downstream
- `late_reply` :: when a response is receive from downstream, but the
caller is no longer for a reply because the request has expired, and
the caller might even have retried it.
2023-01-20 11:36:52 -03:00
8c52264c41
fix: atom leak when doing Kafka bridge dry-run
...
A new atom was created every time one did a dry run of a Kafka bridge
(that is, clicking the Test button in the settings dialog for the
bridge).
After this fix, we will only create a new atom when a bridge with a new
name is created. This should be acceptable as bridges with new names are
created relatively rarely and it seems to be useful to have an unique
atom for every Wolff producer.
Fixes: https://emqx.atlassian.net/browse/EMQX-8739
2023-01-20 11:50:20 +01:00
a5424959c6
refactor: avoid operator precedence bugs
2023-01-18 16:15:34 -03:00
47f796dd12
refactor: rename `emqx_resource_worker` -> `emqx_resource_buffer_worker`
...
To make it more clear that it's purpose is serve as a buffering layer.
2023-01-18 16:15:34 -03:00
f9843de7ae
Merge pull request #9628 from savonarola/fix-flaky-redis-bridge-test
...
chore(ee bridge): fix Redis bridge test flakyness
2023-01-18 20:56:13 +02:00
caf50d1b35
Merge pull request #9792 from savonarola/redis-bridge-examples
...
chore(bridges): add more helpful descriptions
2023-01-18 18:01:35 +01:00
f9c3115cd4
chore(bridges): add more helpful descriptions
...
Add more helpful descriptions for `command_template` option
of Redis bridge.
2023-01-18 18:17:08 +02:00
7abba17b25
Merge pull request #9765 from zmstone/0115-add-password-converter
...
fix(schema): add password converter to ensure its binary() type
2023-01-18 15:09:05 +01:00
2d01e604a5
Merge pull request #9799 from zmstone/0118-fix-key_dispatch-kafka-produce-strategy
...
0118 fix key dispatch kafka produce strategy
2023-01-18 13:52:49 +01:00
f6fbbf3ee3
chore(bridges): reduce Redis bridge flakyness
2023-01-18 14:34:11 +02:00
8f275a66d0
test: add coverage for key_dispatch partition strategy
2023-01-18 11:47:37 +01:00
d4f3b4c8c2
Merge remote-tracking branch 'origin/master' into fix-buffer-clear-replayq-on-delete-v50
2023-01-18 11:39:47 +01:00
3872c4451f
fix(kafka): olp fix was accidentally deleted in 0fd8880d0a
2023-01-18 10:52:29 +01:00
087b667263
fix(buffer_worker): allow signalling unrecoverable errors
2023-01-17 19:50:30 -03:00
fa01deb3eb
chore: retry as much as possible, don't reply to caller too soon
2023-01-17 16:49:15 -03:00
006b4bda97
feat(buffer_worker): monitor async workers and cancel their inflight requests upon death
2023-01-17 16:48:48 -03:00
32a9e60313
feat(buffer_worker): also use the inflight table for sync requests
...
Related: https://emqx.atlassian.net/browse/EMQX-8692
This should also correctly account for `retried.*` metrics for sync
requests.
Also fixes cases where race conditions for retrying async requests
could potentially lead to inconsistent metrics.
Fixes more cases where a stale reference to `replayq` was being held
accidentally after a `pop`.
2023-01-17 16:48:48 -03:00
4c25be8a2c
chore(influxdb): remove deprecated value from example
2023-01-17 16:48:48 -03:00
f37b3e4bc4
test: test against `bridges_probe` API
2023-01-17 15:29:19 +01:00
5fc31dcd9e
fix: use atom keys in example to make sure they exist
2023-01-17 15:29:19 +01:00
4a7e74f5d6
fix(schema): add password converter to ensure its binary() type
2023-01-17 11:12:17 +01:00
1c7fe91a91
fix: typos in emqx_ee_bridge_mongodb.conf
2023-01-17 16:33:54 +08:00
40948f3c0b
fix(influxdb): authentication is not required for influxdb v1 api
2023-01-17 09:29:22 +08:00
a7fc5e8fe1
Merge pull request #9761 from zmstone/0114-fix-kafka-value-template-and-docs
...
feat: introduce 'this' concept for placeholder, and use it in Kafka bridge
2023-01-16 13:37:29 +01:00
47414e0d53
docs: improve kafka key and value field description
2023-01-16 11:32:09 +01:00
376dab7cd8
Merge pull request #9743 from zmstone/0112-docs-fix-some-descriptions
...
0112 docs fix some descriptions
2023-01-16 09:56:56 +01:00
Ilya Averyanov