yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,376 Commits 393 Branches 669 Tags 74 MiB
9e954585ae
Commit Graph

283 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi
fb61c2b266 perf: avoid getting metrics (gen_server:call) for each resource lookup 2023-02-10 19:40:37 +01:00
Zaiming (Stone) Shi
42dfaf3ef2
Merge pull request #9910 from sstrigler/EMQX-8861-improve-bridge-restart-button-behaviour 
EMQX 8861 improve bridge restart button behaviour
 2023-02-09 18:00:48 +01:00
Andrew Mayorov
81b1bab11e
chore: bump emqx_resource version to 0.1.7 
Also add the changelog entry.
 2023-02-08 14:21:30 +03:00
Andrew Mayorov
c6fc0ec8cd
fix(bufworker): do not avoid retry if inflight table is full 
Otherwise there's no other piece of code that would retry the inflight
queries in that case.
 2023-02-08 14:08:04 +03:00
Stefan Strigler
86f3f5787f feat: allow to manually re-connect disconected bridge 2023-02-07 11:58:30 +01:00
Zaiming (Stone) Shi
7ea140599a
Merge pull request #9894 from id/ci-always-run-static-checks 
ci: always run static_checks
 2023-02-02 16:33:19 +01:00
Zaiming (Stone) Shi
feca4cc0a5
Merge pull request #9892 from zmstone/0202-docs-cosmetic 
0202 docs cosmetic
 2023-02-02 15:43:58 +01:00
Zaiming (Stone) Shi
58627b7958 chore(emqx_resource_manager): ignore unused return value for dialyzer 2023-02-02 14:11:12 +01:00
zhongwencool
ee852d8204
Merge pull request #9886 from zhongwencool/mongo-connection-default-async 
fix: remove async mode from mongodb/redis/mysql/pgsql bridge
 2023-02-02 21:08:01 +08:00
Zaiming (Stone) Shi
d5c482b0b0 docs: remove timer unit from description 
the user input has time unit. e.g. "5s" for 5 seconds etc.
 2023-02-02 13:49:20 +01:00
Zaiming (Stone) Shi
9864587389 fix: send to buffer-supported connector even when disconnected 2023-02-02 12:04:17 +01:00
Zhongwen Deng
f8936013b7 chore: replace async with sync 2023-02-02 17:37:18 +08:00
Zhongwen Deng
22c3f50020 fix: add query_mode_sync_only for mysql pgsql redis mongodb bridge 2023-02-02 17:37:18 +08:00
Ilya Averyanov
14f528cc86 fix(resources): fix resource lifecycle 
* do not resume all buffer workers on successful healthcheck
* do not pass undefined state to resource healthcheck callback
 2023-02-01 18:26:13 +02:00
Zaiming (Stone) Shi
b3ad9e97d2
Merge pull request #9870 from keynslug/fix/mqtt-connection-loss-feedback 
feat(mqtt-bridge): avoid middleman process
 2023-01-31 19:12:18 +01:00
Andrew Mayorov
c76311c9c3
fix(buffer): count inflight batches properly 2023-01-31 18:30:42 +03:00
Zaiming (Stone) Shi
b3e486041b
Merge pull request #9853 from zmstone/0127-refactor-buffer-worker-no-need-to-keep-request-for-reply-callback 
0127 refactor buffer worker no need to keep request for reply callback
 2023-01-31 08:44:01 +01:00
Stefan Strigler
27881064dc fix: increase dropped.queue_full by number of messages 2023-01-30 11:37:35 +01:00
Zaiming (Stone) Shi
d47941601d refactor(buffer_worker): rename trace points 2023-01-28 11:52:11 +01:00
Zaiming (Stone) Shi
7f66c6a9e2
Merge pull request #9840 from olcai/redact-influxdb-tokens 
fix: redact influxdb tokens in logs and reduce log level
 2023-01-28 11:47:36 +01:00
Zaiming (Stone) Shi
fc38ea9571 refactor(buffer_worker): do not keep request body in reply context 
the request body can be potentially very large
the reply context is sent to the async call handler and kept
in its memory until the async reply is received from bridge
target service.

this commit tries to minimize the size of the reply context
by replacing the request body with `[]`.
 2023-01-27 17:12:55 +01:00
Zaiming (Stone) Shi
578271ea3d refactor: use lists:map instead of lc for safty 2023-01-27 15:15:46 +01:00
Zaiming (Stone) Shi
f793807bc1 refactor(buffer_worker): rename function 
batch_reply_after_query to handle_async_batch_reply
 2023-01-27 15:04:28 +01:00
Zaiming (Stone) Shi
262c3a2869 refactor(buffer_worker): rename function 
from reply_after_query to handle_async_reply
 2023-01-27 15:03:18 +01:00
Zaiming (Stone) Shi
52b75ada04
Merge pull request #9832 from sstrigler/EMQX-8774-failure-to-handle-timeout-error-in-resource-worker 
EMQX 8774 failure to handle timeout error in resource worker
 2023-01-27 14:36:44 +01:00
Zaiming (Stone) Shi
514609bcf7
Merge pull request #9850 from zmstone/0127-fix-influxdb-bridge-atom-leak 
0127 fix influxdb bridge atom leak
 2023-01-27 14:30:20 +01:00
Zaiming (Stone) Shi
d53106145f fix: stop resource when resource manager terminates 2023-01-27 12:39:05 +01:00
Stefan Strigler
a180bd9aa5 fix: catch error, not exit 2023-01-27 11:40:06 +01:00
Stefan Strigler
b7e3f9d5a6 fix: try-case-of rather than try-of 
try-of catches only what happens within but not after
 2023-01-27 11:40:06 +01:00
Zaiming (Stone) Shi
db2f631a8a refactor(buffer_worker): simplify caller reply 2023-01-27 11:33:45 +01:00
Zaiming (Stone) Shi
d4fab92b72 refactor(buffer_worker): no need to keep request for REPLY macro 2023-01-27 10:41:30 +01:00
Zaiming (Stone) Shi
1f799dfd59 fix: reply with {error, buffer_overflow} when discarded 2023-01-26 17:15:36 +01:00
Zaiming (Stone) Shi
ed28789164 refactor(buffer_worker): no need to return after collect into buf queue 2023-01-26 14:50:40 +01:00
Zaiming (Stone) Shi
25b4821adc refactor: move the the per-message overflow log from error to info level 2023-01-26 14:48:43 +01:00
Zaiming (Stone) Shi
bb26632c8a fix(buffer_worker): fix a wrong assertion 
the assertion is to ensure queue items are not binary
but should not assert the queue itself
 2023-01-26 14:33:16 +01:00
Erik Timan
805d08e823 fix: reduce log level from error to warning in several places 
This reduces the log level from error to warning in places that are
connected to the influxdb bridge. Transient errors for external
resources should not render an error log.
 2023-01-25 14:49:50 +01:00
Zaiming (Stone) Shi
5fdf7fd24c fix(kafka): use async callback to bump success counters 
some telemetry events from wolff are discarded:

* dropped:
    this is double counted in wolff,
    we now only subscribe to the dropped_queue_full event
* retried_failed:
    it has different meanings in wolff,
    in wolff, it means it's the 2nd (or onward) produce attempt
    in EMQX, it means it's eventually failed after some retries

* retried_success
    since we are going to handle the success counters in callbac
    this having this reported from wolff will only make things
    harder to understand

* failed
    wolff never fails (unelss drop which is a different counter)
 2023-01-24 21:12:36 +01:00
Erik Timan
9d20431257 fix(emqx_resource): fix crash while flushing queue 
We used next_event for flushing the queue in emqx_resource, but this
leads to a crash. We now call flush_worker/1 instead.
 2023-01-24 14:13:35 +01:00
Erik Timan
28718edbfd chore: bump application VSNs 2023-01-24 14:12:34 +01:00
Zaiming (Stone) Shi
8fde169abb
Merge pull request #9821 from thalesmg/buffer-worker-expiry-v50 
feat(buffer_worker): add expiration time to requests
 2023-01-24 13:54:04 +01:00
Thales Macedo Garitezi
ca4a262b75 refactor: re-organize dealing with unrecoverable errors 2023-01-20 12:00:17 -03:00
Thales Macedo Garitezi
6fa6c679bb feat(buffer_worker): add expiration time to requests 
With this, we avoid performing work or replying to callers that are no
longer waiting on a result.

Also introduces two new counters:

- `dropped.expired` :: happens when a request expires before being
  sent downstream
- `late_reply` :: when a response is receive from downstream, but the
  caller is no longer for a reply because the request has expired, and
  the caller might even have retried it.
 2023-01-20 11:36:52 -03:00
Thales Macedo Garitezi
47f796dd12 refactor: rename emqx_resource_worker -> emqx_resource_buffer_worker 
To make it more clear that it's purpose is serve as a buffering layer.
 2023-01-18 16:15:34 -03:00
Ilya Averyanov
44a6e5ed15 chore(resources): add missing parameters to emqx_resource schema 2023-01-18 14:33:45 +02:00
Zaiming (Stone) Shi
d4f3b4c8c2 Merge remote-tracking branch 'origin/master' into fix-buffer-clear-replayq-on-delete-v50 2023-01-18 11:39:47 +01:00
Ivan Dyachkov
430b0a03d4
Merge pull request #9780 from id/fix-ensure-no-colon-in-filenames 
fix: ensure no colon in filenames
 2023-01-18 09:36:16 +01:00
Zaiming (Stone) Shi
faf5916ed6 test: relax recoverable/unrecoverable error check 
for now, treat all other errors unrecoverable
 2023-01-18 07:52:28 +01:00
Thales Macedo Garitezi
5c2ac0ac81 chore: don't cancel inflight items upon worker death; retry them 2023-01-17 19:50:30 -03:00
Thales Macedo Garitezi
087b667263 fix(buffer_worker): allow signalling unrecoverable errors 2023-01-17 19:50:30 -03:00
Thales Macedo Garitezi
4ed7bff33f chore: fix dialyzer warnings 2023-01-17 16:49:16 -03:00