Commit Graph

15346 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
bb4fef3cfd
Merge pull request #10752 from thalesmg/fix-pulsar-redact-auth-r50
fix(pulsar): redact error reason
2023-05-19 16:47:05 -03:00
Andrew Mayorov
9aaa0b6fe1
Merge pull request #9927 from emqx/file-transfer
feat: implement file transfer over mqtt
2023-05-19 18:35:13 +03:00
Thales Macedo Garitezi
4327e40740 fix(pulsar): redact error reason
Fixes https://emqx.atlassian.net/browse/EMQX-9940

Error log after fix:

```
2023-05-19T13:09:26.304769+00:00 [error] msg: failed_to_start_pulsar_client, mfa: emqx_bridge_pulsar_impl_producer:on_start/2, line: 104, instance_id: <<"bridge:pulsar_producer:pprodu">>, pulsar_hosts: pulsar://pulsar:6652, reason: {#{{"pulsar",6652} => #{error => 'AuthenticationError',message => "Unable to authenticate",request_id => 18446744073709551615}},{child,undefined,'pulsar_producer:pprodu:emqx@127.0.0.1',{pulsar_client,start_link,['pulsar_producer:pprodu:emqx@127.0.0.1',["pulsar://pulsar:6652"],#{conn_opts => #{auth_data => <<"******">>,auth_method_name => <<"basic">>},ssl_opts => []}]},transient,false,5000,worker,[pulsar_client]}}
```
2023-05-19 10:09:57 -03:00
lafirest
264404e180
Merge pull request #10511 from lafirest/fix/resource_safe_log
fix(resource): make some logging of the resource manager more secure
2023-05-19 20:42:21 +08:00
lafirest
1aa904d019
Merge pull request #10738 from lafirest/fix/tdengine_template
fix(tdengine): add supports for the `super table` feature in the SQL template
2023-05-19 20:41:50 +08:00
Thales Macedo Garitezi
8cfcc8fbaf
Merge pull request #10743 from thalesmg/fix-bridge-metric-aggregation-r50
fix(bridge_api): don't crash when formatting empty/unknown bridge metrics
2023-05-19 09:07:22 -03:00
Kjell Winblad
28dda39e39
Merge pull request #10724 from kjellwinblad/kjell/docs/missing_http_api_endpoint_descriptions/EMQX-9767
docs: make sure there is a summary for all API endpoints
2023-05-19 12:05:41 +02:00
firest
baeb96a6e4 chore: update changes 2023-05-19 15:36:18 +08:00
firest
0eea8438bf fix(resource): make some logging of the resource manager more secure 2023-05-19 15:28:19 +08:00
firest
5a08a7b9de fix(tdengine): minor improvement of code and changes 2023-05-19 11:14:31 +08:00
Paulo Zulato
9c15720fee
Merge pull request #10726 from paulozulato/fix-large-health-check-interval
fix: validate range for some bridge options
2023-05-18 19:25:24 -03:00
Thales Macedo Garitezi
09ea2e2224 fix(bridge_api): don't crash when formatting empty/unknown bridge metrics
Fixes https://emqx.atlassian.net/browse/EMQX-9872
2023-05-18 14:25:13 -03:00
Thales Macedo Garitezi
56634e5238
Merge pull request #10725 from thalesmg/bump-gproc-091-r50
chore: bump gproc -> 0.9.0.1 (r5.0)
2023-05-18 10:38:06 -03:00
Thales Macedo Garitezi
49424cc9a2
Merge pull request #10733 from thalesmg/fix-pulsar-sensitive-struct-r50
fix(pulsar): mark whole auth struct as sensitive, update dashboard version and improve bridge type matching errors (r5.0)
2023-05-18 10:15:10 -03:00
Paulo Zulato
5d289ade56 fix: validate range for some bridge options
Fixes https://emqx.atlassian.net/browse/EMQX-9864

Setting a very large interval can cause `erlang:start_timer` to crash.
Also, setting auto_restart_interval or health_check_interval to "0s"
causes the state machine to be in loop as time 0 is handled separately:

| state_timeout() = timeout() | integer()
| (...)
| If Time is relative and 0 no timer is actually started, instead the the
| time-out event is enqueued to ensure that it gets processed before any
| not yet received external event.
from "https://www.erlang.org/doc/man/gen_statem.html#type-state_timeout"

Therefore, both fields are now validated against the range [1ms, 1h],
which doesn't cause above issues.
2023-05-18 10:10:58 -03:00
firest
f1a3e5965e chore: update apps version && changes 2023-05-18 17:34:28 +08:00
firest
142125b9e4 test(tdengine): add test cases to cover the super table feature 2023-05-18 17:01:27 +08:00
firest
6ff77b221b fix(tdengine): add supports for the automatically create feature in the SQL template 2023-05-18 15:13:08 +08:00
JianBo He
0dc3e10533
Merge pull request #10699 from HJianBo/conf-tests-2
chore(ci): add conf tests
2023-05-18 09:38:06 +08:00
Thales Macedo Garitezi
5d5c16a56d feat(bridges): use union member selector function for better error messages 2023-05-17 17:57:29 -03:00
Thales Macedo Garitezi
659cf64ad7 feat(pulsar): use an union member selector for better error messages 2023-05-17 17:56:53 -03:00
Thales Macedo Garitezi
10f6edd6ea
Merge pull request #10717 from thalesmg/fix-bw-flush-timer-full-infl-r50
fix(buffer_worker): avoid setting flush timer when inflight is full
2023-05-17 14:49:27 -03:00
Thales Macedo Garitezi
dcccc0910a fix(pulsar): mark whole auth struct as sensitive (r5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-9900

I tried to patch hocon itself to filter the sensitive data, but the
way it's currently structured doesn't seem to keep that field
metadata.  So, for now, we can just mark the whole auth union as
sensitive.
2023-05-17 11:50:22 -03:00
Thales Macedo Garitezi
38ef99caf5 chore(dashboard): bump dashboard -> e1.0.7-beta.3 2023-05-17 11:50:22 -03:00
Thales Macedo Garitezi
060efd6964 chore: bump gproc -> 0.9.0.1 (r5.0)
Includes this fix: https://github.com/uwiger/gproc/pull/193
2023-05-17 11:05:44 -03:00
zhongwencool
ea8ac877b0
Merge pull request #10713 from zhongwencool/put-webhook-request-timeout-into-resource-opts
feat: update wehbook's request_timeout into resource_opts
2023-05-17 21:20:42 +08:00
Ilya Averyanov
cf117af4da
Merge pull request #10732 from keynslug/ft/ft-readme
chore(ft): provide more details in README.md
2023-05-17 13:04:25 +03:00
JianBo He
73f1421d1f ci: prefixed config files wit v/e 2023-05-17 17:52:08 +08:00
JianBo He
a05156df9a chore: tests v5.0.20~25 confs 2023-05-17 17:52:08 +08:00
JianBo He
90d1a0096c chore: make spellcheck happy 2023-05-17 17:52:08 +08:00
JianBo He
07e46592a8 chore(ci): add conf tests 2023-05-17 17:52:08 +08:00
Andrew Mayorov
c95ef71fb5
chore(ft): provide more details in README.md 2023-05-17 12:49:20 +03:00
Kjell Winblad
ab8b8ccaad docs: make sure there is a summary for all API endpoints
Many HTTP API endpoints did not have a summary when looking at the API
docs at "http://emqx_host_name:18083/api-docs". This has been fixed by
making sure there is a summary corresponding to all endpoints.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9767
2023-05-17 09:58:56 +02:00
Ilya Averyanov
c54d04450c
Merge pull request #10723 from savonarola/0516-improve-rebalance-logging
Improve rebalance logging
2023-05-17 09:21:10 +03:00
某文
2b99a9b988 feat: hide resource_opts's request_timeout 2023-05-17 13:41:11 +08:00
lafirest
b5da9eb9ad
Merge pull request #10728 from lafirest/fix/port_ruleengine_export_colums
fix: cannot access columns exported by FOREACH in DO clause
2023-05-17 11:46:17 +08:00
firest
1f7ede90a4 chore: update app version && changes 2023-05-17 11:01:31 +08:00
firest
18043150be fix: cannot access columns exported by FOREACH in DO clause 2023-05-17 10:55:47 +08:00
JianBo He
0dd3325d70
Merge pull request #10711 from lafirest/fix/fix_infinity_bucket
fix(limiter): a bucket with an infinity rate shouldn't be added to limiter server
2023-05-17 10:40:59 +08:00
JianBo He
2a1814691e
Merge pull request #10721 from zhongwencool/10340-changlog
chore: update 10340's changelog
2023-05-17 08:54:11 +08:00
Ilya Averyanov
b71955e368 fix(ft): bump application versions 2023-05-17 00:29:15 +03:00
Ilya Averyanov
1a8cf0e392 Merge branch 'release-50' into file-transfer
* release-50:
  fix(pulsar): use a binary duration as default `health_check_interval`
  docs: add changelog entry
  docs: clarify description of bridge username and password
  chore: bump to v5.0.25
  fix(limiter): adjust type for compatibility
  fix(limiter): fix that update node-level limiter config will not working
  chore: upgrade dashboard to v1.2.4-1 for ce
  chore: upgarde rulesql to 0.1.6 to fix invaid utf8 input
  chore: add changelog for 10659
  fix: crash when sysmon.os.mem_check_interval = disabled
  chore: bump influxdb version && update changes
  refactor(influxdb): move influxdb bridge into its own app
  chore: add listener default changelog
  fix: ocsp cache SUITE failed
  fix: ensure atom key for emqx_config:get
  fix: only fill cerf_file default in server side
  fix: authn init is empty
  fix: bad listeners default ssl_options
2023-05-17 00:27:55 +03:00
Thales Macedo Garitezi
85089a3210 fix(buffer_worker): correctly flush the buffer workers when inflight table room is made
The previous commit uncovered another bug that was hidden by it:
`maybe_flush_after_async_reply` was sending a message to the wrong
PID.  It was sending a message to `self()` meaning to target a buffer
worker, but `self()` in that context is never the buffer worker, it's
the connector's worker.

This change also revealed a race condition where the buffer workers
could stop flushing messages.  So we piggy-backed on the atomic update
of the table size count to check if the buffer worker should be poked
to continue flushing.  This allows us to get rid of
`maybe_flush_after_async_reply` altogether.
2023-05-16 17:15:42 -03:00
Thales Macedo Garitezi
4a034c70bf
Merge pull request #10716 from thalesmg/fix-pulsar-health-check-default-r50
fix(pulsar): use a binary duration as default `health_check_interval`
2023-05-16 16:17:56 -03:00
Ilya Averyanov
d4b60c5610 chore(rebalance): improve debug logging 2023-05-16 18:59:12 +03:00
某文
77cf19c96c chore: update 10340's changelog 2023-05-16 22:59:08 +08:00
Thales Macedo Garitezi
cebde87114 fix(pulsar): use a binary duration as default health_check_interval
Fixes https://emqx.atlassian.net/browse/EMQX-9885

The frontend needs the default value to match the duration (binary)
type to display correctly.
2023-05-16 11:29:29 -03:00
Thales Macedo Garitezi
657df05ad9 fix(buffer_worker): avoid setting flush timer when inflight is full
Fixes https://emqx.atlassian.net/browse/EMQX-9902

When the buffer worker inflight window is full, we don’t need to set a
timer to flush the messages again because there’s no more room, and
one of the inflight windows will flush the buffer worker by calling
`flush_worker`.

Currently, we do set the timer on such situation, and this fact
combined with the default batch time of 0 yields a busy loop situation
where the CPU spins a lot while inflight messages do not return.
2023-05-16 11:28:58 -03:00
Andrew Mayorov
6f8f21106b
fix(assert): use unpredictable binding names in macros
Also translate macro vars to ALLCAPS for consistency.
2023-05-16 16:46:05 +03:00
Andrew Mayorov
e3b822c1a0
chore: remove empty header
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-05-16 16:45:41 +03:00