yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

chore: e5.0.3 changelog

This commit is contained in:
Ivan Dyachkov 2023-05-05 16:05:00 +02:00
parent 8ce2122516
commit cd5a375aad
2 changed files with 247 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

187
changes/e5.0.3.en.md Normal file
View File

@ -0,0 +1,187 @@
# e5.0.3
## Enhancements
- [#10077](https://github.com/emqx/emqx/pull/10077) Add support for QUIC TLS password protected certificate file.
- [#10128](https://github.com/emqx/emqx/pull/10128) Add support for OCSP stapling for SSL MQTT listeners.
- [#10156](https://github.com/emqx/emqx/pull/10156) Change the priority of the configuration:
1. If it is a new installation of EMQX, the priority of configuration is `ENV > emqx.conf > HTTP API`.
2. If EMQX is upgraded from an old version (i.e., the cluster-override.conf file still exists in EMQX's data directory), then the configuration priority remains the same as before. That is, `HTTP API > ENV > emqx.conf`.
Deprecated data/configs/local-override.conf.
Stabilizing the HTTP API for hot updates.
- [#10164](https://github.com/emqx/emqx/pull/10164) Add CRL check support for TLS MQTT listeners.
- [#10206](https://github.com/emqx/emqx/pull/10206) Decouple the query mode from the underlying call mode for buffer
workers.
Prior to this change, setting the query mode of a resource
such as a bridge to `sync` would force the buffer to call the
underlying connector in a synchronous way, even if it supports async
calls.
- [#10207](https://github.com/emqx/emqx/pull/10207) Use 'label' from i18n file as 'summary' in OpenAPI spec.
- [#10210](https://github.com/emqx/emqx/pull/10210) Unregister Mnesia post commit hook when Mria is being stopped.
This fixes hook failures occasionally occurring on stopping/restarting Mria.
[Mria PR](https://github.com/emqx/mria/pull/133)
- [#10224](https://github.com/emqx/emqx/pull/10224) Add the option to customize `clusterIP` in Helm chart, so that a user may set it to a fixed IP.
- [#10263](https://github.com/emqx/emqx/pull/10263) Add command 'eval-ex' for Elixir expression evaluation.
- [#10278](https://github.com/emqx/emqx/pull/10278) Refactor the directory structure of all gateways.
- [#10306](https://github.com/emqx/emqx/pull/10306) Add support for `async` query mode for most bridges.
Before this change, some bridges (Cassandra, MongoDB, MySQL, Postgres, Redis, RocketMQ, TDengine) were only allowed to be created with a `sync` query mode.
- [#10318](https://github.com/emqx/emqx/pull/10318) Now, the rule engine language's FROM clause supports both strings enclosed in double quotes (") and single quotes (').
- [#10336](https://github.com/emqx/emqx/pull/10336) Add `/rule_engine` API endpoint to manage configuration of rule engine.
- [#10354](https://github.com/emqx/emqx/pull/10354) More specific error messages when configure with bad max_heap_size value.
Log current value and the max value when the `message_queue_too_long` error is thrown.
- [#10358](https://github.com/emqx/emqx/pull/10358) Hide `flapping_detect/conn_congestion/stats` configuration.
Deprecate `flapping_detect.enable`.
- [#10359](https://github.com/emqx/emqx/pull/10359) Metrics now are not implicitly collected in places where API handlers don't make any use of them. Instead, a separate backplane RPC gathers cluster-wide metrics.
- [#10373](https://github.com/emqx/emqx/pull/10373) Deprecate the trace.payload_encode configuration.
Add payload_encode=[text,hidden,hex] option when creating a trace via HTTP API.
- [#10381](https://github.com/emqx/emqx/pull/10381) Hide the `auto_subscribe` configuration items so that they can be modified later only through the HTTP API.
- [#10385](https://github.com/emqx/emqx/pull/10385) Hide data items(rule_engine/bridge/authz/authn) from configuration files and documentation.
- [#10391](https://github.com/emqx/emqx/pull/10391) Hide a large number of advanced options to simplify the configuration file.
That includes `rewrite`, `topic_metric`, `persistent_session_store`, `overload_protection`,
`flapping_detect`, `conn_congestion`, `stats,auto_subscribe`, `broker_perf`,
`shared_subscription_group`, `slow_subs`, `ssl_options.user_lookup_fun` and some advance items
in `node` and `dashboard` section, [#10358](https://github.com/emqx/emqx/pull/10358),
[#10381](https://github.com/emqx/emqx/pull/10381), [#10385](https://github.com/emqx/emqx/pull/10385).
- [#10404](https://github.com/emqx/emqx/pull/10404) Change the default queue mode for buffer workers to `memory_only`.
Before this change, the default queue mode was `volatile_offload`. When under high message rate pressure and when the resource is not keeping up with such rate, the buffer performance degraded a lot due to the constant disk operations.
- [#10140](https://github.com/emqx/emqx/pull/10140) Integrate `Cassandra` into `bridges` as a new backend. At the current stage:
- Only support Cassandra version 3.x, not yet 4.x.
- [#10143](https://github.com/emqx/emqx/pull/10143) Add `RocketMQ` data integration bridge.
- [#10165](https://github.com/emqx/emqx/pull/10165) Support escaped special characters in InfluxDB data bridge write_syntax.
This update allows to use escaped special characters in string elements in accordance with InfluxDB line protocol.
- [#10294](https://github.com/emqx/emqx/pull/10294) When configuring a MongoDB bridge, you can now use the ${var} syntax to reference fields in the message payload within the collection field. This enables you to select the collection to insert data into dynamically.
- [#10337](https://github.com/emqx/emqx/pull/10337) Add schema registry feature.
With schema registry, one can encode and decode special serialization formats in payloads when transforming messages in Rule Engine. Currently, only [Apache Avro](https://avro.apache.org/) is supported.
- [#10363](https://github.com/emqx/emqx/pull/10363) Implement Microsoft SQL Server bridge.
- [#10573](https://github.com/emqx/emqx/pull/10573) Improved performance of Webhook bridge when using synchronous query mode.
This also should improve the performance of other bridges when they are configured with no batching.
## Bug Fixes
- [#10145](https://github.com/emqx/emqx/pull/10145) Fix `bridges` API to report error conditions for a failing bridge as
`status_reason`. Also when creating an alarm for a failing resource we include
this error condition with the alarm's message.
- [#10172](https://github.com/emqx/emqx/pull/10172) Fix the incorrect default ACL rule, which was:
```
{allow, {username, "^dashboard?"}, subscribe, ["$SYS/#"]}.
```
However, it should use `{re, "^dashboard$"}` to perform a regular expression match:
```
{allow, {username, {re,"^dashboard$"}}, subscribe, ["$SYS/#"]}.
```
- [#10174](https://github.com/emqx/emqx/pull/10174) Upgrade library `esockd` from 5.9.4 to 5.9.6.
Fix an unnecessary error level logging when a connection is closed before proxy protocol header is sent by the proxy.
- [#10195](https://github.com/emqx/emqx/pull/10195) Add labels to API schemas where description contains HTML and breaks formatting of generated documentation otherwise.
- [#10196](https://github.com/emqx/emqx/pull/10196) Use lower-case for schema summaries and descritptions to be used in menu of generated online documentation.
- [#10209](https://github.com/emqx/emqx/pull/10209) Fix bug where a last will testament (LWT) message could be published
when kicking out a banned client.
- [#10211](https://github.com/emqx/emqx/pull/10211) Hide `broker.broker_perf` config and API documents.
The two configs `route_lock_type` and `trie_compaction` are rarely used and requires a full cluster restart to take effect. They are not suitable for being exposed to users.
Detailed changes can be found here: https://gist.github.com/zmstone/01ad5754b9beaeaf3f5b86d14d49a0b7/revisions
- [#10225](https://github.com/emqx/emqx/pull/10225) Allow installing a plugin if its name matches the beginning of another (already installed) plugin name.
For example: if plugin "emqx_plugin_template_a" is installed, it must not block installing plugin "emqx_plugin_template".
- [#10226](https://github.com/emqx/emqx/pull/10226) Don't crash on validation error in `/bridges` API, return `400` instead.
- [#10242](https://github.com/emqx/emqx/pull/10242) Fixed a log data field name clash.
Piror to this fix, some debug logs may report a wrong Erlang PID which may affect troubleshooting session takeover issues.
- [#10257](https://github.com/emqx/emqx/pull/10257) Fixed the issue where `auto_observe` was not working in LwM2M Gateway.
Before the fix, OBSERVE requests were sent without a token, causing failures
that LwM2M clients could not handle.
After the fix, LwM2M Gateway can correctly observe the resource list carried by
client, furthermore, unknown resources will be ignored and printing the following
warning log:
```
2023-03-28T18:50:27.771123+08:00 [warning] msg: ignore_observer_resource, mfa: emqx_lwm2m_session:observe_object_list/3, line: 522, peername: 127.0.0.1:56830, clientid: testlwm2mclient, object_id: 31024, reason: no_xml_definition
```
- [#10286](https://github.com/emqx/emqx/pull/10286) Enhance logging behaviour during boot failure.
When EMQX fails to start due to corrupted configuration files, excessive logging is eliminated and no crash dump file is generated.
- [#10297](https://github.com/emqx/emqx/pull/10297) Keeps `eval` command backward compatible with v4 by evaluating only Erlang expressions, even on Elixir node. For Elixir expressions, use `eval-ex` command.
- [#10300](https://github.com/emqx/emqx/pull/10300) Fixed an issue where a build made with Elixir could not receive uploaded plugins until the `plugins` folder was created manually to receive the uploaded files.
- [#10315](https://github.com/emqx/emqx/pull/10315) Fix crash checking `limit` and `page` parameters in `/mqtt/delayed/messages` API call.
- [#10317](https://github.com/emqx/emqx/pull/10317) Do not expose listener level authentications before extensive verification.
- [#10323](https://github.com/emqx/emqx/pull/10323) For security reasons, the value of the `password` field in the API examples is replaced with `******`.
- [#10410](https://github.com/emqx/emqx/pull/10410) Fix config check failed when gateways are configured in emqx.conf.
This issue was first introduced in v5.0.22 via [#10278](https://github.com/emqx/emqx/pull/10278), the boot-time config check was missing.
- [#10449](https://github.com/emqx/emqx/pull/10449) Validate the ssl_options and header configurations when creating authentication http (`authn_http`).
Prior to this, incorrect `ssl` configuration could result in successful creation but the entire authn being unusable.
- [#10455](https://github.com/emqx/emqx/pull/10455) Fixed an issue that could cause (otherwise harmless) noise in the logs.
During some particularly slow synchronous calls to bridges, some late replies could be sent to connections processes that were no longer expecting a reply, and then emit an error log like:
```
2023-04-19T18:24:35.350233+00:00 [error] msg: unexpected_info, mfa: emqx_channel:handle_info/2, line: 1278, peername: 172.22.0.1:36384, clientid: caribdis_bench_sub_1137967633_4788, info: {#Ref<0.408802983.1941504010.189402>,{ok,200,[{<<"cache-control">>,<<"max-age=0, ...">>}}
```
Those logs are harmless, but they could flood and worry the users without need.
- [#10548](https://github.com/emqx/emqx/pull/10548) Fixed a race condition in the HTTP driver that would result in an error rather than a retry of the request.
Related fix in the driver: https://github.com/emqx/ehttpc/pull/45
- [#10201](https://github.com/emqx/emqx/pull/10201) In TDengine, removed the redundant database name from the SQL template.
- [#10270](https://github.com/emqx/emqx/pull/10270) Clickhouse has got a fix that makes the error message better when users click the test button in the settings dialog.
- [#10324](https://github.com/emqx/emqx/pull/10324) Previously, when attempting to reconnect to a misconfigured Clickhouse bridge through the dashboard, users would not receive an error message. This issue is now resolved, and error messages will now be displayed
- [#10438](https://github.com/emqx/emqx/pull/10438) Fix some configuration item terminology errors in the DynamoDB data bridge:
- Changed `database` to `table`
- Changed `username` to `aws_access_key_id`
- Changed `password` to `aws_secret_access_key`

60
changes/e5.0.3.zh.md Normal file
View File

@ -0,0 +1,60 @@
# e5.0.3
## 增强
- [#10077](https://github.com/emqx/emqx/pull/10077) 增加对 QUIC TLS 密码保护证书文件的支持。
- [#10278](https://github.com/emqx/emqx/pull/10278) 重构所有网关的源码目录结构。
- [#10318](https://github.com/emqx/emqx/pull/10318) 现在，规则引擎语言的 FROM 子句支持使用双引号（"）和单引号（'）括起来的字符串。
- [#10140](https://github.com/emqx/emqx/pull/10140) 支持 Cassandra 数据桥接。在当前阶段：
- 仅支持 Cassandra 3.x 版本，暂不支持 4.x。
- [#10143](https://github.com/emqx/emqx/pull/10143) 为数据桥接增加 `RocketMQ` 支持。
- [#10294](https://github.com/emqx/emqx/pull/10294) 在配置 MongoDB 桥时，现在可以使用 ${var} 语法来引用消息负载中的字段，以便动态选择要插入的集合。
## 修复
- [#10172](https://github.com/emqx/emqx/pull/10172) 修复错误的默认 ACL 规则，之前是：
```
{allow, {username, "^dashboard?"}, subscribe, ["$SYS/#"]}.
```
但执行正则表达式的匹配应该使用 `{re, "^dashboard$"}`：
```
{allow, {username, {re, "^dashboard$"}}, subscribe, ["$SYS/#"]}.
```
- [#10174](https://github.com/emqx/emqx/pull/10174) 依赖库 `esockd` 从 5.9.4 升级到 5.9.6。
修复了一个不必要的错误日志。如果连接在 proxy protocol 包头还没有发送前就关闭了， 则不打印错误日志。
- [#10211](https://github.com/emqx/emqx/pull/10211) 隐藏 `broker.broker_perf` 配置项，不再在 配置和 API 的文档中展示。
`route_lock_type` 和 `trie_compaction` 这两个配置项很少使用，且需要全集群重启才能生效，不适合暴露给用户。
详细对比： https://gist.github.com/zmstone/01ad5754b9beaeaf3f5b86d14d49a0b7/revisions
- [#10242](https://github.com/emqx/emqx/pull/10242) 修复log数据字段名称冲突。
在这个修复之前，一些调试日志可能会报告一个错误的Erlang PID，这可能会影响会话接管问题的故障调查。
- [#10257](https://github.com/emqx/emqx/pull/10257) 修复 LwM2M 网关 `auto_observe` 不工作的问题。
在修复之前，下发的 OBSERVE 请求没有 Token 从而导致 LwM2M 客户端无法处理的失败。
修复后，能正确监听 LwM2M 携带的资源列表、和会忽略未知的资源，并打印以下日志：
```
2023-03-28T18:50:27.771123+08:00 [warning] msg: ignore_observer_resource, mfa: emqx_lwm2m_session:observe_object_list/3, line: 522, peername: 127.0.0.1:56830, clientid: testlwm2mclient, object_id: 31024, reason: no_xml_definition
```
- [#10286](https://github.com/emqx/emqx/pull/10286) 优化启动失败的错误日志。
如果 EMQX 因为损坏的配置文件无法启动时，不会再打印过多的错误日志，也不再生成 crash.dump 文件。
- [#10317](https://github.com/emqx/emqx/pull/10317) 在大量验证完成前不暴露监听器级的认证功能。
- [#10323](https://github.com/emqx/emqx/pull/10323) 出于安全原因，将 API 示例中 `password` 字段的值，统一更换为 `******`。
- [#10201](https://github.com/emqx/emqx/pull/10201) 在 TDengine 桥接的 SQL 模板中，删除了多余的数据库表名。
- [#10270](https://github.com/emqx/emqx/pull/10270) Clickhouse 已经修复了一个问题，当用户在设置对话框中点击测试按钮时，错误信息会更清晰。