yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fix conflicts

This commit is contained in:
turtled 2019-08-10 02:20:38 +08:00
parent 3ebd06a0fa 7fd33ba4e2
commit 303c2414e5
168 changed files with 6333 additions and 7772 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.travis.yml
View File

@ -11,6 +11,7 @@ script:
- make xref
- make eunit
- make ct
- make proper
- make cover
after_success:

9
Makefile
View File

@ -14,10 +14,17 @@ RUN_NODE_NAME = emqxdebug@127.0.0.1
.PHONY: all
all: compile
.PHONY: tests
tests: eunit ct proper
.PHONY: proper
proper:
@rebar3 proper
.PHONY: run
run: run_setup unlock
@rebar3 as test get-deps
@rebar3 as test auto --name $(RUN_NODE_NAME) --script test/run_emqx.escript
@rebar3 as test auto --name $(RUN_NODE_NAME) --script scripts/run_emqx.escript
.PHONY: run_setup
run_setup:

87
README-CN.md Normal file
View File

@ -0,0 +1,87 @@
# EMQ X Broker
[![GitHub Release](https://img.shields.io/github/release/emqx/emqx?color=brightgreen)](https://github.com/emqx/emqx/releases)
[![Build Status](https://travis-ci.org/emqx/emqx.svg)](https://travis-ci.org/emqx/emqx)
[![Coverage Status](https://coveralls.io/repos/github/emqx/emqx/badge.svg)](https://coveralls.io/github/emqx/emqx)
[![Docker Pulls](https://img.shields.io/docker/pulls/emqx/emqx)](https://hub.docker.com/r/emqx/emqx)
[![Slack](https://img.shields.io/badge/Slack-EMQ%20X-brightgreen?logo=slack&style=flat&color=7E4798)](https://emqx.slack.com)
[![Twitter](https://img.shields.io/badge/Twiiter-EMQ%20X-1DA1F2?logo=twitter)](https://twitter.com/emqtt)
[English](./README.md) | 简体中文
*EMQ X* 是一款完全开源，高度可伸缩，高可用的分布式 MQTT 消息服务器，适用于 IoT、M2M 和移动应用程序，可处理千万级别的并发客户端。
从 3.0 版本开始，*EMQ X* 完整支持 MQTT V5.0 协议规范，向下兼容 MQTT V3.1 和 V3.1.1，并支持 MQTT-SN、CoAP、LwM2M、WebSocket 和 STOMP 等通信协议。EMQ X 3.0 单集群可支持千万级别的 MQTT 并发连接。
- 新功能的完整列表，请参阅 [EMQ X Release Notes](https://github.com/emqx/emqx/releases)。
- 获取更多信息，请访问 [EMQ X](https://emqx.io)。
## 安装
*EMQ X* 是跨平台的，支持 Linux、Unix、Mac OS 以及 Windows。这意味着 *EMQ X* 可以部署在 x86_64 架构的服务器上，也可以部署在 Raspberry Pi 这样的 ARM 设备上。
获取适合你的二进制软件包，[点此下载](https://emqx.io/downloads)。
- [单节点安装](https://developer.emqx.io/docs/emq/v3/en/install.html)
- [集群安装](https://developer.emqx.io/docs/emq/v3/en/cluster.html)
## 从源码构建
3.0 版本开始，构建 *EMQ X* 需要 Erlang/OTP R21+。
```
git clone https://github.com/emqx/emqx-rel.git
cd emqx-rel && make
cd _rel/emqx && ./bin/emqx console
```
## 快速入门
```
# Start emqx
./bin/emqx start
# Check Status
./bin/emqx_ctl status
# Stop emqx
./bin/emqx stop
```
*EMQ X* 启动，可以使用浏览器访问 http://localhost:18083 来查看 Dashboard。
## FAQ
访问 [FAQ](https://developer.emqx.io/docs/tutorial/zh/faq/faq.html) 以获取常见问题的帮助。
## 产品路线
通过 [EMQ X Roadmap uses Github milestones](https://github.com/emqx/emqx/milestones) 参与跟踪项目进度。
## 社区、讨论、贡献和支持
你可通过以下途径与 EMQ 社区及开发者联系:
- [EMQX Slack](http://emqx.slack.com)
- [Mailing Lists](<emqtt@googlegroups.com>)
- [Twitter](https://twitter.com/emqtt)
- [Forum](https://groups.google.com/d/forum/emqtt)
- [Blog](https://medium.com/@emqtt)
欢迎你将任何 bug、问题和功能请求提交到 [emqx/emqx](https://github.com/emqx/emqx/issues)。
## MQTT 规范
你可以通过以下链接了解与查阅 MQTT 协议:
[MQTT Version 3.1.1](https://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html)
[MQTT Version 5.0](https://docs.oasis-open.org/mqtt/mqtt/v5.0/cs02/mqtt-v5.0-cs02.html)
[MQTT SN](http://mqtt.org/new/wp-content/uploads/2009/06/MQTT-SN_spec_v1.2.pdf)
## 开源许可
Apache License 2.0, 详见 [LICENSE](./LICENSE)。

35
README.md
View File

@ -1,19 +1,26 @@
# EMQ X Broker
[![GitHub Release](https://img.shields.io/github/release/emqx/emqx?color=brightgreen)](https://github.com/emqx/emqx/releases)
[![Build Status](https://travis-ci.org/emqx/emqx.svg)](https://travis-ci.org/emqx/emqx)
[![Coverage Status](https://coveralls.io/repos/github/emqx/emqx/badge.svg)](https://coveralls.io/github/emqx/emqx)
[![Docker Pulls](https://img.shields.io/docker/pulls/emqx/emqx)](https://hub.docker.com/r/emqx/emqx)
[![Slack](https://img.shields.io/badge/Slack-EMQ%20X-brightgreen?logo=slack&style=flat&color=7E4798)](https://emqx.slack.com)
[![Twitter](https://img.shields.io/badge/Twiiter-EMQ%20X-1DA1F2?logo=twitter)](https://twitter.com/emqtt)
English | [简体中文](./README-CN.md)
*EMQ X* broker is a fully open source, highly scalable, highly available distributed MQTT messaging broker for IoT, M2M and Mobile applications that can handle tens of millions of concurrent clients.
Starting from 3.0 release, *EMQ X* broker fully supports MQTT V5.0 protocol specifications and backward compatible with MQTT V3.1 and V3.1.1, as well as other communication protocols such as MQTT-SN, CoAP, LwM2M, WebSocket and STOMP. The 3.0 release of the *EMQ X* broker can scaled to 10+ million concurrent MQTT connections on one cluster.
- For full list of new features, please read *EMQ X* broker 3.0 [release notes](https://github.com/emqx/emqx/releases/).
- For more information, please visit [EMQ X homepage](http://emqx.io).
- For full list of new features, please read [EMQ X Release Notes](https://github.com/emqx/emqx/releases).
- For more information, please visit [EMQ X homepage](https://emqx.io).
## Installation
The *EMQ X* broker is cross-platform, which can be deployed on Linux, Unix, Mac, Windows and even Raspberry Pi.
The *EMQ X* broker is cross-platform, which supports Linux, Unix, Mac OS and Windows. It means *EMQ X* can be deployed on x86_64 architecture servers and ARM devices like Raspberry Pi.
Download the binary package for your platform from [here](http://emqx.io/downloads).
Download the binary package for your platform from [here](https://emqx.io/downloads).
- [Single Node Install](https://developer.emqx.io/docs/emq/v3/en/install.html)
- [Multi Node Install](https://developer.emqx.io/docs/emq/v3/en/cluster.html)
@ -34,6 +41,7 @@ cd _build/emqx/rel/emqx && ./bin/emqx console
## Quick Start
```
# Start emqx
./bin/emqx start
@ -42,9 +50,13 @@ cd _build/emqx/rel/emqx && ./bin/emqx console
# Stop emqx
./bin/emqx stop
```
To view the dashboard after running, use your browser to open: http://localhost:18083
## FAQ
Visiting [FAQ](https://developer.emqx.io/docs/tutorial/en/faq/faq.html) to get help of common problems.
## Roadmap
@ -54,8 +66,6 @@ The [EMQ X Roadmap uses Github milestones](https://github.com/emqx/emqx/mileston
You can reach the EMQ community and developers via the following channels:
- [EMQX Slack](http://emqx.slack.com)
-[#emqx-users](https://emqx.slack.com/messages/CBUF2TTB8/)
-[#emqx-devs](https://emqx.slack.com/messages/CBSL57DUH/)
- [Mailing Lists](<emqtt@googlegroups.com>)
- [Twitter](https://twitter.com/emqtt)
- [Forum](https://groups.google.com/d/forum/emqtt)
@ -75,11 +85,4 @@ You can read the mqtt protocol via the following links:
## License
Copyright (c) 2013-2019 [EMQ Technologies Co., Ltd](http://emqx.io). All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License at
[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and limitations under the License.
Apache License 2.0, see [LICENSE](./LICENSE).

52
etc/emqx.conf
View File

@ -11,6 +11,16 @@
## Value: String
cluster.name = emqxcl
## Specify the erlang distributed protocol.
##
## Value: Enum
## - inet_tcp: the default; handles TCP streams with IPv4 addressing.
## - inet6_tcp: handles TCP with IPv6 addressing.
## - inet_tls: using TLS for Erlang Distribution.
##
## vm.args: -proto_dist inet_tcp
cluster.proto_dist = inet_tcp
## Cluster auto-discovery strategy.
##
## Value: Enum
@ -251,15 +261,7 @@ node.fullsweep_after = 1000
## Value: Log file
node.crash_dump = {{ platform_log_dir }}/crash.dump
## Specify the erlang distributed protocol.
##
## Value: Enum
## - inet_tcp: the default; handles TCP streams with IPv4 addressing.
## - inet6_tcp: handles TCP with IPv6 addressing.
## - inet_tls: using TLS for Erlang Distribution.
##
## vm.args: -proto_dist inet_tcp
node.proto_dist = inet_tcp
## Specify SSL Options in the file if using SSL for Erlang Distribution.
##
@ -292,6 +294,18 @@ node.dist_listen_max = 6369
##--------------------------------------------------------------------
## RPC
##--------------------------------------------------------------------
## RPC Mode.
##
## Value: sync | async
rpc.mode = async
## Max batch size of async RPC requests.
##
## Value: Integer
## Zero or negative value disables rpc batching.
##
## NOTE: RPC batch won't work when rpc.mode = sync
rpc.async_batch_size = 256
## TCP server port for RPC.
##
@ -303,6 +317,11 @@ rpc.tcp_server_port = 5369
## Value: Port [1024-65535]
rpc.tcp_client_port = 5369
## Number of utgoing RPC connections.
##
## Value: Interger [1-256]
rpc.tcp_client_num = 32
## RCP Client connect timeout.
##
## Value: Seconds
@ -338,6 +357,21 @@ rpc.socket_keepalive_interval = 75s
## Value: Integer
rpc.socket_keepalive_count = 9
## Size of TCP send buffer.
##
## Value: Bytes
rpc.socket_sndbuf = 1MB
## Size of TCP receive buffer.
##
## Value: Seconds
rpc.socket_recbuf = 1MB
## Size of user-level software socket buffer.
##
## Value: Seconds
rpc.socket_buffer = 1MB
##--------------------------------------------------------------------
## Log
##--------------------------------------------------------------------

26
include/emqx.hrl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,17 +12,20 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-ifndef(EMQ_X_HRL).
-define(EMQ_X_HRL, true).
%%--------------------------------------------------------------------
%% Banner
%% Common
%%--------------------------------------------------------------------
-define(COPYRIGHT, "Copyright (c) 2013-2019 EMQ Technologies Co., Ltd").
-define(Otherwise, true).
-define(LICENSE_MESSAGE, "Licensed under the Apache License, Version 2.0").
%%--------------------------------------------------------------------
%% Banner
%%--------------------------------------------------------------------
-define(PROTOCOL_VERSION, "MQTT/5.0").
@ -47,8 +51,6 @@
%% Message and Delivery
%%--------------------------------------------------------------------
-record(session, {sid, pid}).
-record(subscription, {topic, subid, subopts}).
%% See 'Application Message' in MQTT Version 5.0
@ -73,8 +75,7 @@
-record(delivery, {
sender :: pid(), %% Sender of the delivery
message :: #message{}, %% The message delivered
results :: list() %% Dispatches of the message
message :: #message{} %% The message delivered
}).
%%--------------------------------------------------------------------
@ -167,12 +168,3 @@
-endif.
%%--------------------------------------------------------------------
%% Compatible with Windows
%%--------------------------------------------------------------------
-define(compat_windows(Expression, Default),
case os:type() of
{win32, nt} -> Default;
_Unix -> Expression
end).

21
include/emqx_client.hrl
View File

@ -1,21 +0,0 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
-ifndef(EMQX_CLIENT_HRL).
-define(EMQX_CLIENT_HRL, true).
-include("emqx_mqtt.hrl").
-record(mqtt_msg, {qos = ?QOS_0, retain = false, dup = false,
packet_id, topic, props, payload}).
-endif.

10
include/emqx_mqtt.hrl
View File

@ -204,8 +204,7 @@
-define(DEFAULT_SUBOPTS, #{rh => 0, %% Retain Handling
rap => 0, %% Retain as Publish
nl => 0, %% No Local
qos => 0, %% QoS
rc => 0 %% Reason Code
qos => 0 %% QoS
}).
-record(mqtt_packet_connect, {
@ -351,6 +350,13 @@
variable = #mqtt_packet_publish{packet_id = PacketId}
}).
-define(PUBLISH_PACKET(QoS, Topic, PacketId),
#mqtt_packet{header = #mqtt_packet_header{type = ?PUBLISH,
qos = QoS},
variable = #mqtt_packet_publish{topic_name = Topic,
packet_id = PacketId}
}).
-define(PUBLISH_PACKET(QoS, Topic, PacketId, Payload),
#mqtt_packet{header = #mqtt_packet_header{type = ?PUBLISH,
qos = QoS},

5
include/logger.hrl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% debug | info | notice | warning | error | critical | alert | emergency
@ -43,3 +45,4 @@
begin
(logger:log(Level,#{},#{report_cb => fun(_) -> {'$logger_header'()++(Format), (Args)} end}))
end).

5
include/types.hrl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-type(maybe(T) :: undefined | T).
@ -19,3 +21,4 @@
-type(ok_or_error(Reason) :: ok | {error, Reason}).
-type(ok_or_error(Value, Reason) :: {ok, Value} | {error, Reason}).

66
priv/emqx.schema
View File

@ -85,6 +85,13 @@
{datatype, string}
]}.
%% @doc The erlang distributed protocol
{mapping, "cluster.proto_dist", "ekka.proto_dist", [
{default, "inet_tcp"},
{datatype, {enum, [inet_tcp, inet6_tcp, inet_tls]}},
hidden
]}.
{mapping, "cluster.dns.app", "ekka.cluster_discovery", [
{datatype, string}
]}.
@ -198,12 +205,7 @@ end}.
{default, "emqx@127.0.0.1"}
]}.
%% @doc The erlang distributed protocol
{mapping, "node.proto_dist", "vm_args.-proto_dist", [
%{default, "inet_tcp"},
{datatype, {enum, [inet_tcp, inet6_tcp, inet_tls]}},
hidden
]}.
%% @doc Specify SSL Options in the file if using SSL for erlang distribution
{mapping, "node.ssl_dist_optfile", "vm_args.-ssl_dist_optfile", [
@ -338,6 +340,17 @@ end}.
%% RPC
%%--------------------------------------------------------------------
%% RPC Mode.
{mapping, "rpc.mode", "emqx.rpc_mode", [
{default, async},
{datatype, {enum, [sync, async]}}
]}.
{mapping, "rpc.async_batch_size", "gen_rpc.max_batch_size", [
{default, 256},
{datatype, integer}
]}.
%% RPC server port.
{mapping, "rpc.tcp_server_port", "gen_rpc.tcp_server_port", [
{default, 5369},
@ -350,6 +363,13 @@ end}.
{datatype, integer}
]}.
%% Default TCP port for outgoing connections
{mapping, "rpc.tcp_client_num", "gen_rpc.tcp_client_num", [
{default, 32},
{datatype, integer},
{validators, ["range:gt_0_lt_256"]}
]}.
%% Client connect timeout
{mapping, "rpc.connect_timeout", "gen_rpc.connect_timeout", [
{default, "5s"},
@ -392,6 +412,28 @@ end}.
{datatype, integer}
]}.
%% Size of TCP send buffer
{mapping, "rpc.socket_sndbuf", "gen_rpc.socket_sndbuf", [
{default, "1MB"},
{datatype, bytesize}
]}.
%% Size of TCP receive buffer
{mapping, "rpc.socket_recbuf", "gen_rpc.socket_recbuf", [
{default, "1MB"},
{datatype, bytesize}
]}.
%% Size of TCP receive buffer
{mapping, "rpc.socket_buffer", "gen_rpc.socket_buffer", [
{default, "1MB"},
{datatype, bytesize}
]}.
{validator, "range:gt_0_lt_256", "must greater than 0 and less than 256",
fun(X) -> X > 0 andalso X < 256 end
}.
%%--------------------------------------------------------------------
%% Log
%%--------------------------------------------------------------------
@ -634,19 +676,19 @@ end}.
]}.
%% @doc Whether the server supports MQTT retained messages.
{mapping, "mqtt.retain_available", "emqx.mqtt_retain_available", [
{mapping, "mqtt.retain_available", "emqx.retain_available", [
{default, true},
{datatype, {enum, [true, false]}}
]}.
%% @doc Whether the Server supports MQTT Wildcard Subscriptions.
{mapping, "mqtt.wildcard_subscription", "emqx.mqtt_wildcard_subscription", [
{mapping, "mqtt.wildcard_subscription", "emqx.wildcard_subscription", [
{default, true},
{datatype, {enum, [true, false]}}
]}.
%% @doc Whether the Server supports MQTT Shared Subscriptions.
{mapping, "mqtt.shared_subscription", "emqx.mqtt_shared_subscription", [
{mapping, "mqtt.shared_subscription", "emqx.shared_subscription", [
{default, true},
{datatype, {enum, [true, false]}}
]}.
@ -876,7 +918,7 @@ end}.
{translation, "emqx.zones", fun(Conf) ->
Mapping = fun("retain_available", Val) ->
{mqtt_retain_available, Val};
{retain_available, Val};
("flapping_threshold", Val) ->
[Limit, Duration] = string:tokens(Val, ", "),
FlappingThreshold = case cuttlefish_duration:parse(Duration, s) of
@ -887,9 +929,9 @@ end}.
end,
{flapping_threshold, FlappingThreshold};
("wildcard_subscription", Val) ->
{mqtt_wildcard_subscription, Val};
{wildcard_subscription, Val};
("shared_subscription", Val) ->
{mqtt_shared_subscription, Val};
{shared_subscription, Val};
("publish_limit", Val) ->
[Limit, Duration] = string:tokens(Val, ", "),
PubLimit = case cuttlefish_duration:parse(Duration, s) of

14
rebar.config
View File

@ -2,10 +2,9 @@
[{jsx, "2.9.0"}, % hex
{cowboy, "2.6.1"}, % hex
{gproc, "0.8.0"}, % hex
{replayq, "0.1.1"}, %hex
{esockd, "5.5.0"}, %hex
{ekka, {git, "https://github.com/emqx/ekka", {tag, "v0.5.8"}}},
{gen_rpc, {git, "https://github.com/emqx/gen_rpc", {tag, "2.4.0"}}},
{ekka, "0.6.0"}, %hex
{gen_rpc, {git, "https://github.com/emqx/gen_rpc", {tag, "2.4.1"}}},
{cuttlefish, {git, "https://github.com/emqx/cuttlefish", {tag, "v3.0.0"}}}
]}.
@ -22,16 +21,17 @@
{cover_opts, [verbose]}.
{cover_export_enabled, true}.
{plugins, [coveralls]}.
{plugins, [coveralls,
rebar3_proper]}.
{erl_first_files, ["src/emqx_logger.erl"]}.
{profiles,
[{test,
[{deps,
[{meck, "0.8.13"}, % hex
{bbmustache, "1.7.0"}, % hex
{emqx_ct_helpers, "1.1.3"} % hex
[{bbmustache, "1.7.0"}, % hex
{emqtt, {git, "https://github.com/emqx/emqtt", {tag, "v1.0.1"}}},
{emqx_ct_helpers, {git, "https://github.com/emqx/emqx-ct-helpers", {branch, "develop"}}}
]}
]}
]}.

4
test/run_emqx.escript → scripts/run_emqx.escript
View File

@ -4,6 +4,10 @@ main(_) ->
start().
start() ->
ok = application:load(mnesia),
MnesiaName = lists:concat(["Mnesia.", atom_to_list(node())]),
MnesiaDir = filename:join(["_build", "data", MnesiaName]),
ok = application:set_env(mnesia, dir, MnesiaDir),
SpecEmqxConfig = fun(_) -> ok end,
start(SpecEmqxConfig).

12
src/emqx.app.src
View File

@ -1,12 +1,14 @@
{application,emqx,
[{description,"EMQ X Broker"},
{application, emqx, [
{id, "emqx"},
{vsn, "git"},
{description, "EMQ X Broker"},
{modules, []},
{registered,[emqx_sup]},
{registered, []},
{applications, [kernel,stdlib,jsx,gproc,gen_rpc,esockd,cowboy,
replayq,sasl,os_mon]},
sasl,os_mon]},
{env, []},
{mod, {emqx_app,[]}},
{maintainers, ["Feng Lee <feng@emqx.io>"]},
{licenses, ["Apache-2.0"]},
{links,[{"Github","https://github.com/emqx/emqx"}]}]}.
{links, [{"Github", "https://github.com/emqx/emqx"}]}
]}.

35
src/emqx.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx).
@ -61,9 +63,13 @@
-define(APP, ?MODULE).
%%------------------------------------------------------------------------------
-define(COPYRIGHT, "Copyright (c) 2019 EMQ Technologies Co., Ltd").
-define(LICENSE_MESSAGE, "Licensed under the Apache License, Version 2.0").
%%--------------------------------------------------------------------
%% Bootstrap, is_running...
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Start emqx application
-spec(start() -> {ok, list(atom())} | {error, term()}).
@ -95,9 +101,9 @@ is_running(Node) ->
Pid when is_pid(Pid) -> true
end.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% PubSub API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(subscribe(emqx_topic:topic() | string()) -> ok).
subscribe(Topic) ->
@ -114,7 +120,7 @@ subscribe(Topic, SubOpts) when is_map(SubOpts) ->
subscribe(Topic, SubId, SubOpts) when (is_atom(SubId) orelse is_binary(SubId)), is_map(SubOpts) ->
emqx_broker:subscribe(iolist_to_binary(Topic), SubId, SubOpts).
-spec(publish(emqx_types:message()) -> emqx_types:deliver_results()).
-spec(publish(emqx_types:message()) -> emqx_types:publish_result()).
publish(Msg) ->
emqx_broker:publish(Msg).
@ -122,9 +128,9 @@ publish(Msg) ->
unsubscribe(Topic) ->
emqx_broker:unsubscribe(iolist_to_binary(Topic)).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% PubSub management API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(topics() -> list(emqx_topic:topic())).
topics() -> emqx_router:topics().
@ -143,9 +149,9 @@ subscribed(SubPid, Topic) when is_pid(SubPid) ->
subscribed(SubId, Topic) when is_atom(SubId); is_binary(SubId) ->
emqx_broker:subscribed(SubId, iolist_to_binary(Topic)).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Hooks API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(hook(emqx_hooks:hookpoint(), emqx_hooks:action()) -> ok | {error, already_exists}).
hook(HookPoint, Action) ->
@ -177,9 +183,9 @@ run_hook(HookPoint, Args) ->
run_fold_hook(HookPoint, Args, Acc) ->
emqx_hooks:run_fold(HookPoint, Args, Acc).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Shutdown and reboot
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
shutdown() ->
shutdown(normal).
@ -193,12 +199,13 @@ shutdown(Reason) ->
reboot() ->
lists:foreach(fun application:start/1, [gproc, esockd, ranch, cowboy, ekka, emqx]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
reload_config(ConfFile) ->
{ok, [Conf]} = file:consult(ConfFile),
lists:foreach(fun({App, Vals}) ->
[application:set_env(App, Par, Val) || {Par, Val} <- Vals]
end, Conf).

78
src/emqx_access_control.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_access_control).
@ -22,55 +24,61 @@
, reload_acl/0
]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
-spec(authenticate(emqx_types:credentials())
-> {ok, emqx_types:credentials()} | {error, term()}).
authenticate(Credentials) ->
case emqx_hooks:run_fold('client.authenticate', [], init_auth_result(Credentials)) of
#{auth_result := success, anonymous := true} = NewCredentials ->
%%--------------------------------------------------------------------
-spec(authenticate(emqx_types:client())
-> {ok, #{auth_result := emqx_types:auth_result(),
anonymous := boolean}} | {error, term()}).
authenticate(Client) ->
case emqx_hooks:run_fold('client.authenticate',
[Client], default_auth_result(maps:get(zone, Client, undefined))) of
Result = #{auth_result := success, anonymous := true} ->
emqx_metrics:inc('auth.mqtt.anonymous'),
{ok, NewCredentials};
#{auth_result := success} = NewCredentials ->
{ok, NewCredentials};
NewCredentials ->
{error, maps:get(auth_result, NewCredentials, unknown_error)}
{ok, Result};
Result = #{auth_result := success} ->
{ok, Result};
Result ->
{error, maps:get(auth_result, Result, unknown_error)}
end.
%% @doc Check ACL
-spec(check_acl(emqx_types:credentials(), emqx_types:pubsub(), emqx_types:topic()) -> allow | deny).
check_acl(Credentials, PubSub, Topic) ->
-spec(check_acl(emqx_types:cient(), emqx_types:pubsub(), emqx_types:topic())
-> allow | deny).
check_acl(Client, PubSub, Topic) ->
case emqx_acl_cache:is_enabled() of
false ->
do_check_acl(Credentials, PubSub, Topic);
true ->
case emqx_acl_cache:get_acl_cache(PubSub, Topic) of
not_found ->
AclResult = do_check_acl(Credentials, PubSub, Topic),
emqx_acl_cache:put_acl_cache(PubSub, Topic, AclResult),
AclResult;
AclResult ->
AclResult
end
check_acl_cache(Client, PubSub, Topic);
false ->
do_check_acl(Client, PubSub, Topic)
end.
do_check_acl(#{zone := Zone} = Credentials, PubSub, Topic) ->
case emqx_hooks:run_fold('client.check_acl', [Credentials, PubSub, Topic],
emqx_zone:get_env(Zone, acl_nomatch, deny)) of
check_acl_cache(Client, PubSub, Topic) ->
case emqx_acl_cache:get_acl_cache(PubSub, Topic) of
not_found ->
AclResult = do_check_acl(Client, PubSub, Topic),
emqx_acl_cache:put_acl_cache(PubSub, Topic, AclResult),
AclResult;
AclResult -> AclResult
end.
do_check_acl(#{zone := Zone} = Client, PubSub, Topic) ->
Default = emqx_zone:get_env(Zone, acl_nomatch, deny),
case emqx_hooks:run_fold('client.check_acl', [Client, PubSub, Topic], Default) of
allow -> allow;
_ -> deny
_Other -> deny
end.
-spec(reload_acl() -> ok | {error, term()}).
reload_acl() ->
emqx_acl_cache:is_enabled() andalso
emqx_acl_cache:empty_acl_cache(),
emqx_acl_cache:is_enabled()
andalso emqx_acl_cache:empty_acl_cache(),
emqx_mod_acl_internal:reload_acl().
init_auth_result(Credentials) ->
case emqx_zone:get_env(maps:get(zone, Credentials, undefined), allow_anonymous, false) of
true -> Credentials#{auth_result => success, anonymous => true};
false -> Credentials#{auth_result => not_authorized, anonymous => false}
default_auth_result(Zone) ->
case emqx_zone:get_env(Zone, allow_anonymous, false) of
true -> #{auth_result => success, anonymous => true};
false -> #{auth_result => not_authorized, anonymous => false}
end.

76
src/emqx_access_rule.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_access_rule).
@ -21,6 +23,8 @@
, compile/1
]).
-export_type([rule/0]).
-type(acl_result() :: allow | deny).
-type(who() :: all | binary() |
@ -33,15 +37,9 @@
-type(rule() :: {acl_result(), all} |
{acl_result(), who(), access(), list(emqx_topic:topic())}).
-export_type([rule/0]).
-define(ALLOW_DENY(A), ((A =:= allow) orelse (A =:= deny))).
-define(PUBSUB(A), ((A =:= subscribe) orelse (A =:= publish) orelse (A =:= pubsub))).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%% @doc Compile Access Rule.
compile({A, all}) when ?ALLOW_DENY(A) ->
{A, all};
@ -88,23 +86,23 @@ bin(B) when is_binary(B) ->
B.
%% @doc Match access rule
-spec(match(emqx_types:credentials(), emqx_types:topic(), rule())
-spec(match(emqx_types:client(), emqx_types:topic(), rule())
-> {matched, allow} | {matched, deny} | nomatch).
match(_Credentials, _Topic, {AllowDeny, all}) when ?ALLOW_DENY(AllowDeny) ->
match(_Client, _Topic, {AllowDeny, all}) when ?ALLOW_DENY(AllowDeny) ->
{matched, AllowDeny};
match(Credentials, Topic, {AllowDeny, Who, _PubSub, TopicFilters})
match(Client, Topic, {AllowDeny, Who, _PubSub, TopicFilters})
when ?ALLOW_DENY(AllowDeny) ->
case match_who(Credentials, Who)
andalso match_topics(Credentials, Topic, TopicFilters) of
case match_who(Client, Who)
andalso match_topics(Client, Topic, TopicFilters) of
true -> {matched, AllowDeny};
false -> nomatch
end.
match_who(_Credentials, all) ->
match_who(_Client, all) ->
true;
match_who(_Credentials, {user, all}) ->
match_who(_Client, {user, all}) ->
true;
match_who(_Credentials, {client, all}) ->
match_who(_Client, {client, all}) ->
true;
match_who(#{client_id := ClientId}, {client, ClientId}) ->
true;
@ -114,44 +112,44 @@ match_who(#{peername := undefined}, {ipaddr, _Tup}) ->
false;
match_who(#{peername := {IP, _}}, {ipaddr, CIDR}) ->
esockd_cidr:match(IP, CIDR);
match_who(Credentials, {'and', Conds}) when is_list(Conds) ->
match_who(Client, {'and', Conds}) when is_list(Conds) ->
lists:foldl(fun(Who, Allow) ->
match_who(Credentials, Who) andalso Allow
match_who(Client, Who) andalso Allow
end, true, Conds);
match_who(Credentials, {'or', Conds}) when is_list(Conds) ->
match_who(Client, {'or', Conds}) when is_list(Conds) ->
lists:foldl(fun(Who, Allow) ->
match_who(Credentials, Who) orelse Allow
match_who(Client, Who) orelse Allow
end, false, Conds);
match_who(_Credentials, _Who) ->
match_who(_Client, _Who) ->
false.
match_topics(_Credentials, _Topic, []) ->
match_topics(_Client, _Topic, []) ->
false;
match_topics(Credentials, Topic, [{pattern, PatternFilter}|Filters]) ->
TopicFilter = feed_var(Credentials, PatternFilter),
match_topics(Client, Topic, [{pattern, PatternFilter}|Filters]) ->
TopicFilter = feed_var(Client, PatternFilter),
match_topic(emqx_topic:words(Topic), TopicFilter)
orelse match_topics(Credentials, Topic, Filters);
match_topics(Credentials, Topic, [TopicFilter|Filters]) ->
orelse match_topics(Client, Topic, Filters);
match_topics(Client, Topic, [TopicFilter|Filters]) ->
match_topic(emqx_topic:words(Topic), TopicFilter)
orelse match_topics(Credentials, Topic, Filters).
orelse match_topics(Client, Topic, Filters).
match_topic(Topic, {eq, TopicFilter}) ->
Topic == TopicFilter;
match_topic(Topic, TopicFilter) ->
emqx_topic:match(Topic, TopicFilter).
feed_var(Credentials, Pattern) ->
feed_var(Credentials, Pattern, []).
feed_var(_Credentials, [], Acc) ->
feed_var(Client, Pattern) ->
feed_var(Client, Pattern, []).
feed_var(_Client, [], Acc) ->
lists:reverse(Acc);
feed_var(Credentials = #{client_id := undefined}, [<<"%c">>|Words], Acc) ->
feed_var(Credentials, Words, [<<"%c">>|Acc]);
feed_var(Credentials = #{client_id := ClientId}, [<<"%c">>|Words], Acc) ->
feed_var(Credentials, Words, [ClientId |Acc]);
feed_var(Credentials = #{username := undefined}, [<<"%u">>|Words], Acc) ->
feed_var(Credentials, Words, [<<"%u">>|Acc]);
feed_var(Credentials = #{username := Username}, [<<"%u">>|Words], Acc) ->
feed_var(Credentials, Words, [Username|Acc]);
feed_var(Credentials, [W|Words], Acc) ->
feed_var(Credentials, Words, [W|Acc]).
feed_var(Client = #{client_id := undefined}, [<<"%c">>|Words], Acc) ->
feed_var(Client, Words, [<<"%c">>|Acc]);
feed_var(Client = #{client_id := ClientId}, [<<"%c">>|Words], Acc) ->
feed_var(Client, Words, [ClientId |Acc]);
feed_var(Client = #{username := undefined}, [<<"%u">>|Words], Acc) ->
feed_var(Client, Words, [<<"%u">>|Acc]);
feed_var(Client = #{username := Username}, [<<"%u">>|Words], Acc) ->
feed_var(Client, Words, [Username|Acc]);
feed_var(Client, [W|Words], Acc) ->
feed_var(Client, Words, [W|Acc]).

5
src/emqx_acl_cache.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_acl_cache).
@ -124,6 +126,7 @@ map_acl_cache(Fun) ->
%%--------------------------------------------------------------------
%% Internal functions
%%--------------------------------------------------------------------
add_acl(PubSub, Topic, AclResult) ->
K = cache_k(PubSub, Topic),
V = cache_v(AclResult),

24
src/emqx_alarm_handler.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_alarm_handler).
@ -47,9 +49,9 @@
-define(ALARM_TAB, emqx_alarm).
-define(ALARM_HISTORY_TAB, emqx_alarm_history).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Mnesia bootstrap
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
mnesia(boot) ->
ok = ekka_mnesia:create_table(?ALARM_TAB, [
@ -64,13 +66,14 @@ mnesia(boot) ->
{local_content, true},
{record_name, alarm_history},
{attributes, record_info(fields, alarm_history)}]);
mnesia(copy) ->
ok = ekka_mnesia:copy_table(?ALARM_TAB),
ok = ekka_mnesia:copy_table(?ALARM_HISTORY_TAB).
%%----------------------------------------------------------------------
%%--------------------------------------------------------------------
%% API
%%----------------------------------------------------------------------
%%--------------------------------------------------------------------
load() ->
gen_event:swap_handler(alarm_handler, {alarm_handler, swap}, {?MODULE, []}).
@ -89,13 +92,14 @@ get_alarms(history) ->
Alarms = ets:tab2list(?ALARM_HISTORY_TAB),
[{Id, Desc, ClearAt} || #alarm_history{id = Id, desc = Desc, clear_at = ClearAt} <- Alarms].
%%----------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_event callbacks
%%----------------------------------------------------------------------
%%--------------------------------------------------------------------
init({_Args, {alarm_handler, ExistingAlarms}}) ->
init_tables(ExistingAlarms),
{ok, []};
init(_) ->
init_tables([]),
{ok, []}.
@ -188,7 +192,7 @@ clear_alarm_(Id) ->
end.
set_alarm_history(Id, Desc) ->
mnesia:dirty_write(?ALARM_HISTORY_TAB, #alarm_history{id = Id,
His = #alarm_history{id = Id,
desc = Desc,
clear_at = os:timestamp()}).
clear_at = os:timestamp()},
mnesia:dirty_write(?ALARM_HISTORY_TAB, His).

10
src/emqx_app.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_app).
@ -30,10 +32,10 @@ start(_Type, _Args) ->
print_banner(),
ekka:start(),
{ok, Sup} = emqx_sup:start_link(),
emqx_modules:load(),
emqx_plugins:init(),
ok = emqx_modules:load(),
ok = emqx_plugins:init(),
emqx_plugins:load(),
emqx_listeners:start(),
ok = emqx_listeners:start(),
start_autocluster(),
register(emqx, self()),

56
src/emqx_banned.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_banned).
@ -30,9 +32,10 @@
-export([start_link/0]).
-export([ add/1
-export([ check/1
, add/1
, delete/1
, check/1
, info/1
]).
%% gen_server callbacks
@ -44,14 +47,14 @@
, code_change/3
]).
-define(TAB, ?MODULE).
-define(BANNED_TAB, ?MODULE).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Mnesia bootstrap
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
mnesia(boot) ->
ok = ekka_mnesia:create_table(?TAB, [
ok = ekka_mnesia:create_table(?BANNED_TAB, [
{type, set},
{disc_copies, [node()]},
{record_name, banned},
@ -59,32 +62,35 @@ mnesia(boot) ->
{storage_properties, [{ets, [{read_concurrency, true}]}]}]);
mnesia(copy) ->
ok = ekka_mnesia:copy_table(?TAB).
ok = ekka_mnesia:copy_table(?BANNED_TAB).
%% @doc Start the banned server.
-spec(start_link() -> startlink_ret()).
start_link() ->
gen_server:start_link({local, ?MODULE}, ?MODULE, [], []).
-spec(check(emqx_types:credentials()) -> boolean()).
-spec(check(emqx_types:client()) -> boolean()).
check(#{client_id := ClientId, username := Username, peername := {IPAddr, _}}) ->
ets:member(?TAB, {client_id, ClientId})
orelse ets:member(?TAB, {username, Username})
orelse ets:member(?TAB, {ipaddr, IPAddr}).
ets:member(?BANNED_TAB, {client_id, ClientId})
orelse ets:member(?BANNED_TAB, {username, Username})
orelse ets:member(?BANNED_TAB, {ipaddr, IPAddr}).
-spec(add(emqx_types:banned()) -> ok).
add(Banned) when is_record(Banned, banned) ->
mnesia:dirty_write(?TAB, Banned).
mnesia:dirty_write(?BANNED_TAB, Banned).
-spec(delete({client_id, emqx_types:client_id()}
| {username, emqx_types:username()}
| {peername, emqx_types:peername()}) -> ok).
-spec(delete({client_id, emqx_types:client_id()} |
{username, emqx_types:username()} |
{peername, emqx_types:peername()}) -> ok).
delete(Key) ->
mnesia:dirty_delete(?TAB, Key).
mnesia:dirty_delete(?BANNED_TAB, Key).
%%------------------------------------------------------------------------------
info(InfoKey) ->
mnesia:table_info(?BANNED_TAB, InfoKey).
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
{ok, ensure_expiry_timer(#{expiry_timer => undefined})}.
@ -98,7 +104,8 @@ handle_cast(Msg, State) ->
{noreply, State}.
handle_info({timeout, TRef, expire}, State = #{expiry_timer := TRef}) ->
mnesia:async_dirty(fun expire_banned_items/1, [erlang:system_time(second)]),
mnesia:async_dirty(fun expire_banned_items/1,
[erlang:system_time(second)]),
{noreply, ensure_expiry_timer(State), hibernate};
handle_info(Info, State) ->
@ -111,9 +118,9 @@ terminate(_Reason, #{expiry_timer := TRef}) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-ifdef(TEST).
ensure_expiry_timer(State) ->
@ -126,6 +133,7 @@ ensure_expiry_timer(State) ->
expire_banned_items(Now) ->
mnesia:foldl(
fun(B = #banned{until = Until}, _Acc) when Until < Now ->
mnesia:delete_object(?TAB, B, sticky_write);
mnesia:delete_object(?BANNED_TAB, B, sticky_write);
(_, _Acc) -> ok
end, ok, ?TAB).
end, ok, ?BANNED_TAB).

25
src/emqx_base62.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,26 +12,25 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_base62).
%% APIs
-export([ encode/1
, encode/2
, decode/1
, decode/2
]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Encode any data to base62 binary
-spec encode(string() | integer() | binary()) -> binary().
encode(I) when is_integer(I) ->
encode(integer_to_binary(I));
encode(S) when is_list(S)->
encode(list_to_binary(S));
encode(unicode:characters_to_binary(S));
encode(B) when is_binary(B) ->
encode(B, <<>>).
@ -38,17 +38,13 @@ encode(B) when is_binary(B) ->
%% binary_to_list(encode(D)).
%% @doc Decode base62 binary to origin data binary
decode(L) when is_list(L) ->
decode(list_to_binary(L));
decode(B) when is_binary(B) ->
decode(B, <<>>).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Interval Functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
encode(D, string) ->
binary_to_list(encode(D));
encode(<<Index1:6, Index2:6, Index3:6, Index4:6, Rest/binary>>, Acc) ->
CharList = [encode_char(Index1), encode_char(Index2), encode_char(Index3), encode_char(Index4)],
NewAcc = <<Acc/binary,(iolist_to_binary(CharList))/binary>>,
@ -64,10 +60,6 @@ encode(<<Index1:6, Index2:2>>, Acc) ->
encode(<<>>, Acc) ->
Acc.
decode(D, integer) ->
binary_to_integer(decode(D));
decode(D, string) ->
binary_to_list(decode(D));
decode(<<Head:8, Rest/binary>>, Acc)
when bit_size(Rest) >= 8->
case Head == $9 of
@ -112,4 +104,3 @@ decode_char(I) when I >= $A andalso I =< $Z->
decode_char(9, I) ->
I + 61 - $A.

45
src/emqx_batch.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_batch).
@ -22,29 +24,27 @@
, items/1
]).
-record(batch,
{ batch_size :: non_neg_integer()
, batch_q :: list(any())
, linger_ms :: pos_integer()
, linger_timer :: reference() | undefined
, commit_fun :: function()
-export_type([options/0, batch/0]).
-record(batch, {
batch_size :: non_neg_integer(),
batch_q :: list(any()),
linger_ms :: pos_integer(),
linger_timer :: reference() | undefined,
commit_fun :: function()
}).
-type(options() ::
#{ batch_size => non_neg_integer()
, linger_ms => pos_integer()
, commit_fun := function()
-type(options() :: #{
batch_size => non_neg_integer(),
linger_ms => pos_integer(),
commit_fun := function()
}).
-opaque(batch() :: #batch{}).
-export_type([options/0]).
-export_type([batch/0]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(init(options()) -> batch()).
init(Opts) when is_map(Opts) ->
@ -54,14 +54,19 @@ init(Opts) when is_map(Opts) ->
commit_fun = maps:get(commit_fun, Opts)}.
-spec(push(any(), batch()) -> batch()).
push(El, Batch = #batch{batch_q = Q, linger_ms = Ms, linger_timer = undefined}) when length(Q) == 0 ->
Batch#batch{batch_q = [El], linger_timer = erlang:send_after(Ms, self(), batch_linger_expired)};
push(El, Batch = #batch{batch_q = Q,
linger_ms = Ms,
linger_timer = undefined})
when length(Q) == 0 ->
TRef = erlang:send_after(Ms, self(), batch_linger_expired),
Batch#batch{batch_q = [El], linger_timer = TRef};
%% no limit.
push(El, Batch = #batch{batch_size = 0, batch_q = Q}) ->
Batch#batch{batch_q = [El|Q]};
push(El, Batch = #batch{batch_size = MaxSize, batch_q = Q}) when length(Q) >= MaxSize ->
push(El, Batch = #batch{batch_size = MaxSize, batch_q = Q})
when length(Q) >= MaxSize ->
commit(Batch#batch{batch_q = [El|Q]});
push(El, Batch = #batch{batch_q = Q}) ->

98
src/emqx_broker.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_broker).
@ -191,7 +193,7 @@ do_unsubscribe(Group, Topic, SubPid, _SubOpts) ->
%% Publish
%%------------------------------------------------------------------------------
-spec(publish(emqx_types:message()) -> emqx_types:deliver_results()).
-spec(publish(emqx_types:message()) -> emqx_types:publish_result()).
publish(Msg) when is_record(Msg, message) ->
_ = emqx_tracer:trace(publish, Msg),
Headers = Msg#message.headers,
@ -200,8 +202,7 @@ publish(Msg) when is_record(Msg, message) ->
?LOG(notice, "Publishing interrupted: ~s", [emqx_message:format(Msg)]),
[];
#message{topic = Topic} = Msg1 ->
Delivery = route(aggre(emqx_router:match_routes(Topic)), delivery(Msg1)),
Delivery#delivery.results
route(aggre(emqx_router:match_routes(Topic)), delivery(Msg1))
end.
%% Called internally
@ -217,27 +218,27 @@ safe_publish(Msg) when is_record(Msg, message) ->
end.
delivery(Msg) ->
#delivery{sender = self(), message = Msg, results = []}.
#delivery{sender = self(), message = Msg}.
%%------------------------------------------------------------------------------
%% Route
%%------------------------------------------------------------------------------
route([], Delivery = #delivery{message = Msg}) ->
-spec(route([emqx_types:route_entry()], emqx_types:delivery()) -> emqx_types:publish_result()).
route([], #delivery{message = Msg}) ->
emqx_hooks:run('message.dropped', [#{node => node()}, Msg]),
inc_dropped_cnt(Msg#message.topic), Delivery;
route([{To, Node}], Delivery) when Node =:= node() ->
dispatch(To, Delivery);
route([{To, Node}], Delivery = #delivery{results = Results}) when is_atom(Node) ->
forward(Node, To, Delivery#delivery{results = [{route, Node, To}|Results]});
route([{To, Group}], Delivery) when is_tuple(Group); is_binary(Group) ->
emqx_shared_sub:dispatch(Group, To, Delivery);
inc_dropped_cnt(Msg#message.topic),
[];
route(Routes, Delivery) ->
lists:foldl(fun(Route, Acc) -> route([Route], Acc) end, Delivery, Routes).
lists:foldl(fun(Route, Acc) ->
[do_route(Route, Delivery) | Acc]
end, [], Routes).
do_route({To, Node}, Delivery) when Node =:= node() ->
{Node, To, dispatch(To, Delivery)};
do_route({To, Node}, Delivery) when is_atom(Node) ->
{Node, To, forward(Node, To, Delivery, emqx_config:get_env(rpc_mode, async))};
do_route({To, Group}, Delivery) when is_tuple(Group); is_binary(Group) ->
{share, To, emqx_shared_sub:dispatch(Group, To, Delivery)}.
aggre([]) ->
[];
@ -254,45 +255,58 @@ aggre(Routes) ->
end, [], Routes).
%% @doc Forward message to another node.
forward(Node, To, Delivery) ->
%% rpc:call to ensure the delivery, but the latency:(
-spec(forward(node(), emqx_types:topic(), emqx_types:delivery(), RPCMode::sync|async)
-> emqx_types:deliver_result()).
forward(Node, To, Delivery, async) ->
case emqx_rpc:cast(Node, ?BROKER, dispatch, [To, Delivery]) of
true -> ok;
{badrpc, Reason} ->
?LOG(error, "Ansync forward msg to ~s failed: ~p", [Node, Reason]),
{error, badrpc}
end;
forward(Node, To, Delivery, sync) ->
case emqx_rpc:call(Node, ?BROKER, dispatch, [To, Delivery]) of
{badrpc, Reason} ->
?LOG(error, "Failed to forward msg to ~s: ~p", [Node, Reason]),
Delivery;
Delivery1 -> Delivery1
?LOG(error, "Sync forward msg to ~s failed: ~p", [Node, Reason]),
{error, badrpc};
Result -> Result
end.
-spec(dispatch(emqx_topic:topic(), emqx_types:delivery()) -> emqx_types:delivery()).
dispatch(Topic, Delivery = #delivery{message = Msg, results = Results}) ->
-spec(dispatch(emqx_topic:topic(), emqx_types:delivery()) -> emqx_types:deliver_result()).
dispatch(Topic, #delivery{message = Msg}) ->
case subscribers(Topic) of
[] ->
emqx_hooks:run('message.dropped', [#{node => node()}, Msg]),
inc_dropped_cnt(Topic),
Delivery;
{error, no_subscribers};
[Sub] -> %% optimize?
Cnt = dispatch(Sub, Topic, Msg),
Delivery#delivery{results = [{dispatch, Topic, Cnt}|Results]};
dispatch(Sub, Topic, Msg);
Subs ->
Cnt = lists:foldl(
fun(Sub, Acc) ->
dispatch(Sub, Topic, Msg) + Acc
end, 0, Subs),
Delivery#delivery{results = [{dispatch, Topic, Cnt}|Results]}
lists:foldl(
fun(Sub, Res) ->
case dispatch(Sub, Topic, Msg) of
ok -> Res;
Err -> Err
end
end, ok, Subs)
end.
dispatch(SubPid, Topic, Msg) when is_pid(SubPid) ->
case erlang:is_process_alive(SubPid) of
true ->
SubPid ! {dispatch, Topic, Msg},
1;
false -> 0
SubPid ! {deliver, Topic, Msg},
ok;
false -> {error, subscriber_die}
end;
dispatch({shard, I}, Topic, Msg) ->
lists:foldl(
fun(SubPid, Cnt) ->
dispatch(SubPid, Topic, Msg) + Cnt
end, 0, subscribers({shard, Topic, I})).
fun(SubPid, Res) ->
case dispatch(SubPid, Topic, Msg) of
ok -> Res;
Err -> Err
end
end, ok, subscribers({shard, Topic, I})).
inc_dropped_cnt(<<"$SYS/", _/binary>>) ->
ok;
@ -374,9 +388,9 @@ set_subopts(Topic, NewOpts) when is_binary(Topic), is_map(NewOpts) ->
topics() ->
emqx_router:topics().
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Stats fun
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
stats_fun() ->
safe_update_stats(?SUBSCRIBER, 'subscribers.count', 'subscribers.max'),

12
src/emqx_broker_helper.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_broker_helper).
@ -92,9 +94,9 @@ create_seq(Topic) ->
reclaim_seq(Topic) ->
emqx_sequence:reclaim(?SUBSEQ, Topic).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
%% Helper table
@ -142,9 +144,9 @@ terminate(_Reason, _State) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
clean_down(SubPid) ->
case ets:lookup(?SUBMON, SubPid) of

8
src/emqx_broker_sup.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_broker_sup).
@ -23,9 +25,9 @@
start_link() ->
supervisor:start_link({local, ?MODULE}, ?MODULE, []).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Supervisor callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
%% Broker pool

527
src/emqx_channel.erl
View File

@ -14,6 +14,7 @@
%% limitations under the License.
%%--------------------------------------------------------------------
%% MQTT TCP/SSL Channel
-module(emqx_channel).
-behaviour(gen_statem).
@ -21,6 +22,7 @@
-include("emqx.hrl").
-include("emqx_mqtt.hrl").
-include("logger.hrl").
-include("types.hrl").
-logger_header("[Channel]").
@ -32,15 +34,16 @@
, stats/1
]).
-export([kick/1]).
%% for Debug
-export([state/1]).
-export([session/1]).
%% gen_statem callbacks
%% state callbacks
-export([ idle/3
, connected/3
, disconnected/3
]).
%% gen_statem callbacks
-export([ init/1
, callback_mode/0
, code_change/4
@ -48,28 +51,35 @@
]).
-record(state, {
transport,
socket,
peername,
sockname,
conn_state,
active_n,
proto_state,
parse_state,
gc_state,
keepalive,
rate_limit,
pub_limit,
limit_timer,
enable_stats,
stats_timer,
idle_timeout
transport :: esockd:transport(),
socket :: esockd:socket(),
peername :: emqx_types:peername(),
sockname :: emqx_types:peername(),
conn_state :: running | blocked,
active_n :: pos_integer(),
rate_limit :: maybe(esockd_rate_limit:bucket()),
pub_limit :: maybe(esockd_rate_limit:bucket()),
limit_timer :: maybe(reference()),
serialize :: fun((emqx_types:packet()) -> iodata()),
parse_state :: emqx_frame:parse_state(),
proto_state :: emqx_protocol:proto_state(),
gc_state :: emqx_gc:gc_state(),
keepalive :: maybe(emqx_keepalive:keepalive()),
stats_timer :: disabled | maybe(reference()),
idle_timeout :: timeout(),
connected :: boolean(),
connected_at :: erlang:timestamp()
}).
-type(state() :: #state{}).
-define(ACTIVE_N, 100).
-define(HANDLE(T, C, D), handle((T), (C), (D))).
-define(CHAN_STATS, [recv_pkt, recv_msg, send_pkt, send_msg]).
-define(SOCK_STATS, [recv_oct, recv_cnt, send_oct, send_cnt, send_pend]).
-spec(start_link(esockd:transport(), esockd:socket(), proplists:proplist())
-> {ok, pid()}).
start_link(Transport, Socket, Options) ->
{ok, proc_lib:spawn_link(?MODULE, init, [{Transport, Socket, Options}])}.
@ -77,11 +87,10 @@ start_link(Transport, Socket, Options) ->
%% API
%%--------------------------------------------------------------------
%% For debug
-spec(info(pid() | #state{}) -> map()).
%% @doc Get infos of the channel.
-spec(info(pid() | state()) -> emqx_types:infos()).
info(CPid) when is_pid(CPid) ->
call(CPid, info);
info(#state{transport = Transport,
socket = Socket,
peername = Peername,
@ -90,39 +99,57 @@ info(#state{transport = Transport,
active_n = ActiveN,
rate_limit = RateLimit,
pub_limit = PubLimit,
proto_state = ProtoState}) ->
ConnInfo = #{socktype => Transport:type(Socket),
proto_state = ProtoState,
gc_state = GCState,
stats_timer = StatsTimer,
idle_timeout = IdleTimeout,
connected = Connected,
connected_at = ConnectedAt}) ->
ChanInfo = #{socktype => Transport:type(Socket),
peername => Peername,
sockname => Sockname,
conn_state => ConnState,
active_n => ActiveN,
rate_limit => rate_limit_info(RateLimit),
pub_limit => rate_limit_info(PubLimit)
rate_limit => limit_info(RateLimit),
pub_limit => limit_info(PubLimit),
gc_state => emqx_gc:info(GCState),
enable_stats => case StatsTimer of
disabled -> false;
_Otherwise -> true
end,
idle_timeout => IdleTimeout,
connected => Connected,
connected_at => ConnectedAt
},
ProtoInfo = emqx_protocol:info(ProtoState),
maps:merge(ConnInfo, ProtoInfo).
maps:merge(ChanInfo, emqx_protocol:info(ProtoState)).
rate_limit_info(undefined) ->
#{};
rate_limit_info(Limit) ->
limit_info(undefined) ->
undefined;
limit_info(Limit) ->
esockd_rate_limit:info(Limit).
%% For dashboard
%% @doc Get attrs of the channel.
-spec(attrs(pid() | state()) -> emqx_types:attrs()).
attrs(CPid) when is_pid(CPid) ->
call(CPid, attrs);
attrs(#state{peername = Peername,
attrs(#state{transport = Transport,
socket = Socket,
peername = Peername,
sockname = Sockname,
proto_state = ProtoState}) ->
SockAttrs = #{peername => Peername,
sockname => Sockname},
ProtoAttrs = emqx_protocol:attrs(ProtoState),
maps:merge(SockAttrs, ProtoAttrs).
proto_state = ProtoState,
connected = Connected,
connected_at = ConnectedAt}) ->
ConnAttrs = #{socktype => Transport:type(Socket),
peername => Peername,
sockname => Sockname,
connected => Connected,
connected_at => ConnectedAt},
maps:merge(ConnAttrs, emqx_protocol:attrs(ProtoState)).
%% Conn stats
%% @doc Get stats of the channel.
-spec(stats(pid() | state()) -> emqx_types:stats()).
stats(CPid) when is_pid(CPid) ->
call(CPid, stats);
stats(#state{transport = Transport,
socket = Socket,
proto_state = ProtoState}) ->
@ -130,16 +157,13 @@ stats(#state{transport = Transport,
{ok, Ss} -> Ss;
{error, _} -> []
end,
lists:append([SockStats,
emqx_misc:proc_stats(),
emqx_protocol:stats(ProtoState)]).
ChanStats = [{Name, emqx_pd:get_counter(Name)} || Name <- ?CHAN_STATS],
SessStats = emqx_session:stats(emqx_protocol:info(session, ProtoState)),
lists:append([SockStats, ChanStats, SessStats, emqx_misc:proc_stats()]).
kick(CPid) ->
call(CPid, kick).
session(CPid) ->
call(CPid, session).
state(CPid) -> call(CPid, get_state).
%% @private
call(CPid, Req) ->
gen_statem:call(CPid, Req, infinity).
@ -148,7 +172,6 @@ call(CPid, Req) ->
%%--------------------------------------------------------------------
init({Transport, RawSocket, Options}) ->
process_flag(trap_exit, true),
{ok, Socket} = Transport:wait(RawSocket),
{ok, Peername} = Transport:ensure_ok_or_exit(peername, [Socket]),
{ok, Sockname} = Transport:ensure_ok_or_exit(sockname, [Socket]),
@ -158,39 +181,33 @@ init({Transport, RawSocket, Options}) ->
RateLimit = init_limiter(proplists:get_value(rate_limit, Options)),
PubLimit = init_limiter(emqx_zone:get_env(Zone, publish_limit)),
ActiveN = proplists:get_value(active_n, Options, ?ACTIVE_N),
SendFun = fun(Packet, Opts) ->
Data = emqx_frame:serialize(Packet, Opts),
case Transport:async_send(Socket, Data) of
ok -> {ok, Data};
{error, Reason} ->
{error, Reason}
end
end,
MaxSize = emqx_zone:get_env(Zone, max_packet_size, ?MAX_PACKET_SIZE),
ParseState = emqx_frame:initial_parse_state(#{max_size => MaxSize}),
ProtoState = emqx_protocol:init(#{peername => Peername,
sockname => Sockname,
peercert => Peercert,
sendfun => SendFun,
conn_mod => ?MODULE}, Options),
MaxSize = emqx_zone:get_env(Zone, max_packet_size, ?MAX_PACKET_SIZE),
ParseState = emqx_frame:initial_parse_state(#{max_size => MaxSize}),
GcPolicy = emqx_zone:get_env(Zone, force_gc_policy, false),
GcState = emqx_gc:init(GcPolicy),
EnableStats = emqx_zone:get_env(Zone, enable_stats, true),
StatsTimer = if EnableStats -> undefined; ?Otherwise -> disabled end,
IdleTimout = emqx_zone:get_env(Zone, idle_timeout, 30000),
ok = emqx_misc:init_proc_mng_policy(Zone),
State = #state{transport = Transport,
socket = Socket,
peername = Peername,
sockname = Sockname,
conn_state = running,
active_n = ActiveN,
rate_limit = RateLimit,
pub_limit = PubLimit,
proto_state = ProtoState,
parse_state = ParseState,
proto_state = ProtoState,
gc_state = GcState,
enable_stats = EnableStats,
idle_timeout = IdleTimout
stats_timer = StatsTimer,
idle_timeout = IdleTimout,
connected = false
},
ok = emqx_misc:init_proc_mng_policy(Zone),
gen_statem:enter_loop(?MODULE, [{hibernate_after, 2 * IdleTimout}],
idle, State, self(), [IdleTimout]).
@ -206,16 +223,27 @@ callback_mode() ->
%% Idle State
idle(enter, _, State) ->
ok = activate_socket(State),
keep_state_and_data;
case activate_socket(State) of
ok -> keep_state_and_data;
{error, Reason} ->
shutdown(Reason, State)
end;
idle(timeout, _Timeout, State) ->
{stop, {shutdown, idle_timeout}, State};
stop(idle_timeout, State);
idle(cast, {incoming, Packet = ?CONNECT_PACKET(
#mqtt_packet_connect{
proto_ver = ProtoVer}
)}, State) ->
State1 = State#state{serialize = serialize_fun(ProtoVer)},
handle_incoming(Packet, fun(NewSt) ->
{next_state, connected, NewSt}
end, State1);
idle(cast, {incoming, Packet}, State) ->
handle_incoming(Packet, fun(NState) ->
{next_state, connected, NState}
end, State);
?LOG(warning, "Unexpected incoming: ~p", [Packet]),
shutdown(unexpected_incoming_packet, State);
idle(EventType, Content, State) ->
?HANDLE(EventType, Content, State).
@ -223,56 +251,76 @@ idle(EventType, Content, State) ->
%%--------------------------------------------------------------------
%% Connected State
connected(enter, _, _State) ->
%% What to do?
keep_state_and_data;
%% Handle Input
connected(cast, {incoming, Packet = ?PACKET(Type)}, State) ->
ok = emqx_metrics:inc_recv(Packet),
(Type == ?PUBLISH) andalso emqx_pd:update_counter(incoming_pubs, 1),
handle_incoming(Packet, fun(NState) -> {keep_state, NState} end, State);
%% Handle Output
connected(info, {deliver, PubOrAck}, State = #state{proto_state = ProtoState}) ->
case emqx_protocol:deliver(PubOrAck, ProtoState) of
{ok, NProtoState} ->
NState = State#state{proto_state = NProtoState},
{keep_state, maybe_gc(PubOrAck, NState)};
{error, Reason} ->
shutdown(Reason, State)
end;
%% Start Keepalive
connected(info, {keepalive, start, Interval},
State = #state{transport = Transport, socket = Socket}) ->
StatFun = fun() ->
case Transport:getstat(Socket, [recv_oct]) of
{ok, [{recv_oct, RecvOct}]} -> {ok, RecvOct};
Error -> Error
end
end,
case emqx_keepalive:start(StatFun, Interval, {keepalive, check}) of
connected(enter, _PrevSt, State = #state{proto_state = ProtoState}) ->
NState = State#state{connected = true,
connected_at = os:timestamp()},
ClientId = emqx_protocol:info(client_id, ProtoState),
ok = emqx_cm:register_channel(ClientId),
ok = emqx_cm:set_chan_attrs(ClientId, info(NState)),
%% Ensure keepalive after connected successfully.
Interval = emqx_protocol:info(keepalive, ProtoState),
case ensure_keepalive(Interval, NState) of
ignore -> keep_state(NState);
{ok, KeepAlive} ->
{keep_state, State#state{keepalive = KeepAlive}};
{error, Error} ->
shutdown(Error, State)
keep_state(NState#state{keepalive = KeepAlive});
{error, Reason} ->
shutdown(Reason, NState)
end;
connected(cast, {incoming, Packet = ?PACKET(?CONNECT)}, State) ->
?LOG(warning, "Unexpected connect: ~p", [Packet]),
shutdown(unexpected_incoming_connect, State);
connected(cast, {incoming, Packet}, State) when is_record(Packet, mqtt_packet) ->
handle_incoming(Packet, fun keep_state/1, State);
connected(info, Deliver = {deliver, _Topic, _Msg},
State = #state{proto_state = ProtoState}) ->
Delivers = emqx_misc:drain_deliver([Deliver]),
case emqx_protocol:handle_deliver(Delivers, ProtoState) of
{ok, NProtoState} ->
keep_state(State#state{proto_state = NProtoState});
{ok, Packets, NProtoState} ->
NState = State#state{proto_state = NProtoState},
handle_outgoing(Packets, fun keep_state/1, NState);
{error, Reason} ->
shutdown(Reason, State);
{error, Reason, NProtoState} ->
shutdown(Reason, State#state{proto_state = NProtoState})
end;
%% TODO: Improve later.
connected(info, {subscribe, TopicFilters}, State) ->
handle_request({subscribe, TopicFilters}, State);
connected(info, {unsubscribe, TopicFilters}, State) ->
handle_request({unsubscribe, TopicFilters}, State);
%% Keepalive timer
connected(info, {keepalive, check}, State = #state{keepalive = KeepAlive}) ->
case emqx_keepalive:check(KeepAlive) of
{ok, KeepAlive1} ->
{keep_state, State#state{keepalive = KeepAlive1}};
keep_state(State#state{keepalive = KeepAlive1});
{error, timeout} ->
shutdown(keepalive_timeout, State);
{error, Error} ->
shutdown(Error, State)
{error, Reason} ->
shutdown(Reason, State)
end;
connected(EventType, Content, State) ->
?HANDLE(EventType, Content, State).
%%--------------------------------------------------------------------
%% Disconnected State
disconnected(enter, _, _State) ->
%% TODO: What to do?
%% CleanStart is true
keep_state_and_data;
disconnected(EventType, Content, State) ->
?HANDLE(EventType, Content, State).
%% Handle call
handle({call, From}, info, State) ->
reply(From, info(State), State);
@ -283,12 +331,16 @@ handle({call, From}, attrs, State) ->
handle({call, From}, stats, State) ->
reply(From, stats(State), State);
handle({call, From}, kick, State) ->
ok = gen_statem:reply(From, ok),
shutdown(kicked, State);
handle({call, From}, get_state, State) ->
reply(From, State, State);
handle({call, From}, session, State = #state{proto_state = ProtoState}) ->
reply(From, emqx_protocol:session(ProtoState), State);
%%handle({call, From}, kick, State) ->
%% ok = gen_statem:reply(From, ok),
%% shutdown(kicked, State);
%%handle({call, From}, discard, State) ->
%% ok = gen_statem:reply(From, ok),
%% shutdown(discard, State);
handle({call, From}, Req, State) ->
?LOG(error, "Unexpected call: ~p", [Req]),
@ -297,40 +349,49 @@ handle({call, From}, Req, State) ->
%% Handle cast
handle(cast, Msg, State) ->
?LOG(error, "Unexpected cast: ~p", [Msg]),
{keep_state, State};
keep_state(State);
%% Handle Incoming
handle(info, {Inet, _Sock, Data}, State) when Inet == tcp; Inet == ssl ->
%% Handle incoming data
handle(info, {Inet, _Sock, Data}, State) when Inet == tcp;
Inet == ssl ->
Oct = iolist_size(Data),
?LOG(debug, "RECV ~p", [Data]),
emqx_pd:update_counter(incoming_bytes, Oct),
ok = emqx_metrics:inc('bytes.received', Oct),
NState = ensure_stats_timer(maybe_gc({1, Oct}, State)),
process_incoming(Data, [], NState);
NState = maybe_gc(1, Oct, State),
process_incoming(Data, ensure_stats_timer(NState));
handle(info, {Error, _Sock, Reason}, State)
when Error == tcp_error; Error == ssl_error ->
handle(info, {Error, _Sock, Reason}, State) when Error == tcp_error;
Error == ssl_error ->
shutdown(Reason, State);
handle(info, {Closed, _Sock}, State)
when Closed == tcp_closed; Closed == ssl_closed ->
handle(info, {Closed, _Sock}, State) when Closed == tcp_closed;
Closed == ssl_closed ->
shutdown(closed, State);
handle(info, {Passive, _Sock}, State) when Passive == tcp_passive;
Passive == ssl_passive ->
%% Rate limit here:)
NState = ensure_rate_limit(State),
ok = activate_socket(NState),
{keep_state, NState};
case activate_socket(NState) of
ok -> keep_state(NState);
{error, Reason} ->
shutdown(Reason, NState)
end;
handle(info, activate_socket, State) ->
%% Rate limit timer expired.
ok = activate_socket(State#state{conn_state = running}),
{keep_state, State#state{conn_state = running, limit_timer = undefined}};
NState = State#state{conn_state = running},
case activate_socket(NState) of
ok ->
keep_state(NState#state{limit_timer = undefined});
{error, Reason} ->
shutdown(Reason, NState)
end;
handle(info, {inet_reply, _Sock, ok}, State) ->
%% something sent
{keep_state, ensure_stats_timer(State)};
keep_state(ensure_stats_timer(State));
handle(info, {inet_reply, _Sock, {error, Reason}}, State) ->
shutdown(Reason, State);
@ -339,13 +400,13 @@ handle(info, {timeout, Timer, emit_stats},
State = #state{stats_timer = Timer,
proto_state = ProtoState,
gc_state = GcState}) ->
ClientId = emqx_protocol:client_id(ProtoState),
emqx_cm:set_conn_stats(ClientId, stats(State)),
ClientId = emqx_protocol:info(client_id, ProtoState),
ok = emqx_cm:set_chan_stats(ClientId, stats(State)),
NState = State#state{stats_timer = undefined},
Limits = erlang:get(force_shutdown_policy),
case emqx_misc:conn_proc_mng_policy(Limits) of
continue ->
{keep_state, NState};
keep_state(NState);
hibernate ->
%% going to hibernate, reset gc stats
GcState1 = emqx_gc:reset(GcState),
@ -355,6 +416,20 @@ handle(info, {timeout, Timer, emit_stats},
shutdown(Reason, NState)
end;
handle(info, {timeout, Timer, Msg},
State = #state{proto_state = ProtoState}) ->
case emqx_protocol:handle_timeout(Timer, Msg, ProtoState) of
{ok, NProtoState} ->
keep_state(State#state{proto_state = NProtoState});
{ok, Packets, NProtoState} ->
handle_outgoing(Packets, fun keep_state/1,
State#state{proto_state = NProtoState});
{error, Reason} ->
shutdown(Reason, State);
{error, Reason, NProtoState} ->
shutdown(Reason, State#state{proto_state = NProtoState})
end;
handle(info, {shutdown, discard, {ClientId, ByPid}}, State) ->
?LOG(error, "Discarded by ~s:~p", [ClientId, ByPid]),
shutdown(discard, State);
@ -366,19 +441,9 @@ handle(info, {shutdown, conflict, {ClientId, NewPid}}, State) ->
handle(info, {shutdown, Reason}, State) ->
shutdown(Reason, State);
handle(info, Info = {'EXIT', SessionPid, Reason}, State = #state{proto_state = ProtoState}) ->
case emqx_protocol:session(ProtoState) of
undefined ->
?LOG(error, "Unexpected EXIT: ~p", [Info]),
{keep_state, State};
SessionPid ->
?LOG(error, "Session ~p termiated: ~p", [SessionPid, Reason]),
shutdown(Reason, State)
end;
handle(info, Info, State) ->
?LOG(error, "Unexpected info: ~p", [Info]),
{keep_state, State}.
keep_state(State).
code_change(_Vsn, State, Data, _Extra) ->
{ok, State, Data}.
@ -388,27 +453,36 @@ terminate(Reason, _StateName, #state{transport = Transport,
keepalive = KeepAlive,
proto_state = ProtoState}) ->
?LOG(debug, "Terminated for ~p", [Reason]),
Transport:fast_close(Socket),
emqx_keepalive:cancel(KeepAlive),
case {ProtoState, Reason} of
{undefined, _} -> ok;
{_, {shutdown, Error}} ->
emqx_protocol:terminate(Error, ProtoState);
{_, Reason} ->
emqx_protocol:terminate(Reason, ProtoState)
ok = Transport:fast_close(Socket),
ok = emqx_keepalive:cancel(KeepAlive),
emqx_protocol:terminate(Reason, ProtoState).
%%--------------------------------------------------------------------
%% Handle internal request
handle_request(Req, State = #state{proto_state = ProtoState}) ->
case emqx_protocol:handle_req(Req, ProtoState) of
{ok, _Result, NProtoState} -> %% TODO:: how to handle the result?
keep_state(State#state{proto_state = NProtoState});
{error, Reason, NProtoState} ->
shutdown(Reason, State#state{proto_state = NProtoState})
end.
%%--------------------------------------------------------------------
%% Process incoming data
-compile({inline, [process_incoming/2]}).
process_incoming(Data, State) ->
process_incoming(Data, [], State).
process_incoming(<<>>, Packets, State) ->
{keep_state, State, next_events(Packets)};
{keep_state, State, next_incoming_events(Packets)};
process_incoming(Data, Packets, State = #state{parse_state = ParseState}) ->
try emqx_frame:parse(Data, ParseState) of
{ok, NParseState} ->
NState = State#state{parse_state = NParseState},
{keep_state, NState, next_events(Packets)};
{keep_state, NState, next_incoming_events(Packets)};
{ok, Packet, Rest, NParseState} ->
NState = State#state{parse_state = NParseState},
process_incoming(Rest, [Packet|Packets], NState);
@ -421,26 +495,85 @@ process_incoming(Data, Packets, State = #state{parse_state = ParseState}) ->
shutdown(parse_error, State)
end.
next_events(Packets) when is_list(Packets) ->
[next_events(Packet) || Packet <- lists:reverse(Packets)];
next_events(Packet) ->
{next_event, cast, {incoming, Packet}}.
next_incoming_events(Packets) when is_list(Packets) ->
[next_event(cast, {incoming, Packet})
|| Packet <- lists:reverse(Packets)].
%%--------------------------------------------------------------------
%% Handle incoming packet
handle_incoming(Packet, SuccFun, State = #state{proto_state = ProtoState}) ->
case emqx_protocol:received(Packet, ProtoState) of
handle_incoming(Packet = ?PACKET(Type), SuccFun,
State = #state{proto_state = ProtoState}) ->
_ = inc_incoming_stats(Type),
ok = emqx_metrics:inc_recv(Packet),
?LOG(debug, "RECV ~s", [emqx_packet:format(Packet)]),
case emqx_protocol:handle_in(Packet, ProtoState) of
{ok, NProtoState} ->
SuccFun(State#state{proto_state = NProtoState});
{error, Reason} ->
shutdown(Reason, State);
{ok, OutPackets, NProtoState} ->
handle_outgoing(OutPackets, SuccFun,
State#state{proto_state = NProtoState});
{error, Reason, NProtoState} ->
shutdown(Reason, State#state{proto_state = NProtoState});
{error, Reason, OutPacket, NProtoState} ->
Shutdown = fun(NewSt) -> shutdown(Reason, NewSt) end,
handle_outgoing(OutPacket, Shutdown, State#state{proto_state = NProtoState});
{stop, Error, NProtoState} ->
stop(Error, State#state{proto_state = NProtoState})
end.
%%--------------------------------------------------------------------
%% Handle outgoing packets
handle_outgoing(Packets, SuccFun, State = #state{serialize = Serialize})
when is_list(Packets) ->
send(lists:map(Serialize, Packets), SuccFun, State);
handle_outgoing(Packet, SuccFun, State = #state{serialize = Serialize}) ->
send(Serialize(Packet), SuccFun, State).
%%--------------------------------------------------------------------
%% Serialize fun
serialize_fun(ProtoVer) ->
fun(Packet = ?PACKET(Type)) ->
?LOG(debug, "SEND ~s", [emqx_packet:format(Packet)]),
_ = inc_outgoing_stats(Type),
emqx_frame:serialize(Packet, ProtoVer)
end.
%%--------------------------------------------------------------------
%% Send data
send(IoData, SuccFun, State = #state{transport = Transport,
socket = Socket}) ->
Oct = iolist_size(IoData),
ok = emqx_metrics:inc('bytes.sent', Oct),
case Transport:async_send(Socket, IoData) of
ok -> SuccFun(maybe_gc(1, Oct, State));
{error, Reason} ->
shutdown(Reason, State)
end.
%%--------------------------------------------------------------------
%% Ensure keepalive
ensure_keepalive(0, _State) ->
ignore;
ensure_keepalive(Interval, #state{transport = Transport,
socket = Socket,
proto_state = ProtoState}) ->
StatFun = fun() ->
case Transport:getstat(Socket, [recv_oct]) of
{ok, [{recv_oct, RecvOct}]} ->
{ok, RecvOct};
Error -> Error
end
end,
Backoff = emqx_zone:get_env(emqx_protocol:info(zone, ProtoState),
keepalive_backoff, 0.75),
emqx_keepalive:start(StatFun, round(Interval * Backoff), {keepalive, check}).
%%--------------------------------------------------------------------
%% Ensure rate limit
@ -460,58 +593,82 @@ ensure_rate_limit([{Rl, Pos, Cnt}|Limiters], State) ->
{Pause, Rl1} ->
?LOG(debug, "Rate limit pause connection ~pms", [Pause]),
TRef = erlang:send_after(Pause, self(), activate_socket),
setelement(Pos, State#state{conn_state = blocked, limit_timer = TRef}, Rl1)
setelement(Pos, State#state{conn_state = blocked,
limit_timer = TRef}, Rl1)
end.
%%--------------------------------------------------------------------
%% Activate socket
%% Activate Socket
activate_socket(#state{conn_state = blocked}) ->
ok;
activate_socket(#state{transport = Transport,
socket = Socket,
active_n = N}) ->
Transport:setopts(Socket, [{active, N}]).
activate_socket(#state{transport = Transport, socket = Socket, active_n = N}) ->
case Transport:setopts(Socket, [{active, N}]) of
ok -> ok;
{error, Reason} ->
self() ! {shutdown, Reason},
ok
%%--------------------------------------------------------------------
%% Inc incoming/outgoing stats
-compile({inline,
[ inc_incoming_stats/1
, inc_outgoing_stats/1
]}).
inc_incoming_stats(Type) ->
emqx_pd:update_counter(recv_pkt, 1),
case Type == ?PUBLISH of
true ->
emqx_pd:update_counter(recv_msg, 1),
emqx_pd:update_counter(incoming_pubs, 1);
false -> ok
end.
inc_outgoing_stats(Type) ->
emqx_pd:update_counter(send_pkt, 1),
(Type == ?PUBLISH)
andalso emqx_pd:update_counter(send_msg, 1).
%%--------------------------------------------------------------------
%% Ensure stats timer
ensure_stats_timer(State = #state{enable_stats = true,
stats_timer = undefined,
ensure_stats_timer(State = #state{stats_timer = undefined,
idle_timeout = IdleTimeout}) ->
State#state{stats_timer = emqx_misc:start_timer(IdleTimeout, emit_stats)};
TRef = emqx_misc:start_timer(IdleTimeout, emit_stats),
State#state{stats_timer = TRef};
%% disabled or timer existed
ensure_stats_timer(State) -> State.
%%--------------------------------------------------------------------
%% Maybe GC
maybe_gc(_, State = #state{gc_state = undefined}) ->
maybe_gc(_Cnt, _Oct, State = #state{gc_state = undefined}) ->
State;
maybe_gc({publish, _, #message{payload = Payload}}, State) ->
Oct = iolist_size(Payload),
maybe_gc({1, Oct}, State);
maybe_gc(Packets, State) when is_list(Packets) ->
{Cnt, Oct} =
lists:unzip([{1, iolist_size(Payload)}
|| {publish, _, #message{payload = Payload}} <- Packets]),
maybe_gc({lists:sum(Cnt), lists:sum(Oct)}, State);
maybe_gc({Cnt, Oct}, State = #state{gc_state = GCSt}) ->
{_, GCSt1} = emqx_gc:run(Cnt, Oct, GCSt),
State#state{gc_state = GCSt1};
maybe_gc(_, State) -> State.
maybe_gc(Cnt, Oct, State = #state{gc_state = GCSt}) ->
{Ok, GCSt1} = emqx_gc:run(Cnt, Oct, GCSt),
Ok andalso emqx_metrics:inc('channel.gc.cnt'),
State#state{gc_state = GCSt1}.
%%--------------------------------------------------------------------
%% Helper functions
-compile({inline,
[ reply/3
, keep_state/1
, next_event/2
, shutdown/2
, stop/2
]}).
reply(From, Reply, State) ->
{keep_state, State, [{reply, From, Reply}]}.
shutdown(Reason = {shutdown, _}, State) ->
stop(Reason, State);
keep_state(State) ->
{keep_state, State}.
next_event(Type, Content) ->
{next_event, Type, Content}.
shutdown(Reason, State) ->
stop({shutdown, Reason}, State).

5
src/emqx_cli.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_cli).
@ -35,3 +37,4 @@ usage(CmdList) ->
usage(Format, Args) ->
usage([{Format, Args}]).

27
src/emqx_client.erl
View File

@ -20,7 +20,7 @@
-include("logger.hrl").
-include("types.hrl").
-include("emqx_client.hrl").
-include("emqx_mqtt.hrl").
-logger_header("[Client]").
@ -144,7 +144,17 @@
| {force_ping, boolean()}
| {properties, properties()}).
-type(mqtt_msg() :: #mqtt_msg{}).
-record(mqtt_msg, {
qos = ?QOS_0,
retain = false,
dup = false,
packet_id,
topic,
props,
payload
}).
-opaque(mqtt_msg() :: #mqtt_msg{}).
-record(state, {name :: atom(),
owner :: pid(),
@ -160,7 +170,7 @@
clean_start :: boolean(),
username :: maybe(binary()),
password :: maybe(binary()),
proto_ver :: emqx_mqtt_types:version(),
proto_ver :: emqx_types:mqtt_ver(),
proto_name :: iodata(),
keepalive :: non_neg_integer(),
keepalive_timer :: maybe(reference()),
@ -192,11 +202,11 @@
-type(payload() :: iodata()).
-type(packet_id() :: emqx_mqtt_types:packet_id()).
-type(packet_id() :: emqx_types:packet_id()).
-type(properties() :: emqx_mqtt_types:properties()).
-type(properties() :: emqx_types:properties()).
-type(qos() :: emqx_mqtt_types:qos_name() | emqx_mqtt_types:qos()).
-type(qos() :: emqx_types:qos_name() | emqx_types:qos()).
-type(pubopt() :: {retain, boolean()} | {qos, qos()} | {timeout, timeout()}).
@ -205,7 +215,7 @@
| {nl, boolean()}
| {qos, qos()}).
-type(reason_code() :: emqx_mqtt_types:reason_code()).
-type(reason_code() :: emqx_types:reason_code()).
-type(subscribe_ret() :: {ok, properties(), [reason_code()]} | {error, term()}).
@ -1201,7 +1211,7 @@ send(Msg, State) when is_record(Msg, mqtt_msg) ->
send(Packet, State = #state{socket = Sock, proto_ver = Ver})
when is_record(Packet, mqtt_packet) ->
Data = emqx_frame:serialize(Packet, #{version => Ver}),
Data = emqx_frame:serialize(Packet, Ver),
?LOG(debug, "SEND Data: ~1000p", [Packet]),
case emqx_client_sock:send(Sock, Data) of
ok -> {ok, bump_last_packet_id(State)};
@ -1246,4 +1256,3 @@ bump_last_packet_id(State = #state{last_packet_id = Id}) ->
-spec next_packet_id(packet_id()) -> packet_id().
next_packet_id(?MAX_PACKET_ID) -> 1;
next_packet_id(Id) -> Id + 1.

9
src/emqx_client_sock.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_client_sock).
@ -24,6 +26,8 @@
, getstat/2
]).
-export_type([socket/0, option/0]).
-record(ssl_socket, {tcp, ssl}).
-type(socket() :: inet:socket() | #ssl_socket{}).
@ -32,8 +36,6 @@
-type(option() :: gen_tcp:connect_option() | {ssl_opts, [ssl:ssl_option()]}).
-export_type([socket/0, option/0]).
-define(DEFAULT_TCP_OPTIONS, [binary, {packet, raw}, {active, false},
{nodelay, true}, {reuseaddr, true}]).
@ -105,3 +107,4 @@ default_ciphers(TlsVersions) ->
fun(TlsVer, Ciphers) ->
Ciphers ++ ssl:cipher_suites(all, TlsVer)
end, [], TlsVersions).

343
src/emqx_cm.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,7 +12,9 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% Channel Manager
-module(emqx_cm).
-behaviour(gen_server).
@ -24,25 +27,29 @@
-export([start_link/0]).
-export([ register_connection/1
, register_connection/2
, unregister_connection/1
, unregister_connection/2
-export([ register_channel/1
, unregister_channel/1
, unregister_channel/2
]).
-export([ get_conn_attrs/1
, get_conn_attrs/2
, set_conn_attrs/2
, set_conn_attrs/3
-export([ get_chan_attrs/1
, get_chan_attrs/2
, set_chan_attrs/2
]).
-export([ get_conn_stats/1
, get_conn_stats/2
, set_conn_stats/2
, set_conn_stats/3
-export([ get_chan_stats/1
, get_chan_stats/2
, set_chan_stats/2
]).
-export([lookup_conn_pid/1]).
-export([ open_session/3
, discard_session/1
, resume_session/1
]).
-export([ lookup_channels/1
, lookup_channels/2
]).
%% gen_server callbacks
-export([ init/1
@ -53,159 +60,269 @@
, code_change/3
]).
%% internal export
%% Internal export
-export([stats_fun/0]).
-define(CM, ?MODULE).
-type(chan_pid() :: pid()).
%% ETS tables for connection management.
-define(CONN_TAB, emqx_conn).
-define(CONN_ATTRS_TAB, emqx_conn_attrs).
-define(CONN_STATS_TAB, emqx_conn_stats).
%% Tables for channel management.
-define(CHAN_TAB, emqx_channel).
-define(CHAN_P_TAB, emqx_channel_p).
-define(CHAN_ATTRS_TAB, emqx_channel_attrs).
-define(CHAN_STATS_TAB, emqx_channel_stats).
-define(CHAN_STATS,
[{?CHAN_TAB, 'channels.count', 'channels.max'},
{?CHAN_TAB, 'connections.count', 'connections.max'},
{?CHAN_TAB, 'sessions.count', 'sessions.max'},
{?CHAN_P_TAB, 'sessions.persistent.count', 'sessions.persistent.max'}
]).
%% Batch drain
-define(BATCH_SIZE, 100000).
%% @doc Start the connection manager.
%% Server name
-define(CM, ?MODULE).
%% @doc Start the channel manager.
-spec(start_link() -> startlink_ret()).
start_link() ->
gen_server:start_link({local, ?CM}, ?MODULE, [], []).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Register a connection.
-spec(register_connection(emqx_types:client_id()) -> ok).
register_connection(ClientId) when is_binary(ClientId) ->
register_connection(ClientId, self()).
%% @doc Register a channel.
-spec(register_channel(emqx_types:client_id()) -> ok).
register_channel(ClientId) when is_binary(ClientId) ->
register_channel(ClientId, self()).
-spec(register_connection(emqx_types:client_id(), pid()) -> ok).
register_connection(ClientId, ConnPid) when is_binary(ClientId), is_pid(ConnPid) ->
true = ets:insert(?CONN_TAB, {ClientId, ConnPid}),
notify({registered, ClientId, ConnPid}).
%% @doc Register a channel with pid.
-spec(register_channel(emqx_types:client_id(), chan_pid()) -> ok).
register_channel(ClientId, ChanPid) ->
Chan = {ClientId, ChanPid},
true = ets:insert(?CHAN_TAB, Chan),
ok = emqx_cm_registry:register_channel(Chan),
cast({registered, Chan}).
%% @doc Unregister a connection.
-spec(unregister_connection(emqx_types:client_id()) -> ok).
unregister_connection(ClientId) when is_binary(ClientId) ->
unregister_connection(ClientId, self()).
%% @doc Unregister a channel.
-spec(unregister_channel(emqx_types:client_id()) -> ok).
unregister_channel(ClientId) when is_binary(ClientId) ->
unregister_channel(ClientId, self()).
-spec(unregister_connection(emqx_types:client_id(), pid()) -> ok).
unregister_connection(ClientId, ConnPid) when is_binary(ClientId), is_pid(ConnPid) ->
true = do_unregister_connection({ClientId, ConnPid}),
notify({unregistered, ConnPid}).
-spec(unregister_channel(emqx_types:client_id(), chan_pid()) -> ok).
unregister_channel(ClientId, ChanPid) ->
Chan = {ClientId, ChanPid},
true = do_unregister_channel(Chan),
cast({unregistered, Chan}).
do_unregister_connection(Conn) ->
true = ets:delete(?CONN_STATS_TAB, Conn),
true = ets:delete(?CONN_ATTRS_TAB, Conn),
true = ets:delete_object(?CONN_TAB, Conn).
%% @private
do_unregister_channel(Chan) ->
ok = emqx_cm_registry:unregister_channel(Chan),
true = ets:delete_object(?CHAN_P_TAB, Chan),
true = ets:delete(?CHAN_ATTRS_TAB, Chan),
true = ets:delete(?CHAN_STATS_TAB, Chan),
ets:delete_object(?CHAN_TAB, Chan).
%% @doc Get conn attrs
-spec(get_conn_attrs(emqx_types:client_id()) -> list()).
get_conn_attrs(ClientId) when is_binary(ClientId) ->
ConnPid = lookup_conn_pid(ClientId),
get_conn_attrs(ClientId, ConnPid).
%% @doc Get attrs of a channel.
-spec(get_chan_attrs(emqx_types:client_id()) -> maybe(emqx_types:attrs())).
get_chan_attrs(ClientId) ->
with_channel(ClientId, fun(ChanPid) -> get_chan_attrs(ClientId, ChanPid) end).
-spec(get_conn_attrs(emqx_types:client_id(), pid()) -> list()).
get_conn_attrs(ClientId, ConnPid) when is_binary(ClientId) ->
emqx_tables:lookup_value(?CONN_ATTRS_TAB, {ClientId, ConnPid}, []).
-spec(get_chan_attrs(emqx_types:client_id(), chan_pid()) -> maybe(emqx_types:attrs())).
get_chan_attrs(ClientId, ChanPid) when node(ChanPid) == node() ->
Chan = {ClientId, ChanPid},
emqx_tables:lookup_value(?CHAN_ATTRS_TAB, Chan);
get_chan_attrs(ClientId, ChanPid) ->
rpc_call(node(ChanPid), get_chan_attrs, [ClientId, ChanPid]).
%% @doc Set conn attrs
-spec(set_conn_attrs(emqx_types:client_id(), list()) -> true).
set_conn_attrs(ClientId, Attrs) when is_binary(ClientId) ->
set_conn_attrs(ClientId, self(), Attrs).
%% @doc Set attrs of a channel.
-spec(set_chan_attrs(emqx_types:client_id(), emqx_types:attrs()) -> ok).
set_chan_attrs(ClientId, Attrs) when is_binary(ClientId) ->
Chan = {ClientId, self()},
true = ets:insert(?CHAN_ATTRS_TAB, {Chan, Attrs}),
ok.
-spec(set_conn_attrs(emqx_types:client_id(), pid(), list()) -> true).
set_conn_attrs(ClientId, ConnPid, Attrs) when is_binary(ClientId), is_pid(ConnPid) ->
Conn = {ClientId, ConnPid},
ets:insert(?CONN_ATTRS_TAB, {Conn, Attrs}).
%% @doc Get channel's stats.
-spec(get_chan_stats(emqx_types:client_id()) -> maybe(emqx_types:stats())).
get_chan_stats(ClientId) ->
with_channel(ClientId, fun(ChanPid) -> get_chan_stats(ClientId, ChanPid) end).
%% @doc Get conn stats
-spec(get_conn_stats(emqx_types:client_id()) -> list(emqx_stats:stats())).
get_conn_stats(ClientId) when is_binary(ClientId) ->
ConnPid = lookup_conn_pid(ClientId),
get_conn_stats(ClientId, ConnPid).
-spec(get_chan_stats(emqx_types:client_id(), chan_pid()) -> maybe(emqx_types:stats())).
get_chan_stats(ClientId, ChanPid) when node(ChanPid) == node() ->
Chan = {ClientId, ChanPid},
emqx_tables:lookup_value(?CHAN_STATS_TAB, Chan);
get_chan_stats(ClientId, ChanPid) ->
rpc_call(node(ChanPid), get_chan_stats, [ClientId, ChanPid]).
-spec(get_conn_stats(emqx_types:client_id(), pid()) -> list(emqx_stats:stats())).
get_conn_stats(ClientId, ConnPid) when is_binary(ClientId) ->
Conn = {ClientId, ConnPid},
emqx_tables:lookup_value(?CONN_STATS_TAB, Conn, []).
%% @doc Set channel's stats.
-spec(set_chan_stats(emqx_types:client_id(), emqx_types:stats()) -> ok).
set_chan_stats(ClientId, Stats) when is_binary(ClientId) ->
set_chan_stats(ClientId, self(), Stats).
%% @doc Set conn stats.
-spec(set_conn_stats(emqx_types:client_id(), list(emqx_stats:stats())) -> true).
set_conn_stats(ClientId, Stats) when is_binary(ClientId) ->
set_conn_stats(ClientId, self(), Stats).
-spec(set_chan_stats(emqx_types:client_id(), chan_pid(), emqx_types:stats()) -> ok).
set_chan_stats(ClientId, ChanPid, Stats) ->
Chan = {ClientId, ChanPid},
true = ets:insert(?CHAN_STATS_TAB, {Chan, Stats}),
ok.
-spec(set_conn_stats(emqx_types:client_id(), pid(), list(emqx_stats:stats())) -> true).
set_conn_stats(ClientId, ConnPid, Stats) when is_binary(ClientId), is_pid(ConnPid) ->
Conn = {ClientId, ConnPid},
ets:insert(?CONN_STATS_TAB, {Conn, Stats}).
%% @doc Open a session.
-spec(open_session(boolean(), emqx_types:client(), map())
-> {ok, emqx_session:session()} | {error, Reason :: term()}).
open_session(true, Client = #{client_id := ClientId}, Options) ->
CleanStart = fun(_) ->
ok = discard_session(ClientId),
{ok, emqx_session:init(true, Client, Options), false}
end,
emqx_cm_locker:trans(ClientId, CleanStart);
%% @doc Lookup connection pid.
-spec(lookup_conn_pid(emqx_types:client_id()) -> maybe(pid())).
lookup_conn_pid(ClientId) when is_binary(ClientId) ->
emqx_tables:lookup_value(?CONN_TAB, ClientId).
open_session(false, Client = #{client_id := ClientId}, Options) ->
ResumeStart = fun(_) ->
case resume_session(ClientId) of
{ok, Session} ->
{ok, Session, true};
{error, not_found} ->
{ok, emqx_session:init(false, Client, Options), false}
end
end,
emqx_cm_locker:trans(ClientId, ResumeStart).
notify(Msg) ->
gen_server:cast(?CM, {notify, Msg}).
%% @doc Try to resume a session.
-spec(resume_session(emqx_types:client_id())
-> {ok, emqx_session:session()} | {error, Reason :: term()}).
resume_session(ClientId) ->
case lookup_channels(ClientId) of
[] -> {error, not_found};
[_ChanPid] ->
ok;
% emqx_channel:resume(ChanPid);
ChanPids ->
[_ChanPid|StalePids] = lists:reverse(ChanPids),
?LOG(error, "[SM] More than one channel found: ~p", [ChanPids]),
lists:foreach(fun(_StalePid) ->
% catch emqx_channel:discard(StalePid)
ok
end, StalePids),
% emqx_channel:resume(ChanPid)
ok
end.
%%-----------------------------------------------------------------------------
%% @doc Discard all the sessions identified by the ClientId.
-spec(discard_session(emqx_types:client_id()) -> ok).
discard_session(ClientId) when is_binary(ClientId) ->
case lookup_channels(ClientId) of
[] -> ok;
ChanPids ->
lists:foreach(
fun(ChanPid) ->
try ok
% emqx_channel:discard(ChanPid)
catch
_:Error:_Stk ->
?LOG(warning, "[SM] Failed to discard ~p: ~p", [ChanPid, Error])
end
end, ChanPids)
end.
%% @doc Is clean start?
% is_clean_start(#{clean_start := false}) -> false;
% is_clean_start(_Attrs) -> true.
with_channel(ClientId, Fun) ->
case lookup_channels(ClientId) of
[] -> undefined;
[Pid] -> Fun(Pid);
Pids -> Fun(lists:last(Pids))
end.
%% @doc Lookup channels.
-spec(lookup_channels(emqx_types:client_id()) -> list(chan_pid())).
lookup_channels(ClientId) ->
lookup_channels(global, ClientId).
%% @doc Lookup local or global channels.
-spec(lookup_channels(local | global, emqx_types:client_id()) -> list(chan_pid())).
lookup_channels(global, ClientId) ->
case emqx_cm_registry:is_enabled() of
true ->
emqx_cm_registry:lookup_channels(ClientId);
false ->
lookup_channels(local, ClientId)
end;
lookup_channels(local, ClientId) ->
[ChanPid || {_, ChanPid} <- ets:lookup(?CHAN_TAB, ClientId)].
%% @private
rpc_call(Node, Fun, Args) ->
case rpc:call(Node, ?MODULE, Fun, Args) of
{badrpc, Reason} -> error(Reason);
Res -> Res
end.
%% @private
cast(Msg) -> gen_server:cast(?CM, Msg).
%%--------------------------------------------------------------------
%% gen_server callbacks
%%-----------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
TabOpts = [public, set, {write_concurrency, true}],
ok = emqx_tables:new(?CONN_TAB, [{read_concurrency, true} | TabOpts]),
ok = emqx_tables:new(?CONN_ATTRS_TAB, TabOpts),
ok = emqx_tables:new(?CONN_STATS_TAB, TabOpts),
ok = emqx_stats:update_interval(conn_stats, fun ?MODULE:stats_fun/0),
{ok, #{conn_pmon => emqx_pmon:new()}}.
TabOpts = [public, {write_concurrency, true}],
ok = emqx_tables:new(?CHAN_TAB, [bag, {read_concurrency, true} | TabOpts]),
ok = emqx_tables:new(?CHAN_P_TAB, [bag | TabOpts]),
ok = emqx_tables:new(?CHAN_ATTRS_TAB, [set, compressed | TabOpts]),
ok = emqx_tables:new(?CHAN_STATS_TAB, [set | TabOpts]),
ok = emqx_stats:update_interval(chan_stats, fun ?MODULE:stats_fun/0),
{ok, #{chan_pmon => emqx_pmon:new()}}.
handle_call(Req, _From, State) ->
?LOG(error, "Unexpected call: ~p", [Req]),
{reply, ignored, State}.
handle_cast({notify, {registered, ClientId, ConnPid}}, State = #{conn_pmon := PMon}) ->
{noreply, State#{conn_pmon := emqx_pmon:monitor(ConnPid, ClientId, PMon)}};
handle_cast({registered, {ClientId, ChanPid}}, State = #{chan_pmon := PMon}) ->
PMon1 = emqx_pmon:monitor(ChanPid, ClientId, PMon),
{noreply, State#{chan_pmon := PMon1}};
handle_cast({notify, {unregistered, ConnPid}}, State = #{conn_pmon := PMon}) ->
{noreply, State#{conn_pmon := emqx_pmon:demonitor(ConnPid, PMon)}};
handle_cast({unregistered, {_ClientId, ChanPid}}, State = #{chan_pmon := PMon}) ->
PMon1 = emqx_pmon:demonitor(ChanPid, PMon),
{noreply, State#{chan_pmon := PMon1}};
handle_cast(Msg, State) ->
?LOG(error, "Unexpected cast: ~p", [Msg]),
{noreply, State}.
handle_info({'DOWN', _MRef, process, Pid, _Reason}, State = #{conn_pmon := PMon}) ->
ConnPids = [Pid | emqx_misc:drain_down(?BATCH_SIZE)],
{Items, PMon1} = emqx_pmon:erase_all(ConnPids, PMon),
ok = emqx_pool:async_submit(
fun lists:foreach/2, [fun clean_down/1, Items]),
{noreply, State#{conn_pmon := PMon1}};
handle_info({'DOWN', _MRef, process, Pid, _Reason}, State = #{chan_pmon := PMon}) ->
ChanPids = [Pid | emqx_misc:drain_down(?BATCH_SIZE)],
{Items, PMon1} = emqx_pmon:erase_all(ChanPids, PMon),
ok = emqx_pool:async_submit(fun lists:foreach/2, [fun clean_down/1, Items]),
{noreply, State#{chan_pmon := PMon1}};
handle_info(Info, State) ->
?LOG(error, "Unexpected info: ~p", [Info]),
{noreply, State}.
terminate(_Reason, _State) ->
emqx_stats:cancel_update(conn_stats).
emqx_stats:cancel_update(chan_stats).
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
clean_down({Pid, ClientId}) ->
Conn = {ClientId, Pid},
case ets:member(?CONN_TAB, ClientId)
orelse ets:member(?CONN_ATTRS_TAB, Conn) of
true ->
do_unregister_connection(Conn);
false -> false
end.
clean_down({ChanPid, ClientId}) ->
Chan = {ClientId, ChanPid},
do_unregister_channel(Chan).
stats_fun() ->
case ets:info(?CONN_TAB, size) of
lists:foreach(fun update_stats/1, ?CHAN_STATS).
update_stats({Tab, Stat, MaxStat}) ->
case ets:info(Tab, size) of
undefined -> ok;
Size -> emqx_stats:setstat('connections.count', 'connections.max', Size)
Size -> emqx_stats:setstat(Stat, MaxStat, Size)
end.

10
src/emqx_sm_locker.erl → src/emqx_cm_locker.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,8 +12,9 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_sm_locker).
-module(emqx_cm_locker).
-include("emqx.hrl").
-include("types.hrl").
@ -26,10 +28,6 @@
, unlock/1
]).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
-spec(start_link() -> startlink_ret()).
start_link() ->
ekka_locker:start_link(?MODULE).

95
src/emqx_sm_registry.erl → src/emqx_cm_registry.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,8 +12,10 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_sm_registry).
%% Global Channel Registry
-module(emqx_cm_registry).
-behaviour(gen_server).
@ -24,12 +27,14 @@
-export([start_link/0]).
-export([ is_enabled/0
, register_session/1
, lookup_session/1
, unregister_session/1
-export([is_enabled/0]).
-export([ register_channel/1
, unregister_channel/1
]).
-export([lookup_channels/1]).
%% gen_server callbacks
-export([ init/1
, handle_call/3
@ -40,57 +45,67 @@
]).
-define(REGISTRY, ?MODULE).
-define(TAB, emqx_session_registry).
-define(LOCK, {?MODULE, cleanup_sessions}).
-define(TAB, emqx_channel_registry).
-define(LOCK, {?MODULE, cleanup_down}).
-record(global_session, {sid, pid}).
-record(channel, {chid, pid}).
-type(session_pid() :: pid()).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%% @doc Start the global session manager.
%% @doc Start the global channel registry.
-spec(start_link() -> startlink_ret()).
start_link() ->
gen_server:start_link({local, ?REGISTRY}, ?MODULE, [], []).
%%--------------------------------------------------------------------
%% APIs
%%--------------------------------------------------------------------
%% @doc Is the global registry enabled?
-spec(is_enabled() -> boolean()).
is_enabled() ->
emqx_config:get_env(enable_session_registry, true).
emqx_config:get_env(enable_channel_registry, true).
-spec(lookup_session(emqx_types:client_id()) -> list(session_pid())).
lookup_session(ClientId) ->
[SessPid || #global_session{pid = SessPid} <- mnesia:dirty_read(?TAB, ClientId)].
%% @doc Register a global channel.
-spec(register_channel(emqx_types:client_id()
| {emqx_types:client_id(), pid()}) -> ok).
register_channel(ClientId) when is_binary(ClientId) ->
register_channel({ClientId, self()});
-spec(register_session({emqx_types:client_id(), session_pid()}) -> ok).
register_session({ClientId, SessPid}) when is_binary(ClientId), is_pid(SessPid) ->
register_channel({ClientId, ChanPid}) when is_binary(ClientId), is_pid(ChanPid) ->
case is_enabled() of
true -> mnesia:dirty_write(?TAB, record(ClientId, SessPid));
true -> mnesia:dirty_write(?TAB, record(ClientId, ChanPid));
false -> ok
end.
-spec(unregister_session({emqx_types:client_id(), session_pid()}) -> ok).
unregister_session({ClientId, SessPid}) when is_binary(ClientId), is_pid(SessPid) ->
%% @doc Unregister a global channel.
-spec(unregister_channel(emqx_types:client_id()
| {emqx_types:client_id(), pid()}) -> ok).
unregister_channel(ClientId) when is_binary(ClientId) ->
unregister_channel({ClientId, self()});
unregister_channel({ClientId, ChanPid}) when is_binary(ClientId), is_pid(ChanPid) ->
case is_enabled() of
true -> mnesia:dirty_delete_object(?TAB, record(ClientId, SessPid));
true -> mnesia:dirty_delete_object(?TAB, record(ClientId, ChanPid));
false -> ok
end.
record(ClientId, SessPid) ->
#global_session{sid = ClientId, pid = SessPid}.
%% @doc Lookup the global channels.
-spec(lookup_channels(emqx_types:client_id()) -> list(pid())).
lookup_channels(ClientId) ->
[ChanPid || #channel{pid = ChanPid} <- mnesia:dirty_read(?TAB, ClientId)].
%%------------------------------------------------------------------------------
record(ClientId, ChanPid) ->
#channel{chid = ClientId, pid = ChanPid}.
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
ok = ekka_mnesia:create_table(?TAB, [
{type, bag},
{ram_copies, [node()]},
{record_name, global_session},
{attributes, record_info(fields, global_session)},
{record_name, channel},
{attributes, record_info(fields, channel)},
{storage_properties, [{ets, [{read_concurrency, true},
{write_concurrency, true}]}]}]),
ok = ekka_mnesia:copy_table(?TAB),
@ -108,7 +123,7 @@ handle_cast(Msg, State) ->
handle_info({membership, {mnesia, down, Node}}, State) ->
global:trans({?LOCK, self()},
fun() ->
mnesia:transaction(fun cleanup_sessions/1, [Node])
mnesia:transaction(fun cleanup_channels/1, [Node])
end),
{noreply, State};
@ -125,14 +140,14 @@ terminate(_Reason, _State) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
cleanup_sessions(Node) ->
Pat = [{#global_session{pid = '$1', _ = '_'}, [{'==', {node, '$1'}, Node}], ['$_']}],
lists:foreach(fun delete_session/1, mnesia:select(?TAB, Pat, write)).
cleanup_channels(Node) ->
Pat = [{#channel{pid = '$1', _ = '_'}, [{'==', {node, '$1'}, Node}], ['$_']}],
lists:foreach(fun delete_channel/1, mnesia:select(?TAB, Pat, write)).
delete_session(Session) ->
mnesia:delete_object(?TAB, Session, write).
delete_channel(Chan) ->
mnesia:delete_object(?TAB, Chan, write).

32
src/emqx_cm_sup.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_cm_sup).
@ -36,13 +38,33 @@ init([]) ->
shutdown => 1000,
type => worker,
modules => [emqx_flapping]},
%% Channel locker
Locker = #{id => locker,
start => {emqx_cm_locker, start_link, []},
restart => permanent,
shutdown => 5000,
type => worker,
modules => [emqx_cm_locker]
},
%% Channel registry
Registry = #{id => registry,
start => {emqx_cm_registry, start_link, []},
restart => permanent,
shutdown => 5000,
type => worker,
modules => [emqx_cm_registry]
},
%% Channel Manager
Manager = #{id => manager,
start => {emqx_cm, start_link, []},
restart => permanent,
shutdown => 2000,
shutdown => 5000,
type => worker,
modules => [emqx_cm]},
modules => [emqx_cm]
},
SupFlags = #{strategy => one_for_one,
intensity => 100,
period => 10},
{ok, {SupFlags, [Banned, Manager, Flapping]}}.
period => 10
},
{ok, {SupFlags, [Banned, Flapping, Locker, Registry, Manager]}}.

4
src/emqx_config.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc Hot Configuration
%%

4
src/emqx_ctl.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_ctl).

29
src/emqx_flapping.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,9 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc This module is used to garbage clean the flapping records.
-module(emqx_flapping).
@ -21,14 +25,12 @@
-export([start_link/0]).
%% This module is used to garbage clean the flapping records
%% gen_statem callbacks
-export([ terminate/3
, code_change/4
, init/1
-export([ init/1
, initialized/3
, callback_mode/0
, terminate/3
, code_change/4
]).
-define(FLAPPING_TAB, ?MODULE).
@ -37,15 +39,15 @@
-export([check/3]).
-record(flapping,
{ client_id :: binary()
, check_count :: integer()
, timestamp :: integer()
-record(flapping, {
client_id :: binary(),
check_count :: integer(),
timestamp :: integer()
}).
-type(flapping_record() :: #flapping{}).
-type(flapping_state() :: flapping | ok).
-type(flapping_state() :: flapping | ok).
%% @doc This function is used to initialize flapping records
%% the expiry time unit is minutes.
@ -103,14 +105,14 @@ start_link() ->
gen_statem:start_link({local, ?MODULE}, ?MODULE, [], []).
init([]) ->
TimerInterval = emqx_config:get_env(flapping_clean_interval, ?default_flapping_clean_interval),
Interval = emqx_config:get_env(flapping_clean_interval, ?default_flapping_clean_interval),
TabOpts = [ public
, set
, {keypos, 2}
, {write_concurrency, true}
, {read_concurrency, true}],
ok = emqx_tables:new(?FLAPPING_TAB, TabOpts),
{ok, initialized, #{timer_interval => TimerInterval}}.
{ok, initialized, #{timer_interval => Interval}}.
callback_mode() -> [state_functions, state_enter].
@ -137,3 +139,4 @@ clean_expired_records() ->
NowTime = emqx_time:now_secs(),
MatchSpec = [{{'$1', '$2', '$3'},[{'<', '$3', NowTime}], [true]}],
ets:select_delete(?FLAPPING_TAB, MatchSpec).

55
src/emqx_frame.erl
View File

@ -29,22 +29,22 @@
, serialize/2
]).
-export_type([ options/0
, parse_state/0
, parse_result/0
]).
-type(options() :: #{max_size => 1..?MAX_PACKET_SIZE,
version => emqx_mqtt_types:version()
version => emqx_types:version()
}).
-opaque(parse_state() :: {none, options()} | {more, cont_fun()}).
-opaque(parse_result() :: {ok, parse_state()}
| {ok, emqx_mqtt_types:packet(), binary(), parse_state()}).
| {ok, emqx_types:packet(), binary(), parse_state()}).
-type(cont_fun() :: fun((binary()) -> parse_result())).
-export_type([ options/0
, parse_state/0
, parse_result/0
]).
-define(none(Opts), {none, Opts}).
-define(more(Cont), {more, Cont}).
-define(DEFAULT_OPTIONS,
@ -385,15 +385,15 @@ parse_binary_data(<<Len:16/big, Data:Len/binary, Rest/binary>>) ->
%% Serialize MQTT Packet
%%--------------------------------------------------------------------
-spec(serialize(emqx_mqtt_types:packet()) -> iodata()).
-spec(serialize(emqx_types:packet()) -> iodata()).
serialize(Packet) ->
serialize(Packet, ?DEFAULT_OPTIONS).
serialize(Packet, ?MQTT_PROTO_V4).
-spec(serialize(emqx_mqtt_types:packet(), options()) -> iodata()).
-spec(serialize(emqx_types:packet(), emqx_types:version()) -> iodata()).
serialize(#mqtt_packet{header = Header,
variable = Variable,
payload = Payload}, Options) when is_map(Options) ->
serialize(Header, serialize_variable(Variable, merge_opts(Options)), serialize_payload(Payload)).
payload = Payload}, Ver) ->
serialize(Header, serialize_variable(Variable, Ver), serialize_payload(Payload)).
serialize(#mqtt_packet_header{type = Type,
dup = Dup,
@ -420,7 +420,7 @@ serialize_variable(#mqtt_packet_connect{
will_topic = WillTopic,
will_payload = WillPayload,
username = Username,
password = Password}, _Options) ->
password = Password}, _Ver) ->
[serialize_binary_data(ProtoName),
<<(case IsBridge of
true -> 16#80 + ProtoVer;
@ -447,14 +447,12 @@ serialize_variable(#mqtt_packet_connect{
serialize_variable(#mqtt_packet_connack{ack_flags = AckFlags,
reason_code = ReasonCode,
properties = Properties},
#{version := Ver}) ->
properties = Properties}, Ver) ->
[AckFlags, ReasonCode, serialize_properties(Properties, Ver)];
serialize_variable(#mqtt_packet_publish{topic_name = TopicName,
packet_id = PacketId,
properties = Properties},
#{version := Ver}) ->
properties = Properties}, Ver) ->
[serialize_utf8_string(TopicName),
if
PacketId =:= undefined -> <<>>;
@ -462,59 +460,54 @@ serialize_variable(#mqtt_packet_publish{topic_name = TopicName,
end,
serialize_properties(Properties, Ver)];
serialize_variable(#mqtt_packet_puback{packet_id = PacketId},
#{version := Ver})
serialize_variable(#mqtt_packet_puback{packet_id = PacketId}, Ver)
when Ver == ?MQTT_PROTO_V3; Ver == ?MQTT_PROTO_V4 ->
<<PacketId:16/big-unsigned-integer>>;
serialize_variable(#mqtt_packet_puback{packet_id = PacketId,
reason_code = ReasonCode,
properties = Properties},
#{version := ?MQTT_PROTO_V5}) ->
?MQTT_PROTO_V5) ->
[<<PacketId:16/big-unsigned-integer>>, ReasonCode,
serialize_properties(Properties, ?MQTT_PROTO_V5)];
serialize_variable(#mqtt_packet_subscribe{packet_id = PacketId,
properties = Properties,
topic_filters = TopicFilters},
#{version := Ver}) ->
topic_filters = TopicFilters}, Ver) ->
[<<PacketId:16/big-unsigned-integer>>, serialize_properties(Properties, Ver),
serialize_topic_filters(subscribe, TopicFilters, Ver)];
serialize_variable(#mqtt_packet_suback{packet_id = PacketId,
properties = Properties,
reason_codes = ReasonCodes},
#{version := Ver}) ->
reason_codes = ReasonCodes}, Ver) ->
[<<PacketId:16/big-unsigned-integer>>, serialize_properties(Properties, Ver),
serialize_reason_codes(ReasonCodes)];
serialize_variable(#mqtt_packet_unsubscribe{packet_id = PacketId,
properties = Properties,
topic_filters = TopicFilters},
#{version := Ver}) ->
topic_filters = TopicFilters}, Ver) ->
[<<PacketId:16/big-unsigned-integer>>, serialize_properties(Properties, Ver),
serialize_topic_filters(unsubscribe, TopicFilters, Ver)];
serialize_variable(#mqtt_packet_unsuback{packet_id = PacketId,
properties = Properties,
reason_codes = ReasonCodes},
#{version := Ver}) ->
reason_codes = ReasonCodes}, Ver) ->
[<<PacketId:16/big-unsigned-integer>>, serialize_properties(Properties, Ver),
serialize_reason_codes(ReasonCodes)];
serialize_variable(#mqtt_packet_disconnect{}, #{version := Ver})
serialize_variable(#mqtt_packet_disconnect{}, Ver)
when Ver == ?MQTT_PROTO_V3; Ver == ?MQTT_PROTO_V4 ->
<<>>;
serialize_variable(#mqtt_packet_disconnect{reason_code = ReasonCode,
properties = Properties},
#{version := Ver = ?MQTT_PROTO_V5}) ->
Ver = ?MQTT_PROTO_V5) ->
[ReasonCode, serialize_properties(Properties, Ver)];
serialize_variable(#mqtt_packet_disconnect{}, _Ver) ->
<<>>;
serialize_variable(#mqtt_packet_auth{reason_code = ReasonCode,
properties = Properties},
#{version := Ver = ?MQTT_PROTO_V5}) ->
Ver = ?MQTT_PROTO_V5) ->
[ReasonCode, serialize_properties(Properties, Ver)];
serialize_variable(PacketId, ?MQTT_PROTO_V3) when is_integer(PacketId) ->

21
src/emqx_gc.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,13 +12,17 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc This module manages an opaque collection of statistics data used to
%%--------------------------------------------------------------------
%% @doc
%% This module manages an opaque collection of statistics data used to
%% force garbage collection on `self()' process when hitting thresholds.
%% Namely:
%% (1) Total number of messages passed through
%% (2) Total data volume passed through
%% @end
%%--------------------------------------------------------------------
-module(emqx_gc).
@ -29,17 +34,17 @@
, reset/1
]).
-export_type([gc_state/0]).
-type(opts() :: #{count => integer(),
bytes => integer()}).
-type(st() :: #{cnt => {integer(), integer()},
oct => {integer(), integer()}}).
-opaque(gc_state() :: {?MODULE, st()}).
-opaque(gc_state() :: {gc_state, st()}).
-export_type([gc_state/0]).
-define(GCS(St), {?MODULE, St}).
-define(GCS(St), {gc_state, St}).
-define(disabled, disabled).
-define(ENABLED(X), (is_integer(X) andalso X > 0)).
@ -85,9 +90,9 @@ reset(?GCS(St)) ->
reset(undefined) ->
undefined.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(dec(cnt | oct, pos_integer(), st()) -> {boolean(), st()}).
dec(Key, Num, St) ->

4
src/emqx_gen_mod.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_gen_mod).

7
src/emqx_guid.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc Generate global unique id for mqtt message.
%%
@ -136,6 +138,5 @@ to_base62(<<I:128>>) ->
emqx_base62:encode(I).
from_base62(S) ->
I = emqx_base62:decode(S, integer),
I = binary_to_integer( emqx_base62:decode(S)),
<<I:128>>.

40
src/emqx_hooks.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_hooks).
@ -21,7 +23,9 @@
-logger_header("[Hooks]").
-export([start_link/0, stop/0]).
-export([ start_link/0
, stop/0
]).
%% Hooks API
-export([ add/2
@ -42,6 +46,11 @@
, code_change/3
]).
-export_type([ hookpoint/0
, action/0
, filter/0
]).
%% Multiple callbacks can be registered on a hookpoint.
%% The execution order depends on the priority value:
%% - Callbacks with greater priority values will be run before
@ -54,28 +63,32 @@
-type(action() :: function() | mfa()).
-type(filter() :: function() | mfa()).
-record(callback, {action :: action(),
-record(callback, {
action :: action(),
filter :: filter(),
priority :: integer()}).
priority :: integer()
}).
-record(hook, {name :: hookpoint(), callbacks :: list(#callback{})}).
-export_type([hookpoint/0, action/0, filter/0]).
-record(hook, {
name :: hookpoint(),
callbacks :: list(#callback{})
}).
-define(TAB, ?MODULE).
-define(SERVER, ?MODULE).
-spec(start_link() -> startlink_ret()).
start_link() ->
gen_server:start_link({local, ?SERVER}, ?MODULE, [], [{hibernate_after, 1000}]).
gen_server:start_link({local, ?SERVER},
?MODULE, [], [{hibernate_after, 1000}]).
-spec(stop() -> ok).
stop() ->
gen_server:stop(?SERVER, normal, infinity).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Hooks API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Register a callback
-spec(add(hookpoint(), action() | #callback{}) -> ok_or_error(already_exists)).
@ -113,7 +126,6 @@ run(HookPoint, Args) ->
run_fold(HookPoint, Args, Acc) ->
do_run_fold(lookup(HookPoint), Args, Acc).
do_run([#callback{action = Action, filter = Filter} | Callbacks], Args) ->
case filter_passed(Filter, Args) andalso execute(Action, Args) of
%% stop the hook chain and return
@ -165,12 +177,12 @@ lookup(HookPoint) ->
[] -> []
end.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
ok = emqx_tables:new(?TAB, [{keypos, #hook.name}, {read_concurrency, true}, protected]),
ok = emqx_tables:new(?TAB, [{keypos, #hook.name}, {read_concurrency, true}]),
{ok, #{}}.
handle_call({add, HookPoint, Callback = #callback{action = Action}}, _From, State) ->

31
src/emqx_inflight.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,16 +12,20 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_inflight).
-compile(inline).
%% APIs
-export([ new/1
-export([ new/0
, new/1
, contain/2
, lookup/2
, insert/3
, update/3
, update_size/2
, resize/2
, delete/2
, values/1
, to_list/1
@ -31,24 +36,24 @@
, window/1
]).
-export_type([inflight/0]).
-type(key() :: term()).
-type(max_size() :: pos_integer()).
-opaque(inflight() :: {?MODULE, max_size(), gb_trees:tree()}).
-opaque(inflight() :: {inflight, max_size(), gb_trees:tree()}).
-define(Inflight(Tree), {?MODULE, _MaxSize, Tree}).
-define(Inflight(MaxSize, Tree), {?MODULE, MaxSize, (Tree)}).
-define(Inflight(Tree), {inflight, _MaxSize, Tree}).
-export_type([inflight/0]).
-define(Inflight(MaxSize, Tree), {inflight, MaxSize, (Tree)}).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
-spec(new() -> inflight()).
new() -> new(0).
-spec(new(non_neg_integer()) -> inflight()).
new(MaxSize) when MaxSize >= 0 ->
{?MODULE, MaxSize, gb_trees:empty()}.
?Inflight(MaxSize, gb_trees:empty()).
-spec(contain(key(), inflight()) -> boolean()).
contain(Key, ?Inflight(Tree)) ->
@ -70,8 +75,8 @@ delete(Key, ?Inflight(MaxSize, Tree)) ->
update(Key, Val, ?Inflight(MaxSize, Tree)) ->
?Inflight(MaxSize, gb_trees:update(Key, Val, Tree)).
-spec(update_size(integer(), inflight()) -> inflight()).
update_size(MaxSize, ?Inflight(Tree)) ->
-spec(resize(integer(), inflight()) -> inflight()).
resize(MaxSize, ?Inflight(Tree)) ->
?Inflight(MaxSize, Tree).
-spec(is_full(inflight()) -> boolean()).

20
src/emqx_json.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,9 +12,12 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_json).
-compile(inline).
-export([ encode/1
, encode/2
, safe_encode/1
@ -30,17 +34,18 @@
encode(Term) ->
jsx:encode(Term).
-spec(encode(jsx:json_term(), jsx_to_json:config()) -> jsx:json_text()).
-spec(encode(jsx:json_term(), jsx_to_json:config())
-> jsx:json_text()).
encode(Term, Opts) ->
jsx:encode(Term, Opts).
-spec(safe_encode(jsx:json_term())
-> {ok, jsx:json_text()} | {error, term()}).
-> {ok, jsx:json_text()} | {error, Reason :: term()}).
safe_encode(Term) ->
safe_encode(Term, []).
-spec(safe_encode(jsx:json_term(), jsx_to_json:config())
-> {ok, jsx:json_text()} | {error, term()}).
-> {ok, jsx:json_text()} | {error, Reason :: term()}).
safe_encode(Term, Opts) ->
try encode(Term, Opts) of
Json -> {ok, Json}
@ -53,17 +58,18 @@ safe_encode(Term, Opts) ->
decode(Json) ->
jsx:decode(Json).
-spec(decode(jsx:json_text(), jsx_to_json:config()) -> jsx:json_term()).
-spec(decode(jsx:json_text(), jsx_to_json:config())
-> jsx:json_term()).
decode(Json, Opts) ->
jsx:decode(Json, Opts).
-spec(safe_decode(jsx:json_text())
-> {ok, jsx:json_term()} | {error, term()}).
-> {ok, jsx:json_term()} | {error, Reason :: term()}).
safe_decode(Json) ->
safe_decode(Json, []).
-spec(safe_decode(jsx:json_text(), jsx_to_json:config())
-> {ok, jsx:json_term()} | {error, term()}).
-> {ok, jsx:json_term()} | {error, Reason :: term()}).
safe_decode(Json, Opts) ->
try decode(Json, Opts) of
Term -> {ok, Term}

22
src/emqx_keepalive.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_keepalive).
@ -20,15 +22,22 @@
, cancel/1
]).
-record(keepalive, {statfun, statval, tsec, tmsg, tref, repeat = 0}).
-export_type([keepalive/0]).
-record(keepalive, {
statfun,
statval,
tsec,
tmsg,
tref,
repeat = 0
}).
-opaque(keepalive() :: #keepalive{}).
-export_type([keepalive/0]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Start a keepalive
-spec(start(fun(), integer(), any()) -> {ok, keepalive()} | {error, term()}).
@ -79,3 +88,4 @@ cancel(_) ->
timer(Secs, Msg) ->
erlang:send_after(timer:seconds(Secs), self(), Msg).

7
src/emqx_kernel_sup.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_kernel_sup).
@ -30,8 +32,7 @@ init([]) ->
child_spec(emqx_stats, worker),
child_spec(emqx_metrics, worker),
child_spec(emqx_ctl, worker),
child_spec(emqx_zone, worker),
child_spec(emqx_tracer, worker)]}}.
child_spec(emqx_zone, worker)]}}.
child_spec(M, worker) ->
#{id => M,

41
src/emqx_listeners.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc Start/Stop MQTT listeners.
-module(emqx_listeners).
@ -33,9 +35,9 @@
-type(listener() :: {esockd:proto(), esockd:listen_on(), [esockd:option()]}).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Start all listeners.
-spec(start() -> ok).
@ -44,13 +46,15 @@ start() ->
-spec(start_listener(listener()) -> {ok, pid()} | {error, term()}).
start_listener({Proto, ListenOn, Options}) ->
case start_listener(Proto, ListenOn, Options) of
{ok, _} ->
io:format("Start mqtt:~s listener on ~s successfully.~n", [Proto, format(ListenOn)]);
StartRet = start_listener(Proto, ListenOn, Options),
case StartRet of
{ok, _} -> io:format("Start mqtt:~s listener on ~s successfully.~n",
[Proto, format(ListenOn)]);
{error, Reason} ->
io:format(standard_error, "Failed to start mqtt:~s listener on ~s - ~p~n!",
[Proto, format(ListenOn), Reason])
end.
end,
StartRet.
%% Start MQTT/TCP listener
-spec(start_listener(esockd:proto(), esockd:listen_on(), [esockd:option()])
@ -64,11 +68,13 @@ start_listener(Proto, ListenOn, Options) when Proto == ssl; Proto == tls ->
%% Start MQTT/WS listener
start_listener(Proto, ListenOn, Options) when Proto == http; Proto == ws ->
start_http_listener(fun cowboy:start_clear/3, 'mqtt:ws', ListenOn, ranch_opts(Options), ws_opts(Options));
start_http_listener(fun cowboy:start_clear/3, 'mqtt:ws', ListenOn,
ranch_opts(Options), ws_opts(Options));
%% Start MQTT/WSS listener
start_listener(Proto, ListenOn, Options) when Proto == https; Proto == wss ->
start_http_listener(fun cowboy:start_tls/3, 'mqtt:wss', ListenOn, ranch_opts(Options), ws_opts(Options)).
start_http_listener(fun cowboy:start_tls/3, 'mqtt:wss', ListenOn,
ranch_opts(Options), ws_opts(Options)).
start_mqtt_listener(Name, ListenOn, Options) ->
SockOpts = esockd:parse_opt(Options),
@ -82,8 +88,10 @@ mqtt_path(Options) ->
proplists:get_value(mqtt_path, Options, "/mqtt").
ws_opts(Options) ->
Dispatch = cowboy_router:compile([{'_', [{mqtt_path(Options), emqx_ws_channel, Options}]}]),
#{env => #{dispatch => Dispatch}, proxy_header => proplists:get_value(proxy_protocol, Options, false)}.
WsPaths = [{mqtt_path(Options), emqx_ws_channel, Options}],
Dispatch = cowboy_router:compile([{'_', WsPaths}]),
ProxyProto = proplists:get_value(proxy_protocol, Options, false),
#{env => #{dispatch => Dispatch}, proxy_header => ProxyProto}.
ranch_opts(Options) ->
NumAcceptors = proplists:get_value(acceptors, Options, 4),
@ -132,13 +140,15 @@ stop() ->
-spec(stop_listener(listener()) -> ok | {error, term()}).
stop_listener({Proto, ListenOn, Opts}) ->
case stop_listener(Proto, ListenOn, Opts) of
ok ->
io:format("Stop mqtt:~s listener on ~s successfully.~n", [Proto, format(ListenOn)]);
StopRet = stop_listener(Proto, ListenOn, Opts),
case StopRet of
ok -> io:format("Stop mqtt:~s listener on ~s successfully.~n",
[Proto, format(ListenOn)]);
{error, Reason} ->
io:format(standard_error, "Failed to stop mqtt:~s listener on ~s - ~p~n.",
[Proto, format(ListenOn), Reason])
end.
end,
StopRet.
-spec(stop_listener(esockd:proto(), esockd:listen_on(), [esockd:option()])
-> ok | {error, term()}).
@ -167,3 +177,4 @@ format({Addr, Port}) when is_list(Addr) ->
io_lib:format("~s:~w", [Addr, Port]);
format({Addr, Port}) when is_tuple(Addr) ->
io_lib:format("~s:~w", [esockd_net:ntoab(Addr), Port]).

64
src/emqx_logger.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_logger).
@ -50,84 +52,127 @@
-export([parse_transform/2]).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
-type(peername_str() :: list()).
-type(logger_dst() :: file:filename() | console | unknown).
-type(logger_handler_info() :: {logger:handler_id(), logger:level(), logger_dst()}).
%%--------------------------------------------------------------------
%% APIs
%%--------------------------------------------------------------------
-spec(debug(unicode:chardata()) -> ok).
debug(Msg) ->
logger:debug(Msg).
-spec(debug(io:format(), [term()]) -> ok).
debug(Format, Args) ->
logger:debug(Format, Args).
-spec(debug(logger:metadata(), io:format(), [term()]) -> ok).
debug(Metadata, Format, Args) when is_map(Metadata) ->
logger:debug(Format, Args, Metadata).
-spec(info(unicode:chardata()) -> ok).
info(Msg) ->
logger:info(Msg).
-spec(info(io:format(), [term()]) -> ok).
info(Format, Args) ->
logger:info(Format, Args).
-spec(info(logger:metadata(), io:format(), [term()]) -> ok).
info(Metadata, Format, Args) when is_map(Metadata) ->
logger:info(Format, Args, Metadata).
-spec(warning(unicode:chardata()) -> ok).
warning(Msg) ->
logger:warning(Msg).
-spec(warning(io:format(), [term()]) -> ok).
warning(Format, Args) ->
logger:warning(Format, Args).
-spec(warning(logger:metadata(), io:format(), [term()]) -> ok).
warning(Metadata, Format, Args) when is_map(Metadata) ->
logger:warning(Format, Args, Metadata).
-spec(error(unicode:chardata()) -> ok).
error(Msg) ->
logger:error(Msg).
-spec(error(io:format(), [term()]) -> ok).
error(Format, Args) ->
logger:error(Format, Args).
-spec(error(logger:metadata(), io:format(), [term()]) -> ok).
error(Metadata, Format, Args) when is_map(Metadata) ->
logger:error(Format, Args, Metadata).
-spec(critical(unicode:chardata()) -> ok).
critical(Msg) ->
logger:critical(Msg).
-spec(critical(io:format(), [term()]) -> ok).
critical(Format, Args) ->
logger:critical(Format, Args).
-spec(critical(logger:metadata(), io:format(), [term()]) -> ok).
critical(Metadata, Format, Args) when is_map(Metadata) ->
logger:critical(Format, Args, Metadata).
-spec(set_metadata_client_id(emqx_types:client_id()) -> ok).
set_metadata_client_id(ClientId) ->
set_proc_metadata(#{client_id => ClientId}).
-spec(set_metadata_peername(peername_str()) -> ok).
set_metadata_peername(Peername) ->
set_proc_metadata(#{peername => Peername}).
-spec(set_proc_metadata(logger:metadata()) -> ok).
set_proc_metadata(Meta) ->
logger:update_process_metadata(Meta).
-spec(get_primary_log_level() -> logger:level()).
get_primary_log_level() ->
#{level := Level} = logger:get_primary_config(),
Level.
-spec(set_primary_log_level(logger:level()) -> ok | {error, term()}).
set_primary_log_level(Level) ->
logger:set_primary_config(level, Level).
-spec(get_log_handlers() -> [logger_handler_info()]).
get_log_handlers() ->
lists:map(fun log_hanlder_info/1, logger:get_handler_config()).
-spec(get_log_handler(logger:handler_id()) -> logger_handler_info()).
get_log_handler(HandlerId) ->
{ok, Conf} = logger:get_handler_config(HandlerId),
log_hanlder_info(Conf).
-spec(set_log_handler_level(logger:handler_id(), logger:level()) -> ok | {error, term()}).
set_log_handler_level(HandlerId, Level) ->
logger:set_handler_config(HandlerId, level, Level).
%% Set both the primary and all handlers level in one command
%% @doc Set both the primary and all handlers level in one command
-spec(set_log_level(logger:handler_id()) -> ok | {error, term()}).
set_log_level(Level) ->
case set_primary_log_level(Level) of
ok -> set_all_log_handlers_level(Level);
{error, Error} -> {error, {primary_logger_level, Error}}
end.
%% @doc The parse transform for prefixing a module-specific logger header to the logs.
%% The logger header can be specified by "-logger_header(Header)", where Header
%% must be a string (list).
%% @end
parse_transform(AST, _Opts) ->
trans(AST, "", []).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal Functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
log_hanlder_info(#{id := Id, level := Level, module := logger_std_h,
config := #{type := Type}}) when Type =:= standard_io;
@ -165,7 +210,10 @@ rollback([{ID, Level} | List]) ->
rollback(List);
rollback([]) -> ok.
%% Generate a function '$logger_header'/0 into the source code
%% @doc The following parse-transforms stripped off the module attribute named
%% `-logger_header(Header)` (if there's one) from the source code, and then
%% generate a function named '$logger_header'/0, which returns the logger header.
%% @end
trans([], LogHeader, ResAST) ->
lists:reverse([header_fun(LogHeader) | ResAST]);
trans([{eof, L} | AST], LogHeader, ResAST) ->

20
src/emqx_logger_formatter.erl
View File

@ -34,18 +34,22 @@
-define(IS_STRING(String),
(is_list(String) orelse is_binary(String))).
%%%-----------------------------------------------------------------
%%% Types
-type config() :: #{chars_limit => pos_integer() | unlimited,
%%--------------------------------------------------------------------
%% Types
-type(config() :: #{chars_limit => pos_integer() | unlimited,
depth => pos_integer() | unlimited,
max_size => pos_integer() | unlimited,
report_cb => logger:report_cb(),
quit => template()}.
-type template() :: [metakey() | {metakey(),template(),template()} | string()].
-type metakey() :: atom() | [atom()].
quit => template()}).
-type(template() :: [metakey() | {metakey(),template(),template()} | string()]).
-type(metakey() :: atom() | [atom()]).
%%--------------------------------------------------------------------
%% API
%%%-----------------------------------------------------------------
%%% API
-spec format(LogEvent,Config) -> unicode:chardata() when
LogEvent :: logger:log_event(),
Config :: config().

8
src/emqx_message.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_message).
@ -74,7 +76,7 @@ make(From, Topic, Payload) ->
make(From, ?QOS_0, Topic, Payload).
-spec(make(atom() | emqx_types:client_id(),
emqx_mqtt_types:qos(),
emqx_types:qos(),
emqx_topic:topic(),
emqx_types:payload()) -> emqx_types:message()).
make(From, QoS, Topic, Payload) when ?QOS_0 =< QoS, QoS =< ?QOS_2 ->
@ -89,7 +91,7 @@ make(From, QoS, Topic, Payload) when ?QOS_0 =< QoS, QoS =< ?QOS_2 ->
-spec(id(emqx_types:message()) -> maybe(binary())).
id(#message{id = Id}) -> Id.
-spec(qos(emqx_types:message()) -> emqx_mqtt_types:qos()).
-spec(qos(emqx_types:message()) -> emqx_types:qos()).
qos(#message{qos = QoS}) -> QoS.
-spec(from(emqx_types:message()) -> atom() | binary()).

37
src/emqx_metrics.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_metrics).
@ -58,12 +60,12 @@
, code_change/3
]).
-export_type([metric_idx/0]).
-opaque(metric_idx() :: 1..1024).
-type(metric_name() :: atom() | string() | binary()).
-export_type([metric_idx/0]).
-define(MAX_SIZE, 1024).
-define(RESERVED_IDX, 256).
-define(TAB, ?MODULE).
@ -92,6 +94,7 @@
{counter, 'packets.puback.missed'}, % PUBACK packets missed
{counter, 'packets.pubrec.received'}, % PUBREC packets received
{counter, 'packets.pubrec.sent'}, % PUBREC packets sent
{counter, 'packets.pubrec.inuse'}, % PUBREC packet_id inuse
{counter, 'packets.pubrec.missed'}, % PUBREC packets missed
{counter, 'packets.pubrel.received'}, % PUBREL packets received
{counter, 'packets.pubrel.sent'}, % PUBREL packets sent
@ -132,10 +135,15 @@
{counter, 'messages.forward'} % Messages forward
]).
-define(CHAN_METRICS, [
{counter, 'channel.gc.cnt'}
]).
-define(MQTT_METRICS, [
{counter, 'auth.mqtt.anonymous'}
]).
-record(state, {next_idx = 1}).
-record(metric, {name, type, idx}).
@ -149,9 +157,9 @@ start_link() ->
stop() ->
gen_server:stop(?SERVER).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Metrics API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(new(metric_name()) -> ok).
new(Name) ->
@ -255,12 +263,12 @@ update_counter(Name, Value) ->
end,
counters:add(CRef, CIdx, Value).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Inc Received/Sent metrics
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Inc packets received.
-spec(inc_recv(emqx_mqtt_types:packet()) -> ok).
-spec(inc_recv(emqx_types:packet()) -> ok).
inc_recv(Packet) ->
inc('packets.received'),
do_inc_recv(Packet).
@ -297,7 +305,7 @@ do_inc_recv(_Packet) ->
ignore.
%% @doc Inc packets sent. Will not count $SYS PUBLISH.
-spec(inc_sent(emqx_mqtt_types:packet()) -> ok | ignore).
-spec(inc_sent(emqx_types:packet()) -> ok | ignore).
inc_sent(?PUBLISH_PACKET(_QoS, <<"$SYS/", _/binary>>, _, _)) ->
ignore;
inc_sent(Packet) ->
@ -343,9 +351,9 @@ do_inc_sent(?PACKET(?AUTH)) ->
do_inc_sent(_Packet) ->
ignore.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
% Create counters array
@ -395,9 +403,9 @@ terminate(_Reason, _State) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
reserved_idx('bytes.received') -> 01;
reserved_idx('bytes.sent') -> 02;
@ -451,4 +459,7 @@ reserved_idx('messages.dropped') -> 49;
reserved_idx('messages.expired') -> 50;
reserved_idx('messages.forward') -> 51;
reserved_idx('auth.mqtt.anonymous') -> 52;
reserved_idx('channel.gc.cnt') -> 53;
reserved_idx('packets.pubrec.inuse') -> 54;
reserved_idx(_) -> undefined.

38
src/emqx_misc.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_misc).
@ -27,7 +29,14 @@
, conn_proc_mng_policy/1
]).
-export([drain_down/1]).
-export([ drain_deliver/1
, drain_down/1
]).
-compile({inline,
[ start_timer/2
, start_timer/3
]}).
%% @doc Merge options
-spec(merge_opts(list(), list()) -> list()).
@ -66,9 +75,12 @@ proc_stats() ->
-spec(proc_stats(pid()) -> list()).
proc_stats(Pid) ->
Stats = process_info(Pid, [message_queue_len, heap_size, reductions]),
{value, {_, V}, Stats1} = lists:keytake(message_queue_len, 1, Stats),
[{mailbox_len, V} | Stats1].
case process_info(Pid, [message_queue_len, heap_size,
total_heap_size, reductions, memory]) of
undefined -> [];
[{message_queue_len, Len}|Stats] ->
[{mailbox_len, Len}|Stats]
end.
-define(DISABLED, 0).
@ -113,24 +125,34 @@ check([{Pred, Result} | Rest]) ->
is_message_queue_too_long(Qlength, Max) ->
is_enabled(Max) andalso Qlength > Max.
is_enabled(Max) -> is_integer(Max) andalso Max > ?DISABLED.
is_enabled(Max) ->
is_integer(Max) andalso Max > ?DISABLED.
proc_info(Key) ->
{Key, Value} = erlang:process_info(self(), Key),
Value.
%% @doc Drain delivers from the channel's mailbox.
drain_deliver(Acc) ->
receive
Deliver = {deliver, _Topic, _Msg} ->
drain_deliver([Deliver|Acc])
after 0 ->
lists:reverse(Acc)
end.
%% @doc Drain process down events.
-spec(drain_down(pos_integer()) -> list(pid())).
drain_down(Cnt) when Cnt > 0 ->
drain_down(Cnt, []).
drain_down(0, Acc) ->
lists:reverse(Acc);
drain_down(Cnt, Acc) ->
receive
{'DOWN', _MRef, process, Pid, _Reason} ->
drain_down(Cnt - 1, [Pid|Acc])
after 0 ->
lists:reverse(Acc)
drain_down(0, Acc)
end.

30
src/emqx_mod_acl_internal.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_mod_acl_internal).
@ -37,9 +39,9 @@
-type(acl_rules() :: #{publish => [emqx_access_rule:rule()],
subscribe => [emqx_access_rule:rule()]}).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
load(_Env) ->
Rules = rules_from_file(acl_file()),
@ -54,16 +56,16 @@ unload(_Env) ->
all_rules() ->
rules_from_file(acl_file()).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% ACL callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Check ACL
-spec(check_acl(emqx_types:credentials(), emqx_types:pubsub(), emqx_topic:topic(),
-spec(check_acl(emqx_types:client(), emqx_types:pubsub(), emqx_topic:topic(),
emqx_access_rule:acl_result(), acl_rules())
-> {ok, allow} | {ok, deny} | ok).
check_acl(Credentials, PubSub, Topic, _AclResult, Rules) ->
case match(Credentials, Topic, lookup(PubSub, Rules)) of
check_acl(Client, PubSub, Topic, _AclResult, Rules) ->
case match(Client, Topic, lookup(PubSub, Rules)) of
{matched, allow} -> {ok, allow};
{matched, deny} -> {ok, deny};
nomatch -> ok
@ -73,9 +75,9 @@ check_acl(Credentials, PubSub, Topic, _AclResult, Rules) ->
reload_acl() ->
unload([]), load([]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal Functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
acl_file() ->
emqx_config:get_env(acl_file).
@ -83,12 +85,12 @@ acl_file() ->
lookup(PubSub, Rules) ->
maps:get(PubSub, Rules, []).
match(_Credentials, _Topic, []) ->
match(_Client, _Topic, []) ->
nomatch;
match(Credentials, Topic, [Rule|Rules]) ->
case emqx_access_rule:match(Credentials, Topic, Rule) of
match(Client, Topic, [Rule|Rules]) ->
case emqx_access_rule:match(Client, Topic, Rule) of
nomatch ->
match(Credentials, Topic, Rules);
match(Client, Topic, Rules);
{matched, AllowDeny} ->
{matched, AllowDeny}
end.

52
src/emqx_mod_presence.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_mod_presence).
@ -31,25 +33,34 @@
, unload/1
]).
-define(ATTR_KEYS, [clean_start, proto_ver, proto_name, keepalive]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
load(Env) ->
emqx_hooks:add('client.connected', fun ?MODULE:on_client_connected/4, [Env]),
emqx_hooks:add('client.disconnected', fun ?MODULE:on_client_disconnected/3, [Env]).
emqx_hooks:add('client.connected', {?MODULE, on_client_connected, [Env]}),
emqx_hooks:add('client.disconnected', {?MODULE, on_client_disconnected, [Env]}).
on_client_connected(#{client_id := ClientId,
username := Username,
peername := {IpAddr, _}}, ConnAck, ConnAttrs, Env) ->
Attrs = maps:filter(fun(K, _) ->
lists:member(K, ?ATTR_KEYS)
end, ConnAttrs),
case emqx_json:safe_encode(Attrs#{clientid => ClientId,
peername := {IpAddr, _}
}, ConnAck,
#{session := #{clean_start := CleanStart,
expiry_interval := Interval
},
proto_name := ProtoName,
proto_ver := ProtoVer,
keepalive := Keepalive
}, Env) ->
case emqx_json:safe_encode(#{clientid => ClientId,
username => Username,
ipaddress => iolist_to_binary(esockd_net:ntoa(IpAddr)),
proto_name => ProtoName,
proto_ver => ProtoVer,
keepalive => Keepalive,
clean_start => CleanStart,
expiry_interval => Interval,
connack => ConnAck,
ts => erlang:system_time(millisecond)
}) of
@ -59,11 +70,13 @@ on_client_connected(#{client_id := ClientId,
?LOG(error, "Encoding connected event error: ~p", [Reason])
end.
on_client_disconnected(#{client_id := ClientId, username := Username}, Reason, Env) ->
case emqx_json:safe_encode([{clientid, ClientId},
{username, Username},
{reason, reason(Reason)},
{ts, erlang:system_time(millisecond)}]) of
on_client_disconnected(#{client_id := ClientId,
username := Username}, Reason, Env) ->
case emqx_json:safe_encode(#{clientid => ClientId,
username => Username,
reason => reason(Reason),
ts => erlang:system_time(millisecond)
}) of
{ok, Payload} ->
emqx_broker:publish(message(qos(Env), topic(disconnected, ClientId), Payload));
{error, Reason} ->
@ -71,8 +84,8 @@ on_client_disconnected(#{client_id := ClientId, username := Username}, Reason, E
end.
unload(_Env) ->
emqx_hooks:del('client.connected', fun ?MODULE:on_client_connected/4),
emqx_hooks:del('client.disconnected', fun ?MODULE:on_client_disconnected/3).
emqx_hooks:del('client.connected', {?MODULE, on_client_connected}),
emqx_hooks:del('client.disconnected', {?MODULE, on_client_disconnected}).
message(QoS, Topic, Payload) ->
emqx_message:set_flag(
@ -89,3 +102,4 @@ qos(Env) -> proplists:get_value(qos, Env, 0).
reason(Reason) when is_atom(Reason) -> Reason;
reason({Error, _}) when is_atom(Error) -> Error;
reason(_) -> internal_error.

37
src/emqx_mod_rewrite.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_mod_rewrite).
@ -20,8 +22,8 @@
-include_lib("emqx_mqtt.hrl").
%% APIs
-export([ rewrite_subscribe/3
, rewrite_unsubscribe/3
-export([ rewrite_subscribe/4
, rewrite_unsubscribe/4
, rewrite_publish/2
]).
@ -30,33 +32,33 @@
, unload/1
]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Load/Unload
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
load(RawRules) ->
Rules = compile(RawRules),
emqx_hooks:add('client.subscribe', fun ?MODULE:rewrite_subscribe/3, [Rules]),
emqx_hooks:add('client.unsubscribe', fun ?MODULE:rewrite_unsubscribe/3, [Rules]),
emqx_hooks:add('message.publish', fun ?MODULE:rewrite_publish/2, [Rules]).
emqx_hooks:add('client.subscribe', {?MODULE, rewrite_subscribe, [Rules]}),
emqx_hooks:add('client.unsubscribe', {?MODULE, rewrite_unsubscribe, [Rules]}),
emqx_hooks:add('message.publish', {?MODULE, rewrite_publish, [Rules]}).
rewrite_subscribe(_Credentials, TopicTable, Rules) ->
{ok, [{match_rule(Topic, Rules), Opts} || {Topic, Opts} <- TopicTable]}.
rewrite_subscribe(_Client, _Properties, TopicFilters, Rules) ->
{ok, [{match_rule(Topic, Rules), Opts} || {Topic, Opts} <- TopicFilters]}.
rewrite_unsubscribe(_Credentials, TopicTable, Rules) ->
{ok, [{match_rule(Topic, Rules), Opts} || {Topic, Opts} <- TopicTable]}.
rewrite_unsubscribe(_Client, _Properties, TopicFilters, Rules) ->
{ok, [{match_rule(Topic, Rules), Opts} || {Topic, Opts} <- TopicFilters]}.
rewrite_publish(Message = #message{topic = Topic}, Rules) ->
{ok, Message#message{topic = match_rule(Topic, Rules)}}.
unload(_) ->
emqx_hooks:del('client.subscribe', fun ?MODULE:rewrite_subscribe/3),
emqx_hooks:del('client.unsubscribe', fun ?MODULE:rewrite_unsubscribe/3),
emqx_hooks:del('message.publish', fun ?MODULE:rewrite_publish/2).
emqx_hooks:del('client.subscribe', {?MODULE, rewrite_subscribe}),
emqx_hooks:del('client.unsubscribe', {?MODULE, rewrite_unsubscribe}),
emqx_hooks:del('message.publish', {?MODULE, rewrite_publish}).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
match_rule(Topic, []) ->
Topic;
@ -84,3 +86,4 @@ compile(Rules) ->
{ok, MP} = re:compile(Re),
{rewrite, Topic, MP, Dest}
end, Rules).

22
src/emqx_mod_subscription.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_mod_subscription).
@ -20,28 +22,30 @@
-include_lib("emqx_mqtt.hrl").
%% APIs
-export([on_session_created/3]).
-export([on_client_connected/4]).
%% emqx_gen_mod callbacks
-export([ load/1
, unload/1
]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Load/Unload Hook
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
load(Topics) ->
emqx_hooks:add('session.created', fun ?MODULE:on_session_created/3, [Topics]).
emqx_hooks:add('client.connected', {?MODULE, on_client_connected, [Topics]}).
on_session_created(#{client_id := ClientId}, SessAttrs, Topics) ->
Username = proplists:get_value(username, SessAttrs),
on_client_connected(#{client_id := ClientId,
username := Username}, ?RC_SUCCESS, _ConnAttrs, Topics) ->
Replace = fun(Topic) ->
rep(<<"%u">>, Username, rep(<<"%c">>, ClientId, Topic))
end,
emqx_session:subscribe(self(), [{Replace(Topic), #{qos => QoS}} || {Topic, QoS} <- Topics]).
TopicFilters = [{Replace(Topic), #{qos => QoS}} || {Topic, QoS} <- Topics],
self() ! {subscribe, TopicFilters}.
unload(_) ->
emqx_hooks:del('session.created', fun ?MODULE:on_session_created/3).
emqx_hooks:del('client.connected', {?MODULE, on_client_connected}).
%%------------------------------------------------------------------------------
%% Internal functions

19
src/emqx_mod_sup.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,11 +12,14 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_mod_sup).
-behaviour(supervisor).
-include("types.hrl").
-export([ start_link/0
, start_child/1
, start_child/2
@ -25,8 +29,14 @@
-export([init/1]).
%% Helper macro for declaring children of supervisor
-define(CHILD(Mod, Type), {Mod, {Mod, start_link, []}, permanent, 5000, Type, [Mod]}).
-define(CHILD(Mod, Type), #{id => Mod,
start => {Mod, start_link, []},
restart => permanent,
shutdown => 5000,
type => Type,
modules => [Mod]}).
-spec(start_link() -> startlink_ret()).
start_link() ->
supervisor:start_link({local, ?MODULE}, ?MODULE, []).
@ -43,9 +53,10 @@ stop_child(ChildId) ->
Error -> Error
end.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Supervisor callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
{ok, {{one_for_one, 10, 100}, []}}.

27
src/emqx_modules.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_modules).
@ -22,20 +24,25 @@
, unload/0
]).
%% @doc Load all the extended modules.
-spec(load() -> ok).
load() ->
ok = emqx_mod_acl_internal:load([]),
lists:foreach(
fun({Mod, Env}) ->
ok = Mod:load(Env),
?LOG(info, "Load ~s module successfully.", [Mod])
end, emqx_config:get_env(modules, [])).
lists:foreach(fun load/1, modules()).
load({Mod, Env}) ->
ok = Mod:load(Env),
?LOG(info, "Load ~s module successfully.", [Mod]).
modules() ->
emqx_config:get_env(modules, []).
%% @doc Unload all the extended modules.
-spec(unload() -> ok).
unload() ->
ok = emqx_mod_acl_internal:unload([]),
lists:foreach(
fun({Mod, Env}) ->
Mod:unload(Env) end,
emqx_config:get_env(modules, [])).
lists:foreach(fun unload/1, modules()).
unload({Mod, Env}) ->
Mod:unload(Env).

55
src/emqx_mountpoint.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,13 +12,12 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_mountpoint).
-include("emqx.hrl").
-include("logger.hrl").
-logger_header("[Mountpoint]").
-include("types.hrl").
-export([ mount/2
, unmount/2
@ -25,37 +25,50 @@
-export([replvar/2]).
-type(mountpoint() :: binary()).
-export_type([mountpoint/0]).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
-type(mountpoint() :: binary()).
-spec(mount(maybe(mountpoint()), Any) -> Any
when Any :: emqx_types:topic()
| emqx_types:message()
| emqx_types:topic_filters()).
mount(undefined, Any) ->
Any;
mount(MountPoint, Topic) when is_binary(Topic) ->
prefix(MountPoint, Topic);
mount(MountPoint, Msg = #message{topic = Topic}) ->
Msg#message{topic = <<MountPoint/binary, Topic/binary>>};
Msg#message{topic = prefix(MountPoint, Topic)};
mount(MountPoint, TopicFilters) when is_list(TopicFilters) ->
[{<<MountPoint/binary, Topic/binary>>, SubOpts} || {Topic, SubOpts} <- TopicFilters].
[{prefix(MountPoint, Topic), SubOpts} || {Topic, SubOpts} <- TopicFilters].
unmount(undefined, Msg) ->
Msg;
%% @private
-compile({inline, [prefix/2]}).
prefix(MountPoint, Topic) ->
<<MountPoint/binary, Topic/binary>>.
-spec(unmount(maybe(mountpoint()), Any) -> Any
when Any :: emqx_types:topic()
| emqx_types:message()).
unmount(undefined, Any) ->
Any;
unmount(MountPoint, Topic) when is_binary(Topic) ->
case string:prefix(Topic, MountPoint) of
nomatch -> Topic;
Topic1 -> Topic1
end;
unmount(MountPoint, Msg = #message{topic = Topic}) ->
try split_binary(Topic, byte_size(MountPoint)) of
{MountPoint, Topic1} -> Msg#message{topic = Topic1}
catch
_Error:Reason ->
?LOG(error, "Unmount error : ~p", [Reason]),
Msg
case string:prefix(Topic, MountPoint) of
nomatch -> Msg;
Topic1 -> Msg#message{topic = Topic1}
end.
-spec(replvar(maybe(mountpoint()), map()) -> maybe(mountpoint())).
replvar(undefined, _Vars) ->
undefined;
replvar(MountPoint, #{client_id := ClientId, username := Username}) ->
lists:foldl(fun feed_var/2, MountPoint, [{<<"%c">>, ClientId}, {<<"%u">>, Username}]).
lists:foldl(fun feed_var/2, MountPoint,
[{<<"%c">>, ClientId}, {<<"%u">>, Username}]).
feed_var({<<"%c">>, ClientId}, MountPoint) ->
emqx_topic:feed_var(<<"%c">>, ClientId, MountPoint);

206
src/emqx_mqtt_caps.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,146 +12,139 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc MQTTv5 capabilities
%% @doc MQTTv5 Capabilities
-module(emqx_mqtt_caps).
-include("emqx.hrl").
-include("emqx_mqtt.hrl").
-include("types.hrl").
-export([ check_pub/2
, check_sub/2
, get_caps/1
, check_sub/3
]).
-export([ get_caps/1
, get_caps/2
]).
-export([default_caps/0]).
-export([default/0]).
-export_type([caps/0]).
-type(caps() :: #{max_packet_size => integer(),
max_clientid_len => integer(),
max_topic_alias => integer(),
max_topic_levels => integer(),
max_qos_allowed => emqx_mqtt_types:qos(),
mqtt_retain_available => boolean(),
mqtt_shared_subscription => boolean(),
mqtt_wildcard_subscription => boolean()}).
-export_type([caps/0]).
max_qos_allowed => emqx_types:qos(),
retain_available => boolean(),
wildcard_subscription => boolean(),
subscription_identifiers => boolean(),
shared_subscription => boolean()
}).
-define(UNLIMITED, 0).
-define(DEFAULT_CAPS, [{max_packet_size, ?MAX_PACKET_SIZE},
{max_clientid_len, ?MAX_CLIENTID_LEN},
{max_topic_alias, ?UNLIMITED},
{max_topic_levels, ?UNLIMITED},
{max_qos_allowed, ?QOS_2},
{mqtt_retain_available, true},
{mqtt_shared_subscription, true},
{mqtt_wildcard_subscription, true}]).
-define(PUBCAP_KEYS, [max_qos_allowed,
mqtt_retain_available,
max_topic_alias
-define(PUBCAP_KEYS, [max_topic_alias,
max_qos_allowed,
retain_available
]).
-define(SUBCAP_KEYS, [max_qos_allowed,
max_topic_levels,
mqtt_shared_subscription,
mqtt_wildcard_subscription]).
-spec(check_pub(emqx_types:zone(), map()) -> ok | {error, emqx_mqtt_types:reason_code()}).
check_pub(Zone, Props) when is_map(Props) ->
do_check_pub(Props, maps:to_list(get_caps(Zone, publish))).
-define(SUBCAP_KEYS, [max_topic_levels,
max_qos_allowed,
wildcard_subscription,
shared_subscription
]).
do_check_pub(_Props, []) ->
ok;
do_check_pub(Props = #{qos := QoS}, [{max_qos_allowed, MaxQoS}|Caps]) ->
case QoS > MaxQoS of
true -> {error, ?RC_QOS_NOT_SUPPORTED};
false -> do_check_pub(Props, Caps)
end;
do_check_pub(Props = #{ topic_alias := TopicAlias}, [{max_topic_alias, MaxTopicAlias}| Caps]) ->
case TopicAlias =< MaxTopicAlias andalso TopicAlias > 0 of
false -> {error, ?RC_TOPIC_ALIAS_INVALID};
true -> do_check_pub(Props, Caps)
end;
do_check_pub(#{retain := true}, [{mqtt_retain_available, false}|_Caps]) ->
-define(DEFAULT_CAPS, #{max_packet_size => ?MAX_PACKET_SIZE,
max_clientid_len => ?MAX_CLIENTID_LEN,
max_topic_alias => ?UNLIMITED,
max_topic_levels => ?UNLIMITED,
max_qos_allowed => ?QOS_2,
retain_available => true,
wildcard_subscription => true,
subscription_identifiers => true,
shared_subscription => true
}).
-spec(check_pub(emqx_types:zone(),
#{qos => emqx_types:qos(),
retain => boolean()})
-> ok_or_error(emqx_types:reason_code())).
check_pub(Zone, Flags) when is_map(Flags) ->
do_check_pub(Flags, get_caps(Zone, publish)).
do_check_pub(#{qos := QoS}, #{max_qos_allowed := MaxQoS})
when QoS > MaxQoS ->
{error, ?RC_QOS_NOT_SUPPORTED};
do_check_pub(#{retain := true}, #{retain_available := false}) ->
{error, ?RC_RETAIN_NOT_SUPPORTED};
do_check_pub(Props, [{max_topic_alias, _} | Caps]) ->
do_check_pub(Props, Caps);
do_check_pub(Props, [{mqtt_retain_available, _}|Caps]) ->
do_check_pub(Props, Caps).
do_check_pub(#{topic_alias := TopicAlias},
#{max_topic_alias := MaxTopicAlias})
when 0 == TopicAlias; TopicAlias >= MaxTopicAlias ->
{error, ?RC_TOPIC_ALIAS_INVALID};
do_check_pub(_Flags, _Caps) -> ok.
-spec(check_sub(emqx_types:zone(), emqx_mqtt_types:topic_filters())
-> {ok | error, emqx_mqtt_types:topic_filters()}).
check_sub(Zone, TopicFilters) ->
Caps = maps:to_list(get_caps(Zone, subscribe)),
lists:foldr(fun({Topic, Opts}, {Ok, Result}) ->
case check_sub(Topic, Opts, Caps) of
{ok, Opts1} ->
{Ok, [{Topic, Opts1}|Result]};
{error, Opts1} ->
{error, [{Topic, Opts1}|Result]}
end
end, {ok, []}, TopicFilters).
-spec(check_sub(emqx_types:zone(),
emqx_types:topic(),
emqx_types:subopts())
-> ok_or_error(emqx_types:reason_code())).
check_sub(Zone, Topic, SubOpts) ->
Caps = get_caps(Zone, subscribe),
Flags = lists:foldl(
fun(max_topic_levels, Map) ->
Map#{topic_levels => emqx_topic:levels(Topic)};
(wildcard_subscription, Map) ->
Map#{is_wildcard => emqx_topic:wildcard(Topic)};
(shared_subscription, Map) ->
Map#{is_shared => maps:is_key(share, SubOpts)};
(_Key, Map) -> Map %% Ignore
end, #{}, maps:keys(Caps)),
do_check_sub(Flags, Caps).
check_sub(_Topic, Opts, []) ->
{ok, Opts};
check_sub(Topic, Opts = #{qos := QoS}, [{max_qos_allowed, MaxQoS}|Caps]) ->
check_sub(Topic, Opts#{qos := min(QoS, MaxQoS)}, Caps);
check_sub(Topic, Opts, [{mqtt_shared_subscription, true}|Caps]) ->
check_sub(Topic, Opts, Caps);
check_sub(Topic, Opts, [{mqtt_shared_subscription, false}|Caps]) ->
case maps:is_key(share, Opts) of
true ->
{error, Opts#{rc := ?RC_SHARED_SUBSCRIPTIONS_NOT_SUPPORTED}};
false -> check_sub(Topic, Opts, Caps)
end;
check_sub(Topic, Opts, [{mqtt_wildcard_subscription, true}|Caps]) ->
check_sub(Topic, Opts, Caps);
check_sub(Topic, Opts, [{mqtt_wildcard_subscription, false}|Caps]) ->
case emqx_topic:wildcard(Topic) of
true ->
{error, Opts#{rc := ?RC_WILDCARD_SUBSCRIPTIONS_NOT_SUPPORTED}};
false -> check_sub(Topic, Opts, Caps)
end;
check_sub(Topic, Opts, [{max_topic_levels, ?UNLIMITED}|Caps]) ->
check_sub(Topic, Opts, Caps);
check_sub(Topic, Opts, [{max_topic_levels, Limit}|Caps]) ->
case emqx_topic:levels(Topic) of
Levels when Levels > Limit ->
{error, Opts#{rc := ?RC_TOPIC_FILTER_INVALID}};
_ -> check_sub(Topic, Opts, Caps)
end.
do_check_sub(#{topic_levels := Levels}, #{max_topic_levels := Limit})
when Limit > 0, Levels > Limit ->
{error, ?RC_TOPIC_FILTER_INVALID};
do_check_sub(#{is_wildcard := true}, #{wildcard_subscription := false}) ->
{error, ?RC_WILDCARD_SUBSCRIPTIONS_NOT_SUPPORTED};
do_check_sub(#{is_shared := true}, #{shared_subscription := false}) ->
{error, ?RC_SHARED_SUBSCRIPTIONS_NOT_SUPPORTED};
do_check_sub(_Flags, _Caps) -> ok.
default_caps() ->
?DEFAULT_CAPS.
-spec(get_caps(emqx_zone:zone()) -> caps()).
get_caps(Zone) ->
with_env(Zone, '$mqtt_caps', fun all_caps/1).
-spec(get_caps(emqx_zone:zone(), publish|subscribe) -> caps()).
get_caps(Zone, publish) ->
with_env(Zone, '$mqtt_pub_caps',
fun() ->
filter_caps(?PUBCAP_KEYS, get_caps(Zone))
end);
with_env(Zone, '$mqtt_pub_caps', fun pub_caps/1);
get_caps(Zone, subscribe) ->
with_env(Zone, '$mqtt_sub_caps',
fun() ->
filter_caps(?SUBCAP_KEYS, get_caps(Zone))
end).
with_env(Zone, '$mqtt_sub_caps', fun sub_caps/1).
get_caps(Zone) ->
with_env(Zone, '$mqtt_caps',
fun() ->
maps:from_list([{Cap, emqx_zone:get_env(Zone, Cap, Def)}
|| {Cap, Def} <- ?DEFAULT_CAPS])
end).
pub_caps(Zone) ->
filter_caps(?PUBCAP_KEYS, get_caps(Zone)).
sub_caps(Zone) ->
filter_caps(?SUBCAP_KEYS, get_caps(Zone)).
all_caps(Zone) ->
maps:map(fun(Cap, Def) ->
emqx_zone:get_env(Zone, Cap, Def)
end, ?DEFAULT_CAPS).
filter_caps(Keys, Caps) ->
maps:filter(fun(Key, _Val) -> lists:member(Key, Keys) end, Caps).
with_env(Zone, Key, InitFun) ->
case emqx_zone:get_env(Zone, Key) of
undefined -> Caps = InitFun(),
undefined ->
Caps = InitFun(Zone),
ok = emqx_zone:set_env(Zone, Key, Caps),
Caps;
ZoneCaps -> ZoneCaps
Caps -> Caps
end.
-spec(default() -> caps()).
default() -> ?DEFAULT_CAPS.

4
src/emqx_mqtt_props.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc MQTT5 Properties
-module(emqx_mqtt_props).

43
src/emqx_mqtt_types.erl
View File

@ -1,43 +0,0 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
-module(emqx_mqtt_types).
-include("emqx_mqtt.hrl").
-export_type([version/0, qos/0, qos_name/0]).
-export_type([connack/0, reason_code/0]).
-export_type([properties/0, subopts/0]).
-export_type([topic_filters/0]).
-export_type([packet_id/0, packet_type/0, packet/0]).
-type(qos() :: ?QOS_0 | ?QOS_1 | ?QOS_2).
-type(version() :: ?MQTT_PROTO_V3 | ?MQTT_PROTO_V4 | ?MQTT_PROTO_V5).
-type(qos_name() :: qos0 | at_most_once |
qos1 | at_least_once |
qos2 | exactly_once).
-type(packet_type() :: ?RESERVED..?AUTH).
-type(connack() :: ?CONNACK_ACCEPT..?CONNACK_AUTH).
-type(reason_code() :: 0..16#FF).
-type(packet_id() :: 1..16#FFFF).
-type(properties() :: #{atom() => term()}).
-type(subopts() :: #{rh := 0 | 1 | 2,
rap := 0 | 1,
nl := 0 | 1,
qos := qos(),
rc => reason_code()
}).
-type(topic_filters() :: [{emqx_topic:topic(), subopts()}]).
-type(packet() :: #mqtt_packet{}).

6
src/emqx_mqueue.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,7 +12,9 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc A Simple in-memory message queue.
%%
%% Notice that MQTT is not a (on-disk) persistent messaging queue.
@ -42,6 +45,7 @@
%% unless `max_len' is set to `0' which implies (`infinity').
%%
%% @end
%%--------------------------------------------------------------------
-module(emqx_mqueue).

86
src/emqx_os_mon.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_os_mon).
@ -18,19 +20,10 @@
-include("logger.hrl").
-logger_header("[OS Monitor]").
-logger_header("[OS_MON]").
-export([start_link/1]).
%% gen_server callbacks
-export([ init/1
, handle_call/3
, handle_cast/2
, handle_info/2
, terminate/2
, code_change/3
]).
-export([ get_cpu_check_interval/0
, set_cpu_check_interval/1
, get_cpu_high_watermark/0
@ -45,17 +38,26 @@
, set_procmem_high_watermark/1
]).
-define(OS_MON, ?MODULE).
%% gen_server callbacks
-export([ init/1
, handle_call/3
, handle_cast/2
, handle_info/2
, terminate/2
, code_change/3
]).
-include("emqx.hrl").
%%------------------------------------------------------------------------------
%% API
%%------------------------------------------------------------------------------
-define(OS_MON, ?MODULE).
start_link(Opts) ->
gen_server:start_link({local, ?OS_MON}, ?MODULE, [Opts], []).
%%--------------------------------------------------------------------
%% API
%%--------------------------------------------------------------------
get_cpu_check_interval() ->
call(get_cpu_check_interval).
@ -92,12 +94,14 @@ get_procmem_high_watermark() ->
set_procmem_high_watermark(Float) ->
memsup:set_procmem_high_watermark(Float).
%%------------------------------------------------------------------------------
call(Req) ->
gen_server:call(?OS_MON, Req, infinity).
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([Opts]) ->
_ = ?compat_windows(cpu_sup:util(), windows),
set_mem_check_interval(proplists:get_value(mem_check_interval, Opts, 60)),
set_sysmem_high_watermark(proplists:get_value(sysmem_high_watermark, Opts, 0.70)),
set_procmem_high_watermark(proplists:get_value(procmem_high_watermark, Opts, 0.05)),
@ -109,49 +113,56 @@ init([Opts]) ->
handle_call(get_cpu_check_interval, _From, State) ->
{reply, maps:get(cpu_check_interval, State, undefined), State};
handle_call({set_cpu_check_interval, Seconds}, _From, State) ->
{reply, ok, State#{cpu_check_interval := Seconds}};
handle_call(get_cpu_high_watermark, _From, State) ->
{reply, maps:get(cpu_high_watermark, State, undefined), State};
handle_call({set_cpu_high_watermark, Float}, _From, State) ->
{reply, ok, State#{cpu_high_watermark := Float}};
handle_call(get_cpu_low_watermark, _From, State) ->
{reply, maps:get(cpu_low_watermark, State, undefined), State};
handle_call({set_cpu_low_watermark, Float}, _From, State) ->
{reply, ok, State#{cpu_low_watermark := Float}};
handle_call(_Request, _From, State) ->
{reply, ok, State}.
handle_call(Req, _From, State) ->
?LOG(error, "Unexpected call: ~p", [Req]),
{reply, ignored, State}.
handle_cast(_Request, State) ->
handle_cast(Msg, State) ->
?LOG(error, "Unexpected cast: ~p", [Msg]),
{noreply, State}.
handle_info({timeout, Timer, check}, State = #{timer := Timer,
cpu_high_watermark := CPUHighWatermark,
cpu_low_watermark := CPULowWatermark,
is_cpu_alarm_set := IsCPUAlarmSet}) ->
case ?compat_windows(cpu_sup:util(), windows) of
0 ->
{noreply, State#{timer := undefined}};
NState =
case emqx_vm:cpu_util() of %% TODO: should be improved?
0 -> State#{timer := undefined};
{error, Reason} ->
?LOG(error, "Failed to get cpu utilization: ~p", [Reason]),
{noreply, ensure_check_timer(State)};
windows ->
{noreply, State};
ensure_check_timer(State);
Busy when Busy / 100 >= CPUHighWatermark ->
alarm_handler:set_alarm({cpu_high_watermark, Busy}),
{noreply, ensure_check_timer(State#{is_cpu_alarm_set := true})};
ensure_check_timer(State#{is_cpu_alarm_set := true});
Busy when Busy / 100 < CPULowWatermark ->
case IsCPUAlarmSet of
true -> alarm_handler:clear_alarm(cpu_high_watermark);
false -> ok
end,
{noreply, ensure_check_timer(State#{is_cpu_alarm_set := false})};
_Busy ->
{noreply, ensure_check_timer(State)}
end.
ensure_check_timer(State#{is_cpu_alarm_set := false});
_Busy -> ensure_check_timer(State)
end,
{noreply, NState};
handle_info(Info, State) ->
?LOG(error, "unexpected info: ~p", [Info]),
{noreply, State}.
terminate(_Reason, #{timer := Timer}) ->
emqx_misc:cancel_timer(Timer).
@ -159,11 +170,12 @@ terminate(_Reason, #{timer := Timer}) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
call(Req) ->
gen_server:call(?OS_MON, Req, infinity).
%%--------------------------------------------------------------------
ensure_check_timer(State = #{cpu_check_interval := Interval}) ->
State#{timer := emqx_misc:start_timer(timer:seconds(Interval), check)}.
case erlang:system_info(system_architecture) of
"x86_64-pc-linux-musl" -> State;
_ -> State#{timer := emqx_misc:start_timer(timer:seconds(Interval), check)}
end.

24
src/emqx_packet.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_packet).
@ -27,7 +29,7 @@
]).
%% @doc Protocol name of version
-spec(protocol_name(emqx_mqtt_types:version()) -> binary()).
-spec(protocol_name(emqx_types:version()) -> binary()).
protocol_name(?MQTT_PROTO_V3) ->
<<"MQIsdp">>;
protocol_name(?MQTT_PROTO_V4) ->
@ -36,14 +38,15 @@ protocol_name(?MQTT_PROTO_V5) ->
<<"MQTT">>.
%% @doc Name of MQTT packet type
-spec(type_name(emqx_mqtt_types:packet_type()) -> atom()).
-spec(type_name(emqx_types:packet_type()) -> atom()).
type_name(Type) when Type > ?RESERVED andalso Type =< ?AUTH ->
lists:nth(Type, ?TYPE_NAMES).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Validate MQTT Packet
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(validate(emqx_types:packet()) -> true).
validate(?SUBSCRIBE_PACKET(_PacketId, _Properties, [])) ->
error(topic_filters_invalid);
validate(?SUBSCRIBE_PACKET(PacketId, Properties, TopicFilters)) ->
@ -110,7 +113,8 @@ validate_qos(QoS) when ?QOS_0 =< QoS, QoS =< ?QOS_2 ->
validate_qos(_) -> error(bad_qos).
%% @doc From message to packet
-spec(from_message(emqx_mqtt_types:packet_id(), emqx_types:message()) -> emqx_mqtt_types:packet()).
-spec(from_message(emqx_types:packet_id(), emqx_types:message())
-> emqx_types:packet()).
from_message(PacketId, #message{qos = QoS, flags = Flags, headers = Headers,
topic = Topic, payload = Payload}) ->
Flags1 = if Flags =:= undefined ->
@ -138,7 +142,7 @@ publish_props(Headers) ->
'Message-Expiry-Interval'], Headers).
%% @doc Message from Packet
-spec(to_message(emqx_types:credentials(), emqx_mqtt_types:packet())
-spec(to_message(emqx_types:client(), emqx_ypes:packet())
-> emqx_types:message()).
to_message(#{client_id := ClientId, username := Username, peername := Peername},
#mqtt_packet{header = #mqtt_packet_header{type = ?PUBLISH,
@ -173,7 +177,7 @@ merge_props(Headers, Props) ->
maps:merge(Headers, Props).
%% @doc Format packet
-spec(format(emqx_mqtt_types:packet()) -> iolist()).
-spec(format(emqx_types:packet()) -> iolist()).
format(#mqtt_packet{header = Header, variable = Variable, payload = Payload}) ->
format_header(Header, format_variable(Variable, Payload)).
@ -233,11 +237,11 @@ format_variable(#mqtt_packet_puback{packet_id = PacketId}) ->
format_variable(#mqtt_packet_subscribe{packet_id = PacketId,
topic_filters = TopicFilters}) ->
io_lib:format("PacketId=~p, TopicFilters=~p", [PacketId, TopicFilters]);
io_lib:format("PacketId=~p, TopicFilters=~0p", [PacketId, TopicFilters]);
format_variable(#mqtt_packet_unsubscribe{packet_id = PacketId,
topic_filters = Topics}) ->
io_lib:format("PacketId=~p, TopicFilters=~p", [PacketId, Topics]);
io_lib:format("PacketId=~p, TopicFilters=~0p", [PacketId, Topics]);
format_variable(#mqtt_packet_suback{packet_id = PacketId,
reason_codes = ReasonCodes}) ->

10
src/emqx_pd.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc The utility functions for erlang process dictionary.
-module(emqx_pd).
@ -22,6 +24,12 @@
, reset_counter/1
]).
-compile({inline,
[ update_counter/2
, get_counter/1
, reset_counter/1
]}).
-type(key() :: term()).
-spec(update_counter(key(), number()) -> maybe(number())).

12
src/emqx_plugins.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_plugins).
@ -30,9 +32,9 @@
, load_expand_plugin/1
]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Init plugins' config
-spec(init() -> ok).
@ -47,8 +49,8 @@ init() ->
init_config(CfgFile) ->
{ok, [AppsEnv]} = file:consult(CfgFile),
lists:foreach(fun({AppName, Envs}) ->
[application:set_env(AppName, Par, Val) || {Par, Val} <- Envs]
lists:foreach(fun({App, Envs}) ->
[application:set_env(App, Par, Val) || {Par, Val} <- Envs]
end, AppsEnv).
%% @doc Load all plugins when the broker started.

54
src/emqx_pmon.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_pmon).
@ -30,50 +32,53 @@
-export([count/1]).
-type(pmon() :: {?MODULE, map()}).
-export_type([pmon/0]).
%%------------------------------------------------------------------------------
-opaque(pmon() :: {?MODULE, map()}).
-define(PMON(Map), {?MODULE, Map}).
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(new() -> pmon()).
new() ->
{?MODULE, maps:new()}.
new() -> ?PMON(maps:new()).
-spec(monitor(pid(), pmon()) -> pmon()).
monitor(Pid, PM) ->
?MODULE:monitor(Pid, undefined, PM).
monitor(Pid, PMon) ->
?MODULE:monitor(Pid, undefined, PMon).
-spec(monitor(pid(), term(), pmon()) -> pmon()).
monitor(Pid, Val, {?MODULE, PM}) ->
{?MODULE, case maps:is_key(Pid, PM) of
true -> PM;
false -> Ref = erlang:monitor(process, Pid),
maps:put(Pid, {Ref, Val}, PM)
end}.
monitor(Pid, Val, PMon = ?PMON(Map)) ->
case maps:is_key(Pid, Map) of
true -> PMon;
false ->
Ref = erlang:monitor(process, Pid),
?PMON(maps:put(Pid, {Ref, Val}, Map))
end.
-spec(demonitor(pid(), pmon()) -> pmon()).
demonitor(Pid, {?MODULE, PM}) ->
{?MODULE, case maps:find(Pid, PM) of
demonitor(Pid, PMon = ?PMON(Map)) ->
case maps:find(Pid, Map) of
{ok, {Ref, _Val}} ->
%% flush
_ = erlang:demonitor(Ref, [flush]),
maps:remove(Pid, PM);
error -> PM
end}.
?PMON(maps:remove(Pid, Map));
error -> PMon
end.
-spec(find(pid(), pmon()) -> error | {ok, term()}).
find(Pid, {?MODULE, PM}) ->
case maps:find(Pid, PM) of
find(Pid, ?PMON(Map)) ->
case maps:find(Pid, Map) of
{ok, {_Ref, Val}} ->
{ok, Val};
error -> error
end.
-spec(erase(pid(), pmon()) -> pmon()).
erase(Pid, {?MODULE, PM}) ->
{?MODULE, maps:remove(Pid, PM)}.
erase(Pid, ?PMON(Map)) ->
?PMON(maps:remove(Pid, Map)).
-spec(erase_all([pid()], pmon()) -> {[{pid(), term()}], pmon()}).
erase_all(Pids, PMon0) ->
@ -87,6 +92,5 @@ erase_all(Pids, PMon0) ->
end, {[], PMon0}, Pids).
-spec(count(pmon()) -> non_neg_integer()).
count({?MODULE, PM}) ->
maps:size(PM).
count(?PMON(Map)) -> maps:size(Map).

16
src/emqx_pool.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_pool).
@ -47,9 +49,9 @@
-type(task() :: fun() | mfa() | {fun(), Args :: list(any())}).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Start pool.
-spec(start_link(atom(), pos_integer()) -> startlink_ret()).
@ -87,9 +89,9 @@ cast(Msg) ->
worker() ->
gproc_pool:pick_worker(?POOL).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([Pool, Id]) ->
true = gproc_pool:connect_worker(Pool, {Pool, Id}),
@ -123,9 +125,9 @@ terminate(_Reason, #{pool := Pool, id := Id}) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
run({M, F, A}) ->
erlang:apply(M, F, A);

16
src/emqx_pool_sup.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_pool_sup).
@ -44,7 +46,8 @@ spec(ChildId, Args) ->
start_link() ->
start_link(?POOL, random, {?POOL, start_link, []}).
-spec(start_link(atom() | tuple(), atom(), mfa()) -> {ok, pid()} | {error, term()}).
-spec(start_link(atom() | tuple(), atom(), mfa())
-> {ok, pid()} | {error, term()}).
start_link(Pool, Type, MFA) ->
start_link(Pool, Type, emqx_vm:schedulers(), MFA).
@ -54,11 +57,16 @@ start_link(Pool, Type, Size, MFA) ->
supervisor:start_link(?MODULE, [Pool, Type, Size, MFA]).
init([Pool, Type, Size, {M, F, Args}]) ->
ensure_pool(Pool, Type, [{size, Size}]),
ok = ensure_pool(Pool, Type, [{size, Size}]),
{ok, {{one_for_one, 10, 3600}, [
begin
ensure_pool_worker(Pool, {Pool, I}, I),
{{M, I}, {M, F, [Pool, I | Args]}, transient, 5000, worker, [M]}
#{id => {M, I},
start => {M, F, [Pool, I | Args]},
restart => transient,
shutdown => 5000,
type => worker,
modules => [M]}
end || I <- lists:seq(1, Size)]}}.
ensure_pool(Pool, Type, Opts) ->

4
src/emqx_pqueue.erl
View File

@ -57,6 +57,8 @@
, highest/1
]).
-export_type([q/0]).
%%----------------------------------------------------------------------------
-type(priority() :: integer() | 'infinity').
@ -64,8 +66,6 @@
-type(pqueue() :: squeue() | {pqueue, [{priority(), squeue()}]}).
-type(q() :: pqueue()).
-export_type([q/0]).
%%----------------------------------------------------------------------------
-spec(new() -> pqueue()).

1596
src/emqx_protocol.erl
View File

File diff suppressed because it is too large Load Diff

4
src/emqx_psk.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_psk).

13
src/emqx_reason_codes.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,15 +12,18 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc MQTT5 reason codes
-module(emqx_reason_codes).
-include("emqx_mqtt.hrl").
-export([ name/2
-export([ name/1
, name/2
, text/1
, connack_error/1
, puback/1
]).
-export([compat/2]).
@ -159,3 +163,8 @@ connack_error(server_busy) -> ?RC_SERVER_BUSY;
connack_error(banned) -> ?RC_BANNED;
connack_error(bad_authentication_method) -> ?RC_BAD_AUTHENTICATION_METHOD;
connack_error(_) -> ?RC_NOT_AUTHORIZED.
%%TODO: This function should be removed.
puback([]) -> ?RC_NO_MATCHING_SUBSCRIBERS;
puback(L) when is_list(L) -> ?RC_SUCCESS.

60
src/emqx_router.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_router).
@ -64,16 +66,16 @@
-type(group() :: binary()).
-type(destination() :: node() | {group(), node()}).
-type(dest() :: node() | {group(), node()}).
-define(ROUTE, emqx_route).
-define(ROUTE_TAB, emqx_route).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Mnesia bootstrap
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
mnesia(boot) ->
ok = ekka_mnesia:create_table(?ROUTE, [
ok = ekka_mnesia:create_table(?ROUTE_TAB, [
{type, bag},
{ram_copies, [node()]},
{record_name, route},
@ -81,26 +83,26 @@ mnesia(boot) ->
{storage_properties, [{ets, [{read_concurrency, true},
{write_concurrency, true}]}]}]);
mnesia(copy) ->
ok = ekka_mnesia:copy_table(?ROUTE).
ok = ekka_mnesia:copy_table(?ROUTE_TAB).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Start a router
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(start_link(atom(), pos_integer()) -> startlink_ret()).
start_link(Pool, Id) ->
gen_server:start_link({local, emqx_misc:proc_name(?MODULE, Id)},
?MODULE, [Pool, Id], [{hibernate_after, 1000}]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Route APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(add_route(emqx_topic:topic()) -> ok | {error, term()}).
add_route(Topic) when is_binary(Topic) ->
add_route(Topic, node()).
-spec(add_route(emqx_topic:topic(), destination()) -> ok | {error, term()}).
-spec(add_route(emqx_topic:topic(), dest()) -> ok | {error, term()}).
add_route(Topic, Dest) when is_binary(Topic) ->
call(pick(Topic), {add_route, Topic, Dest}).
@ -108,7 +110,7 @@ add_route(Topic, Dest) when is_binary(Topic) ->
do_add_route(Topic) when is_binary(Topic) ->
do_add_route(Topic, node()).
-spec(do_add_route(emqx_topic:topic(), destination()) -> ok | {error, term()}).
-spec(do_add_route(emqx_topic:topic(), dest()) -> ok | {error, term()}).
do_add_route(Topic, Dest) when is_binary(Topic) ->
Route = #route{topic = Topic, dest = Dest},
case lists:member(Route, lookup_routes(Topic)) of
@ -140,17 +142,17 @@ match_trie(Topic) ->
-spec(lookup_routes(emqx_topic:topic()) -> [emqx_types:route()]).
lookup_routes(Topic) ->
ets:lookup(?ROUTE, Topic).
ets:lookup(?ROUTE_TAB, Topic).
-spec(has_routes(emqx_topic:topic()) -> boolean()).
has_routes(Topic) when is_binary(Topic) ->
ets:member(?ROUTE, Topic).
ets:member(?ROUTE_TAB, Topic).
-spec(delete_route(emqx_topic:topic()) -> ok | {error, term()}).
delete_route(Topic) when is_binary(Topic) ->
delete_route(Topic, node()).
-spec(delete_route(emqx_topic:topic(), destination()) -> ok | {error, term()}).
-spec(delete_route(emqx_topic:topic(), dest()) -> ok | {error, term()}).
delete_route(Topic, Dest) when is_binary(Topic) ->
call(pick(Topic), {delete_route, Topic, Dest}).
@ -158,7 +160,7 @@ delete_route(Topic, Dest) when is_binary(Topic) ->
do_delete_route(Topic) when is_binary(Topic) ->
do_delete_route(Topic, node()).
-spec(do_delete_route(emqx_topic:topic(), destination()) -> ok | {error, term()}).
-spec(do_delete_route(emqx_topic:topic(), dest()) -> ok | {error, term()}).
do_delete_route(Topic, Dest) ->
Route = #route{topic = Topic, dest = Dest},
case emqx_topic:wildcard(Topic) of
@ -168,7 +170,7 @@ do_delete_route(Topic, Dest) ->
-spec(topics() -> list(emqx_topic:topic())).
topics() ->
mnesia:dirty_all_keys(?ROUTE).
mnesia:dirty_all_keys(?ROUTE_TAB).
%% @doc Print routes to a topic
-spec(print_routes(emqx_topic:topic()) -> ok).
@ -183,9 +185,9 @@ call(Router, Msg) ->
pick(Topic) ->
gproc_pool:pick_worker(router_pool, Topic).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([Pool, Id]) ->
true = gproc_pool:connect_worker(Pool, {Pool, Id}),
@ -217,30 +219,30 @@ terminate(_Reason, #{pool := Pool, id := Id}) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
insert_direct_route(Route) ->
mnesia:async_dirty(fun mnesia:write/3, [?ROUTE, Route, sticky_write]).
mnesia:async_dirty(fun mnesia:write/3, [?ROUTE_TAB, Route, sticky_write]).
insert_trie_route(Route = #route{topic = Topic}) ->
case mnesia:wread({?ROUTE, Topic}) of
case mnesia:wread({?ROUTE_TAB, Topic}) of
[] -> emqx_trie:insert(Topic);
_ -> ok
end,
mnesia:write(?ROUTE, Route, sticky_write).
mnesia:write(?ROUTE_TAB, Route, sticky_write).
delete_direct_route(Route) ->
mnesia:async_dirty(fun mnesia:delete_object/3, [?ROUTE, Route, sticky_write]).
mnesia:async_dirty(fun mnesia:delete_object/3, [?ROUTE_TAB, Route, sticky_write]).
delete_trie_route(Route = #route{topic = Topic}) ->
case mnesia:wread({?ROUTE, Topic}) of
case mnesia:wread({?ROUTE_TAB, Topic}) of
[Route] -> %% Remove route and trie
ok = mnesia:delete_object(?ROUTE, Route, sticky_write),
ok = mnesia:delete_object(?ROUTE_TAB, Route, sticky_write),
emqx_trie:delete(Topic);
[_|_] -> %% Remove route only
mnesia:delete_object(?ROUTE, Route, sticky_write);
mnesia:delete_object(?ROUTE_TAB, Route, sticky_write);
[] -> ok
end.

20
src/emqx_router_helper.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_router_helper).
@ -51,9 +53,9 @@
-define(ROUTING_NODE, emqx_routing_node).
-define(LOCK, {?MODULE, cleanup_routes}).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Mnesia bootstrap
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
mnesia(boot) ->
ok = ekka_mnesia:create_table(?ROUTING_NODE, [
@ -66,9 +68,9 @@ mnesia(boot) ->
mnesia(copy) ->
ok = ekka_mnesia:copy_table(?ROUTING_NODE).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Starts the router helper
-spec(start_link() -> startlink_ret()).
@ -86,9 +88,9 @@ monitor(Node) when is_atom(Node) ->
false -> mnesia:dirty_write(?ROUTING_NODE, #routing_node{name = Node})
end.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
ok = ekka:monitor(membership),
@ -154,9 +156,9 @@ terminate(_Reason, _State) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
stats_fun() ->
case ets:info(?ROUTE, size) of

4
src/emqx_router_sup.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_router_sup).

14
src/emqx_rpc.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
%% @doc wrap gen_rpc?
-module(emqx_rpc).
@ -20,6 +22,11 @@
, multicall/4
]).
-compile({inline,
[ rpc_node/1
, rpc_nodes/1
]}).
-define(RPC, gen_rpc).
call(Node, Mod, Fun, Args) ->
@ -32,7 +39,8 @@ cast(Node, Mod, Fun, Args) ->
filter_result(?RPC:cast(rpc_node(Node), Mod, Fun, Args)).
rpc_node(Node) ->
{Node, erlang:system_info(scheduler_id)}.
{ok, ClientNum} = application:get_env(gen_rpc, tcp_client_num),
{Node, rand:uniform(ClientNum)}.
rpc_nodes(Nodes) ->
rpc_nodes(Nodes, []).
@ -42,9 +50,9 @@ rpc_nodes([], Acc) ->
rpc_nodes([Node | Nodes], Acc) ->
rpc_nodes(Nodes, [rpc_node(Node) | Acc]).
filter_result({Error, Reason})
when Error =:= badrpc; Error =:= badtcp ->
{badrpc, Reason};
filter_result(Delivery) ->
Delivery.

12
src/emqx_sequence.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_sequence).
@ -21,17 +23,17 @@
, delete/1
]).
-export_type([seqid/0]).
-type(key() :: term()).
-type(name() :: atom()).
-type(seqid() :: non_neg_integer()).
-export_type([seqid/0]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Create a sequence.
-spec(create(name()) -> ok).

1464
src/emqx_session.erl
View File

File diff suppressed because it is too large Load Diff

267
src/emqx_session_sup.erl
View File

@ -1,267 +0,0 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
-module(emqx_session_sup).
-behaviour(gen_server).
-include("logger.hrl").
-include("types.hrl").
-logger_header("[Session Supervisor]").
-export([start_link/1]).
-export([ start_session/1
, count_sessions/0
]).
%% gen_server callbacks
-export([ init/1
, handle_call/3
, handle_cast/2
, handle_info/2
, terminate/2
, code_change/3
]).
-type(shutdown() :: brutal_kill | infinity | pos_integer()).
-record(state,
{ sessions :: #{pid() => emqx_types:client_id()}
, mfargs :: mfa()
, shutdown :: shutdown()
, clean_down :: fun()
}).
-define(SUP, ?MODULE).
-define(BATCH_EXIT, 100000).
%% @doc Start session supervisor.
-spec(start_link(map()) -> startlink_ret()).
start_link(SessSpec) when is_map(SessSpec) ->
gen_server:start_link({local, ?SUP}, ?MODULE, [SessSpec], []).
%%------------------------------------------------------------------------------
%% API
%%------------------------------------------------------------------------------
%% @doc Start a session.
-spec(start_session(map()) -> startlink_ret()).
start_session(SessAttrs) ->
gen_server:call(?SUP, {start_session, SessAttrs}, infinity).
%% @doc Count sessions.
-spec(count_sessions() -> non_neg_integer()).
count_sessions() ->
gen_server:call(?SUP, count_sessions, infinity).
%%------------------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
init([Spec]) ->
process_flag(trap_exit, true),
MFA = maps:get(start, Spec),
Shutdown = maps:get(shutdown, Spec, brutal_kill),
CleanDown = maps:get(clean_down, Spec, undefined),
State = #state{sessions = #{},
mfargs = MFA,
shutdown = Shutdown,
clean_down = CleanDown
},
{ok, State}.
handle_call({start_session, SessAttrs = #{client_id := ClientId}}, _From,
State = #state{sessions = SessMap, mfargs = {M, F, Args}}) ->
try erlang:apply(M, F, [SessAttrs | Args]) of
{ok, Pid} ->
reply({ok, Pid}, State#state{sessions = maps:put(Pid, ClientId, SessMap)});
ignore ->
reply(ignore, State);
{error, Reason} ->
reply({error, Reason}, State)
catch
_:Error:Stk ->
?LOG(error, "Failed to start session ~p: ~p, stacktrace:~n~p",
[ClientId, Error, Stk]),
reply({error, Error}, State)
end;
handle_call(count_sessions, _From, State = #state{sessions = SessMap}) ->
{reply, maps:size(SessMap), State};
handle_call(Req, _From, State) ->
?LOG(error, "Unexpected call: ~p", [Req]),
{reply, ignored, State}.
handle_cast(Msg, State) ->
?LOG(error, "Unexpected cast: ~p", [Msg]),
{noreply, State}.
handle_info({'EXIT', Pid, _Reason}, State = #state{sessions = SessMap, clean_down = CleanDown}) ->
SessPids = [Pid | drain_exit(?BATCH_EXIT, [])],
{SessItems, SessMap1} = erase_all(SessPids, SessMap),
(CleanDown =:= undefined)
orelse emqx_pool:async_submit(
fun lists:foreach/2, [CleanDown, SessItems]),
{noreply, State#state{sessions = SessMap1}};
handle_info(Info, State) ->
?LOG(error, "Unexpected info: ~p", [Info]),
{noreply, State}.
terminate(_Reason, State) ->
terminate_children(State).
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
drain_exit(0, Acc) ->
lists:reverse(Acc);
drain_exit(Cnt, Acc) ->
receive
{'EXIT', Pid, _Reason} ->
drain_exit(Cnt - 1, [Pid|Acc])
after 0 ->
lists:reverse(Acc)
end.
erase_all(Pids, Map) ->
lists:foldl(
fun(Pid, {Acc, M}) ->
case maps:take(Pid, M) of
{Val, M1} ->
{[{Val, Pid}|Acc], M1};
error ->
{Acc, M}
end
end, {[], Map}, Pids).
terminate_children(State = #state{sessions = SessMap, shutdown = Shutdown}) ->
{Pids, EStack0} = monitor_children(SessMap),
Sz = sets:size(Pids),
EStack =
case Shutdown of
brutal_kill ->
sets:fold(fun(P, _) -> exit(P, kill) end, ok, Pids),
wait_children(Shutdown, Pids, Sz, undefined, EStack0);
infinity ->
sets:fold(fun(P, _) -> exit(P, shutdown) end, ok, Pids),
wait_children(Shutdown, Pids, Sz, undefined, EStack0);
Time when is_integer(Time) ->
sets:fold(fun(P, _) -> exit(P, shutdown) end, ok, Pids),
TRef = erlang:start_timer(Time, self(), kill),
wait_children(Shutdown, Pids, Sz, TRef, EStack0)
end,
%% Unroll stacked errors and report them
dict:fold(fun(Reason, Pid, _) ->
report_error(connection_shutdown_error, Reason, Pid, State)
end, ok, EStack).
monitor_children(SessMap) ->
lists:foldl(
fun(Pid, {Pids, EStack}) ->
case monitor_child(Pid) of
ok ->
{sets:add_element(Pid, Pids), EStack};
{error, normal} ->
{Pids, EStack};
{error, Reason} ->
{Pids, dict:append(Reason, Pid, EStack)}
end
end, {sets:new(), dict:new()}, maps:keys(SessMap)).
%% Help function to shutdown/2 switches from link to monitor approach
monitor_child(Pid) ->
%% Do the monitor operation first so that if the child dies
%% before the monitoring is done causing a 'DOWN'-message with
%% reason noproc, we will get the real reason in the 'EXIT'-message
%% unless a naughty child has already done unlink...
erlang:monitor(process, Pid),
unlink(Pid),
receive
%% If the child dies before the unlik we must empty
%% the mail-box of the 'EXIT'-message and the 'DOWN'-message.
{'EXIT', Pid, Reason} ->
receive
{'DOWN', _, process, Pid, _} ->
{error, Reason}
end
after 0 ->
%% If a naughty child did unlink and the child dies before
%% monitor the result will be that shutdown/2 receives a
%% 'DOWN'-message with reason noproc.
%% If the child should die after the unlink there
%% will be a 'DOWN'-message with a correct reason
%% that will be handled in shutdown/2.
ok
end.
wait_children(_Shutdown, _Pids, 0, undefined, EStack) ->
EStack;
wait_children(_Shutdown, _Pids, 0, TRef, EStack) ->
%% If the timer has expired before its cancellation, we must empty the
%% mail-box of the 'timeout'-message.
erlang:cancel_timer(TRef),
receive
{timeout, TRef, kill} ->
EStack
after 0 ->
EStack
end;
%%TODO: Copied from supervisor.erl, rewrite it later.
wait_children(brutal_kill, Pids, Sz, TRef, EStack) ->
receive
{'DOWN', _MRef, process, Pid, killed} ->
wait_children(brutal_kill, sets:del_element(Pid, Pids), Sz-1, TRef, EStack);
{'DOWN', _MRef, process, Pid, Reason} ->
wait_children(brutal_kill, sets:del_element(Pid, Pids),
Sz-1, TRef, dict:append(Reason, Pid, EStack))
end;
wait_children(Shutdown, Pids, Sz, TRef, EStack) ->
receive
{'DOWN', _MRef, process, Pid, shutdown} ->
wait_children(Shutdown, sets:del_element(Pid, Pids), Sz-1, TRef, EStack);
{'DOWN', _MRef, process, Pid, normal} ->
wait_children(Shutdown, sets:del_element(Pid, Pids), Sz-1, TRef, EStack);
{'DOWN', _MRef, process, Pid, Reason} ->
wait_children(Shutdown, sets:del_element(Pid, Pids), Sz-1,
TRef, dict:append(Reason, Pid, EStack));
{timeout, TRef, kill} ->
sets:fold(fun(P, _) -> exit(P, kill) end, ok, Pids),
wait_children(Shutdown, Pids, Sz-1, undefined, EStack)
end.
report_error(Error, Reason, Pid, #state{mfargs = MFA}) ->
SupName = list_to_atom("esockd_connection_sup - " ++ pid_to_list(self())),
ErrorMsg = [{supervisor, SupName},
{errorContext, Error},
{reason, Reason},
{offender, [{pid, Pid},
{name, connection},
{mfargs, MFA}]}],
error_logger:error_report(supervisor_report, ErrorMsg).
reply(Repy, State) ->
{reply, Repy, State}.

41
src/emqx_shared_sub.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_shared_sub).
@ -67,11 +69,12 @@
-define(no_ack, no_ack).
-record(state, {pmon}).
-record(emqx_shared_subscription, {group, topic, subpid}).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Mnesia bootstrap
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
mnesia(boot) ->
ok = ekka_mnesia:create_table(?TAB, [
@ -83,9 +86,9 @@ mnesia(boot) ->
mnesia(copy) ->
ok = ekka_mnesia:copy_table(?TAB).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% API
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(start_link() -> startlink_ret()).
start_link() ->
@ -103,19 +106,19 @@ record(Group, Topic, SubPid) ->
#emqx_shared_subscription{group = Group, topic = Topic, subpid = SubPid}.
-spec(dispatch(emqx_topic:group(), emqx_topic:topic(), emqx_types:delivery())
-> emqx_types:delivery()).
-> emqx_types:deliver_result()).
dispatch(Group, Topic, Delivery) ->
dispatch(Group, Topic, Delivery, _FailedSubs = []).
dispatch(Group, Topic, Delivery = #delivery{message = Msg, results = Results}, FailedSubs) ->
dispatch(Group, Topic, Delivery = #delivery{message = Msg}, FailedSubs) ->
#message{from = ClientId} = Msg,
case pick(strategy(), ClientId, Group, Topic, FailedSubs) of
false ->
Delivery;
{error, no_subscribers};
{Type, SubPid} ->
case do_dispatch(SubPid, Topic, Msg, Type) of
ok ->
Delivery#delivery{results = [{dispatch, {Group, Topic}, 1} | Results]};
ok;
{error, _Reason} ->
%% Failed to dispatch to this sub, try next.
dispatch(Group, Topic, Delivery, [SubPid | FailedSubs])
@ -132,7 +135,7 @@ ack_enabled() ->
do_dispatch(SubPid, Topic, Msg, _Type) when SubPid =:= self() ->
%% Deadlock otherwise
_ = erlang:send(SubPid, {dispatch, Topic, Msg}),
_ = erlang:send(SubPid, {deliver, Topic, Msg}),
ok;
do_dispatch(SubPid, Topic, Msg, Type) ->
dispatch_per_qos(SubPid, Topic, Msg, Type).
@ -140,18 +143,18 @@ do_dispatch(SubPid, Topic, Msg, Type) ->
%% return either 'ok' (when everything is fine) or 'error'
dispatch_per_qos(SubPid, Topic, #message{qos = ?QOS_0} = Msg, _Type) ->
%% For QoS 0 message, send it as regular dispatch
_ = erlang:send(SubPid, {dispatch, Topic, Msg}),
_ = erlang:send(SubPid, {deliver, Topic, Msg}),
ok;
dispatch_per_qos(SubPid, Topic, Msg, retry) ->
%% Retry implies all subscribers nack:ed, send again without ack
_ = erlang:send(SubPid, {dispatch, Topic, Msg}),
_ = erlang:send(SubPid, {deliver, Topic, Msg}),
ok;
dispatch_per_qos(SubPid, Topic, Msg, fresh) ->
case ack_enabled() of
true ->
dispatch_with_ack(SubPid, Topic, Msg);
false ->
_ = erlang:send(SubPid, {dispatch, Topic, Msg}),
_ = erlang:send(SubPid, {deliver, Topic, Msg}),
ok
end.
@ -159,7 +162,7 @@ dispatch_with_ack(SubPid, Topic, Msg) ->
%% For QoS 1/2 message, expect an ack
Ref = erlang:monitor(process, SubPid),
Sender = self(),
_ = erlang:send(SubPid, {dispatch, Topic, with_ack_ref(Msg, {Sender, Ref})}),
_ = erlang:send(SubPid, {deliver, Topic, with_ack_ref(Msg, {Sender, Ref})}),
Timeout = case Msg#message.qos of
?QOS_1 -> timer:seconds(?SHARED_SUB_QOS1_DISPATCH_TIMEOUT_SECONDS);
?QOS_2 -> infinity
@ -275,12 +278,12 @@ do_pick_subscriber(Group, Topic, round_robin, _ClientId, Count) ->
subscribers(Group, Topic) ->
ets:select(?TAB, [{{emqx_shared_subscription, Group, Topic, '$1'}, [], ['$1']}]).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
mnesia:subscribe({table, ?TAB, simple}),
{ok, _} = mnesia:subscribe({table, ?TAB, simple}),
{atomic, PMon} = mnesia:transaction(fun init_monitors/0),
ok = emqx_tables:new(?SHARED_SUBS, [protected, bag]),
ok = emqx_tables:new(?ALIVE_SUBS, [protected, set, {read_concurrency, true}]),
@ -345,9 +348,9 @@ terminate(_Reason, _State) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% keep track of alive remote pids
maybe_insert_alive_tab(Pid) when ?IS_LOCAL_PID(Pid) -> ok;

298
src/emqx_sm.erl
View File

@ -1,298 +0,0 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
-module(emqx_sm).
-behaviour(gen_server).
-include("emqx.hrl").
-include("logger.hrl").
-include("types.hrl").
-logger_header("[SM]").
%% APIs
-export([start_link/0]).
-export([ open_session/1
, close_session/1
, resume_session/2
, discard_session/1
, discard_session/2
, register_session/1
, register_session/2
, unregister_session/1
, unregister_session/2
]).
-export([ get_session_attrs/1
, get_session_attrs/2
, set_session_attrs/2
, set_session_attrs/3
, get_session_stats/1
, get_session_stats/2
, set_session_stats/2
, set_session_stats/3
]).
-export([lookup_session_pids/1]).
%% Internal functions for rpc
-export([dispatch/3]).
%% Internal function for stats
-export([stats_fun/0]).
%% Internal function for emqx_session_sup
-export([clean_down/1]).
%% gen_server callbacks
-export([ init/1
, handle_call/3
, handle_cast/2
, handle_info/2
, terminate/2
, code_change/3
]).
-define(SM, ?MODULE).
%% ETS Tables for session management.
-define(SESSION_TAB, emqx_session).
-define(SESSION_P_TAB, emqx_session_p).
-define(SESSION_ATTRS_TAB, emqx_session_attrs).
-define(SESSION_STATS_TAB, emqx_session_stats).
-define(BATCH_SIZE, 100000).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
-spec(start_link() -> startlink_ret()).
start_link() ->
gen_server:start_link({local, ?SM}, ?MODULE, [], []).
%% @doc Open a session.
-spec(open_session(map()) -> {ok, pid()} | {ok, pid(), boolean()} | {error, term()}).
open_session(SessAttrs = #{clean_start := true, client_id := ClientId, conn_pid := ConnPid}) ->
CleanStart = fun(_) ->
ok = discard_session(ClientId, ConnPid),
emqx_session_sup:start_session(SessAttrs)
end,
emqx_sm_locker:trans(ClientId, CleanStart);
open_session(SessAttrs = #{clean_start := false, client_id := ClientId}) ->
ResumeStart = fun(_) ->
case resume_session(ClientId, SessAttrs) of
{ok, SessPid} ->
{ok, SessPid, true};
{error, not_found} ->
emqx_session_sup:start_session(SessAttrs)
end
end,
emqx_sm_locker:trans(ClientId, ResumeStart).
%% @doc Discard all the sessions identified by the ClientId.
-spec(discard_session(emqx_types:client_id()) -> ok).
discard_session(ClientId) when is_binary(ClientId) ->
discard_session(ClientId, self()).
-spec(discard_session(emqx_types:client_id(), pid()) -> ok).
discard_session(ClientId, ConnPid) when is_binary(ClientId) ->
lists:foreach(
fun(SessPid) ->
try emqx_session:discard(SessPid, ConnPid)
catch
_:Error:_Stk ->
unregister_session(ClientId, SessPid),
?LOG(warning, "Failed to discard ~p: ~p", [SessPid, Error])
end
end, lookup_session_pids(ClientId)).
%% @doc Try to resume a session.
-spec(resume_session(emqx_types:client_id(), map()) -> {ok, pid()} | {error, term()}).
resume_session(ClientId, SessAttrs = #{conn_pid := ConnPid}) ->
case lookup_session_pids(ClientId) of
[] -> {error, not_found};
[SessPid] ->
ok = emqx_session:resume(SessPid, SessAttrs),
{ok, SessPid};
SessPids ->
[SessPid|StalePids] = lists:reverse(SessPids),
?LOG(error, "More than one session found: ~p", [SessPids]),
lists:foreach(fun(StalePid) ->
catch emqx_session:discard(StalePid, ConnPid)
end, StalePids),
ok = emqx_session:resume(SessPid, SessAttrs),
{ok, SessPid}
end.
%% @doc Close a session.
-spec(close_session(emqx_types:client_id() | pid()) -> ok).
close_session(ClientId) when is_binary(ClientId) ->
case lookup_session_pids(ClientId) of
[] -> ok;
[SessPid] -> close_session(SessPid);
SessPids -> lists:foreach(fun close_session/1, SessPids)
end;
close_session(SessPid) when is_pid(SessPid) ->
emqx_session:close(SessPid).
%% @doc Register a session.
-spec(register_session(emqx_types:client_id()) -> ok).
register_session(ClientId) when is_binary(ClientId) ->
register_session(ClientId, self()).
-spec(register_session(emqx_types:client_id(), pid()) -> ok).
register_session(ClientId, SessPid) when is_binary(ClientId), is_pid(SessPid) ->
Session = {ClientId, SessPid},
true = ets:insert(?SESSION_TAB, Session),
emqx_sm_registry:register_session(Session).
%% @doc Unregister a session
-spec(unregister_session(emqx_types:client_id()) -> ok).
unregister_session(ClientId) when is_binary(ClientId) ->
unregister_session(ClientId, self()).
-spec(unregister_session(emqx_types:client_id(), pid()) -> ok).
unregister_session(ClientId, SessPid) when is_binary(ClientId), is_pid(SessPid) ->
Session = {ClientId, SessPid},
true = ets:delete(?SESSION_STATS_TAB, Session),
true = ets:delete(?SESSION_ATTRS_TAB, Session),
true = ets:delete_object(?SESSION_P_TAB, Session),
true = ets:delete_object(?SESSION_TAB, Session),
emqx_sm_registry:unregister_session(Session).
%% @doc Get session attrs
-spec(get_session_attrs(emqx_types:client_id()) -> list(emqx_session:attr())).
get_session_attrs(ClientId) when is_binary(ClientId) ->
case lookup_session_pids(ClientId) of
[] -> [];
[SessPid|_] -> get_session_attrs(ClientId, SessPid)
end.
-spec(get_session_attrs(emqx_types:client_id(), pid()) -> list(emqx_session:attr())).
get_session_attrs(ClientId, SessPid) when is_binary(ClientId), is_pid(SessPid) ->
emqx_tables:lookup_value(?SESSION_ATTRS_TAB, {ClientId, SessPid}, []).
%% @doc Set session attrs
-spec(set_session_attrs(emqx_types:client_id(), list(emqx_session:attr())) -> true).
set_session_attrs(ClientId, SessAttrs) when is_binary(ClientId) ->
set_session_attrs(ClientId, self(), SessAttrs).
-spec(set_session_attrs(emqx_types:client_id(), pid(), list(emqx_session:attr())) -> true).
set_session_attrs(ClientId, SessPid, SessAttrs) when is_binary(ClientId), is_pid(SessPid) ->
Session = {ClientId, SessPid},
true = ets:insert(?SESSION_ATTRS_TAB, {Session, SessAttrs}),
proplists:get_value(clean_start, SessAttrs, true) orelse ets:insert(?SESSION_P_TAB, Session).
%% @doc Get session stats
-spec(get_session_stats(emqx_types:client_id()) -> list(emqx_stats:stats())).
get_session_stats(ClientId) when is_binary(ClientId) ->
case lookup_session_pids(ClientId) of
[] -> [];
[SessPid|_] ->
get_session_stats(ClientId, SessPid)
end.
-spec(get_session_stats(emqx_types:client_id(), pid()) -> list(emqx_stats:stats())).
get_session_stats(ClientId, SessPid) when is_binary(ClientId) ->
emqx_tables:lookup_value(?SESSION_STATS_TAB, {ClientId, SessPid}, []).
%% @doc Set session stats
-spec(set_session_stats(emqx_types:client_id(), emqx_stats:stats()) -> true).
set_session_stats(ClientId, Stats) when is_binary(ClientId) ->
set_session_stats(ClientId, self(), Stats).
-spec(set_session_stats(emqx_types:client_id(), pid(), emqx_stats:stats()) -> true).
set_session_stats(ClientId, SessPid, Stats) when is_binary(ClientId), is_pid(SessPid) ->
ets:insert(?SESSION_STATS_TAB, {{ClientId, SessPid}, Stats}).
%% @doc Lookup session pid.
-spec(lookup_session_pids(emqx_types:client_id()) -> list(pid())).
lookup_session_pids(ClientId) ->
case emqx_sm_registry:is_enabled() of
true -> emqx_sm_registry:lookup_session(ClientId);
false ->
case emqx_tables:lookup_value(?SESSION_TAB, ClientId) of
undefined -> [];
SessPid when is_pid(SessPid) -> [SessPid]
end
end.
%% @doc Dispatch a message to the session.
-spec(dispatch(emqx_types:client_id(), emqx_topic:topic(), emqx_types:message()) -> any()).
dispatch(ClientId, Topic, Msg) ->
case lookup_session_pids(ClientId) of
[SessPid|_] when is_pid(SessPid) ->
SessPid ! {dispatch, Topic, Msg};
[] ->
emqx_hooks:run('message.dropped', [#{client_id => ClientId}, Msg])
end.
%%------------------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
init([]) ->
TabOpts = [public, set, {write_concurrency, true}],
ok = emqx_tables:new(?SESSION_TAB, [{read_concurrency, true} | TabOpts]),
ok = emqx_tables:new(?SESSION_P_TAB, TabOpts),
ok = emqx_tables:new(?SESSION_ATTRS_TAB, TabOpts),
ok = emqx_tables:new(?SESSION_STATS_TAB, TabOpts),
ok = emqx_stats:update_interval(sess_stats, fun ?MODULE:stats_fun/0),
{ok, #{}}.
handle_call(Req, _From, State) ->
?LOG(error, "Unexpected call: ~p", [Req]),
{reply, ignored, State}.
handle_cast(Msg, State) ->
?LOG(error, "Unexpected cast: ~p", [Msg]),
{noreply, State}.
handle_info(Info, State) ->
?LOG(error, "Unexpected info: ~p", [Info]),
{noreply, State}.
terminate(_Reason, _State) ->
emqx_stats:cancel_update(sess_stats).
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
clean_down(Session = {ClientId, SessPid}) ->
case ets:member(?SESSION_TAB, ClientId)
orelse ets:member(?SESSION_ATTRS_TAB, Session) of
true ->
unregister_session(ClientId, SessPid);
false -> ok
end.
stats_fun() ->
safe_update_stats(?SESSION_TAB, 'sessions.count', 'sessions.max'),
safe_update_stats(?SESSION_P_TAB, 'sessions.persistent.count', 'sessions.persistent.max').
safe_update_stats(Tab, Stat, MaxStat) ->
case ets:info(Tab, size) of
undefined -> ok;
Size -> emqx_stats:setstat(Stat, MaxStat, Size)
end.

64
src/emqx_sm_sup.erl
View File

@ -1,64 +0,0 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
-module(emqx_sm_sup).
-behaviour(supervisor).
-export([start_link/0]).
-export([init/1]).
start_link() ->
supervisor:start_link({local, ?MODULE}, ?MODULE, []).
init([]) ->
%% Session locker
Locker = #{id => locker,
start => {emqx_sm_locker, start_link, []},
restart => permanent,
shutdown => 5000,
type => worker,
modules => [emqx_sm_locker]
},
%% Session registry
Registry = #{id => registry,
start => {emqx_sm_registry, start_link, []},
restart => permanent,
shutdown => 5000,
type => worker,
modules => [emqx_sm_registry]
},
%% Session Manager
Manager = #{id => manager,
start => {emqx_sm, start_link, []},
restart => permanent,
shutdown => 5000,
type => worker,
modules => [emqx_sm]
},
%% Session Sup
SessSpec = #{start => {emqx_session, start_link, []},
shutdown => brutal_kill,
clean_down => fun emqx_sm:clean_down/1
},
SessionSup = #{id => session_sup,
start => {emqx_session_sup, start_link, [SessSpec ]},
restart => transient,
shutdown => infinity,
type => supervisor,
modules => [emqx_session_sup]
},
{ok, {{rest_for_one, 10, 3600}, [Locker, Registry, Manager, SessionSup]}}.

26
src/emqx_stats.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_stats).
@ -49,14 +51,18 @@
, code_change/3
]).
-export_type([stats/0]).
-record(update, {name, countdown, interval, func}).
-record(state, {timer, updates :: [#update{}], tick_ms :: timeout()}).
-record(state, {
timer :: reference(),
updates :: [#update{}],
tick_ms :: timeout()
}).
-type(stats() :: list({atom(), non_neg_integer()})).
-export_type([stats/0]).
%% Connection stats
-define(CONNECTION_STATS, [
'connections.count', % current connections
@ -168,9 +174,9 @@ rec(Name, Secs, UpFun) ->
cast(Msg) ->
gen_server:cast(?SERVER, Msg).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init(#{tick_ms := TickMs}) ->
ok = emqx_tables:new(?TAB, [public, set, {write_concurrency, true}]),
@ -201,7 +207,8 @@ handle_cast({setstat, Stat, MaxStat, Val}, State) ->
safe_update_element(Stat, Val),
{noreply, State};
handle_cast({update_interval, Update = #update{name = Name}}, State = #state{updates = Updates}) ->
handle_cast({update_interval, Update = #update{name = Name}},
State = #state{updates = Updates}) ->
case lists:keyfind(Name, #update.name, Updates) of
#update{} ->
?LOG(warning, "Duplicated update: ~s", [Name]),
@ -242,9 +249,9 @@ terminate(_Reason, #state{timer = TRef}) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
safe_update_element(Key, Val) ->
try ets:update_element(?TAB, Key, {2, Val}) of
@ -256,3 +263,4 @@ safe_update_element(Key, Val) ->
error:badarg ->
?LOG(warning, "Update ~p to ~p failed", [Key, Val])
end.

66
src/emqx_sup.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,11 +12,14 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_sup).
-behaviour(supervisor).
-include("types.hrl").
-export([ start_link/0
, start_child/1
, start_child/2
@ -28,29 +32,28 @@
| {ok, supervisor:child(), term()}
| {error, term()}).
-define(SUPERVISOR, ?MODULE).
-define(SUP, ?MODULE).
%%--------------------------------------------------------------------
%% API
%%--------------------------------------------------------------------
-spec(start_link() -> startlink_ret()).
start_link() ->
supervisor:start_link({local, ?SUPERVISOR}, ?MODULE, []).
supervisor:start_link({local, ?SUP}, ?MODULE, []).
-spec(start_child(supervisor:child_spec()) -> startchild_ret()).
start_child(ChildSpec) when is_tuple(ChildSpec) ->
supervisor:start_child(?SUPERVISOR, ChildSpec).
supervisor:start_child(?SUP, ChildSpec).
-spec(start_child(module(), worker | supervisor) -> startchild_ret()).
start_child(Mod, worker) ->
start_child(worker_spec(Mod));
start_child(Mod, supervisor) ->
start_child(supervisor_spec(Mod)).
start_child(Mod, Type) ->
start_child(child_spec(Mod, Type)).
-spec(stop_child(supervisor:child_id()) -> ok | {error, term()}).
stop_child(ChildId) ->
case supervisor:terminate_child(?SUPERVISOR, ChildId) of
ok -> supervisor:delete_child(?SUPERVISOR, ChildId);
case supervisor:terminate_child(?SUP, ChildId) of
ok -> supervisor:delete_child(?SUP, ChildId);
Error -> Error
end.
@ -60,30 +63,37 @@ stop_child(ChildId) ->
init([]) ->
%% Kernel Sup
KernelSup = supervisor_spec(emqx_kernel_sup),
KernelSup = child_spec(emqx_kernel_sup, supervisor),
%% Router Sup
RouterSup = supervisor_spec(emqx_router_sup),
RouterSup = child_spec(emqx_router_sup, supervisor),
%% Broker Sup
BrokerSup = supervisor_spec(emqx_broker_sup),
%% Session Manager
SMSup = supervisor_spec(emqx_sm_sup),
%% Connection Manager
CMSup = supervisor_spec(emqx_cm_sup),
BrokerSup = child_spec(emqx_broker_sup, supervisor),
%% CM Sup
CMSup = child_spec(emqx_cm_sup, supervisor),
%% Sys Sup
SysSup = supervisor_spec(emqx_sys_sup),
SysSup = child_spec(emqx_sys_sup, supervisor),
{ok, {{one_for_all, 0, 1},
[KernelSup,
RouterSup,
BrokerSup,
SMSup,
CMSup,
SysSup]}}.
[KernelSup, RouterSup, BrokerSup, CMSup, SysSup]}}.
%%--------------------------------------------------------------------
%% Internal functions
%%--------------------------------------------------------------------
worker_spec(M) ->
{M, {M, start_link, []}, permanent, 30000, worker, [M]}.
supervisor_spec(M) ->
{M, {M, start_link, []}, permanent, infinity, supervisor, [M]}.
child_spec(Mod, supervisor) ->
#{id => Mod,
start => {Mod, start_link, []},
restart => permanent,
shutdown => infinity,
type => supervisor,
modules => [Mod]
};
child_spec(Mod, worker) ->
#{id => Mod,
start => {Mod, start_link, []},
restart => permanent,
shutdown => 15000,
type => worker,
modules => [Mod]
}.

4
src/emqx_sys.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_sys).

14
src/emqx_sys_mon.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_sys_mon).
@ -43,18 +45,14 @@
-define(SYSMON, ?MODULE).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%% @doc Start system monitor
%% @doc Start the system monitor.
-spec(start_link(list(option())) -> startlink_ret()).
start_link(Opts) ->
gen_server:start_link({local, ?SYSMON}, ?MODULE, [Opts], []).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([Opts]) ->
erlang:system_monitor(self(), parse_opt(Opts)),

29
src/emqx_sys_sup.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_sys_sup).
@ -24,23 +26,28 @@ start_link() ->
supervisor:start_link({local, ?MODULE}, ?MODULE, []).
init([]) ->
{ok, {{one_for_one, 10, 100}, [child_spec(emqx_sys, worker),
child_spec(emqx_sys_mon, worker, [emqx_config:get_env(sysmon, [])]),
child_spec(emqx_os_mon, worker, [emqx_config:get_env(os_mon, [])]),
child_spec(emqx_vm_mon, worker, [emqx_config:get_env(vm_mon, [])])]}}.
Childs = [child_spec(emqx_sys),
child_spec(emqx_sys_mon, [config(sysmon)]),
child_spec(emqx_os_mon, [config(os_mon)]),
child_spec(emqx_vm_mon, [config(vm_mon)])],
{ok, {{one_for_one, 10, 100}, Childs}}.
%%--------------------------------------------------------------------
%% Internal functions
%%--------------------------------------------------------------------
child_spec(M, worker) ->
child_spec(M, worker, []).
child_spec(Mod) ->
child_spec(Mod, []).
child_spec(M, worker, A) ->
#{id => M,
start => {M, start_link, A},
child_spec(Mod, Args) ->
#{id => Mod,
start => {Mod, start_link, Args},
restart => permanent,
shutdown => 5000,
type => worker,
modules => [M]}.
modules => [Mod]
}.
config(Name) ->
emqx_config:get_env(Name, []).

45
src/emqx_tables.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,15 +12,25 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_tables).
-export([new/2, delete/1]).
-export([ new/1
, new/2
]).
-export([ lookup_value/2
, lookup_value/3
]).
-export([delete/1]).
%% Create an ets table.
-spec(new(atom()) -> ok).
new(Tab) ->
new(Tab, []).
%% Create a named_table ets.
-spec(new(atom(), list()) -> ok).
new(Tab, Opts) ->
@ -30,25 +41,25 @@ new(Tab, Opts) ->
Tab -> ok
end.
-spec(delete(atom()) -> ok).
%% KV lookup
-spec(lookup_value(ets:tab(), term()) -> any()).
lookup_value(Tab, Key) ->
lookup_value(Tab, Key, undefined).
-spec(lookup_value(ets:tab(), term(), any()) -> any()).
lookup_value(Tab, Key, Def) ->
try ets:lookup_element(Tab, Key, 2)
catch
error:badarg -> Def
end.
%% Delete the ets table.
-spec(delete(ets:tab()) -> ok).
delete(Tab) ->
case ets:info(Tab, name) of
undefined ->
ok;
undefined -> ok;
Tab ->
ets:delete(Tab),
ok
end.
%% KV lookup
-spec(lookup_value(atom(), term()) -> any()).
lookup_value(Tab, Key) ->
lookup_value(Tab, Key, undefined).
-spec(lookup_value(atom(), term(), any()) -> any()).
lookup_value(Tab, Key, Def) ->
try
ets:lookup_element(Tab, Key, 2)
catch
error:badarg -> Def
end.

15
src/emqx_time.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_time).
@ -19,9 +21,16 @@
, now_secs/1
, now_ms/0
, now_ms/1
, ts_from_ms/1
]).
-compile({inline,
[ seed/0
, now_secs/0
, now_secs/1
, now_ms/0
, now_ms/1
]}).
seed() ->
rand:seed(exsplus, erlang:timestamp()).
@ -37,5 +46,3 @@ now_ms() ->
now_ms({MegaSecs, Secs, MicroSecs}) ->
(MegaSecs * 1000000 + Secs) * 1000 + round(MicroSecs/1000).
ts_from_ms(Ms) ->
{Ms div 1000000, Ms rem 1000000, 0}.

73
src/emqx_topic.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,11 +12,10 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_topic).
-include("emqx_mqtt.hrl").
%% APIs
-export([ match/2
, validate/1
@ -33,19 +33,23 @@
, parse/2
]).
-export_type([ group/0
, topic/0
, word/0
, triple/0
]).
-type(group() :: binary()).
-type(topic() :: binary()).
-type(word() :: '' | '+' | '#' | binary()).
-type(words() :: list(word())).
-opaque(triple() :: {root | binary(), word(), binary()}).
-export_type([group/0, topic/0, word/0, triple/0]).
-define(MAX_TOPIC_LEN, 4096).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Is wildcard topic?
-spec(wildcard(topic() | words()) -> true | false).
@ -60,7 +64,7 @@ wildcard(['+'|_]) ->
wildcard([_H|T]) ->
wildcard(T).
%% @doc Match Topic name with filter
%% @doc Match Topic name with filter.
-spec(match(Name, Filter) -> boolean() when
Name :: topic() | words(),
Filter :: topic() | words()).
@ -68,7 +72,7 @@ match(<<$$, _/binary>>, <<$+, _/binary>>) ->
false;
match(<<$$, _/binary>>, <<$#, _/binary>>) ->
false;
match(Name, Filter) when is_binary(Name) and is_binary(Filter) ->
match(Name, Filter) when is_binary(Name), is_binary(Filter) ->
match(words(Name), words(Filter));
match([], []) ->
true;
@ -95,13 +99,15 @@ validate({Type, Topic}) when Type =:= name; Type =:= filter ->
-spec(validate(name | filter, topic()) -> true).
validate(_, <<>>) ->
error(empty_topic);
validate(_, Topic) when is_binary(Topic) and (size(Topic) > ?MAX_TOPIC_LEN) ->
validate(_, Topic) when is_binary(Topic) andalso (size(Topic) > ?MAX_TOPIC_LEN) ->
error(topic_too_long);
validate(filter, Topic) when is_binary(Topic) ->
validate2(words(Topic));
validate(name, Topic) when is_binary(Topic) ->
Words = words(Topic),
validate2(Words) and (not wildcard(Words)).
validate2(Words)
andalso (not wildcard(Words))
orelse error(topic_name_error).
validate2([]) ->
true;
@ -123,7 +129,7 @@ validate3(<<C/utf8, _Rest/binary>>) when C == $#; C == $+; C == 0 ->
validate3(<<_/utf8, Rest/binary>>) ->
validate3(Rest).
%% @doc Topic to triples
%% @doc Topic to triples.
-spec(triples(topic()) -> list(triple())).
triples(Topic) when is_binary(Topic) ->
triples(words(Topic), root, []).
@ -206,27 +212,28 @@ join(Words) ->
end, {true, <<>>}, [bin(W) || W <- Words]),
Bin.
-spec(parse(topic()) -> {topic(), #{}}).
parse(Topic) when is_binary(Topic) ->
parse(Topic, #{}).
-spec(parse(topic() | {topic(), map()}) -> {topic(), #{share => binary()}}).
parse(TopicFilter) when is_binary(TopicFilter) ->
parse(TopicFilter, #{});
parse({TopicFilter, Options}) when is_binary(TopicFilter) ->
parse(TopicFilter, Options).
parse(Topic = <<"$queue/", _/binary>>, #{share := _Group}) ->
error({invalid_topic, Topic});
parse(Topic = <<?SHARE, "/", _/binary>>, #{share := _Group}) ->
error({invalid_topic, Topic});
parse(<<"$queue/", Topic1/binary>>, Options) ->
parse(Topic1, maps:put(share, <<"$queue">>, Options));
parse(Topic = <<?SHARE, "/", Topic1/binary>>, Options) ->
case binary:split(Topic1, <<"/">>) of
[<<>>] -> error({invalid_topic, Topic});
[_] -> error({invalid_topic, Topic});
[Group, Topic2] ->
case binary:match(Group, [<<"/">>, <<"+">>, <<"#">>]) of
nomatch -> {Topic2, maps:put(share, Group, Options)};
_ -> error({invalid_topic, Topic})
-spec(parse(topic(), map()) -> {topic(), map()}).
parse(TopicFilter = <<"$queue/", _/binary>>, #{share := _Group}) ->
error({invalid_topic_filter, TopicFilter});
parse(TopicFilter = <<"$share/", _/binary>>, #{share := _Group}) ->
error({invalid_topic_filter, TopicFilter});
parse(<<"$queue/", TopicFilter/binary>>, Options) ->
parse(TopicFilter, Options#{share => <<"$queue">>});
parse(TopicFilter = <<"$share/", Rest/binary>>, Options) ->
case binary:split(Rest, <<"/">>) of
[_Any] -> error({invalid_topic_filter, TopicFilter});
[ShareName, Filter] ->
case binary:match(ShareName, [<<"+">>, <<"#">>]) of
nomatch -> parse(Filter, Options#{share => ShareName});
_ -> error({invalid_topic_filter, TopicFilter})
end
end;
parse(Topic, Options = #{qos := QoS}) ->
{Topic, Options#{rc => QoS}};
parse(Topic, Options) ->
{Topic, Options}.
parse(TopicFilter, Options) ->
{TopicFilter, Options}.

152
src/emqx_tracer.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,37 +12,23 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_tracer).
-behaviour(gen_server).
-include("emqx.hrl").
-include("logger.hrl").
-logger_header("[Tracer]").
%% APIs
-export([start_link/0]).
-export([ trace/2
, start_trace/3
, lookup_traces/0
, stop_trace/1
]).
%% gen_server callbacks
-export([ init/1
, handle_call/3
, handle_cast/2
, handle_info/2
, terminate/2
, code_change/3
]).
-record(state, {traces}).
-type(trace_who() :: {client_id | topic, binary()}).
-type(trace_who() :: {client_id | topic, binary() | list()}).
-define(TRACER, ?MODULE).
-define(FORMAT, {emqx_logger_formatter,
@ -55,65 +42,62 @@
[peername," "],
[]}]},
msg,"\n"]}}).
-define(TOPIC_TRACE_ID(T), "trace_topic_"++T).
-define(CLIENT_TRACE_ID(C), "trace_clientid_"++C).
-define(TOPIC_TRACE(T), {topic,T}).
-define(CLIENT_TRACE(C), {client_id,C}).
-define(is_log_level(L),
L =:= emergency orelse
L =:= alert orelse
L =:= critical orelse
L =:= error orelse
L =:= warning orelse
L =:= notice orelse
L =:= info orelse
L =:= debug).
%%------------------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
-spec(start_link() -> {ok, pid()} | ignore | {error, term()}).
start_link() ->
gen_server:start_link({local, ?TRACER}, ?MODULE, [], []).
trace(publish, #message{topic = <<"$SYS/", _/binary>>}) ->
%% Dont' trace '$SYS' publish
%% Do not trace '$SYS' publish
ignore;
trace(publish, #message{from = From, topic = Topic, payload = Payload})
when is_binary(From); is_atom(From) ->
emqx_logger:info(#{topic => Topic}, "PUBLISH to ~s: ~p", [Topic, Payload]).
%%------------------------------------------------------------------------------
%% Start/Stop trace
%%------------------------------------------------------------------------------
%% @doc Start to trace client_id or topic.
-spec(start_trace(trace_who(), logger:level(), string()) -> ok | {error, term()}).
start_trace({client_id, ClientId}, Level, LogFile) ->
do_start_trace({client_id, ClientId}, Level, LogFile);
start_trace({topic, Topic}, Level, LogFile) ->
do_start_trace({topic, Topic}, Level, LogFile).
do_start_trace(Who, Level, LogFile) ->
start_trace(Who, all, LogFile) ->
start_trace(Who, debug, LogFile);
start_trace(Who, Level, LogFile) ->
case ?is_log_level(Level) of
true ->
#{level := PrimaryLevel} = logger:get_primary_config(),
try logger:compare_levels(log_level(Level), PrimaryLevel) of
try logger:compare_levels(Level, PrimaryLevel) of
lt ->
{error, io_lib:format("Cannot trace at a log level (~s) lower than the primary log level (~s)", [Level, PrimaryLevel])};
_GtOrEq ->
gen_server:call(?MODULE, {start_trace, Who, Level, LogFile}, 5000)
install_trace_handler(Who, Level, LogFile)
catch
_:Error ->
{error, Error}
end;
false -> {error, {invalid_log_level, Level}}
end.
%% @doc Stop tracing client_id or topic.
-spec(stop_trace(trace_who()) -> ok | {error, term()}).
stop_trace({client_id, ClientId}) ->
gen_server:call(?MODULE, {stop_trace, {client_id, ClientId}});
stop_trace({topic, Topic}) ->
gen_server:call(?MODULE, {stop_trace, {topic, Topic}}).
stop_trace(Who) ->
uninstall_trance_handler(Who).
%% @doc Lookup all traces
-spec(lookup_traces() -> [{Who :: trace_who(), LogFile :: string()}]).
lookup_traces() ->
gen_server:call(?TRACER, lookup_traces).
lists:foldl(fun filter_traces/2, [], emqx_logger:get_log_handlers()).
%%------------------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
init([]) ->
{ok, #state{traces = #{}}}.
handle_call({start_trace, Who, Level, LogFile}, _From, State = #state{traces = Traces}) ->
install_trace_handler(Who, Level, LogFile) ->
case logger:add_handler(handler_id(Who), logger_disk_log_h,
#{level => Level,
formatter => ?FORMAT,
@ -121,54 +105,37 @@ handle_call({start_trace, Who, Level, LogFile}, _From, State = #state{traces = T
config => #{type => halt, file => LogFile},
filter_default => stop,
filters => [{meta_key_filter,
{fun filter_by_meta_key/2, Who} }]}) of
{fun filter_by_meta_key/2, Who}}]})
of
ok ->
?LOG(info, "Start trace for ~p", [Who]),
{reply, ok, State#state{traces = maps:put(Who, {Level, LogFile}, Traces)}};
?LOG(info, "Start trace for ~p", [Who]);
{error, Reason} ->
?LOG(error, "Start trace for ~p failed, error: ~p", [Who, Reason]),
{reply, {error, Reason}, State}
end;
{error, Reason}
end.
handle_call({stop_trace, Who}, _From, State = #state{traces = Traces}) ->
case maps:find(Who, Traces) of
{ok, _LogFile} ->
uninstall_trance_handler(Who) ->
case logger:remove_handler(handler_id(Who)) of
ok ->
?LOG(info, "Stop trace for ~p", [Who]);
{error, Reason} ->
?LOG(error, "Stop trace for ~p failed, error: ~p", [Who, Reason])
end,
{reply, ok, State#state{traces = maps:remove(Who, Traces)}};
error ->
{reply, {error, not_found}, State}
end;
?LOG(error, "Stop trace for ~p failed, error: ~p", [Who, Reason]),
{error, Reason}
end.
handle_call(lookup_traces, _From, State = #state{traces = Traces}) ->
{reply, [{Who, LogFile} || {Who, LogFile} <- maps:to_list(Traces)], State};
filter_traces({Id, Level, Dst}, Acc) ->
case atom_to_list(Id) of
?TOPIC_TRACE_ID(T)->
[{?TOPIC_TRACE(T), {Level,Dst}} | Acc];
?CLIENT_TRACE_ID(C) ->
[{?CLIENT_TRACE(C), {Level,Dst}} | Acc];
_ -> Acc
end.
handle_call(Req, _From, State) ->
?LOG(error, "Unexpected call: ~p", [Req]),
{reply, ignored, State}.
handle_cast(Msg, State) ->
?LOG(error, "Unexpected cast: ~p", [Msg]),
{noreply, State}.
handle_info(Info, State) ->
?LOG(error, "Unexpected info: ~p", [Info]),
{noreply, State}.
terminate(_Reason, _State) ->
ok.
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
handler_id({topic, Topic}) ->
list_to_atom("topic_" ++ binary_to_list(Topic));
handler_id({client_id, ClientId}) ->
list_to_atom("clientid_" ++ binary_to_list(ClientId)).
handler_id(?TOPIC_TRACE(Topic)) ->
list_to_atom(?TOPIC_TRACE_ID(str(Topic)));
handler_id(?CLIENT_TRACE(ClientId)) ->
list_to_atom(?CLIENT_TRACE_ID(str(ClientId))).
filter_by_meta_key(#{meta:=Meta}=LogEvent, {MetaKey, MetaValue}) ->
case maps:find(MetaKey, Meta) of
@ -181,13 +148,6 @@ filter_by_meta_key(#{meta:=Meta}=LogEvent, {MetaKey, MetaValue}) ->
_ -> ignore
end.
log_level(emergency) -> emergency;
log_level(alert) -> alert;
log_level(critical) -> critical;
log_level(error) -> error;
log_level(warning) -> warning;
log_level(notice) -> notice;
log_level(info) -> info;
log_level(debug) -> debug;
log_level(all) -> debug;
log_level(_) -> throw(invalid_log_level).
str(Bin) when is_binary(Bin) -> binary_to_list(Bin);
str(Atom) when is_atom(Atom) -> atom_to_list(Atom);
str(Str) when is_list(Str) -> Str.

60
src/emqx_trie.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_trie).
@ -32,12 +34,12 @@
-export([empty/0]).
%% Mnesia tables
-define(TRIE, emqx_trie).
-define(TRIE_NODE, emqx_trie_node).
-define(TRIE_TAB, emqx_trie).
-define(TRIE_NODE_TAB, emqx_trie_node).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Mnesia bootstrap
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Create or replicate trie tables.
-spec(mnesia(boot | copy) -> ok).
@ -46,13 +48,13 @@ mnesia(boot) ->
StoreProps = [{ets, [{read_concurrency, true},
{write_concurrency, true}]}],
%% Trie table
ok = ekka_mnesia:create_table(?TRIE, [
ok = ekka_mnesia:create_table(?TRIE_TAB, [
{ram_copies, [node()]},
{record_name, trie},
{attributes, record_info(fields, trie)},
{storage_properties, StoreProps}]),
%% Trie node table
ok = ekka_mnesia:create_table(?TRIE_NODE, [
ok = ekka_mnesia:create_table(?TRIE_NODE_TAB, [
{ram_copies, [node()]},
{record_name, trie_node},
{attributes, record_info(fields, trie_node)},
@ -60,18 +62,18 @@ mnesia(boot) ->
mnesia(copy) ->
%% Copy trie table
ok = ekka_mnesia:copy_table(?TRIE),
ok = ekka_mnesia:copy_table(?TRIE_TAB),
%% Copy trie_node table
ok = ekka_mnesia:copy_table(?TRIE_NODE).
ok = ekka_mnesia:copy_table(?TRIE_NODE_TAB).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Trie APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @doc Insert a topic filter into the trie.
-spec(insert(emqx_topic:topic()) -> ok).
insert(Topic) when is_binary(Topic) ->
case mnesia:wread({?TRIE_NODE, Topic}) of
case mnesia:wread({?TRIE_NODE_TAB, Topic}) of
[#trie_node{topic = Topic}] ->
ok;
[TrieNode = #trie_node{topic = undefined}] ->
@ -92,14 +94,14 @@ match(Topic) when is_binary(Topic) ->
%% @doc Lookup a trie node.
-spec(lookup(NodeId :: binary()) -> [#trie_node{}]).
lookup(NodeId) ->
mnesia:read(?TRIE_NODE, NodeId).
mnesia:read(?TRIE_NODE_TAB, NodeId).
%% @doc Delete a topic filter from the trie.
-spec(delete(emqx_topic:topic()) -> ok).
delete(Topic) when is_binary(Topic) ->
case mnesia:wread({?TRIE_NODE, Topic}) of
case mnesia:wread({?TRIE_NODE_TAB, Topic}) of
[#trie_node{edge_count = 0}] ->
ok = mnesia:delete({?TRIE_NODE, Topic}),
ok = mnesia:delete({?TRIE_NODE_TAB, Topic}),
delete_path(lists:reverse(emqx_topic:triples(Topic)));
[TrieNode] ->
write_trie_node(TrieNode#trie_node{topic = undefined});
@ -109,19 +111,19 @@ delete(Topic) when is_binary(Topic) ->
%% @doc Is the trie empty?
-spec(empty() -> boolean()).
empty() ->
ets:info(?TRIE, size) == 0.
ets:info(?TRIE_TAB, size) == 0.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% @private
%% @doc Add a path to the trie.
add_path({Node, Word, Child}) ->
Edge = #trie_edge{node_id = Node, word = Word},
case mnesia:wread({?TRIE_NODE, Node}) of
case mnesia:wread({?TRIE_NODE_TAB, Node}) of
[TrieNode = #trie_node{edge_count = Count}] ->
case mnesia:wread({?TRIE, Edge}) of
case mnesia:wread({?TRIE_TAB, Edge}) of
[] ->
ok = write_trie_node(TrieNode#trie_node{edge_count = Count + 1}),
write_trie(#trie{edge = Edge, node_id = Child});
@ -141,11 +143,11 @@ match_node(NodeId, Words) ->
match_node(NodeId, Words, []).
match_node(NodeId, [], ResAcc) ->
mnesia:read(?TRIE_NODE, NodeId) ++ 'match_#'(NodeId, ResAcc);
mnesia:read(?TRIE_NODE_TAB, NodeId) ++ 'match_#'(NodeId, ResAcc);
match_node(NodeId, [W|Words], ResAcc) ->
lists:foldl(fun(WArg, Acc) ->
case mnesia:read(?TRIE, #trie_edge{node_id = NodeId, word = WArg}) of
case mnesia:read(?TRIE_TAB, #trie_edge{node_id = NodeId, word = WArg}) of
[#trie{node_id = ChildId}] -> match_node(ChildId, Words, Acc);
[] -> Acc
end
@ -154,9 +156,9 @@ match_node(NodeId, [W|Words], ResAcc) ->
%% @private
%% @doc Match node with '#'.
'match_#'(NodeId, ResAcc) ->
case mnesia:read(?TRIE, #trie_edge{node_id = NodeId, word = '#'}) of
case mnesia:read(?TRIE_TAB, #trie_edge{node_id = NodeId, word = '#'}) of
[#trie{node_id = ChildId}] ->
mnesia:read(?TRIE_NODE, ChildId) ++ ResAcc;
mnesia:read(?TRIE_NODE_TAB, ChildId) ++ ResAcc;
[] -> ResAcc
end.
@ -165,10 +167,10 @@ match_node(NodeId, [W|Words], ResAcc) ->
delete_path([]) ->
ok;
delete_path([{NodeId, Word, _} | RestPath]) ->
ok = mnesia:delete({?TRIE, #trie_edge{node_id = NodeId, word = Word}}),
case mnesia:wread({?TRIE_NODE, NodeId}) of
ok = mnesia:delete({?TRIE_TAB, #trie_edge{node_id = NodeId, word = Word}}),
case mnesia:wread({?TRIE_NODE_TAB, NodeId}) of
[#trie_node{edge_count = 1, topic = undefined}] ->
ok = mnesia:delete({?TRIE_NODE, NodeId}),
ok = mnesia:delete({?TRIE_NODE_TAB, NodeId}),
delete_path(RestPath);
[TrieNode = #trie_node{edge_count = 1, topic = _}] ->
write_trie_node(TrieNode#trie_node{edge_count = 0});
@ -180,9 +182,9 @@ delete_path([{NodeId, Word, _} | RestPath]) ->
%% @private
write_trie(Trie) ->
mnesia:write(?TRIE, Trie, write).
mnesia:write(?TRIE_TAB, Trie, write).
%% @private
write_trie_node(TrieNode) ->
mnesia:write(?TRIE_NODE, TrieNode, write).
mnesia:write(?TRIE_NODE_TAB, TrieNode, write).

126
src/emqx_types.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,34 +12,48 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_types).
-include("emqx.hrl").
-include("emqx_mqtt.hrl").
-include("types.hrl").
-export_type([zone/0]).
-export_type([ pubsub/0
, topic/0
, subid/0
, subopts/0
-export_type([ ver/0
, qos/0
, qos_name/0
]).
-export_type([ client_id/0
-export_type([ zone/0
, pubsub/0
, topic/0
, subid/0
]).
-export_type([ conn/0
, client/0
, client_id/0
, username/0
, password/0
, peername/0
, protocol/0
]).
-export_type([ credentials/0
, session/0
-export_type([ connack/0
, subopts/0
, reason_code/0
, properties/0
]).
-export_type([ packet_id/0
, packet_type/0
, packet/0
]).
-export_type([ subscription/0
, subscriber/0
, topic_table/0
, topic_filters/0
]).
-export_type([ payload/0
@ -46,10 +61,13 @@
]).
-export_type([ delivery/0
, deliver_results/0
, publish_result/0
, deliver_result/0
]).
-export_type([route/0]).
-export_type([ route/0
, route_entry/0
]).
-export_type([ alarm/0
, plugin/0
@ -57,19 +75,52 @@
, command/0
]).
-type(zone() :: atom()).
-export_type([ caps/0
, infos/0
, attrs/0
, stats/0
]).
-type(ver() :: ?MQTT_PROTO_V3
| ?MQTT_PROTO_V4
| ?MQTT_PROTO_V5).
-type(qos() :: ?QOS_0 | ?QOS_1 | ?QOS_2).
-type(qos_name() :: qos0 | at_most_once |
qos1 | at_least_once |
qos2 | exactly_once).
-type(zone() :: emqx_zone:zone()).
-type(pubsub() :: publish | subscribe).
-type(topic() :: binary()).
-type(topic() :: emqx_topic:topic()).
-type(subid() :: binary() | atom()).
-type(subopts() :: #{qos := emqx_mqtt_types:qos(),
share => binary(),
-type(conn() :: #{peername := peername(),
sockname := peername(),
peercert := esockd_peercert:peercert(),
conn_mod := module(),
atom() => term()
}).
-type(client() :: #{zone := zone(),
conn_mod := maybe(module()),
peername := peername(),
sockname := peername(),
client_id := client_id(),
username := username(),
peercert := esockd_peercert:peercert(),
is_bridge := boolean(),
is_superuser := boolean(),
mountpoint := maybe(binary()),
ws_cookie := maybe(list()),
password => maybe(binary()),
auth_result => auth_result(),
anonymous => boolean(),
atom() => term()
}).
-type(session() :: #session{}).
-type(client_id() :: binary() | atom()).
-type(username() :: maybe(binary())).
-type(password() :: maybe(binary())).
-type(peername() :: {inet:ip_address(), inet:port_number()}).
-type(protocol() :: mqtt | 'mqtt-sn' | coap | stomp | none | atom()).
-type(auth_result() :: success
| client_identifier_not_valid
| bad_username_or_password
@ -79,29 +130,40 @@
| server_busy
| banned
| bad_authentication_method).
-type(protocol() :: mqtt | 'mqtt-sn' | coap | stomp | none | atom()).
-type(credentials() :: #{zone := zone(),
client_id := client_id(),
username := username(),
sockname := peername(),
peername := peername(),
ws_cookie := undefined | list(),
mountpoint := binary(),
password => binary(),
auth_result => auth_result(),
anonymous => boolean(),
-type(packet_type() :: ?RESERVED..?AUTH).
-type(connack() :: ?CONNACK_ACCEPT..?CONNACK_AUTH).
-type(subopts() :: #{rh := 0 | 1 | 2,
rap := 0 | 1,
nl := 0 | 1,
qos := qos(),
share => binary(),
atom() => term()
}).
-type(reason_code() :: 0..16#FF).
-type(packet_id() :: 1..16#FFFF).
-type(properties() :: #{atom() => term()}).
-type(topic_filters() :: list({topic(), subopts()})).
-type(packet() :: #mqtt_packet{}).
-type(subscription() :: #subscription{}).
-type(subscriber() :: {pid(), subid()}).
-type(topic_table() :: [{topic(), subopts()}]).
-type(payload() :: binary() | iodata()).
-type(message() :: #message{}).
-type(banned() :: #banned{}).
-type(delivery() :: #delivery{}).
-type(deliver_results() :: [{route, node(), topic()} |
{dispatch, topic(), pos_integer()}]).
-type(deliver_result() :: ok | {error, term()}).
-type(publish_result() :: [ {node(), topic(), deliver_result()}
| {share, topic(), deliver_result()}]).
-type(route() :: #route{}).
-type(sub_group() :: tuple() | binary()).
-type(route_entry() :: {topic(), node()} | {topic, sub_group()}).
-type(alarm() :: #alarm{}).
-type(plugin() :: #plugin{}).
-type(command() :: #command{}).
-type(caps() :: emqx_mqtt_caps:caps()).
-type(infos() :: #{atom() => term()}).
-type(attrs() :: #{atom() => term()}).
-type(stats() :: list({atom(), non_neg_integer()})).

37
src/emqx_vm.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_vm).
@ -45,6 +47,8 @@
, get_port_info/1
]).
-export([cpu_util/0]).
-define(UTIL_ALLOCATORS, [temp_alloc,
eheap_alloc,
binary_alloc,
@ -159,8 +163,6 @@
sndbuf,
tos]).
-include("emqx.hrl").
schedulers() ->
erlang:system_info(schedulers).
@ -169,9 +171,9 @@ microsecs() ->
(Mega * 1000000 + Sec) * 1000000 + Micro.
loads() ->
[{load1, ftos(?compat_windows(cpu_sup:avg1()/256, 0.0))},
{load5, ftos(?compat_windows(cpu_sup:avg5()/256, 0.0))},
{load15, ftos(?compat_windows(cpu_sup:avg15()/256, 0.0))}].
[{load1, ftos(avg1()/256)},
{load5, ftos(avg5()/256)},
{load15, ftos(avg15()/256)}].
get_system_info() ->
[{Key, format_system_info(Key, get_system_info(Key))} || Key <- ?SYSTEM_INFO].
@ -447,3 +449,26 @@ mapping([{owner, V}|Entries], Acc) when is_pid(V) ->
mapping(Entries, [{owner, Owner}|Acc]);
mapping([{Key, Value}|Entries], Acc) ->
mapping(Entries, [{Key, Value}|Acc]).
avg1() ->
compat_windows(fun cpu_sup:avg1/0).
avg5() ->
compat_windows(fun cpu_sup:avg5/0).
avg15() ->
compat_windows(fun cpu_sup:avg15/0).
cpu_util() ->
compat_windows(fun cpu_sup:util/0).
compat_windows(Fun) ->
case os:type() of
{win32, nt} -> 0;
_Other -> handle_error(Fun())
end.
handle_error(Value) when is_number(Value) ->
Value;
handle_error({error, _Reason}) ->
0.

58
src/emqx_vm_mon.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,11 +12,14 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_vm_mon).
-behaviour(gen_server).
-include("logger.hrl").
%% APIs
-export([start_link/1]).
@ -38,13 +42,13 @@
-define(VM_MON, ?MODULE).
%%----------------------------------------------------------------------
%% API
%%----------------------------------------------------------------------
start_link(Opts) ->
gen_server:start_link({local, ?VM_MON}, ?MODULE, [Opts], []).
%%--------------------------------------------------------------------
%% API
%%--------------------------------------------------------------------
get_check_interval() ->
call(get_check_interval).
@ -63,9 +67,12 @@ get_process_low_watermark() ->
set_process_low_watermark(Float) ->
call({set_process_low_watermark, Float}).
%%----------------------------------------------------------------------
call(Req) ->
gen_server:call(?VM_MON, Req, infinity).
%%--------------------------------------------------------------------
%% gen_server callbacks
%%----------------------------------------------------------------------
%%--------------------------------------------------------------------
init([Opts]) ->
{ok, ensure_check_timer(#{check_interval => proplists:get_value(check_interval, Opts, 30),
@ -76,43 +83,53 @@ init([Opts]) ->
handle_call(get_check_interval, _From, State) ->
{reply, maps:get(check_interval, State, undefined), State};
handle_call({set_check_interval, Seconds}, _From, State) ->
{reply, ok, State#{check_interval := Seconds}};
handle_call(get_process_high_watermark, _From, State) ->
{reply, maps:get(process_high_watermark, State, undefined), State};
handle_call({set_process_high_watermark, Float}, _From, State) ->
{reply, ok, State#{process_high_watermark := Float}};
handle_call(get_process_low_watermark, _From, State) ->
{reply, maps:get(process_low_watermark, State, undefined), State};
handle_call({set_process_low_watermark, Float}, _From, State) ->
{reply, ok, State#{process_low_watermark := Float}};
handle_call(_Request, _From, State) ->
{reply, ok, State}.
handle_call(Req, _From, State) ->
?LOG(error, "[VM_MON] Unexpected call: ~p", [Req]),
{reply, ignored, State}.
handle_cast(_Request, State) ->
handle_cast(Msg, State) ->
?LOG(error, "[VM_MON] Unexpected cast: ~p", [Msg]),
{noreply, State}.
handle_info({timeout, Timer, check}, State = #{timer := Timer,
handle_info({timeout, Timer, check},
State = #{timer := Timer,
process_high_watermark := ProcHighWatermark,
process_low_watermark := ProcLowWatermark,
is_process_alarm_set := IsProcessAlarmSet}) ->
ProcessCount = erlang:system_info(process_count),
case ProcessCount / erlang:system_info(process_limit) of
NState = case ProcessCount / erlang:system_info(process_limit) of
Percent when Percent >= ProcHighWatermark ->
alarm_handler:set_alarm({too_many_processes, ProcessCount}),
{noreply, ensure_check_timer(State#{is_process_alarm_set := true})};
State#{is_process_alarm_set := true};
Percent when Percent < ProcLowWatermark ->
case IsProcessAlarmSet of
true -> alarm_handler:clear_alarm(too_many_processes);
false -> ok
end,
{noreply, ensure_check_timer(State#{is_process_alarm_set := false})};
_Precent ->
{noreply, ensure_check_timer(State)}
end.
State#{is_process_alarm_set := false};
_Precent -> State
end,
{noreply, ensure_check_timer(NState)};
handle_info(Info, State) ->
?LOG(error, "[VM_MON] Unexpected info: ~p", [Info]),
{noreply, State}.
terminate(_Reason, #{timer := Timer}) ->
emqx_misc:cancel_timer(Timer).
@ -120,11 +137,10 @@ terminate(_Reason, #{timer := Timer}) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%----------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%----------------------------------------------------------------------
call(Req) ->
gen_server:call(?VM_MON, Req, infinity).
%%--------------------------------------------------------------------
ensure_check_timer(State = #{check_interval := Interval}) ->
State#{timer := emqx_misc:start_timer(timer:seconds(Interval), check)}.

511
src/emqx_ws_channel.erl
View File

@ -14,22 +14,22 @@
%% limitations under the License.
%%--------------------------------------------------------------------
%% MQTT WebSocket Channel
-module(emqx_ws_channel).
-include("emqx.hrl").
-include("emqx_mqtt.hrl").
-include("logger.hrl").
-include("types.hrl").
-logger_header("[WS Channel]").
-logger_header("[WsChannel]").
-export([ info/1
, attrs/1
, stats/1
, kick/1
, session/1
]).
%% websocket callbacks
%% WebSocket callbacks
-export([ init/2
, websocket_init/1
, websocket_handle/2
@ -38,66 +38,82 @@
]).
-record(state, {
request,
options,
peername,
sockname,
proto_state,
parse_state,
keepalive,
enable_stats,
stats_timer,
idle_timeout,
shutdown
peername :: emqx_types:peername(),
sockname :: emqx_types:peername(),
fsm_state :: idle | connected | disconnected,
serialize :: fun((emqx_types:packet()) -> iodata()),
parse_state :: emqx_frame:parse_state(),
proto_state :: emqx_protocol:proto_state(),
gc_state :: emqx_gc:gc_state(),
keepalive :: maybe(emqx_keepalive:keepalive()),
pendings :: list(),
stats_timer :: disabled | maybe(reference()),
idle_timeout :: timeout(),
connected :: boolean(),
connected_at :: erlang:timestamp(),
reason :: term()
}).
-type(state() :: #state{}).
-define(SOCK_STATS, [recv_oct, recv_cnt, send_oct, send_cnt]).
-define(CHAN_STATS, [recv_pkt, recv_msg, send_pkt, send_msg]).
%%--------------------------------------------------------------------
%% API
%%--------------------------------------------------------------------
%% for debug
-spec(info(pid() | state()) -> emqx_types:infos()).
info(WSPid) when is_pid(WSPid) ->
call(WSPid, info);
info(#state{peername = Peername,
sockname = Sockname,
proto_state = ProtoState}) ->
ProtoInfo = emqx_protocol:info(ProtoState),
ConnInfo = #{socktype => websocket,
conn_state => running,
proto_state = ProtoState,
gc_state = GCState,
stats_timer = StatsTimer,
idle_timeout = IdleTimeout,
connected = Connected,
connected_at = ConnectedAt}) ->
ChanInfo = #{socktype => websocket,
peername => Peername,
sockname => Sockname},
maps:merge(ProtoInfo, ConnInfo).
sockname => Sockname,
conn_state => running,
gc_state => emqx_gc:info(GCState),
enable_stats => enable_stats(StatsTimer),
idle_timeout => IdleTimeout,
connected => Connected,
connected_at => ConnectedAt
},
maps:merge(ChanInfo, emqx_protocol:info(ProtoState)).
%% for dashboard
enable_stats(disabled) -> false;
enable_stats(_MaybeRef) -> true.
-spec(attrs(pid() | state()) -> emqx_types:attrs()).
attrs(WSPid) when is_pid(WSPid) ->
call(WSPid, attrs);
attrs(#state{peername = Peername,
sockname = Sockname,
proto_state = ProtoState}) ->
SockAttrs = #{peername => Peername,
sockname => Sockname},
ProtoAttrs = emqx_protocol:attrs(ProtoState),
maps:merge(SockAttrs, ProtoAttrs).
proto_state = ProtoState,
connected = Connected,
connected_at = ConnectedAt}) ->
ConnAttrs = #{socktype => websocket,
peername => Peername,
sockname => Sockname,
connected => Connected,
connected_at => ConnectedAt
},
maps:merge(ConnAttrs, emqx_protocol:attrs(ProtoState)).
-spec(stats(pid() | state()) -> emqx_types:stats()).
stats(WSPid) when is_pid(WSPid) ->
call(WSPid, stats);
stats(#state{proto_state = ProtoState}) ->
lists:append([wsock_stats(),
emqx_misc:proc_stats(),
emqx_protocol:stats(ProtoState)
]).
kick(WSPid) when is_pid(WSPid) ->
call(WSPid, kick).
session(WSPid) when is_pid(WSPid) ->
call(WSPid, session).
ProcStats = emqx_misc:proc_stats(),
SessStats = emqx_session:stats(emqx_protocol:info(session, ProtoState)),
lists:append([ProcStats, SessStats, chan_stats(), wsock_stats()]).
%% @private
call(WSPid, Req) when is_pid(WSPid) ->
Mref = erlang:monitor(process, WSPid),
WSPid ! {call, {self(), Mref}, Req},
@ -121,24 +137,27 @@ init(Req, Opts) ->
DeflateOptions = maps:from_list(proplists:get_value(deflate_options, Opts, [])),
MaxFrameSize = case proplists:get_value(max_frame_size, Opts, 0) of
0 -> infinity;
MFS -> MFS
I -> I
end,
Compress = proplists:get_value(compress, Opts, false),
Options = #{compress => Compress,
WsOpts = #{compress => Compress,
deflate_opts => DeflateOptions,
max_frame_size => MaxFrameSize,
idle_timeout => IdleTimeout},
idle_timeout => IdleTimeout
},
case cowboy_req:parse_header(<<"sec-websocket-protocol">>, Req) of
undefined ->
{cowboy_websocket, Req, #state{}, Options};
%% TODO: why not reply 500???
{cowboy_websocket, Req, [Req, Opts], WsOpts};
[<<"mqtt", Vsn/binary>>] ->
Resp = cowboy_req:set_resp_header(<<"sec-websocket-protocol">>, <<"mqtt", Vsn/binary>>, Req),
{cowboy_websocket, Resp, #state{request = Req, options = Opts}, Options};
Resp = cowboy_req:set_resp_header(
<<"sec-websocket-protocol">>, <<"mqtt", Vsn/binary>>, Req),
{cowboy_websocket, Resp, [Req, Opts], WsOpts};
_ ->
{ok, cowboy_req:reply(400, Req), #state{}}
end.
websocket_init(#state{request = Req, options = Options}) ->
websocket_init([Req, Opts]) ->
Peername = cowboy_req:peer(Req),
Sockname = cowboy_req:sock(Req),
Peercert = cowboy_req:cert(Req),
@ -148,84 +167,64 @@ websocket_init(#state{request = Req, options = Options}) ->
?LOG(error, "Illegal cookie"),
undefined;
Error:Reason ->
?LOG(error,
"Cookie is parsed failed, Error: ~p, Reason ~p",
?LOG(error, "Cookie is parsed failed, Error: ~p, Reason ~p",
[Error, Reason]),
undefined
end,
ProtoState = emqx_protocol:init(#{peername => Peername,
sockname => Sockname,
peercert => Peercert,
sendfun => send_fun(self()),
ws_cookie => WsCookie,
conn_mod => ?MODULE}, Options),
Zone = proplists:get_value(zone, Options),
conn_mod => ?MODULE}, Opts),
Zone = proplists:get_value(zone, Opts),
MaxSize = emqx_zone:get_env(Zone, max_packet_size, ?MAX_PACKET_SIZE),
ParseState = emqx_frame:initial_parse_state(#{max_size => MaxSize}),
GcPolicy = emqx_zone:get_env(Zone, force_gc_policy, false),
GcState = emqx_gc:init(GcPolicy),
EnableStats = emqx_zone:get_env(Zone, enable_stats, true),
StatsTimer = if EnableStats -> undefined; ?Otherwise-> disabled end,
IdleTimout = emqx_zone:get_env(Zone, idle_timeout, 30000),
emqx_logger:set_metadata_peername(esockd_net:format(Peername)),
ok = emqx_misc:init_proc_mng_policy(Zone),
{ok, #state{peername = Peername,
sockname = Sockname,
fsm_state = idle,
parse_state = ParseState,
proto_state = ProtoState,
enable_stats = EnableStats,
idle_timeout = IdleTimout}}.
send_fun(WsPid) ->
fun(Packet, Options) ->
Data = emqx_frame:serialize(Packet, Options),
BinSize = iolist_size(Data),
emqx_pd:update_counter(send_cnt, 1),
emqx_pd:update_counter(send_oct, BinSize),
WsPid ! {binary, iolist_to_binary(Data)},
{ok, Data}
end.
gc_state = GcState,
pendings = [],
stats_timer = StatsTimer,
idle_timeout = IdleTimout,
connected = false
}}.
stat_fun() ->
fun() -> {ok, emqx_pd:get_counter(recv_oct)} end.
websocket_handle({binary, <<>>}, State) ->
{ok, ensure_stats_timer(State)};
websocket_handle({binary, [<<>>]}, State) ->
{ok, ensure_stats_timer(State)};
websocket_handle({binary, Data}, State = #state{parse_state = ParseState}) ->
websocket_handle({binary, Data}, State) when is_list(Data) ->
websocket_handle({binary, iolist_to_binary(Data)}, State);
websocket_handle({binary, Data}, State) when is_binary(Data) ->
?LOG(debug, "RECV ~p", [Data]),
BinSize = iolist_size(Data),
emqx_pd:update_counter(recv_oct, BinSize),
ok = emqx_metrics:inc('bytes.received', BinSize),
try emqx_frame:parse(iolist_to_binary(Data), ParseState) of
{ok, NParseState} ->
{ok, State#state{parse_state = NParseState}};
{ok, Packet, Rest, NParseState} ->
ok = emqx_metrics:inc_recv(Packet),
Oct = iolist_size(Data),
emqx_pd:update_counter(recv_cnt, 1),
handle_incoming(Packet, fun(NState) ->
websocket_handle({binary, Rest}, NState)
end,
State#state{parse_state = NParseState});
{error, Reason} ->
?LOG(error, "Frame error: ~p", [Reason]),
shutdown(Reason, State)
catch
error:Reason:Stk ->
?LOG(error, "Parse failed for ~p~n\
Stacktrace:~p~nFrame data: ~p", [Reason, Stk, Data]),
shutdown(parse_error, State)
end;
emqx_pd:update_counter(recv_oct, Oct),
ok = emqx_metrics:inc('bytes.received', Oct),
NState = maybe_gc(1, Oct, State),
process_incoming(Data, ensure_stats_timer(NState));
%% Pings should be replied with pongs, cowboy does it automatically
%% Pongs can be safely ignored. Clause here simply prevents crash.
websocket_handle(Frame, State)
when Frame =:= ping; Frame =:= pong ->
{ok, ensure_stats_timer(State)};
{ok, State};
websocket_handle({FrameType, _}, State)
when FrameType =:= ping; FrameType =:= pong ->
{ok, ensure_stats_timer(State)};
{ok, State};
%% According to mqtt spec[https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901285]
websocket_handle({_OtherFrameType, _}, State) ->
?LOG(error, "Frame error: Other type of data frame"),
shutdown(other_frame_type, State).
websocket_handle({FrameType, _}, State) ->
?LOG(error, "Frame error: unexpected frame - ~p", [FrameType]),
stop(unexpected_ws_frame, State).
websocket_info({call, From, info}, State) ->
gen_server:reply(From, info(State)),
@ -241,33 +240,41 @@ websocket_info({call, From, stats}, State) ->
websocket_info({call, From, kick}, State) ->
gen_server:reply(From, ok),
shutdown(kick, State);
stop(kick, State);
websocket_info({call, From, session}, State = #state{proto_state = ProtoState}) ->
gen_server:reply(From, emqx_protocol:session(ProtoState)),
{ok, State};
websocket_info({incoming, Packet = ?CONNECT_PACKET(
#mqtt_packet_connect{
proto_ver = ProtoVer}
)},
State = #state{fsm_state = idle}) ->
handle_incoming(Packet, fun connected/1,
State#state{serialize = serialize_fun(ProtoVer)});
websocket_info({deliver, PubOrAck}, State = #state{proto_state = ProtoState}) ->
case emqx_protocol:deliver(PubOrAck, ProtoState) of
{ok, ProtoState1} ->
{ok, ensure_stats_timer(State#state{proto_state = ProtoState1})};
websocket_info({incoming, Packet}, State = #state{fsm_state = idle}) ->
?LOG(warning, "Unexpected incoming: ~p", [Packet]),
stop(unexpected_incoming_packet, State);
websocket_info({incoming, Packet = ?PACKET(?CONNECT)},
State = #state{fsm_state = connected}) ->
?LOG(warning, "Unexpected connect: ~p", [Packet]),
stop(unexpected_incoming_connect, State);
websocket_info({incoming, Packet}, State = #state{fsm_state = connected})
when is_record(Packet, mqtt_packet) ->
handle_incoming(Packet, fun reply/1, State);
websocket_info(Deliver = {deliver, _Topic, _Msg},
State = #state{proto_state = ProtoState}) ->
Delivers = emqx_misc:drain_deliver([Deliver]),
case emqx_protocol:handle_deliver(Delivers, ProtoState) of
{ok, NProtoState} ->
reply(State#state{proto_state = NProtoState});
{ok, Packets, NProtoState} ->
reply(enqueue(Packets, State#state{proto_state = NProtoState}));
{error, Reason} ->
shutdown(Reason, State)
end;
websocket_info({timeout, Timer, emit_stats},
State = #state{stats_timer = Timer, proto_state = ProtoState}) ->
emqx_cm:set_conn_stats(emqx_protocol:client_id(ProtoState), stats(State)),
{ok, State#state{stats_timer = undefined}, hibernate};
websocket_info({keepalive, start, Interval}, State) ->
?LOG(debug, "Keepalive at the interval of ~p", [Interval]),
case emqx_keepalive:start(stat_fun(), Interval, {keepalive, check}) of
{ok, KeepAlive} ->
{ok, State#state{keepalive = KeepAlive}};
{error, Error} ->
?LOG(warning, "Keepalive error: ~p", [Error]),
shutdown(Error, State)
stop(Reason, State);
{error, Reason, NProtoState} ->
stop(Reason, State#state{proto_state = NProtoState})
end;
websocket_info({keepalive, check}, State = #state{keepalive = KeepAlive}) ->
@ -275,100 +282,242 @@ websocket_info({keepalive, check}, State = #state{keepalive = KeepAlive}) ->
{ok, KeepAlive1} ->
{ok, State#state{keepalive = KeepAlive1}};
{error, timeout} ->
?LOG(debug, "Keepalive Timeout!"),
shutdown(keepalive_timeout, State);
stop(keepalive_timeout, State);
{error, Error} ->
?LOG(error, "Keepalive error: ~p", [Error]),
shutdown(keepalive_error, State)
stop(keepalive_error, State)
end;
websocket_info({timeout, Timer, emit_stats},
State = #state{stats_timer = Timer,
proto_state = ProtoState,
gc_state = GcState}) ->
ClientId = emqx_protocol:info(client_id, ProtoState),
ok = emqx_cm:set_chan_stats(ClientId, stats(State)),
NState = State#state{stats_timer = undefined},
Limits = erlang:get(force_shutdown_policy),
case emqx_misc:conn_proc_mng_policy(Limits) of
continue ->
{ok, NState};
hibernate ->
%% going to hibernate, reset gc stats
GcState1 = emqx_gc:reset(GcState),
{ok, NState#state{gc_state = GcState1}, hibernate};
{shutdown, Reason} ->
?LOG(error, "Shutdown exceptionally due to ~p", [Reason]),
stop(Reason, NState)
end;
websocket_info({timeout, Timer, Msg},
State = #state{proto_state = ProtoState}) ->
case emqx_protocol:handle_timeout(Timer, Msg, ProtoState) of
{ok, NProtoState} ->
{ok, State#state{proto_state = NProtoState}};
{ok, Packets, NProtoState} ->
reply(enqueue(Packets, State#state{proto_state = NProtoState}));
{error, Reason} ->
stop(Reason, State);
{error, Reason, NProtoState} ->
stop(Reason, State#state{proto_state = NProtoState})
end;
websocket_info({subscribe, TopicFilters}, State) ->
handle_request({subscribe, TopicFilters}, State);
websocket_info({unsubscribe, TopicFilters}, State) ->
handle_request({unsubscribe, TopicFilters}, State);
websocket_info({shutdown, discard, {ClientId, ByPid}}, State) ->
?LOG(warning, "Discarded by ~s:~p", [ClientId, ByPid]),
shutdown(discard, State);
stop(discard, State);
websocket_info({shutdown, conflict, {ClientId, NewPid}}, State) ->
?LOG(warning, "Clientid '~s' conflict with ~p", [ClientId, NewPid]),
shutdown(conflict, State);
stop(conflict, State);
websocket_info({binary, Data}, State) ->
{reply, {binary, Data}, State};
%% websocket_info({binary, Data}, State) ->
%% {reply, {binary, Data}, State};
websocket_info({shutdown, Reason}, State) ->
shutdown(Reason, State);
stop(Reason, State);
websocket_info(stop, State) ->
{stop, State};
websocket_info(Info = {'EXIT', SessionPid, Reason}, State = #state{proto_state = ProtoState}) ->
case emqx_protocol:session(ProtoState) of
undefined ->
?LOG(error, "Unexpected EXIT: ~p", [Info]),
{ok, State};
SessionPid ->
?LOG(error, "Session ~p termiated: ~p", [SessionPid, Reason]),
shutdown(Reason, State)
end;
websocket_info({stop, Reason}, State) ->
stop(Reason, State);
websocket_info(Info, State) ->
?LOG(error, "Unexpected info: ~p", [Info]),
{ok, State}.
terminate(WsReason, _Req, #state{keepalive = Keepalive,
terminate(SockError, _Req, #state{keepalive = Keepalive,
proto_state = ProtoState,
shutdown = Shutdown}) ->
?LOG(debug, "Terminated for ~p, websocket reason: ~p",
[Shutdown, WsReason]),
reason = Reason}) ->
?LOG(debug, "Terminated for ~p, sockerror: ~p",
[Reason, SockError]),
emqx_keepalive:cancel(Keepalive),
case {ProtoState, Shutdown} of
{undefined, _} -> ok;
{_, {shutdown, Reason}} ->
terminate_session(Reason, ProtoState);
{_, _Error} ->
?LOG(info, "Terminate for unexpected error: ~p", [WsReason]),
terminate_session(unknown, ProtoState)
emqx_protocol:terminate(Reason, ProtoState).
%%--------------------------------------------------------------------
%% Connected callback
connected(State = #state{proto_state = ProtoState}) ->
NState = State#state{fsm_state = connected,
connected = true,
connected_at = os:timestamp()
},
ClientId = emqx_protocol:info(client_id, ProtoState),
ok = emqx_cm:register_channel(ClientId),
ok = emqx_cm:set_chan_attrs(ClientId, info(NState)),
%% Ensure keepalive after connected successfully.
Interval = emqx_protocol:info(keepalive, ProtoState),
case ensure_keepalive(Interval, NState) of
ignore -> reply(NState);
{ok, KeepAlive} ->
reply(NState#state{keepalive = KeepAlive});
{error, Reason} ->
stop(Reason, NState)
end.
%%--------------------------------------------------------------------
%% Internal functions
%%--------------------------------------------------------------------
%% Ensure keepalive
terminate_session(Reason, ProtoState) ->
emqx_protocol:terminate(Reason, ProtoState),
case emqx_protocol:session(ProtoState) of
undefined ->
ok;
SessionPid ->
unlink(SessionPid),
SessionPid ! {'EXIT', self(), Reason}
ensure_keepalive(0, _State) ->
ignore;
ensure_keepalive(Interval, #state{proto_state = ProtoState}) ->
Backoff = emqx_zone:get_env(emqx_protocol:info(zone, ProtoState),
keepalive_backoff, 0.75),
emqx_keepalive:start(stat_fun(), round(Interval * Backoff), {keepalive, check}).
%%--------------------------------------------------------------------
%% Handle internal request
handle_request(Req, State = #state{proto_state = ProtoState}) ->
case emqx_protocol:handle_req(Req, ProtoState) of
{ok, _Result, NProtoState} -> %% TODO:: how to handle the result?
{ok, State#state{proto_state = NProtoState}};
{error, Reason, NProtoState} ->
stop(Reason, State#state{proto_state = NProtoState})
end.
handle_incoming(Packet, SuccFun, State = #state{proto_state = ProtoState}) ->
case emqx_protocol:received(Packet, ProtoState) of
%%--------------------------------------------------------------------
%% Process incoming data
process_incoming(<<>>, State) ->
{ok, State};
process_incoming(Data, State = #state{parse_state = ParseState}) ->
try emqx_frame:parse(Data, ParseState) of
{ok, NParseState} ->
{ok, State#state{parse_state = NParseState}};
{ok, Packet, Rest, NParseState} ->
self() ! {incoming, Packet},
process_incoming(Rest, State#state{parse_state = NParseState});
{error, Reason} ->
?LOG(error, "Frame error: ~p", [Reason]),
stop(Reason, State)
catch
error:Reason:Stk ->
?LOG(error, "Parse failed for ~p~n\
Stacktrace:~p~nFrame data: ~p", [Reason, Stk, Data]),
stop(parse_error, State)
end.
%%--------------------------------------------------------------------
%% Handle incoming packets
handle_incoming(Packet = ?PACKET(Type), SuccFun,
State = #state{proto_state = ProtoState}) ->
_ = inc_incoming_stats(Type),
ok = emqx_metrics:inc_recv(Packet),
?LOG(debug, "RECV ~s", [emqx_packet:format(Packet)]),
case emqx_protocol:handle_in(Packet, ProtoState) of
{ok, NProtoState} ->
SuccFun(State#state{proto_state = NProtoState});
{error, Reason} ->
?LOG(error, "Protocol error: ~p", [Reason]),
shutdown(Reason, State);
{ok, OutPackets, NProtoState} ->
SuccFun(enqueue(OutPackets, State#state{proto_state = NProtoState}));
{error, Reason, NProtoState} ->
shutdown(Reason, State#state{proto_state = NProtoState});
stop(Reason, State#state{proto_state = NProtoState});
{error, Reason, OutPacket, NProtoState} ->
stop(Reason, enqueue(OutPacket, State#state{proto_state = NProtoState}));
{stop, Error, NProtoState} ->
shutdown(Error, State#state{proto_state = NProtoState})
stop(Error, State#state{proto_state = NProtoState})
end.
ensure_stats_timer(State = #state{enable_stats = true,
stats_timer = undefined,
idle_timeout = IdleTimeout}) ->
State#state{stats_timer = emqx_misc:start_timer(IdleTimeout, emit_stats)};
ensure_stats_timer(State) ->
State.
%%--------------------------------------------------------------------
%% Handle outgoing packets
shutdown(Reason = {shutdown, _}, State) ->
self() ! stop,
{ok, State#state{shutdown = Reason}};
shutdown(Reason, State) ->
%% Fix the issue#2591(https://github.com/emqx/emqx/issues/2591#issuecomment-500278696)
self() ! stop,
{ok, State#state{shutdown = {shutdown, Reason}}}.
handle_outgoing(Packets, #state{serialize = Serialize}) ->
Data = lists:map(Serialize, Packets),
emqx_pd:update_counter(send_oct, iolist_size(Data)),
{binary, Data}.
%%--------------------------------------------------------------------
%% Serialize fun
serialize_fun(ProtoVer) ->
fun(Packet = ?PACKET(Type)) ->
?LOG(debug, "SEND ~s", [emqx_packet:format(Packet)]),
_ = inc_outgoing_stats(Type),
emqx_frame:serialize(Packet, ProtoVer)
end.
%%--------------------------------------------------------------------
%% Inc incoming/outgoing stats
inc_incoming_stats(Type) ->
emqx_pd:update_counter(recv_pkt, 1),
(Type == ?PUBLISH)
andalso emqx_pd:update_counter(recv_msg, 1).
inc_outgoing_stats(Type) ->
emqx_pd:update_counter(send_cnt, 1),
emqx_pd:update_counter(send_pkt, 1),
(Type == ?PUBLISH)
andalso emqx_pd:update_counter(send_msg, 1).
%%--------------------------------------------------------------------
%% Reply or Stop
reply(State = #state{pendings = []}) ->
{ok, State};
reply(State = #state{pendings = Pendings}) ->
Reply = handle_outgoing(Pendings, State),
{reply, Reply, State#state{pendings = []}}.
stop(Reason, State = #state{pendings = []}) ->
{stop, State#state{reason = Reason}};
stop(Reason, State = #state{pendings = Pendings}) ->
Reply = handle_outgoing(Pendings, State),
{reply, [Reply, close],
State#state{pendings = [], reason = Reason}}.
enqueue(Packet, State) when is_record(Packet, mqtt_packet) ->
enqueue([Packet], State);
enqueue(Packets, State = #state{pendings = Pendings}) ->
State#state{pendings = lists:append(Pendings, Packets)}.
%%--------------------------------------------------------------------
%% Ensure stats timer
ensure_stats_timer(State = #state{stats_timer = undefined,
idle_timeout = IdleTimeout}) ->
TRef = emqx_misc:start_timer(IdleTimeout, emit_stats),
State#state{stats_timer = TRef};
%% disabled or timer existed
ensure_stats_timer(State) -> State.
wsock_stats() ->
[{Key, emqx_pd:get_counter(Key)} || Key <- ?SOCK_STATS].
chan_stats() ->
[{Name, emqx_pd:get_counter(Name)} || Name <- ?CHAN_STATS].
%%--------------------------------------------------------------------
%% Maybe GC
maybe_gc(_Cnt, _Oct, State = #state{gc_state = undefined}) ->
State;
maybe_gc(Cnt, Oct, State = #state{gc_state = GCSt}) ->
{Ok, GCSt1} = emqx_gc:run(Cnt, Oct, GCSt),
Ok andalso emqx_metrics:inc('channel.gc.cnt'),
State#state{gc_state = GCSt1}.

37
src/emqx_zone.erl
View File

@ -1,4 +1,5 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%--------------------------------------------------------------------
%% Copyright (c) 2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
@ -11,6 +12,7 @@
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emqx_zone).
@ -28,6 +30,7 @@
-export([ get_env/2
, get_env/3
, set_env/3
, unset_env/2
, force_reload/0
]).
@ -43,28 +46,32 @@
, code_change/3
]).
-export_type([zone/0]).
%% dummy state
-record(state, {}).
-type(zone() :: atom()).
-define(TAB, ?MODULE).
-define(SERVER, ?MODULE).
-define(KEY(Zone, Key), {?MODULE, Zone, Key}).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% APIs
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
-spec(start_link() -> startlink_ret()).
start_link() ->
gen_server:start_link({local, ?SERVER}, ?MODULE, [], []).
-spec(get_env(maybe(emqx_types:zone()), atom()) -> maybe(term())).
-spec(get_env(maybe(zone()), atom()) -> maybe(term())).
get_env(undefined, Key) ->
emqx_config:get_env(Key);
get_env(Zone, Key) ->
get_env(Zone, Key, undefined).
-spec(get_env(maybe(emqx_types:zone()), atom(), term()) -> maybe(term())).
-spec(get_env(maybe(zone()), atom(), term()) -> maybe(term())).
get_env(undefined, Key, Def) ->
emqx_config:get_env(Key, Def);
get_env(Zone, Key, Def) ->
@ -73,9 +80,13 @@ get_env(Zone, Key, Def) ->
emqx_config:get_env(Key, Def)
end.
-spec(set_env(emqx_types:zone(), atom(), term()) -> ok).
-spec(set_env(zone(), atom(), term()) -> ok).
set_env(Zone, Key, Val) ->
gen_server:cast(?SERVER, {set_env, Zone, Key, Val}).
persistent_term:put(?KEY(Zone, Key), Val).
-spec(unset_env(zone(), atom()) -> boolean()).
unset_env(Zone, Key) ->
persistent_term:erase(?KEY(Zone, Key)).
-spec(force_reload() -> ok).
force_reload() ->
@ -85,9 +96,9 @@ force_reload() ->
stop() ->
gen_server:stop(?SERVER, normal, infinity).
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% gen_server callbacks
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
init([]) ->
_ = do_reload(),
@ -101,10 +112,6 @@ handle_call(Req, _From, State) ->
?LOG(error, "Unexpected call: ~p", [Req]),
{reply, ignored, State}.
handle_cast({set_env, Zone, Key, Val}, State) ->
ok = persistent_term:put(?KEY(Zone, Key), Val),
{noreply, State};
handle_cast(Msg, State) ->
?LOG(error, "Unexpected cast: ~p", [Msg]),
{noreply, State}.
@ -119,9 +126,9 @@ terminate(_Reason, _State) ->
code_change(_OldVsn, State, _Extra) ->
{ok, State}.
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
%% Internal functions
%%------------------------------------------------------------------------------
%%--------------------------------------------------------------------
do_reload() ->
[persistent_term:put(?KEY(Zone, Key), Val)

206
test/emqx_SUITE.erl
View File

@ -1,206 +0,0 @@
%% Copyright (c) 2013-2019 EMQ Technologies Co., Ltd. All Rights Reserved.
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
-module(emqx_SUITE).
-compile(export_all).
-compile(nowarn_export_all).
-define(APP, emqx).
-include_lib("eunit/include/eunit.hrl").
-include_lib("common_test/include/ct.hrl").
-include("emqx_mqtt.hrl").
-record(ssl_socket, {tcp, ssl}).
-define(CLIENT, ?CONNECT_PACKET(#mqtt_packet_connect{
client_id = <<"mqtt_client">>,
username = <<"admin">>,
password = <<"public">>})).
-define(CLIENT2, ?CONNECT_PACKET(#mqtt_packet_connect{
username = <<"admin">>,
clean_start = false,
password = <<"public">>})).
-define(CLIENT3, ?CONNECT_PACKET(#mqtt_packet_connect{
username = <<"admin">>,
proto_ver = ?MQTT_PROTO_V5,
clean_start = false,
password = <<"public">>,
will_props = #{'Will-Delay-Interval' => 2}})).
-define(SUBCODE, [0]).
-define(PACKETID, 1).
-define(PUBQOS, 1).
-define(SUBPACKET, ?SUBSCRIBE_PACKET(?PACKETID, [{<<"sub/topic">>, ?DEFAULT_SUBOPTS}])).
-define(PUBPACKET, ?PUBLISH_PACKET(?PUBQOS, <<"sub/topic">>, ?PACKETID, <<"publish">>)).
-define(PAYLOAD, [{type,"dsmSimulationData"},
{id, 9999},
{status, "running"},
{soc, 1536702170},
{fracsec, 451000},
{data, lists:seq(1, 20480)}]).
-define(BIG_PUBPACKET, ?PUBLISH_PACKET(?PUBQOS, <<"sub/topic">>, ?PACKETID, emqx_json:encode(?PAYLOAD))).
all() ->
[{group, connect},
{group, publish}].
groups() ->
[{connect, [non_parallel_tests],
[mqtt_connect,
mqtt_connect_with_tcp,
mqtt_connect_with_will_props,
mqtt_connect_with_ssl_oneway,
mqtt_connect_with_ssl_twoway,
mqtt_connect_with_ws]},
{publish, [non_parallel_tests],
[packet_size]}].
init_per_suite(Config) ->
emqx_ct_helpers:start_apps([]),
Config.
end_per_suite(_Config) ->
emqx_ct_helpers:stop_apps([]).
%%--------------------------------------------------------------------
%% Protocol Test
%%--------------------------------------------------------------------
mqtt_connect(_) ->
%% Issue #599
%% Empty clientId and clean_session = false
?assertEqual(<<32,2,0,2>>, connect_broker_(<<16,12,0,4,77,81,84,84,4,0,0,90,0,0>>, 4)),
%% Empty clientId and clean_session = true
?assertEqual(<<32,2,0,0>>, connect_broker_(<<16,12,0,4,77,81,84,84,4,2,0,90,0,0>>, 4)).
connect_broker_(Packet, RecvSize) ->
{ok, Sock} = emqx_client_sock:connect({127,0,0,1}, 1883, [binary, {packet, raw}, {active, false}], 3000),
emqx_client_sock:send(Sock, Packet),
{ok, Data} = gen_tcp:recv(Sock, RecvSize, 3000),
emqx_client_sock:close(Sock),
Data.
mqtt_connect_with_tcp(_) ->
%% Issue #599
%% Empty clientId and clean_session = false
{ok, Sock} = emqx_client_sock:connect({127,0,0,1}, 1883, [binary, {packet, raw}, {active, false}], 3000),
Packet = raw_send_serialize(?CLIENT2),
emqx_client_sock:send(Sock, Packet),
{ok, Data} = gen_tcp:recv(Sock, 0),
{ok, ?CONNACK_PACKET(?CONNACK_INVALID_ID), <<>>, _} = raw_recv_pase(Data),
emqx_client_sock:close(Sock).
mqtt_connect_with_will_props(_) ->
%% Issue #599
%% Empty clientId and clean_session = false
{ok, Sock} = emqx_client_sock:connect({127,0,0,1}, 1883, [binary, {packet, raw}, {active, false}], 3000),
Packet = raw_send_serialize(?CLIENT3),
emqx_client_sock:send(Sock, Packet),
emqx_client_sock:close(Sock).
mqtt_connect_with_ssl_oneway(_) ->
emqx:shutdown(),
emqx_ct_helpers:change_emqx_opts(ssl_oneway),
emqx:start(),
ClientSsl = emqx_ct_helpers:client_ssl(),
{ok, #ssl_socket{tcp = _Sock1, ssl = SslSock} = Sock}
= emqx_client_sock:connect("127.0.0.1", 8883, [{ssl_opts, ClientSsl}], 3000),
Packet = raw_send_serialize(?CLIENT),
emqx_client_sock:setopts(Sock, [{active, once}]),
emqx_client_sock:send(Sock, Packet),
?assert(
receive {ssl, _, ConAck}->
{ok, ?CONNACK_PACKET(?CONNACK_ACCEPT), <<>>, _} = raw_recv_pase(ConAck), true
after 1000 ->
false
end),
ssl:close(SslSock).
mqtt_connect_with_ssl_twoway(_Config) ->
emqx:shutdown(),
emqx_ct_helpers:change_emqx_opts(ssl_twoway),
emqx:start(),
ClientSsl = emqx_ct_helpers:client_ssl_twoway(),
{ok, #ssl_socket{tcp = _Sock1, ssl = SslSock} = Sock}
= emqx_client_sock:connect("127.0.0.1", 8883, [{ssl_opts, ClientSsl}], 3000),
Packet = raw_send_serialize(?CLIENT),
emqx_client_sock:setopts(Sock, [{active, once}]),
emqx_client_sock:send(Sock, Packet),
timer:sleep(500),
?assert(
receive {ssl, _, Data}->
{ok, ?CONNACK_PACKET(?CONNACK_ACCEPT), <<>>, _} = raw_recv_pase(Data), true
after 1000 ->
false
end),
ssl:close(SslSock),
emqx_client_sock:close(Sock).
mqtt_connect_with_ws(_Config) ->
WS = rfc6455_client:new("ws://127.0.0.1:8083" ++ "/mqtt", self()),
{ok, _} = rfc6455_client:open(WS),
%% Connect Packet
Packet = raw_send_serialize(?CLIENT),
ok = rfc6455_client:send_binary(WS, Packet),
{binary, CONACK} = rfc6455_client:recv(WS),
{ok, ?CONNACK_PACKET(?CONNACK_ACCEPT), <<>>, _} = raw_recv_pase(CONACK),
%% Sub Packet
SubPacket = raw_send_serialize(?SUBPACKET),
rfc6455_client:send_binary(WS, SubPacket),
{binary, SubAck} = rfc6455_client:recv(WS),
{ok, ?SUBACK_PACKET(?PACKETID, ?SUBCODE), <<>>, _} = raw_recv_pase(SubAck),
%% Pub Packet QoS 1
PubPacket = raw_send_serialize(?PUBPACKET),
rfc6455_client:send_binary(WS, PubPacket),
{binary, PubAck} = rfc6455_client:recv(WS),
{ok, ?PUBACK_PACKET(?PACKETID), <<>>, _} = raw_recv_pase(PubAck),
{close, _} = rfc6455_client:close(WS),
ok.
%%issue 1811
packet_size(_Config) ->
{ok, Sock} = emqx_client_sock:connect({127,0,0,1}, 1883, [binary, {packet, raw}, {active, false}], 3000),
Packet = raw_send_serialize(?CLIENT),
emqx_client_sock:send(Sock, Packet),
{ok, Data} = gen_tcp:recv(Sock, 0),
{ok, ?CONNACK_PACKET(?CONNACK_ACCEPT), <<>>, _} = raw_recv_pase(Data),
%% Pub Packet QoS 1
PubPacket = raw_send_serialize(?BIG_PUBPACKET),
emqx_client_sock:send(Sock, PubPacket),
{ok, Data1} = gen_tcp:recv(Sock, 0),
{ok, ?PUBACK_PACKET(?PACKETID), <<>>, _} = raw_recv_pase(Data1),
emqx_client_sock:close(Sock).
raw_send_serialize(Packet) ->
emqx_frame:serialize(Packet).
raw_recv_pase(Bin) ->
emqx_frame:parse(Bin).

Some files were not shown because too many files have changed in this diff Show More