This commit is contained in:
Feng 2016-02-27 15:24:46 +08:00
parent 4e73476dfe
commit 8b8ceaac2f
4 changed files with 682 additions and 3 deletions

View File

@ -1,3 +1,330 @@
============== ==============
Design Guide Design Guide
============== ==============
---------------
Pubsub Sequence
---------------
## PubSub Sequence
### Clean Session = 1
```
title PubSub Sequence(Clean Session = 1)
ClientA-->PubSub: Publish Message
PubSub-->ClientB: Dispatch Message
```
![PubSub_CleanSess_1](http://emqtt.io/static/img/design/PubSub_CleanSess_1.png)
### Clean Session = 0
```
title PubSub Sequence(Clean Session = 0)
ClientA-->SessionA: Publish Message
SessionA-->PubSub: Publish Message
PubSub-->SessionB: Dispatch Message
SessionB-->ClientB: Dispatch Message
```
![PubSub_CleanSess_0](http://emqtt.io/static/img/design/PubSub_CleanSess_0.png)
## Qos
PubQos | SubQos | In Message | Out Message
-------|--------|------------|-------------
0 | 0 | 0 | 0
0 | 1 | 0 | 0
0 | 2 | 0 | 0
1 | 0 | 1 | 0
1 | 1 | 1 | 1
1 | 2 | 1 | 1
2 | 0 | 2 | 0
2 | 1 | 2 | 1
2 | 2 | 2 | 2
## Topic Functions Benchmark
Mac Air(11):
Function | Time(microseconds)
-------------|--------------------
match | 6.25086
triples | 13.86881
words | 3.41177
binary:split | 3.03776
iMac:
Function | Time(microseconds)
-------------|--------------------
match | 3.2348
triples | 6.93524
words | 1.89616
binary:split | 1.65243
--------------
Cluster Design
--------------
## Cluster Design
1. One 'disc_copies' node and many 'ram_copies' nodes.
2. Topic trie tree will be copied to every clusterd node.
3. Subscribers to topic will be stored in each node and will not be copied.
## Cluster Strategy
TODO:...
1. A message only gets forwarded to other cluster nodes if a cluster node is interested in it. this reduces the network traffic tremendously, because it prevents nodes from forwarding unnecessary messages.
2. As soon as a client on a node subscribes to a topic it becomes known within the cluster. If one of the clients somewhere in the cluster is publishing to this topic, the message will be delivered to its subscriber no matter to which cluster node it is connected.
....
## Cluster Architecture
![Cluster Design](http://emqtt.io/static/img/Cluster.png)
## Cluster Command
```sh
./bin/emqttd_ctl cluster DiscNode
```
## Mnesia Example
```
(emqttd3@127.0.0.1)3> mnesia:info().
---> Processes holding locks <---
---> Processes waiting for locks <---
---> Participant transactions <---
---> Coordinator transactions <---
---> Uncertain transactions <---
---> Active tables <---
mqtt_retained : with 6 records occupying 221 words of mem
topic_subscriber: with 0 records occupying 305 words of mem
topic_trie_node: with 129 records occupying 3195 words of mem
topic_trie : with 128 records occupying 3986 words of mem
topic : with 93 records occupying 1797 words of mem
schema : with 6 records occupying 1081 words of mem
===> System info in version "4.12.4", debug level = none <===
opt_disc. Directory "/Users/erylee/Projects/emqttd/rel/emqttd3/data/mnesia" is NOT used.
use fallback at restart = false
running db nodes = ['emqttd2@127.0.0.1','emqttd@127.0.0.1','emqttd3@127.0.0.1']
stopped db nodes = []
master node tables = []
remote = []
ram_copies = [mqtt_retained,schema,topic,topic_subscriber,topic_trie,
topic_trie_node]
disc_copies = []
disc_only_copies = []
[{'emqttd2@127.0.0.1',ram_copies},
{'emqttd3@127.0.0.1',ram_copies},
{'emqttd@127.0.0.1',disc_copies}] = [schema]
[{'emqttd2@127.0.0.1',ram_copies},
{'emqttd3@127.0.0.1',ram_copies},
{'emqttd@127.0.0.1',ram_copies}] = [topic,topic_trie,topic_trie_node,
mqtt_retained]
[{'emqttd3@127.0.0.1',ram_copies}] = [topic_subscriber]
44 transactions committed, 5 aborted, 0 restarted, 0 logged to disc
0 held locks, 0 in queue; 0 local transactions, 0 remote
0 transactions waits for other nodes: []
```
## Cluster vs Bridge
Cluster will copy topic trie tree between nodes, Bridge will not.
-------------
Hooks Design
-------------
## Overview
emqttd supported a simple hooks mechanism in 0.8.0 release to extend the broker. The designed is improved in 0.9.0 release.
## API
emqttd_broker Hook API:
```
-export([hook/3, unhook/2, foreach_hooks/2, foldl_hooks/3]).
```
### Hook
```
-spec hook(Hook :: atom(), Name :: any(), MFA :: mfa()) -> ok | {error, any()}.
hook(Hook, Name, MFA) ->
...
```
### Unhook
```
-spec unhook(Hook :: atom(), Name :: any()) -> ok | {error, any()}.
unhook(Hook, Name) ->
...
```
### Foreach Hooks
```
-spec foreach_hooks(Hook :: atom(), Args :: list()) -> any().
foreach_hooks(Hook, Args) ->
...
```
### Foldl Hooks
```
-spec foldl_hooks(Hook :: atom(), Args :: list(), Acc0 :: any()) -> any().
foldl_hooks(Hook, Args, Acc0) ->
...
```
## Hooks
Name | Type | Description
--------------- | ----------| --------------
client.connected | foreach | Run when client connected successfully
client.subscribe | foldl | Run before client subscribe topics
client.subscribe.after | foreach | Run After client subscribe topics
client.unsubscribe | foldl | Run when client unsubscribe topics
message.publish | foldl | Run when message is published
message.acked | foreach | Run when message is acked
client.disconnected | foreach | Run when client is disconnnected
## End-to-End Message Pub/Ack
Could use 'message.publish', 'message.acked' hooks to implement end-to-end message pub/ack:
```
PktId <-- --> MsgId <-- --> MsgId <-- --> PktId
|<--- Qos --->|<---PubSub--->|<-- Qos -->|
```
## Limit
The design is experimental.
--------------
Plugin Design
--------------
## Overview
**Notice that 0.11.0 release use rebar to manage plugin's deps.**
A plugin is just an erlang application that extends emqttd broker.
The plugin application should be put in "emqttd/plugins/" folder to build.
## Plugin Project
You could create a standalone plugin project outside emqttd, and then add it to "emqttd/plugins/" folder by "git submodule".
Git submodule to compile emqttd_dashboard plugin with the broker, For example:
```
git submodule add https://github.com/emqtt/emqttd_dashboard.git plugins/emqttd_dashboard
make && make dist
```
## plugin.config
**Each plugin should have a 'etc/plugin.config' file**
For example, project structure of emqttd_dashboard plugin:
```
LICENSE
README.md
ebin
etc
priv
rebar.config
src
```
etc/plugin.config for emqttd_dashboard plugin:
```
[
{emqttd_dashboard, [
{listener,
{emqttd_dashboard, 18083, [
{acceptors, 4},
{max_clients, 512}]}}
]}
].
```
## rebar.config
**Plugin should use 'rebar.config' to manage depencies**
emqttd_plugin_pgsql plugin's rebar.config, for example:
```
%% -*- erlang -*-
{deps, [
{epgsql, ".*",{git, "https://github.com/epgsql/epgsql.git", {branch, "master"}}}
]}.
```
## Build emqttd with plugins
Put all the plugins you required in 'plugins/' folder of emqttd project, and then:
```
make && make dist
```
## Load Plugin
'./bin/emqttd_ctl' to load/unload plugin, when emqttd broker started.
```
./bin/emqttd_ctl plugins load emqttd_plugin_demo
./bin/emqttd_ctl plugins unload emqttd_plugin_demo
```
## List Plugins
```
./bin/emqttd_ctl plugins list
```
## API
```
%% Load all active plugins after broker started
emqttd_plugins:load()
%% Load new plugin
emqttd_plugins:load(Name)
%% Unload all active plugins before broker stopped
emqttd_plugins:unload()
%% Unload a plugin
emqttd_plugins:unload(Name)
```

77
docs/source/faq.rst Normal file
View File

@ -0,0 +1,77 @@
===
FAQ
===
##### Q1. Is port 4369 and another random port secure?
```
HI, when start emqttd , I found the port 4369 and another random port(63703) is open, are this security ?
Example:
tcp 0 0 0.0.0.0:4369 0.0.0.0:* LISTEN 13736/epmd
tcp 0 0 0.0.0.0:8083 0.0.0.0:* LISTEN 16745/beam.smp
tcp 0 0 0.0.0.0:8883 0.0.0.0:* LISTEN 16745/beam.smp
tcp 0 0 0.0.0.0:63703 0.0.0.0:* LISTEN 16745/beam.smp
tcp 0 0 0.0.0.0:1883 0.0.0.0:* LISTEN 16745/beam.smp
1883: mqtt no ssl
8883: mqtt with ssl
8083: websocket
```
4369 and some random ports are opened by erlang node for internal communication. Configure your firewall to allow 1883, 8883, 8083 ports to be accessed from outside for security.
Access control of emqttd broker has two layers:
eSockd TCP Acceptor, Ipaddress based Access Control, Example:
```
{access, [{deny, "192.168.1.1"},
{allow, "192.168.1.0/24"},
{deny, all}]}
```
MQTT Subscribe/Publish Access Control by etc/acl.config, Example:
```
{allow, {ipaddr, "127.0.0.1"}, pubsub, ["$SYS/#", "#"]}.
{deny, all, subscribe, ["$SYS/#", {eq, "#"}]}.
{allow, all}.
```
##### Q2. cannot compile emqttd under Chinese folder?
It seems that rebar cannot support Chinese folder name.
##### Q3. emqttd is ready for production ?
The core features are solid and scalable. A small full-time team with many contributors are developing this project. You could submit issues if any feature requests or bug reports.
##### Q4. Benchmark and performance issue
Wiki: https://github.com/emqtt/emqttd/wiki/One-Million-Connections
##### Q5. 'session' identified by clientIDwhen session expired what will happenAll queued messages will be deleted and subscribed topics will be deleted toowhen reconnected need redo subscription(#150)
When a client connected to broker with 'clean session' flag 0, a session identified by clientId will be created. The session will expire after 48 hours(configured in etc/emqttd.config) if no client connections bind with it, and all queued messages and subscriptions will be dropped.
##### Q6. "{max_queued_messages, 100}" in 0.8 release or "{queue, {max_length, 1000},..." means queue for one session or one topicIf it stands for sessionwhile one topic has lots of offline messages(100), the user's other topic offline messages will be flushed? (#150)
For session. Topic just dispatch messages to clients or sessions that matched the subscriptions. Will Flood.
##### Q7. About the retained message, how to config one topic only keep the latest retained message, the older retained messages will be auto deleted?(#150)
By default, the broker only keep the latest retained message of one topic.
##### Q8. When the persistent client with 'clean session' flag 0 is offline but not expired, will the broker put session's subscribed topic new messages to session's queue?(#150)
Yes
##### Q9. If max_length of queue is 100, when the session subscribed topic1 and topic2, what will happen when topic1 fill 70 messages, then topic2 fill 80 messages? After the reconnection, will the session lose first 50 message?(#150)
Lose the oldest 50 messages.

View File

@ -29,8 +29,283 @@ Will Message
HTTP Publish API HTTP Publish API
---------------- ----------------
emqttd support HTTP API to publish message from your application server to MQTT clients. For example::
curl -v --basic -u user:passwd -d "qos=1&retain=0&topic=/a/b/c&message=hello from http..." -k http://localhost:8083/mqtt/publish
HTTP API URL
------------
HTTP POST http://host:8083/mqtt/publish
HTTP Parameters
---------------
Name | Description
--------|---------------
client | ClientId
qos | QoS(0, 1, 2)
retain | Retain(0, 1)
topic | Topic
message | Message
------------------- -------------------
MQTT Over WebSocket MQTT Over WebSocket
------------------- -------------------
## Overview
emqttd 0.7.0 release support MQTT over WebSocket(SSL).
## URI
```
ws(s)://host:8083/mqtt
```
## Sec-WebSocket-Protocol
'mqttv3.1', 'mqttv3.1.1'
-----------------------
Trace Topic or Client
-----------------------
## Overview
emqttd could trace packets received/sent from/to specific client, or trace publish/subscribe to specific topic.
emqttd use lager:trace_file api and write trace log to file.
## Trace Commands
### Trace client
```
./bin/emqttd_ctl trace client "ClientId" "trace_clientid.log"
```
### Trace topic
```
./bin/emqttd_ctl trace topic "Topic" "trace_topic.log"
```
### Stop Trace
```
./bin/emqttd_ctl trace client "ClientId" off
./bin/emqttd_ctl trace topic "Topic" off
```
### Lookup Traces
```
./bin/emqttd_ctl trace list
```
-------------------
$SYS Topics
-------------------
NOTICE: This is the design of 0.9.0 release
## Overview
For emqttd is clustered, $SYS Topics of broker is started with:
```
$SYS/brokers/${node}
```
${node} is erlang node of clustered brokers. For example:
```
$SYS/brokers/emqttd@host1/version
$SYS/brokers/emqttd@host2/version
```
## Broker $SYS Topics
Topic | Description
-------------------------------|------------
$SYS/brokers | Broker nodes
$SYS/brokers/${node}/version | Broker Version
$SYS/brokers/${node}/uptime | Broker Uptime
$SYS/brokers/${node}/datetime | Broker DateTime
$SYS/brokers/${node}/sysdescr | Broker Description
 
## Client $SYS Topics
Start with: $SYS/brokers/${node}/clients/
Topic | Payload(json) | Description
----------------------|---------------------|---------------
${clientid}/connected | {ipaddress: "127.0.0.1", username: "test", session: false, version: 3, connack: 0, ts: 1432648482} | Publish when client connected
${clientid}/disconnected | {reason: "normal" | "keepalive_timeout" | "conn_closed"}
Parameters of 'connected' Payload:
```
ipaddress: "127.0.0.1",
username: "test",
session: false,
protocol: 3,
connack: 0,
ts: 1432648482
```
Parameters of 'disconnected' Payload:
```
reason: normal,
ts: 1432648486
```
## Statistics $SYS Topics
Start with '$SYS/brokers/${node}/stats/'
### Client Stats
Topic | Description
-------------------------------------|------------
clients/count | count of current connected clients
clients/max | max connected clients in the same time
### Session Stats
Topic | Description
-----------------|------------
sessions/count | count of current sessions
sessions/max | max number of sessions
### Subscriber Stats
Topic | Description
------------------|------------
subscriptions/count | count of current subscriptions
subscriptions/max | max number of subscriptions
### Topic Stats
Topic | Description
------------------|------------
topics/count | count of current topics
topics/max | max number of topics
### Queue Stats
Topic | Description
------------------|------------
queues/count | count of current queues
queues/max | max number of queues
## Metrics $SYS Topics
Start with '$SYS/brokers/${node}/metrics/'
### Bytes sent and received
Topic | Description
------------------------------------|------------
bytes/received | MQTT Bytes Received since broker started
bytes/sent | MQTT Bytes Sent since the broker started
### Packets sent and received
Topic | Description
-------------------------|------------
packets/received | MQTT Packets received
packets/sent | MQTT Packets sent
packets/connect | MQTT CONNECT Packet received
packets/connack | MQTT CONNACK Packet sent
packets/publish/received | MQTT PUBLISH packets received
packets/publish/sent | MQTT PUBLISH packets sent
packets/subscribe | MQTT SUBSCRIBE Packets received
packets/suback | MQTT SUBACK packets sent
packets/unsubscribe | MQTT UNSUBSCRIBE Packets received
packets/unsuback | MQTT UNSUBACK Packets sent
packets/pingreq | MQTT PINGREQ packets received
packets/pingresp | MQTT PINGRESP Packets sent
packets/disconnect | MQTT DISCONNECT Packets received
### Messages sent and received
Topic | Description
---------------------------------------|-------------------
messages/received | Messages Received
messages/sent | Messages Sent
messages/retained | Messages Retained
messages/stored | TODO: Messages Stored
messages/dropped | Messages Dropped
## Alarm Topics
Start with '$SYS/brokers/${node}/alarms/'
Topic | Description
-----------------|-------------------
${alarmId}/alert | New Alarm
${alarmId}/clear | Clear Alarm
## Log
'$SYS/brokers/${node}/logs/${severity}'
Severity | Description
-----------|-------------------
debug | Debug Log
info | Info Log
notice | Notice Log
warning | Warning Log
error | Error Log
critical | Critical Log
## Sysmon
Start with '$SYS/brokers/${node}/sysmon/'
Topic | Description
-----------------|-------------------
long_gc | Long GC Warning
long_schedule | Long Schedule
large_heap | Large Heap Warning
busy_port | Busy Port Warning
busy_dist_port | Busy Dist Port
## Log
'$SYS/brokers/${node}/log/${severity}'
Severity | Description
------------|-------------------
debug | Debug
info | Info Log
notice | Notice Log
warning | Warning Log
error | Error Log
critical | Critical Log
alert | Alert Log
## VM Load Topics
Start with '$SYS/brokers/${node}/vm/'
Topic | Description
-----------------|-------------------
memory/* | TODO
cpu/* | TODO
processes/* | TODO
## Sys Interval
sys_interval: 1 minute default

View File

@ -79,9 +79,9 @@ Timeout for FIN-WAIT-2 sockets::
net.ipv4.tcp_fin_timeout = 15 net.ipv4.tcp_fin_timeout = 15
--------- ----------------
Erlang VM Erlang VM Tuning
--------- ----------------
Tuning and optimize the Erlang VM in etc/vm.args file:: Tuning and optimize the Erlang VM in etc/vm.args file::