yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

config

This commit is contained in:
Feng 2016-02-26 22:39:17 +08:00
parent f7d6ba5c78 385ab3ec77
commit 062e154dec
3 changed files with 187 additions and 171 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

190
docs/source/config.rst
View File

@ -5,31 +5,29 @@
Configuration
=============
emqttd消息服务器通过etc/目录下配置文件进行设置，主要配置文件包括:
Configuration files of the broker are under 'etc/' folder, including:
+-------------------+-----------------------------------+
| 配置文件 | 说明 |
| File | Description |
+-------------------+-----------------------------------+
| etc/vm.args | Erlang 虚拟机的参数设置 |
| etc/vm.args | Erlang VM Arguments |
+-------------------+-----------------------------------+
| etc/emqttd.config | emqttd消息服务器参数设置 |
| etc/emqttd.config | emqttd broker Config |
+-------------------+-----------------------------------+
| etc/acl.config | ACL(访问控制规则)设置 |
| etc/acl.config | ACL Config |
+-------------------+-----------------------------------+
| etc/clients.config| 基于ClientId认证设置 |
| etc/clients.config| ClientId Authentication |
+-------------------+-----------------------------------+
| etc/rewrite.config| Rewrite扩展模块规则配置 |
| etc/rewrite.config| Rewrite Rules |
+-------------------+-----------------------------------+
| etc/ssl/* | SSL证书设置 |
| etc/ssl/* | SSL certificate and key files |
+-------------------+-----------------------------------+
-----------
etc/vm.args
-----------
Configure parameters of Erlang VM:
.. code::
Configure and Optimize Erlang VM::
##-------------------------------------------------------------------------
## Name of the node
@ -84,35 +82,33 @@ Configure parameters of Erlang VM:
## Tweak GC to run more often
-env ERL_FULLSWEEP_AFTER 1000
etc/vm.args中两个最重要的参数:
The two most important parameters in etc/vm.args:
+-------+----------------------------------------------------------------------------------------------+
| +P | Erlang虚拟机允许的最大进程数，一个MQTT连接会消耗2个Erlang进程，所以参数值 > 最大连接数 * 2 |
+-------+----------------------------------------------------------------------------------------------+
| +Q | Erlang虚拟机允许的最大Port数量，一个MQTT连接消耗1个Port，所以参数值 > 最大连接数 |
+-------+----------------------------------------------------------------------------------------------+
+-------+---------------------------------------------------------------------------+
| +P | Max number of Erlang proccesses. A MQTT client consumes two proccesses. |
| | The value should be larger than max_clients * 2 |
+-------+---------------------------------------------------------------------------+
| +Q | Max number of Erlang Ports. A MQTT client consumes one port. |
| | The value should be larger than max_clients. |
+-------+---------------------------------------------------------------------------+
etc/vm.args设置Erlang节点名、节点间通信Cookie::
The name and cookie of Erlang Node should be configured when clustering::
-name emqttd@127.0.0.1
-name emqttd@host_or_ip
## Cookie for distributed erlang
-setcookie emqttdsecretcookie
.. NOTE::
Erlang/OTP平台应用多由分布的Erlang节点(进程)组成，每个Erlang节点(进程)需指配一个节点名，用于节点间通信互访。
所有互相通信的Erlang节点(进程)间通过一个共用的Cookie进行安全认证。
-----------------
etc/emqttd.config
-----------------
etc/emqttd.config是消息服务器的核心配置文件。Erlang程序由多个应用(application)组成，每个应用(application)有自身的环境参数，
The main configuration file for emqttd broker.
启动时候通过etc/emqttd.config文件加载。
File Syntax
-----------
etc/emqttd.config文件采用的是Erlang数据格式，kernel, sasl, emqttd是Erlang应用(application)名称，'[]'内是应用的环境参数列表。
The config consists of a list of Erlang Applications and their environments.
.. code:: erlang
@ -131,24 +127,22 @@ etc/emqttd.config文件采用的是Erlang数据格式，kernel, sasl, emqttd是E
]}
].
emqttd.config格式简要说明:
The file adopts Erlang Term Syntax:
1. [ ] : 列表，逗号分隔元素
1. [ ]: List, seperated by comma
2. { }: Tuple, Usually {Env, Value}
3. % : comment
2. { } : 元组，配置元组一般两个元素{Env, Value}
Log Level and File
------------------
3. % : 注释
Log Level and Destination
-------------------------
emqttd消息服务器日志由lager应用(application)提供，日志相关设置在lager应用段落::
Logger of emqttd broker is implemented by 'lager' application::
{lager, [
...
]},
产品环境下默认只开启error日志，日志输出到logs/emqttd_error.log文件。'handlers'段落启用其他级别日志::
Configure log handlers::
{handlers, [
{lager_console_backend, info},
@ -172,47 +166,45 @@ emqttd消息服务器日志由lager应用(application)提供，日志相关设
]}
]}
.. WARNING:: 过多日志打印严重影响服务器性能，产品环境下建议开启error级别日志。
Broker Parameters
emqttd Application
------------------
emqttd消息服务器参数设置在emqttd应用段落，包括用户认证与访问控制设置，MQTT协议、会话、队列设置，扩展模块设置，TCP服务监听器设置::
The MQTT broker is implemented by erlang 'emqttd' application::
{emqttd, [
%% 用户认证与访问控制设置
%% Authentication and Authorization
{access, [
...
]},
%% MQTT连接、协议、会话、队列设置
%% MQTT Protocol Options
{mqtt, [
...
]},
%% 消息服务器设置
%% Broker Options
{broker, [
...
]},
%% 扩展模块设置
%% Modules
{modules, [
...
]},
%% 插件目录设置
%% Plugins
{plugins, [
...
]},
%% TCP监听器设置
%% Listeners
{listeners, [
...
]},
%% Erlang虚拟机监控设置
%% Erlang System Monitor
{sysmon, [
]}
]}
access用户认证设置
------------------
Authentication
--------------
emqttd消息服务器认证由一系列认证模块(module)或插件(plugin)提供，系统默认支持用户名、ClientID、LDAP、匿名(anonymouse)认证模块::
@ -511,10 +503,8 @@ broker pubsub路由设置
{route_aging, 5}
]},
broker bridge桥接参数
-----------------------
桥接参数设置::
Bridge Parameters
-----------------
{bridge, [
%% 最大缓存桥接消息数
@ -525,16 +515,14 @@ broker bridge桥接参数
]}
modules扩展模块设置
-----------------------
Enable Modules
--------------
emqtt消息服务器支持简单的扩展模块，用于定制服务器功能。默认支持presence、subscription、rewrite模块。
'presence'扩展模块会向$SYS主题(Topic)发布客户端上下线消息::
'presence' module will publish presence message to $SYS topic when a client connected or disconnected::
{presence, [{qos, 0}]},
'subscription'扩展模块支持客户端上线时，自动订阅或恢复订阅某些主题(Topic)::
'subscription' module forces the client to subscribe some topics when connected to the broker::
%% Subscribe topics automatically when client connected
{subscription, [
@ -548,15 +536,14 @@ emqtt消息服务器支持简单的扩展模块，用于定制服务器功能。
{"$Q/client/$c", 1}
]}
'rewrite'扩展模块支持重写主题(Topic)路径, 重写规则定义在etc/rewrite.config文件::
'rewrite' module supports to rewrite the topic path::
%% Rewrite rules
%% {rewrite, [{file, "etc/rewrite.config"}]}
{rewrite, [{file, "etc/rewrite.config"}]}
关于扩展模块详细介绍，请参考<用户指南>文档。
plugins插件目录设置
-------------------
Plugins Folder
--------------
.. code:: erlang
@ -569,19 +556,21 @@ plugins插件目录设置
]},
listeners监听器设置
-----------------------
TCP Listeners
-------------
emqttd消息服务器开启的MQTT协议、HTTP协议服务端，可通过listener设置TCP服务端口、最大允许连接数等参数。
Congfigure the TCP listener for MQTT, MQTT(SSL) and HTTP Protocols.
emqttd消息服务器默认开启的TCP服务端口包括:
The most important parameter is 'max_clients' - max concurrent clients allowed.
The TCP Ports occupied by emqttd broker by default:
+-----------+-----------------------------------+
| 1883 | MQTT协议端口 |
| 1883 | MQTT Port |
+-----------+-----------------------------------+
| 8883 | MQTT(SSL)端口 |
| 8883 | MQTT(SSL) Port |
+-----------+-----------------------------------+
| 8083 | MQTT(WebSocket), HTTP API端口 |
| 8083 | MQTT(WebSocket), HTTP API Port |
+-----------+-----------------------------------+
.. code:: erlang
@ -669,19 +658,19 @@ emqttd消息服务器默认开启的TCP服务端口包括:
]}
]},
listener参数说明:
Listener Parameters:
+-------------+-----------------------------------------------------------+
| acceptors | TCP Acceptor池 |
+-------------+-----------------------------------------------------------+
| max_clients | 最大允许TCP连接数 |
+-------------+-----------------------------------------------------------+
| access | 允许访问的IP地址段设置，例如: [{allow, "192.168.1.0/24"}] |
+-------------+-----------------------------------------------------------+
| connopts | 连接限速配置，例如限速10KB/秒: {rate_limit, "100,10"} |
+-------------+-----------------------------------------------------------+
| sockopts | Socket参数设置 |
+-------------+-----------------------------------------------------------+
+-------------+----------------------------------------------------------------+
| acceptors | TCP Acceptor Pool |
+-------------+----------------------------------------------------------------+
| max_clients | Maximum number of concurrent TCP connections allowed |
+-------------+----------------------------------------------------------------+
| access | Access Control by IP, for example: [{allow, "192.168.1.0/24"}] |
+-------------+----------------------------------------------------------------+
| connopts | Rate Limit Control, for example: {rate_limit, "100,10"} |
+-------------+----------------------------------------------------------------+
| sockopts | TCP Socket parameters |
+-------------+----------------------------------------------------------------+
.. _config_acl:
@ -689,9 +678,21 @@ listener参数说明:
etc/acl.config
--------------
emqttd消息服务器默认访问控制规则配置在etc/acl.config文件。
The 'etc/acl.config' is the default ACL config for emqttd broker. The rules by default::
访问控制规则采用Erlang元组格式，访问控制模块逐条匹配规则::
%% Allow 'dashboard' to subscribe '$SYS/#'
{allow, {user, "dashboard"}, subscribe, ["$SYS/#"]}.
%% Allow clients from localhost to subscribe any topics
{allow, {ipaddr, "127.0.0.1"}, pubsub, ["$SYS/#", "#"]}.
%% Deny clients to subscribe '$SYS#' and '#'
{deny, all, subscribe, ["$SYS/#", {eq, "#"}]}.
%% Allow all by default
{allow, all}.
An ACL rule is an Erlang tuple. The Access control module of emqttd broker matches the rule one by one from top to bottom::
--------- --------- ---------
Client -> | Rule1 | --nomatch--> | Rule2 | --nomatch--> | Rule3 | --> Default
@ -701,27 +702,13 @@ emqttd消息服务器默认访问控制规则配置在etc/acl.config文件。
\|/ \|/ \|/
allow | deny allow | deny allow | deny
etc/acl.config文件默认规则设置::
%% 允许'dashboard'用户订阅 '$SYS/#'
{allow, {user, "dashboard"}, subscribe, ["$SYS/#"]}.
%% 允许本机用户发布订阅全部主题
{allow, {ipaddr, "127.0.0.1"}, pubsub, ["$SYS/#", "#"]}.
%% 拒绝用户订阅'$SYS#'与'#'主题
{deny, all, subscribe, ["$SYS/#", {eq, "#"}]}.
%% 上述规则无匹配，允许
{allow, all}.
.. _config_rewrite:
------------------
etc/rewrite.config
------------------
Rewrite扩展模块的规则配置文件，示例配置::
The Rewrite Rules for emqttd_mod_rewrite::
{topic, "x/#", [
{rewrite, "^x/y/(.+)$", "z/y/$1"},
@ -731,3 +718,4 @@ Rewrite扩展模块的规则配置文件，示例配置::
{topic, "y/+/z/#", [
{rewrite, "^y/(.+)/z/(.+)$", "y/z/$2"}
]}.

167
docs/source/getstarted.rst
View File

@ -9,19 +9,19 @@ Get Started
Overview
--------
emqttd(Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. Erlang/OTP is a concurrent, fault-tolerant, soft-realtime and distributed programming platform. MQTT is anextremely lightweight publish/subscribe messaging protocol powering IoT, M2M applications.
emqttd(Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. Erlang/OTP is a concurrent, fault-tolerant, soft-realtime and distributed programming platform. MQTT is an extremely lightweight publish/subscribe messaging protocol powering IoT, M2M and Mobile applications.
The emqttd project is aimed to implement a scalable, distributed, extensible open-source MQTT broker for IoT, M2M and Mobile applications that hope to handle ten millions of concurrent MQTT clients.
The emqttd broker is:
Highlights of the emqttd broker:
* Full MQTT V3.1/3.1.1 Protocol Specifications Support
* Easy to Install - Quick Install on Linux, FreeBSD, Mac and Windows
* Massively scalable - Scaling to 1 million connections on a single server
* Cluster and Bridge Support
* Easy to extend - Hooks and plugins to customize or extend the broker
* Pluggable Authentication - LDAP, MySQL, PostgreSQL, Redis Authentication Plugins
--------
Features
--------
@ -32,22 +32,22 @@ Features
* Retained Message
* Last Will Message
* TCP/SSL Connection
* MQTT Over Websocket(SSL)
* MQTT Over WebSocket(SSL)
* HTTP Publish API
* STOMP protocol
* STOMP over SockJS
* $SYS/# Topics
* Client Authentication with clientId, ipaddress
* Client Authentication with username, password
* Client ACL control with ipaddress, clientid, username
* LDAP, Redis, MySQL, PostgreSQL authentication
* Cluster brokers on several servers.
* ClientID Authentication
* IpAddress Authentication
* Username and Password Authentication
* Access control based on IpAddress, ClientID, Username
* Authentication with LDAP, Redis, MySQL, PostgreSQL
* Cluster brokers on several servers
* Bridge brokers locally or remotely
* mosquitto, RSMB bridge
* Extensible architecture with Hooks, Modules and Plugins
* Passed eclipse paho interoperability tests
-----------
Quick Start
-----------
@ -55,7 +55,11 @@ Quick Start
Download and Install
--------------------
Download binary package for Linux, Mac, FreeBSD and Windows platform from http://emqtt.io/downloads.
The emqttd broker is cross-platform, could be deployed on Linux, Mac, FreeBSD, Windows and Raspberry Pi.
Download binary package from: http://emqtt.io/downloads.
Installing on Mac, For example:
.. code:: console
@ -71,9 +75,9 @@ Download binary package for Linux, Mac, FreeBSD and Windows platform from http:/
./bin/emqttd stop
Installing from Source
-----------------------
----------------------
.. NOTE:: emqttd requires Erlang R17+ to build.
.. NOTE:: emqttd broker requires Erlang R17+ to build.
.. code:: console
@ -81,39 +85,54 @@ Installing from Source
cd emqttd && make && make dist
cd rel/emqttd && ./bin/emqttd console
-------------
Web Dashboard
-------------
.. image:: ./_static/images/dashboard.png
A Web Dashboard will be loaded automatically when the emqttd broker is started successfully.
The Dashboard helps to check running status of the broker, monitor statistics and metrics of MQTT packets, query clients, sessions, topics and subscriptions.
+------------------+---------------------------+
| Default Address | http://localhost:18083 |
+------------------+---------------------------+
| Default User | admin |
+------------------+---------------------------+
| Default Password | public |
+------------------+---------------------------+
.. image:: ./_static/images/dashboard.png
-------------------
Modules and Plugins
-------------------
The emqttd broker could be extended by modules and plugins.
The Authentication and ACL mechanism is usually implemented by a Module or Plugin.
Modules
-------
+-------------------------+-----------------------------------+
| emqttd_auth_clientid | ClientId认证 |
+-------------------------+-----------------------------------+
| emqttd_auth_username | 用户名密码认证 |
+-------------------------+-----------------------------------+
| emqttd_auth_ldap | LDAP认证 |
+-------------------------+-----------------------------------+
| emqttd_mod_presence | 客户端上下线状态消息发布 |
+-------------------------+-----------------------------------+
| emqttd_mod_subscription | 客户端上线自动主题订阅 |
+-------------------------+-----------------------------------+
| emqttd_mod_rewrite | 重写客户端订阅主题(Topic) |
+-------------------------+-----------------------------------+
+-------------------------+--------------------------------------------+
| emqttd_auth_clientid | Authentication with ClientId |
+-------------------------+--------------------------------------------+
| emqttd_auth_username | Authentication with Username and Password |
+-------------------------+--------------------------------------------+
| emqttd_auth_ldap | Authentication with LDAP |
+-------------------------+--------------------------------------------+
| emqttd_mod_presence | Publish presence message to $SYS topics |
| | when client connected or disconnected |
+-------------------------+--------------------------------------------+
| emqttd_mod_subscription | Subscribe topics automatically when client |
| | connected |
+-------------------------+--------------------------------------------+
| emqttd_mod_rewrite | Topics rewrite like HTTP rewrite module |
+-------------------------+--------------------------------------------+
扩展模块通过'etc/emqttd.config'配置文件的auth, modules段落启用。
Configure the 'auth', 'module' paragraph in 'etc/emqttd.config' to enable a module.
例如启用用户名密码认证::
Enable 'emqttd_auth_username' module::
{access, [
%% Authetication. Anonymous Default
@ -123,7 +142,7 @@ Modules
...
启用客户端状态发布模块::
Enable 'emqttd_mod_presence' module::
{modules, [
%% Client presence management module.
@ -131,71 +150,70 @@ Modules
{presence, [{qos, 0}]}
Plugins
--------
-------
+-------------------------+-----------------------------------+
| emqttd_plugin_template | 插件模版与演示代码 |
+-------------------------+-----------------------------------+
| emqttd_dashboard | Web管理控制台，默认加载 |
+-------------------------+-----------------------------------+
| emqttd_plugin_mysql | MySQL认证插件 |
+-------------------------+-----------------------------------+
| emqttd_plugin_pgsql | PostgreSQL认证插件 |
+-------------------------+-----------------------------------+
| emqttd_plugin_redis | Redis认证插件 |
+-------------------------+-----------------------------------+
| emqttd_plugin_mongo | MongoDB认证插件 |
+-------------------------+-----------------------------------+
| emqttd_stomp | Stomp协议插件 |
+-------------------------+-----------------------------------+
| emqttd_sockjs | SockJS插件 |
+-------------------------+-----------------------------------+
| emqttd_recon | Recon优化调测插件 |
+-------------------------+-----------------------------------+
A plugin is an Erlang application to extend the emqttd broker.
扩展插件通过'bin/emqttd_ctl'管理命令行，加载启动运行。
+----------------------------+-----------------------------------+
| `emqttd_plugin_template`_ | Plugin template and demo |
+----------------------------+-----------------------------------+
| `emqttd_dashboard`_ | Web Dashboard |
+----------------------------+-----------------------------------+
| `emqttd_plugin_mysql`_ | Authentication with MySQL |
+----------------------------+-----------------------------------+
| `emqttd_plugin_pgsql`_ | Authentication with PostgreSQL |
+----------------------------+-----------------------------------+
| `emqttd_plugin_redis`_ | Authentication with Redis |
+----------------------------+-----------------------------------+
| `emqttd_plugin_mongo`_ | Authentication with MongoDB |
+----------------------------+-----------------------------------+
| `emqttd_stomp`_ | STOMP Protocol Plugin |
+----------------------------+-----------------------------------+
| `emqttd_sockjs`_ | SockJS(Stomp) Plugin |
+----------------------------+-----------------------------------+
| `emqttd_recon`_ | Recon Plugin |
+----------------------------+-----------------------------------+
例如启用PostgreSQL认证插件::
A plugin could be enabled by 'bin/emqttd_ctl plugins load' command.
For example, enable 'emqttd_plugin_pgsql' plugin::
./bin/emqttd_ctl plugins load emqttd_plugin_pgsql
----------------------------------
-----------------------
One million Connections
----------------------------------
-----------------------
Latest release of the emqttd broker is scaling to 1.3 million MQTT connections on a 12 Core, 32G CentOS server.
.. NOTE::
emqttd消息服务器默认设置，允许最大客户端连接是512，因为大部分操作系统'ulimit -n'限制为1024。
The emqttd broker only allows 512 concurrent connections by default, for 'ulimit -n' limit is 1024 on most platform.
emqttd消息服务器当前版本，连接压力测试到130万线，8核心/32G内存的CentOS云服务器。
操作系统内核参数、TCP协议栈参数、Erlang虚拟机参数、emqttd最大允许连接数设置简述如下：
We need tune the OS Kernel, TCP Stack, Erlang VM and emqttd broker for one million connections benchmark.
Linux Kernel Parameters
-----------------------
# 2M - 系统所有进程可打开的文件数量::
.. code::
# 2M:
sysctl -w fs.file-max=2097152
sysctl -w fs.nr_open=2097152
# 1M - 系统允许当前进程打开的文件数量::
# 1M:
ulimit -n 1048576
TCP Stack Parameters
-----------------------
# backlog - Socket监听队列长度::
--------------------
.. code::
# backlog
sysctl -w net.core.somaxconn=65536
Erlang VM
-----------------
---------
emqttd/etc/vm.args::
@ -210,8 +228,8 @@ emqttd/etc/vm.args::
-env ERTS_MAX_PORTS 1048576
emqttd.config
-----------------
emqttd broker
-------------
emqttd/etc/emqttd.config::
@ -239,6 +257,7 @@ Test Client
sysctl -w net.ipv4.ip_local_port_range="500 65535"
echo 1000000 > /proc/sys/fs/nr_open
ulimit -n 100000
---------------------
MQTT Client Libraries
@ -261,3 +280,13 @@ GitHub: https://github.com/emqtt
.. _CocoaMQTT: https://github.com/emqtt/CocoaMQTT
.. _QMQTT: https://github.com/emqtt/qmqtt
.. _emqttd_plugin_template: https://github.com/emqtt/emqttd_plugin_template
.. _emqttd_dashboard: https://github.com/emqtt/emqttd_dashboard
.. _emqttd_plugin_mysql: https://github.com/emqtt/emqttd_plugin_mysql
.. _emqttd_plugin_pgsql: https://github.com/emqtt/emqttd_plugin_pgsql
.. _emqttd_plugin_redis: https://github.com/emqtt/emqttd_plugin_redis
.. _emqttd_plugin_mongo: https://github.com/emqtt/emqttd_plugin_mongo
.. _emqttd_stomp: https://github.com/emqtt/emqttd_stomp
.. _emqttd_sockjs: https://github.com/emqtt/emqttd_sockjs
.. _emqttd_recon: https://github.com/emqtt/emqttd_recon

1
docs/source/index.rst
View File

@ -47,7 +47,6 @@ Contents:
config
cluster
bridge
guide
commands
plugins