diff --git a/docs/source/config.rst b/docs/source/config.rst index 4acf4e27b..461dde235 100644 --- a/docs/source/config.rst +++ b/docs/source/config.rst @@ -5,31 +5,29 @@ Configuration ============= -emqttd消息服务器通过etc/目录下配置文件进行设置，主要配置文件包括: +Configuration files of the broker are under 'etc/' folder, including: +-------------------+-----------------------------------+ -| 配置文件 | 说明 | +| File | Description | +-------------------+-----------------------------------+ -| etc/vm.args | Erlang 虚拟机的参数设置 | +| etc/vm.args | Erlang VM Arguments | +-------------------+-----------------------------------+ -| etc/emqttd.config | emqttd消息服务器参数设置 | +| etc/emqttd.config | emqttd broker Config | +-------------------+-----------------------------------+ -| etc/acl.config | ACL(访问控制规则)设置 | +| etc/acl.config | ACL Config | +-------------------+-----------------------------------+ -| etc/clients.config| 基于ClientId认证设置 | +| etc/clients.config| ClientId Authentication | +-------------------+-----------------------------------+ -| etc/rewrite.config| Rewrite扩展模块规则配置 | +| etc/rewrite.config| Rewrite Rules | +-------------------+-----------------------------------+ -| etc/ssl/* | SSL证书设置 | +| etc/ssl/* | SSL certificate and key files | +-------------------+-----------------------------------+ ----------- etc/vm.args ----------- -Configure parameters of Erlang VM: - -.. code:: +Configure and Optimize Erlang VM:: ##------------------------------------------------------------------------- ## Name of the node @@ -84,35 +82,33 @@ Configure parameters of Erlang VM: ## Tweak GC to run more often -env ERL_FULLSWEEP_AFTER 1000 -etc/vm.args中两个最重要的参数: +The two most important parameters in etc/vm.args: -+-------+----------------------------------------------------------------------------------------------+ -| +P | Erlang虚拟机允许的最大进程数，一个MQTT连接会消耗2个Erlang进程，所以参数值 > 最大连接数 * 2 | -+-------+----------------------------------------------------------------------------------------------+ -| +Q | Erlang虚拟机允许的最大Port数量，一个MQTT连接消耗1个Port，所以参数值 > 最大连接数 | -+-------+----------------------------------------------------------------------------------------------+ ++-------+---------------------------------------------------------------------------+ +| +P | Max number of Erlang proccesses. A MQTT client consumes two proccesses. | +| | The value should be larger than max_clients * 2 | ++-------+---------------------------------------------------------------------------+ +| +Q | Max number of Erlang Ports. A MQTT client consumes one port. | +| | The value should be larger than max_clients. | ++-------+---------------------------------------------------------------------------+ -etc/vm.args设置Erlang节点名、节点间通信Cookie:: +The name and cookie of Erlang Node should be configured when clustering:: - -name emqttd@127.0.0.1 + -name emqttd@host_or_ip ## Cookie for distributed erlang -setcookie emqttdsecretcookie -.. NOTE:: - - Erlang/OTP平台应用多由分布的Erlang节点(进程)组成，每个Erlang节点(进程)需指配一个节点名，用于节点间通信互访。 - 所有互相通信的Erlang节点(进程)间通过一个共用的Cookie进行安全认证。 - ----------------- etc/emqttd.config ----------------- -etc/emqttd.config是消息服务器的核心配置文件。Erlang程序由多个应用(application)组成，每个应用(application)有自身的环境参数， +The main configuration file for emqttd broker. -启动时候通过etc/emqttd.config文件加载。 +File Syntax +----------- -etc/emqttd.config文件采用的是Erlang数据格式，kernel, sasl, emqttd是Erlang应用(application)名称，'[]'内是应用的环境参数列表。 +The config consists of a list of Erlang Applications and their environments. .. code:: erlang @@ -131,24 +127,22 @@ etc/emqttd.config文件采用的是Erlang数据格式，kernel, sasl, emqttd是E ]} ]. -emqttd.config格式简要说明: +The file adopts Erlang Term Syntax: -1. [ ] : 列表，逗号分隔元素 +1. [ ]: List, seperated by comma +2. { }: Tuple, Usually {Env, Value} +3. % : comment -2. { } : 元组，配置元组一般两个元素{Env, Value} +Log Level and File +------------------ -3. % : 注释 - -Log Level and Destination -------------------------- - -emqttd消息服务器日志由lager应用(application)提供，日志相关设置在lager应用段落:: +Logger of emqttd broker is implemented by 'lager' application:: {lager, [ ... ]}, -产品环境下默认只开启error日志，日志输出到logs/emqttd_error.log文件。'handlers'段落启用其他级别日志:: +Configure log handlers:: {handlers, [ {lager_console_backend, info}, @@ -172,47 +166,45 @@ emqttd消息服务器日志由lager应用(application)提供，日志相关设 ]} ]} -.. WARNING:: 过多日志打印严重影响服务器性能，产品环境下建议开启error级别日志。 - -Broker Parameters +emqttd Application ------------------ -emqttd消息服务器参数设置在emqttd应用段落，包括用户认证与访问控制设置，MQTT协议、会话、队列设置，扩展模块设置，TCP服务监听器设置:: +The MQTT broker is implemented by erlang 'emqttd' application:: {emqttd, [ - %% 用户认证与访问控制设置 + %% Authentication and Authorization {access, [ ... ]}, - %% MQTT连接、协议、会话、队列设置 + %% MQTT Protocol Options {mqtt, [ ... ]}, - %% 消息服务器设置 + %% Broker Options {broker, [ ... ]}, - %% 扩展模块设置 + %% Modules {modules, [ ... ]}, - %% 插件目录设置 + %% Plugins {plugins, [ ... ]}, - %% TCP监听器设置 + %% Listeners {listeners, [ ... ]}, - %% Erlang虚拟机监控设置 + %% Erlang System Monitor {sysmon, [ ]} ]} -access用户认证设置 ------------------- +Authentication +-------------- emqttd消息服务器认证由一系列认证模块(module)或插件(plugin)提供，系统默认支持用户名、ClientID、LDAP、匿名(anonymouse)认证模块:: @@ -511,10 +503,8 @@ broker pubsub路由设置 {route_aging, 5} ]}, -broker bridge桥接参数 ------------------------ - -桥接参数设置:: +Bridge Parameters +----------------- {bridge, [ %% 最大缓存桥接消息数 @@ -525,16 +515,14 @@ broker bridge桥接参数 ]} -modules扩展模块设置 ------------------------ +Enable Modules +-------------- -emqtt消息服务器支持简单的扩展模块，用于定制服务器功能。默认支持presence、subscription、rewrite模块。 - -'presence'扩展模块会向$SYS主题(Topic)发布客户端上下线消息:: +'presence' module will publish presence message to $SYS topic when a client connected or disconnected:: {presence, [{qos, 0}]}, -'subscription'扩展模块支持客户端上线时，自动订阅或恢复订阅某些主题(Topic):: +'subscription' module forces the client to subscribe some topics when connected to the broker:: %% Subscribe topics automatically when client connected {subscription, [ @@ -548,15 +536,14 @@ emqtt消息服务器支持简单的扩展模块，用于定制服务器功能。 {"$Q/client/$c", 1} ]} -'rewrite'扩展模块支持重写主题(Topic)路径, 重写规则定义在etc/rewrite.config文件:: +'rewrite' module supports to rewrite the topic path:: %% Rewrite rules - %% {rewrite, [{file, "etc/rewrite.config"}]} + {rewrite, [{file, "etc/rewrite.config"}]} -关于扩展模块详细介绍，请参考<用户指南>文档。 -plugins插件目录设置 -------------------- +Plugins Folder +-------------- .. code:: erlang @@ -569,19 +556,21 @@ plugins插件目录设置 ]}, -listeners监听器设置 ------------------------ +TCP Listeners +------------- -emqttd消息服务器开启的MQTT协议、HTTP协议服务端，可通过listener设置TCP服务端口、最大允许连接数等参数。 +Congfigure the TCP listener for MQTT, MQTT(SSL) and HTTP Protocols. -emqttd消息服务器默认开启的TCP服务端口包括: +The most important parameter is 'max_clients' - max concurrent clients allowed. + +The TCP Ports occupied by emqttd broker by default: +-----------+-----------------------------------+ -| 1883 | MQTT协议端口 | +| 1883 | MQTT Port | +-----------+-----------------------------------+ -| 8883 | MQTT(SSL)端口 | +| 8883 | MQTT(SSL) Port | +-----------+-----------------------------------+ -| 8083 | MQTT(WebSocket), HTTP API端口 | +| 8083 | MQTT(WebSocket), HTTP API Port | +-----------+-----------------------------------+ .. code:: erlang @@ -669,19 +658,19 @@ emqttd消息服务器默认开启的TCP服务端口包括: ]} ]}, -listener参数说明: +Listener Parameters: -+-------------+-----------------------------------------------------------+ -| acceptors | TCP Acceptor池 | -+-------------+-----------------------------------------------------------+ -| max_clients | 最大允许TCP连接数 | -+-------------+-----------------------------------------------------------+ -| access | 允许访问的IP地址段设置，例如: [{allow, "192.168.1.0/24"}] | -+-------------+-----------------------------------------------------------+ -| connopts | 连接限速配置，例如限速10KB/秒: {rate_limit, "100,10"} | -+-------------+-----------------------------------------------------------+ -| sockopts | Socket参数设置 | -+-------------+-----------------------------------------------------------+ ++-------------+----------------------------------------------------------------+ +| acceptors | TCP Acceptor Pool | ++-------------+----------------------------------------------------------------+ +| max_clients | Maximum number of concurrent TCP connections allowed | ++-------------+----------------------------------------------------------------+ +| access | Access Control by IP, for example: [{allow, "192.168.1.0/24"}] | ++-------------+----------------------------------------------------------------+ +| connopts | Rate Limit Control, for example: {rate_limit, "100,10"} | ++-------------+----------------------------------------------------------------+ +| sockopts | TCP Socket parameters | ++-------------+----------------------------------------------------------------+ .. _config_acl: @@ -689,9 +678,21 @@ listener参数说明: etc/acl.config -------------- -emqttd消息服务器默认访问控制规则配置在etc/acl.config文件。 +The 'etc/acl.config' is the default ACL config for emqttd broker. The rules by default:: -访问控制规则采用Erlang元组格式，访问控制模块逐条匹配规则:: + %% Allow 'dashboard' to subscribe '$SYS/#' + {allow, {user, "dashboard"}, subscribe, ["$SYS/#"]}. + + %% Allow clients from localhost to subscribe any topics + {allow, {ipaddr, "127.0.0.1"}, pubsub, ["$SYS/#", "#"]}. + + %% Deny clients to subscribe '$SYS#' and '#' + {deny, all, subscribe, ["$SYS/#", {eq, "#"}]}. + + %% Allow all by default + {allow, all}. + +An ACL rule is an Erlang tuple. The Access control module of emqttd broker matches the rule one by one from top to bottom:: --------- --------- --------- Client -> | Rule1 | --nomatch--> | Rule2 | --nomatch--> | Rule3 | --> Default @@ -701,27 +702,13 @@ emqttd消息服务器默认访问控制规则配置在etc/acl.config文件。 \|/ \|/ \|/ allow | deny allow | deny allow | deny -etc/acl.config文件默认规则设置:: - - %% 允许'dashboard'用户订阅 '$SYS/#' - {allow, {user, "dashboard"}, subscribe, ["$SYS/#"]}. - - %% 允许本机用户发布订阅全部主题 - {allow, {ipaddr, "127.0.0.1"}, pubsub, ["$SYS/#", "#"]}. - - %% 拒绝用户订阅'$SYS#'与'#'主题 - {deny, all, subscribe, ["$SYS/#", {eq, "#"}]}. - - %% 上述规则无匹配，允许 - {allow, all}. - .. _config_rewrite: ------------------ etc/rewrite.config ------------------ -Rewrite扩展模块的规则配置文件，示例配置:: +The Rewrite Rules for emqttd_mod_rewrite:: {topic, "x/#", [ {rewrite, "^x/y/(.+)$", "z/y/$1"}, @@ -731,3 +718,4 @@ Rewrite扩展模块的规则配置文件，示例配置:: {topic, "y/+/z/#", [ {rewrite, "^y/(.+)/z/(.+)$", "y/z/$2"} ]}. + diff --git a/docs/source/getstarted.rst b/docs/source/getstarted.rst index 2b0b407e8..a9c9fe8a2 100644 --- a/docs/source/getstarted.rst +++ b/docs/source/getstarted.rst @@ -9,19 +9,19 @@ Get Started Overview -------- -emqttd(Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. Erlang/OTP is a concurrent, fault-tolerant, soft-realtime and distributed programming platform. MQTT is anextremely lightweight publish/subscribe messaging protocol powering IoT, M2M applications. +emqttd(Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. Erlang/OTP is a concurrent, fault-tolerant, soft-realtime and distributed programming platform. MQTT is an extremely lightweight publish/subscribe messaging protocol powering IoT, M2M and Mobile applications. The emqttd project is aimed to implement a scalable, distributed, extensible open-source MQTT broker for IoT, M2M and Mobile applications that hope to handle ten millions of concurrent MQTT clients. -The emqttd broker is: +Highlights of the emqttd broker: * Full MQTT V3.1/3.1.1 Protocol Specifications Support * Easy to Install - Quick Install on Linux, FreeBSD, Mac and Windows * Massively scalable - Scaling to 1 million connections on a single server +* Cluster and Bridge Support * Easy to extend - Hooks and plugins to customize or extend the broker * Pluggable Authentication - LDAP, MySQL, PostgreSQL, Redis Authentication Plugins - -------- Features -------- @@ -32,22 +32,22 @@ Features * Retained Message * Last Will Message * TCP/SSL Connection -* MQTT Over Websocket(SSL) +* MQTT Over WebSocket(SSL) * HTTP Publish API * STOMP protocol * STOMP over SockJS * $SYS/# Topics -* Client Authentication with clientId, ipaddress -* Client Authentication with username, password -* Client ACL control with ipaddress, clientid, username -* LDAP, Redis, MySQL, PostgreSQL authentication -* Cluster brokers on several servers. +* ClientID Authentication +* IpAddress Authentication +* Username and Password Authentication +* Access control based on IpAddress, ClientID, Username +* Authentication with LDAP, Redis, MySQL, PostgreSQL +* Cluster brokers on several servers * Bridge brokers locally or remotely * mosquitto, RSMB bridge * Extensible architecture with Hooks, Modules and Plugins * Passed eclipse paho interoperability tests - ----------- Quick Start ----------- @@ -55,7 +55,11 @@ Quick Start Download and Install -------------------- -Download binary package for Linux, Mac, FreeBSD and Windows platform from http://emqtt.io/downloads. +The emqttd broker is cross-platform, could be deployed on Linux, Mac, FreeBSD, Windows and Raspberry Pi. + +Download binary package from: http://emqtt.io/downloads. + +Installing on Mac, For example: .. code:: console @@ -71,9 +75,9 @@ Download binary package for Linux, Mac, FreeBSD and Windows platform from http:/ ./bin/emqttd stop Installing from Source ------------------------ +---------------------- -.. NOTE:: emqttd requires Erlang R17+ to build. +.. NOTE:: emqttd broker requires Erlang R17+ to build. .. code:: console @@ -81,39 +85,54 @@ Installing from Source cd emqttd && make && make dist + cd rel/emqttd && ./bin/emqttd console + ------------- Web Dashboard ------------- -.. image:: ./_static/images/dashboard.png +A Web Dashboard will be loaded automatically when the emqttd broker is started successfully. +The Dashboard helps to check running status of the broker, monitor statistics and metrics of MQTT packets, query clients, sessions, topics and subscriptions. + ++------------------+---------------------------+ +| Default Address | http://localhost:18083 | ++------------------+---------------------------+ +| Default User | admin | ++------------------+---------------------------+ +| Default Password | public | ++------------------+---------------------------+ + +.. image:: ./_static/images/dashboard.png ------------------- Modules and Plugins ------------------- -The emqttd broker could be extended by modules and plugins. +The Authentication and ACL mechanism is usually implemented by a Module or Plugin. Modules ------- -+-------------------------+-----------------------------------+ -| emqttd_auth_clientid | ClientId认证 | -+-------------------------+-----------------------------------+ -| emqttd_auth_username | 用户名密码认证 | -+-------------------------+-----------------------------------+ -| emqttd_auth_ldap | LDAP认证 | -+-------------------------+-----------------------------------+ -| emqttd_mod_presence | 客户端上下线状态消息发布 | -+-------------------------+-----------------------------------+ -| emqttd_mod_subscription | 客户端上线自动主题订阅 | -+-------------------------+-----------------------------------+ -| emqttd_mod_rewrite | 重写客户端订阅主题(Topic) | -+-------------------------+-----------------------------------+ ++-------------------------+--------------------------------------------+ +| emqttd_auth_clientid | Authentication with ClientId | ++-------------------------+--------------------------------------------+ +| emqttd_auth_username | Authentication with Username and Password | ++-------------------------+--------------------------------------------+ +| emqttd_auth_ldap | Authentication with LDAP | ++-------------------------+--------------------------------------------+ +| emqttd_mod_presence | Publish presence message to $SYS topics | +| | when client connected or disconnected | ++-------------------------+--------------------------------------------+ +| emqttd_mod_subscription | Subscribe topics automatically when client | +| | connected | ++-------------------------+--------------------------------------------+ +| emqttd_mod_rewrite | Topics rewrite like HTTP rewrite module | ++-------------------------+--------------------------------------------+ -扩展模块通过'etc/emqttd.config'配置文件的auth, modules段落启用。 +Configure the 'auth', 'module' paragraph in 'etc/emqttd.config' to enable a module. -例如启用用户名密码认证:: +Enable 'emqttd_auth_username' module:: {access, [ %% Authetication. Anonymous Default @@ -123,7 +142,7 @@ Modules ... -启用客户端状态发布模块:: +Enable 'emqttd_mod_presence' module:: {modules, [ %% Client presence management module. @@ -131,71 +150,70 @@ Modules {presence, [{qos, 0}]} Plugins --------- +------- -+-------------------------+-----------------------------------+ -| emqttd_plugin_template | 插件模版与演示代码 | -+-------------------------+-----------------------------------+ -| emqttd_dashboard | Web管理控制台，默认加载 | -+-------------------------+-----------------------------------+ -| emqttd_plugin_mysql | MySQL认证插件 | -+-------------------------+-----------------------------------+ -| emqttd_plugin_pgsql | PostgreSQL认证插件 | -+-------------------------+-----------------------------------+ -| emqttd_plugin_redis | Redis认证插件 | -+-------------------------+-----------------------------------+ -| emqttd_plugin_mongo | MongoDB认证插件 | -+-------------------------+-----------------------------------+ -| emqttd_stomp | Stomp协议插件 | -+-------------------------+-----------------------------------+ -| emqttd_sockjs | SockJS插件 | -+-------------------------+-----------------------------------+ -| emqttd_recon | Recon优化调测插件 | -+-------------------------+-----------------------------------+ +A plugin is an Erlang application to extend the emqttd broker. -扩展插件通过'bin/emqttd_ctl'管理命令行，加载启动运行。 ++----------------------------+-----------------------------------+ +| `emqttd_plugin_template`_ | Plugin template and demo | ++----------------------------+-----------------------------------+ +| `emqttd_dashboard`_ | Web Dashboard | ++----------------------------+-----------------------------------+ +| `emqttd_plugin_mysql`_ | Authentication with MySQL | ++----------------------------+-----------------------------------+ +| `emqttd_plugin_pgsql`_ | Authentication with PostgreSQL | ++----------------------------+-----------------------------------+ +| `emqttd_plugin_redis`_ | Authentication with Redis | ++----------------------------+-----------------------------------+ +| `emqttd_plugin_mongo`_ | Authentication with MongoDB | ++----------------------------+-----------------------------------+ +| `emqttd_stomp`_ | STOMP Protocol Plugin | ++----------------------------+-----------------------------------+ +| `emqttd_sockjs`_ | SockJS(Stomp) Plugin | ++----------------------------+-----------------------------------+ +| `emqttd_recon`_ | Recon Plugin | ++----------------------------+-----------------------------------+ -例如启用PostgreSQL认证插件:: +A plugin could be enabled by 'bin/emqttd_ctl plugins load' command. + +For example, enable 'emqttd_plugin_pgsql' plugin:: ./bin/emqttd_ctl plugins load emqttd_plugin_pgsql ----------------------------------- +----------------------- One million Connections ----------------------------------- +----------------------- + +Latest release of the emqttd broker is scaling to 1.3 million MQTT connections on a 12 Core, 32G CentOS server. .. NOTE:: - emqttd消息服务器默认设置，允许最大客户端连接是512，因为大部分操作系统'ulimit -n'限制为1024。 + The emqttd broker only allows 512 concurrent connections by default, for 'ulimit -n' limit is 1024 on most platform. -emqttd消息服务器当前版本，连接压力测试到130万线，8核心/32G内存的CentOS云服务器。 - -操作系统内核参数、TCP协议栈参数、Erlang虚拟机参数、emqttd最大允许连接数设置简述如下： +We need tune the OS Kernel, TCP Stack, Erlang VM and emqttd broker for one million connections benchmark. Linux Kernel Parameters ----------------------- -# 2M - 系统所有进程可打开的文件数量:: - .. code:: + # 2M: sysctl -w fs.file-max=2097152 sysctl -w fs.nr_open=2097152 -# 1M - 系统允许当前进程打开的文件数量:: - + # 1M: ulimit -n 1048576 TCP Stack Parameters ------------------------ - -# backlog - Socket监听队列长度:: +-------------------- .. code:: + # backlog sysctl -w net.core.somaxconn=65536 Erlang VM ------------------ +--------- emqttd/etc/vm.args:: @@ -210,8 +228,8 @@ emqttd/etc/vm.args:: -env ERTS_MAX_PORTS 1048576 -emqttd.config ------------------ +emqttd broker +------------- emqttd/etc/emqttd.config:: @@ -239,6 +257,7 @@ Test Client sysctl -w net.ipv4.ip_local_port_range="500 65535" echo 1000000 > /proc/sys/fs/nr_open + ulimit -n 100000 --------------------- MQTT Client Libraries @@ -261,3 +280,13 @@ GitHub: https://github.com/emqtt .. _CocoaMQTT: https://github.com/emqtt/CocoaMQTT .. _QMQTT: https://github.com/emqtt/qmqtt +.. _emqttd_plugin_template: https://github.com/emqtt/emqttd_plugin_template +.. _emqttd_dashboard: https://github.com/emqtt/emqttd_dashboard +.. _emqttd_plugin_mysql: https://github.com/emqtt/emqttd_plugin_mysql +.. _emqttd_plugin_pgsql: https://github.com/emqtt/emqttd_plugin_pgsql +.. _emqttd_plugin_redis: https://github.com/emqtt/emqttd_plugin_redis +.. _emqttd_plugin_mongo: https://github.com/emqtt/emqttd_plugin_mongo +.. _emqttd_stomp: https://github.com/emqtt/emqttd_stomp +.. _emqttd_sockjs: https://github.com/emqtt/emqttd_sockjs +.. _emqttd_recon: https://github.com/emqtt/emqttd_recon + diff --git a/docs/source/index.rst b/docs/source/index.rst index 123c96275..941265591 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -47,7 +47,6 @@ Contents: config cluster bridge - guide commands plugins