From f63b826521c209ffb9a82cc64b6e54341e9094a9 Mon Sep 17 00:00:00 2001 From: Feng Lee Date: Fri, 26 Feb 2016 19:35:02 +0800 Subject: [PATCH 1/3] getstarted --- docs/source/getstarted.rst | 153 ++++++++++++++++++++++--------------- 1 file changed, 90 insertions(+), 63 deletions(-) diff --git a/docs/source/getstarted.rst b/docs/source/getstarted.rst index 1145d614f..e04890246 100644 --- a/docs/source/getstarted.rst +++ b/docs/source/getstarted.rst @@ -13,7 +13,7 @@ emqttd(Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. The emqttd project is aimed to implement a scalable, distributed, extensible open-source MQTT broker for IoT, M2M and Mobile applications that hope to handle ten millions of concurrent MQTT clients. -The emqttd broker is: +Highlights of the emqttd broker: * Full MQTT V3.1/3.1.1 Protocol Specifications Support * Easy to Install - Quick Install on Linux, FreeBSD, Mac and Windows @@ -21,7 +21,6 @@ The emqttd broker is: * Easy to extend - Hooks and plugins to customize or extend the broker * Pluggable Authentication - LDAP, MySQL, PostgreSQL, Redis Authentication Plugins - -------- Features -------- @@ -47,7 +46,6 @@ Features * Extensible architecture with Hooks, Modules and Plugins * Passed eclipse paho interoperability tests - ----------- Quick Start ----------- @@ -55,7 +53,9 @@ Quick Start Download and Install -------------------- -Download binary package for Linux, Mac, FreeBSD and Windows platform from http://emqtt.io/downloads. +The emqttd broker is cross-platform, could be deployed on Linux, Mac, FreeBSD, Windows and Raspberry Pi. + +Download binary package from: http://emqtt.io/downloads. Installing on Mac, For example: .. code:: console @@ -71,7 +71,7 @@ Download binary package for Linux, Mac, FreeBSD and Windows platform from http:/ ./bin/emqttd stop Installing from Source ------------------------ +---------------------- .. NOTE:: emqttd requires Erlang R17+ to build. @@ -81,39 +81,56 @@ Installing from Source cd emqttd && make && make dist --------------------- +------------- Web Dashboard --------------------- +------------- + +A Web Dashboard will be loaded automatically when the emqttd broker is started successfully. + +The Dashboard helps to check running status of the broker, monitor statistics and metrics of MQTT packets, + +query clients, sessions, topics and subscriptions. + ++------------------+---------------------------+ +| Default Address | http://localhost:18083 | ++------------------+---------------------------+ +| Default User | admin | ++------------------+---------------------------+ +| Default Password | public | ++------------------+---------------------------+ .. image:: ./_static/images/dashboard.png - ------------------- Modules and Plugins ------------------- -The emqttd broker could be extended by modules and plugins. +The emqttd broker could be extended by Modules and Plugins. + +The authentication and ACL mechanism is usally implemented by a module or plugin. Modules ------- -+-------------------------+-----------------------------------+ -| emqttd_auth_clientid | ClientId认证 | -+-------------------------+-----------------------------------+ -| emqttd_auth_username | 用户名密码认证 | -+-------------------------+-----------------------------------+ -| emqttd_auth_ldap | LDAP认证 | -+-------------------------+-----------------------------------+ -| emqttd_mod_presence | 客户端上下线状态消息发布 | -+-------------------------+-----------------------------------+ -| emqttd_mod_subscription | 客户端上线自动主题订阅 | -+-------------------------+-----------------------------------+ -| emqttd_mod_rewrite | 重写客户端订阅主题(Topic) | -+-------------------------+-----------------------------------+ ++-------------------------+--------------------------------------------+ +| emqttd_auth_clientid | Authentication with ClientId | ++-------------------------+--------------------------------------------+ +| emqttd_auth_username | Authentication with Username and Password | ++-------------------------+--------------------------------------------+ +| emqttd_auth_ldap | Authentication with LDAP | ++-------------------------+--------------------------------------------+ +| emqttd_mod_presence | Publish presence message to $SYS topics | +| | when client connected or disconnected | ++-------------------------+--------------------------------------------+ +| emqttd_mod_subscription | Subscribe topics automatically when client | +| | connected | ++-------------------------+--------------------------------------------+ +| emqttd_mod_rewrite | Topics rewrite like HTTP rewrite module | ++-------------------------+--------------------------------------------+ -扩展模块通过'etc/emqttd.config'配置文件的auth, modules段落启用。 +Configure the 'auth', 'module' paragraph in 'etc/emqttd.config' to enable a module. -例如启用用户名密码认证:: +Enable 'emqttd_auth_username' module:: {access, [ %% Authetication. Anonymous Default @@ -123,7 +140,7 @@ Modules ... -启用客户端状态发布模块:: +Enable 'emqttd_mod_presence' module:: {modules, [ %% Client presence management module. @@ -131,71 +148,70 @@ Modules {presence, [{qos, 0}]} Plugins --------- +------- -+-------------------------+-----------------------------------+ -| emqttd_plugin_template | 插件模版与演示代码 | -+-------------------------+-----------------------------------+ -| emqttd_dashboard | Web管理控制台，默认加载 | -+-------------------------+-----------------------------------+ -| emqttd_plugin_mysql | MySQL认证插件 | -+-------------------------+-----------------------------------+ -| emqttd_plugin_pgsql | PostgreSQL认证插件 | -+-------------------------+-----------------------------------+ -| emqttd_plugin_redis | Redis认证插件 | -+-------------------------+-----------------------------------+ -| emqttd_plugin_mongo | MongoDB认证插件 | -+-------------------------+-----------------------------------+ -| emqttd_stomp | Stomp协议插件 | -+-------------------------+-----------------------------------+ -| emqttd_sockjs | SockJS插件 | -+-------------------------+-----------------------------------+ -| emqttd_recon | Recon优化调测插件 | -+-------------------------+-----------------------------------+ +A plugin is an Erlang application to extend the emqttd broker. -扩展插件通过'bin/emqttd_ctl'管理命令行，加载启动运行。 ++----------------------------+-----------------------------------+ +| `emqttd_plugin_template`_ | Plugin template and demo | ++----------------------------+-----------------------------------+ +| `emqttd_dashboard`_ | Web Dashboard | ++----------------------------+-----------------------------------+ +| `emqttd_plugin_mysql`_ | Authentication with MySQL | ++----------------------------+-----------------------------------+ +| `emqttd_plugin_pgsql`_ | Authentication with PostgreSQL | ++----------------------------+-----------------------------------+ +| `emqttd_plugin_redis`_ | Authentication with Redis | ++----------------------------+-----------------------------------+ +| `emqttd_plugin_mongo`_ | Authentication with MongoDB | ++----------------------------+-----------------------------------+ +| `emqttd_stomp`_ | STOMP Protocol Plugin | ++----------------------------+-----------------------------------+ +| `emqttd_sockjs`_ | SockJS(Stomp) Plugin | ++----------------------------+-----------------------------------+ +| `emqttd_recon`_ | Recon Plugin | ++----------------------------+-----------------------------------+ -例如启用PostgreSQL认证插件:: +A plugin could be enabled by 'bin/emqttd_ctl plugins load' command. + +For example, enable 'emqttd_plugin_pgsql' plugin:: ./bin/emqttd_ctl plugins load emqttd_plugin_pgsql ----------------------------------- +----------------------- One million Connections ----------------------------------- +----------------------- + +Latest release of the emqttd broker is scaling to 1.3 million MQTT connections on a 12 Core, 32G CentOS server. .. NOTE:: - emqttd消息服务器默认设置，允许最大客户端连接是512，因为大部分操作系统'ulimit -n'限制为1024。 + The emqttd broker only allows 512 concurrent connections by default, for 'ulimit -n' limit is 1024 on most platform. -emqttd消息服务器当前版本，连接压力测试到130万线，8核心/32G内存的CentOS云服务器。 - -操作系统内核参数、TCP协议栈参数、Erlang虚拟机参数、emqttd最大允许连接数设置简述如下： +We need tune the OS Kernel, TCP Stack, Erlang VM and emqttd broker for one million connections benchmark. Linux Kernel Parameters ----------------------- -# 2M - 系统所有进程可打开的文件数量:: - .. code:: + # 2M: sysctl -w fs.file-max=2097152 sysctl -w fs.nr_open=2097152 -# 1M - 系统允许当前进程打开的文件数量:: - + # 1M: ulimit -n 1048576 TCP Stack Parameters ------------------------ - -# backlog - Socket监听队列长度:: +-------------------- .. code:: + # backlog sysctl -w net.core.somaxconn=65536 Erlang VM ------------------ +--------- emqttd/etc/vm.args:: @@ -210,8 +226,8 @@ emqttd/etc/vm.args:: -env ERTS_MAX_PORTS 1048576 -emqttd.config ------------------ +emqttd broker +------------- emqttd/etc/emqttd.config:: @@ -239,6 +255,7 @@ Test Client sysctl -w net.ipv4.ip_local_port_range="500 65535" echo 1000000 > /proc/sys/fs/nr_open + ulimit -n 100000 ---------------------- emqtt Client Libraries @@ -261,3 +278,13 @@ GitHub: https://github.com/emqtt .. _CocoaMQTT: https://github.com/emqtt/CocoaMQTT .. _QMQTT: https://github.com/emqtt/qmqtt +.. _emqttd_plugin_template: https://github.com/emqtt/emqttd_plugin_template +.. _emqttd_dashboard: https://github.com/emqtt/emqttd_dashboard +.. _emqttd_plugin_mysql: https://github.com/emqtt/emqttd_plugin_mysql +.. _emqttd_plugin_pgsql: https://github.com/emqtt/emqttd_plugin_pgsql +.. _emqttd_plugin_redis: https://github.com/emqtt/emqttd_plugin_redis +.. _emqttd_plugin_mongo: https://github.com/emqtt/emqttd_plugin_mongo +.. _emqttd_stomp: https://github.com/emqtt/emqttd_stomp +.. _emqttd_sockjs: https://github.com/emqtt/emqttd_sockjs +.. _emqttd_recon: https://github.com/emqtt/emqttd_recon + From 1158ccbbcdf6f60a71d7b0f288d2279f21713625 Mon Sep 17 00:00:00 2001 From: Feng Lee Date: Fri, 26 Feb 2016 20:06:49 +0800 Subject: [PATCH 2/3] misc fix --- docs/source/getstarted.rst | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-) diff --git a/docs/source/getstarted.rst b/docs/source/getstarted.rst index e04890246..e3972600a 100644 --- a/docs/source/getstarted.rst +++ b/docs/source/getstarted.rst @@ -9,7 +9,7 @@ Get Started Overview -------- -emqttd(Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. Erlang/OTP is a concurrent, fault-tolerant, soft-realtime and distributed programming platform. MQTT is anextremely lightweight publish/subscribe messaging protocol powering IoT, M2M applications. +emqttd(Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. Erlang/OTP is a concurrent, fault-tolerant, soft-realtime and distributed programming platform. MQTT is an extremely lightweight publish/subscribe messaging protocol powering IoT, M2M and Mobile applications. The emqttd project is aimed to implement a scalable, distributed, extensible open-source MQTT broker for IoT, M2M and Mobile applications that hope to handle ten millions of concurrent MQTT clients. @@ -31,16 +31,17 @@ Features * Retained Message * Last Will Message * TCP/SSL Connection -* MQTT Over Websocket(SSL) +* MQTT Over WebSocket(SSL) * HTTP Publish API * STOMP protocol * STOMP over SockJS * $SYS/# Topics -* Client Authentication with clientId, ipaddress -* Client Authentication with username, password -* Client ACL control with ipaddress, clientid, username -* LDAP, Redis, MySQL, PostgreSQL authentication -* Cluster brokers on several servers. +* ClientID Authentication +* IpAddress Authentication +* Username and Password Authentication +* Access control based on IpAddress, ClientID, Username +* Authentication with LDAP, Redis, MySQL, PostgreSQL +* Cluster brokers on several servers * Bridge brokers locally or remotely * mosquitto, RSMB bridge * Extensible architecture with Hooks, Modules and Plugins @@ -55,7 +56,9 @@ Download and Install The emqttd broker is cross-platform, could be deployed on Linux, Mac, FreeBSD, Windows and Raspberry Pi. -Download binary package from: http://emqtt.io/downloads. Installing on Mac, For example: +Download binary package from: http://emqtt.io/downloads. + +Installing on Mac, For example: .. code:: console @@ -81,6 +84,8 @@ Installing from Source cd emqttd && make && make dist + cd rel/emqttd && ./bin/emqttd console + ------------- Web Dashboard ------------- @@ -107,7 +112,7 @@ Modules and Plugins The emqttd broker could be extended by Modules and Plugins. -The authentication and ACL mechanism is usally implemented by a module or plugin. +The authentication and ACL mechanism is usually implemented by a module or plugin. Modules ------- From 385ab3ec77eb510b9babec83c8e15d05a76cf288 Mon Sep 17 00:00:00 2001 From: Feng Lee Date: Fri, 26 Feb 2016 21:52:18 +0800 Subject: [PATCH 3/3] config --- docs/source/config.rst | 92 +++++++++++++++++--------------------- docs/source/getstarted.rst | 11 ++--- docs/source/index.rst | 1 - 3 files changed, 46 insertions(+), 58 deletions(-) diff --git a/docs/source/config.rst b/docs/source/config.rst index 4acf4e27b..72215f476 100644 --- a/docs/source/config.rst +++ b/docs/source/config.rst @@ -5,31 +5,29 @@ Configuration ============= -emqttd消息服务器通过etc/目录下配置文件进行设置，主要配置文件包括: +Configuration files are under 'etc/' folder, including: +-------------------+-----------------------------------+ -| 配置文件 | 说明 | +| File | Description | +-------------------+-----------------------------------+ -| etc/vm.args | Erlang 虚拟机的参数设置 | +| etc/vm.args | Erlang VM Arguments | +-------------------+-----------------------------------+ -| etc/emqttd.config | emqttd消息服务器参数设置 | +| etc/emqttd.config | emqttd broker Config | +-------------------+-----------------------------------+ -| etc/acl.config | ACL(访问控制规则)设置 | +| etc/acl.config | ACL Config | +-------------------+-----------------------------------+ -| etc/clients.config| 基于ClientId认证设置 | +| etc/clients.config| ClientId Authentication | +-------------------+-----------------------------------+ -| etc/rewrite.config| Rewrite扩展模块规则配置 | +| etc/rewrite.config| Rewrite Rules | +-------------------+-----------------------------------+ -| etc/ssl/* | SSL证书设置 | +| etc/ssl/* | SSL certificate and key files | +-------------------+-----------------------------------+ ----------- etc/vm.args ----------- -Configure parameters of Erlang VM: - -.. code:: +Configure and Optimize Erlang VM:: ##------------------------------------------------------------------------- ## Name of the node @@ -84,35 +82,33 @@ Configure parameters of Erlang VM: ## Tweak GC to run more often -env ERL_FULLSWEEP_AFTER 1000 -etc/vm.args中两个最重要的参数: +The two most important parameters in etc/vm.args: -+-------+----------------------------------------------------------------------------------------------+ -| +P | Erlang虚拟机允许的最大进程数，一个MQTT连接会消耗2个Erlang进程，所以参数值 > 最大连接数 * 2 | -+-------+----------------------------------------------------------------------------------------------+ -| +Q | Erlang虚拟机允许的最大Port数量，一个MQTT连接消耗1个Port，所以参数值 > 最大连接数 | -+-------+----------------------------------------------------------------------------------------------+ ++-------+---------------------------------------------------------------------------+ +| +P | Max number of Erlang proccesses. A MQTT client consumes two proccesses. | +| | The value should be larger than max_clients * 2 | ++-------+---------------------------------------------------------------------------+ +| +Q | Max number of Erlang Ports. A MQTT client consumes one port. | +| | The value should be larger than max_clients. | ++-------+---------------------------------------------------------------------------+ -etc/vm.args设置Erlang节点名、节点间通信Cookie:: +The name and cookie of Erlang Node should be configured when clustering:: - -name emqttd@127.0.0.1 + -name emqttd@host_or_ip ## Cookie for distributed erlang -setcookie emqttdsecretcookie -.. NOTE:: - - Erlang/OTP平台应用多由分布的Erlang节点(进程)组成，每个Erlang节点(进程)需指配一个节点名，用于节点间通信互访。 - 所有互相通信的Erlang节点(进程)间通过一个共用的Cookie进行安全认证。 - ----------------- etc/emqttd.config ----------------- -etc/emqttd.config是消息服务器的核心配置文件。Erlang程序由多个应用(application)组成，每个应用(application)有自身的环境参数， +The main configuration file for emqttd broker. -启动时候通过etc/emqttd.config文件加载。 +File Syntax +----------- -etc/emqttd.config文件采用的是Erlang数据格式，kernel, sasl, emqttd是Erlang应用(application)名称，'[]'内是应用的环境参数列表。 +The config consists of a list of Erlang Applications and their environments. .. code:: erlang @@ -131,24 +127,22 @@ etc/emqttd.config文件采用的是Erlang数据格式，kernel, sasl, emqttd是E ]} ]. -emqttd.config格式简要说明: +The file adopts Erlang Term Syntax: -1. [ ] : 列表，逗号分隔元素 +1. [ ]: List, seperated by comma +2. { }: Tuple, Usually {Env, Value} +3. % : comment -2. { } : 元组，配置元组一般两个元素{Env, Value} +Log Level and File +------------------ -3. % : 注释 - -Log Level and Destination -------------------------- - -emqttd消息服务器日志由lager应用(application)提供，日志相关设置在lager应用段落:: +Logger of emqttd broker is implemented by 'lager' application:: {lager, [ ... ]}, -产品环境下默认只开启error日志，日志输出到logs/emqttd_error.log文件。'handlers'段落启用其他级别日志:: +Configure log handlers:: {handlers, [ {lager_console_backend, info}, @@ -172,47 +166,45 @@ emqttd消息服务器日志由lager应用(application)提供，日志相关设 ]} ]} -.. WARNING:: 过多日志打印严重影响服务器性能，产品环境下建议开启error级别日志。 - -Broker Parameters +emqttd Application ------------------ -emqttd消息服务器参数设置在emqttd应用段落，包括用户认证与访问控制设置，MQTT协议、会话、队列设置，扩展模块设置，TCP服务监听器设置:: +The MQTT broker is implemented by erlang 'emqttd' application:: {emqttd, [ - %% 用户认证与访问控制设置 + %% Authentication and Authorization {access, [ ... ]}, - %% MQTT连接、协议、会话、队列设置 + %% MQTT Protocol Options {mqtt, [ ... ]}, - %% 消息服务器设置 + %% Broker Options {broker, [ ... ]}, - %% 扩展模块设置 + %% Modules {modules, [ ... ]}, - %% 插件目录设置 + %% Plugins {plugins, [ ... ]}, - %% TCP监听器设置 + %% Listeners {listeners, [ ... ]}, - %% Erlang虚拟机监控设置 + %% Erlang System Monitor {sysmon, [ ]} ]} -access用户认证设置 ------------------- +Authentication +-------------- emqttd消息服务器认证由一系列认证模块(module)或插件(plugin)提供，系统默认支持用户名、ClientID、LDAP、匿名(anonymouse)认证模块:: diff --git a/docs/source/getstarted.rst b/docs/source/getstarted.rst index e3972600a..d24e9bb3f 100644 --- a/docs/source/getstarted.rst +++ b/docs/source/getstarted.rst @@ -18,6 +18,7 @@ Highlights of the emqttd broker: * Full MQTT V3.1/3.1.1 Protocol Specifications Support * Easy to Install - Quick Install on Linux, FreeBSD, Mac and Windows * Massively scalable - Scaling to 1 million connections on a single server +* Cluster and Bridge Support * Easy to extend - Hooks and plugins to customize or extend the broker * Pluggable Authentication - LDAP, MySQL, PostgreSQL, Redis Authentication Plugins @@ -76,7 +77,7 @@ Installing on Mac, For example: Installing from Source ---------------------- -.. NOTE:: emqttd requires Erlang R17+ to build. +.. NOTE:: emqttd broker requires Erlang R17+ to build. .. code:: console @@ -92,9 +93,7 @@ Web Dashboard A Web Dashboard will be loaded automatically when the emqttd broker is started successfully. -The Dashboard helps to check running status of the broker, monitor statistics and metrics of MQTT packets, - -query clients, sessions, topics and subscriptions. +The Dashboard helps to check running status of the broker, monitor statistics and metrics of MQTT packets, query clients, sessions, topics and subscriptions. +------------------+---------------------------+ | Default Address | http://localhost:18083 | @@ -110,9 +109,7 @@ query clients, sessions, topics and subscriptions. Modules and Plugins ------------------- -The emqttd broker could be extended by Modules and Plugins. - -The authentication and ACL mechanism is usually implemented by a module or plugin. +The Authentication and ACL mechanism is usually implemented by a Module or Plugin. Modules ------- diff --git a/docs/source/index.rst b/docs/source/index.rst index 123c96275..941265591 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -47,7 +47,6 @@ Contents: config cluster bridge - guide commands plugins