From 79176c5224f514d3eb87639150c8c47e53e89bf9 Mon Sep 17 00:00:00 2001 From: Feng Date: Wed, 19 Oct 2016 20:50:01 +0800 Subject: [PATCH] 2.0-rc.2 - update docs --- docs/source/config.rst | 16 +- docs/source/design.rst | 36 +- docs/source/getstarted.rst | 192 ++++----- docs/source/guide.rst | 565 ++++++++++++++----------- docs/source/install.rst | 94 ++--- docs/source/plugins.rst | 824 ++++++++++++++++++++++--------------- docs/source/tune.rst | 55 +-- 7 files changed, 1005 insertions(+), 777 deletions(-) diff --git a/docs/source/config.rst b/docs/source/config.rst index c69491073..d623ef0bc 100644 --- a/docs/source/config.rst +++ b/docs/source/config.rst @@ -21,7 +21,7 @@ The main configuration files of the EMQ broker are under 'etc/' folder: EMQ 2.0 Config Syntax --------------------- -The EMQ 2.0-rc.2 release integrated with `cuttlefish` library, and adopt a more user-friendly '`k = v` syntax for configuration file: +The *EMQ* 2.0-rc.2 release integrated with `cuttlefish` library, and adopt a more user-friendly `k = v` syntax for configuration file: .. code-block:: properties @@ -64,7 +64,7 @@ OS Environment Variables EMQ Node and Cookie ------------------- -The node name and cookie of *EMQ* should be configured when clustering:: +The node name and cookie of *EMQ* should be configured when clustering: .. code-block:: properties @@ -117,7 +117,7 @@ Configure and Optimize Erlang VM: ## node.dist_listen_min = 6000 ## node.dist_listen_max = 6999 -The two most important parameters in releases/2.0/vm.args: +The two most important parameters for Erlang VM: +--------------------------+---------------------------------------------------------------------------+ | node.process_limit | Max number of Erlang proccesses. A MQTT client consumes two proccesses. | @@ -229,7 +229,7 @@ Define ACL rules in etc/acl.conf. The rules by default: %% Allow all by default {allow, all}. -An ACL rule is an Erlang tuple. The Access control module of emqttd broker matches the rule one by one from top to bottom:: +An ACL rule is an Erlang tuple. The Access control module of *EMQ* broker matches the rule one by one from top to bottom:: --------- --------- --------- Client -> | Rule1 | --nomatch--> | Rule2 | --nomatch--> | Rule3 | --> Default @@ -433,7 +433,7 @@ MQTT Listeners Configure the TCP listeners for MQTT, MQTT(SSL), HTTP and HTTPS Protocols. -The most important parameter is 'max_clients' - max concurrent clients allowed. +The most important parameter for MQTT listener is `max_clients`: max concurrent clients allowed. The TCP Ports occupied by the *EMQ* broker by default: @@ -552,9 +552,9 @@ System Monitor ## Busy Dist Port sysmon.busy_dist_port = true ----------------------------- -Plugins' Configuration Files ----------------------------- +-------------------------- +Plugin Configuration Files +-------------------------- +----------------------------------------+-----------------------------------+ | File | Description | diff --git a/docs/source/design.rst b/docs/source/design.rst index 8e5ea8852..7982a8b7b 100644 --- a/docs/source/design.rst +++ b/docs/source/design.rst @@ -11,11 +11,11 @@ Design Architecture ------------ -The emqttd broker 1.0 is more like a network Switch or Router, not a traditional enterprise message queue. Compared to a network router that routes packets based on IP or MPLS label, the emqttd broker routes MQTT messages based on topic trie. +The *EMQ* broker 1.0 is more like a network Switch or Router, not a traditional enterprise message queue. Compared to a network router that routes packets based on IP or MPLS label, the *EMQ* broker routes MQTT messages based on topic trie. .. image:: _static/images/concept.png -The EMQ 2.0 seperated the Message Flow Plane and Monitor/Control Plane, the Architecture is something like:: +The *EMQ* 2.0 seperated the Message Flow Plane and Monitor/Control Plane, the Architecture is something like:: Control Plane -------------------- @@ -200,7 +200,7 @@ The routing design follows two rules: Authentication and ACL ---------------------- -The emqttd broker supports an extensible authentication/ACL mechanism, which is implemented by emqttd_access_control, emqttd_auth_mod and emqttd_acl_mod modules. +The *EMQ* broker supports an extensible authentication/ACL mechanism, which is implemented by emqttd_access_control, emqttd_auth_mod and emqttd_acl_mod modules. emqttd_access_control module provides two APIs that help register/unregister auth or ACL module: @@ -241,18 +241,26 @@ The emqttd_auth_mod defines an Erlang behaviour for authentication module: -endif. -The authentication modules implemented by default: +The authentication modules implemented by plugins: +-----------------------+--------------------------------+ -| Module | Authentication | +| Plugin | Authentication | +-----------------------+--------------------------------+ -| emqttd_auth_username | Username and Password | +| emq_auth_username | Username and Password | +-----------------------+--------------------------------+ -| emqttd_auth_clientid | ClientID | +| emq_auth_clientid | ClientID and Password | +-----------------------+--------------------------------+ -| emqttd_auth_ldap | LDAP | +| emq_auth_ldap | LDAP | +-----------------------+--------------------------------+ -| emqttd_auth_anonymous | Anonymous | +| emq_auth_http | HTTP API | ++-----------------------+--------------------------------+ +| emq_auth_mysql | MySQL | ++-----------------------+--------------------------------+ +| emq_auth_pgsql | PostgreSQL | ++-----------------------+--------------------------------+ +| emq_auth_redis | Redis | ++-----------------------+--------------------------------+ +| emq_auth_mongo | MongoDB | +-----------------------+--------------------------------+ Authorization(ACL) @@ -290,7 +298,7 @@ The emqttd_acl_mod defines an Erlang behavihour for ACL module: -endif. -emqttd_acl_internal implements the default ACL based on etc/acl.config file: +emqttd_acl_internal implements the default ACL based on etc/acl.conf file: .. code-block:: erlang @@ -326,9 +334,9 @@ emqttd_acl_internal implements the default ACL based on etc/acl.config file: Hooks Design ------------ -The emqttd broker implements a simple but powerful hooks mechanism to help users develop plugin. The broker would run the hooks when a client is connected/disconnected, a topic is subscribed/unsubscribed or a MQTT message is published/delivered/acked. +The *EMQ* broker implements a simple but powerful hooks mechanism to help users develop plugin. The broker would run the hooks when a client is connected/disconnected, a topic is subscribed/unsubscribed or a MQTT message is published/delivered/acked. -Hooks defined by the emqttd 1.0 broker: +Hooks defined by the *EMQ* 2.0 broker: +------------------------+------------------------------------------------------+ | Hook | Description | @@ -352,7 +360,7 @@ Hooks defined by the emqttd 1.0 broker: | client.disconnected | Run when client disconnected from broker | +------------------------+------------------------------------------------------+ -The emqttd broker uses the `Chain-of-responsibility_pattern`_ to implement hook mechanism. The callback functions registered to hook will be executed one by one:: +The *EMQ* broker uses the `Chain-of-responsibility_pattern`_ to implement hook mechanism. The callback functions registered to hook will be executed one by one:: -------- ok | {ok, NewAcc} -------- ok | {ok, NewAcc} -------- (Args, Acc) --> | Fun1 | -------------------> | Fun2 | -------------------> | Fun3 | --> {ok, Acc} | {stop, Acc} @@ -455,7 +463,7 @@ The `emqttd_plugin_template`_ project provides the examples for hook usage: Plugin Design ------------- -Plugin is a normal erlang application that can be started/stopped dynamically by a running emqttd broker. +Plugin is a normal erlang application that can be started/stopped dynamically by a running *EMQ* broker. emqttd_plugins Module --------------------- diff --git a/docs/source/getstarted.rst b/docs/source/getstarted.rst index 1cb36ecae..4cb5bcef7 100644 --- a/docs/source/getstarted.rst +++ b/docs/source/getstarted.rst @@ -9,11 +9,11 @@ Get Started Overview -------- -emqttd(Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. Erlang/OTP is a concurrent, fault-tolerant, soft-realtime and distributed programming platform. MQTT is an extremely lightweight publish/subscribe messaging protocol powering IoT, M2M and Mobile applications. +*EMQ* (Erlang MQTT Broker) is an open source MQTT broker written in Erlang/OTP. Erlang/OTP is a concurrent, fault-tolerant, soft-realtime and distributed programming platform. MQTT is an extremely lightweight publish/subscribe messaging protocol powering IoT, M2M and Mobile applications. -The emqttd project is aimed to implement a scalable, distributed, extensible open-source MQTT broker for IoT, M2M and Mobile applications that hope to handle millions of concurrent MQTT clients. +The *EMQ* project is aimed to implement a scalable, distributed, extensible open-source MQTT broker for IoT, M2M and Mobile applications that hope to handle millions of concurrent MQTT clients. -Highlights of the emqttd broker: +Highlights of the *EMQ* broker: * Full MQTT V3.1/3.1.1 Protocol Specifications Support * Easy to Install - Quick Install on Linux, FreeBSD, Mac and Windows @@ -49,6 +49,8 @@ Features * mosquitto, RSMB bridge * Extensible architecture with Hooks, Modules and Plugins * Passed eclipse paho interoperability tests +* Local subscription +* Shared subscription ----------- Quick Start @@ -57,7 +59,7 @@ Quick Start Download and Install -------------------- -The emqttd broker is cross-platform, which could be deployed on Linux, FreeBSD, Mac, Windows and even Raspberry Pi. +The *EMQ* broker is cross-platform, which could be deployed on Linux, FreeBSD, Mac, Windows and even Raspberry Pi. Download binary package from: http://emqtt.io/downloads. @@ -65,7 +67,7 @@ Installing on Mac, for example: .. code-block:: bash - unzip emqttd-macosx-v2.0-rc.2-20160910.zip && cd emqttd + unzip emqttd-macosx-v2.0-rc.2-20161019.zip && cd emqttd # Start emqttd ./bin/emqttd start @@ -93,7 +95,7 @@ Installing from Source Web Dashboard ------------- -A Web Dashboard will be loaded when the emqttd broker is started successfully. +A Web Dashboard will be loaded when the *EMQ* broker is started successfully. The Dashboard helps check running status of the broker, monitor statistics and metrics of MQTT packets, query clients, sessions, topics and subscriptions. @@ -111,15 +113,13 @@ The Dashboard helps check running status of the broker, monitor statistics and m Modules and Plugins ------------------- -The Authentication and Authorization(ACL) are usually implemented by a Module or Plugin. +The *EMQ* broker could be extended by Modules ofr Plugins. Modules ------- +-------------------------+--------------------------------------------+ -| emqttd_auth_clientid | Authentication with ClientId | -+-------------------------+--------------------------------------------+ -| emqttd_auth_username | Authentication with Username and Password | +| emqttd_mod_retainer | Retained Messages Storage | +-------------------------+--------------------------------------------+ | emqttd_mod_presence | Publish presence message to $SYS topics | | | when client connected or disconnected | @@ -127,68 +127,77 @@ Modules | emqttd_mod_subscription | Subscribe topics automatically when client | | | connected | +-------------------------+--------------------------------------------+ -| emqttd_mod_rewrite | Topics rewrite like HTTP rewrite module | -+-------------------------+--------------------------------------------+ -Configure the 'auth', 'module' paragraph in 'etc/emqttd.config' to enable a module. +Modules could enabled by configuring 'etc/emq.conf': -Enable 'emqttd_auth_username' module: +.. code-block:: properties -.. code-block:: erlang + ##------------------------------------------------------------------- + ## MQTT Modules + ##------------------------------------------------------------------- - %% Authentication with username, password - {auth, username, [{passwd, "etc/modules/passwd.conf"}]}. + ## Enable presence module + mqtt.module.presence = on -Enable 'emqttd_mod_presence' module: + mqtt.module.presence.qos = 0 -.. code-block:: erlang + ## Enable subscription module + mqtt.module.subscription = on - %% Client presence management module. Publish presence messages when - %% client connected or disconnected. - {module, presence, [{qos, 0}]}. + mqtt.module.subscription.topics = $client/%c=1,$user/%u=1 Plugins ------- -A plugin is an Erlang application to extend the emqttd broker. +A plugin is an Erlang application to extend the *EMQ* broker. -+----------------------------+-----------------------------------+ -| `emqttd_plugin_template`_ | Plugin template and demo | -+----------------------------+-----------------------------------+ -| `emqttd_dashboard`_ | Web Dashboard | -+----------------------------+-----------------------------------+ -| `emqttd_auth_ldap`_ | LDAP Auth Plugin | -+----------------------------+-----------------------------------+ -| `emqttd_auth_http`_ | Authentication/ACL with HTTP API | -+----------------------------+-----------------------------------+ -| `emqttd_auth_mysql` _ | Authentication with MySQL | -+----------------------------+-----------------------------------+ -| `emqttd_auth_pgsql`_ | Authentication with PostgreSQL | -+----------------------------+-----------------------------------+ -| `emqttd_auth_redis`_ | Authentication with Redis | -+----------------------------+-----------------------------------+ -| `emqttd_plugin_mongo`_ | Authentication with MongoDB | -+----------------------------+-----------------------------------+ -| `emqttd_sn`_ | MQTT-SN Protocol Plugin | -+----------------------------+-----------------------------------+ -| `emqttd_stomp`_ | STOMP Protocol Plugin | -+----------------------------+-----------------------------------+ -| `emqttd_sockjs`_ | SockJS(Stomp) Plugin | -+----------------------------+-----------------------------------+ -| `emqttd_recon`_ | Recon Plugin | -+----------------------------+-----------------------------------+ ++----------------------------+--------------------------------------------+ +| `emq_auth_clientid`_ | Authentication with ClientId | ++----------------------------+--------------------------------------------+ +| `emq_auth_username`_ | Authentication with Username and Password | ++----------------------------+--------------------------------------------+ +| `emq_plugin_template`_ | Plugin template and demo | ++----------------------------+--------------------------------------------+ +| `emq_dashboard`_ | Web Dashboard | ++----------------------------+--------------------------------------------+ +| `emq_auth_ldap`_ | LDAP Auth Plugin | ++----------------------------+--------------------------------------------+ +| `emq_auth_http`_ | Authentication/ACL with HTTP API | ++----------------------------+--------------------------------------------+ +| `emq_auth_mysql` _ | Authentication with MySQL | ++----------------------------+--------------------------------------------+ +| `emq_auth_pgsql`_ | Authentication with PostgreSQL | ++----------------------------+--------------------------------------------+ +| `emq_auth_redis`_ | Authentication with Redis | ++----------------------------+--------------------------------------------+ +| `emq_mod_rewrite`_ | Topics rewrite like HTTP rewrite module | ++----------------------------+--------------------------------------------+ +| `emq_plugin_mongo`_ | Authentication with MongoDB | ++----------------------------+--------------------------------------------+ +| `emq_sn`_ | MQTT-SN Protocol Plugin | ++----------------------------+--------------------------------------------+ +| `emq_coap`_ | CoAP Protocol Plugin | ++----------------------------+--------------------------------------------+ +| `emq_stomp`_ | STOMP Protocol Plugin | ++----------------------------+--------------------------------------------+ +| `emq_sockjs`_ | SockJS(Stomp) Plugin | ++----------------------------+--------------------------------------------+ +| `emq_recon`_ | Recon Plugin | ++----------------------------+--------------------------------------------+ +| `emq_reloader`_ | Reloader Plugin | ++----------------------------+--------------------------------------------+ A plugin could be enabled by 'bin/emqttd_ctl plugins load' command. For example, enable 'emqttd_auth_pgsql' plugin:: - ./bin/emqttd_ctl plugins load emqttd_auth_pgsql + ./bin/emqttd_ctl plugins load emq_auth_pgsql ----------------------- One Million Connections ----------------------- -Latest release of emqttd broker is scaling to 1.3 million MQTT connections on a 12 Core, 32G CentOS server. +Latest release of the *EMQ* broker is scaling to 1.3 million MQTT connections on a 12 Core, 32G CentOS server. .. NOTE:: @@ -220,43 +229,28 @@ TCP Stack Parameters Erlang VM --------- -emqttd/release/2.0/vm.args:: +emqttd/etc/emq.conf: - ## max process numbers - +P 2097152 +.. code-block:: properties + + ## Erlang Process Limit + node.process_limit = 2097152 ## Sets the maximum number of simultaneously existing ports for this system - +Q 1048576 + node.max_ports = 1048576 - ## Increase number of concurrent ports/sockets - -env ERL_MAX_PORTS 1048576 +Max Allowed Connections +----------------------- - -env ERTS_MAX_PORTS 1048576 +emqttd/etc/emq.conf 'listeners': -emqttd broker -------------- +.. code-block:: properties -emqttd/etc/emqttd.conf: + ## Size of acceptor pool + mqtt.listener.tcp.acceptors = 64 -.. code-block:: erlang - - {listener, mqtt, 1883, [ - %% Size of acceptor pool - {acceptors, 64}, - - %% Maximum number of concurrent clients - {max_clients, 1000000}, - - %% Socket Access Control - {access, [{allow, all}]}, - - %% Connection Options - {connopts, [ - %% Rate Limit. Format is 'burst, rate', Unit is KB/Sec - %% {rate_limit, "100,10"} %% 100K burst, 10K rate - ]}, - ... - ]}. + ## Maximum number of concurrent clients + mqtt.listener.tcp.max_clients = 1000000 Test Client ----------- @@ -283,22 +277,30 @@ GitHub: https://github.com/emqtt | `QMQTT`_ | QT MQTT Client | +--------------------+----------------------+ -.. _emqttc: https://github.com/emqtt/emqttc +Eclipse Paho: https://www.eclipse.org/paho/ + +MQTT.org: https://github.com/mqtt/mqtt.github.io/wiki/libraries + +.. _emqttc: https://github.com/emqtt/emqttc .. _emqtt_benchmark: https://github.com/emqtt/emqtt_benchmark -.. _CocoaMQTT: https://github.com/emqtt/CocoaMQTT -.. _QMQTT: https://github.com/emqtt/qmqtt +.. _CocoaMQTT: https://github.com/emqtt/CocoaMQTT +.. _QMQTT: https://github.com/emqtt/qmqtt -.. _emqttd_plugin_template: https://github.com/emqtt/emqttd_plugin_template -.. _emqttd_dashboard: https://github.com/emqtt/emqttd_dashboard -.. _emqttd_auth_ldap: https://github.com/emqtt/emqttd_auth_ldap -.. _emqttd_auth_http: https://github.com/emqtt/emqttd_auth_http -.. _emqttd_auth_mysql: https://github.com/emqtt/emqttd_plugin_mysql -.. _emqttd_auth_pgsql: https://github.com/emqtt/emqttd_plugin_pgsql -.. _emqttd_auth_redis: https://github.com/emqtt/emqttd_plugin_redis -.. _emqttd_auth_mongo: https://github.com/emqtt/emqttd_plugin_mongo -.. _emqttd_reloader: https://github.com/emqtt/emqttd_reloader -.. _emqttd_stomp: https://github.com/emqtt/emqttd_stomp -.. _emqttd_sockjs: https://github.com/emqtt/emqttd_sockjs -.. _emqttd_recon: https://github.com/emqtt/emqttd_recon -.. _emqttd_sn: https://github.com/emqtt/emqttd_sn +.. _emq_plugin_template: https://github.com/emqtt/emqttd_plugin_template +.. _emq_dashboard: https://github.com/emqtt/emqttd_dashboard +.. _emq_mod_rewrite: https://github.com/emqtt/emq_mod_rewrite +.. _emq_auth_clientid: https://github.com/emqtt/emq_auth_clientid +.. _emq_auth_username: https://github.com/emqtt/emq_auth_username +.. _emq_auth_ldap: https://github.com/emqtt/emqttd_auth_ldap +.. _emq_auth_http: https://github.com/emqtt/emqttd_auth_http +.. _emq_auth_mysql: https://github.com/emqtt/emqttd_plugin_mysql +.. _emq_auth_pgsql: https://github.com/emqtt/emqttd_plugin_pgsql +.. _emq_auth_redis: https://github.com/emqtt/emqttd_plugin_redis +.. _emq_auth_mongo: https://github.com/emqtt/emqttd_plugin_mongo +.. _emq_reloader: https://github.com/emqtt/emqttd_reloader +.. _emq_stomp: https://github.com/emqtt/emqttd_stomp +.. _emq_sockjs: https://github.com/emqtt/emqttd_sockjs +.. _emq_recon: https://github.com/emqtt/emqttd_recon +.. _emq_sn: https://github.com/emqtt/emqttd_sn +.. _emq_coap: https://github.com/emqtt/emqttd_coap diff --git a/docs/source/guide.rst b/docs/source/guide.rst index 709c4329f..031616feb 100644 --- a/docs/source/guide.rst +++ b/docs/source/guide.rst @@ -9,43 +9,11 @@ User Guide Authentication -------------- -The emqttd broker supports to authenticate MQTT clients with ClientID, Username/Password, IpAddress and even HTTP Cookies. +The *EMQ* broker supports to authenticate MQTT clients with ClientID, Username/Password, IpAddress and even HTTP Cookies. -The authentication is provided by a list of extended modules, or MySQL, PostgreSQL and Redis Plugins. +The authentication is provided by a list of plugins such as MySQL, PostgreSQL and Redis... -Enable an authentication module in etc/emqttd.config: - -.. code-block:: erlang - - %% Authentication and Authorization - {access, [ - %% Authetication. Anonymous Default - {auth, [ - %% Authentication with username, password - %{username, []}, - - %% Authentication with clientid - %{clientid, [{password, no}, {file, "etc/clients.config"}]}, - - %% Authentication with LDAP - % {ldap, [ - % {servers, ["localhost"]}, - % {port, 389}, - % {timeout, 30}, - % {user_dn, "uid=$u,ou=People,dc=example,dc=com"}, - % {ssl, fasle}, - % {sslopts, [ - % {"certfile", "ssl.crt"}, - % {"keyfile", "ssl.key"}]} - % ]}, - - %% Allow all - {anonymous, []} - ]}, - -.. NOTE:: "%" comments the line. - -If we enable several modules at the same time, the authentication process:: +If we enable several authentication plugins at the same time, the authentication process:: ---------------- ---------------- ------------- Client --> | Username | -ignore-> | ClientID | -ignore-> | Anonymous | @@ -54,77 +22,123 @@ If we enable several modules at the same time, the authentication process:: \|/ \|/ \|/ allow | deny allow | deny allow | deny -The authentication plugins developed by emqttd: +The authentication plugins implemented by default: +---------------------------+---------------------------+ | Plugin | Description | +===========================+===========================+ -| `emqttd_plugin_mysql`_ | MySQL Auth/ACL Plugin | +| `emq_auth_clientid`_ | ClientId Auth Plugin | +---------------------------+---------------------------+ -| `emqttd_plugin_pgsql`_ | PostgreSQL Auth/ACL Plugin| +| `emq_auth_username`_ | Username Auth Plugin | +---------------------------+---------------------------+ -| `emqttd_plugin_redis`_ | Redis Auth/ACL Plugin | +| `emq_auth_ldap`_ | LDAP Auth Plugin | ++---------------------------+---------------------------+ +| `emq_auth_http`_ | HTTP Auth/ACL Plugin | ++---------------------------+---------------------------+ +| `emq_auth_mysql`_ | MySQL Auth/ACL Plugin | ++---------------------------+---------------------------+ +| `emq_auth_pgsql`_ | Postgre Auth/ACL Plugin | ++---------------------------+---------------------------+ +| `emq_auth_redis`_ | Redis Auth/ACL Plugin | ++---------------------------+---------------------------+ +| `emq_auth_mongo`_ | MongoDB Auth/ACL Plugin | +---------------------------+---------------------------+ -.. NOTE:: If we load an authentication plugin, the authentication modules will be disabled. +--------------- +Allow Anonymous +--------------- -Username --------- +Configure etc/emq.conf to allow anonymous authentication: + +.. code-block:: properties + + ## Allow Anonymous authentication + mqtt.allow_anonymous = true + +Username/Password +----------------- Authenticate MQTT client with Username/Password:: - {username, [{client1, "passwd1"}, {client1, "passwd2"}]}, +Configure default users in etc/plugins/emq_auth_username.conf: -Two ways to add users: +.. code-block:: properties -1. Configure username and plain password directly:: + auth.username.$name=$password - {username, [{client1, "passwd1"}, {client1, "passwd2"}]}, +Enable `emq_auth_username`_ plugin: -2. Add user by './bin/emqttd_ctl users' command:: +.. code-block:: bash + + ./bin/emqttd_ctl plugins load emq_auth_username + +Add user by './bin/emqttd_ctl users' command:: $ ./bin/emqttd_ctl users add ClientId -------- -.. code-block:: erlang +Authentication with MQTT ClientId. - {clientid, [{password, no}, {file, "etc/clients.config"}]}, +Configure Client Ids in etc/plugins/emq_auth_clientid.conf: -Configure ClientIDs in etc/clients.config:: +.. code-block:: properties - testclientid0 - testclientid1 127.0.0.1 - testclientid2 192.168.0.1/24 + auth.clientid.$id=$password + +Enable `emq_auth_clientid`_ plugin: + +.. code-block:: bash + + ./bin/emqttd_ctl plugins load emq_auth_clientid LDAP ---- -.. code-block:: erlang +etc/plugins/emq_auth_ldap.conf: - {ldap, [ - {servers, ["localhost"]}, - {port, 389}, - {timeout, 30}, - {user_dn, "uid=$u,ou=People,dc=example,dc=com"}, - {ssl, fasle}, - {sslopts, [ - {"certfile", "ssl.crt"}, - {"keyfile", "ssl.key"}]} - ]}, +.. code-block:: properties -Anonymous ---------- + auth.ldap.servers = 127.0.0.1 -Allow any client to connect to the broker:: + auth.ldap.port = 389 - {anonymous, []} + auth.ldap.timeout = 30 + + auth.ldap.user_dn = uid=%u,ou=People,dc=example,dc=com + + auth.ldap.ssl = false + +Enable LDAP plugin:: + + ./bin/emqttd_ctl plugins load emq_auth_ldap + +HTTP +---- + +etc/plugins/emq_auth_http.conf: + +.. code-block:: properties + + ## Variables: %u = username, %c = clientid, %a = ipaddress, %P = password, %t = topic + + auth.http.auth_req = http://127.0.0.1:8080/mqtt/auth + auth.http.auth_req.method = post + auth.http.auth_req.params = clientid=%c,username=%u,password=%P + + auth.http.super_req = http://127.0.0.1:8080/mqtt/superuser + auth.http.super_req.method = post + auth.http.super_req.params = clientid=%c,username=%u + +Enable HTTP Plugin:: + + ./bin/emqttd_ctl plugins load emq_auth_http MySQL ----- -Authenticate against MySQL database. Support we create a mqtt_user table: +Authenticate with MySQL database. Suppose that we create a mqtt_user table: .. code-block:: sql @@ -138,36 +152,46 @@ Authenticate against MySQL database. Support we create a mqtt_user table: UNIQUE KEY `mqtt_username` (`username`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8; -Configure the 'authquery' and 'password_hash' in emqttd_plugin_mysql/etc/plugin.config: +Configure the 'auth_query' and 'password_hash' in etc/plugins/emq_auth_mysql.conf: -.. code-block:: erlang +.. code-block:: properties - [ + ## Mysql Server + auth.mysql.server = 127.0.0.1:3306 - {emqttd_plugin_mysql, [ + ## Mysql Pool Size + auth.mysql.pool = 8 - ... + ## Mysql Username + ## auth.mysql.username = - %% select password only - {authquery, "select password from mqtt_user where username = '%u' limit 1"}, + ## Mysql Password + ## auth.mysql.password = - %% hash algorithm: md5, sha, sha256, pbkdf2? - {password_hash, sha256}, + ## Mysql Database + auth.mysql.database = mqtt - ... + ## Variables: %u = username, %c = clientid - ]} - ]. + ## Authentication Query: select password only + auth.mysql.auth_query = select password from mqtt_user where username = '%u' limit 1 -Load the plugin:: + ## Password hash: plain, md5, sha, sha256, pbkdf2 + auth.mysql.password_hash = sha256 - ./bin/emqttd_ctl plugins load emqttd_plugin_mysql + ## %% Superuser Query + auth.mysql.super_query = select is_superuser from mqtt_user where username = '%u' limit 1 +Enable MySQL plugin: + +.. code-block:: bash + + ./bin/emqttd_ctl plugins load emq_plugin_mysql PostgreSQL ---------- -Authenticate against PostgreSQL database. Create a mqtt_user table: +Authenticate with PostgreSQL database. Create a mqtt_user table: .. code-block:: sql @@ -178,71 +202,152 @@ Authenticate against PostgreSQL database. Create a mqtt_user table: salt character varying(40) ); -Configure the 'authquery' and 'password_hash' in emqttd_plugin_pgsql/etc/plugin.config: +Configure the 'auth_query' and 'password_hash' in etc/plugins/emq_auth_pgsql.conf: -.. code-block:: erlang +.. code-block:: properties - [ + ## Postgre Server + auth.pgsql.server = 127.0.0.1:5432 - {emqttd_plugin_pgsql, [ + auth.pgsql.pool = 8 - ... + auth.pgsql.username = root - %% select password only - {authquery, "select password from mqtt_user where username = '%u' limit 1"}, + #auth.pgsql.password = - %% hash algorithm: md5, sha, sha256, pbkdf2? - {password_hash, sha256}, + auth.pgsql.database = mqtt - ... + auth.pgsql.encoding = utf8 - ]} - ]. + auth.pgsql.ssl = false -Load the plugin:: + ## Variables: %u = username, %c = clientid, %a = ipaddress - ./bin/emqttd_ctl plugins load emqttd_plugin_pgsql + ## Authentication Query: select password only + auth.pgsql.auth_query = select password from mqtt_user where username = '%u' limit 1 + + ## Password hash: plain, md5, sha, sha256, pbkdf2 + auth.pgsql.password_hash = sha256 + + ## sha256 with salt prefix + ## auth.pgsql.password_hash = salt sha256 + + ## sha256 with salt suffix + ## auth.pgsql.password_hash = sha256 salt + + ## Superuser Query + auth.pgsql.super_query = select is_superuser from mqtt_user where username = '%u' limit 1 + +Enable the plugin: + +.. code-block:: bash + + ./bin/emqttd_ctl plugins load emq_plugin_pgsql Redis ----- -Authenticate against Redis. MQTT users could be stored in redis HASH, the key is "mqtt_user:". +Authenticate with Redis. MQTT users could be stored in redis HASH, the key is "mqtt_user:". -Configure 'authcmd' and 'password_hash' in emqttd_plugin_redis/etc/plugin.config: +Configure 'auth_cmd' and 'password_hash' in etc/plugins/emq_auth_redis.conf: -.. code-block:: erlang +.. code-block:: properties - [ - {emqttd_plugin_redis, [ + ## Redis Server + auth.redis.server = 127.0.0.1:6379 - ... + ## Redis Pool Size + auth.redis.pool = 8 - %% HMGET mqtt_user:%u password - {authcmd, ["HGET", "mqtt_user:%u", "password"]}, + ## Redis Database + auth.redis.database = 0 - %% Password hash algorithm: plain, md5, sha, sha256, pbkdf2? - {password_hash, sha256}, + ## Redis Password + ## auth.redis.password = - ... + ## Variables: %u = username, %c = clientid - ]} - ]. + ## Authentication Query Command + auth.redis.auth_cmd = HGET mqtt_user:%u password -Load the plugin:: + ## Password hash: plain, md5, sha, sha256, pbkdf2 + auth.redis.password_hash = sha256 - ./bin/emqttd_ctl plugins load emqttd_plugin_redis + ## Superuser Query Command + auth.redis.super_cmd = HGET mqtt_user:%u is_superuser + +Enable the plugin: + +.. code-block:: bash + + ./bin/emqttd_ctl plugins load emq_auth_redis + +MongoDB +------- + +Create a `mqtt_user` collection:: + + { + username: "user", + password: "password hash", + is_superuser: boolean (true, false), + created: "datetime" + } + +Configure `super_query`, `auth_query` in etc/plugins/emq_auth_mongo.conf: + +.. code-block:: properties + + ## Mongo Server + auth.mongo.server = 127.0.0.1:27017 + + ## Mongo Pool Size + auth.mongo.pool = 8 + + ## Mongo User + ## auth.mongo.user = + + ## Mongo Password + ## auth.mongo.password = + + ## Mongo Database + auth.mongo.database = mqtt + + ## auth_query + auth.mongo.auth_query.collection = mqtt_user + + auth.mongo.auth_query.password_field = password + + auth.mongo.auth_query.password_hash = sha256 + + auth.mongo.auth_query.selector = username=%u + + ## super_query + auth.mongo.super_query.collection = mqtt_user + + auth.mongo.super_query.super_field = is_superuser + + auth.mongo.super_query.selector = username=%u + +Enable the plugin: + +.. code-block:: bash + + ./bin/emqttd_ctl plugins load emq_auth_mongo + +.. _acl: --- ACL --- -The ACL of emqttd broker is responsbile for authorizing MQTT clients to publish/subscribe topics. +The ACL of *EMQ* broker is responsbile for authorizing MQTT clients to publish/subscribe topics. The ACL rules define:: Allow|Deny Who Publish|Subscribe Topics -Access Control Module of emqttd broker will match the rules one by one:: +Access Control Module of *EMQ* broker will match the rules one by one:: --------- --------- --------- Client -> | Rule1 | --nomatch--> | Rule2 | --nomatch--> | Rule3 | --> Default @@ -255,18 +360,16 @@ Access Control Module of emqttd broker will match the rules one by one:: Internal -------- -The default ACL of emqttd broker is implemented by an 'internal' module. +The default ACL of *EMQ* broker is implemented by an 'internal' module. -Enable the 'internal' ACL module in etc/emqttd.config: +Enable the 'internal' ACL module in etc/emq.conf: -.. code-block:: erlang +.. code-block:: properties - {acl, [ - %% Internal ACL module - {internal, [{file, "etc/acl.config"}, {nomatch, allow}]} - ]} + ## Default ACL File + mqtt.acl_file = etc/acl.conf -The ACL rules of 'internal' module are defined in 'etc/acl.config' file: +The ACL rules of 'internal' module are defined in 'etc/acl.conf' file: .. code-block:: erlang @@ -282,10 +385,26 @@ The ACL rules of 'internal' module are defined in 'etc/acl.config' file: %% Allow all by default {allow, all}. +HTTP API +-------- + +ACL by HTTP API: https://github.com/emqtt/emq_auth_http + +Configure etc/plugins/emq_auth_http.conf and enable the plugin: + +.. code-block:: properties + + ## 'access' parameter: sub = 1, pub = 2 + auth.http.acl_req = http://127.0.0.1:8080/mqtt/acl + auth.http.acl_req.method = get + auth.http.acl_req.params = access=%A,username=%u,clientid=%c,ipaddr=%a,topic=%t + + auth.http.acl_nomatch = deny + MySQL ----- -ACL against MySQL database. The mqtt_acl table and default data: +ACL with MySQL database. The `mqtt_acl` table and default data: .. code-block:: sql @@ -309,29 +428,20 @@ ACL against MySQL database. The mqtt_acl table and default data: (6,1,'127.0.0.1',NULL,NULL,2,'#'), (7,1,NULL,'dashboard',NULL,1,'$SYS/#'); -Configure 'aclquery' and 'acl_nomatch' in emqttd_plugin_mysql/etc/plugin.config: +Configure 'acl-query' and 'acl_nomatch' in etc/plugins/emq_auth_mysql.conf: -.. code-block:: erlang +.. code-block:: properties - [ + ## ACL Query Command + auth.mysql.acl_query = select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c' - {emqttd_plugin_mysql, [ - - ... - - %% comment this query, the acl will be disabled - {aclquery, "select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'"}, - - %% If no rules matched, return... - {acl_nomatch, allow} - - ]} - ]. + ## ACL nomatch + auth.mysql.acl_nomatch = deny PostgreSQL ---------- -ACL against PostgreSQL database. The mqtt_acl table and default data: +ACL with PostgreSQL database. The mqtt_acl table and default data: .. code-block:: sql @@ -354,52 +464,66 @@ ACL against PostgreSQL database. The mqtt_acl table and default data: (6,1,'127.0.0.1',NULL,NULL,2,'#'), (7,1,NULL,'dashboard',NULL,1,'$SYS/#'); -Configure 'aclquery' and 'acl_nomatch' in emqttd_plugin_pgsql/etc/plugin.config: +Configure 'acl_query' and 'acl_nomatch' in etc/plugins/emq_auth_pgsql.conf: -.. code-block:: erlang +.. code-block:: properties - [ + ## ACL Query. Comment this query, the acl will be disabled. + auth.pgsql.acl_query = select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c' - {emqttd_plugin_pgsql, [ - - ... - - %% Comment this query, the acl will be disabled. Notice: don't edit this query! - {aclquery, "select allow, ipaddr, username, clientid, access, topic from mqtt_acl - where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'"}, - - %% If no rules matched, return... - {acl_nomatch, allow} - - ... - - ]} - ]. + ## If no rules matched, return... + auth.pgsql.acl_nomatch = deny Redis ----- -ACL against Redis. We store ACL rules for each MQTT client in a Redis List by defualt. The key is "mqtt_acl:", the value is a list of "publish ", "subscribe " or "pubsub ". +ACL with Redis. The ACL rules are stored in a Redis HashSet:: -Configure 'aclcmd' and 'acl_nomatch' in emqttd_plugin_redis/etc/plugin.config: + HSET mqtt_acl: topic1 1 + HSET mqtt_acl: topic2 2 + HSET mqtt_acl: topic3 3 -.. code-block:: erlang +Configure `acl_cmd` and `acl_nomatch` in etc/plugins/emq_auth_redis.conf: - [ - {emqttd_plugin_redis, [ +.. code-block:: properties - ... + ## ACL Query Command + auth.redis.acl_cmd = HGETALL mqtt_acl:%u - %% SMEMBERS mqtt_acl:%u - {aclcmd, ["SMEMBERS", "mqtt_acl:%u"]}, + ## ACL nomatch + auth.redis.acl_nomatch = deny - %% If no rules matched, return... - {acl_nomatch, deny}, +MongoDB +------- - ... +Store ACL Rules in a `mqtt_acl` collection: - ]} - ]. +.. code-block:: json + + { + username: "username", + clientid: "clientid", + publish: ["topic1", "topic2", ...], + subscribe: ["subtop1", "subtop2", ...], + pubsub: ["topic/#", "topic1", ...] + } + +For example, insert rules into `mqtt_acl` collection:: + + db.mqtt_acl.insert({username: "test", publish: ["t/1", "t/2"], subscribe: ["user/%u", "client/%c"]}) + db.mqtt_acl.insert({username: "admin", pubsub: ["#"]}) + +Configure `acl_query` and `acl_nomatch` in etc/plugins/emq_auth_mongo.conf: + +.. code-block:: properties + + ## acl_query + auth.mongo.acl_query.collection = mqtt_user + + auth.mongo.acl_query.selector = username=%u + + ## acl_nomatch + auth.mongo.acl_nomatch = deny ---------------------- MQTT Publish/Subscribe @@ -409,7 +533,7 @@ MQTT is a an extremely lightweight publish/subscribe messaging protocol desgined .. image:: _static/images/pubsub_concept.png -Install and start the emqttd broker, and then any MQTT client could connect to the broker, subscribe topics and publish messages. +Install and start the *EMQ* broker, and then any MQTT client could connect to the broker, subscribe topics and publish messages. MQTT Client Libraries: https://github.com/mqtt/mqtt.github.io/wiki/libraries @@ -420,67 +544,39 @@ For example, we use mosquitto_sub/pub commands:: MQTT V3.1.1 Protocol Specification: http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html -MQTT Listener of emqttd broker is configured in etc/emqttd.config: +MQTT Listener of emqttd broker is configured in etc/emq.conf: .. code-block:: erlang - {mqtt, 1883, [ - %% Size of acceptor pool - {acceptors, 16}, +.. code-block:: properties - %% Maximum number of concurrent clients - {max_clients, 512}, + ## TCP Listener: 1883, 127.0.0.1:1883, ::1:1883 + mqtt.listener.tcp = 1883 - %% Socket Access Control - {access, [{allow, all}]}, + ## Size of acceptor pool + mqtt.listener.tcp.acceptors = 8 - %% Connection Options - {connopts, [ - %% Rate Limit. Format is 'burst, rate', Unit is KB/Sec - %% {rate_limit, "100,10"} %% 100K burst, 10K rate - ]}, - - %% Socket Options - {sockopts, [ - %Set buffer if hight thoughtput - %{recbuf, 4096}, - %{sndbuf, 4096}, - %{buffer, 4096}, - %{nodelay, true}, - {backlog, 512} - ]} - ]}, + ## Maximum number of concurrent clients + mqtt.listener.tcp.max_clients = 1024 MQTT(SSL) Listener, Default Port is 8883: -.. code-block:: erlang +.. code-block:: properties - {mqtts, 8883, [ - %% Size of acceptor pool - {acceptors, 4}, + ## SSL Listener: 8883, 127.0.0.1:8883, ::1:8883 + mqtt.listener.ssl = 8883 - %% Maximum number of concurrent clients - {max_clients, 512}, + ## Size of acceptor pool + mqtt.listener.ssl.acceptors = 4 - %% Socket Access Control - {access, [{allow, all}]}, - - %% SSL certificate and key files - {ssl, [{certfile, "etc/ssl/ssl.crt"}, - {keyfile, "etc/ssl/ssl.key"}]}, - - %% Socket Options - {sockopts, [ - {backlog, 1024} - %{buffer, 4096}, - ]} - ]}, + ## Maximum number of concurrent clients + mqtt.listener.ssl.max_clients = 512 ---------------- HTTP Publish API ---------------- -The emqttd broker provides a HTTP API to help application servers publish messages to MQTT clients. +The *EMQ* broker provides a HTTP API to help application servers publish messages to MQTT clients. HTTP API: POST http://host:8083/mqtt/publish @@ -524,30 +620,20 @@ The Dashboard plugin provides a test page for WebSocket:: Listener of WebSocket and HTTP Publish API is configured in etc/emqttd.config: -.. code-block:: erlang +.. code-block:: properties - %% HTTP and WebSocket Listener - {http, 8083, [ - %% Size of acceptor pool - {acceptors, 4}, - %% Maximum number of concurrent clients - {max_clients, 64}, - %% Socket Access Control - {access, [{allow, all}]}, - %% Socket Options - {sockopts, [ - {backlog, 1024} - %{buffer, 4096}, - ]} - ]} + ## HTTP and WebSocket Listener + mqtt.listener.http = 8083 + mqtt.listener.http.acceptors = 4 + mqtt.listener.http.max_clients = 64 ----------- $SYS Topics ----------- -The emqttd broker periodically publishes internal status, MQTT statistics, metrics and client online/offline status to $SYS/# topics. +The *EMQ* broker periodically publishes internal status, MQTT statistics, metrics and client online/offline status to $SYS/# topics. -For emqttd broker is clustered, the $SYS topic path is started with:: +For the *EMQ* broker could be clustered, the $SYS topic path is started with:: $SYS/brokers/${node}/ @@ -760,7 +846,6 @@ Topic path started with: '$SYS/brokers/${node}/sysmon/' | busy_dist_port | Busy Dist Port | +------------------+--------------------+ - ----- Trace ----- @@ -785,6 +870,12 @@ Stop a Trace:: ./bin/emqttd_ctl trace topic "topic" off -.. _emqttd_plugin_mysql: https://github.com/emqtt/emqttd_plugin_mysql -.. _emqttd_plugin_pgsql: https://github.com/emqtt/emqttd_plugin_pgsql -.. _emqttd_plugin_redis: https://github.com/emqtt/emqttd_plugin_redis +.. _emq_auth_clientid: https://github.com/emqtt/emq_auth_clientid +.. _emq_auth_username: https://github.com/emqtt/emq_auth_username +.. _emq_auth_ldap: https://github.com/emqtt/emqttd_auth_ldap +.. _emq_auth_http: https://github.com/emqtt/emqttd_auth_http +.. _emq_auth_mysql: https://github.com/emqtt/emqttd_plugin_mysql +.. _emq_auth_pgsql: https://github.com/emqtt/emqttd_plugin_pgsql +.. _emq_auth_redis: https://github.com/emqtt/emqttd_plugin_redis +.. _emq_auth_mongo: https://github.com/emqtt/emqttd_plugin_mongo + diff --git a/docs/source/install.rst b/docs/source/install.rst index a245cd612..37320e6ce 100644 --- a/docs/source/install.rst +++ b/docs/source/install.rst @@ -5,7 +5,7 @@ Installation ============ -The emqttd broker is cross-platform, which could be deployed on Linux, FreeBSD, Mac, Windows and even Raspberry Pi. +The *EMQ* broker is cross-platform, which could be deployed on Linux, FreeBSD, Mac, Windows and even Raspberry Pi. .. NOTE:: @@ -32,10 +32,12 @@ Download binary packages from: http://emqtt.io/downloads +-----------+------------------------------------------+ | Windows | http://emqtt.io/downloads/latest/windows | +-----------+------------------------------------------+ +| Docker | http://emqtt.com/downloads/latest/docker | ++-----------+------------------------------------------+ The package name consists of platform, version and release time. -For example: emqttd-centos64-v2.0-beta.2-20160910.zip +For example: emqttd-centos64-v2.0-rc.2-20161019.zip .. _install_on_linux: @@ -47,7 +49,7 @@ Download CentOS Package from: http://emqtt.io/downloads/latest/centos, and then .. code-block:: bash - unzip emqttd-centos64-v2.0-beta.2-20160910.zip + unzip emqttd-centos64-v2.0-rc.2-20161019.zip Start the broker in console mode: @@ -92,8 +94,6 @@ Start the broker in daemon mode: ./bin/emqttd start -The boot logs in log/emqttd_sasl.log file. - Check the running status of the broker: .. code-block:: bash @@ -130,17 +130,18 @@ We could install the broker on Mac OS X to develop and debug MQTT applications. Download Mac Package from: http://emqtt.io/downloads/latest/macosx -Configure 'lager' log level in 'releases/2.0/sys.config', all MQTT messages recevied/sent will be printed on console: +Configure log level in `etc/emq.conf`, all MQTT messages recevied/sent will be printed on console: -.. code-block:: erlang +.. code-block:: - {lager, [ - ... - {handlers, [ - {lager_console_backend, info}, - ... - ]} - ]}, + ## Console log. Enum: off, file, console, both + log.console = both + + ## Console log level. Enum: debug, info, notice, warning, error, critical, alert, emergency + log.console.level = debug + + ## Console log file + log.console.file = log/console.log The install and boot process on Mac are same to Linux. @@ -186,7 +187,7 @@ Uninstall emqttd service:: Installing From Source ---------------------- -The emqttd broker requires Erlang/OTP R17+ and git client to build: +The *EMQ* broker requires Erlang/OTP R18+ and git client to build: Install Erlang: http://www.erlang.org/ @@ -226,22 +227,16 @@ TCP Ports Used The TCP ports used can be configured in etc/emqttd.config: -.. code-block:: erlang +.. code-block:: properties - %% Plain MQTT - {listener, mqtt, 1883, [ - ... - ]}. + ## TCP Listener: 1883, 127.0.0.1:1883, ::1:1883 + mqtt.listener.tcp = 1883 - %% MQTT/SSL - {listener, mqtts, 8883, [ - ... - ]}. + ## SSL Listener: 8883, 127.0.0.1:8883, ::1:8883 + mqtt.listener.ssl = 8883 - %% HTTP and WebSocket Listener - {listener, http, 8083, [ - ... - ]}. + ## HTTP and WebSocket Listener + mqtt.listener.http = 8083 The 18083 port is used by Web Dashboard of the broker. Default login: admin, Password: public @@ -251,45 +246,38 @@ The 18083 port is used by Web Dashboard of the broker. Default login: admin, Pas Quick Setup ----------- -Two main configuration files of the emqttd broker: +Two main configuration files of the *EMQ* broker: +-----------------------+-----------------------------------+ -| releases/2.0/vm.args | Erlang VM Arguments | +| etc/emq.conf | EMQ Broker Config | +-----------------------+-----------------------------------+ -| etc/emqttd.conf | emqttd broker Config | +| etc/plugins/\*.conf | EMQ Plugins' Config | +-----------------------+-----------------------------------+ -Two important parameters in releases/2.0/vm.args: +Two important parameters in etc/emq.conf: -+-------+---------------------------------------------------------------------------+ -| +P | Max number of Erlang proccesses. A MQTT client consumes two proccesses. | -| | The value should be larger than max_clients * 2 | -+-------+---------------------------------------------------------------------------+ -| +Q | Max number of Erlang Ports. A MQTT client consumes one port. | -| | The value should be larger than max_clients. | -+-------+---------------------------------------------------------------------------+ ++--------------------+-------------------------------------------------------------------------+ +| node.process_limit | Max number of Erlang proccesses. A MQTT client consumes two proccesses. | +| | The value should be larger than max_clients * 2 | ++--------------------+-------------------------------------------------------------------------+ +| node.max_ports | Max number of Erlang Ports. A MQTT client consumes one port. | +| | The value should be larger than max_clients. | ++--------------------+-------------------------------------------------------------------------+ .. NOTE:: - +Q > maximum number of allowed concurrent clients - +P > maximum number of allowed concurrent clients * 2 + node.process_limit > maximum number of allowed concurrent clients * 2 + node.max_ports > maximum number of allowed concurrent clients The maximum number of allowed MQTT clients: -.. code-block:: erlang +.. code-block:: properties - %% Plain MQTT - {listener, mqtt, 1883, [ + mqtt.listener.tcp = 1883 - %% Size of acceptor pool - {acceptors, 16}, + mqtt.listener.tcp.acceptors = 8 - %% Maximum number of concurrent clients - {max_clients, 8192}, - - ... - - ]}. + mqtt.listener.tcp.max_clients = 1024 .. _init_d_emqttd: @@ -357,4 +345,6 @@ boot test:: ## erlexec: HOME must be set uncomment '# export HOME=/root' if "HOME must be set" error. +.. _emq_dashboard: https://github.com/emqtt/emqttd_dashboard + diff --git a/docs/source/plugins.rst b/docs/source/plugins.rst index 2b90427cb..a8f327c13 100644 --- a/docs/source/plugins.rst +++ b/docs/source/plugins.rst @@ -5,47 +5,55 @@ Plugins ======= -The emqttd broker could be extended by plugins. Users could develop plugins to customize authentication, ACL and functions of the broker, or integrate the broker with other systems. +The *EMQ* broker could be extended by plugins. Users could develop plugins to customize authentication, ACL and functions of the broker, or integrate the broker with other systems. -The plugins that emqttd 2.0 released: +The plugins that *EMQ* 2.0-rc.2 released: -+---------------------------+---------------------------+ -| Plugin | Description | -+===========================+===========================+ -| `emqttd_dashboard`_ | Web Dashboard | -+---------------------------+---------------------------+ -| `emqttd_plugin_template`_ | Template Plugin | -+---------------------------+---------------------------+ -| `emqttd_auth_ldap`_ | LDAP Auth | -+---------------------------+---------------------------+ -| `emqttd_auth_http`_ | HTTP Auth/ACL Plugin | -+---------------------------+---------------------------+ -| `emqttd_auth_mysql`_ | MySQL Auth/ACL Plugin | -+---------------------------+---------------------------+ -| `emqttd_auth_pgsql`_ | PostgreSQL Auth/ACL Plugin| -+---------------------------+---------------------------+ -| `emqttd_auth_redis`_ | Redis Auth/ACL Plugin | -+---------------------------+---------------------------+ -| `emqttd_auth_mongo`_ | MongoDB Auth/ACL Plugin | -+---------------------------+---------------------------+ -| `emqttd_sn`_ | MQTT-SN Protocol Plugin | -+---------------------------+---------------------------+ -| `emqttd_stomp`_ | STOMP Protocol Plugin | -+---------------------------+---------------------------+ -| `emqttd_sockjs`_ | STOMP over SockJS Plugin | -+---------------------------+---------------------------+ -| `emqttd_recon`_ | Recon Plugin | -+---------------------------+---------------------------+ -| `emqttd_reloader`_ | Reloader Plugin | -+---------------------------+---------------------------+ ++------------------------+-------------------------------+ +| Plugin | Description | ++========================+===============================+ +| `emq_dashboard`_ | Web Dashboard | ++------------------------+-------------------------------+ +| `emq_auth_clientid`_ | ClientId Auth Plugin | ++------------------------+-------------------------------+ +| `emq_auth_username`_ | Username/Password Auth Plugin | ++------------------------+-------------------------------+ +| `emq_auth_ldap`_ | LDAP Auth | ++------------------------+-------------------------------+ +| `emq_auth_http`_ | HTTP Auth/ACL Plugin | ++------------------------+-------------------------------+ +| `emq_auth_mysql`_ | MySQL Auth/ACL Plugin | ++------------------------+-------------------------------+ +| `emq_auth_pgsql`_ | PostgreSQL Auth/ACL Plugin | ++------------------------+-------------------------------+ +| `emq_auth_redis`_ | Redis Auth/ACL Plugin | ++------------------------+-------------------------------+ +| `emq_auth_mongo`_ | MongoDB Auth/ACL Plugin | ++------------------------+-------------------------------+ +| `emq_mod_rewrite`_ | Topic Rewrite Plugin | ++------------------------+-------------------------------+ +| `emq_coap`_ | CoAP Protocol Plugin | ++------------------------+-------------------------------+ +| `emq_sn`_ | MQTT-SN Protocol Plugin | ++------------------------+-------------------------------+ +| `emq_stomp`_ | STOMP Protocol Plugin | ++------------------------+-------------------------------+ +| `emq_sockjs`_ | STOMP over SockJS Plugin | ++------------------------+-------------------------------+ +| `emq_recon`_ | Recon Plugin | ++------------------------+-------------------------------+ +| `emq_reloader`_ | Reloader Plugin | ++------------------------+-------------------------------+ +| `emq_plugin_template`_ | Template Plugin | ++------------------------+-------------------------------+ ----------------------------------------- -emqttd_plugin_template - Template Plugin ----------------------------------------- +------------------------------------- +emq_plugin_template - Template Plugin +------------------------------------- -A plugin is just a normal Erlang application which has its own configuration file: 'etc/.config'. +A plugin is just a normal Erlang application which has its own configuration file: 'etc/.conf|config'. -emqttd_plugin_template is a demo plugin. +emq_plugin_template is a plugin template. Load, unload Plugin ------------------- @@ -58,11 +66,69 @@ Use 'bin/emqttd_ctl plugins' CLI to load, unload a plugin:: ./bin/emqttd_ctl plugins list ------------------------------------ -emqttd_dashboard - Dashboard Plugin ------------------------------------ +---------------------------------------- +emq_auth_clientid - ClientID Auth Plugin +---------------------------------------- -The Web Dashboard for emqttd broker. The plugin will be loaded automatically when the broker started successfully. +Released in 2.0-rc.2: https://github.com/emqtt/emq_auth_clientid + +Configure ClientID Auth Plugin +------------------------------ + +etc/plugins/emq_auth_clientid.conf: + +.. code-block:: properties + + ## auth.client.$clientid = $password + ## Examples + ## auth.client.id = passwd + ## auth.client.dev:devid = passwd2 + ## auth.client.app:appid = passwd2 + +Load ClientId Auth Plugin +------------------------- + +.. code-block:: bash + + ./bin/emqttd_ctl plugins load emq_auth_clientid + +---------------------------------------- +emq_auth_username - Username Auth Plugin +---------------------------------------- + +Released in 2.0-rc.2: https://github.com/emqtt/emq_auth_username + +Configure Username Auth Plugin +------------------------------ + +etc/plugins/emq_auth_username.conf: + +.. code-block:: properties + + ##auth.username.$name=$password + + ## Examples: + ##auth.username.admin=public + ##auth.username.feng@emqtt.io=public + +Add username/password by `./bin/emqttd_ctl users` CLI: + +.. code-block:: bash + + $ ./bin/emqttd_ctl users add + +Load Username Auth Plugin +------------------------- + +.. code-block:: bash + + ./bin/emqttd_ctl plugins load emq_auth_username + +-------------------------------- +emq_dashboard - Dashboard Plugin +-------------------------------- + +The Web Dashboard for *EMQ* broker. The plugin will be loaded automatically when the broker started successfully. +------------------+---------------------------+ | Address | http://localhost:18083 | @@ -77,99 +143,87 @@ The Web Dashboard for emqttd broker. The plugin will be loaded automatically whe Configure Dashboard Plugin -------------------------- -etc/plugins/emqttd_dashboard.conf: +etc/plugins/emq_dashboard.conf: -.. code-block:: erlang +.. code-block:: properties - {listener, - {dashboard, 18083, [ - {acceptors, 4}, - {max_clients, 512} - ]} - }. + ## HTTP Listener + dashboard.listener.http = 18083 + dashboard.listener.http.acceptors = 2 + dashboard.listener.http.max_clients = 512 ----------------------------------- -emqttd_auth_ldap: LDAP Auth Plugin ----------------------------------- + ## HTTPS Listener + ## dashboard.listener.https = 18084 + ## dashboard.listener.https.acceptors = 2 + ## dashboard.listener.https.max_clients = 512 + ## dashboard.listener.https.handshake_timeout = 15 + ## dashboard.listener.https.certfile = etc/certs/cert.pem + ## dashboard.listener.https.keyfile = etc/certs/key.pem + ## dashboard.listener.https.cacertfile = etc/certs/cacert.pem + ## dashboard.listener.https.verify = verify_peer + ## dashboard.listener.https.failed_if_no_peer_cert = true + +------------------------------- +emq_auth_ldap: LDAP Auth Plugin +------------------------------- LDAP Auth Plugin: https://github.com/emqtt/emqttd_auth_ldap -.. NOTE:: Supported in 2.0-beta1 release +.. NOTE:: Released in 2.0-beta.1 Configure LDAP Plugin --------------------- -etc/plugins/emqttd_auth_ldap.conf: +etc/plugins/emq_auth_ldap.conf: -.. code-block:: erlang +.. code-block:: properties - {ldap, [ - {servers, ["localhost"]}, - {port, 389}, - {timeout, 30}, - {user_dn, "uid=$u,ou=People,dc=example,dc=com"}, - {ssl, fasle}, - {sslopts, [ - {certfile, "ssl.crt"}, - {keyfile, "ssl.key"} - ]} - ]}. + auth.ldap.servers = 127.0.0.1 + + auth.ldap.port = 389 + + auth.ldap.timeout = 30 + + auth.ldap.user_dn = uid=%u,ou=People,dc=example,dc=com + + auth.ldap.ssl = false Load LDAP Plugin ---------------- -./bin/emqttd_ctl plugins load emqttd_auth_ldap +./bin/emqttd_ctl plugins load emq_auth_ldap ---------------------------------------- -emqttd_auth_http - HTTP Auth/ACL Plugin ---------------------------------------- +------------------------------------ +emq_auth_http - HTTP Auth/ACL Plugin +------------------------------------ -MQTT Authentication/ACL with HTTP API: https://github.com/emqtt/emqttd_auth_http +MQTT Authentication/ACL with HTTP API: https://github.com/emqtt/emq_auth_http .. NOTE:: Supported in 1.1 release Configure HTTP Auth/ACL Plugin ------------------------------ -etc/plugins/emqttd_auth_http.conf: +etc/plugins/emq_auth_http.conf: -.. code-block:: erlang +.. code-block:: properties - %% Variables: %u = username, %c = clientid, %a = ipaddress, %t = topic + ## Variables: %u = username, %c = clientid, %a = ipaddress, %P = password, %t = topic - {super_req, [ - {method, post}, - {url, "http://localhost:8080/mqtt/superuser"}, - {params, [ - {username, "%u"}, - {clientid, "%c"} - ]} - ]}. + auth.http.auth_req = http://127.0.0.1:8080/mqtt/auth + auth.http.auth_req.method = post + auth.http.auth_req.params = clientid=%c,username=%u,password=%P - {auth_req, [ - {method, post}, - {url, "http://localhost:8080/mqtt/auth"}, - {params, [ - {clientid, "%c"}, - {username, "%u"}, - {password, "%P"} - ]} - ]}. + auth.http.super_req = http://127.0.0.1:8080/mqtt/superuser + auth.http.super_req.method = post + auth.http.super_req.params = clientid=%c,username=%u - %% 'access' parameter: sub = 1, pub = 2 - - {acl_req, [ - {method, post}, - {url, "http://localhost:8080/mqtt/acl"}, - {params, [ - {access, "%A"}, - {username, "%u"}, - {clientid, "%c"}, - {ipaddr, "%a"}, - {topic, "%t"} - ]} - ]}. + ## 'access' parameter: sub = 1, pub = 2 + auth.http.acl_req = http://127.0.0.1:8080/mqtt/acl + auth.http.acl_req.method = get + auth.http.acl_req.params = access=%A,username=%u,clientid=%c,ipaddr=%a,topic=%t + auth.http.acl_nomatch = deny HTTP Auth/ACL API ----------------- @@ -179,15 +233,15 @@ Return 200 if ok Return 4xx if unauthorized Load HTTP Auth/ACL Plugin ----------------------------- +------------------------- .. code:: bash - ./bin/emqttd_ctl plugins load emqttd_auth_http + ./bin/emqttd_ctl plugins load emq_auth_http -------------------------------------------- -emqttd_plugin_mysql - MySQL Auth/ACL Plugin -------------------------------------------- +---------------------------------------- +emq_plugin_mysql - MySQL Auth/ACL Plugin +---------------------------------------- MQTT Authentication, ACL with MySQL database. @@ -235,64 +289,54 @@ MQTT ACL Table Configure MySQL Auth/ACL Plugin ------------------------------- -etc/plugins/emqttd_plugin_mysql.conf: +etc/plugins/emq_plugin_mysql.conf: -.. code-block:: erlang +.. code-block:: properties - {mysql_pool, [ - %% pool options - {pool_size, 8}, - {auto_reconnect, 1}, + ## Mysql Server + auth.mysql.server = 127.0.0.1:3306 - %% mysql options - {host, "localhost"}, - {port, 3306}, - {user, ""}, - {password, ""}, - {database, "mqtt"}, - {encoding, utf8}, - {keep_alive, true} - ]}. + ## Mysql Pool Size + auth.mysql.pool = 8 - %% Variables: %u = username, %c = clientid, %a = ipaddress + ## Mysql Username + ## auth.mysql.username = - %% Superuser Query - {superquery, "select is_superuser from mqtt_user where username = '%u' limit 1"}. + ## Mysql Password + ## auth.mysql.password = - %% Authentication Query: select password only - {authquery, "select password from mqtt_user where username = '%u' limit 1"}. + ## Mysql Database + auth.mysql.database = mqtt - %% hash algorithm: plain, md5, sha, sha256, pbkdf2? - {password_hash, sha256}. + ## Variables: %u = username, %c = clientid - %% select password with salt - %% {authquery, "select password, salt from mqtt_user where username = '%u'"}. + ## Authentication Query: select password only + auth.mysql.auth_query = select password from mqtt_user where username = '%u' limit 1 - %% sha256 with salt prefix - %% {password_hash, {salt, sha256}}. + ## Password hash: plain, md5, sha, sha256, pbkdf2 + auth.mysql.password_hash = sha256 - %% sha256 with salt suffix - %% {password_hash, {sha256, salt}}. + ## %% Superuser Query + auth.mysql.super_query = select is_superuser from mqtt_user where username = '%u' limit 1 - %% '%a' = ipaddress, '%u' = username, '%c' = clientid - %% Comment this query, the acl will be disabled - {aclquery, "select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'"}. + ## ACL Query Command + auth.mysql.acl_query = select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c' - %% If no ACL rules matched, return... - {acl_nomatch, allow}. + ## ACL nomatch + auth.mysql.acl_nomatch = deny Load MySQL Auth/ACL plugin -------------------------- .. code-block:: bash - ./bin/emqttd_ctl plugins load emqttd_plugin_mysql + ./bin/emqttd_ctl plugins load emq_auth_mysql ------------------------------------------------- -emqttd_plugin_pgsql - PostgreSQL Auth/ACL Plugin ------------------------------------------------- +------------------------------------------- +emq_auth_pgsql - PostgreSQL Auth/ACL Plugin +------------------------------------------- -MQTT Authentication, ACL with PostgreSQL Database. +MQTT Authentication/ACL with PostgreSQL Database. Postgre MQTT User Table ----------------------- @@ -332,106 +376,100 @@ Postgre MQTT ACL Table (7,1,NULL,'dashboard',NULL,1,'$SYS/#'); Configure Postgre Auth/ACL Plugin ------------------------------------------------ +---------------------------------- -Plugin Config: etc/plugins/emqttd_plugin_pgsql.conf. +Plugin Config: etc/plugins/emq_auth_pgsql.conf. Configure host, username, password and database of PostgreSQL: -.. code-block:: erlang +.. code-block:: properties - {pgsql_pool, [ - %% pool options - {pool_size, 8}, - {auto_reconnect, 3}, + ## Postgre Server + auth.pgsql.server = 127.0.0.1:5432 - %% pgsql options - {host, "localhost"}, - {port, 5432}, - {ssl, false}, - {username, "feng"}, - {password, ""}, - {database, "mqtt"}, - {encoding, utf8} - ]}. + auth.pgsql.pool = 8 - %% Variables: %u = username, %c = clientid, %a = ipaddress + auth.pgsql.username = root - %% Superuser Query - {superquery, "select is_superuser from mqtt_user where username = '%u' limit 1"}. + #auth.pgsql.password = - %% Authentication Query: select password only - {authquery, "select password from mqtt_user where username = '%u' limit 1"}. + auth.pgsql.database = mqtt - %% hash algorithm: plain, md5, sha, sha256, pbkdf2? - {password_hash, sha256}. + auth.pgsql.encoding = utf8 - %% select password with salt - %% {authquery, "select password, salt from mqtt_user where username = '%u'"}. + auth.pgsql.ssl = false - %% sha256 with salt prefix - %% {password_hash, {salt, sha256}}. + ## Variables: %u = username, %c = clientid, %a = ipaddress - %% sha256 with salt suffix - %% {password_hash, {sha256, salt}}. + ## Authentication Query: select password only + auth.pgsql.auth_query = select password from mqtt_user where username = '%u' limit 1 - %% Comment this query, the acl will be disabled. Notice: don't edit this query! - {aclquery, "select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c'"}. + ## Password hash: plain, md5, sha, sha256, pbkdf2 + auth.pgsql.password_hash = sha256 - %% If no rules matched, return... - {acl_nomatch, allow}. + ## sha256 with salt prefix + ## auth.pgsql.password_hash = salt sha256 + + ## sha256 with salt suffix + ## auth.pgsql.password_hash = sha256 salt + + ## Superuser Query + auth.pgsql.super_query = select is_superuser from mqtt_user where username = '%u' limit 1 + + ## ACL Query. Comment this query, the acl will be disabled. + auth.pgsql.acl_query = select allow, ipaddr, username, clientid, access, topic from mqtt_acl where ipaddr = '%a' or username = '%u' or username = '$all' or clientid = '%c' + + ## If no rules matched, return... + auth.pgsql.acl_nomatch = deny Load Postgre Auth/ACL Plugin ----------------------------- .. code-block:: bash - ./bin/emqttd_ctl plugins load emqttd_auth_pgsql + ./bin/emqttd_ctl plugins load emq_auth_pgsql -------------------------------------------- -emqttd_plugin_redis - Redis Auth/ACL Plugin -------------------------------------------- +-------------------------------------- +emq_auth_redis - Redis Auth/ACL Plugin +-------------------------------------- -MQTT Authentication, ACL with Redis: https://github.com/emqtt/emqttd_plugin_redis +MQTT Authentication, ACL with Redis: https://github.com/emqtt/emq_auth_redis Configure Redis Auth/ACL Plugin ------------------------------- -etc/plugins/emqttd_auth_redis.conf: +etc/plugins/emq_auth_redis.conf: -.. code-block:: erlang +.. code-block:: properties - {redis_pool, [ - %% pool options - {pool_size, 8}, - {auto_reconnect, 2}, + ## Redis Server + auth.redis.server = 127.0.0.1:6379 - %% redis options - {host, "127.0.0.1"}, - {port, 6379}, - {database, 0}, - {password, ""} - ]}. + ## Redis Pool Size + auth.redis.pool = 8 - %% Variables: %u = username, %c = clientid + ## Redis Database + auth.redis.database = 0 - %% HMGET mqtt_user:%u password - {authcmd, "HGET mqtt_user:%u password"}. + ## Redis Password + ## auth.redis.password = - %% Password hash algorithm: plain, md5, sha, sha256, pbkdf2? - {password_hash, sha256}. + ## Variables: %u = username, %c = clientid - %% HMGET mqtt_user:%u is_superuser - {supercmd, "HGET mqtt_user:%u is_superuser"}. + ## Authentication Query Command + auth.redis.auth_cmd = HGET mqtt_user:%u password - %% HGETALL mqtt_acl:%u - {aclcmd, "HGETALL mqtt_acl:%u"}. + ## Password hash: plain, md5, sha, sha256, pbkdf2 + auth.redis.password_hash = sha256 - %% If no rules matched, return... - {acl_nomatch, deny}. + ## Superuser Query Command + auth.redis.super_cmd = HGET mqtt_user:%u is_superuser - %% Load Subscriptions form Redis when client connected. - {subcmd, "HGETALL mqtt_sub:%u"}. + ## ACL Query Command + auth.redis.acl_cmd = HGETALL mqtt_acl:%u + + ## ACL nomatch + auth.redis.acl_nomatch = deny Redis User Hash --------------- @@ -466,59 +504,59 @@ Load Redis Auth/ACL Plugin .. code-block:: bash - ./bin/emqttd_ctl plugins load emqttd_auth_redis + ./bin/emqttd_ctl plugins load emq_auth_redis ---------------------------------------------- -emqttd_plugin_mongo - MongoDB Auth/ACL Plugin ---------------------------------------------- +---------------------------------------- +emq_auth_mongo - MongoDB Auth/ACL Plugin +---------------------------------------- -MQTT Authentication, ACL with MongoDB: https://github.com/emqtt/emqttd_plugin_mongo +MQTT Authentication, ACL with MongoDB: https://github.com/emqtt/emqttd_auth_mongo Configure MongoDB Auth/ACL Plugin --------------------------------- -etc/plugins/emqttd_plugin_mongo.conf: +etc/plugins/emq_plugin_mongo.conf: -.. code-block:: erlang +.. code-block:: properties - {mongo_pool, [ - {pool_size, 8}, - {auto_reconnect, 3}, + ## Mongo Server + auth.mongo.server = 127.0.0.1:27017 - %% Mongodb Opts - {host, "localhost"}, - {port, 27017}, - %% {login, ""}, - %% {password, ""}, - {database, "mqtt"} - ]}. + ## Mongo Pool Size + auth.mongo.pool = 8 - %% Variables: %u = username, %c = clientid + ## Mongo User + ## auth.mongo.user = - %% Superuser Query - {superquery, [ - {collection, "mqtt_user"}, - {super_field, "is_superuser"}, - {selector, {"username", "%u"}} - ]}. + ## Mongo Password + ## auth.mongo.password = - %% Authentication Query - {authquery, [ - {collection, "mqtt_user"}, - {password_field, "password"}, - %% Hash Algorithm: plain, md5, sha, sha256, pbkdf2? - {password_hash, sha256}, - {selector, {"username", "%u"}} - ]}. + ## Mongo Database + auth.mongo.database = mqtt - %% ACL Query: "%u" = username, "%c" = clientid - {aclquery, [ - {collection, "mqtt_acl"}, - {selector, {"username", "%u"}} - ]}. + ## auth_query + auth.mongo.auth_query.collection = mqtt_user - %% If no ACL rules matched, return... - {acl_nomatch, deny}. + auth.mongo.auth_query.password_field = password + + auth.mongo.auth_query.password_hash = sha256 + + auth.mongo.auth_query.selector = username=%u + + ## super_query + auth.mongo.super_query.collection = mqtt_user + + auth.mongo.super_query.super_field = is_superuser + + auth.mongo.super_query.selector = username=%u + + ## acl_query + auth.mongo.acl_query.collection = mqtt_user + + auth.mongo.acl_query.selector = username=%u + + ## acl_nomatch + auth.mongo.acl_nomatch = deny MongoDB Database ---------------- @@ -570,106 +608,178 @@ Load MongoDB Auth/ACL Plugin .. code-block:: bash - ./bin/emqttd_ctl plugins load emqttd_auth_mongo + ./bin/emqttd_ctl plugins load emq_auth_mongo ---------------------------- -emqttd_sn: MQTT-SN Protocol --------------------------- +-------------------------------------- +emq_mod_rewrite - Topic Rewrite Plugin +-------------------------------------- + +Released in 2.0-rc.2: https://github.com/emqtt/emq_mod_rewrite + +Configure Rewrite Plugin +------------------------ + +etc/plugins/emq_mod_rewrite.conf: + +.. code-block:: erlang + + [ + {emq_mod_rewrite, [ + {rules, [ + %% {rewrite, Topic, Re, Dest} + + %% Example: x/y/ -> z/y/ + %% {rewrite, "x/#", "^x/y/(.+)$", "z/y/$1"}, + + %% {rewrite, "y/+/z/#", "^y/(.+)/z/(.+)$", "y/z/$2"} + ]} + ]} + ]. + +Load Rewrite Plugin +------------------- + +.. code:: bash + + ./bin/emqttd_ctl plugins load emq_mod_rewrite + +------------------------------ +emq_coap: CoAP Protocol Plugin +------------------------------ + +CoAP Protocol Plugin: https://github.com/emqtt/emqttd_coap + +Configure CoAP Plugin +--------------------- + +.. code-block:: properties + + coap.server = 5683 + + coap.prefix.mqtt = mqtt + + coap.handler.mqtt = emq_coap_gateway + +Load CoAP Protocol Plugin +------------------------- + +.. code:: bash + + ./bin/emqttd_ctl plugins load emq_coap + +libcoap Client +-------------- + +.. code:: bash + + yum install libcoap + + % coap client publish message + coap-client -m post -e "qos=0&retain=0&message=payload&topic=hello" coap://localhost/mqtt + +------------------------ +emq_sn: MQTT-SN Protocol +------------------------ MQTT-SN Protocol/Gateway Plugin. Configure MQTT-SN Plugin -------------------------- +------------------------ .. NOTE:: UDP Port for MQTT-SN: 1884 -etc/plugins/emqttd_sn.conf:: +etc/plugins/emq_sn.conf: - {listener, {1884, []}}. +.. code-block:: properties + + mqtt.sn.port = 1884 Load MQTT-SN Plugin ------------------- .. code:: - ./bin/emqttd_ctl plugins load emqttd_sn + ./bin/emqttd_ctl plugins load emq_sn ------------------------------ -emqttd_stomp - STOMP Protocol ------------------------------ +-------------------------- +emq_stomp - STOMP Protocol +-------------------------- Support STOMP 1.0/1.1/1.2 clients to connect to emqttd broker and communicate with MQTT Clients. Configure Stomp Plugin ---------------------- -etc/plugins/emqttd_stomp.conf: +etc/plugins/emq_stomp.conf: .. NOTE:: Default Port for STOMP Protocol: 61613 -.. code-block:: erlang +.. code-block:: properties - {default_user, [ - {login, "guest"}, - {passcode, "guest"} - ]}. + stomp.default_user.login = guest - {allow_anonymous, true}. + stomp.default_user.passcode = guest - {frame, [ - {max_headers, 10}, - {max_header_length, 1024}, - {max_body_length, 8192} - ]}. + stomp.allow_anonymous = true - {listener, emqttd_stomp, 61613, [ - {acceptors, 4}, - {max_clients, 512} - ]}. + stomp.frame.max_headers = 10 + stomp.frame.max_header_length = 1024 + + stomp.frame.max_body_length = 8192 + + stomp.listener = 61613 + + stomp.listener.acceptors = 4 + + stomp.listener.max_clients = 512 Load Stomp Plugin ----------------- .. code-block:: bash - ./bin/emqttd_ctl plugins load emqttd_stomp + ./bin/emqttd_ctl plugins load emq_stomp ------------------------------------ -emqttd_sockjs - STOMP/SockJS Plugin ------------------------------------ +-------------------------------- +emq_sockjs - STOMP/SockJS Plugin +-------------------------------- -emqttd_sockjs plugin enables web browser to connect to emqttd broker and communicate with MQTT clients. +emq_sockjs plugin enables web browser to connect to emqttd broker and communicate with MQTT clients. + +.. WARNING:: The plugin is deprecated in 2.0 + +Configure SockJS Plugin +----------------------- .. NOTE:: Default TCP Port: 61616 -Configure emqttd_sockjs ------------------------ - .. code-block:: erlang - {sockjs, []}. + [ + {emq_sockjs, [ - {cowboy_listener, {stomp_sockjs, 61616, 4}}. + {sockjs, []}, - %% TODO: unused... - {stomp, [ - {frame, [ - {max_headers, 10}, - {max_header_length, 1024}, - {max_body_length, 8192} + {cowboy_listener, {stomp_sockjs, 61616, 4}}, + + %% TODO: unused... + {stomp, [ + {frame, [ + {max_headers, 10}, + {max_header_length, 1024}, + {max_body_length, 8192} + ]} ]} - ]}. + ]} + ]. + Load SockJS Plugin ------------------ -.. NOTE:: emqttd_stomp Plugin required. - .. code-block:: bash - ./bin/emqttd_ctl plugins load emqttd_stomp - ./bin/emqttd_ctl plugins load emqttd_sockjs SockJS Demo Page @@ -677,18 +787,18 @@ SockJS Demo Page http://localhost:61616/index.html ---------------------------- -emqttd_recon - Recon Plugin ---------------------------- +------------------------ +emq_recon - Recon Plugin +------------------------ -The plugin loads `recon`_ library on a running emqttd broker. Recon libray helps debug and optimize an Erlang application. +The plugin loads `recon`_ library on a running *EMQ* broker. Recon libray helps debug and optimize an Erlang application. Load Recon Plugin ----------------- .. code-block:: bash - ./bin/emqttd_ctl plugins load emqttd_recon + ./bin/emqttd_ctl plugins load emq_recon Recon CLI --------- @@ -703,20 +813,20 @@ Recon CLI recon node_stats #recon:node_stats(10, 1000) recon remote_load Mod #recon:remote_load(Mod) ---------------------------------- -emqttd_reloader - Reloader Plugin ---------------------------------- +------------------------------ +emq_reloader - Reloader Plugin +------------------------------ Erlang Module Reloader for Development .. NOTE:: Don't load the plugin in production! -Load 'Reloader' Plugin +Load `Reloader` Plugin ---------------------- .. code-block:: bash - ./bin/emqttd_ctl plugins load emqttd_reloader + ./bin/emqttd_ctl plugins load emq_reloader reload CLI ---------- @@ -744,8 +854,8 @@ Create a plugin project with erlang.mk and depends on 'emqttd' application, the PROJECT_DESCRIPTION = emqttd abc plugin PROJECT_VERSION = 1.0 - DEPS = emqttd - dep_emqttd = git https://github.com/emqtt/emqttd emq20 + BUILD_DEPS = emqttd + dep_emqttd = git https://github.com/emqtt/emqttd master COVER = true @@ -756,11 +866,11 @@ Template Plugin: https://github.com/emqtt/emqttd_plugin_template Register Auth/ACL Modules ------------------------- -emqttd_auth_demo.erl - demo authentication module: +emq_auth_demo.erl - demo authentication module: .. code-block:: erlang - -module(emqttd_auth_demo). + -module(emq_auth_demo). -behaviour(emqttd_auth_mod). @@ -777,11 +887,11 @@ emqttd_auth_demo.erl - demo authentication module: description() -> "Demo Auth Module". -emqttd_acl_demo.erl - demo ACL module: +emq_acl_demo.erl - demo ACL module: .. code-block:: erlang - -module(emqttd_acl_demo). + -module(emq_acl_demo). -include_lib("emqttd/include/emqttd.hrl"). @@ -800,13 +910,12 @@ emqttd_acl_demo.erl - demo ACL module: description() -> "ACL Module Demo". -emqttd_plugin_template_app.erl - Register the auth/ACL modules: +emq_plugin_template_app.erl - Register the auth/ACL modules: .. code-block:: erlang - ok = emqttd_access_control:register_mod(auth, emqttd_auth_demo, []), - ok = emqttd_access_control:register_mod(acl, emqttd_acl_demo, []), - + ok = emqttd_access_control:register_mod(auth, emq_auth_demo, []), + ok = emqttd_access_control:register_mod(acl, emq_acl_demo, []), Register Callbacks for Hooks ----------------------------- @@ -836,7 +945,7 @@ The plugin could register callbacks for hooks. The hooks will be run by the brok | client.disconnected | Run when a client is disconnnected | +------------------------+-----------------------------------------+ -emqttd_plugin_template.erl for example: +emq_plugin_template.erl for example: .. code-block:: erlang @@ -855,7 +964,7 @@ emqttd_plugin_template.erl for example: Register CLI Modules -------------------- -emqttd_cli_demo.erl: +emq_cli_demo.erl: .. code-block:: erlang @@ -871,28 +980,75 @@ emqttd_cli_demo.erl: cmd(_) -> ?USAGE([{"cmd arg1 arg2", "cmd demo"}]). -emqttd_plugin_template_app.erl - register the CLI module to emqttd broker: +emq_plugin_template_app.erl - register the CLI module to *EMQ* broker: .. code-block:: erlang - emqttd_ctl:register_cmd(cmd, {emqttd_cli_demo, cmd}, []). + emqttd_ctl:register_cmd(cmd, {emq_cli_demo, cmd}, []). There will be a new CLI after the plugin loaded:: ./bin/emqttd_ctl cmd arg1 arg2 -.. _emqttd_dashboard: https://github.com/emqtt/emqttd_dashboard -.. _emqttd_auth_ldap: https://github.com/emqtt/emqttd_auth_ldap -.. _emqttd_auth_http: https://github.com/emqtt/emqttd_auth_http -.. _emqttd_auth_mysql: https://github.com/emqtt/emqttd_auth_mysql -.. _emqttd_auth_pgsql: https://github.com/emqtt/emqttd_auth_pgsql -.. _emqttd_auth_redis: https://github.com/emqtt/emqttd_auth_redis -.. _emqttd_auth_mongo: https://github.com/emqtt/emqttd_auth_mongo -.. _emqttd_sn: https://github.com/emqtt/emqttd_sn -.. _emqttd_stomp: https://github.com/emqtt/emqttd_stomp -.. _emqttd_sockjs: https://github.com/emqtt/emqttd_sockjs -.. _emqttd_recon: https://github.com/emqtt/emqttd_recon -.. _emqttd_reloader: https://github.com/emqtt/emqttd_reloader -.. _emqttd_plugin_template: https://github.com/emqtt/emqttd_plugin_template -.. _recon: http://ferd.github.io/recon/ +Create Configuration File +------------------------- + +Create `etc/${plugin_name}.conf|config` file for the plugin. The *EMQ* broker supports two type of config syntax: + +1. ${plugin_name}.config with erlang syntax: + +.. code-block:: erlang + + [ + {plugin_name, [ + {key, value} + ]} + ]. + +2. ${plugin_name}.conf with a general `k = v` syntax: + +.. code-block:: properties + + plugin_name.key = value + +Build and Release the Plugin +---------------------------- + +1. clone emqttd-relx project: + +.. code-block:: bash + + git clone https://github.com/emqtt/emqttd-relx.git + +2. Add `DEPS` in Makefile: + +.. code-block:: makefile + + DEPS += plugin_name + dep_plugin_name = git url_of_plugin + +3. Add the plugin in relx.config: + +.. code-block:: erlang + + {plugin_name, load}, + +.. _emq_dashboard: https://github.com/emqtt/emqttd_dashboard +.. _emq_auth_clientid: https://github.com/emqtt/emq_auth_clientid +.. _emq_auth_username: https://github.com/emqtt/emq_auth_username +.. _emq_auth_ldap: https://github.com/emqtt/emqttd_auth_ldap +.. _emq_auth_http: https://github.com/emqtt/emqttd_auth_http +.. _emq_auth_mysql: https://github.com/emqtt/emqttd_auth_mysql +.. _emq_auth_pgsql: https://github.com/emqtt/emqttd_auth_pgsql +.. _emq_auth_redis: https://github.com/emqtt/emqttd_auth_redis +.. _emq_auth_mongo: https://github.com/emqtt/emqttd_auth_mongo +.. _emq_mod_rewrite: https://github.com/emqtt/emq_mod_rewrite +.. _emq_sn: https://github.com/emqtt/emqttd_sn +.. _emq_coap: https://github.com/emqtt/emqttd_coap +.. _emq_stomp: https://github.com/emqtt/emqttd_stomp +.. _emq_sockjs: https://github.com/emqtt/emqttd_sockjs +.. _emq_recon: https://github.com/emqtt/emqttd_recon +.. _emq_reloader: https://github.com/emqtt/emqttd_reloader +.. _emq_plugin_template: https://github.com/emqtt/emqttd_plugin_template +.. _recon: http://ferd.github.io/recon/ diff --git a/docs/source/tune.rst b/docs/source/tune.rst index 05db453c9..71a7234d6 100644 --- a/docs/source/tune.rst +++ b/docs/source/tune.rst @@ -5,7 +5,7 @@ Tuning Guide ============ -Tuning the Linux Kernel, Networking, Erlang VM and emqttd broker for one million concurrent MQTT connections. +Tuning the Linux Kernel, Networking, Erlang VM and the *EMQ* broker for one million concurrent MQTT connections. ------------------- Linux Kernel Tuning @@ -89,55 +89,36 @@ Timeout for FIN-WAIT-2 sockets:: Erlang VM Tuning ---------------- -Tuning and optimize the Erlang VM in releases/2.0/vm.args file:: +Tuning and optimize the Erlang VM in etc/emq.conf file: - ## max number of erlang processes - +P 2097152 +.. code-block:: properties + + ## Erlang Process Limit + node.process_limit = 2097152 ## Sets the maximum number of simultaneously existing ports for this system - +Q 1048576 + node.max_ports = 1048576 - ## Increase number of concurrent ports/sockets, deprecated in R17 - -env ERL_MAX_PORTS 1048576 - - -env ERTS_MAX_PORTS 1048576 - - ## Mnesia and SSL will create temporary ets tables. - -env ERL_MAX_ETS_TABLES 1024 - - ## Tweak GC to run more often - -env ERL_FULLSWEEP_AFTER 1000 - -------------- -emqttd broker -------------- +-------------- +The EMQ Broker +-------------- Tune the acceptor pool, max_clients limit and sockopts for TCP listener in etc/emqttd.config: -.. code-block:: erlang +.. code-block:: properties - {mqtt, 1883, [ - %% Size of acceptor pool - {acceptors, 64}, - - %% Maximum number of concurrent clients - {max_clients, 1000000}, - - %% Socket Access Control - {access, [{allow, all}]}, - - %% Connection Options - {connopts, [ - %% Rate Limit. Format is 'burst, rate', Unit is KB/Sec - %% {rate_limit, "100,10"} %% 100K burst, 10K rate - ]}, - ... + ## TCP Listener + mqtt.listener.tcp = 1883 + mqtt.listener.tcp.acceptors = 64 + mqtt.listener.tcp.max_clients = 1000000 -------------- Client Machine -------------- -Tune the client machine to benchmark emqttd broker:: +Tune the client machine to benchmark emqttd broker: + +.. code-block:: bash sysctl -w net.ipv4.ip_local_port_range="500 65535" sysctl -w fs.file-max=1000000