134 lines
3.2 KiB
ReStructuredText
134 lines
3.2 KiB
ReStructuredText
|
|
.. _tune:
|
|
|
|
============
|
|
Tuning Guide
|
|
============
|
|
|
|
Tuning the Linux Kernel, Networking, Erlang VM and the *EMQ* broker for one million concurrent MQTT connections.
|
|
|
|
-------------------
|
|
Linux Kernel Tuning
|
|
-------------------
|
|
|
|
The system-wide limit on max opened file handles::
|
|
|
|
# 2 million system-wide
|
|
sysctl -w fs.file-max=2097152
|
|
sysctl -w fs.nr_open=2097152
|
|
echo 2097152 > /proc/sys/fs/nr_open
|
|
|
|
The limit on opened file handles for current session::
|
|
|
|
ulimit -n 1048576
|
|
|
|
/etc/sysctl.conf
|
|
----------------
|
|
|
|
Add the 'fs.file-max' to /etc/sysctl.conf, make the changes permanent::
|
|
|
|
fs.file-max = 1048576
|
|
|
|
/etc/security/limits.conf
|
|
-------------------------
|
|
|
|
Persist the limits on opened file handles for users in /etc/security/limits.conf::
|
|
|
|
* soft nofile 1048576
|
|
* hard nofile 1048576
|
|
|
|
--------------
|
|
Network Tuning
|
|
--------------
|
|
|
|
Increase number of incoming connections backlog::
|
|
|
|
sysctl -w net.core.somaxconn=32768
|
|
sysctl -w net.ipv4.tcp_max_syn_backlog=16384
|
|
sysctl -w net.core.netdev_max_backlog=16384
|
|
|
|
Local Port Range::
|
|
|
|
sysctl -w net.ipv4.ip_local_port_range="1000 65535"
|
|
|
|
Read/Write Buffer for TCP connections::
|
|
|
|
sysctl -w net.core.rmem_default=262144
|
|
sysctl -w net.core.wmem_default=262144
|
|
sysctl -w net.core.rmem_max=16777216
|
|
sysctl -w net.core.wmem_max=16777216
|
|
sysctl -w net.core.optmem_max=16777216
|
|
|
|
#sysctl -w net.ipv4.tcp_mem='16777216 16777216 16777216'
|
|
sysctl -w net.ipv4.tcp_rmem='1024 4096 16777216'
|
|
sysctl -w net.ipv4.tcp_wmem='1024 4096 16777216'
|
|
|
|
Connection Tracking::
|
|
|
|
sysctl -w net.nf_conntrack_max=1000000
|
|
sysctl -w net.netfilter.nf_conntrack_max=1000000
|
|
sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=30
|
|
|
|
The TIME-WAIT Buckets Pool, Recycling and Reuse::
|
|
|
|
net.ipv4.tcp_max_tw_buckets=1048576
|
|
|
|
# Enable fast recycling of TIME_WAIT sockets. Enabling this
|
|
# option is not recommended for devices communicating with the
|
|
# general Internet or using NAT (Network Address Translation).
|
|
# Since some NAT gateways pass through IP timestamp values, one
|
|
# IP can appear to have non-increasing timestamps.
|
|
# net.ipv4.tcp_tw_recycle = 1
|
|
# net.ipv4.tcp_tw_reuse = 1
|
|
|
|
Timeout for FIN-WAIT-2 sockets::
|
|
|
|
net.ipv4.tcp_fin_timeout = 15
|
|
|
|
----------------
|
|
Erlang VM Tuning
|
|
----------------
|
|
|
|
Tuning and optimize the Erlang VM in etc/emq.conf file:
|
|
|
|
.. code-block:: properties
|
|
|
|
## Erlang Process Limit
|
|
node.process_limit = 2097152
|
|
|
|
## Sets the maximum number of simultaneously existing ports for this system
|
|
node.max_ports = 1048576
|
|
|
|
--------------
|
|
The EMQ Broker
|
|
--------------
|
|
|
|
Tune the acceptor pool, max_clients limit and sockopts for TCP listener in etc/emqttd.config:
|
|
|
|
.. code-block:: properties
|
|
|
|
## TCP Listener
|
|
mqtt.listener.tcp = 1883
|
|
mqtt.listener.tcp.acceptors = 64
|
|
mqtt.listener.tcp.max_clients = 1000000
|
|
|
|
--------------
|
|
Client Machine
|
|
--------------
|
|
|
|
Tune the client machine to benchmark emqttd broker:
|
|
|
|
.. code-block:: bash
|
|
|
|
sysctl -w net.ipv4.ip_local_port_range="500 65535"
|
|
sysctl -w fs.file-max=1000000
|
|
echo 1000000 > /proc/sys/fs/nr_open
|
|
ulimit -n 100000
|
|
|
|
---------------
|
|
emqtt_benchmark
|
|
---------------
|
|
|
|
Test tool for concurrent connections: http://github.com/emqtt/emqtt_benchmark
|
|
|