Commit Graph

310 Commits

Author SHA1 Message Date
terry-xiaoyu 20a7ed6f53 RPC batch 2019-07-29 09:35:06 +08:00
terry-xiaoyu b0e2b7db0c Support inter-node messages via RPC cast 2019-07-27 12:59:07 +08:00
terry-xiaoyu b61d72cdcd Add options for gen_rpc 2019-07-26 11:22:25 +08:00
turtled 01f8ce1dc7 Add default k8s.suffix 2019-07-19 18:03:53 +08:00
turtled caf6dab808 Support K8S hostname auto discovery cluster 2019-07-19 18:03:53 +08:00
terry-xiaoyu 86333802b9 Add configuration for sys_heartbeat 2019-07-19 17:33:24 +08:00
terry-xiaoyu 96fff39535 Change default logger level to warning 2019-06-28 21:34:21 +08:00
Gilbert 9df6345a6c Remove bridge functions from emqx (#2652)
* Remove bridge functions from emqx
2019-06-26 19:33:17 +08:00
terry-xiaoyu c4de0b1792 No chars limit by default 2019-06-13 13:53:59 +08:00
Gilbert 0857b976ed Better zone (#2548) 2019-05-25 22:29:23 +08:00
turtled 54f2a768a8 Rm start logger:set_primary_config 2019-05-23 10:51:21 +08:00
tigercl 7e0ac5bd34
Fix bad config type of long_gc (#2504) 2019-05-08 10:19:49 +08:00
terry-xiaoyu 7b356e80f4 Rename varibles 2019-05-07 17:27:06 +08:00
terry-xiaoyu a1eb7ca7d2 Adapt max-heap-size to Arch-32 systems
The erlang:process_flag(max_heap_size, MaxHeapSize) can only set a MaxHeapSize of `small integer`, which is smaller than 2^59 on 64-bit systems, and 2^27 on 32-bit systems.
2019-05-07 17:27:06 +08:00
terry-xiaoyu 0ea6697a57 Enlarge chars_limit to 8192 2019-05-05 21:28:59 +08:00
turtled 6dd09ef681 Optimize log configuration 2019-05-05 14:35:43 +08:00
Gilbert eaa1deab20 Refactor flapping code (#2476)
* Refactor flapping code
2019-04-24 10:42:26 +08:00
terry-xiaoyu a8d2497480 Truncate logs that is too long 2019-04-19 16:49:17 +08:00
Gilbert Wong bcbb4b68e9 Add flapping detect feature 2019-04-19 16:34:33 +08:00
spring2maz 8821cfcfc7 Fix typo round_robbin -> round_robin in emqx.schema 2019-04-17 10:24:48 +08:00
YoukiLin fee94525db Add etcd client certificate configuration (#2367)
* Add etcd client certificate
2019-03-27 13:53:40 +08:00
Gilbert 2534b8dc64 Support to pass ws compressing options (#2356)
Add new config entries about websocket
2019-03-27 10:19:35 +08:00
spring2maz a72a914fb8 Fix issue #2312 (#2313)
* Fix SSL option parser

* Fix bridge disconnect log formatting

* Set no headers if new headers is undefined
2019-03-16 19:28:57 +08:00
Shawn 111b66121c
Support TLS/DTLS PSK (#2297)
* Improve filter functions for emqx-hook

* Add PSK hook

* Reset hook args for filter functions
2019-03-13 22:24:19 +08:00
spring2maz 48450d1d37 Move request response out of emqx client (#2293)
* Delete response_topic_prefix

The Response-Infomation in CONNACK was misinterpreated, it should
not be a broker's global config, instead, it should be per-client
or even per-session prefix/suffix mechanism

* Delete request-response code from emqx_client

* Add request-response test code

* meck as default dependency --- temp workaround for dep-vsn-check
2019-03-13 11:27:25 +08:00
Gilbert Wong 076f3e3068 Fix schema bug 2019-03-13 10:14:31 +08:00
Gilbert 93162c52b6
Fix conf parse bug (#2296)
Prior to this change, there are some config entry which is not parsed
correctly. So if user modify config such as reconnect_interval for
bridge, the new config can not be taken into effect.
2019-03-11 16:31:21 +08:00
tigercl da755b88c7 Add monitors and alarm handler (#2266)
* Add monitors and alarm handler
2019-02-28 18:25:17 +08:00
Gilbert Wong 7efd7b3ec0 Rename portal to bridge 2019-02-28 15:31:54 +08:00
Gilbert Wong d18f4ba550 Fix wrong config entries 2019-02-28 11:18:12 +08:00
Gilbert 796fc3b1ba Fix app config generation (#2245) 2019-02-28 11:17:27 +08:00
spring2maz 086a1d56b9 Drop unused config schema
bridge.$name.transport was added before we decided to derive
transport portocol based on the 'address' config.
i.e. when it's a remote erlang node, use gen_rpc
otherwise (must be IP or hostnmae), we should estabilish mqtt connection
2019-02-28 11:17:27 +08:00
spring2maz fbe67e6784 Introduce new bridge impl 2019-02-28 11:17:27 +08:00
tigercl a0fd9e63e0 Add data_dir to emqx.conf (#2271)
* Add data_dir to emqx.conf
2019-02-28 09:28:37 +08:00
张奇怪 44d3eff094 Auto-pull-request-by-2019-02-19 (#2232)
* Update Copyright to 2019 (#2191)

* Disable bridges by default (#2189)

* Fix warning logger args emqx#2195

* Ambiguity elimination (#2217)

* Fix emqx_ws_connection: prevent crashes on pong (#2210)

* Issue#2184 (#2194)

* Delete dep-vsn-check

* Format app.src
2019-02-19 14:00:34 +08:00
terry-xiaoyu de65e21270 Disable lager
This PR removes lager handlers and the `crash.log`, in case of some dependencies or plugins of emqx use lager.
2019-01-09 12:21:20 +08:00
Frank 981d785c06 Fix data type of mountpoint from string to binary 2019-01-09 10:31:38 +08:00
turtleDeng 6a1ebe299a
Merge emqx32 to emqx30 (#2112) 2018-12-28 19:44:41 +08:00
turtled 7d3357e0f3 Merge emqx32 2018-12-21 16:01:32 +08:00
Gilbert 7d9e350bbe Add option to disconnect client in case acl deny (#2059)
* Add option to disconnect client in case acl deny
2018-12-19 10:34:06 +08:00
Shawn c7fa4b1b15 Revert vm args in emqx conf b (#2070)
* Revert changes in emqx.conf for backward compatibility
2018-12-18 16:50:47 +08:00
Feng Lee 721b72b96a Add 'active_n' option to optimize the CPU usage of emqx_connection (#2060)
* Add 'active_n' option to optimize the CPU usage of emqx_connection

* Supports batch processing 'DOWN' events
2018-12-17 19:53:29 +08:00
terry-xiaoyu 68a6a88eb9 Move all vm args into separate file vm.args 2018-12-14 09:53:13 +08:00
terry-xiaoyu d445c17e6c Move some vm args to file vm.args 2018-12-14 09:53:13 +08:00
Feng Lee abe9aff062 Add 'enable_session_registry' config 2018-12-13 18:28:32 +08:00
周子博 ea62b15c87 Alter apis provided by emqx_metrics, and use existing timer to commit metrics 2018-11-30 17:44:29 +08:00
周子博 194dbc02c8 Add batch commit for metrics 2018-11-30 17:44:29 +08:00
Shawn 76d5dedb3b
Remove simple log handler at startup (#2000)
The simple logger handler is not removed if we set the `log.to=file` in
emqx.conf.

This might be an issue of OTP logger:
https://bugs.erlang.org/browse/ERL-788

I set the error_logger to silent as a workaround.
2018-11-30 14:29:34 +08:00
Shawn 6fa183f847 Add CLI for log level (#1977) 2018-11-23 11:04:33 +08:00
tigercl 14b8036576 Use username replace id(issue#1737) (#1961)
* Use username replace id(issue#1737)

* Add test case for issue#1737

* Make with_connection/2 support batch connect
2018-11-21 22:51:33 +08:00
spring2maz a2c658ba19 Add acking mechamism for shared dispatch (#1872)
* Add acking mechamism for shared dispatch

For QoS0 messages, no acking
For QoS1/2 messages, 'ACK' at any of events below:
 - ACK when QoS is downgraded to 0
 - Message is sent to connection process
'NACK' at any of events below:
 - Message queue is full and the receiving session starts to drop old messages
 - The receiving session crash
Upon 'NACK', messages are dispatched to the 'next' subscriber in the group,
depending on the shared subscription dispatch strategy.
2018-11-21 22:49:45 +08:00
Gilbert 16821490ce Fix issue#1874 (#1964)
* Fix issue#1874
Prior to this change, if user use one client connect emqx with mqtt
v3.1.1, the client subscribe the topic and publish message to this
topic, it would receive this message itself published, this commit
provide a configure option to let user ignore the message itself published.

This change fix issue 1874.

* Small Fix

* Fix bug

* Better design

* Fix compile warning and improve coverage

* Better design to solve the performance issue

* Fix typo

* Fix typo

* Delete spaces in end of lines.

* Do not use anonymous function

* Better performance
2018-11-19 13:34:03 +08:00
terry-xiaoyu 2269967f1a Lazy evaluation when logging messages
Formatting variables and then passing them into the logger functions leads
to performance issues. i.e.

```erlang
logger:debug("RECV ~s", [emqx_packet:format(Packet)])
```

Above message will only be printed when the current log level set to
`debug`, but the function emqx_packet:format/1 will always be evaluated no
matter what the current log level is.

OTP 21 provides a special meta-data named `report_cb`, which can be used
for lazy evaluation. The fun is only evaluated if the primary/handler log level
check passes, and is therefore recommended if it is expensive to generate
the message.
2018-11-19 11:17:58 +08:00
turtled faeda253e1 Fix conflicts 2018-11-10 11:44:55 +08:00
terry-xiaoyu 997958aed1 Change the start_link API for emqx_client
Prior to this change, emqx_client:start_link does 2 works in one call:
- init an erlang process for emqx_client
- send MQTT CONNECT to remote broker

But this solution have some drawbacks:

- the return value of `start_link` compiles the return values of the 2
 works: {ok, Pid, MqttResult}. It is inconsistent with the return value
 of `gen_statem:start_link`, may causes confusions.

- the return mode of the 2 works are different:
  `start_link` should always return {ok, Pid} or {error, Reason}, but
 connecting to mqtt may throw out exceptions as it handles the
 socket. But the caller couldn't have thought of the exception, he would
 pattern match on the result of `emqx_client:start_link`, but it crashed!

- If the init work succeed but the connection failed, the caller couldn't
get a Pid from the return value, but indeed it was created inside the
emqx_client. This hides the fact that the Pid was created, and when the
Pid dies, the caller would receive an message from a Pid it doesn' know about.

This change divived these 2 work into 2 APIs:
- `start_link/1` is to build and verify the options, and returns {ok,Pid}
 (on success) or {error, Reason} (on failure).
- `connect/1` is to send MQTT CONNECT, and returns {ok, MQTTResult::properties()} or
 {error, MQTTReason}. MQTT reason codes will contains in the `MQTTReason`.
2018-11-09 17:43:04 +08:00
terry-xiaoyu 2dc8ec8b11 Bridge via TLS 2018-11-09 17:43:04 +08:00
terry-xiaoyu cca27d1a5a Change filesync_repeat_interval to no_repeat 2018-11-05 16:04:58 +08:00
terry-xiaoyu 92cc171aaf Support tracing log files for specific topics or clients 2018-11-02 18:38:07 +08:00
terry-xiaoyu 94dbdffd59 New logger formatter with meta-data 2018-11-02 18:38:07 +08:00
terry-xiaoyu 2c63aef3f6 Update schema 2018-11-02 18:32:42 +08:00
spring2maz ae743ad1f0 Rewrite emqx_mqueue.erl
Fixed bugs:

- Priority queue lack of a `len + 1` logic in `in/2`

Changed behaviors:

- Topics not found in priority table (from config) will be treated with default priority,
  instead of hasing topic name to a priority number.
- Default priority is now configurable (it was always lower than all configured priorities)
- The dropped message due to reaching `max_len` is now returned from `in/2`,
  so the queue owner (`in/2` caller) can perform autopsy on it
2018-10-27 14:21:53 +02:00
Gilbert Wong 3f761cbe6a Support use certifate as username
Prior to this change, you can just use CN or EN field from the client
certificate as username.

This change add a new option to allow user to use Certificate directly as
username.
2018-10-23 14:37:05 +08:00
Gilbert 4c40f75f4b Request & Response (broker and client) (#1819)
Add request & response support for CONNECT & CONNACK

Prior to this change, there is no validate and specified process for
Request-Response-Information and Response-Information

Also added basic Request/Response functionality to emqx_client implementation
2018-10-18 19:21:05 +02:00
spring2maz e3f2ae8db8 Change from customized total heap size check to set process flag
The `max_heap_size` process flag can be used to limit total
heap size of a process, and it gives much more detailed
crash log if the limit is hit.
2018-09-29 21:36:34 +08:00
HuangDan 1bc175e0ce Add mountpoint config to zone configs 2018-09-28 18:53:52 +08:00
turtleDeng 925e98a3e5
Merge pull request #1823 from emqx/add-shared-sub-strategy
Add new shared subscription dispatch strategy
2018-09-22 19:30:08 +08:00
spring2maz b35d37c92d Add new shared subscription dispatch strategy
'random' was already there before this change
Added two new strategies: 'sticky' and 'round_robin'
'sticky' is made default as it is the cheapest
2018-09-22 10:50:34 +02:00
Feng Lee 5689ad3485 Fix issue #1847.
Change the config of 'zone.$name.publish_limit'
2018-09-22 16:26:39 +08:00
turtleDeng 8f35d13e17 Improve bridges design (#1849)
Improve the design of bridges
2018-09-22 16:19:28 +08:00
tigercl 03f607c1b2 Fix issue#1833 and #1834 (#1845) 2018-09-22 16:10:24 +08:00
turtleDeng 8653732bae Revert "Calculate the 1.5 keep alive time exactly" 2018-09-22 14:52:31 +08:00
Feng Lee 02ddcf37cd
Merge branch 'emqx30' into more-gc-enforcement-policies 2018-09-22 06:35:11 +08:00
Feng Lee 3dfe4168cb
Change '0KB' to '0MB' 2018-09-22 06:24:41 +08:00
Feng Lee 05a5ad0f8c
Use '0MB' to configure size of force_shutdown_policy 2018-09-22 06:14:43 +08:00
Feng Lee 29787d8945
Use '0MB' to configure bytes of force_gc_policy 2018-09-22 06:08:53 +08:00
Feng Lee 33f9e895ad
Merge branch 'emqx30' into more-gc-enforcement-policies 2018-09-22 05:52:41 +08:00
Feng Lee fbac9ce43e Improve the foce_gc_policy config. 2018-09-22 05:42:38 +08:00
spring2maz b61615323b Move shutdown policy config to zone configs 2018-09-21 18:58:18 +02:00
spring2maz 6fca651a84 Add connection/session shutdown policy
The hibernation behaviour is also changed (implicitly) in this commit:
Prior to this change, connection/session always hibernates after
the stats timer expires regardless of messages in mailbox.
After this commit, connection/session process only goes to hibernate
when the timer expires AND there is nothing left in the mailbox to
process
2018-09-21 18:58:18 +02:00
spring2maz f9f09f66dd Add new shutdown-policy config schemas 2018-09-21 18:58:18 +02:00
spring2maz 721f237bc4 Rewrite emqx_gc.erl
The implementation prior to this commit supports
only one gc enforcement policy which is message count threshold.
The new implementation introduces 1 more: volume threshold based
2018-09-21 22:56:34 +08:00
周子博 073bf481c9 Calculate the 1.5 keep alive time exactly 2018-09-20 15:51:28 +08:00
spring2maz e940c1c970 Rewrite emqx_gc.erl
The implementation prior to this commit supports
only one gc enforcement policy which is message count threshold.
The new implementation introduces 1 more: volume threshold based
2018-09-19 21:08:16 +02:00
周子博 5eb92e37cc Remove check for MQTT path, and normalize code 2018-09-14 11:49:17 +08:00
周子博 6f536eaac4 Add customized mqtt path for websocket 2018-09-14 11:49:17 +08:00
周子博 88b3460715 Add feature for issue#1809 2018-09-14 11:49:17 +08:00
周子博 c7928235c3 Remove check for MQTT path, and normalize code 2018-09-14 11:29:38 +08:00
周子博 e33414aca1 Add customized mqtt path for websocket 2018-09-14 10:10:32 +08:00
周子博 d29069a50d Add feature for issue#1809 2018-09-12 15:17:18 +08:00
Feng Lee 3045ec10ab Add banned feature 2018-08-31 14:04:26 +08:00
Feng Lee e6bed24bb3 Add server_keepalive config 2018-08-31 00:32:56 +08:00
Feng Lee bbb58dad68 Change the default value of 'zone.external.await_rel_timeout' to 300s 2018-08-30 21:02:14 +08:00
Feng Lee 7b58718280 Add mqueue_type, mqueue_priorities options 2018-08-30 18:26:31 +08:00
HuangDan 18eee0f1b0 Fixed Conflicting files 2018-08-30 10:53:35 +08:00
HuangDan 53d7d0a9d4 Update the peer_cert_as_username conf desc 2018-08-30 10:35:20 +08:00
Feng Lee 465963fec3
Merge branch 'emqx30' into emqx30-feng 2018-08-29 17:36:38 +08:00
Feng Lee 8e11845f87 Remove ignore_loop_deliver option 2018-08-28 07:36:22 +08:00
turtled 2e4fb3f946 emqttd -> emqx 2018-08-27 13:58:08 +08:00
turtled 91eb79967c Add syslog 2018-08-27 13:50:38 +08:00
terry-xiaoyu 9d29dd0e10 use config enable_acl_cache 2018-08-26 18:25:17 +08:00
terry-xiaoyu 8cd20744be improve cache datastruct using keys-queue 2018-08-26 13:33:00 +08:00
terry-xiaoyu a904031979 acl cache using proc_dict 2018-08-26 13:30:18 +08:00
Feng Lee 68cfcf6e0e Pass paho mqtt interoperability tests 2018-08-24 23:19:11 +08:00
Feng Lee 694485252a Merge branch 'emqx30-dev' of github.com:emqtt/emqttd into emqx30-dev 2018-08-24 18:43:36 +08:00
Feng Lee 0f052ce352 Upgrade connection, protocol and session modules for MQTT 5.0 2018-08-24 18:39:59 +08:00
turtled 22e8b07a3d Receive/send messages by bridge 2018-08-19 20:31:44 +08:00
Feng Lee f80cd2d986 Improve the MQTT over Websocket connection 2018-08-11 17:57:19 +08:00
Feng Lee bc8302dae9 Change default value of max_mqueue_len to 1000 2018-08-10 15:39:24 +08:00
Feng Lee d9004d4cfb Add MQTT section in configuration file 2018-08-10 12:43:32 +08:00
Feng Lee 3ac4be84e4 Remove 'listener.wss.external.handshake_timeout' for cowboy does not support this option 2018-08-09 14:26:39 +08:00
Feng Lee 854132d0c3 Add max_conn_rate, handshake_timeout options for wss listeners 2018-08-09 09:13:30 +08:00
Feng Lee e23a6e87d8 Merge listeners, zones, bridges config 2018-08-08 19:37:57 +08:00
Feng Lee 4cf1815030 Add more configurations for EMQ X R3.0 2018-08-08 19:23:32 +08:00
turtled 4d9e03a803 Refactor websocket conn using cowboy 2018-08-08 18:36:14 +08:00
Feng Lee 7d0cba9427 Add MQTT 5.0 supports for connection, protocol and session modules 2018-08-06 16:33:10 +08:00
Feng Lee cbbc231210 Comment log.syslog.* to fix building issue 2018-07-19 10:08:37 +08:00
Feng Lee 7ee54aac28 Add 'qos' field to message record 2018-07-18 23:41:00 +08:00
Feng Lee b3adcc8971 Add RPC Args 2018-07-18 14:24:35 +08:00
Feng Lee c79fa29587 Fix the schema of lager console backend 2018-03-30 17:17:18 +08:00
Feng Lee f4fd6efe16 Merge the emqx-common, emqx-router libraries 2018-02-26 23:29:53 +08:00
Feng Lee 057ef7b668 Merge branch 'master' 2018-02-26 01:05:08 +08:00
Feng Lee fcb2ec8427 Merge the latest enterprise branch 2018-02-24 15:56:32 +08:00
Feng Lee b63c34ab14 Merge the latest master branch 2018-02-24 11:27:09 +08:00
Feng Lee 2674c145a9 Fix the 'inet_ssl' distribution module 2018-02-16 20:40:44 +08:00
Feng Lee 79902be40e Fix issue #1492 - add 'node.ssl_dist_optfile' to support ssl distributation 2018-02-16 18:40:03 +08:00
Feng Lee bd0409879b Add 'proxy_port_address', 'proxy_port_header' options for WebSocket listener 2018-01-28 15:26:56 +08:00
Feng Lee 707d338b0e
Merge branch 'develop' into ws-proxy 2018-01-28 15:11:08 +08:00
Feng Lee f70bf23440 Add 'listener.<proto>.<name>.reuseaddr' option 2018-01-24 09:48:06 +08:00
Feng Lee 87ae76b6b4 Fix issue #1460 - Add node.proto_dist option to support inet6_dist 2018-01-23 15:22:05 +08:00
Feng Lee a1cbdc5122 Update emq.conf and emq.schema 2018-01-12 10:46:35 +08:00
Feng Lee 3a39706d84 Add more options for 'listener.wss.' 2018-01-04 16:05:44 +08:00
Feng Lee b2b78c178c Add documenation for SSL configurations 2018-01-04 12:25:05 +08:00
Feng Lee bcec25092b Merge version 2.3.2 to X edition 2017-12-26 21:09:24 +08:00
Feng Lee d5222dcc9b Add proxy_protocol, proxy_protocol_timeout options for ws/wss 2017-12-25 20:38:09 +08:00
Feng Lee db954aeb6d Support X.509 certificate based authentication with HAProxy 2017-12-15 19:23:50 +08:00
Feng Lee f1640f5b85 Merge emq24 to X branch 2017-12-07 11:02:44 +08:00
Feng Lee 8e41aeeeb8 Add send_timeout, send_timeout_close options 2017-12-05 19:47:17 +08:00
Feng Lee bceb72853d Fix issue #1335 - Forward real client IP using a reverse proxy for websocket 2017-12-02 17:59:29 +08:00
HeeeJianBo 5006dbba6e Add ws/wss proxy cofingurations for getting client original ip address 2017-12-01 22:12:27 +08:00
Feng Lee aaf19787af Remove the duplicated InfoHandler 2017-11-22 18:11:21 +08:00
Feng Lee 99ed0e46b6 Remove RPC args 2017-11-22 17:07:11 +08:00
Feng Lee 2cec86eba7 Merge with version 2.3.0 2017-11-21 14:14:12 +08:00
Feng Lee c751e958db Remove the translation for 'by_clientid' 2017-11-18 10:37:48 +08:00
Feng Lee 1545e18cd9 Remove the 'mqtt.pubsub.by_clientid' option 2017-11-17 20:55:11 +08:00
HuangDan 97cf04d752 Add more lager configuration 2017-10-11 16:01:39 +08:00
turtled 989d2fd9e7 Add more lager configuration 2017-10-10 13:10:34 +08:00
Feng Lee 4b8cd18f5d Merge with EMQ X project 2017-10-09 19:43:06 +08:00
turtled d3f52898ee Add Hot configuration plugin 2017-09-05 16:52:07 +08:00
Feng Lee 88f84a4a0c Support to configure keepalive backoff 2017-08-07 18:27:16 +08:00
Feng Lee 09b4f17065 Merge the configuration of ekka 2017-07-23 18:42:54 +08:00