Commit Graph

227 Commits

Author SHA1 Message Date
terry-xiaoyu 2269967f1a Lazy evaluation when logging messages
Formatting variables and then passing them into the logger functions leads
to performance issues. i.e.

```erlang
logger:debug("RECV ~s", [emqx_packet:format(Packet)])
```

Above message will only be printed when the current log level set to
`debug`, but the function emqx_packet:format/1 will always be evaluated no
matter what the current log level is.

OTP 21 provides a special meta-data named `report_cb`, which can be used
for lazy evaluation. The fun is only evaluated if the primary/handler log level
check passes, and is therefore recommended if it is expensive to generate
the message.
2018-11-19 11:17:58 +08:00
turtled faeda253e1 Fix conflicts 2018-11-10 11:44:55 +08:00
terry-xiaoyu 997958aed1 Change the start_link API for emqx_client
Prior to this change, emqx_client:start_link does 2 works in one call:
- init an erlang process for emqx_client
- send MQTT CONNECT to remote broker

But this solution have some drawbacks:

- the return value of `start_link` compiles the return values of the 2
 works: {ok, Pid, MqttResult}. It is inconsistent with the return value
 of `gen_statem:start_link`, may causes confusions.

- the return mode of the 2 works are different:
  `start_link` should always return {ok, Pid} or {error, Reason}, but
 connecting to mqtt may throw out exceptions as it handles the
 socket. But the caller couldn't have thought of the exception, he would
 pattern match on the result of `emqx_client:start_link`, but it crashed!

- If the init work succeed but the connection failed, the caller couldn't
get a Pid from the return value, but indeed it was created inside the
emqx_client. This hides the fact that the Pid was created, and when the
Pid dies, the caller would receive an message from a Pid it doesn' know about.

This change divived these 2 work into 2 APIs:
- `start_link/1` is to build and verify the options, and returns {ok,Pid}
 (on success) or {error, Reason} (on failure).
- `connect/1` is to send MQTT CONNECT, and returns {ok, MQTTResult::properties()} or
 {error, MQTTReason}. MQTT reason codes will contains in the `MQTTReason`.
2018-11-09 17:43:04 +08:00
terry-xiaoyu 2dc8ec8b11 Bridge via TLS 2018-11-09 17:43:04 +08:00
terry-xiaoyu cca27d1a5a Change filesync_repeat_interval to no_repeat 2018-11-05 16:04:58 +08:00
terry-xiaoyu 92cc171aaf Support tracing log files for specific topics or clients 2018-11-02 18:38:07 +08:00
terry-xiaoyu 94dbdffd59 New logger formatter with meta-data 2018-11-02 18:38:07 +08:00
terry-xiaoyu 2c63aef3f6 Update schema 2018-11-02 18:32:42 +08:00
spring2maz ae743ad1f0 Rewrite emqx_mqueue.erl
Fixed bugs:

- Priority queue lack of a `len + 1` logic in `in/2`

Changed behaviors:

- Topics not found in priority table (from config) will be treated with default priority,
  instead of hasing topic name to a priority number.
- Default priority is now configurable (it was always lower than all configured priorities)
- The dropped message due to reaching `max_len` is now returned from `in/2`,
  so the queue owner (`in/2` caller) can perform autopsy on it
2018-10-27 14:21:53 +02:00
Gilbert Wong 3f761cbe6a Support use certifate as username
Prior to this change, you can just use CN or EN field from the client
certificate as username.

This change add a new option to allow user to use Certificate directly as
username.
2018-10-23 14:37:05 +08:00
Gilbert 4c40f75f4b Request & Response (broker and client) (#1819)
Add request & response support for CONNECT & CONNACK

Prior to this change, there is no validate and specified process for
Request-Response-Information and Response-Information

Also added basic Request/Response functionality to emqx_client implementation
2018-10-18 19:21:05 +02:00
spring2maz e3f2ae8db8 Change from customized total heap size check to set process flag
The `max_heap_size` process flag can be used to limit total
heap size of a process, and it gives much more detailed
crash log if the limit is hit.
2018-09-29 21:36:34 +08:00
HuangDan 1bc175e0ce Add mountpoint config to zone configs 2018-09-28 18:53:52 +08:00
turtleDeng 925e98a3e5
Merge pull request #1823 from emqx/add-shared-sub-strategy
Add new shared subscription dispatch strategy
2018-09-22 19:30:08 +08:00
spring2maz b35d37c92d Add new shared subscription dispatch strategy
'random' was already there before this change
Added two new strategies: 'sticky' and 'round_robin'
'sticky' is made default as it is the cheapest
2018-09-22 10:50:34 +02:00
Feng Lee 5689ad3485 Fix issue #1847.
Change the config of 'zone.$name.publish_limit'
2018-09-22 16:26:39 +08:00
turtleDeng 8f35d13e17 Improve bridges design (#1849)
Improve the design of bridges
2018-09-22 16:19:28 +08:00
tigercl 03f607c1b2 Fix issue#1833 and #1834 (#1845) 2018-09-22 16:10:24 +08:00
turtleDeng 8653732bae Revert "Calculate the 1.5 keep alive time exactly" 2018-09-22 14:52:31 +08:00
Feng Lee 02ddcf37cd
Merge branch 'emqx30' into more-gc-enforcement-policies 2018-09-22 06:35:11 +08:00
Feng Lee 3dfe4168cb
Change '0KB' to '0MB' 2018-09-22 06:24:41 +08:00
Feng Lee 05a5ad0f8c
Use '0MB' to configure size of force_shutdown_policy 2018-09-22 06:14:43 +08:00
Feng Lee 29787d8945
Use '0MB' to configure bytes of force_gc_policy 2018-09-22 06:08:53 +08:00
Feng Lee 33f9e895ad
Merge branch 'emqx30' into more-gc-enforcement-policies 2018-09-22 05:52:41 +08:00
Feng Lee fbac9ce43e Improve the foce_gc_policy config. 2018-09-22 05:42:38 +08:00
spring2maz b61615323b Move shutdown policy config to zone configs 2018-09-21 18:58:18 +02:00
spring2maz 6fca651a84 Add connection/session shutdown policy
The hibernation behaviour is also changed (implicitly) in this commit:
Prior to this change, connection/session always hibernates after
the stats timer expires regardless of messages in mailbox.
After this commit, connection/session process only goes to hibernate
when the timer expires AND there is nothing left in the mailbox to
process
2018-09-21 18:58:18 +02:00
spring2maz f9f09f66dd Add new shutdown-policy config schemas 2018-09-21 18:58:18 +02:00
spring2maz 721f237bc4 Rewrite emqx_gc.erl
The implementation prior to this commit supports
only one gc enforcement policy which is message count threshold.
The new implementation introduces 1 more: volume threshold based
2018-09-21 22:56:34 +08:00
周子博 073bf481c9 Calculate the 1.5 keep alive time exactly 2018-09-20 15:51:28 +08:00
spring2maz e940c1c970 Rewrite emqx_gc.erl
The implementation prior to this commit supports
only one gc enforcement policy which is message count threshold.
The new implementation introduces 1 more: volume threshold based
2018-09-19 21:08:16 +02:00
周子博 5eb92e37cc Remove check for MQTT path, and normalize code 2018-09-14 11:49:17 +08:00
周子博 6f536eaac4 Add customized mqtt path for websocket 2018-09-14 11:49:17 +08:00
周子博 88b3460715 Add feature for issue#1809 2018-09-14 11:49:17 +08:00
周子博 c7928235c3 Remove check for MQTT path, and normalize code 2018-09-14 11:29:38 +08:00
周子博 e33414aca1 Add customized mqtt path for websocket 2018-09-14 10:10:32 +08:00
周子博 d29069a50d Add feature for issue#1809 2018-09-12 15:17:18 +08:00
Feng Lee 3045ec10ab Add banned feature 2018-08-31 14:04:26 +08:00
Feng Lee e6bed24bb3 Add server_keepalive config 2018-08-31 00:32:56 +08:00
Feng Lee bbb58dad68 Change the default value of 'zone.external.await_rel_timeout' to 300s 2018-08-30 21:02:14 +08:00
Feng Lee 7b58718280 Add mqueue_type, mqueue_priorities options 2018-08-30 18:26:31 +08:00
HuangDan 18eee0f1b0 Fixed Conflicting files 2018-08-30 10:53:35 +08:00
HuangDan 53d7d0a9d4 Update the peer_cert_as_username conf desc 2018-08-30 10:35:20 +08:00
Feng Lee 465963fec3
Merge branch 'emqx30' into emqx30-feng 2018-08-29 17:36:38 +08:00
Feng Lee 8e11845f87 Remove ignore_loop_deliver option 2018-08-28 07:36:22 +08:00
turtled 2e4fb3f946 emqttd -> emqx 2018-08-27 13:58:08 +08:00
turtled 91eb79967c Add syslog 2018-08-27 13:50:38 +08:00
terry-xiaoyu 9d29dd0e10 use config enable_acl_cache 2018-08-26 18:25:17 +08:00
terry-xiaoyu 8cd20744be improve cache datastruct using keys-queue 2018-08-26 13:33:00 +08:00
terry-xiaoyu a904031979 acl cache using proc_dict 2018-08-26 13:30:18 +08:00
Feng Lee 68cfcf6e0e Pass paho mqtt interoperability tests 2018-08-24 23:19:11 +08:00
Feng Lee 694485252a Merge branch 'emqx30-dev' of github.com:emqtt/emqttd into emqx30-dev 2018-08-24 18:43:36 +08:00
Feng Lee 0f052ce352 Upgrade connection, protocol and session modules for MQTT 5.0 2018-08-24 18:39:59 +08:00
turtled 22e8b07a3d Receive/send messages by bridge 2018-08-19 20:31:44 +08:00
Feng Lee f80cd2d986 Improve the MQTT over Websocket connection 2018-08-11 17:57:19 +08:00
Feng Lee bc8302dae9 Change default value of max_mqueue_len to 1000 2018-08-10 15:39:24 +08:00
Feng Lee d9004d4cfb Add MQTT section in configuration file 2018-08-10 12:43:32 +08:00
Feng Lee 3ac4be84e4 Remove 'listener.wss.external.handshake_timeout' for cowboy does not support this option 2018-08-09 14:26:39 +08:00
Feng Lee 854132d0c3 Add max_conn_rate, handshake_timeout options for wss listeners 2018-08-09 09:13:30 +08:00
Feng Lee e23a6e87d8 Merge listeners, zones, bridges config 2018-08-08 19:37:57 +08:00
Feng Lee 4cf1815030 Add more configurations for EMQ X R3.0 2018-08-08 19:23:32 +08:00
turtled 4d9e03a803 Refactor websocket conn using cowboy 2018-08-08 18:36:14 +08:00
Feng Lee 7d0cba9427 Add MQTT 5.0 supports for connection, protocol and session modules 2018-08-06 16:33:10 +08:00
Feng Lee cbbc231210 Comment log.syslog.* to fix building issue 2018-07-19 10:08:37 +08:00
Feng Lee 7ee54aac28 Add 'qos' field to message record 2018-07-18 23:41:00 +08:00
Feng Lee b3adcc8971 Add RPC Args 2018-07-18 14:24:35 +08:00
Feng Lee c79fa29587 Fix the schema of lager console backend 2018-03-30 17:17:18 +08:00
Feng Lee f4fd6efe16 Merge the emqx-common, emqx-router libraries 2018-02-26 23:29:53 +08:00
Feng Lee 057ef7b668 Merge branch 'master' 2018-02-26 01:05:08 +08:00
Feng Lee fcb2ec8427 Merge the latest enterprise branch 2018-02-24 15:56:32 +08:00
Feng Lee b63c34ab14 Merge the latest master branch 2018-02-24 11:27:09 +08:00
Feng Lee bcec25092b Merge version 2.3.2 to X edition 2017-12-26 21:09:24 +08:00
Feng Lee f1640f5b85 Merge emq24 to X branch 2017-12-07 11:02:44 +08:00
Feng Lee aaf19787af Remove the duplicated InfoHandler 2017-11-22 18:11:21 +08:00
Feng Lee 99ed0e46b6 Remove RPC args 2017-11-22 17:07:11 +08:00
Feng Lee 2cec86eba7 Merge with version 2.3.0 2017-11-21 14:14:12 +08:00
Feng Lee 4b8cd18f5d Merge with EMQ X project 2017-10-09 19:43:06 +08:00