Commit Graph

229 Commits

Author SHA1 Message Date
spring2maz a2c658ba19 Add acking mechamism for shared dispatch (#1872)
* Add acking mechamism for shared dispatch

For QoS0 messages, no acking
For QoS1/2 messages, 'ACK' at any of events below:
 - ACK when QoS is downgraded to 0
 - Message is sent to connection process
'NACK' at any of events below:
 - Message queue is full and the receiving session starts to drop old messages
 - The receiving session crash
Upon 'NACK', messages are dispatched to the 'next' subscriber in the group,
depending on the shared subscription dispatch strategy.
2018-11-21 22:49:45 +08:00
Gilbert 16821490ce Fix issue#1874 (#1964)
* Fix issue#1874
Prior to this change, if user use one client connect emqx with mqtt
v3.1.1, the client subscribe the topic and publish message to this
topic, it would receive this message itself published, this commit
provide a configure option to let user ignore the message itself published.

This change fix issue 1874.

* Small Fix

* Fix bug

* Better design

* Fix compile warning and improve coverage

* Better design to solve the performance issue

* Fix typo

* Fix typo

* Delete spaces in end of lines.

* Do not use anonymous function

* Better performance
2018-11-19 13:34:03 +08:00
terry-xiaoyu 2269967f1a Lazy evaluation when logging messages
Formatting variables and then passing them into the logger functions leads
to performance issues. i.e.

```erlang
logger:debug("RECV ~s", [emqx_packet:format(Packet)])
```

Above message will only be printed when the current log level set to
`debug`, but the function emqx_packet:format/1 will always be evaluated no
matter what the current log level is.

OTP 21 provides a special meta-data named `report_cb`, which can be used
for lazy evaluation. The fun is only evaluated if the primary/handler log level
check passes, and is therefore recommended if it is expensive to generate
the message.
2018-11-19 11:17:58 +08:00
turtled faeda253e1 Fix conflicts 2018-11-10 11:44:55 +08:00
terry-xiaoyu 997958aed1 Change the start_link API for emqx_client
Prior to this change, emqx_client:start_link does 2 works in one call:
- init an erlang process for emqx_client
- send MQTT CONNECT to remote broker

But this solution have some drawbacks:

- the return value of `start_link` compiles the return values of the 2
 works: {ok, Pid, MqttResult}. It is inconsistent with the return value
 of `gen_statem:start_link`, may causes confusions.

- the return mode of the 2 works are different:
  `start_link` should always return {ok, Pid} or {error, Reason}, but
 connecting to mqtt may throw out exceptions as it handles the
 socket. But the caller couldn't have thought of the exception, he would
 pattern match on the result of `emqx_client:start_link`, but it crashed!

- If the init work succeed but the connection failed, the caller couldn't
get a Pid from the return value, but indeed it was created inside the
emqx_client. This hides the fact that the Pid was created, and when the
Pid dies, the caller would receive an message from a Pid it doesn' know about.

This change divived these 2 work into 2 APIs:
- `start_link/1` is to build and verify the options, and returns {ok,Pid}
 (on success) or {error, Reason} (on failure).
- `connect/1` is to send MQTT CONNECT, and returns {ok, MQTTResult::properties()} or
 {error, MQTTReason}. MQTT reason codes will contains in the `MQTTReason`.
2018-11-09 17:43:04 +08:00
terry-xiaoyu 2dc8ec8b11 Bridge via TLS 2018-11-09 17:43:04 +08:00
terry-xiaoyu cca27d1a5a Change filesync_repeat_interval to no_repeat 2018-11-05 16:04:58 +08:00
terry-xiaoyu 92cc171aaf Support tracing log files for specific topics or clients 2018-11-02 18:38:07 +08:00
terry-xiaoyu 94dbdffd59 New logger formatter with meta-data 2018-11-02 18:38:07 +08:00
terry-xiaoyu 2c63aef3f6 Update schema 2018-11-02 18:32:42 +08:00
spring2maz ae743ad1f0 Rewrite emqx_mqueue.erl
Fixed bugs:

- Priority queue lack of a `len + 1` logic in `in/2`

Changed behaviors:

- Topics not found in priority table (from config) will be treated with default priority,
  instead of hasing topic name to a priority number.
- Default priority is now configurable (it was always lower than all configured priorities)
- The dropped message due to reaching `max_len` is now returned from `in/2`,
  so the queue owner (`in/2` caller) can perform autopsy on it
2018-10-27 14:21:53 +02:00
Gilbert Wong 3f761cbe6a Support use certifate as username
Prior to this change, you can just use CN or EN field from the client
certificate as username.

This change add a new option to allow user to use Certificate directly as
username.
2018-10-23 14:37:05 +08:00
Gilbert 4c40f75f4b Request & Response (broker and client) (#1819)
Add request & response support for CONNECT & CONNACK

Prior to this change, there is no validate and specified process for
Request-Response-Information and Response-Information

Also added basic Request/Response functionality to emqx_client implementation
2018-10-18 19:21:05 +02:00
spring2maz e3f2ae8db8 Change from customized total heap size check to set process flag
The `max_heap_size` process flag can be used to limit total
heap size of a process, and it gives much more detailed
crash log if the limit is hit.
2018-09-29 21:36:34 +08:00
HuangDan 1bc175e0ce Add mountpoint config to zone configs 2018-09-28 18:53:52 +08:00
turtleDeng 925e98a3e5
Merge pull request #1823 from emqx/add-shared-sub-strategy
Add new shared subscription dispatch strategy
2018-09-22 19:30:08 +08:00
spring2maz b35d37c92d Add new shared subscription dispatch strategy
'random' was already there before this change
Added two new strategies: 'sticky' and 'round_robin'
'sticky' is made default as it is the cheapest
2018-09-22 10:50:34 +02:00
Feng Lee 5689ad3485 Fix issue #1847.
Change the config of 'zone.$name.publish_limit'
2018-09-22 16:26:39 +08:00
turtleDeng 8f35d13e17 Improve bridges design (#1849)
Improve the design of bridges
2018-09-22 16:19:28 +08:00
tigercl 03f607c1b2 Fix issue#1833 and #1834 (#1845) 2018-09-22 16:10:24 +08:00
turtleDeng 8653732bae Revert "Calculate the 1.5 keep alive time exactly" 2018-09-22 14:52:31 +08:00
Feng Lee 02ddcf37cd
Merge branch 'emqx30' into more-gc-enforcement-policies 2018-09-22 06:35:11 +08:00
Feng Lee 3dfe4168cb
Change '0KB' to '0MB' 2018-09-22 06:24:41 +08:00
Feng Lee 05a5ad0f8c
Use '0MB' to configure size of force_shutdown_policy 2018-09-22 06:14:43 +08:00
Feng Lee 29787d8945
Use '0MB' to configure bytes of force_gc_policy 2018-09-22 06:08:53 +08:00
Feng Lee 33f9e895ad
Merge branch 'emqx30' into more-gc-enforcement-policies 2018-09-22 05:52:41 +08:00
Feng Lee fbac9ce43e Improve the foce_gc_policy config. 2018-09-22 05:42:38 +08:00
spring2maz b61615323b Move shutdown policy config to zone configs 2018-09-21 18:58:18 +02:00
spring2maz 6fca651a84 Add connection/session shutdown policy
The hibernation behaviour is also changed (implicitly) in this commit:
Prior to this change, connection/session always hibernates after
the stats timer expires regardless of messages in mailbox.
After this commit, connection/session process only goes to hibernate
when the timer expires AND there is nothing left in the mailbox to
process
2018-09-21 18:58:18 +02:00
spring2maz f9f09f66dd Add new shutdown-policy config schemas 2018-09-21 18:58:18 +02:00
spring2maz 721f237bc4 Rewrite emqx_gc.erl
The implementation prior to this commit supports
only one gc enforcement policy which is message count threshold.
The new implementation introduces 1 more: volume threshold based
2018-09-21 22:56:34 +08:00
周子博 073bf481c9 Calculate the 1.5 keep alive time exactly 2018-09-20 15:51:28 +08:00
spring2maz e940c1c970 Rewrite emqx_gc.erl
The implementation prior to this commit supports
only one gc enforcement policy which is message count threshold.
The new implementation introduces 1 more: volume threshold based
2018-09-19 21:08:16 +02:00
周子博 5eb92e37cc Remove check for MQTT path, and normalize code 2018-09-14 11:49:17 +08:00
周子博 6f536eaac4 Add customized mqtt path for websocket 2018-09-14 11:49:17 +08:00
周子博 88b3460715 Add feature for issue#1809 2018-09-14 11:49:17 +08:00
周子博 c7928235c3 Remove check for MQTT path, and normalize code 2018-09-14 11:29:38 +08:00
周子博 e33414aca1 Add customized mqtt path for websocket 2018-09-14 10:10:32 +08:00
周子博 d29069a50d Add feature for issue#1809 2018-09-12 15:17:18 +08:00
Feng Lee 3045ec10ab Add banned feature 2018-08-31 14:04:26 +08:00
Feng Lee e6bed24bb3 Add server_keepalive config 2018-08-31 00:32:56 +08:00
Feng Lee bbb58dad68 Change the default value of 'zone.external.await_rel_timeout' to 300s 2018-08-30 21:02:14 +08:00
Feng Lee 7b58718280 Add mqueue_type, mqueue_priorities options 2018-08-30 18:26:31 +08:00
HuangDan 18eee0f1b0 Fixed Conflicting files 2018-08-30 10:53:35 +08:00
HuangDan 53d7d0a9d4 Update the peer_cert_as_username conf desc 2018-08-30 10:35:20 +08:00
Feng Lee 465963fec3
Merge branch 'emqx30' into emqx30-feng 2018-08-29 17:36:38 +08:00
Feng Lee 8e11845f87 Remove ignore_loop_deliver option 2018-08-28 07:36:22 +08:00
turtled 2e4fb3f946 emqttd -> emqx 2018-08-27 13:58:08 +08:00
turtled 91eb79967c Add syslog 2018-08-27 13:50:38 +08:00
terry-xiaoyu 9d29dd0e10 use config enable_acl_cache 2018-08-26 18:25:17 +08:00
terry-xiaoyu 8cd20744be improve cache datastruct using keys-queue 2018-08-26 13:33:00 +08:00
terry-xiaoyu a904031979 acl cache using proc_dict 2018-08-26 13:30:18 +08:00
Feng Lee 68cfcf6e0e Pass paho mqtt interoperability tests 2018-08-24 23:19:11 +08:00
Feng Lee 694485252a Merge branch 'emqx30-dev' of github.com:emqtt/emqttd into emqx30-dev 2018-08-24 18:43:36 +08:00
Feng Lee 0f052ce352 Upgrade connection, protocol and session modules for MQTT 5.0 2018-08-24 18:39:59 +08:00
turtled 22e8b07a3d Receive/send messages by bridge 2018-08-19 20:31:44 +08:00
Feng Lee f80cd2d986 Improve the MQTT over Websocket connection 2018-08-11 17:57:19 +08:00
Feng Lee bc8302dae9 Change default value of max_mqueue_len to 1000 2018-08-10 15:39:24 +08:00
Feng Lee d9004d4cfb Add MQTT section in configuration file 2018-08-10 12:43:32 +08:00
Feng Lee 3ac4be84e4 Remove 'listener.wss.external.handshake_timeout' for cowboy does not support this option 2018-08-09 14:26:39 +08:00
Feng Lee 854132d0c3 Add max_conn_rate, handshake_timeout options for wss listeners 2018-08-09 09:13:30 +08:00
Feng Lee e23a6e87d8 Merge listeners, zones, bridges config 2018-08-08 19:37:57 +08:00
Feng Lee 4cf1815030 Add more configurations for EMQ X R3.0 2018-08-08 19:23:32 +08:00
turtled 4d9e03a803 Refactor websocket conn using cowboy 2018-08-08 18:36:14 +08:00
Feng Lee 7d0cba9427 Add MQTT 5.0 supports for connection, protocol and session modules 2018-08-06 16:33:10 +08:00
Feng Lee cbbc231210 Comment log.syslog.* to fix building issue 2018-07-19 10:08:37 +08:00
Feng Lee 7ee54aac28 Add 'qos' field to message record 2018-07-18 23:41:00 +08:00
Feng Lee b3adcc8971 Add RPC Args 2018-07-18 14:24:35 +08:00
Feng Lee c79fa29587 Fix the schema of lager console backend 2018-03-30 17:17:18 +08:00
Feng Lee f4fd6efe16 Merge the emqx-common, emqx-router libraries 2018-02-26 23:29:53 +08:00
Feng Lee 057ef7b668 Merge branch 'master' 2018-02-26 01:05:08 +08:00
Feng Lee fcb2ec8427 Merge the latest enterprise branch 2018-02-24 15:56:32 +08:00
Feng Lee b63c34ab14 Merge the latest master branch 2018-02-24 11:27:09 +08:00
Feng Lee bcec25092b Merge version 2.3.2 to X edition 2017-12-26 21:09:24 +08:00
Feng Lee f1640f5b85 Merge emq24 to X branch 2017-12-07 11:02:44 +08:00
Feng Lee aaf19787af Remove the duplicated InfoHandler 2017-11-22 18:11:21 +08:00
Feng Lee 99ed0e46b6 Remove RPC args 2017-11-22 17:07:11 +08:00
Feng Lee 2cec86eba7 Merge with version 2.3.0 2017-11-21 14:14:12 +08:00
Feng Lee 4b8cd18f5d Merge with EMQ X project 2017-10-09 19:43:06 +08:00