Commit Graph

181 Commits

Author SHA1 Message Date
GilbertWong 0c14484b0a Disable the useless error log info 2019-07-19 17:36:50 +08:00
GilbertWong 953d320667 Implement better websocket exit mechanism 2019-07-19 17:36:50 +08:00
GilbertWong 234037aee1 Fix will message mechanism for websocket channel 2019-07-11 09:43:52 +08:00
周子博 61d64cbb83 Fix the issue that emqx_session:attrs/1 returns wrong value 2019-07-05 22:54:57 +08:00
Feng Lee 42da51e1c5 Implement batch delivery for the new channel design
- Implement a new emqx_channel module
- Improve the emqx_frame:serialize/2 API
- Improve the emqx_connection, emqx_ws_connection modules
2019-07-02 13:40:52 +08:00
terry-xiaoyu 0e2e4e73ca Clean staled sesssions 2019-06-26 20:04:32 +08:00
tradingtrace 0b116bc208 Fix the massages order when batch dispatching 2019-06-26 19:32:15 +08:00
Feng Lee 4974eab20e Improve the channel design 2019-06-25 14:53:45 +08:00
Feng Lee 3877c4db1a Merge branch 'channel2' into channel3 2019-06-18 16:49:26 +08:00
terry-xiaoyu 7092748266 Improve log messages using logger header 2019-06-18 14:37:32 +08:00
Feng Lee 21162f88b8 Update copyright 2019-06-18 14:27:06 +08:00
Feng Lee 1d88f8fd9e Improve the emqx_connection, emqx_ws_connection and emqx_frame modules (#2611)
* Adopt channel architecture and improve the MQTT frame parser

* Update the test cases for emqx_channel, emqx_protocol

- Improve emqx_client to Use the new emqx_frame:parse/2
- Update the ct suites for emqx_channel, emqx_ws_channel

* Fix test case
2019-06-14 12:44:27 +08:00
Feng Lee d386b27e8a Adopt channel architecture and improve the MQTT frame parser 2019-06-11 23:18:38 +08:00
Feng Lee f801ff1e53 Optimize metrics using counters and persistent_term (#2597)
* Optimize emqx_metrics module using persistent_term and counters in OTP 22
2019-06-05 23:16:35 +08:00
Gilbert 0857b976ed Better zone (#2548) 2019-05-25 22:29:23 +08:00
周子博 6a4c318acb Update metrics when session process terminates 2019-04-11 10:08:26 +08:00
Gilbert Wong 15f8d3208f Fix message dropped
Prior to this change, when store_qos0 option is set to false, the hook
`message.dropped` would not be triggered. It is wrong design and when
the ignore_loop is enabled or nl is set to 1, the hook
`message.dropped` also would not be triggered.
2019-04-03 20:33:58 +08:00
HeeeJianBo 24f8c3a391 Fix drain the session process mailbox handling err 2019-03-28 10:32:03 +08:00
周子博 0fbf771543 Readjust log level into error when got unexpected message 2019-03-27 11:07:58 +08:00
周子博 58e1b4d485 Readjust log level and unify log format 2019-03-26 22:18:31 +08:00
HeeeJianBo fc2df9aff5 Adapt to new export style 2019-03-22 14:58:24 +08:00
HeeeJianBo 998684fc4e Fix badmatch error in the batch_process of session 2019-03-21 17:40:02 +08:00
tigercl 5a401c44fd Clear dup flag if message published first (#2337) 2019-03-21 10:13:41 +08:00
Shawn 02fe8560e2
Improve emqx_hooks and credentials (#2309)
* Improve emqx_hooks and credentials

1. Modify the return modes of emqx hooks.

Change the return value of hook functions to:
- ok: stop the hook chain and return ok
- {error, Reason}: stop the hook chain and return error
- continue: continue the hook chain

And the return value of emqx_hooks:run/2 is changed to:
- ok
- {error, Reason}

And the return value of emqx_hooks:run/3:
- {ok, Acc}
- {error, Reason, Acc}

2. Treat password as a member of credentials.

Password should be wrapped in the `credentials` data-structure, as the
username/password pair together consists of an authentication method.
There can be some methods using some other credential data (e.g.
a JWT token), and these credential data should also be wrapped in the
the `credentials` data-structure.

An event `client.authenticate` is triggered when an user logs in:
```erlang
emqx_hooks:run('client.authenticate', [], Credentials)
```

A `default callback` that deny/allow any user (according to the
`allow_anonymous` config) should be appended to the end of the
callback chain.

The `credentails` is passed through all of the callbacks, and
can be changed over in this process.

* Refactor emqx hooks return mode

* Remove password from PState
2019-03-16 10:43:53 +08:00
Feng Lee 3e0fa87209 Remove the io:format 2019-02-28 15:09:13 +08:00
Feng Lee e3bc8e4f0a Support batch delivery
- Upgrade the emqx_session module to support batch delivery
- Update emqx_protocol:deliver/2 to support batch delivery
- Update some test cases
2019-02-28 15:09:13 +08:00
Feng Lee 08204fc7a8 Improve the emqx_connection module using gen_statem behaviour (#2235) 2019-02-28 15:08:29 +08:00
Feng Lee 7a645dd9cc Add 'include/types.hrl' and introduce some common types 2019-02-19 17:16:11 +08:00
turtled 31e95b56c4 Update Copyright to 2019 2019-01-29 10:43:25 +08:00
turtleDeng 00863acea3
Merge emqx30 (#2181)
* Change the reason code in will topic acl check (#2168)

* Fix bridge bug (#2160)

* Limit bridge QoS less than 1

* Improve shared sub dispatch implementation. (#2144)

* Upgrade ekka, esockd libraries

* Improve the 'try_open_session' function

* Reload config (#2180)
2019-01-25 13:01:48 +08:00
Shawn b8929a46c1 Fix the EXIT outputs in Travis CI (#2154)
* Fix the Exit in testcases

* Fix Exit in emqx_mod_sup_SUITE

* Update testcases for log_tracer

* Fix Exit in emqx_protocol_SUITE

* Add will_acl_check

* Fix more Exits
2019-01-17 17:14:51 +08:00
tigercl bc31faac6b Fix will msg (#2156)
* Remove will_msg and will_topic from protocol state

* Modify try_open_session/1
2019-01-17 13:57:42 +08:00
Feng Lee d7254cdc7f Add more macros for logger 2019-01-04 17:30:27 +08:00
terry-xiaoyu 3b8eb41347 Remove the log header macro 2019-01-04 17:30:27 +08:00
Shawn 69954480bf
Hook args (#2119) 2019-01-03 13:54:31 +08:00
turtled f7596b8131 Fix session shutdown bug 2018-12-21 22:44:58 +08:00
Feng Lee bb45825e77 Inc deliver_stats, enqueue_stats with emqx_pd:update_counter/2 (#2100) 2018-12-21 18:31:04 +08:00
turtled 7d3357e0f3 Merge emqx32 2018-12-21 16:01:32 +08:00
Feng Lee 367b717c40 Implement a new 'emqx_gc' module (#2090)
Update connection/session module to using the new emqx_gc API
2018-12-20 22:42:18 +08:00
Feng Lee d827604213 Remove the ensure_stats_timer/1 call from reply/2 and noreply/1 2018-12-20 11:33:18 +08:00
Feng Lee 42fc8f5811 Improve the session module 2018-12-20 10:21:40 +08:00
turtled 7a1ec580b0 Update broker test cases 2018-12-19 17:18:26 +08:00
turtled 666d9706a3 Fixed conflicts 2018-12-18 16:41:10 +08:00
Feng Lee dc06c0beab Remove 'topic_alias_maximum' from session's state 2018-12-18 15:11:04 +08:00
tigercl 95ad67b47c Fix a bug that will not send a will message in some cases (#2068)
* Fix a bug that will not send a will message in some cases
2018-12-18 14:55:37 +08:00
周子博 b7a39f25f2 Revert "Fix a bug that will not send a will message in some cases"
This reverts commit 675edf3fab.
2018-12-18 12:02:18 +08:00
周子博 675edf3fab Fix a bug that will not send a will message in some cases 2018-12-18 12:01:41 +08:00
Feng Lee 721b72b96a Add 'active_n' option to optimize the CPU usage of emqx_connection (#2060)
* Add 'active_n' option to optimize the CPU usage of emqx_connection

* Supports batch processing 'DOWN' events
2018-12-17 19:53:29 +08:00
Feng Lee 4aaf0a7db4 Remove emqx_session_sup to handle massive concurrent sessions 2018-12-13 17:10:02 +08:00
Feng Lee d8cbf72da1 Fix 'badarg' crash 2018-12-12 16:32:58 +08:00
Feng Lee 8d50c62a94 Optimize connection and session management 2018-12-12 16:10:16 +08:00
Feng Lee bce1ddc5c4 Implement a hash-based subscription sharding 2018-12-06 18:45:07 +08:00
周子博 ea62b15c87 Alter apis provided by emqx_metrics, and use existing timer to commit metrics 2018-11-30 17:44:29 +08:00
周子博 194dbc02c8 Add batch commit for metrics 2018-11-30 17:44:29 +08:00
Shawn eeb0cab3e3 Update proc meta-data for empty clientid (#1980) 2018-11-23 18:11:46 +08:00
spring2maz a2c658ba19 Add acking mechamism for shared dispatch (#1872)
* Add acking mechamism for shared dispatch

For QoS0 messages, no acking
For QoS1/2 messages, 'ACK' at any of events below:
 - ACK when QoS is downgraded to 0
 - Message is sent to connection process
'NACK' at any of events below:
 - Message queue is full and the receiving session starts to drop old messages
 - The receiving session crash
Upon 'NACK', messages are dispatched to the 'next' subscriber in the group,
depending on the shared subscription dispatch strategy.
2018-11-21 22:49:45 +08:00
Gilbert 16821490ce Fix issue#1874 (#1964)
* Fix issue#1874
Prior to this change, if user use one client connect emqx with mqtt
v3.1.1, the client subscribe the topic and publish message to this
topic, it would receive this message itself published, this commit
provide a configure option to let user ignore the message itself published.

This change fix issue 1874.

* Small Fix

* Fix bug

* Better design

* Fix compile warning and improve coverage

* Better design to solve the performance issue

* Fix typo

* Fix typo

* Delete spaces in end of lines.

* Do not use anonymous function

* Better performance
2018-11-19 13:34:03 +08:00
turtled faeda253e1 Fix conflicts 2018-11-10 11:44:55 +08:00
Gilbert Wong 32d3826440 Replace macro QOS$i to QOS_$I
The two styles of qos macro name in one project is chaotic. It is not
a good practice.

So I change the QOS$i to QOS_$i
2018-11-08 20:42:31 +08:00
terry-xiaoyu 94dbdffd59 New logger formatter with meta-data 2018-11-02 18:38:07 +08:00
spring2maz ae743ad1f0 Rewrite emqx_mqueue.erl
Fixed bugs:

- Priority queue lack of a `len + 1` logic in `in/2`

Changed behaviors:

- Topics not found in priority table (from config) will be treated with default priority,
  instead of hasing topic name to a priority number.
- Default priority is now configurable (it was always lower than all configured priorities)
- The dropped message due to reaching `max_len` is now returned from `in/2`,
  so the queue owner (`in/2` caller) can perform autopsy on it
2018-10-27 14:21:53 +02:00
周子博 3cea06f88f Remove resume_session/1, 2018-10-27 14:25:10 +08:00
周子博 7c14ba11d6 Set some attributes when session resumed 2018-10-27 13:59:17 +08:00
周子博 92251d4a8a Make more normalize 2018-10-26 17:45:13 +08:00
周子博 db2e47470a Fix bugs 2018-10-25 16:19:57 +08:00
周子博 6675e3d496 Implement will message delay publish in session, add test case for clean start and will message in connect packet 2018-10-25 14:26:31 +08:00
周子博 29beb42aa2 Using client id rather then session pid 2018-10-10 14:00:17 +08:00
周子博 9bcd4c3e08 improve will message 2018-10-09 13:35:27 +08:00
spring2maz e3f2ae8db8 Change from customized total heap size check to set process flag
The `max_heap_size` process flag can be used to limit total
heap size of a process, and it gives much more detailed
crash log if the limit is hit.
2018-09-29 21:36:34 +08:00
周子博 1705f25db6 Fix bug in session expiry interval 2018-09-29 21:30:22 +08:00
周子博 2d354ca883 Improve topic alias maximum in connect packet 2018-09-29 14:47:21 +08:00
周子博 064db65206 improve receive maximum in connect packet 2018-09-29 14:47:21 +08:00
HuangDan 96b5d71a67 Fix args errors on emqx_hook:run('message.acked') 2018-09-27 11:49:23 +08:00
tigercl 03f607c1b2 Fix issue#1833 and #1834 (#1845) 2018-09-22 16:10:24 +08:00
spring2maz b61615323b Move shutdown policy config to zone configs 2018-09-21 18:58:18 +02:00
spring2maz 6fca651a84 Add connection/session shutdown policy
The hibernation behaviour is also changed (implicitly) in this commit:
Prior to this change, connection/session always hibernates after
the stats timer expires regardless of messages in mailbox.
After this commit, connection/session process only goes to hibernate
when the timer expires AND there is nothing left in the mailbox to
process
2018-09-21 18:58:18 +02:00
spring2maz 721f237bc4 Rewrite emqx_gc.erl
The implementation prior to this commit supports
only one gc enforcement policy which is message count threshold.
The new implementation introduces 1 more: volume threshold based
2018-09-21 22:56:34 +08:00
terry-xiaoyu 389b3c80bc Fix the message delivery to remote 2018-09-21 20:00:43 +08:00
周子博 3caa41f751 Add run hook when duplicated subscription 2018-09-08 19:01:03 +08:00
周子博 40977e529a Retain flag in retained message must set to 1 2018-09-08 18:31:44 +08:00
周子博 f8471afb97 Add handling of retain handling subscription option 2018-09-07 13:50:12 +08:00
周子博 47955f4309 fix bug in retain as published flag 2018-09-05 15:18:26 +08:00
周子博 aa34258f1e Support Retain As Published in Subscription Options 2018-09-05 14:25:33 +08:00
terry-xiaoyu 3f42f1271b bug fix 2018-08-31 18:14:10 +08:00
Feng Lee ea1ae70833 Fix errors found by dialyzer 2018-08-31 16:46:51 +08:00
Feng Lee 0379219a04 Improve the design of session discard 2018-08-30 23:14:09 +08:00
Feng Lee 78a8ccd0f2 Only store packet_id and timestamp for qos2 message 2018-08-30 21:17:20 +08:00
Feng Lee 826daace61 Align the state record 2018-08-30 18:44:58 +08:00
Feng Lee c49e5dfddc
Merge branch 'emqx30' into emqx30-feng 2018-08-30 18:42:29 +08:00
Feng Lee 7b5f2577d3 Support message ttl and expiration 2018-08-30 18:30:33 +08:00
Gilbert Wong b0ed953708 fix emqx_session:unsubscribe bug 2018-08-30 13:56:55 +08:00
Feng Lee 2342a7db6d Rename 'Pid' to 'ConnPid' 2018-08-30 11:48:49 +08:00
Feng Lee 53a2f93b7e Add emqx_connection:attrs/1, emqx_protocol:attrs/1, emqx_session:attrs/1 APIs 2018-08-30 03:03:19 +08:00
Feng Lee 8492513094
Merge branch 'emqx30' into emqx30-feng 2018-08-29 23:11:24 +08:00
Feng Lee 567aeb274f Define types in emqx_types, emqx_mqtt_types modules 2018-08-29 23:08:55 +08:00
turtled 1629215b18 Fix hook args bug 2018-08-29 20:29:03 +08:00
Feng Lee 465963fec3
Merge branch 'emqx30' into emqx30-feng 2018-08-29 17:36:38 +08:00
Feng Lee 1a7d60a7e3 Improve the hooks design for emqx 3.0 2018-08-29 17:27:56 +08:00
Feng Lee 594819b752 Upgrade the publish sequence of QoS1/2 messages 2018-08-29 02:53:22 +08:00
Feng Lee 8e11845f87 Remove ignore_loop_deliver option 2018-08-28 07:36:22 +08:00