Commit Graph

810 Commits

Author SHA1 Message Date
Zaiming Shi faecde9ce1 fix(emqx_plugins): allow loading conf for plugin app dir
Prior to this change, plugin config files are only allowed
to be placed in the collective config dir etc/plugins.
In order to support external plugin's drop-in deployment,
this commit made emqx_plugins module to read conf file
in application's etc dir
2021-05-24 10:45:56 +02:00
Zaiming Shi 4e1798e3f3 fix(emqx_cm): do not log noproc as error
1. websocket call exit with noproc reason.
2. do not capture stacktrace when no need for it
2021-05-19 23:16:11 +02:00
Zaiming Shi 55316b3ac3 perf: micro optimisation: no lookup for non-wildcard in trie 2021-05-14 10:45:58 +02:00
Zaiming Shi 3c03047c9f fix(emqx_trie): performance issue when many levels 2021-05-14 10:45:58 +02:00
Zaiming Shi 05c5378265 feat(http_lib): add normalise_headers API 2021-05-07 10:14:19 +08:00
Zaiming Shi f8700e3f27 fix(emqx_packet): no crash if publish packet has no data 2021-05-04 13:48:35 +02:00
Zaiming Shi 29475eb610 feat(emqx_http_lib): try to parse host ip 2021-04-28 10:43:36 +02:00
Zaiming Shi eb946eb80c fix(emqx_trie): do not try to match wildcard topics 2021-04-26 14:18:40 +02:00
Zaiming Shi 22e72cdd82 refactor(emqx_trie): rename record from eqmx_topic to emqx_trie 2021-04-26 14:18:40 +02:00
Zaiming Shi bc6a87946c refactor(emqx_trie): store only prefixes
This commit refactors emqx_trie implementation
with mainly two changes

1. Deleted the edge table.

   In the old implementation, trie consist of nodes and edges.
   e.g. for topic 'a/b/+/d', node 'a' with edge 'b'
   points to node 'a/b' and so on.

   However, edges can be computed at runtime,
   so there is no need to store them in a table.
2021-04-26 14:18:40 +02:00
Zaiming Shi d61b100cc9 chore: more copyright updates 2021-04-23 20:43:00 +02:00
Zaiming Shi 6354e75626 chore: update copyrights 2021-04-23 20:43:00 +02:00
Zaiming Shi c81df7278f test(trace): split test case 2021-04-23 06:54:10 +02:00
Zaiming Shi 3547dc4c93 fix(log): client id as string for log metadata
so it does not print <<"...">> to the logs
2021-04-23 06:54:10 +02:00
Zaiming Shi fd69969014 refactor(logging): Log to single line 2021-04-23 06:54:10 +02:00
JianBo He 1f258a0499 test(proper): fix cant_generate error 2021-04-19 22:03:51 +02:00
Shawn 47817ea28d fix(emqx): meck emqx_connection:call/3 in testcase 2021-04-16 13:37:24 +08:00
Shawn 4885171e4f fix(emqx): add timeout for open/kick a session 2021-04-15 23:09:58 +08:00
Shawn ebac8c1612 fix(emqx): set the timetrap of emqx_cm_SUITE to 1 min 2021-04-15 23:09:58 +08:00
JianBo He 04cbb3751d test(proper): wait tcp port closed 2021-04-15 21:18:17 +08:00
JianBo He 4a388400a5 test(proper): ensure epmd started 2021-04-14 19:38:56 +08:00
JianBo He c413af5c35 test(proper): track to the lastest code
- use limited atom type
- keep webhook confs testing to lastest schema
2021-04-14 19:38:56 +08:00
JianBo He eb8c8021f0 test(proper): spefic the nodename for checking 2021-04-14 19:38:56 +08:00
z8674558 a48e7df4f5 feat(emqx_ws_connection): check http header to know real IP/port 2021-04-14 06:45:39 +00:00
JianBo He 16c999ed9b
Start emqx-modules application by default (#4518)
* fix(modules): start emqx_modules by default

* chore(test): eliminate some compile warnings
2021-04-12 09:45:31 +08:00
Zaiming Shi 25e0e121e5 chore(test): ensure snabbkaffe block until is not timeout 2021-04-09 10:18:25 +02:00
Zaiming Shi b3b821e8c4 fix(flase): false 2021-04-08 00:34:43 +02:00
Zaiming Shi 6f5aa88562 chore(emqx_connection): Add a test case to cover oom kill 2021-03-30 10:08:13 +02:00
Zaiming Shi 5e2ba1cdc3 chore: Fix emqx_channel_SUITE meck orders 2021-03-29 09:42:07 +02:00
Zaiming Shi e579c7624b chore(channel): Add test case for receive-maximum check and override 2021-03-29 09:42:07 +02:00
zhanghongtong 72c1ee264c chore(elvis): fix elvis error 2021-03-19 16:11:23 +08:00
zhanghongtong 568f4de6e5 fix(channel): fix the wrong acl check order
when subscribe, check the acl rule before run 'client.subscribe' hooks
2021-03-19 16:11:23 +08:00
Shawn 81602c973c fix(emqx): deny pingreq when mqtt not connected #4370 2021-03-19 14:17:49 +08:00
zhouzb fa8b2a5ac5 chore(test-cswsh): add test case for cswsh 2021-03-19 10:28:13 +08:00
Karol Kaczmarek c7b44caa1d feat(acl): Add possibility to remove all acl cache 2021-03-17 19:48:42 +01:00
Shawn d6797760a1 fix(test): add testcases for ws subprotocols 2021-03-17 14:59:05 +08:00
z8674558 9087f0c138 feat(emqx_message): add from_map 2021-03-11 16:01:30 +09:00
Shawn b400571dbb fix(emqx): validate mqtt malformed varible byte integer 2021-03-10 10:41:57 +01:00
Zaiming Shi c752f3bec5
Refactor http lib add uri parse (#4292)
* feat(http_lib): Add uri parse to emqx_http_lib

* fix(webhook): call emqx_http_lib to parse uri

* fix(auth-http): Call emqx_http_lib to parse uri

* fix(rule-engine): call emqx_http_lib to parse uri
2021-03-06 13:35:02 +08:00
Zaiming Shi 63c001a7aa refactor(http-lib): Add emqx_http_lib
So far only uri_encode and uri_decode APIs
2021-03-03 21:38:10 +01:00
Zaiming Shi 6e0b53fbb9 fix(eunit): fix emqx_tls_lib_tests 2021-03-01 21:00:17 +08:00
Zaiming Shi 3dfa9f45c4
Merge pull request #4183 from zmstone/merge-e4.2.4-to-dev-4.3.0
Merge e4.2.4 to dev 4.3.0
2021-02-19 10:26:47 +01:00
Shawn eec64e440d
fix(tracer): create short handler-id if clientid is too long (#4164)
* fix(tracer): create short handler-id if clientid is too long

No need to support lists and atoms for when tracing clients/topics.
2021-02-16 14:45:33 +08:00
Zaiming Shi f9465dda16 Merge branch 'dev/v4.3.0' into merge-e4.2.4-to-dev-4.3.0 2021-02-14 21:41:23 +01:00
Zaiming Shi b4b318cb86 chore(style): fix elvis check for emqx_connection_SUITE 2021-02-13 11:41:18 +01:00
Zaiming Shi 9b82112b7c Merge remote-tracking branch 'origin/dev/e4.3.0' into merge-e4.2.4-to-dev-4.3.0 2021-02-13 11:10:08 +01:00
Zaiming Shi 507759b9f1 Revert "refactor(emqx_modules): Move modules config to app dir"
This reverts commit 9cae8784c0.
Because the reverted change broke compatibility with 4.2.x
configuration.
2021-02-11 16:44:12 +01:00
Zaiming Shi 514c2c0af5
Merge pull request #4169 from zmstone/refactor-move-modules-config-to-emqx-modules-app
refactor(emqx_modules): Move modules config to app dir
2021-02-10 09:03:04 +01:00
Zaiming Shi d74bb81ff1 chore(style): make elvis happy 2021-02-10 00:02:52 +01:00
Zaiming Shi 9cae8784c0 refactor(emqx_modules): Move modules config to app dir 2021-02-10 00:02:52 +01:00
Zaiming Shi 2852ac79d4 feat(tls): Add a tls lib module
emqx_tls_lib is added to help ensure integrity of tls version
and cipher options.
2021-02-09 21:20:26 +01:00
Zaiming Shi 7aff861f9b refactor(emqx_modules): Move emqx_modules to lib-opensource
emqx_moduels for enterprise is refactored as a lib/plugin
in order to avoid merge conflicts, we make sure they can
co-exist in the same branch, because lib-opensource is compiled
in opensource project and lib-enterprise (to be added) is
compiled in enterprise project
2021-02-08 21:35:44 +01:00
Zaiming Shi 7c94a02bdd
Merge pull request #4134 from jovdipp/tlsv1.3-as-default
Tlsv1.3 as default
2021-02-05 21:33:03 +01:00
Jóvan S. Dippenaar 6541c589ab test(tls1.3): Tests for default ssl config, as well as specific tls versions ( 1.1, 1.2, 1.3 ) 2021-02-05 16:21:28 +13:00
emqx-ci-robot 6c1129dc6a
Auto-pull-request-on-2021-01-29 (#4114)
* fix(share sub): fix the issue that the number of subscriptions dropped to 0 during the picking subscriber and caused a crash

* Connection Busy Alarms (#3992)

feat(emqx_connection): improve port_busy alarm

* fix(emqx_connection): tune the congestion alarm params

* chore(deps): upgrade esockd to 5.7.5
* fix(appup): correct the appup file
2021-01-30 10:30:18 +08:00
tigercl 5878950dc3
feat(websocket): support for checking subprotocols (#4099) 2021-01-28 20:42:57 +08:00
Zaiming Shi 4680ba574c chore(emqx_vm): Delete get_ets_object, not used 2021-01-22 09:21:45 +01:00
Zaiming Shi 47df29ae49 chore(emqx_vm): delete stale code
The port binary format is no longer compatible in OTP 23
2021-01-22 07:50:00 +01:00
Zaiming Shi ca32ac310f chore(style): make elvis happy 2021-01-21 21:05:56 +01:00
Zaiming Shi 0087303b25 feat(shared_sub): Support hashing from source topic. 2021-01-21 21:05:56 +01:00
zhouzb bcb63bcc18 fix(test case): fix test case 2020-12-28 17:45:19 +08:00
JianBo He 739e49218f
fix(cm): fix the problem of registering a channel twice (#3831) 2020-12-28 11:03:29 +08:00
zhanghongtong 3ef08f9ce3 test(meck): add meck unload after meck load 2020-12-22 13:39:49 +08:00
zhanghongtong ce6411532b test(emqx): fix test case error for emqx 2020-12-22 13:39:49 +08:00
JianBo He 826419df5f test(limiter): more robust asserting logic 2020-12-14 11:24:07 +08:00
Shawn 10dbb5bc96
Fix testcases warns (#3906) 2020-12-11 18:54:48 +08:00
Zaiming Shi d67212e7fa chore(test): Make test profile build 2020-12-11 08:19:26 +01:00
JianBo He 7f556d36d1
Merge branch 'dev/v4.3.0' into e422_to_v430 2020-12-09 11:28:36 +08:00
Zaiming Shi 6d1cd6b7ea refactor(emqx_password): move the module to emqx application
Future plugins will require emqx as a dependency.
This commit also brings back bcrypt to the system which was temporary
removed when converting to umbrella project
2020-12-08 22:09:59 +01:00
Shawn 28b0e874f6 fix(tests): test cases for receiving will msgs 2020-12-07 15:53:22 +08:00
Shawn d00ea48752 feature(tcp): alarm when tcp connection congested 2020-12-07 14:35:38 +08:00
JianBo He f1b3bbd7bc chore: supply the code_change logic 2020-12-05 17:20:50 +08:00
JianBo He 3b1074d11f refactor(conn): not saving anonymous func 2020-12-05 17:11:58 +08:00
Zaiming Shi 73d02beace refactor(apps): Merge emqx_libs back to emqx
And move emqx app back to root level

After realising the challenges of developing plugins in standalone
rebar projects. it is perhaps more smooth to keep using emqx as
a rebar dependency
2020-12-04 19:58:58 +01:00
Turtle f8369f5280 test: comment the will message cases 2020-12-04 22:33:29 +08:00
JianBo He ca0f6b69a4 chore: supply the code_change logic 2020-12-04 22:33:29 +08:00
JianBo He 80c503fd64 refactor(conn): not saving anonymous func 2020-12-04 22:33:29 +08:00
Shawn cc69225b6d feature(tcp): alarm when tcp connection congested (#3818) 2020-11-10 23:36:11 +08:00
JianBo He 139306b61c fix(json): fix decoding/encoding empty object error 2020-11-10 23:36:11 +08:00
JianBo He b883e238bf fix(ws): take ws_cookie to clientinfo
see: https://github.com/emqx/emqx/issues/3747#issuecomment-702268570
2020-11-10 23:36:11 +08:00
terry-xiaoyu 18d6647b03 feature(project): move test to apps/emqx/test 2020-10-31 10:17:47 +08:00
JianBo He b373c0cf1b fix(json): fix decoding/encoding empty object error 2020-10-24 15:49:15 +08:00
JianBo He 2feeb4d91f fix(ws): take ws_cookie to clientinfo
see: https://github.com/emqx/emqx/issues/3747#issuecomment-702268570
2020-10-24 11:21:39 +08:00
turtled eb4b9936dc fix(modules): fix conflicts 2020-10-11 11:12:39 +08:00
tigercl da4d71917f fix(ignore-loop-deliver): fix issue#3738 (#3741) 2020-09-29 15:35:16 +08:00
turtleDeng edcbdcce87 improve: module_acl_internal load/unload api (#3706) 2020-09-05 17:23:22 +08:00
JianBo He 90b33b044d feature(ratelimit): support to set ratelimit & quota policy 2020-09-05 17:23:22 +08:00
terry-xiaoyu 9985e2675c fix(logger): resolve issues when starting and stopping log handlers 2020-08-28 17:10:26 +08:00
zhouzb e232032ec0 fix(mqtt-piggyback): fix test case 2020-08-22 15:27:11 +08:00
Rory Z f47e10e08a feature(emqx_mod_rewrite): separate rewrite rules for pub and sub (#3676) 2020-08-22 15:27:11 +08:00
zhouzb dbeabf3de0 refactor(telemetry): remove telemetry code from emqx core 2020-08-22 15:27:11 +08:00
tigercl e37a70509b feature(telemetry): support for collecting telemetry data and disable it by default (#3653) 2020-08-14 16:58:08 +08:00
JianBo He 98faa698f3 feature(quota): add quota confs to limit the number of message forwards 2020-08-14 16:58:08 +08:00
zhouzb aaf29ee164 fix(alarm): fix the issue that memory watermark were not set correctly 2020-07-31 18:07:20 +08:00
zhouzb 1ba4743213 refactor(alarm): new data structure and support regular cleaning of deactivated alarms 2020-07-31 18:07:20 +08:00
zhouzb 43b49edd28 feature(alarm): new design for alarm 2020-07-31 18:07:20 +08:00
zhanghongtong 0817761aee fix(emqx_channel): EMQ X replaces MQTT properties when node connects using "topic-alias-maximum" property 2020-07-31 18:07:20 +08:00
JianBo He 1c316cb5f0 fix(test): fix badmatch error 2020-07-31 18:07:20 +08:00
JianBo He 83a2af812f feat(ratelimit): refactor ratelimit 2020-07-31 18:07:20 +08:00
Rory Z 492d224728
Auto-pull-request-on-2020-07-17 (#3600)
* refactor(channel): skip the ACL checking for inner subscribe
* fix(props): fix the prop_emqx_sys results of judgment
* Update esockd to 5.7.1
* test(topic-metrics): add test cases for topic metrics
* perf(emqx_vm): make emqx_vm:get_memory/0 more efficiency
2020-07-17 18:25:46 +08:00
JianBo He 600b1055f3 fix(json): convert a proplists to ejson instead of map 2020-06-20 15:42:07 +08:00
tigercl 1b6a586948
feature(mqtt): support response information (#3533) 2020-06-20 15:07:37 +08:00
zhanghongtong 8707afa54c fix(enhanced_auth): fix bug when "conn_props = undefined" 2020-06-20 14:29:20 +08:00
JianBo He 9a8859a44c refactor(topic): move triples/1 func into emqx_trie module 2020-06-18 17:38:34 +08:00
JianBo He 1700a7a98a refactor(test): put all property cases to test/props directory 2020-06-18 17:38:34 +08:00
JianBo He e3489b9d46 refactor(style): improve all types declaration
- Add dialyzer for code analysis
- Correct all the module type declarations
- Use `-type()` to declaration a type instead of `-opaque`
  (It is advantageous to the code dialyzer)

BROKEN CHANGES:
- Add a ?DEFAULT_SUBOPTS for emqx_broker:subscribe/1,2,3
- Remove the needless internal function `emqx_vm:port_info(PortTerm, Keys)`
2020-06-18 17:38:34 +08:00
terry-xiaoyu 6ad65d587e feature(properties): add properties in message header
1. Add MQTT5.0 properties when invoking the hooks.

- Add a new key `properties` into the message header:

  We used to put all of the properties into the `#message.headers`:

  ```erlang
  #message{headers = #{'User-Property' => {<<"reason">>,<<"normal">>},
                       'Message-Expiry-Interval' => 60,
                       allow_publish => true}
  ```

  I put them instead under a single key:

  ```erlang
  #message{headers = #{properties =>
                        #{'User-Property' => {<<"reason">>,<<"normal">>},
                          'Message-Expiry-Interval' => 60},
                       allow_publish => true}
  ```

  Note that the column `properties` may contain all the properties for
  Publish and Will messages.

- Add `disconn_props` into the `Channel.conninfo`

- Add `puback_props` also into the message header:

  ```erlang
  #message{headers = #{puback_props =>
                        #{'User-Property' => {<<"reason">>,<<"normal">>}},
                       allow_publish => true}
  ```

2. Change the data type of `#message.headers` and `#message.flags` to map.

   It used to support `undefined`.
2020-06-17 18:46:26 +08:00
JianBo He ea2a424ce2
Add ensure/2 funcs to avoid needless logs (#3483) 2020-06-04 10:46:53 +08:00
JianBo He 3fb82f7234 Fix(connect): fix the race condition for openning session
- Remove the register_channel/1,2 functions
2020-06-02 17:21:06 +08:00
zhouzb 83b1f3e6c8 Fix unexpected packet before connected 2020-05-22 18:18:52 +08:00
tigercl 7a0ad0f8af
Update certs (#3447)
* Update certs

* Update certs

* Fix test cases fail

Co-authored-by: turtleDeng <deng@emqx.io>
2020-05-12 10:37:26 +08:00
qingchuwudi 7ba801c8d4
Fix reload acl module and clean acl caches (#3409) 2020-04-26 15:45:55 +08:00
zhouzb 7cf97acddd Delete expiried client 2020-04-26 15:38:29 +08:00
zhouzb d5f405259d Fix test case 2020-04-17 20:46:57 +08:00
zhanghongtong dccf3560f0 Add hook to auth packet 2020-04-16 16:07:01 +08:00
张奇怪 9ba938b9d9
EMQX failed to start when the listening port was occupied (#3354) 2020-04-11 10:18:35 +08:00
turtleDeng d49f4118fe
Mgmt emqx modules (#3376) 2020-04-10 19:55:21 +08:00
张奇怪 d4932533ca
Auth packet (#3374) 2020-04-10 19:45:18 +08:00
zhanghongtong e5a6e0dcce Fix test case error 2020-03-27 19:40:30 +08:00
JianBo He 85f0ced88a Ensure delayed test case 2020-03-27 18:01:00 +08:00
zhouzb 792a14aaf7 Fix plugin test case 2020-03-27 15:31:11 +08:00
tigercl 5eadca1782
Merge pull request #3335 from emqx/reload_plugin
Read new configuration items when plugins is loaded
2020-03-26 18:35:49 +08:00
JianBo He 9e6966110d
Move emqx_delay_publish here (#3323)
Move emqx_delay_publish here
2020-03-26 18:32:24 +08:00
张奇怪 59adb9a652
Supports broker-to-client topic aliases (#3325) 2020-03-24 15:23:01 +08:00
张奇怪 2ab6ec418e
Add subscription option for proxy subscribing 2020-03-24 11:31:25 +08:00
zhouzb 3b537760d5 Read new configuration items when plugins is loaded 2020-03-23 15:23:38 +08:00
JianBo He 7e00b58db5
Remove the CONNACK's `Maximum-QoS` key if it equal to 2 (#3321) 2020-03-17 17:56:34 +08:00
JianBo He b2e5b523dd
Change global_gc to major mode (#3317) 2020-03-17 14:28:58 +08:00
JianBo He 47720b906f Fix `peer_cert_as_username` option not working
see #3281
2020-03-05 15:55:39 +08:00
JianBo He 750b088922 Fix testcase 2020-03-05 14:14:22 +08:00
JianBo He 369db50ff6 Fix testcase 2020-03-05 14:14:22 +08:00
JianBo He 9b6a8ebe84 Fix emqx_json:to_ejson 2020-02-21 13:17:12 +08:00
JianBo He 3f6f0eca40
Add an option to allow client bypass auth plugins 2020-02-21 09:42:26 +08:00
zhanghongtong 7bc6a29f4e Receive for the EXIT message on test cases 2020-02-07 16:23:02 +08:00
张奇怪 f29a8b0d5c
Mqtt protocol tests (#3237)
Add more test cases for MQTT
2020-02-07 10:07:36 +08:00
JianBo He b9bdb03005 Decode/Encode json from nested proplist 2020-02-06 12:00:18 +08:00
JianBo He 3c459e8e74
Compatible jsx encode/decode (#3230) 2020-02-04 16:18:19 +08:00
JianBo He 7ab3da399d
Fix testcase (#3216) 2020-01-20 09:41:46 +08:00
Feng Lee eea377eb8f Replace 'jsx' with 'jiffy' and add more test cases 2020-01-19 16:58:51 +08:00
JianBo He e17241884c Send DISCONNECT packet for mqttv5 (#3183) (#3208) 2020-01-17 19:48:39 +08:00
turtleDeng 7a2234c608 Improve emqx_mqtt_caps:get_caps/1 (#3198) 2020-01-17 16:29:46 +08:00
Feng Lee 2b3003b323 Move the global GC server from 'emqx-recon' to 'emqx' project (#3190) 2020-01-16 23:45:17 +08:00
turtleDeng f6b2c9a69f Correct timestamp for banned (#3188) 2020-01-16 23:11:19 +08:00
turtleDeng 0362a28557
Update mqtt_protocol_v5_SUITE.erl 2020-01-10 17:48:44 +08:00
zhanghongtong 4dde1fcf7c Fix error for test case 2020-01-09 20:01:46 +08:00
zhanghongtong 7bd6b249da Increased wait time for test cases 2020-01-09 20:01:46 +08:00
zhanghongtong 16f829a345 Increased wait time for test cases 2020-01-09 20:01:46 +08:00
zhanghongtong 693bd493f0 Fix error for test case 2020-01-09 20:01:46 +08:00
zhanghongtong fdeade3212 Add more test cases 2020-01-09 20:01:46 +08:00
JianBo He b1a22931f4
Add testcase for takeover (#3160) 2020-01-09 12:26:24 +08:00