Commit Graph

3066 Commits

Author SHA1 Message Date
Shawn 14089a572e fix(bridge): changes timeouts from 30s to 15s 2021-12-30 23:48:41 +08:00
Shawn aefcd6275b fix(bridges): ingress MQTT bridges didn't increase counters on msg received 2021-12-30 23:48:41 +08:00
Shawn c23436166b fix(bridge): HTTP connector should failed on non-200 status codes 2021-12-30 23:48:41 +08:00
Shawn ea1aaa9806 fix(bridge): remove clientid config from MQTT bridges
Don't allow the user provide the clientid for connecting the remote broker.
We generate the clientid using the bridge id and node name.
2021-12-30 23:48:41 +08:00
zhongwencool d16362af96
Merge pull request #6547 from zhongwencool/trace-formatter
feat(trace): replace logger_formatter by emqx_trace_formatter
2021-12-30 23:40:59 +08:00
zhongwencool 489fb7f806 fix(trace): copy binary:encode_hex/2 from binary.erl 2021-12-30 21:38:23 +08:00
JimMoen ff3707fbdc
Merge pull request #6565 from JimMoen/fix-redis-servers-field
fix(connector): redis cluster `servers` field
2021-12-30 19:33:12 +08:00
Ilya Averyanov c98e0efdad chore(mongodb): update client 2021-12-30 13:57:48 +03:00
DDDHuang c2e36b041b fix: dashboard return type 2021-12-30 18:53:09 +08:00
Ilya Averyanov 7208669946 chore(authn): reduce number of pre-generated certificates, refactor
tests
2021-12-30 13:49:14 +03:00
lafirest 0e62a6772c fix(emqx_slow_subs): fix config update error 2021-12-30 16:58:33 +08:00
JimMoen da68dfc8f4 test(authz): `servers` field use string 2021-12-30 15:38:04 +08:00
JimMoen 8cb9482542 fix(authz): `servers` field use string 2021-12-30 15:37:53 +08:00
JimMoen 39c29b2396 fix(connector): redis cluster `servers` field 2021-12-30 15:37:53 +08:00
lafirest f466a9f2de fix(emqx_slow_subs): fix timer management error 2021-12-30 15:30:25 +08:00
lafirest c6cc92c608
Merge pull request #6575 from lafirest/fix/slow_subs_api
fix(emqx_slow_subs): limit the max size of top-k table
2021-12-30 14:53:37 +08:00
DDDHuang bd241f4220
Merge pull request #6568 from DDDHuang/api_authbeta3
fix(api): auth failed with basic type check
2021-12-30 14:53:16 +08:00
lafirest ed086728cf fix: Revert "fix(emqx_retainer): add support for RAP falg"
This reverts commit b80a01554b.
2021-12-30 11:32:08 +08:00
lafirest d60c586bfb fix(emqx_slow_subs): limit the max size of top-k table 2021-12-30 10:38:32 +08:00
DDDHuang db9d269034 fix(dashboard): batter auth failed response message 2021-12-30 10:29:20 +08:00
JianBo He e8acec7f56
Merge pull request #6549 from HJianBo/gw-review-r3
- Add endpoint_name, like_endpoint_name, gte_lifetime, lte_lifetime query parameters for Swagger docs
- Optimize the atomicity of the gateway creation procedure and reduce the side effects if creation failed
2021-12-30 09:54:54 +08:00
Thales Macedo Garitezi 3fc8d4049f
Merge pull request #6571 from emqx/fix-sticky-shared-sub-conf
fix(config): add missing enum constructors for `shared_subscription_strategy`

Currently, there's [code](048afa6d2d/apps/emqx/src/emqx_shared_sub.erl (L270-L279)) and tests that expect the possibility of
values `sticky`, `hash_topic` and `hash_clientid` in the
`node.shared_subscritpion_strategy` configuration, but the schema enum
does not contain those constructors.

There's also a `hash` constructor expected, but apparently it's just for backwards compatibility (same as `hash_clientid`), so maybe we can skip it in 5.0
2021-12-29 15:08:49 -03:00
zhongwencool 43141dffee
Merge branch 'release-5.0-beta.3' into trace-formatter 2021-12-29 22:37:59 +08:00
zhongwencool b8bb5ff738 fix(trace): delete duplicate topic from msg 2021-12-29 22:33:34 +08:00
Thales Macedo Garitezi 048afa6d2d
fix(config): add missing enum constructors for `shared_subscription_strategy`
Currently, there's code and tests that expect the possibility of a
value `sticky`, `hash_topic` and `hash_clientid` in the
`node.shared_subscritpion_strategy` configuration, but the schema enum
does not contain those constructors.
2021-12-29 10:34:20 -03:00
Ilya Averyanov 2b3d3ebb0a chore(authn): test JWT authn with HTTPS key server 2021-12-29 16:24:57 +03:00
Ilya Averyanov f230c20521 chore(authn): test HTTPS authn 2021-12-29 16:24:57 +03:00
Ilya Averyanov a7ca6cb39f chore(authn): test MongoDB authn via ssl connection 2021-12-29 16:24:52 +03:00
Zaiming (Stone) Shi f5049b696a docs: fix a misspell in emqx_mgmt_cli 2021-12-29 13:29:51 +01:00
DDDHuang 34dfc327f3 fix(api): auth failed with basic type check 2021-12-29 19:34:13 +08:00
zhongwencool 04313dc044 fix(trace): download trace api not working 2021-12-29 19:03:54 +08:00
zhongwencool 121d906992 chore(log): add SLOG/3 to add meta info 2021-12-29 14:48:48 +08:00
JianBo He 6280c8c9f0 chore: fix dialyzer warnings 2021-12-29 14:02:45 +08:00
tigercl d769401869
Merge pull request #6555 from tigercl/chore/new-epgsql
chore: support prepared query in postgresql connector
2021-12-29 13:37:29 +08:00
zhongwencool 8b5b3a448a fix(test): trace_handler ct fail 2021-12-29 11:06:31 +08:00
Thales Macedo Garitezi 8f4453fedb
fix(gateway): undo move headers to root `include`
Keeping headers in the src directory makes sense for the interfaces we
don't want to expose outside, but the `include_lib` directive fails to
compile under Mix. Changing `include_lib("emqx_gateway/src/...")` to
`-include("src/...")` pleases both Mix and Rebar.
2021-12-28 17:34:47 -03:00
zhongwencool 4b6bba11eb feat(trace): struct log for trace 2021-12-29 00:26:54 +08:00
zhongwencool b78cfa0a1c fix(api_key): support expired_at never expired when undefined 2021-12-29 00:22:59 +08:00
zhongwencool 0712fc1e75 chore(schema): remove emqx_schema:unicode_binary/0 2021-12-28 23:50:12 +08:00
Thales Macedo Garitezi 0020cf592f
Merge remote-tracking branch 'origin/master' into build-with-mix-mkII 2021-12-28 11:28:40 -03:00
Ilya Averyanov f8ffb9c021
Merge pull request #6506 from savonarola/test-authz-mnezia
Test HTTP, Mria, File authorization
2021-12-28 15:53:37 +03:00
zhongwencool 47be5b9ef2
Merge pull request #6539 from zhongwencool/fix-override-conf-not-load
fix(conf): can't include local/overide-conf file
2021-12-28 18:03:56 +08:00
Zaiming (Stone) Shi 0f14f73a50
Merge pull request #6552 from zmstone/fix-5.0-portable-shebang
fix: portable shebang
2021-12-28 10:51:15 +01:00
Shawn 3a39db4f97
Merge pull request #6550 from terry-xiaoyu/fix_utf8_topic_rewrite
Fix utf8 topic rewrite
2021-12-28 17:08:23 +08:00
Shawn 1d4be368cb
Merge pull request #6556 from terry-xiaoyu/fixes_for_bridges
Bug fixes for data bridges
2021-12-28 17:07:54 +08:00
zhongwencool e44f7de596 chore(conf): update hocon to 0.22.1 2021-12-28 16:17:19 +08:00
Ilya Averyanov 9363b6110e chore(authz): make test http server more robust 2021-12-28 11:06:52 +03:00
tigercl 6a686c3314
Merge pull request #6503 from tigercl/feat/authz-config
feat(authz): support sync configuration in the cluster
2021-12-28 14:39:56 +08:00
Shawn 07997ab865 fix(bridge): Bridges should send a JSON message if payload template not set 2021-12-28 14:18:14 +08:00
Shawn 4406589980 fix(bridge): time unit error for MQTT keepalive 2021-12-28 14:12:28 +08:00
lafirest e48df910d8
Merge pull request #6493 from lafirest/refactor/exhook
refactor(emqx_exhook): refactore exhook and add api module
2021-12-28 14:00:19 +08:00
zhouzb 487b84f166 test: fix test case 2021-12-28 11:37:51 +08:00
Shawn eb992ad2ad fix(bridge): add test cases for sending msgs via http bridge 2021-12-28 11:35:47 +08:00
zhouzb 1a22d5ca46 fix: fix undef function call 2021-12-28 11:21:41 +08:00
Shawn bf2392f682 chore(typerefl): update typerefl to 0.8.6 2021-12-28 10:52:30 +08:00
lafirest c68499e39a fix(api): fix the payload in the result of the api of emqx_retainer & emqx_dealy 2021-12-28 10:25:53 +08:00
lafirest e2240bdf03 feat(emqx_limiter): improve burst implementation 2021-12-28 10:13:55 +08:00
Shawn 1cd226c18b fix(bridge): make direction defaults to egress if not provided 2021-12-28 10:12:39 +08:00
zhouzb d26042703e chore: provide prepared_query and equery 2021-12-28 09:52:31 +08:00
lafirest afe51446b0
Merge pull request #6538 from lafirest/fix/rap
fix(emqx_retainer): add support for RAP falg
2021-12-28 09:31:57 +08:00
zhouzb fc89fb0f8a chore: using prepared query in postgresql connector 2021-12-28 09:16:00 +08:00
Zaiming (Stone) Shi 68eb13d478 fix: portable shebang 2021-12-27 15:45:12 +01:00
JianBo He 8203b1f328 refactor(gw): simplify massive repeated codes 2021-12-27 16:50:39 +08:00
Shawn e9c2b5d1cf fix(rewrite): don't check and translate body for topic rewrite APIs 2021-12-27 16:17:59 +08:00
Ilya Averyanov e2e2c98679 chore(authn): test Mysql authn via ssl connection 2021-12-27 11:16:31 +03:00
Ilya Averyanov 6de89d1207 chore(authn): test PostgreSQL authn via ssl connection 2021-12-27 11:16:31 +03:00
Ilya Averyanov 520629d6e2 chore(authn): test Redis authn via ssl connection 2021-12-27 11:16:31 +03:00
JianBo He 76b35910b7 chore(gw): fix elvis warnings 2021-12-27 16:03:31 +08:00
JianBo He fca5a3bc21 chore(gw): add cli testcases 2021-12-27 16:03:31 +08:00
zhongwencool 668180388c feat(trace): replace logger_formatter by emqx_trace_formatter 2021-12-27 14:40:40 +08:00
tigercl d781dc73a5
Merge pull request #6536 from tigercl/fix/authn-ssl
fix(authn): attempt to convert certificate content only when TLS is enabled
2021-12-27 09:50:11 +08:00
JianBo He 523b5761b7
Merge pull request #6527 from HJianBo/fix-receive-maximum-50
fix: disconnect the client due to exceed receive-maximum packets
2021-12-27 09:13:24 +08:00
Thales Macedo Garitezi cc652c3343
ci(mix): add elixir deps check
Checks whether there are discrepancies between rebar3 and mix deps.
2021-12-24 14:44:49 -03:00
Ilya Averyanov 2bada0bab8 chore(authz): test Mria authz 2021-12-24 16:18:46 +03:00
Ilya Averyanov d75e0104cc chore(authz): test file authz with real files 2021-12-24 16:18:41 +03:00
Ilya Averyanov e0f860d7d9 chore(authz): fix HTTP authz, cover with tests 2021-12-24 16:18:36 +03:00
zhongwencool 0b099432c5 fix(conf): can't include local/overide-conf file 2021-12-24 19:20:36 +08:00
lafirest b80a01554b fix(emqx_retainer): add support for RAP falg 2021-12-24 18:03:55 +08:00
zhongwencool 6326e436d3
Merge pull request #6532 from zhongwencool/fix-banned-bad-peerhost-crash
fix(banned): crash by bad peerhost; add banned http API tests
2021-12-24 17:04:26 +08:00
tigercl 41694b7b34
Merge pull request #6526 from emqx/release-5.0-beta.3
Sync 5.0-beta.3 fixes into master
2021-12-24 13:49:13 +08:00
zhouzb 72a27e9d8f fix: fix undefined function call 2021-12-24 13:44:28 +08:00
zhouzb 9769ddeb14 fix(authn): attempt to convert certificate content only when TLS is enabled 2021-12-24 13:40:13 +08:00
zhongwencool 33a6568654 fix(banned): crash by bad peerhost; add banned http API tests 2021-12-24 10:56:41 +08:00
lafirest d2d50443ce fix(emqx_retainer): use base64 to encode payload in api's result 2021-12-24 10:48:12 +08:00
lafirest 8c42b623c6
Merge pull request #6529 from lafirest/fix/retain_payload
fix(emqx_retainer): use base64 to encode payload in api's result
2021-12-24 10:26:00 +08:00
lafirest b6755d5953 fix(emqx_retainer): use base64 to encode payload in api's result 2021-12-24 10:22:10 +08:00
Thales Macedo Garitezi 11c169501e
chore(mix): fix app dependencies and reboot apps (emqx_conf)
While declaring `emqx_conf` as an application dependency of
`emqx_resource` worked for releases, it messed up the startup
relationship during tests.  Since only removing `emqx_conf` from the
`applications` key in `emqx_resource` breaks the list of apps that
need to be rebooted on config changes (since `emqx_conf` is not on any
apps dependencies list, it was not being added to the final
topologically sorted list), we now always add it as a vertex to ensure
its presence there.  This (apparently) makes the rebar3 release, the
mix release and test runs behave normally.
2021-12-23 13:48:28 -03:00
k32 bd31b52e35 chore(ekka): Bump version to 0.11.2 2021-12-23 15:29:35 +01:00
Spycsh 41e373dbc4
Merge pull request #6478 from Spycsh/slog
chore: refactor ?SLOG
2021-12-23 22:21:33 +08:00
lafirest 7c9c7b6a60 refactor(emqx_exhook): refactore exhook and add api module 2021-12-23 17:51:13 +08:00
JianBo He 52502e29c3 fix: disconnect the client due to exceed receive-maximum packets
As described in the 5.0 specification, we should disconnect clients that
exceed the receive-maximum limit.

> If it receives more than Receive Maximum QoS 1 and QoS 2 PUBLISH packets
where it has not sent a PUBACK or PUBCOMP in response, **the Server uses a
DISCONNECT packet with Reason Code 0x9**

fix: #6447
2021-12-23 14:06:11 +08:00
zhouzb 80c1128b7a fix(authz): fix deadlock issues 2021-12-23 11:45:58 +08:00
JianBo He 967989075c
Merge pull request #6512 from terry-xiaoyu/fix_rule_api_swagger
Fix rule api swagger
2021-12-23 10:37:53 +08:00
Thales Macedo Garitezi 238895a301
chore(mix): correctly define release version
With this, we no longer need the last hack in `mix_release.sh`.

A version schema with 3 parts (Major.Minor.Patch) is required by
Elixir:

https://hexdocs.pm/elixir/1.13.1/Version.html#module-versions
2021-12-22 15:43:43 -03:00
Thales Macedo Garitezi e137555052
fix(tests): fix include paths that were moved 2021-12-22 15:15:19 -03:00
Thales Macedo Garitezi 68cbded7c6
fix(dialyzer): fix dialyzer warnings
Adding hocon to emqx solves the issue, since other offending apps
depend on emqx.  The warnings were that it was not able to find info
on the `hocon_schema` behaviour.

Example:
```
Callback info about the hocon_schema behaviour is not available
```
2021-12-22 15:15:19 -03:00
JianBo He b846193556
Merge pull request #6518 from HJianBo/gw-review-r2 2021-12-22 22:10:18 +08:00
Thales Macedo Garitezi 34d6dbca61
chore(mix): simplify build by using rebar3 for umbrella apps
By treating the apps in the umbrella as dependencies to be managed and
built by rebar3, we can simplify the maintenance of the release, at
the cost of increased build times: using Mix as before, it could track
changed files better than using rebar.  But the complexity and
possibility of discrepancies make it using rebar much more compelling.
2021-12-22 09:40:01 -03:00
JianBo He 1bcdbf3a06 chore(gw): make some fields required 2021-12-22 16:49:34 +08:00
JianBo He 6d4aac1600 chore(gw): improve http error messages 2021-12-22 16:49:33 +08:00
JianBo He 3fd9061418 fix(gw): save coap channel info 2021-12-22 16:49:33 +08:00
zhongwencool 2e10a5700c
Merge pull request #6515 from zhongwencool/fix-bann-utf8
fix(banned): create banned with utf8 failed by 500 response
2021-12-22 16:08:34 +08:00
zhongwencool b11a15fa00 fix(banned): create banned with utf8 failed by 500 2021-12-22 14:29:15 +08:00
Shawn cd4227b851 fix(rules): don't show the module name 'emqx_rule_outputs' in outputs 2021-12-22 10:26:22 +08:00
Shawn 9b4b3d2e8c fix(rules): make the 'name' field of POST /rules mandatory 2021-12-22 10:26:22 +08:00
Shawn 5f050b149b fix(rules): the schema for unsubscribe is messing from rule_test API 2021-12-22 10:26:22 +08:00
Thales Macedo Garitezi 917575de5a
chore(mix): minimal elixir mix release build
This commit enables a minimal working build of EMQX release using
Mix.  However, to properly start the release, several configuration
steps are still missing.  A `mix_release.sh` script does a few hacks
to get the release built with Mix to start properly, by first assuming
that `make emqx` has been run prior to the release, ran once to
generate the `app.*.config` files, and then it copies that and some
other files to the expected places.

Also, `emqx_telemetry` hangs the start procedure because it thinks
it's in an official release and tries to make a request.  We disable
it temporarily via config just to get a working build for now.
2021-12-21 13:51:22 -03:00
zhouzb 25b7719db5 feat(authz): support sync configuration in the cluster 2021-12-21 17:00:49 +08:00
JimMoen 99a5f14301 fix(telemetry): use required fields, rolling distro use PRETTY_NAME 2021-12-21 10:44:54 +08:00
JimMoen 6f28e103d0 fix(dashobard): statistical diagram timestamp use UTC time 2021-12-20 19:21:19 +08:00
JianBo He db08cee20f
Merge pull request #6456 from HJianBo/gw-review-r1 2021-12-20 13:48:24 +08:00
Shawn efb0365cdd
Merge pull request #6481 from terry-xiaoyu/fix_rule_apis
fix(connector): some desc for connector APIs
2021-12-20 13:46:29 +08:00
Shawn 27de3139a4 fix(http_connect): don't check status on_start 2021-12-20 11:36:52 +08:00
zhongwencool f7a55d3f8d
Merge pull request #6238 from zhongwencool/fix-keepalive-ct-failed
fix: keepalive ct failed
2021-12-20 11:00:20 +08:00
JianBo He 5b1ed76a44 chore(gw-frame): export types for dialyzer check 2021-12-20 10:59:16 +08:00
Shawn a879ec0f3a feat(resource): add option 'force_create' to emqx_resource:create/4 2021-12-20 10:26:27 +08:00
zhongwencool 96ee51fe57
Merge pull request #6460 from zmstone/feat-add-plugin-mgmt
refactor(plugins): refactor plugins configuration interface
2021-12-20 09:24:38 +08:00
Shawn 05e24b457a fix(bridge): update emqx_bridge.conf 2021-12-18 20:53:14 +08:00
Shawn a9c9d9d805 fix(rule): rename enabled to enable 2021-12-18 20:28:14 +08:00
Shawn cfaad15364 fix(connector): update the connector config failed 2021-12-18 20:19:34 +08:00
Shawn a44e18e869 fix(bridge): filter out some extra fields from the request body 2021-12-18 19:19:58 +08:00
Shawn 11e8e0db69 fix(bridge): stop http failed due to econnrefused 2021-12-18 15:33:25 +08:00
Shawn f08f37ec9c fix(bridge): create bridge failed but the config was saved 2021-12-18 11:19:22 +08:00
Shawn 20091f8d2a fix(connector): some desc for connector APIs 2021-12-18 09:52:15 +08:00
Shawn b2e7a6a249
Merge pull request #6473 from terry-xiaoyu/fix_connector_apis
fix(connector): add type and name in repsonse of GET /connectors
2021-12-18 09:35:45 +08:00
Spycsh ca2660d609 chore: refactor ?SLOG 2021-12-18 00:17:45 +01:00
Shawn 494c08f849 refactor(rule): add name field to request body of POST /rules 2021-12-18 06:34:44 +08:00
Ilya Averyanov 7cceecc11e
Merge pull request #6446 from savonarola/test-authz-more
Test authz more
2021-12-17 19:06:52 +03:00
k32 93acf8a3a7
Merge pull request #6451 from k32/dev/fix-dispatch-when-emqx-is-stopped
fix(broker): Check broker status before dispatch
2021-12-17 15:19:34 +01:00
Shawn 673a545aa2 fix(dialyzer): bad type specs 2021-12-17 22:15:56 +08:00
Shawn 7bcd38c101 fix(bridge): update qos fields for swagger examples 2021-12-17 21:41:25 +08:00
Shawn 0699682f38 refactor(bridges): rename some config entries for MQTT bridge 2021-12-17 21:22:04 +08:00
Ilya Averyanov e24cdb067c chore(authz): fix Mysql variable substitution 2021-12-17 14:20:53 +03:00
Ilya Averyanov 462ec39cb1 chore(authz): test Postgresql backend with real Postgresql 2021-12-17 14:20:50 +03:00
Ilya Averyanov 0a1a68245d chore(authz): test Mysql backend with real Mysql 2021-12-17 14:18:54 +03:00
Ilya Averyanov a21f67a67e chore(authz): test Mongo backend with real Mongo 2021-12-17 14:18:54 +03:00
Ilya Averyanov 28c319b6c4 chore(authz): test Mongo backend with real Mongo 2021-12-17 14:18:54 +03:00
Ilya Averyanov 6ad71a483e chore(authz): test Redis backend with real Redis 2021-12-17 14:18:54 +03:00
Shawn 9b34f6f9a3 fix(connector): add type and name in repsonse of GET /connectors 2021-12-17 16:45:42 +08:00
zhongwencool 870af6df41 fix: Limit interval between 0~65535 2021-12-17 15:51:54 +08:00
zhongwencool fdff3c5a53 fix: keepalive ct failed 2021-12-17 15:51:51 +08:00
Zaiming (Stone) Shi 58f32c2572 fix(plugins): catch stop_app exception 2021-12-17 08:35:10 +01:00
JianBo He 92fd9c1eca chore(gw): fix adding subscription 2021-12-17 15:31:07 +08:00
zhongwencool 38d42517f7 fix: misc_SUITE:t_drain_down sometimes fails 2021-12-17 14:58:50 +08:00
JianBo He 52c78974d6 chore(gw): fix elvis warnings 2021-12-17 14:16:00 +08:00
JianBo He 561c95c31a feat(lwm2m): support subscribe/unsubscribe operations 2021-12-17 14:16:00 +08:00
JianBo He bfbf377a45 feat(coap): support subscribe/unsubscribe operations 2021-12-17 14:16:00 +08:00
JianBo He 6b8bdfd113 chore(gw): support to get subscriptions from coap 2021-12-17 14:16:00 +08:00
JianBo He 3443aeff18 fix(gw): handle discard/kick event 2021-12-17 14:16:00 +08:00
JianBo He a2823f2ad6 chore(gw): improve some type specs 2021-12-17 14:16:00 +08:00
zhongwencool 8be2aaf72c
Merge pull request #6452 from zhongwencool/log-trace-api
feat(trace): trace http api schema
2021-12-17 12:44:15 +08:00
JimMoen 3a3226fc78 fix(dashboard): enable dashboard by default 2021-12-17 11:20:23 +08:00
zhongwencool 38438415b8 fix: replace LOG by SLOG 2021-12-17 10:52:14 +08:00
zhongwencool 89d904b7ef feat(trace): trace http api schema 2021-12-17 09:29:37 +08:00
Zaiming (Stone) Shi f15efb4930 chore: add a function comment 2021-12-16 22:08:16 +01:00
Zaiming (Stone) Shi b1ec8c1aba fix(plugins): fix relative positioning 2021-12-16 21:57:08 +01:00
Zaiming (Stone) Shi 48ad0adbaa refactor: delete plugin management APIs from emqx_mgmt
Will move the APIs to emqx_plugins_api.erl
2021-12-16 21:53:42 +01:00
Zaiming (Stone) Shi 3a7924d0fd refactor(plugins): new CLI for plugins 2021-12-16 21:21:16 +01:00
Zaiming (Stone) Shi 47661042b9 fix(cli): make emqx_ctl aware of max width 2021-12-16 20:12:15 +01:00
JimMoen 477b62200d fix(authz): placeholder regular expression escape 2021-12-16 23:33:26 +08:00
Zaiming (Stone) Shi b9a3b558ee refactor(plugins): refactor plugins configuration interface 2021-12-16 13:59:20 +01:00
k32 618113d507 fix(broker): Check broker status before dispatch
Fixes: #6388
2021-12-15 19:49:46 +01:00
zhongwencool a43ac1b778 fix: improve rfc3339 format error 2021-12-15 16:58:07 +08:00
zhongwencool e4a21fd90d fix: merge from master 2021-12-15 16:24:25 +08:00
zhongwencool f95ef04cf7
Merge branch 'master' into api-key-secret 2021-12-15 16:01:09 +08:00
lafirest 141921bfc4
Merge pull request #6449 from zhongwencool/slow_sub
fix: slow_sub api return field's notice_interval is missing
2021-12-15 14:45:33 +08:00
zhongwencool 29ccc98a42 fix: slow_sub api return field's notice_interval is missing 2021-12-15 10:36:13 +08:00
Thales Macedo Garitezi 95de2d3467
fix(monitor): fix return value on badrpc
If the rpc in `emqx_dashboard_monitor_api:get_collect/1` fails, an
empty map is return.  But the current function expects a 4-tuple,
which results in a 500 error returned on such occasions.

```
curl -su admin:public localhost:8888/api/v5/monitor/current | jq .
{
  "code": "INTERNAL_ERROR",
  "message": "error, function_clause, [{emqx_dashboard_monitor_api,format_current_metrics,[[#{},{0,0,0,0},{0,0,0,0}],{0,0,0,0}],[{file,\"emqx_dashboard_monitor_api.erl\"},{line,179}]},{emqx_dashboard_monitor_api,current_counters,2,[{file,\"emqx_dashboard_monitor_api.erl\"},{line,167}]},{minirest_handler,apply_callback,3,[{file,\"minirest_handler.erl\"},{line,112}]},{minirest_handler,init,2,[{file,\"minirest_handler.erl\"},{line,38}]},{cowboy_handler,execute,2,[{file,\"cowboy_handler.erl\"},{line,41}]},{cowboy_stream_h,execute,3,[{file,\"cowboy_stream_h.erl\"},{line,318}]},{cowboy_stream_h,request_process,3,[{file,\"cowboy_stream_h.erl\"},{line,302}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,226}]}]"
}
```
2021-12-14 17:56:32 -03:00
Thales Macedo Garitezi 0300403fc0
fix(admin): inconsistent return type on empty inputs
If an empty username or password are given during the initialization
of `emqx_dashboard_admin`, it'll return just the atom `ok`, which
breaks the expected `{ok, _}` in `emqx_dashboard_app:start/2`.

```
{'EXIT',
 {{failed_to_start_app,emqx_dashboard,
   {emqx_dashboard,
    {bad_return,
     {{emqx_dashboard_app,start,[normal,[]]},
      {'EXIT',
       {{badmatch,ok},
        [{emqx_dashboard_app,start,2,
          [{file,
            "/home/thales/dev/emqx/emqx/apps/emqx_dashboard/src/emqx_dashboard_app.erl"},
           {line,32}]},
         {application_master,start_it_old,4,
          [{file,"application_master.erl"},{line,293}]}]}}}}}}}}
```
2021-12-14 17:51:50 -03:00
Ilya Averyanov b8a68d7a9f
Merge pull request #6384 from savonarola/refactor-password-hashing
refactor(authn): unify password hashing
2021-12-14 16:21:02 +03:00
Ilya Averyanov 5538cd3708
Merge pull request #6441 from savonarola/refactor-acl
refactor(authz): hide mnesia authz implementation details
2021-12-14 11:32:25 +03:00
zhongwencool 56804ef733
Merge pull request #6427 from zmstone/feat-add-emqx-plugins-app
feat(plugins): add emqx_plugins skeleton
2021-12-14 16:26:58 +08:00
Zaiming (Stone) Shi 867ebbc8a0 docs: fix review comments for plugins config doc 2021-12-13 20:52:20 +01:00
Ilya Averyanov 708d9cfc6c refactor(authn): restore pbkdf2 password hashing functionality 2021-12-13 21:40:09 +03:00
Ilya Averyanov abdb98ffa2 refactor(authz): hide mnesia authz implementation details
* Eliminate type and record sharing through `emqx_authz.hrl`.
* Hide all mria/mnesia interactions inside mnesia authz backend.
2021-12-13 21:23:58 +03:00
zhongwencool 3e892a337a
Merge pull request #6436 from tigercl/chore/compatibility-otp24
chore(otp): be compatibility with crypto in otp 24
2021-12-13 22:31:15 +08:00
zhongwencool b4e2aa0dcf
Merge branch 'master' into feat-add-emqx-plugins-app 2021-12-13 22:28:03 +08:00
lafirest cbe683c6a3
Merge pull request #6353 from lafirest/slow_subs
feat(emqx_slow_sus): ported emqx_slow_subs from v4.4
2021-12-13 17:35:17 +08:00
JimMoen 40c4880a71
Merge pull request #6435 from JimMoen/fix-ssl-file-content
fix(ssl): file content return type
2021-12-13 16:49:21 +08:00
zhouzb 69800a9e88 chore(otp): be compatibility with crypto in otp 24 2021-12-13 16:16:57 +08:00
JimMoen 5785b0c20b fix(ssl): file content return type 2021-12-13 16:08:41 +08:00
Zaiming (Stone) Shi d3d8b3a01f fix: allow listener not_found before restart 2021-12-12 22:18:56 +01:00
Zaiming (Stone) Shi 38ac10d3e2 feat(plugins): add emqx_plugins skeleton
* added emqx_plugins app.
* emqx_plugins.erl is moved from emqx app to emqx_plugins app
  same for the test SUITE
2021-12-12 19:40:34 +01:00
Zaiming (Stone) Shi fcb7c2c70f docs: change <module> to {module} so markdown render will not crash
<module> is treated as an HTML tag
2021-12-12 19:34:29 +01:00
Zaiming (Stone) Shi 41aa958ac9 refactor(authn): root type as array of union
Deleted the wrapping union.
i.e. previously it's
union([union([...]), array(union([...]))]).
not it's
array(union([...]))

It is after-all a lazy type, and the dynamic check allows
single-elemented arrays not to have `[]` around it.

the old union of array of union schema was just adding confusion
2021-12-11 22:20:53 +01:00
Zaiming (Stone) Shi 7d36079f0a docs(connector): better namespace for http schema 2021-12-11 22:17:52 +01:00
Ilya Averyanov 59f1bfeaac
Merge pull request #6373 from savonarola/authz-backend-behaviours
chore(authz): encapsulate authz backend logic
2021-12-11 00:11:16 +03:00
Ilya Averyanov fddb28a4b0 fix(wss): update cowboy & ranch for OTP24 compatibility 2021-12-10 20:38:44 +03:00
JianBo He 5224da57dc
Merge pull request #6411 from HJianBo/gw-improve-apis-3
More precise gateway http-api examples
2021-12-10 16:52:49 +08:00
lafirest 82746134bb feat(emqx_slow_sus): ported emqx_slow_subs from v4.4 2021-12-10 16:42:55 +08:00
firest 8493b61cb5 refactor(eqmx_limiter): use the new hierarchical token bucket to replace the old ratelimit code 2021-12-10 14:37:23 +08:00
JianBo He fa11c6df9c test(gw): fix bad test cases 2021-12-10 10:41:38 +08:00
zhongwencool 64341fdc23 fix(swagger): remote api_secret field form api_key api 2021-12-09 17:57:57 +08:00
k32 98be2e15b1
Merge pull request #6405 from k32/sysmon
feat(system_monitor): Add system_monitor application
2021-12-09 10:22:40 +01:00
zhongwencool 0614c157ec fix(elvis): elvis warning 2021-12-09 16:42:10 +08:00
zhongwencool cf11f3f8ee feat(api-key): support api_key/secret 2021-12-09 16:42:10 +08:00
JianBo He 159ba63530 chore: upgrade grpc-erl to 0.6.4 2021-12-09 15:06:18 +08:00
JianBo He 5d4604701c chore: fix elvis warnings 2021-12-09 15:06:18 +08:00
JianBo He 34cc8cc7f5 chore: put the pool_size default value to avoid hot upgrade failure 2021-12-09 15:06:18 +08:00
JianBo He e7ccd88719 test(props): cover messages headers 2021-12-09 15:06:18 +08:00
JianBo He f132d09483 chore(exhook): parse pool_size 2021-12-09 15:06:18 +08:00
JianBo He 143c685452 feat(exhook): expose process pool_size for grpc client 2021-12-09 15:06:18 +08:00
JianBo He 2b7c311807 chore: upgrade grpc to 0.6.4 2021-12-09 15:06:18 +08:00
JianBo He 262fb13a71 chore(exhook): fix diaylzer warnings 2021-12-09 15:06:17 +08:00
JianBo He 20ba9d285f feat(exhook): expose headers for on_messages_publish hook 2021-12-09 15:06:17 +08:00
lafirest f25c8cdf0f
Merge pull request #6410 from lafirest/fix/lw_cmd_record
fix(emqx_lwm2m): limit the max size of cmd record
2021-12-09 14:57:01 +08:00
JianBo He e662628581 chore(gw): fix bad default value in query-string 2021-12-09 14:50:28 +08:00
lafirest c9cbaeff95 fix(emqx_lwm2m): limit the max size of cmd record 2021-12-09 14:39:11 +08:00
JianBo He 028f1fa71c chore(gw): fix listener examples 2021-12-09 11:33:09 +08:00
Shawn f7941cd2bb
Merge pull request #6387 from terry-xiaoyu/improve_connector_apis
Improve bridge and connector APIs
2021-12-09 10:54:55 +08:00
DDDHuang 7aa08a82d5
Merge pull request #6283 from DDDHuang/as_doc_example
fix: add auto subscription swagger example
2021-12-09 10:35:08 +08:00
Ilya Averyanov 1c9d867c5e chore(authz): add test for authz backend creation dry run 2021-12-08 21:38:10 +03:00
Ilya Averyanov b7d227d2ef chore(authz): move file backend to a separate module 2021-12-08 21:38:10 +03:00
Ilya Averyanov 056558e445 chore(authz): encapsulate authz backend logic 2021-12-08 21:38:10 +03:00
Thales Macedo Garitezi 2b5fe9179e
fix(live_conn): fix live connection count on race condition (5.0)
Port from #6406 to 5.0.

When multiple clients try to connect concurrently using the same
client ID, they all call `emqx_channel:ensure_connected`, increasing
the live connection count, but only one will successfully acquire the
lock for that client ID.  This means that all other clients that
increased the live connection count will not get to call neither
`emqx_channel:ensure_disconnected` nor be monitored for `DOWN`
messages, effectively causing a count leak.

By moving the increment to `emqx_cm:register_channel`, which is only
called inside the lock, we can remove this leakage.

Also, during the handling of `DOWN` messages, we now iterate over all
channel PIDs returned by `eqmx_misc:drain_down`, since it could be
that one or more PIDs are not contained in the `pmon` state.
2021-12-08 15:15:27 -03:00
k32 9fbc247e29 feat(system_monitor): Add system_monitor application 2021-12-08 16:34:10 +01:00
Shawn d76f82d3d2 fix(bridges): add new option 'enable' for bridge configs 2021-12-08 18:16:38 +08:00
JianBo He afb2cf19c2
Merge pull request #6297 from HJianBo/gw-improve-apis-2
Standardize HTTP-APIs return value and Status/Error Code
2021-12-08 18:02:13 +08:00
Tobias Lindahl 92f116afa4
Merge pull request #6393 from emqx/fix-takeover-race-on-enqueued-messages 2021-12-08 11:00:46 +01:00
JianBo He 7c261b5356 chore(gw): improve examples for _gateway_api 2021-12-08 17:18:39 +08:00
JianBo He e48f10d2d6 chore(gw): ignore needless args 2021-12-08 11:33:11 +08:00
JianBo He a937a3d4dc chore(gw): fix bad argument type 2021-12-08 11:13:00 +08:00
JianBo He 0eff5358ca chore(gw): add examples for gw&client api 2021-12-08 10:52:27 +08:00
Ilya Averyanov 2b0a3e8ba3 refactor(authn): unify password hashing 2021-12-07 23:38:26 +03:00
Zaiming (Stone) Shi 9276a5bc3c
Merge pull request #6394 from zmstone/fix-typo-in-hook-point-name
fix: typo takeovered -> takenover
2021-12-07 21:06:30 +01:00
Tobias Lindahl 14bef1ba31 refactor: make clauses safe for future reordering 2021-12-07 16:14:32 +01:00
Zaiming (Stone) Shi 5440b431a1 fix: typo takeovered -> takenover 2021-12-07 16:06:23 +01:00
Tobias Lindahl e6ecc6ca60 fix(emqx_channel): fix race condition in session takeover
Sessions must not enqueue messages when another process is taking over
the client id, since it already passed on the message queue in the
session state.

Without this fix, messages arriving after `{takeover, 'begin'} to a
channel with no connection (i.e., a persistent session) would be lost.
2021-12-07 14:17:59 +01:00
Shawn 12fbcd8181 feat(connector): add num_of_bridges to the response of GET /connectors 2021-12-07 15:09:08 +08:00
Shawn 098d8eacb7 fix(metrics): update testcases for plugin_libs_metrics 2021-12-07 10:12:04 +08:00
Shawn cc96880f18 fix(dialyzer): some dialyzer issue 2021-12-07 09:56:57 +08:00
Shawn 6903997b94 feat(connector): add new option 'mode' to mqtt connectors 2021-12-07 09:56:57 +08:00
Shawn 9b4fe87ed0 fix(bridges): update testcases for bridge APIs 2021-12-07 09:56:57 +08:00
Shawn e1794fbce6 fix(connector): create different schema for POST,PUT,GET methods 2021-12-07 09:56:57 +08:00
Shawn 56d46c80eb refactor(rule): generate swagger from hocon schema for /bridges 2021-12-07 09:56:57 +08:00
Shawn affe69afd6 fix(rules): update the test cases for rule APIs 2021-12-07 09:48:47 +08:00
Shawn 416b9f8d7c refactor(rule): generate swagger from hocon schema for /rules 2021-12-07 09:48:47 +08:00
Shawn 24bded99d5 refactor(metrics): rename speed to rate in emqx_plugin_libs_metrics 2021-12-07 09:47:53 +08:00
Shawn e07fce791f feat(connector): add examples to the swagger doc 2021-12-07 09:47:53 +08:00
Tobias Lindahl 4eef9a5bac
Merge pull request #6378 from emqx/fix-flaky-tests-persistent-session-SUITE
test(flaky_tests): fix persistent_session_SUITE
2021-12-06 14:01:34 +01:00
Thales Macedo Garitezi b6dede3679
Merge pull request #6372 from emqx/fix-typo-listeners
fix(listeners): fix typo in listener type

`emqx_listeners:{current,max}_conns` were matching on type `tcl`.
However, this type doesn't exist (it's not defined in
`?TYPES_STRING`). Therefore, this clause would never match.  It seems
that the intention was that it shouldbe `tcp`.
2021-12-06 09:06:54 -03:00
Tobias Lindahl 6ed3dbd39b test(persistent_session_SUITE): protect against non-atom sinks 2021-12-06 12:40:58 +01:00
Thales Macedo Garitezi 2d46ae7517
fix(typo): fix more typos and restore rate_limit config after test 2021-12-06 08:14:50 -03:00
Tobias Lindahl e62b094f3b test(snabbkaffe): Update to latest snabbkaffe
In particular, this should remove the flaky snabbkaffe failures in
persistent session SUITE where the snabbkaffe_nemesis is trying to
make an ets:lookup in a table that no longer exists.
2021-12-06 11:22:25 +01:00
JimMoen a6a1b288d2
Merge pull request #6365 from JimMoen/fix-vm-mem-info
fix(vm_mem): info calc and display
2021-12-06 17:47:52 +08:00
Tobias Lindahl 366c757959 test(flaky_tests): fix persistent_session_SUITE
Increase timeout on message receive and ensure more info when the test fails.
2021-12-06 10:42:19 +01:00
DDDHuang b0b2adca37
Merge pull request #6377 from DDDHuang/alram_api_info
fix(alarm): alarm message with .2f format
2021-12-06 17:42:02 +08:00
JianBo He d735e2e755 chore: delete 500 error 2021-12-06 16:34:00 +08:00
JianBo He f1dbeb4bda chore: fix bad result match 2021-12-06 16:34:00 +08:00
JianBo He dab3e14686 fix(gw): fix static checking 2021-12-06 16:34:00 +08:00
JianBo He 7f04837ad2 fix(gw): fix elvis warnings 2021-12-06 16:34:00 +08:00
JianBo He 3f35604d16 refactor(gw): fix gateway http api schema 2021-12-06 16:34:00 +08:00
JianBo He f3f3f12635 refactor: improve the _api_clieints implement 2021-12-06 16:33:59 +08:00
JianBo He 01c50992e9 refactor: improve gw api 2021-12-06 16:33:38 +08:00
JianBo He 169848c027 feat(gw): return the created/updated resource info 2021-12-06 16:32:46 +08:00
JianBo He bc012d6554 chore(gw): change lwm2m&coap api path 2021-12-06 16:30:59 +08:00
Zaiming (Stone) Shi 68a7c096b0
Merge pull request #6362 from zmstone/emqx-config-put-raw-with-env-vars
fix: config put raw with env vars
2021-12-06 09:22:32 +01:00
zhongwencool fbe66cb9f2
Merge pull request #6351 from zhongwencool/prometheus-accept-header
chore(HTTP): prometheus or json format by accept header
2021-12-06 16:12:54 +08:00
JianBo He 1ae1460e44
Merge pull request #6364 from HJianBo/stop_publish_clear_msg_for_5
Allow to stop publish clear message
2021-12-06 16:06:38 +08:00
Zaiming (Stone) Shi a1ad6098b3 build: generate document to dashboard priv dir 2021-12-06 08:52:10 +01:00
Zaiming (Stone) Shi 96de7e6b30 docs: refine authn and auth config docs 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi ed073f9966 test: fix emqx_rewrite_SUITE with new path info in schema check failure 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi e774e976a7 fix: ensure atom connector is loaded when emqx_conf_schema is loaded 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi 89614245f6 test: use macro for authentication root name 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi a1f79850b4 refactor: allow accessing override config paths 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi a723e41390 test: fix a flaky test which gets interfered by other test cases 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi 5525cb3262 refactor: only apply environment variables on base config 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi a681079ba7 refactor: remove special env override from schema
EMQX_NODE_NAME and EMQX_NODE_COOKIE are translated into
EMQX_NODE__NAME and EMQX_NODE__COOKIE
2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi d9a980c7e1 docs: fix config doc, avoid using <text> in doc body 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi e6e28f9002 docs: fix emqx_schema doc style 2021-12-06 08:00:42 +01:00
Zaiming (Stone) Shi bfc36efa0e fix: pin hocon 0.21.1 2021-12-06 08:00:37 +01:00
DDDHuang 1b0da3cc58 fix(alarm): alarm message with .2f format 2021-12-06 14:36:51 +08:00
tigercl 9e8b4ab237
Merge pull request #6298 from tigercl/feat/cluster-authn
feat(authn): support sync configuration in the cluster
2021-12-06 10:04:48 +08:00
Zaiming (Stone) Shi 867cc3c4ad test: ensure emqx_connector is loaded 2021-12-04 09:33:14 +01:00
Zaiming (Stone) Shi 5d3cb6ae1c refactor(authn): align authn config root name
authn configs are checked independently per-auth provider,
this was to make authn providers more plugable.

in order to make environment variable overrides work for authn,
we need to have a unified view of the config layout,
no matter from root level, or partially checking per-provider
config independently, i.e. we try to use the same config envelop.
2021-12-04 09:33:14 +01:00
Thales Macedo Garitezi bc8a0d7060
fix(listeners): fix typo in listener type
`emqx_listeners:{current,max}_conns` were matching on type `tcl`.
However, this type doesn't exist (it's not defined in
`?TYPES_STRING`). Therefore, this clause would never match.  It seems
that the intention was that it shouldbe `tcp`.
2021-12-03 15:55:51 -03:00
k32 dfa45c909a fix(crash_dump): Fix hocon schema related to the erlang crash dump 2021-12-03 18:45:45 +01:00
k32 fd9f6bd140 fix(crash_dump): Replace crash dump directory with a file 2021-12-03 18:18:24 +01:00
k32 954179b959
Merge pull request #6370 from k32/perf
chore(mria): Bump version to 0.1.4
2021-12-03 17:39:21 +01:00
k32 2ceb660344 chore(mria): Bump version to 0.1.4 2021-12-03 17:13:12 +01:00
zhouzb 1dfc37cd18 test(authn): fix test cases 2021-12-03 15:55:34 +08:00
JianBo He f645a4eada test(retainer): add test case for stopping publish clear msg 2021-12-03 14:42:27 +08:00
JianBo He 4b3b29873a feat(retainer): allow to stop publish clear msg 2021-12-03 14:41:42 +08:00
JimMoen d4246ad2f6 fix(vm): add literal_alloc memory calculation 2021-12-03 14:41:33 +08:00
JimMoen a7cd1ad30b fix(mgmt): node memory usage info 2021-12-03 14:41:33 +08:00
zhouzb e1bcbd012c feat(authn): support sync configuration in the cluster 2021-12-03 14:14:40 +08:00
zhongwencool 85a6f0f1e8
Merge pull request #6350 from zhongwencool/emqx-conf
feat: support eqmx_ctl cluster_call command.
2021-12-03 11:27:21 +08:00
zhongwencool 858891dbf5 fix: rewrite ct badmatch 2021-12-03 09:54:44 +08:00
zhongwencool f6c61189c4 chore(HTTP): prometheus or json format by accept header 2021-12-02 10:30:50 +08:00
JimMoen a58493f7a4 test(authz): fix test suite for schema check and app start 2021-12-02 10:12:10 +08:00
JimMoen effa3b8b90 style: make elvis happy 2021-12-02 10:12:10 +08:00
JimMoen d7ec368884 fix(authz): fix mongo resources create 2021-12-02 10:12:10 +08:00
JimMoen 68af284570 fix(authz): http source create and update 2021-12-02 10:12:10 +08:00
zhongwencool e7a7d64004 feat(conf): skip/fast_forward tnx_id via cluster_call cli 2021-12-02 09:30:27 +08:00
zhouzb b983a18cdf fix(auth): replace query with cmd 2021-12-01 20:24:43 +08:00
zhongwencool c1a7d7bede fix(conf): emqx_conf return nest error 2021-12-01 16:08:49 +08:00
zhongwencool 25c4f4aa4e
Merge pull request #6339 from zhongwencool/swagger-ct-test
chore(swagger): add test for sub fields
2021-12-01 14:12:14 +08:00
tigercl e888e1ef8d
Merge pull request #6338 from tigercl/fix/superuser
fix(authn): fix superuser when missing is_superuser_field
2021-12-01 14:00:56 +08:00
zhongwencool 1f6e2e7397 chore: add test for sub fields 2021-12-01 12:01:44 +08:00
zhouzb 6b8997cbda fix(authn): fix superuser when missing is_superuser_field 2021-12-01 09:23:36 +08:00