Commit Graph

2961 Commits

Author SHA1 Message Date
Shawn 658f819aab fix(bridges): keep multiple bridges from affecting each other on crash 2021-12-31 21:28:32 +08:00
Shawn f65eca4c47 fix(authn): update testcase for resource not running 2021-12-31 21:08:07 +08:00
Shawn 657ecef67b fix(resource): don't crash on resource stopped 2021-12-31 20:57:34 +08:00
Zaiming (Stone) Shi 071b03b29c refactor: statsd supervisor API no need to return error
Also to make dialyzer happy
2021-12-31 12:27:25 +01:00
zhongwencool 2a0d8752bd fix(statsd): already_started crash and diaylizer warning 2021-12-31 17:32:51 +08:00
zhongwencool d0e8285e93 fix(trace): download trace request not compelete but the zip file is delete sometimes 2021-12-31 17:32:07 +08:00
zhongwencool 87bff075ad fix(api_key): set api_key unexpired when update expired_at=undefined 2021-12-31 17:32:07 +08:00
JianBo He 231aeb6068 fix(statsd): fix dialyzer warnings 2021-12-31 16:41:35 +08:00
DDDHuang 54ea7a5871
Merge pull request #6582 from DDDHuang/config_bugfix
fix(config):  update configs in cluster
2021-12-31 16:03:41 +08:00
DDDHuang d27aecc985
Merge pull request #6590 from DDDHuang/delayed_b64
fix(delayed): base64 encode twice
2021-12-31 16:02:32 +08:00
lafirest 6404efd0ec fix(emqx_slow_subs): fix default expire value 2021-12-31 15:59:47 +08:00
Shawn e299d8d138 fix(rule): rules not triggered after the ingress mqtt bridge received some msg 2021-12-31 15:47:03 +08:00
Zaiming (Stone) Shi b09d0d8e4c
Merge pull request #6586 from zmstone/docs-explain-layered-config-at-the-top
docs: explain layered config at the top
2021-12-31 08:10:15 +01:00
JianBo He 2411c22b42 fix(gw): fix bad listners field on http response 2021-12-31 14:59:17 +08:00
JianBo He 990514a048 fix(gw): stop xml_object_db process synchronously 2021-12-31 14:32:03 +08:00
JianBo He 2a20f110b9 fix(gw): use emqx_http_lib to parse uri 2021-12-31 14:32:03 +08:00
DDDHuang 6cde540fd1 fix(test): close app at end_per_suite 2021-12-31 14:17:50 +08:00
DDDHuang 388fdc6057
Merge pull request #6589 from DDDHuang/statsd_cluster_config
fix(statsd): update config in cluster
2021-12-31 14:09:05 +08:00
DDDHuang d18a2ab57c fix(delayed): base64 encode twice 2021-12-31 14:07:34 +08:00
EMQ-YangM c7693246fe feat(emqx_resource): add health_ckeck process, it will periodically perform health checks, and print error logs and generate alarms when the checks fail. 2021-12-30 21:53:51 -08:00
JianBo He 6e6643f974 fix: fix dialyzer warnings 2021-12-31 12:51:35 +08:00
JianBo He ebbb473d7a refactor(topic-metrics): cluster supported 2021-12-31 12:50:23 +08:00
DDDHuang 3a38e8d68d fix(statsd): update config in cluster 2021-12-31 12:48:24 +08:00
DDDHuang 173ae46538 fix(auto_subscribe): bad test suite 2021-12-31 12:46:29 +08:00
DDDHuang a630044688 fix: update result by row_config 2021-12-31 12:46:29 +08:00
DDDHuang f0330d9334 fix(event_message): update config in cluster 2021-12-31 12:46:29 +08:00
DDDHuang 6c52fb4806 fix: code format 2021-12-31 12:46:29 +08:00
DDDHuang 23cf74d829 fix(delayed): update config in cluster 2021-12-31 12:46:29 +08:00
DDDHuang f19ccdfcde fix(auto_subscribe): update config in cluster 2021-12-31 12:46:29 +08:00
Shawn 9ba454a63d fix(bridge): filter the topic of received msgs got from remote MQTT broker 2021-12-31 12:00:43 +08:00
Shawn 14ee053a0e
Merge pull request #6563 from terry-xiaoyu/bridge_bug_fixes_1
Bridge bug fixes 1
2021-12-31 11:57:28 +08:00
lafirest 922fc0ad9f
Merge pull request #6578 from lafirest/fix/retainer_config
fix(emqx_retainer): fix config update error
2021-12-31 11:13:44 +08:00
lafirest 86001765cb fix(emqx_retainer): fix config update error 2021-12-31 09:56:25 +08:00
Ilya Averyanov 2ffcaad41e chore(authz): increase coverage 2021-12-31 01:24:18 +03:00
Thales Macedo Garitezi c29bc126ef
Merge pull request #6507 from emqx/build-with-mix-mkII
Build with Elixir Mix Release
2021-12-30 15:00:38 -03:00
Zaiming (Stone) Shi d88a648092 docs: update config doc
Explain layered style at the very top of the doc
2021-12-30 18:38:01 +01:00
Zaiming (Stone) Shi d191f7cf7e chore: update to hocon 0.22.2 2021-12-30 18:36:26 +01:00
Shawn 94a5965560 fix(rule): dead lock when update configs for rules 2021-12-31 00:03:03 +08:00
Shawn 626a4c4713 fix(machine): some apps not restarted after joining into the cluster 2021-12-31 00:03:03 +08:00
Shawn a42ab3d9da fix(rule): use emqx_conf:update/3 to make changes to all nodes 2021-12-31 00:03:03 +08:00
Shawn ea2d4674df fix(resource): metrics were cleared after updating the resource 2021-12-31 00:03:03 +08:00
Shawn e2d899ad6e fix(bridge): HTTP reqeust crash if using GET an DELETE method 2021-12-31 00:03:03 +08:00
Shawn 9d733c2ec5 fix(resource): typos on restart a resource 2021-12-30 23:48:41 +08:00
Shawn d11cf6ad64 fix(bridges): store connector name and bridge name to config files 2021-12-30 23:48:41 +08:00
Shawn 110ae62b24 fix(bridge): don't concat names into ids 2021-12-30 23:48:41 +08:00
Shawn 14089a572e fix(bridge): changes timeouts from 30s to 15s 2021-12-30 23:48:41 +08:00
Shawn aefcd6275b fix(bridges): ingress MQTT bridges didn't increase counters on msg received 2021-12-30 23:48:41 +08:00
Shawn c23436166b fix(bridge): HTTP connector should failed on non-200 status codes 2021-12-30 23:48:41 +08:00
Shawn ea1aaa9806 fix(bridge): remove clientid config from MQTT bridges
Don't allow the user provide the clientid for connecting the remote broker.
We generate the clientid using the bridge id and node name.
2021-12-30 23:48:41 +08:00
zhongwencool d16362af96
Merge pull request #6547 from zhongwencool/trace-formatter
feat(trace): replace logger_formatter by emqx_trace_formatter
2021-12-30 23:40:59 +08:00
zhongwencool 489fb7f806 fix(trace): copy binary:encode_hex/2 from binary.erl 2021-12-30 21:38:23 +08:00
JimMoen ff3707fbdc
Merge pull request #6565 from JimMoen/fix-redis-servers-field
fix(connector): redis cluster `servers` field
2021-12-30 19:33:12 +08:00
Ilya Averyanov c98e0efdad chore(mongodb): update client 2021-12-30 13:57:48 +03:00
DDDHuang c2e36b041b fix: dashboard return type 2021-12-30 18:53:09 +08:00
Ilya Averyanov 7208669946 chore(authn): reduce number of pre-generated certificates, refactor
tests
2021-12-30 13:49:14 +03:00
lafirest 0e62a6772c fix(emqx_slow_subs): fix config update error 2021-12-30 16:58:33 +08:00
JimMoen da68dfc8f4 test(authz): `servers` field use string 2021-12-30 15:38:04 +08:00
JimMoen 8cb9482542 fix(authz): `servers` field use string 2021-12-30 15:37:53 +08:00
JimMoen 39c29b2396 fix(connector): redis cluster `servers` field 2021-12-30 15:37:53 +08:00
lafirest f466a9f2de fix(emqx_slow_subs): fix timer management error 2021-12-30 15:30:25 +08:00
lafirest c6cc92c608
Merge pull request #6575 from lafirest/fix/slow_subs_api
fix(emqx_slow_subs): limit the max size of top-k table
2021-12-30 14:53:37 +08:00
DDDHuang bd241f4220
Merge pull request #6568 from DDDHuang/api_authbeta3
fix(api): auth failed with basic type check
2021-12-30 14:53:16 +08:00
lafirest ed086728cf fix: Revert "fix(emqx_retainer): add support for RAP falg"
This reverts commit b80a01554b.
2021-12-30 11:32:08 +08:00
lafirest d60c586bfb fix(emqx_slow_subs): limit the max size of top-k table 2021-12-30 10:38:32 +08:00
DDDHuang db9d269034 fix(dashboard): batter auth failed response message 2021-12-30 10:29:20 +08:00
JianBo He e8acec7f56
Merge pull request #6549 from HJianBo/gw-review-r3
- Add endpoint_name, like_endpoint_name, gte_lifetime, lte_lifetime query parameters for Swagger docs
- Optimize the atomicity of the gateway creation procedure and reduce the side effects if creation failed
2021-12-30 09:54:54 +08:00
Thales Macedo Garitezi 3fc8d4049f
Merge pull request #6571 from emqx/fix-sticky-shared-sub-conf
fix(config): add missing enum constructors for `shared_subscription_strategy`

Currently, there's [code](048afa6d2d/apps/emqx/src/emqx_shared_sub.erl (L270-L279)) and tests that expect the possibility of
values `sticky`, `hash_topic` and `hash_clientid` in the
`node.shared_subscritpion_strategy` configuration, but the schema enum
does not contain those constructors.

There's also a `hash` constructor expected, but apparently it's just for backwards compatibility (same as `hash_clientid`), so maybe we can skip it in 5.0
2021-12-29 15:08:49 -03:00
zhongwencool 43141dffee
Merge branch 'release-5.0-beta.3' into trace-formatter 2021-12-29 22:37:59 +08:00
zhongwencool b8bb5ff738 fix(trace): delete duplicate topic from msg 2021-12-29 22:33:34 +08:00
Thales Macedo Garitezi 048afa6d2d
fix(config): add missing enum constructors for `shared_subscription_strategy`
Currently, there's code and tests that expect the possibility of a
value `sticky`, `hash_topic` and `hash_clientid` in the
`node.shared_subscritpion_strategy` configuration, but the schema enum
does not contain those constructors.
2021-12-29 10:34:20 -03:00
Ilya Averyanov 2b3d3ebb0a chore(authn): test JWT authn with HTTPS key server 2021-12-29 16:24:57 +03:00
Ilya Averyanov f230c20521 chore(authn): test HTTPS authn 2021-12-29 16:24:57 +03:00
Ilya Averyanov a7ca6cb39f chore(authn): test MongoDB authn via ssl connection 2021-12-29 16:24:52 +03:00
Zaiming (Stone) Shi f5049b696a docs: fix a misspell in emqx_mgmt_cli 2021-12-29 13:29:51 +01:00
DDDHuang 34dfc327f3 fix(api): auth failed with basic type check 2021-12-29 19:34:13 +08:00
zhongwencool 04313dc044 fix(trace): download trace api not working 2021-12-29 19:03:54 +08:00
zhongwencool 121d906992 chore(log): add SLOG/3 to add meta info 2021-12-29 14:48:48 +08:00
JianBo He 6280c8c9f0 chore: fix dialyzer warnings 2021-12-29 14:02:45 +08:00
tigercl d769401869
Merge pull request #6555 from tigercl/chore/new-epgsql
chore: support prepared query in postgresql connector
2021-12-29 13:37:29 +08:00
zhongwencool 8b5b3a448a fix(test): trace_handler ct fail 2021-12-29 11:06:31 +08:00
Thales Macedo Garitezi 8f4453fedb
fix(gateway): undo move headers to root `include`
Keeping headers in the src directory makes sense for the interfaces we
don't want to expose outside, but the `include_lib` directive fails to
compile under Mix. Changing `include_lib("emqx_gateway/src/...")` to
`-include("src/...")` pleases both Mix and Rebar.
2021-12-28 17:34:47 -03:00
zhongwencool 4b6bba11eb feat(trace): struct log for trace 2021-12-29 00:26:54 +08:00
zhongwencool b78cfa0a1c fix(api_key): support expired_at never expired when undefined 2021-12-29 00:22:59 +08:00
zhongwencool 0712fc1e75 chore(schema): remove emqx_schema:unicode_binary/0 2021-12-28 23:50:12 +08:00
Thales Macedo Garitezi 0020cf592f
Merge remote-tracking branch 'origin/master' into build-with-mix-mkII 2021-12-28 11:28:40 -03:00
Ilya Averyanov f8ffb9c021
Merge pull request #6506 from savonarola/test-authz-mnezia
Test HTTP, Mria, File authorization
2021-12-28 15:53:37 +03:00
zhongwencool 47be5b9ef2
Merge pull request #6539 from zhongwencool/fix-override-conf-not-load
fix(conf): can't include local/overide-conf file
2021-12-28 18:03:56 +08:00
Zaiming (Stone) Shi 0f14f73a50
Merge pull request #6552 from zmstone/fix-5.0-portable-shebang
fix: portable shebang
2021-12-28 10:51:15 +01:00
Shawn 3a39db4f97
Merge pull request #6550 from terry-xiaoyu/fix_utf8_topic_rewrite
Fix utf8 topic rewrite
2021-12-28 17:08:23 +08:00
Shawn 1d4be368cb
Merge pull request #6556 from terry-xiaoyu/fixes_for_bridges
Bug fixes for data bridges
2021-12-28 17:07:54 +08:00
zhongwencool e44f7de596 chore(conf): update hocon to 0.22.1 2021-12-28 16:17:19 +08:00
Ilya Averyanov 9363b6110e chore(authz): make test http server more robust 2021-12-28 11:06:52 +03:00
tigercl 6a686c3314
Merge pull request #6503 from tigercl/feat/authz-config
feat(authz): support sync configuration in the cluster
2021-12-28 14:39:56 +08:00
Shawn 07997ab865 fix(bridge): Bridges should send a JSON message if payload template not set 2021-12-28 14:18:14 +08:00
Shawn 4406589980 fix(bridge): time unit error for MQTT keepalive 2021-12-28 14:12:28 +08:00
lafirest e48df910d8
Merge pull request #6493 from lafirest/refactor/exhook
refactor(emqx_exhook): refactore exhook and add api module
2021-12-28 14:00:19 +08:00
zhouzb 487b84f166 test: fix test case 2021-12-28 11:37:51 +08:00
Shawn eb992ad2ad fix(bridge): add test cases for sending msgs via http bridge 2021-12-28 11:35:47 +08:00
zhouzb 1a22d5ca46 fix: fix undef function call 2021-12-28 11:21:41 +08:00
Shawn bf2392f682 chore(typerefl): update typerefl to 0.8.6 2021-12-28 10:52:30 +08:00
lafirest c68499e39a fix(api): fix the payload in the result of the api of emqx_retainer & emqx_dealy 2021-12-28 10:25:53 +08:00
lafirest e2240bdf03 feat(emqx_limiter): improve burst implementation 2021-12-28 10:13:55 +08:00
Shawn 1cd226c18b fix(bridge): make direction defaults to egress if not provided 2021-12-28 10:12:39 +08:00
zhouzb d26042703e chore: provide prepared_query and equery 2021-12-28 09:52:31 +08:00
lafirest afe51446b0
Merge pull request #6538 from lafirest/fix/rap
fix(emqx_retainer): add support for RAP falg
2021-12-28 09:31:57 +08:00
zhouzb fc89fb0f8a chore: using prepared query in postgresql connector 2021-12-28 09:16:00 +08:00
Zaiming (Stone) Shi 68eb13d478 fix: portable shebang 2021-12-27 15:45:12 +01:00
JianBo He 8203b1f328 refactor(gw): simplify massive repeated codes 2021-12-27 16:50:39 +08:00
Shawn e9c2b5d1cf fix(rewrite): don't check and translate body for topic rewrite APIs 2021-12-27 16:17:59 +08:00
Ilya Averyanov e2e2c98679 chore(authn): test Mysql authn via ssl connection 2021-12-27 11:16:31 +03:00
Ilya Averyanov 6de89d1207 chore(authn): test PostgreSQL authn via ssl connection 2021-12-27 11:16:31 +03:00
Ilya Averyanov 520629d6e2 chore(authn): test Redis authn via ssl connection 2021-12-27 11:16:31 +03:00
JianBo He 76b35910b7 chore(gw): fix elvis warnings 2021-12-27 16:03:31 +08:00
JianBo He fca5a3bc21 chore(gw): add cli testcases 2021-12-27 16:03:31 +08:00
zhongwencool 668180388c feat(trace): replace logger_formatter by emqx_trace_formatter 2021-12-27 14:40:40 +08:00
tigercl d781dc73a5
Merge pull request #6536 from tigercl/fix/authn-ssl
fix(authn): attempt to convert certificate content only when TLS is enabled
2021-12-27 09:50:11 +08:00
JianBo He 523b5761b7
Merge pull request #6527 from HJianBo/fix-receive-maximum-50
fix: disconnect the client due to exceed receive-maximum packets
2021-12-27 09:13:24 +08:00
Thales Macedo Garitezi cc652c3343
ci(mix): add elixir deps check
Checks whether there are discrepancies between rebar3 and mix deps.
2021-12-24 14:44:49 -03:00
Ilya Averyanov 2bada0bab8 chore(authz): test Mria authz 2021-12-24 16:18:46 +03:00
Ilya Averyanov d75e0104cc chore(authz): test file authz with real files 2021-12-24 16:18:41 +03:00
Ilya Averyanov e0f860d7d9 chore(authz): fix HTTP authz, cover with tests 2021-12-24 16:18:36 +03:00
zhongwencool 0b099432c5 fix(conf): can't include local/overide-conf file 2021-12-24 19:20:36 +08:00
lafirest b80a01554b fix(emqx_retainer): add support for RAP falg 2021-12-24 18:03:55 +08:00
zhongwencool 6326e436d3
Merge pull request #6532 from zhongwencool/fix-banned-bad-peerhost-crash
fix(banned): crash by bad peerhost; add banned http API tests
2021-12-24 17:04:26 +08:00
tigercl 41694b7b34
Merge pull request #6526 from emqx/release-5.0-beta.3
Sync 5.0-beta.3 fixes into master
2021-12-24 13:49:13 +08:00
zhouzb 72a27e9d8f fix: fix undefined function call 2021-12-24 13:44:28 +08:00
zhouzb 9769ddeb14 fix(authn): attempt to convert certificate content only when TLS is enabled 2021-12-24 13:40:13 +08:00
zhongwencool 33a6568654 fix(banned): crash by bad peerhost; add banned http API tests 2021-12-24 10:56:41 +08:00
lafirest d2d50443ce fix(emqx_retainer): use base64 to encode payload in api's result 2021-12-24 10:48:12 +08:00
lafirest 8c42b623c6
Merge pull request #6529 from lafirest/fix/retain_payload
fix(emqx_retainer): use base64 to encode payload in api's result
2021-12-24 10:26:00 +08:00
lafirest b6755d5953 fix(emqx_retainer): use base64 to encode payload in api's result 2021-12-24 10:22:10 +08:00
Thales Macedo Garitezi 11c169501e
chore(mix): fix app dependencies and reboot apps (emqx_conf)
While declaring `emqx_conf` as an application dependency of
`emqx_resource` worked for releases, it messed up the startup
relationship during tests.  Since only removing `emqx_conf` from the
`applications` key in `emqx_resource` breaks the list of apps that
need to be rebooted on config changes (since `emqx_conf` is not on any
apps dependencies list, it was not being added to the final
topologically sorted list), we now always add it as a vertex to ensure
its presence there.  This (apparently) makes the rebar3 release, the
mix release and test runs behave normally.
2021-12-23 13:48:28 -03:00
k32 bd31b52e35 chore(ekka): Bump version to 0.11.2 2021-12-23 15:29:35 +01:00
Spycsh 41e373dbc4
Merge pull request #6478 from Spycsh/slog
chore: refactor ?SLOG
2021-12-23 22:21:33 +08:00
lafirest 7c9c7b6a60 refactor(emqx_exhook): refactore exhook and add api module 2021-12-23 17:51:13 +08:00
JianBo He 52502e29c3 fix: disconnect the client due to exceed receive-maximum packets
As described in the 5.0 specification, we should disconnect clients that
exceed the receive-maximum limit.

> If it receives more than Receive Maximum QoS 1 and QoS 2 PUBLISH packets
where it has not sent a PUBACK or PUBCOMP in response, **the Server uses a
DISCONNECT packet with Reason Code 0x9**

fix: #6447
2021-12-23 14:06:11 +08:00
zhouzb 80c1128b7a fix(authz): fix deadlock issues 2021-12-23 11:45:58 +08:00
JianBo He 967989075c
Merge pull request #6512 from terry-xiaoyu/fix_rule_api_swagger
Fix rule api swagger
2021-12-23 10:37:53 +08:00
Thales Macedo Garitezi 238895a301
chore(mix): correctly define release version
With this, we no longer need the last hack in `mix_release.sh`.

A version schema with 3 parts (Major.Minor.Patch) is required by
Elixir:

https://hexdocs.pm/elixir/1.13.1/Version.html#module-versions
2021-12-22 15:43:43 -03:00
Thales Macedo Garitezi e137555052
fix(tests): fix include paths that were moved 2021-12-22 15:15:19 -03:00
Thales Macedo Garitezi 68cbded7c6
fix(dialyzer): fix dialyzer warnings
Adding hocon to emqx solves the issue, since other offending apps
depend on emqx.  The warnings were that it was not able to find info
on the `hocon_schema` behaviour.

Example:
```
Callback info about the hocon_schema behaviour is not available
```
2021-12-22 15:15:19 -03:00
JianBo He b846193556
Merge pull request #6518 from HJianBo/gw-review-r2 2021-12-22 22:10:18 +08:00
Thales Macedo Garitezi 34d6dbca61
chore(mix): simplify build by using rebar3 for umbrella apps
By treating the apps in the umbrella as dependencies to be managed and
built by rebar3, we can simplify the maintenance of the release, at
the cost of increased build times: using Mix as before, it could track
changed files better than using rebar.  But the complexity and
possibility of discrepancies make it using rebar much more compelling.
2021-12-22 09:40:01 -03:00
JianBo He 1bcdbf3a06 chore(gw): make some fields required 2021-12-22 16:49:34 +08:00
JianBo He 6d4aac1600 chore(gw): improve http error messages 2021-12-22 16:49:33 +08:00
JianBo He 3fd9061418 fix(gw): save coap channel info 2021-12-22 16:49:33 +08:00
zhongwencool 2e10a5700c
Merge pull request #6515 from zhongwencool/fix-bann-utf8
fix(banned): create banned with utf8 failed by 500 response
2021-12-22 16:08:34 +08:00
zhongwencool b11a15fa00 fix(banned): create banned with utf8 failed by 500 2021-12-22 14:29:15 +08:00
Shawn cd4227b851 fix(rules): don't show the module name 'emqx_rule_outputs' in outputs 2021-12-22 10:26:22 +08:00
Shawn 9b4b3d2e8c fix(rules): make the 'name' field of POST /rules mandatory 2021-12-22 10:26:22 +08:00