Commit Graph

5563 Commits

Author SHA1 Message Date
zhongwencool 77a5125366
chore: update apps/emqx_management/src/emqx_mgmt_api_trace.erl
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2022-10-17 09:45:00 +08:00
Zhongwen Deng 58b990064c chore: update bpapi.verions 2022-10-17 09:43:17 +08:00
Zhongwen Deng 4d9cae0387 feat: get trace file's detail via /trace/:name/log_detail 2022-10-17 09:43:15 +08:00
zhongwencool bc0cc80d32
Merge pull request #8924 from zhongwencool/authn-config-check
fix: authentication should be array, not struct
2022-10-17 09:14:05 +08:00
Ivan Dyachkov f2db35db39 chore: delete unused function clause 2022-10-14 10:46:11 +02:00
zhongwencool bfc780f95b
Merge branch 'master' into authn-config-check 2022-10-14 16:36:50 +08:00
Ivan Dyachkov 5a8cab46a9 test: update publish api test suite 2022-10-14 10:17:22 +02:00
Zaiming (Stone) Shi c2f176c1e7 fix(api): publish API only returns message ID 2022-10-14 09:49:46 +02:00
Zhongwen Deng a02fea0a19 chore: update bpapi.verions 2022-10-14 15:16:09 +08:00
Zhongwen Deng 4579d7fff8 feat: get trace file's detail via /trace/:name/log_detail 2022-10-14 11:38:39 +08:00
Thales Macedo Garitezi 2d01726b22 fix: account calls when resource is not connected as matched 2022-10-13 15:32:04 -03:00
Thales Macedo Garitezi 1b2b629cdd feat: emit telemetry events for all resource worker metrics 2022-10-13 15:32:04 -03:00
JianBo He 56000cbf3e refactor: more rigorous checking of flapping to improve stability of the system
port: https://github.com/emqx/emqx/pull/9045
2022-10-12 10:03:47 +08:00
JianBo He 5a66c8b117
Merge pull request #9082 from HJianBo/update-api-docs
chore(i18n): enrich HTTP API desc
2022-10-12 09:36:11 +08:00
Thales Macedo Garitezi f0ff32c031 test: fix tests after counter changes 2022-10-11 17:45:48 -03:00
Thales Macedo Garitezi 357e5919ce chore: add copyright disclaimer 2022-10-11 09:51:16 -03:00
Kjell Winblad 57270fb8fc feat: add support for counters and gauges to the Kafka Bridge
This commit adds support for counters and gauges to the Kafka Brige.
The Kafka bridge uses [Wolff](https://github.com/kafka4beam/wolff) for
the  Kafka connection. Wolff does its own batching and does not use the
batching functionality in `emqx_resource_worker` that is used by other
bridge types. Therefore, the counter events have to be generated by
Wolff. We have added
[telemetry](https://github.com/beam-telemetry/telemetry) events to Wolff
that we hook into to change counters and gauges for the Kafka bridge. The
counter called `matched` does not depend on specific functionality of
any bridge type so the updates of this counter is moved higher up in the
call chain then previously so that it also gets updated for Kafka
bridges.
2022-10-10 14:40:57 -03:00
firest b4eb0f18f9 fix(shared_subs): rename redispatch function name and simply test case 2022-10-10 18:06:55 +08:00
firest f0769cb765 fix(shared_sub): kick session should not cause session message redispatch 2022-10-10 13:52:09 +08:00
Zaiming (Stone) Shi 63774ba5d3
Merge pull request #9104 from zmstone/1005-fix-shared-sub-dispatch
fix(shared): re-dispatch inflight (QoS1) and mqueue messages
2022-10-09 11:41:30 +02:00
lafirest d600c870af
Merge pull request #9060 from lafirest/fix/delayed_pub
fix(delayed): Improve time precision of delayed messages
2022-10-09 10:39:31 +08:00
Zaiming (Stone) Shi 1c29e2806a test(shared): add a test case to ensure shared dispatch retry
to ensure retry will not enter a dead loop
2022-10-05 22:33:49 +02:00
Zaiming (Stone) Shi e8279a02ef fix(shared): re-dispatch inflight (QoS1) and mqueue messages
when session terminates (not due to take over)
shared delivery should be re-dispatched to other members
in the group
2022-10-05 22:33:48 +02:00
Zaiming (Stone) Shi abb941d161 docs: remove port 8081
port 8081 is the management API in 4.x
in 5.0 api port is 18083 (same as for serving the dashboard)
2022-10-05 17:15:19 +02:00
JianBo He d43b680f14 chore: add comments to explain`/status` cant be found in swagger docs 2022-10-01 09:40:27 +08:00
JianBo He 7d7b05ae4e chore: update app.src 2022-09-30 23:32:23 +08:00
JianBo He 93d3fec73f chore(i18n): correct tags name 2022-09-30 23:14:19 +08:00
JianBo He 6589a66710 Merge branch 'update-api-docs' of github.com:HJianBo/emqx into update-api-docs 2022-09-30 23:05:07 +08:00
JianBo He a84c6f938c chore(swagger): caption tags name 2022-09-30 23:04:26 +08:00
JianBo He 9c6c13789e
chore: update apps/emqx_gateway/i18n/emqx_gateway_api_authn_i18n.conf
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2022-09-30 23:03:01 +08:00
JianBo He 8292572da6
chore: apply suggestions from code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2022-09-30 23:02:28 +08:00
JianBo He d17c473dc2 chore(i18n): fix the bad desc for statsd 2022-09-30 18:57:08 +08:00
JianBo He 5b219deb4a Merge remote-tracking branch 'zibo/chore/api-desc' into update-api-docs 2022-09-30 18:48:05 +08:00
zhouzb 0975bf21a9 docs: improve api desc for statsd 2022-09-30 17:55:12 +08:00
JianBo He 5fa1e63392 chore(gw): update HTTP API docs for gateway 2022-09-30 17:44:36 +08:00
zhouzb f08982be93 chore: improve api desc for alarms 2022-09-30 15:34:49 +08:00
zhouzb d1a15f1a43 chore: improve api descs for banned 2022-09-30 15:05:01 +08:00
Zaiming (Stone) Shi 242a69359c
Merge pull request #9047 from HJianBo/remove-colon-port
Improve the `ip_port()` definition and `bind` print format
2022-09-29 05:56:50 +02:00
JianBo He 834910b6f6 chore: update app.src 2022-09-27 17:24:44 +08:00
JianBo He 39ad172377 chore: replace ip_port() wuth host_port() for all connectors 2022-09-27 15:19:42 +08:00
JianBo He 6581911d5c Merge remote-tracking branch 'stone/0922-upgrade-ip-port-schema' into remove-colon-port 2022-09-27 15:08:27 +08:00
Zaiming (Stone) Shi 00e4b4da5a fix(schema): support hostname.domain:port for mqtt bridge 2022-09-27 09:02:49 +02:00
firest 56816559d9 chore: bump emqx_modules && update CHANGES 2022-09-27 13:39:25 +08:00
firest 6836a502ac fix(delayed): Improve time precision of delayed messages
Use milliseconds internally in emqx_delayed to store the publish time, improving precision
2022-09-27 13:24:54 +08:00
JianBo He 607f68bab1 chore(schema): don't automaticly add ipv4 address if configured as
`:1883`
2022-09-27 13:01:39 +08:00
JianBo He 03eaa07c02 chore: add more comments for bind display perference 2022-09-27 12:51:00 +08:00
Thales Macedo Garitezi 129f09f88b chore: remove extra information from denied lwt log 2022-09-26 15:23:59 -03:00
Shawn 19e3f5919e chore: release e5.0.0-beta.4 2022-09-26 19:54:01 +08:00
Shawn dfe14be8b2 chore: bump app vsns 2022-09-26 18:23:28 +08:00
JianBo He c9904e78aa chore: remove colon port in HTTP-API and CLI
In the https://github.com/emqx/emqx/pull/8571, we add a colon before the
port. But it was not popular, so we removed ta in this PR
2022-09-26 17:30:54 +08:00
Shawn 4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
Zaiming (Stone) Shi f6ac4c3a76
Merge pull request #8798 from zmstone/0815-feat-add-kafka-connector
feat: Add Kafka connector
2022-09-24 22:57:50 +02:00
Shawn d24441b204 fix(bridges): some issues found in code review 2022-09-23 18:32:39 +08:00
Zaiming (Stone) Shi 7b601bf970 chore: delete bad parse_bridge function clause 2022-09-23 11:53:02 +02:00
JimMoen 2984397e73 chore: apps vsn bump 2022-09-23 17:09:15 +08:00
JimMoen fc3fdfb994 fix(build): apply PR#8956 changes 2022-09-23 17:09:10 +08:00
Kjell Winblad adc67b165b test: test cases for Kafka bridge REST API 2022-09-23 10:09:07 +02:00
JimMoen 85835256f1 Merge tag 'v5.0.8' into merge-release-v5.0.8-into-master 2022-09-23 16:06:44 +08:00
Zaiming (Stone) Shi 516d60c7da build: fix deps consistency check 2022-09-23 09:00:17 +02:00
firest 5ef1a978bd chore: bump slow subs version && update CHANGES 2022-09-22 17:59:11 +08:00
firest 89af9c4a5f fix(slow_subs): fix timestamp unit error and update config not work
1. The unit of `deliver_begin_at` is incorrectly used the `seconds`, the resulting is incorrect when `stats_type` is `internal` or `response`
2. The config update doesn't actually take effect, the `load` function always used the old config
2022-09-22 17:57:51 +08:00
Kjell Winblad 9ea0147a8c feat(rule engine jq function): config for changing implementation
This commit adds a rule engine configuration option for changing the
implementation module used for the rule engine function jq. The two
options are `jq_port` (uses Erlang port programs to interact with jq)
and `jq_nif` (uses an Erlang NIF library to interact with jq).

Thanks to @terry-xiaoyu (Xinyu Liu <506895667@qq.com>) for Chinese
translations
2022-09-22 09:02:30 +02:00
JimMoen 6d37fc8dcb
Merge pull request #9012 from JimMoen/rm-influxdb-udp
Hide influxdb udp bridge.
2022-09-22 10:33:05 +08:00
Shawn b325633390 refactor(resource): resume from queue/inflight-window with async-sending and batching 2022-09-21 22:58:47 +08:00
Ilya Averyanov c11afc357e fix(auth): use empty strings for absent placeholder values 2022-09-20 15:20:55 +03:00
zhongwencool 8c718d891f
Merge pull request #9002 from zhongwencool/fix-listener-crash
fix: /listeners API crash when there are nodes still in initial conf
2022-09-20 15:25:03 +08:00
JimMoen 6788881642 fix: hide influxdb udp config 2022-09-20 14:46:59 +08:00
Zhongwen Deng 2551c51a8c fix: ct failed 2022-09-20 11:37:49 +08:00
ieQu1 62957b298b chore(ekka): Bump version to 0.13.5 2022-09-19 18:21:32 +02:00
Zhongwen Deng 0241185225 chore: Add test coverage for listeners API 2022-09-19 18:25:25 +08:00
Zhongwen Deng 386110ae1e fix: /listeners API crash when some nodes still in init configuration 2022-09-19 15:24:36 +08:00
Xinyu Liu cc327629c3
Merge pull request #8992 from terry-xiaoyu/fast_resume_from_inflight_full
Fast resume from inflight full
2022-09-18 17:49:04 +08:00
Thales Macedo Garitezi c20ad3733a fix: check for authorization on topic before publishing last will testament
fixes #8978

Without checking for authorization, a client can, on abnormal
termination, publish a message to any topic, including `$SYS` ones.
2022-09-16 17:31:22 -03:00
Thales Macedo Garitezi dca522d7d3 test: add tests for publishing lwt when deny_action is disconnect 2022-09-16 15:11:54 -03:00
Thales Macedo Garitezi 87ab2e3a2d fix: check conn state before sending will message 2022-09-16 15:11:54 -03:00
Shawn 9aa7e826cb refactor(resource): fast resume resource worker if inflight msgs are ACKed 2022-09-17 00:34:30 +08:00
zhouzb ba37ab4938 chore: bump vsn to 5.0.8 2022-09-16 19:32:17 +08:00
Shawn 8307f04c2e refactor(resource): save inflight size into the ETS table 2022-09-16 16:52:08 +08:00
JimMoen a660004fa4 fix(bridge): mqtt bridge client info `username` & `password`
for non configured `username` and `password`, default values `<<>>` should be
used. emqtt needs the values.
2022-09-16 15:09:51 +08:00
JimMoen 0336b935a0
Merge pull request #8901 from JimMoen/fix-bridge-api
- for connectors using ecpool, parse connect error `Reason` when start pool failed.
- Refine InfluxDB bridge desc.
2022-09-16 10:22:06 +08:00
zhongwencool 5056cbebf4 chore: update apps/emqx_authn/src/emqx_authn_app.erl
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-09-16 09:45:37 +08:00
Zhongwen Deng 6bd72fe5a1 fix: check authn's mechanism field 2022-09-16 09:45:37 +08:00
Zhongwen Deng 49829caaa0 fix: check authn(lazy type) config when boot 2022-09-16 09:45:36 +08:00
Zhongwen Deng 0b07561e9b fix: ensure authentication is array, not struct 2022-09-16 09:45:36 +08:00
firest 14b5977996 feat(authz): Support `cert_common_name` and `cert_subject` in Authz 2022-09-15 18:02:54 +08:00
zhouzb 8054144357 chore: bump vsn 2022-09-15 10:20:25 +08:00
Shawn d5d3972ff5 chore: add test cases for MQTT Bridge reconnecting 2022-09-15 10:19:33 +08:00
JimMoen 0390a5e547 fix(bridge): mysql bridge error case 2022-09-15 09:36:52 +08:00
Xinyu Liu d9c9a1dec3
Merge pull request #8958 from JimMoen/fix-ssl-drop-invalid-certs
fix(bridge/authz/exhook): no need to drop invalid certs
2022-09-15 09:01:08 +08:00
Shawn 4e211c12d3 fix(mqtt_bridge): return value of sending messages was discarded 2022-09-15 08:57:01 +08:00
JianBo He 8845da130e
Merge pull request #8957 from HJianBo/strip-double-quote
chore: support strip double quote in authn/authz
2022-09-14 17:35:41 +08:00
lafirest 9ffbff11c2
Merge pull request #8956 from lafirest/fix/redis_authn
fix(authn_redis): Add new clause for non-existent key check
2022-09-14 17:05:34 +08:00
firest 8590fef829 fix(authn_redis): Avoid duplicating check for non-existent keys 2022-09-14 16:30:59 +08:00
JianBo He a461375b30 chore: support strip double quote in authn/authz
more compatibility for https://github.com/emqx/emqx/pull/8827
2022-09-14 15:25:31 +08:00
Shawn 1c03c236f5 fix(mqtt_bridge): handle send_to_remote in idle state 2022-09-14 15:19:30 +08:00
Shawn f41adb0997 refactor: change some default values of resource_opts 2022-09-14 15:18:07 +08:00
firest 62fd955a0e fix(authn_redis): make dialyzer happy 2022-09-14 14:16:43 +08:00
JimMoen f018b8ab49 fix(bridge/authz/exhook): no need to drop invalid certs 2022-09-14 14:08:55 +08:00
firest 9e97760520 fix(authn_redis): Add new clause for non-existent key check
fix #8800
when the key not-existing, redis may return a list that all elements are `undefined`
2022-09-14 11:30:26 +08:00
zhongwencool 53bc6f47e7
Merge pull request #8948 from zhongwencool/deprecated-etc-dir
chore: deprecated etc_dir since 5.0.8
2022-09-14 10:05:10 +08:00
Zaiming (Stone) Shi 0c1595be02 feat: Add Kafka connector 2022-09-13 19:46:56 +02:00
JimMoen b5bf5d3adb fix: mqtt bridge payload default value 2022-09-13 18:58:01 +08:00
zhouzb 2c7d518c19
Merge pull request #8851 from zhongwencool/denny-update-local-override-conf
feat: deny hot updates to the configuration in local-override.conf
2022-09-13 18:49:51 +08:00
Zhongwen Deng 56c1ac5a82 chore: deprecated etc_dir since 5.0.8 2022-09-13 18:38:36 +08:00
JianBo He 9e07f074bf
Merge pull request #8866 from HJianBo/port-exhook-exproto-bugfixes
fix(exproto): avoid udp client process leaking
2022-09-13 18:29:41 +08:00
Zaiming (Stone) Shi 4534966e36 Merge remote-tracking branch 'origin/master' into dev/ee5.0 2022-09-13 12:16:54 +02:00
Shawn b9ae4ea276 refactor: rename some metrics for emqx_resource 2022-09-13 14:04:25 +08:00
JianBo He e282e3986f Merge branch 'using-erlang-system-time-5' into release-v5.0.8 2022-09-13 11:22:57 +08:00
JianBo He 522f650096 chore: apply review suggestions 2022-09-13 10:28:53 +08:00
JianBo He 44f8108228 chore: update app vsn 2022-09-13 09:15:50 +08:00
JianBo He 4384cae29e test: fix failed tests 2022-09-13 09:15:50 +08:00
JianBo He 533569ad2a chore: fix app vsn check 2022-09-13 09:15:50 +08:00
JianBo He fbc2130865 test: add test for idle_timeout parameter 2022-09-13 09:15:28 +08:00
JianBo He f8614196ac test: ensure udp client keepalive value getting right value 2022-09-13 09:15:28 +08:00
JianBo He ebb2824e15 test: ensure hooks has unloaded if grpc is blocked 2022-09-13 09:15:28 +08:00
JianBo He dfc6e34680 fix(exproto): avoid udp client process leaking
porting from v4.x:
- https://github.com/emqx/emqx/pull/8575
- https://github.com/emqx/emqx/pull/8628
- https://github.com/emqx/emqx/pull/8725
2022-09-13 09:15:28 +08:00
JianBo He a715573c64
Merge branch 'master' into using-erlang-system-time-5 2022-09-13 09:09:06 +08:00
firest 061e3c264b fix(authn_redis): fix that redis authn will deny the unknown users 2022-09-09 15:28:01 +08:00
firest a770447e75 fix(authn_redis): fix that redis authn will deny the unknown users 2022-09-09 14:53:31 +08:00
firest 5ba31cb192 fix(api): add listener create API from `POST /listeners/{type:name}` to `POST /listeners`
Old:
    API: POST /listeners/{type:demo}
    Body: {"type" : "tcp", "id" : "tcp:demo"}
New:
    API: POST /listeners
    Body: {"type" : "tcp", "name" : "demo"}
2022-09-09 10:46:42 +08:00
firest 9368ae9fa8 test(retainer): add test case for start after retianer is disabled 2022-09-09 10:46:42 +08:00
firest 8768884e2b chore: bump emqx_retainer version && update CHANGES-5.0.md 2022-09-09 10:46:42 +08:00
firest 42e2d00cfa fix(retainer): fix that EMQX can't start when the retainer is disabled 2022-09-09 10:46:41 +08:00
lafirest de06d1c539
Merge pull request #8876 from lafirest/fix/listeners_api
fix(api): add listener create API from `POST /listeners/{type:name…
2022-09-09 10:14:46 +08:00
lafirest 67945515c9
Merge pull request #8911 from lafirest/fix/retainer_start_error
fix(retainer): fix that EMQX can't start when the retainer is disabled
2022-09-09 10:14:34 +08:00
firest 47b35f0c6a fix(api): add listener create API from `POST /listeners/{type:name}` to `POST /listeners`
Old:
    API: POST /listeners/{type:demo}
    Body: {"type" : "tcp", "id" : "tcp:demo"}
New:
    API: POST /listeners
    Body: {"type" : "tcp", "name" : "demo"}
2022-09-08 20:28:40 +08:00
firest 1a835e9bd4 test(retainer): add test case for start after retianer is disabled 2022-09-08 19:13:02 +08:00
zhongwencool 0e633f87b3
Merge branch 'master' into denny-update-local-override-conf 2022-09-08 17:17:07 +08:00
JianBo He 641763d4d7 chore: update app.src 2022-09-08 16:14:42 +08:00
JianBo He 8521d8dac5 fix(time): replace os:system_time with erlang:system_time
Avoid the problem of inaccurate timers caused by mixing erlang:system_time/0-1
and os:system_time/0-1
2022-09-08 16:13:05 +08:00
JianBo He 9f6fb4f1ce
Merge branch 'master' into fix-coap-topic 2022-09-07 17:48:44 +08:00
Zaiming (Stone) Shi 81ec4c4403
Merge pull request #8906 from zmstone/merge-master-to-ee50-a
Merge master to ee50
2022-09-07 11:19:41 +02:00
Shawn 2b33ca6d49 fix: no error log print if insert bool values into mysql 2022-09-07 16:00:09 +08:00
firest a526c39ff5 chore: bump emqx_retainer version && update CHANGES-5.0.md 2022-09-07 14:49:33 +08:00
firest de36b77261 fix(retainer): fix that EMQX can't start when the retainer is disabled 2022-09-07 14:48:28 +08:00
zhongwencool 2943cbc261
Merge pull request #8857 from zhongwencool/cluster-rpc-when-node-down
feat: cluster-rpc failed fast when some nodes is down
2022-09-07 14:34:36 +08:00
Zaiming (Stone) Shi c9b6dfd7f9
Merge pull request #8905 from zmstone/0906-warn-default-erlang-cookie
refactor(cookie): Warning message when boot with default Erlang cookie
2022-09-07 08:06:55 +02:00
Zhongwen Deng 33341011d8 chore: improve wait_for_nodes_commit/4 function 2022-09-07 10:19:46 +08:00
zhongwencool 758b1979ab chore: Apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-09-07 10:19:46 +08:00
Zhongwen Deng 1175008a74 feat: cluster-rpc failed fast when some nodes is down 2022-09-07 10:19:46 +08:00
Xinyu Liu a67484422d
Merge pull request #8902 from terry-xiaoyu/check_dependent_actions
Check dependent actions before removing a data-bridge
2022-09-07 08:54:44 +08:00
Zaiming (Stone) Shi befc4acced Merge remote-tracking branch 'origin/master' into merge-master-to-ee50-a 2022-09-06 20:31:38 +02:00
Zaiming (Stone) Shi f785da075b ci: only start required docker for integration tests 2022-09-06 19:25:53 +02:00
Zaiming (Stone) Shi ac840e318e refactor(cookie): Warning message when boot with default Erlang cookie 2022-09-06 18:58:35 +02:00
firest 3f758ca36f fix(tls): fix ciphers cache return `ok` at the first time call 2022-09-06 16:42:02 +08:00
Zhongwen Deng 499da1ebe0 chore: add emqx_release:edition_longstr() 2022-09-06 15:46:18 +08:00
JianBo He 0caaccaa0f test: add tests for `/` leading topic 2022-09-06 15:38:32 +08:00
JianBo He da4efc11c2 chore: log authn-http parsing failed reason 2022-09-06 15:38:32 +08:00
JianBo He d6b222d1ff test: fix coap authz suite failures 2022-09-06 15:38:32 +08:00
JianBo He 55c1a1868a fix(coap): remove the leading `/` in assembling publish topic 2022-09-06 15:38:32 +08:00