Commit Graph

6046 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi cdd8de11b0 chore: fix a typo in function name 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi 618b97870b refactor: call local function queue_count everywhere 2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi 249c4c1c79 refactor: use 'bufs' for resource worker replayq dir 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi af6807e863 refactor: cancel flush timer sooner
Avoids the cancellation being delayed.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 477c55d8ef fix: sanitizy replayq dir filepath
Colons (`:`) are not allowed in Windows.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 4c04a01370 refactor(buffer_worker): remove `?Q_ITEM` wrapping and use lightweight size estimate 2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi 32a9e60313 feat(buffer_worker): also use the inflight table for sync requests
Related: https://emqx.atlassian.net/browse/EMQX-8692

This should also correctly account for `retried.*` metrics for sync
requests.

Also fixes cases where race conditions for retrying async requests
could potentially lead to inconsistent metrics.

Fixes more cases where a stale reference to `replayq` was being held
accidentally after a `pop`.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi ff23d25e8b chore(replayq): update replayq -> 0.3.6 and use `clean_start` for buffer workers
So we can truly avoid resuming work after a node restart.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi c383558467 fix(buffer): fix `replayq` usages in buffer workers (5.0)
https://emqx.atlassian.net/browse/EMQX-8700

Fixes a few errors in the usage of `replayq` queues.

- Close `replayq` when `emqx_resource_worker` terminates.
- Do not keep old references to `replayq` after any `pop`s.
- Clear `replayq`'s data directories when removing a resource.
2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi fc992f28bc test: add test coverage 2023-01-17 20:13:01 +01:00
Zaiming (Stone) Shi fff6bf921f refactor(authz): call emqx_resource:simple_sync_query
there is no need to route the request through the buffer workers
2023-01-17 20:01:45 +01:00
Zaiming (Stone) Shi 7a471faf67 refactor(authn): use simple sync query for authn
there is no need to route the authn queries through the buffer worker
2023-01-17 19:57:14 +01:00
Ilya Averyanov 1d7364dab1 fix(authn): fix authn hook chain evaluation
Stop authentication handling when
`emqx_authentication` provides a definitive result.
2023-01-17 20:06:20 +02:00
Stefan Strigler d164e5bc0f fix: don't require ssl conf 2023-01-17 15:29:19 +01:00
Stefan Strigler e54f2f83b3 test: use same default timeout as elsewhere 2023-01-17 15:29:19 +01:00
Erik Timan 30a5cfaa83 fix(emqx_management): remove trace files after zip download
We only deleted the resulting zip after a trace file download, not the
actual trace files. This adds a deletion of the uncompressed trace files
as well. It also creates unique directories when collecting trace files
so that concurrent downloads doesn't overwrite files in transit.
2023-01-17 14:24:10 +01:00
Erik Timan 94296258b4 test(emqx_management): refactor test suite to use common test utils for API calls 2023-01-17 14:24:10 +01:00
Zhongwen Deng f98786c91f fix: bad crash_dump_file default value in rpm 2023-01-17 20:47:54 +08:00
Zhongwen Deng cf99089f84 chore: add some trace func to user_default 2023-01-17 20:33:01 +08:00
Zhongwen Deng 5a56013193 docs: more detail about crashdump config 2023-01-17 20:33:01 +08:00
zhongwencool 9f80eb2497
Merge pull request #9777 from zmstone/0116-refactor-config_files-is-readonly
refactor: config_files is a readonly config
2023-01-17 20:32:39 +08:00
Zaiming (Stone) Shi 263deae1f3 refactor: add a more generic name for password_converter 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 2afbf6a406 test: unlink process which is getting shutdown 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 4a7e74f5d6 fix(schema): add password converter to ensure its binary() type 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 8b3ec48406 refactor: node.config_files should be hidden as it's not configurable
this config schema exists only for app-env mapping.
users can not change it because the bootstraping config
has to be etc/emqx.conf
2023-01-17 11:08:40 +01:00
Stefan Strigler 74ae7c4264
Merge pull request #9782 from sstrigler/EMQX-8361-fix-ct-suite-for-plugins-api
test: get plugin template as artifact
2023-01-17 11:08:07 +01:00
firest c3d5c25c26 fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12 2023-01-17 17:42:10 +08:00
zhongwencool cb5314bb9e
Merge pull request #9774 from zhongwencool/dashboard-password-validate-v50
feat: add dashboard password validation
2023-01-17 17:02:51 +08:00
Stefan Strigler bc69a00329 test: get plugin template as artifact 2023-01-17 09:50:17 +01:00
Shawn 44f635300d chore: update mysql-otp to 1.7.2 2023-01-17 13:36:26 +08:00
Zhongwen Deng 3dfdad9cc9 test: dashboard default password 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi b9911e281e docs: Update apps/emqx_dashboard/src/emqx_dashboard_admin.erl 2023-01-17 11:30:42 +08:00
zhongwencool c549379b7c chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-17 11:30:42 +08:00
Zhongwen Deng 82af854c4a test: dashbashboard new password failed 2023-01-17 11:30:42 +08:00
Zhongwen Deng a523fa2fa2 feat: add dashboard password validate 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi 33be7672ad test: recover boot_modules app env after test 2023-01-17 00:43:25 +01:00
Ivan Dyachkov 676f017ec0 fix: ensure no colon in filenames 2023-01-16 21:27:01 +01:00
Andrew Mayorov 5fd157fb1d
Merge pull request #9776 from keynslug/chore/bump-epgsql-otp-25
chore: update epgsql to 4.7.0.1 with full OTP-25 compat
2023-01-16 23:51:09 +04:00
Zaiming (Stone) Shi e668044ebc fix: version number previs in restricted shell 2023-01-16 13:47:00 +01:00
Andrew Mayorov 34571c779d
feat: make `suboption` table ordering more natural 2023-01-16 15:39:10 +03:00
Andrew Mayorov ce2dba15b4
feat: turn tables queried with search APIs into ordered sets
This is needed to ensure more or less consistent client experience
for the new planned cursor-based search APIs.
2023-01-16 15:39:10 +03:00
Zaiming (Stone) Shi a7fc5e8fe1
Merge pull request #9761 from zmstone/0114-fix-kafka-value-template-and-docs
feat: introduce 'this' concept for placeholder, and use it in Kafka bridge
2023-01-16 13:37:29 +01:00
Andrew Mayorov 54cea3b3f2
chore: update epgsql to 4.7.0.1 with full OTP-25 compat 2023-01-16 15:35:51 +03:00
Stefan Strigler 67909f0b40 fix: testing metrics for emqx_bridge_mqtt_SUITE 2023-01-16 12:10:06 +01:00
Stefan Strigler a19226326c fix: don't test for strict values as that makes test flaky 2023-01-16 12:10:06 +01:00
Stefan Strigler 9a53410efb fix: don't add empty list element to options proplist 2023-01-16 12:00:03 +01:00
Stefan Strigler edfcea7f43 fix: add log for timeout 2023-01-16 12:00:03 +01:00
Zaiming (Stone) Shi 47414e0d53 docs: improve kafka key and value field description 2023-01-16 11:32:09 +01:00
Zaiming (Stone) Shi 376dab7cd8
Merge pull request #9743 from zmstone/0112-docs-fix-some-descriptions
0112 docs fix some descriptions
2023-01-16 09:56:56 +01:00
JimMoen 54ebc27d24
Merge pull request #9672 from JimMoen/0103-fix-mqtt-bridge
Fix the problem that the bridge is not available when the Payload template is empty in the MQTT bridge.
2023-01-16 09:57:20 +08:00
Zaiming (Stone) Shi 089389100e
Merge pull request #9763 from zmstone/0114-fix-crash-when-password-undefined
0114 fix crash when password undefined
2023-01-15 12:41:02 +01:00
Zaiming (Stone) Shi df202c69ad
Merge pull request #9746 from zmstone/0113-fix-mqtt-connect-packet-format-when-password-is-empty
fix(emqx_packet): log empty string if password value is missing
2023-01-14 19:42:07 +01:00
Zaiming (Stone) Shi 0f2f5fbbe0 fix(authn): no exception when password is 'undefined' 2023-01-14 17:58:55 +01:00
Zaiming (Stone) Shi 7073c62dd9 feat: add a 'this' for rule engine put env
now a dot is allowed in front of a var (path)
e.g. ${.clientid} is equivalent to ${clientid}

${.} however, means everything.
the parsed var is interally represented as $_THIS_
to make it easier to read/search.
2023-01-14 11:03:58 +01:00
Zaiming (Stone) Shi 002f5dbd89
Merge pull request #9750 from zhongwencool/bootstrap-app-env
fix: save app's env when bootstrapping and reloading
2023-01-14 07:33:13 +01:00
Zhongwen Deng b07befbf56 test: conf_app ct failed 2023-01-14 01:03:57 +08:00
Stefan Strigler 6fe09447ed fix: stale test using old resource paths after merge 2023-01-13 17:23:25 +01:00
Zhongwen Deng 09160383f5 chore: bump emqx_resource to 0.1.5 2023-01-13 23:42:41 +08:00
Zhongwen Deng 588140b842 chore: why we should ingore save kernel app env when init 2023-01-13 23:35:45 +08:00
Zhongwen Deng 1fe0061123 fix: node.config_files type is hocon:array(string())) 2023-01-13 23:35:45 +08:00
Zhongwen Deng c9f0355f5a fix: save app env when bootstrap and reload 2023-01-13 23:35:45 +08:00
Zhongwen Deng 1085492171 chore: don't refresh when CT is running 2023-01-13 23:35:45 +08:00
Zhongwen Deng 91d38b410d fix: refresh logger handler when starting 2023-01-13 23:35:45 +08:00
Stefan Strigler 8f5881d1a5 fix: remove stale request/3 from merge error 2023-01-13 16:19:35 +01:00
Stefan Strigler e08c1d2229 Merge remote-tracking branch 'olcai/refactor-bridges-api' into dev/api-refactor 2023-01-13 15:49:52 +01:00
Stefan Strigler 1690a6dcfc
Merge branch 'master' into dev/api-refactor 2023-01-13 15:34:13 +01:00
Erik Timan 61e98900be chore: bump app vsn of emqx_resource 2023-01-13 15:13:35 +01:00
Erik Timan c3133fb6a2 fix(emqx_bridge): small fixes from review 2023-01-13 14:54:54 +01:00
Zaiming (Stone) Shi a77d33b97d
Merge pull request #9751 from lafirest/fix/un_cleared_ca
fix that obsoleted cert file will not be deleted after the listener is updated/deleted
2023-01-13 14:38:55 +01:00
Kjell Winblad 1ac03ab208
Merge pull request #9730 from kjellwinblad/kjell/fix/resource_atom_leak/EMQX-8583
fix: remove atom leaks
2023-01-13 14:38:28 +01:00
Zaiming (Stone) Shi bb3dceb456
Merge pull request #9749 from keynslug/fix/count-respect-matchspec
fix(paging): respect matchspec even if qs is empty when counting
2023-01-13 14:29:03 +01:00
Erik Timan 8dd52e5a6e chore: add translations to schemas 2023-01-13 14:19:23 +01:00
Erik Timan f1c58c34ed test(emqx_bridge): fix fetching of metrics in emqx_bridge_mqtt_SUITE 2023-01-13 14:19:23 +01:00
Zaiming (Stone) Shi 2974269f2a
Merge pull request #9723 from emqx/release-50
chore: sync release-50 to master
2023-01-13 14:13:12 +01:00
firest 0a965879dd fix: fix that obsoleted cert file will not be deleted after the listener is updated/deleted 2023-01-13 18:42:27 +08:00
Zhongwen Deng f15b29b1ef chore: upgrade app version 2023-01-13 18:22:29 +08:00
JimMoen b7259d9a20
test(mqtt-bridge): use empty payload template for ingress/egress mqtt bridge 2023-01-13 18:12:53 +08:00
Kjell Winblad 734e6b9c96 chore: fix flaky test cases, log labels and review comments
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-01-13 11:05:02 +01:00
Andrew Mayorov b3e62bd8f8
fix(paging): respect matchspec even if qs is empty when counting 2023-01-13 12:56:29 +03:00
zhongwencool 383a6812d1
Merge pull request #9748 from zhongwencool/listener-max-connections-default-value
fix: default max_connections in raw config should be binary not atom
2023-01-13 17:55:22 +08:00
JimMoen 59ac0b1424
fix(mqtt-bridge): transmit raw msg payload with empty template 2023-01-13 17:23:04 +08:00
zhongwencool 0049b4a294
Merge branch 'master' into release-50 2023-01-13 16:39:35 +08:00
Zaiming (Stone) Shi e7883e0ee2
Merge pull request #9741 from id/fix-description-for-duration-fields
fix: remove time unit from duration fields description
2023-01-13 09:31:48 +01:00
Zhongwen Deng eacdaffe3c fix: default max_connections in raw config should be binary not atom 2023-01-13 16:01:57 +08:00
Andrew Mayorov 33d88c95de
Merge pull request #9745 from keynslug/fix/EMQX-8702/changelog
chore: add changelog for the fuzzy search API fix
2023-01-13 11:59:37 +04:00
Zaiming (Stone) Shi 2bc19c3a62 fix(emqx_packet): log empty string if password value is missing
to keep it a consistent behaviour comparing when logging username
2023-01-13 08:21:09 +01:00
Zaiming (Stone) Shi 153d64068a refactor: rename root level cache type to authz_cache 2023-01-13 07:47:54 +01:00
Zaiming (Stone) Shi c7ac55520d docs: update pool_size doc 2023-01-13 07:46:23 +01:00
Zaiming (Stone) Shi b40ce0fc2d
Merge pull request #9626 from id/fix-enable-authz-cache-by-default
fix: enable authorization cache by default
2023-01-13 07:39:38 +01:00
Xinyu Liu 6b75077c47
Merge pull request #9734 from terry-xiaoyu/influxdb-default-value-of-write-syntax
refactor: remove default value of timestamp field of influxdb
2023-01-13 10:14:14 +08:00
Erik Timan 42f42de4d9 feat(emqx_bridge): add separate endpoint for enable/disable of bridge
In order to improve the consistency with other API endpoints, we move
the enable/disable operations to a separate endpoint
/bridges/{id}/enable/[true,false].
2023-01-12 19:39:43 +01:00
Erik Timan 860e21d40f feat(emqx_bridge): move metrics to own endpoint, rename reset_metrics
In order for the /bridges APIs to be consistent with other APIs, we move
out metrics from GET /bridges/{id} to its own endpoint,
/bridges/{id}/metrics. We also rename /bridges/reset_metrics to
/bridges/metrics/reset.
2023-01-12 19:39:41 +01:00
Erik Timan c2fd1a4482 feat(emqx_bridge): shorten operation endpoint URLs
This shortens and simplifies URLs for performing bridge operations so
that the API looks more congruent.
2023-01-12 19:36:22 +01:00
Andrew Mayorov 90e837783f
fix(paging): rename `partial` to `overflow` for clarity 2023-01-12 19:25:40 +03:00
lafirest a26c05f4f6
Merge pull request #9586 from lafirest/fix/disable_basic_auth_api
fix: disable basic auth for HTTP API
2023-01-12 23:02:29 +08:00
Zaiming (Stone) Shi 370b6b0d2f
Merge pull request #9735 from kjellwinblad/kjell/fix/password_log_leak/EMQX-8718
fix: remove password from info logs
2023-01-12 15:34:16 +01:00
Shawn 1711823487 refactor: remove default value of timestamp field of influxdb 2023-01-12 22:25:44 +08:00
firest c5f557e315 fix: disable basic auth for HTTP API 2023-01-12 21:35:49 +08:00
Stefan Strigler 2a81fa1522 fix: remove redundant log message 2023-01-12 14:29:11 +01:00
Stefan Strigler 04f46f5227 feat: make http connector report errors 2023-01-12 14:26:36 +01:00