Commit Graph

6139 Commits

Author SHA1 Message Date
Thales Macedo Garitezi ff23d25e8b chore(replayq): update replayq -> 0.3.6 and use `clean_start` for buffer workers
So we can truly avoid resuming work after a node restart.
2023-01-17 16:48:48 -03:00
Thales Macedo Garitezi c383558467 fix(buffer): fix `replayq` usages in buffer workers (5.0)
https://emqx.atlassian.net/browse/EMQX-8700

Fixes a few errors in the usage of `replayq` queues.

- Close `replayq` when `emqx_resource_worker` terminates.
- Do not keep old references to `replayq` after any `pop`s.
- Clear `replayq`'s data directories when removing a resource.
2023-01-17 16:48:48 -03:00
Zaiming (Stone) Shi fc992f28bc test: add test coverage 2023-01-17 20:13:01 +01:00
Zaiming (Stone) Shi fff6bf921f refactor(authz): call emqx_resource:simple_sync_query
there is no need to route the request through the buffer workers
2023-01-17 20:01:45 +01:00
Zaiming (Stone) Shi 7a471faf67 refactor(authn): use simple sync query for authn
there is no need to route the authn queries through the buffer worker
2023-01-17 19:57:14 +01:00
Ilya Averyanov 1d7364dab1 fix(authn): fix authn hook chain evaluation
Stop authentication handling when
`emqx_authentication` provides a definitive result.
2023-01-17 20:06:20 +02:00
Stefan Strigler d164e5bc0f fix: don't require ssl conf 2023-01-17 15:29:19 +01:00
Stefan Strigler e54f2f83b3 test: use same default timeout as elsewhere 2023-01-17 15:29:19 +01:00
Erik Timan 30a5cfaa83 fix(emqx_management): remove trace files after zip download
We only deleted the resulting zip after a trace file download, not the
actual trace files. This adds a deletion of the uncompressed trace files
as well. It also creates unique directories when collecting trace files
so that concurrent downloads doesn't overwrite files in transit.
2023-01-17 14:24:10 +01:00
Erik Timan 94296258b4 test(emqx_management): refactor test suite to use common test utils for API calls 2023-01-17 14:24:10 +01:00
Zhongwen Deng f98786c91f fix: bad crash_dump_file default value in rpm 2023-01-17 20:47:54 +08:00
Zhongwen Deng cf99089f84 chore: add some trace func to user_default 2023-01-17 20:33:01 +08:00
Zhongwen Deng 5a56013193 docs: more detail about crashdump config 2023-01-17 20:33:01 +08:00
zhongwencool 9f80eb2497
Merge pull request #9777 from zmstone/0116-refactor-config_files-is-readonly
refactor: config_files is a readonly config
2023-01-17 20:32:39 +08:00
Zaiming (Stone) Shi 263deae1f3 refactor: add a more generic name for password_converter 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 2afbf6a406 test: unlink process which is getting shutdown 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 4a7e74f5d6 fix(schema): add password converter to ensure its binary() type 2023-01-17 11:12:17 +01:00
Zaiming (Stone) Shi 8b3ec48406 refactor: node.config_files should be hidden as it's not configurable
this config schema exists only for app-env mapping.
users can not change it because the bootstraping config
has to be etc/emqx.conf
2023-01-17 11:08:40 +01:00
Stefan Strigler 74ae7c4264
Merge pull request #9782 from sstrigler/EMQX-8361-fix-ct-suite-for-plugins-api
test: get plugin template as artifact
2023-01-17 11:08:07 +01:00
firest c3d5c25c26 fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12 2023-01-17 17:42:10 +08:00
zhongwencool cb5314bb9e
Merge pull request #9774 from zhongwencool/dashboard-password-validate-v50
feat: add dashboard password validation
2023-01-17 17:02:51 +08:00
Stefan Strigler bc69a00329 test: get plugin template as artifact 2023-01-17 09:50:17 +01:00
Shawn 44f635300d chore: update mysql-otp to 1.7.2 2023-01-17 13:36:26 +08:00
Zhongwen Deng 3dfdad9cc9 test: dashboard default password 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi b9911e281e docs: Update apps/emqx_dashboard/src/emqx_dashboard_admin.erl 2023-01-17 11:30:42 +08:00
zhongwencool c549379b7c chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-17 11:30:42 +08:00
Zhongwen Deng 82af854c4a test: dashbashboard new password failed 2023-01-17 11:30:42 +08:00
Zhongwen Deng a523fa2fa2 feat: add dashboard password validate 2023-01-17 11:30:42 +08:00
Zaiming (Stone) Shi 33be7672ad test: recover boot_modules app env after test 2023-01-17 00:43:25 +01:00
Ivan Dyachkov 676f017ec0 fix: ensure no colon in filenames 2023-01-16 21:27:01 +01:00
Andrew Mayorov 5fd157fb1d
Merge pull request #9776 from keynslug/chore/bump-epgsql-otp-25
chore: update epgsql to 4.7.0.1 with full OTP-25 compat
2023-01-16 23:51:09 +04:00
Zaiming (Stone) Shi e668044ebc fix: version number previs in restricted shell 2023-01-16 13:47:00 +01:00
Andrew Mayorov 34571c779d
feat: make `suboption` table ordering more natural 2023-01-16 15:39:10 +03:00
Andrew Mayorov ce2dba15b4
feat: turn tables queried with search APIs into ordered sets
This is needed to ensure more or less consistent client experience
for the new planned cursor-based search APIs.
2023-01-16 15:39:10 +03:00
Zaiming (Stone) Shi a7fc5e8fe1
Merge pull request #9761 from zmstone/0114-fix-kafka-value-template-and-docs
feat: introduce 'this' concept for placeholder, and use it in Kafka bridge
2023-01-16 13:37:29 +01:00
Andrew Mayorov 54cea3b3f2
chore: update epgsql to 4.7.0.1 with full OTP-25 compat 2023-01-16 15:35:51 +03:00
Stefan Strigler 67909f0b40 fix: testing metrics for emqx_bridge_mqtt_SUITE 2023-01-16 12:10:06 +01:00
Stefan Strigler a19226326c fix: don't test for strict values as that makes test flaky 2023-01-16 12:10:06 +01:00
Stefan Strigler 9a53410efb fix: don't add empty list element to options proplist 2023-01-16 12:00:03 +01:00
Stefan Strigler edfcea7f43 fix: add log for timeout 2023-01-16 12:00:03 +01:00
Zaiming (Stone) Shi 47414e0d53 docs: improve kafka key and value field description 2023-01-16 11:32:09 +01:00
Zaiming (Stone) Shi 376dab7cd8
Merge pull request #9743 from zmstone/0112-docs-fix-some-descriptions
0112 docs fix some descriptions
2023-01-16 09:56:56 +01:00
JimMoen 54ebc27d24
Merge pull request #9672 from JimMoen/0103-fix-mqtt-bridge
Fix the problem that the bridge is not available when the Payload template is empty in the MQTT bridge.
2023-01-16 09:57:20 +08:00
Zaiming (Stone) Shi 089389100e
Merge pull request #9763 from zmstone/0114-fix-crash-when-password-undefined
0114 fix crash when password undefined
2023-01-15 12:41:02 +01:00
Zaiming (Stone) Shi df202c69ad
Merge pull request #9746 from zmstone/0113-fix-mqtt-connect-packet-format-when-password-is-empty
fix(emqx_packet): log empty string if password value is missing
2023-01-14 19:42:07 +01:00
Zaiming (Stone) Shi 0f2f5fbbe0 fix(authn): no exception when password is 'undefined' 2023-01-14 17:58:55 +01:00
Zaiming (Stone) Shi 7073c62dd9 feat: add a 'this' for rule engine put env
now a dot is allowed in front of a var (path)
e.g. ${.clientid} is equivalent to ${clientid}

${.} however, means everything.
the parsed var is interally represented as $_THIS_
to make it easier to read/search.
2023-01-14 11:03:58 +01:00
Zaiming (Stone) Shi 002f5dbd89
Merge pull request #9750 from zhongwencool/bootstrap-app-env
fix: save app's env when bootstrapping and reloading
2023-01-14 07:33:13 +01:00
Zhongwen Deng b07befbf56 test: conf_app ct failed 2023-01-14 01:03:57 +08:00
Stefan Strigler 6fe09447ed fix: stale test using old resource paths after merge 2023-01-13 17:23:25 +01:00
Zhongwen Deng 09160383f5 chore: bump emqx_resource to 0.1.5 2023-01-13 23:42:41 +08:00
Zhongwen Deng 588140b842 chore: why we should ingore save kernel app env when init 2023-01-13 23:35:45 +08:00
Zhongwen Deng 1fe0061123 fix: node.config_files type is hocon:array(string())) 2023-01-13 23:35:45 +08:00
Zhongwen Deng c9f0355f5a fix: save app env when bootstrap and reload 2023-01-13 23:35:45 +08:00
Zhongwen Deng 1085492171 chore: don't refresh when CT is running 2023-01-13 23:35:45 +08:00
Zhongwen Deng 91d38b410d fix: refresh logger handler when starting 2023-01-13 23:35:45 +08:00
Stefan Strigler 8f5881d1a5 fix: remove stale request/3 from merge error 2023-01-13 16:19:35 +01:00
Stefan Strigler e08c1d2229 Merge remote-tracking branch 'olcai/refactor-bridges-api' into dev/api-refactor 2023-01-13 15:49:52 +01:00
Stefan Strigler 1690a6dcfc
Merge branch 'master' into dev/api-refactor 2023-01-13 15:34:13 +01:00
Erik Timan 61e98900be chore: bump app vsn of emqx_resource 2023-01-13 15:13:35 +01:00
Erik Timan c3133fb6a2 fix(emqx_bridge): small fixes from review 2023-01-13 14:54:54 +01:00
Zaiming (Stone) Shi a77d33b97d
Merge pull request #9751 from lafirest/fix/un_cleared_ca
fix that obsoleted cert file will not be deleted after the listener is updated/deleted
2023-01-13 14:38:55 +01:00
Kjell Winblad 1ac03ab208
Merge pull request #9730 from kjellwinblad/kjell/fix/resource_atom_leak/EMQX-8583
fix: remove atom leaks
2023-01-13 14:38:28 +01:00
Zaiming (Stone) Shi bb3dceb456
Merge pull request #9749 from keynslug/fix/count-respect-matchspec
fix(paging): respect matchspec even if qs is empty when counting
2023-01-13 14:29:03 +01:00
Erik Timan 8dd52e5a6e chore: add translations to schemas 2023-01-13 14:19:23 +01:00
Erik Timan f1c58c34ed test(emqx_bridge): fix fetching of metrics in emqx_bridge_mqtt_SUITE 2023-01-13 14:19:23 +01:00
Zaiming (Stone) Shi 2974269f2a
Merge pull request #9723 from emqx/release-50
chore: sync release-50 to master
2023-01-13 14:13:12 +01:00
firest 0a965879dd fix: fix that obsoleted cert file will not be deleted after the listener is updated/deleted 2023-01-13 18:42:27 +08:00
Zhongwen Deng f15b29b1ef chore: upgrade app version 2023-01-13 18:22:29 +08:00
JimMoen b7259d9a20
test(mqtt-bridge): use empty payload template for ingress/egress mqtt bridge 2023-01-13 18:12:53 +08:00
Kjell Winblad 734e6b9c96 chore: fix flaky test cases, log labels and review comments
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-01-13 11:05:02 +01:00
Andrew Mayorov b3e62bd8f8
fix(paging): respect matchspec even if qs is empty when counting 2023-01-13 12:56:29 +03:00
zhongwencool 383a6812d1
Merge pull request #9748 from zhongwencool/listener-max-connections-default-value
fix: default max_connections in raw config should be binary not atom
2023-01-13 17:55:22 +08:00
JimMoen 59ac0b1424
fix(mqtt-bridge): transmit raw msg payload with empty template 2023-01-13 17:23:04 +08:00
zhongwencool 0049b4a294
Merge branch 'master' into release-50 2023-01-13 16:39:35 +08:00
Zaiming (Stone) Shi e7883e0ee2
Merge pull request #9741 from id/fix-description-for-duration-fields
fix: remove time unit from duration fields description
2023-01-13 09:31:48 +01:00
Zhongwen Deng eacdaffe3c fix: default max_connections in raw config should be binary not atom 2023-01-13 16:01:57 +08:00
Andrew Mayorov 33d88c95de
Merge pull request #9745 from keynslug/fix/EMQX-8702/changelog
chore: add changelog for the fuzzy search API fix
2023-01-13 11:59:37 +04:00
Zaiming (Stone) Shi 2bc19c3a62 fix(emqx_packet): log empty string if password value is missing
to keep it a consistent behaviour comparing when logging username
2023-01-13 08:21:09 +01:00
Zaiming (Stone) Shi 153d64068a refactor: rename root level cache type to authz_cache 2023-01-13 07:47:54 +01:00
Zaiming (Stone) Shi c7ac55520d docs: update pool_size doc 2023-01-13 07:46:23 +01:00
Zaiming (Stone) Shi b40ce0fc2d
Merge pull request #9626 from id/fix-enable-authz-cache-by-default
fix: enable authorization cache by default
2023-01-13 07:39:38 +01:00
Xinyu Liu 6b75077c47
Merge pull request #9734 from terry-xiaoyu/influxdb-default-value-of-write-syntax
refactor: remove default value of timestamp field of influxdb
2023-01-13 10:14:14 +08:00
Erik Timan 42f42de4d9 feat(emqx_bridge): add separate endpoint for enable/disable of bridge
In order to improve the consistency with other API endpoints, we move
the enable/disable operations to a separate endpoint
/bridges/{id}/enable/[true,false].
2023-01-12 19:39:43 +01:00
Erik Timan 860e21d40f feat(emqx_bridge): move metrics to own endpoint, rename reset_metrics
In order for the /bridges APIs to be consistent with other APIs, we move
out metrics from GET /bridges/{id} to its own endpoint,
/bridges/{id}/metrics. We also rename /bridges/reset_metrics to
/bridges/metrics/reset.
2023-01-12 19:39:41 +01:00
Erik Timan c2fd1a4482 feat(emqx_bridge): shorten operation endpoint URLs
This shortens and simplifies URLs for performing bridge operations so
that the API looks more congruent.
2023-01-12 19:36:22 +01:00
Andrew Mayorov 90e837783f
fix(paging): rename `partial` to `overflow` for clarity 2023-01-12 19:25:40 +03:00
lafirest a26c05f4f6
Merge pull request #9586 from lafirest/fix/disable_basic_auth_api
fix: disable basic auth for HTTP API
2023-01-12 23:02:29 +08:00
Zaiming (Stone) Shi 370b6b0d2f
Merge pull request #9735 from kjellwinblad/kjell/fix/password_log_leak/EMQX-8718
fix: remove password from info logs
2023-01-12 15:34:16 +01:00
Shawn 1711823487 refactor: remove default value of timestamp field of influxdb 2023-01-12 22:25:44 +08:00
firest c5f557e315 fix: disable basic auth for HTTP API 2023-01-12 21:35:49 +08:00
Stefan Strigler 2a81fa1522 fix: remove redundant log message 2023-01-12 14:29:11 +01:00
Stefan Strigler 04f46f5227 feat: make http connector report errors 2023-01-12 14:26:36 +01:00
Stefan Strigler 77f043dedf fix: don't require username and password
not enforced by schema so we shouldn't either
2023-01-12 14:26:36 +01:00
Ivan Dyachkov b5d3e9d8b8 fix: remove time unit from duration fields description 2023-01-12 14:18:55 +01:00
Ivan Dyachkov f90c41f769 fix: set default value in schema 2023-01-12 13:49:52 +01:00
Ivan Dyachkov 1f57e7b538 fix: enable authorization cache by default 2023-01-12 12:36:31 +01:00
Andrew Mayorov 9e9d97b4d8
Merge pull request #9726 from keynslug/fix/EMQX-8702/fuzzy-search-paging
fix(api): augment paged search responses with `hasnext` flag
2023-01-12 13:35:59 +04:00
Kjell Winblad 82e80e5610 fix: remove password from info logs
Information log messages for several connectors contained sensitive
password information. This commit fixes this by hiding the sensitive
information from the log messages.

Fixes: https://emqx.atlassian.net/browse/EMQX-8718
2023-01-12 09:28:01 +01:00
JianBo He 13fdbd695e
Merge pull request #9722 from HJianBo/more-confs-for-prometheus
feat: more confs for prometheus pushing
2023-01-12 16:06:21 +08:00
Andrew Mayorov 50c9321b9e
fix(api): anticipate empty set of nodes handling cluster query 2023-01-12 10:57:50 +03:00
Andrew Mayorov c89b227687
fix(test): adapt affected testcases 2023-01-12 10:57:50 +03:00
Andrew Mayorov abf6f143e5
fix(paging): return subset of rows if page start is in the middle
This change should cover this example scenario where select ended
returning 4 rows.

Row 3
Row 4
Row 5 <- page start
Row 6

Here, only rows 5 and 6 should end up in the response. Before this
change some rows could be duplicated across adjacent search pages.
2023-01-12 10:57:50 +03:00
Andrew Mayorov e07aa2086b
fix(api): augment paged search responses with `hasnext` flag
This flag indicates whether there are more results available on the
next pages. It is needed in cases when the total number of search
results is not known in advance.

Also, in such cases there's no `count` field in responses anymore
because responding with `0` was confusing for clients: it's not possible
to differentiate between "there are no results" and "we don't know how
much".

Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-01-12 10:57:30 +03:00
Zhongwen Deng 1194d07a07 refactor: update emqx_prometheus's headers from map() to list() 2023-01-12 12:48:26 +08:00
Xinyu Liu bae811e8b4
Merge pull request #9725 from terry-xiaoyu/remove_the_auto_reconnect_field
refactor: remove the auto_reconnect field
2023-01-12 11:11:00 +08:00
Kjell Winblad 8c482e03d1 fix: remove atom leaks
Both emqx_resource_managers and emqx_resource_workers leaked atoms as they
created an unique atoms to use as registered names. This is fixed by
removing the need to register the names.

Fixes: https://emqx.atlassian.net/browse/EMQX-8583
2023-01-11 17:03:28 +01:00
Stefan Strigler 417d95d76f
Merge pull request #9708 from sstrigler/EMQX-8588-refactor-emqx-resource-manager
EMQX 8588 refactor emqx resource manager
2023-01-11 16:13:55 +01:00
Shawn 3e9c4f444f refactor: remove the auto_reconnect field 2023-01-11 21:47:06 +08:00
Thales Macedo Garitezi 254b92a3c1 docs: add comma to please spellcheck 2023-01-11 10:31:23 -03:00
Stefan Strigler 8ad8288195 feat: report error in create_dry_run 2023-01-11 14:22:37 +01:00
Thales Macedo Garitezi d5a462486e feat: tag another emqx schema as `emqx` 2023-01-11 09:31:38 -03:00
Thales Macedo Garitezi e6a8be1ff6 docs: fix closing tag 2023-01-11 09:10:04 -03:00
Thales Macedo Garitezi 48e1ba4832 feat(docs): add tags to schemas
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
2023-01-11 09:10:03 -03:00
Kjell Winblad fde8752452
Merge pull request #9710 from kjellwinblad/kjell/mongodb_upgrade_in_emqx_5_master_fix
chore: upgrade mongodb-erlang to v3.0.19
2023-01-11 12:41:11 +01:00
JianBo He 9f6df27759 chore: bump app.src vsn 2023-01-11 17:47:06 +08:00
JianBo He b1292d6d4f feat(prom): support configuring job_name for pushing mode 2023-01-11 17:31:58 +08:00
JianBo He 50b7ac6a22 feat(prom): support headers for pushing 2023-01-11 17:31:58 +08:00
Zhongwen Deng fdc0682c3f fix: elvis warning 2023-01-11 17:30:45 +08:00
zhongwencool 6f0e228a9b
Merge pull request #9717 from zmstone/0110-do-not-start-resource-after-recreate-if-disabled
fix(emqx_resource_manager): do not start when disabled
2023-01-11 16:12:53 +08:00
Zaiming (Stone) Shi 85a8eff90b fix(emqx_resource_manager): do not start when disabled 2023-01-11 08:33:48 +01:00
Zhongwen Deng d9429bba00 fix: bump hocon to 0.35.0(don't log overriding structs) 2023-01-11 14:44:34 +08:00
Zhongwen Deng ba738d7707 chore: bump to 5.0.14 2023-01-11 14:39:32 +08:00
zhongwencool ce2b4d48ad
Merge pull request #9714 from zhongwencool/auto-subscribe-api-schema
fix: bad auto_subscribe api schema
2023-01-11 14:32:40 +08:00
zhongwencool e7d6d26186
Merge pull request #9713 from zhongwencool/api-keys-bootstrap-file
feat: introduce API keys bootstrap_file config
2023-01-11 11:30:45 +08:00
Zhongwen Deng 52ccd0762e test: QOS ct failed 2023-01-11 10:34:09 +08:00
Zaiming (Stone) Shi 6bbb5edb53
Merge pull request #9667 from olcai/remove-clientid-in-publish-request-body
fix(mgmt_api): remove possibility to set clientid in /publish API
2023-01-10 22:31:35 +01:00
Zaiming (Stone) Shi ee89de816c fix: upgrade bridge config converter 2023-01-10 20:42:30 +01:00
Erik Timan fb97096405 fix(mgmt_api): deprecate clientid field instead of removing it 2023-01-10 16:48:04 +01:00
Zhongwen Deng a2be9947f5 chore: update auto_subscribe app version 2023-01-10 22:12:23 +08:00
Zhongwen Deng 5415e341fb fix: bad auto_subscribe api schema 2023-01-10 21:53:54 +08:00
zhongwencool 918ba0dffd
chore: apply suggestions from code review
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-01-10 20:39:31 +08:00
Zhongwen Deng 21dbc21dc8 chore: add changelog for api_key.bootstrap_file 2023-01-10 19:15:01 +08:00
Zhongwen Deng 5d4a1933a2 chore: add more test for api_key 2023-01-10 19:04:00 +08:00
Zhongwen Deng f6a47e5cf6 chore: support api_key.bootstrap_file config 2023-01-10 19:03:49 +08:00
Zhongwen Deng 142826c4c3 chore: limit the maximum number of api keys to 100 2023-01-10 19:03:39 +08:00
Xinyu Liu 0f70786d13
Merge pull request #9705 from terry-xiaoyu/remove-default-value-of-webhook-body
fix: remove the default value of webhook body field
2023-01-10 18:23:50 +08:00
Xinyu Liu 12de104033
Merge pull request #9712 from emqx/fix-bulk-subscribe-failed-in-client.connected-hook
fix: bulk subscribe topics failed in the client.connected hook
2023-01-10 18:22:15 +08:00
ieQu1 13f1ab6344
Merge pull request #9671 from ieQu1/metrics
feat(emqx_metrics): Sliding window samples
2023-01-10 11:08:43 +01:00
Stefan Strigler f27f573109 refactor: move to /authorization/sources/built_in_database/rules 2023-01-10 11:00:22 +01:00
Shawn 73643e932f fix: bulk subscribe topics failed in the client.connected hook 2023-01-10 16:43:24 +08:00
Xinyu Liu 52343e58c7
Merge pull request #9703 from emqx/subscribe-api-function-clause
fix: missing default value for qos field of API /clients/:clientid/subscribe
2023-01-10 10:13:58 +08:00
Shawn 724015af47 fix: remove the default value of webhook body field 2023-01-10 09:45:25 +08:00
Zaiming (Stone) Shi 67f2159a27
Merge pull request #9653 from zmstone/0101-authz-schema-union-member-selection
0101 authz schema union member selection
2023-01-09 22:17:51 +01:00
Kjell Winblad 0bd5bec982 chore: upgrade mongodb-erlang to v3.0.19
This upgrades mongodb-erlang from v3.0.18 to v3.0.19. This fixes a bug
that occurred when one issued a command specified as a map*. This fix
should not affect present EMQX as this functionality is currently not
used by EMQX. However, I think it is good to do the upgrade anyway in
case we will use the fixed functionality in the future. There is no need
to update the changelog.

* https://github.com/emqx/mongodb-erlang/pull/36
2023-01-09 18:14:11 +01:00
lafirest f1ebfe40a5
Merge pull request #9697 from lafirest/fix/ldap_i18n
chore: add i18n file for ldap connector
2023-01-09 22:54:09 +08:00
Thales Macedo Garitezi 7f5150b2e9
Merge pull request #9689 from thalesmg/fix-http-authz-failure-v50
fix(authz_http): handle `ignore` results (request failures)
2023-01-09 11:07:52 -03:00
Zaiming (Stone) Shi c3635f537a ci: wait for redis in emqx_ee_bridge_redis_SUITE 2023-01-09 14:39:56 +01:00
Zaiming (Stone) Shi 0697c692ed refactor: mongo_type and redis_type are not mandatory
the connector schemas are shared between authn, auth and bridges,
the difference is that authn and authz configs are unions like:

[
    {mongo_type = rs,
     ...
    }
    {another backend config}
]

however the brdige types are maps like, for example:

mongodb_rs.$name {
    mongo_type = rs
    ...
}

in which case, the mongo_type is not required.

in order to keep the schema static as much as possible,
this field is chanegd to 'required => false' with a default value.

However, for authn and authz, the union selector will still raise
exception if the there is no type provided.
2023-01-09 14:26:16 +01:00
Zaiming (Stone) Shi e52f9d5920 refactor: use union member type selector for authz sources 2023-01-09 14:26:16 +01:00
Kjell Winblad ba8c304723
Merge pull request #9691 from kjellwinblad/kjell/mongodb_upgrade_in_emqx_5_master
feat: upgrade MongoDB library to support MongoDB 5.1+
2023-01-09 14:15:48 +01:00
Kjell Winblad 692c1f0d1b
Merge pull request #9678 from kjellwinblad/kjell/fix/jira/EMQX-8648
fix: no feedback when deleting non-existing configuration
2023-01-09 13:52:40 +01:00
Stefan Strigler c85a988a43 fix: split 'server' param into host and port for mqtt bridge 2023-01-09 13:27:42 +01:00
Andrew Mayorov 0db31f30a1
Merge pull request #9675 from keynslug/chore/otp-25-compat-deps
chore: bump deps versions with full OTP-25 compat
2023-01-09 14:54:14 +04:00
Shawn 50717a5567 fix: set the topic field of API /clients/:clientid/subscribe as requried 2023-01-09 17:57:39 +08:00
Stefan Strigler c42c99f94f fix: set a timeout for tcp connect 2023-01-09 10:01:35 +01:00
Stefan Strigler 96ca0d9f49 feat(emqx_bridge): add /bridges_probe API endpoint 2023-01-09 10:01:33 +01:00
Stefan Strigler 903ae9a644 style: fix typo in fun name 2023-01-09 09:59:42 +01:00
Erik Timan 3319a8d28e fix(mgmt_api): remove possibility to set clientid in /publish API
To avoid security confusion, we remove the possibility to specify the
client ID in the request body for /publish and /publish/bulk.
2023-01-09 09:30:34 +01:00
Zaiming (Stone) Shi 8d53d154d9 refactor: remove license from configs API
license has its own config APIs
2023-01-09 09:16:19 +01:00
Shawn 7ed1384623 fix: missing default value for qos field of API /clients/:clientid/subscribe 2023-01-09 16:02:10 +08:00
Zaiming (Stone) Shi 3859878985
Merge pull request #9684 from zmstone/0105-do-not-use-testcase-skip
test: do not use tc_user_skip for test cases
2023-01-09 09:02:06 +01:00
JianBo He 6f5057b9dd
Merge pull request #9637 from HJianBo/fix-clients-api
fix(clients): fix expiry_interval unit error
2023-01-09 09:49:16 +08:00
Xinyu Liu 336b310d56
Merge pull request #9687 from emqx/function_clause_when_send_msg_to_bridges
fix: function_clause when sending messages to bridges
2023-01-09 09:23:17 +08:00
Andrew Mayorov 1336a57b3b
chore: bump to emqx/eredis_cluster 0.7.5
* Full OTP-25 compat.
* Password censoring in status and crash reports.
* No more crashes on query timeout.
2023-01-08 13:13:31 +03:00
Zaiming (Stone) Shi 5653d113ac
Merge pull request #9663 from zmstone/0102-fix-stopped-nodes-detection
0102 fix stopped nodes detection
2023-01-08 10:03:48 +01:00
firest d2456f9fbe chore: add i18n file for ldap connector 2023-01-06 17:28:06 +08:00
firest f7b50c56da feat(bridges): integrate PostgreSQL into bridges
support both simple and batch query
2023-01-06 11:47:22 +08:00
JianBo He 5d55e3a7b3 chore: updage mgmt app vsn 2023-01-06 10:31:27 +08:00
JianBo He 65729cd640 fix(clients): fix expiry_interval unit error
It should be second not millisecond
2023-01-06 10:31:27 +08:00
Shawn 13b2f45405 fix: function_clause when sending messages to bridges 2023-01-06 09:24:38 +08:00
Zaiming (Stone) Shi 7a4e7a6de9 test: wait for redis 2023-01-05 22:47:08 +01:00
Zaiming (Stone) Shi 5f12cdff6c ci: should not skip tests in github actions 2023-01-05 22:47:08 +01:00
Thales Macedo Garitezi 22707495ac
Merge pull request #9690 from thalesmg/fix-flaky-jwt-test
test(flaky): fix flaky jwt worker test
2023-01-05 14:00:39 -03:00
Kjell Winblad 5266722e51 feat: upgrade MongoDB library to support MongoDB 5.1+ 2023-01-05 16:40:45 +01:00
Kjell Winblad b59c4c34c5 fix(Bridge REST API): no feedback when deleting bridge
This fixes https://emqx.atlassian.net/browse/EMQX-8648. The issue
described in `EMQX-8648` is that when deleting a non-existing bridge the
server gives a success response. See below:

```
curl --head -u admin:public2 -X 'DELETE' 'http://localhost:18083/api/v5/bridges/webhook:i_do_not_exist'
HTTP/1.1 204 No Content
date: Tue, 03 Jan 2023 16:59:01 GMT
server: Cowboy
```

After the fix, deleting a non existing bridge will give the following
response:

```
HTTP/1.1 404 Not Found
content-length: 49
content-type: application/json
date: Thu, 05 Jan 2023 12:40:35 GMT
server: Cowboy
```

Closes: EMQX-8648
2023-01-05 16:19:46 +01:00
Thales Macedo Garitezi c6b8e614df fix(authz_http): handle `ignore` results (request failures)
Related issue: https://github.com/emqx/emqx/issues/9683

When the HTTP request for authz fails (e.g.: resource is down or
server is down), then the HTTP authorizer returns `ignore`, which was
not handled correctly by the authorization callback.
2023-01-05 11:34:23 -03:00
Thales Macedo Garitezi cf6996cbea test(flaky): fix flaky jwt worker test 2023-01-05 11:19:43 -03:00
Thales Macedo Garitezi 70eb5ffb58 refactor: remove unused function 2023-01-05 10:16:01 -03:00
Thales Macedo Garitezi 56437228dc docs: improve descriptions
Thanks to @qzhuyan for the corrections.
2023-01-05 10:16:01 -03:00
Thales Macedo Garitezi fd360ac6c0 feat(buffer_worker): refactor buffer/resource workers to always use queue
This makes the buffer/resource workers always use `replayq` for
queuing, along with collecting multiple requests in a single call.
This is done to avoid long message queues for the buffer workers and
rely on `replayq`'s capabilities of offloading to disk and detecting
overflow.

Also, this deprecates the `enable_batch` and `enable_queue` resource
creation options, as: i) queuing is now always enables; ii) batch_size
> 1 <=> batch_enabled.  The corresponding metric
`dropped.queue_not_enabled` is dropped, along with `batching`.  The
batching is too ephemeral, especially considering a default batch time
of 20 ms, and is not shown in the dashboard, so it was removed.
2023-01-05 10:15:09 -03:00
Thales Macedo Garitezi bf3983e7c4 feat(buffer_worker): use offload mode for `replayq`
To avoid confusion for the users as to what persistence guarantees we
offer when buffering bridges/resources, we will always enable offload
mode for `replayq`.  With this, when the buffer size is above the max
segment size, it'll flush the queue to disk, but on recovery after a
restart it'll clean the existing segments rather than resuming from
them.
2023-01-05 10:11:59 -03:00
ieQu1 5d9f9671e9 feat(emqx_metrics): Sliding window samples 2023-01-05 13:55:52 +01:00
Thales Macedo Garitezi fa4f90d43e
Merge pull request #9676 from thalesmg/fix-change-retainer-storage-type-v50
fix(retainer): change mnesia table storage types during update
2023-01-05 09:07:10 -03:00
ieQu1 93c205db7c fix(router_helper): Reduce log verbosity of expected events 2023-01-05 12:17:27 +01:00
Thales Macedo Garitezi 51ad27cb4b test(retainer): assert that retained messages are not lost when changing storage type 2023-01-04 13:52:52 -03:00
Thales Macedo Garitezi 2f13bfd452 fix(retainer): change mnesia table storage types during update
https://emqx.atlassian.net/browse/EMQX-8650
2023-01-04 13:52:52 -03:00
Thales Macedo Garitezi 86a4ac87a7
Merge pull request #9674 from thalesmg/reset-rule-metrics-on-disable-v50
chore(rule_engine): reset metrics when disabling a rule
2023-01-04 13:30:39 -03:00
Thales Macedo Garitezi 00b91d8a60
Merge pull request #9664 from thalesmg/fix-mongo-duration-field-descs-v50
docs(mongodb_connector): fix duration field descriptions
2023-01-04 10:03:38 -03:00
Thales Macedo Garitezi eb7dca3691 chore(rule_engine): reset metrics when disabling a rule
https://emqx.atlassian.net/browse/EMQX-8502

When a bridge is disabled, its metrics are reset.  With this change,
we make rule actions behave like that: disabling a rule will reset its
metrics.
2023-01-04 09:30:47 -03:00
Erik Timan f767db4d8f
Merge pull request #9660 from olcai/refactor-ingress-bridge-counter
refactor(emqx_resource): ingress bridge counter
2023-01-04 10:43:07 +01:00
Thales Macedo Garitezi c3cfbae3c2 docs: improve descriptions
Thanks to @qzhuyan for the corrections
2023-01-03 11:38:09 -03:00
firest b43be50a98 test: add the `redacted` test case for bridges api 2023-01-03 22:16:34 +08:00
firest b016695cb1 fix(bridges): obfuscate the password in bridges API responses 2023-01-03 22:09:29 +08:00
Thales Macedo Garitezi 7bbdc20993 docs(mongodb_connector): fix duration field descriptions
https://emqx.atlassian.net/browse/EMQX-8642

Adds different descriptions for the different fields of type duration
for MongoDB.

Based off: https://www.mongodb.com/docs/manual/reference/connection-string/
2023-01-03 10:01:56 -03:00
Zaiming (Stone) Shi ced5a27053 test: fix emqx_conf test cases 2023-01-02 21:30:40 +01:00
ieQu1 2f208c3ab9 fix(cluster_rpc): Detect stopped replicant nodes 2023-01-02 21:30:40 +01:00
lafirest 7985cd3536
Merge pull request #9638 from lafirest/fix/mysql_dup
fix(mysql): fix the problem of data loss and bad match when mysql is disconnected
2023-01-02 23:15:44 +08:00
Erik Timan b9d012e072 refactor(emqx_resource): ingress bridge counter
Unify code paths for resource metrics by removing
emqx_resource:inc_received/1 and adding
emqx_resource_metrics:received_inc/1 & friends.
2023-01-02 15:11:52 +01:00
Thales Macedo Garitezi 7e02eac3bc
Merge pull request #9619 from thalesmg/refactor-gauges-v50
refactor(metrics): use absolute gauge values rather than deltas (v5.0)
2023-01-02 10:56:47 -03:00