Commit Graph

1115 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 5427ebc5f1 fix: log error when failed to install plugin
the error return is a map with details.
prior to this change only the reason is returned in the api response,
now the error map is logged at error level to help troubleshooting
2023-11-30 16:49:30 +01:00
Zaiming (Stone) Shi 14644988e0 chore: change triple-quotes to single-quotes 2023-11-29 16:15:18 +01:00
Serge Tupchii 009a15c7d0 fix(emqx_mgmt_api_data_backup): validate empty file in upload API
fixes EMQX-11488
2023-11-29 12:12:42 +02:00
JianBo He c8b5c51bbc chore: fix failed test cases 2023-11-28 09:53:46 +08:00
Serge Tupchii 89fdbf3e73 fix(emqx_mgmt_cli): remove duplicated word in the log message 2023-11-27 17:30:58 +02:00
Serge Tupchii 64ee29af81 feat(emqx_management): implement data backup API 2023-11-27 17:30:58 +02:00
zhongwencool b5a00ec6b2
Merge pull request #12023 from emqx/master
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
Andrew Mayorov 110a5a4896
feat(schema): introduce separate root for new session persistence
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
2023-11-20 23:49:14 +07:00
Andrew Mayorov d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Andrew Mayorov 6812ee9d0f
fix(mgmt): hide route selection behind router interface
Also introduce a generic _stream_ concept, mostly to deal with
iterating over 2 ETS tables at once with `ets:match_object/3`.
2023-11-14 18:36:25 +07:00
Ivan Dyachkov 28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Zaiming (Stone) Shi 86110824eb feat: upgrade hocon to 0.40.0 which supports union type display name 2023-11-10 13:41:51 +01:00
Ivan Dyachkov 0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
Ivan Dyachkov b1ab213081 chore: merge 'upstream/release-53' into 1031-sync-r53 2023-10-31 11:06:25 +01:00
Serge Tupchii 0562585c8f fix(emqx_mgmt_data_backup): implement importer modules ordering
`emqx_bridge_v2` depends on `emqx_connector`, so connectors must be imported first.
2023-10-30 19:14:58 +02:00
Serge Tupchii d1cd5dd817 fix(emqx_mgmt_data_backup): upgrade raw conf before validating and importing 2023-10-30 17:52:39 +02:00
JimMoen 3a09fdc495
refactor: check match topic before do subscriptions query 2023-10-30 16:45:29 +08:00
JimMoen e9de7316b6
test: shared-sub topics/subscription api 2023-10-30 14:42:59 +08:00
JimMoen 814e22feb3
fix: topics/subscripton mgmt api searching 2023-10-30 14:42:47 +08:00
JimMoen a2015f37ae
Merge pull request #10976 from JimMoen/fix-shared-sub-unsub 2023-10-27 21:22:27 +08:00
firest 4ba34f8f3e chore(rbac): fix CI errors & update change 2023-10-27 17:17:04 +08:00
JimMoen b5411da770
refactor: subscribe process to fix shared-sub 2023-10-27 09:23:43 +08:00
firest ec41479633 feat(rbac): supports setting role in API bootstrap file 2023-10-24 23:03:18 +08:00
firest e175c213a1 fix(rbac): for compatibility with old data schema, extend the existing field as extra 2023-10-24 22:28:53 +08:00
firest 7a8a5926ab fix(rbac): adjust the role names 2023-10-24 19:01:12 +08:00
JimMoen 99fab8dc5d
chore: missing change log for duplicated apikey fixing 2023-10-24 09:44:39 +08:00
JimMoen e6576951ef
test: cleanup duplicated apikey with different name 2023-10-20 22:33:26 +08:00
JimMoen d467289bb2
fix: avoid duplicated apikey from data import 2023-10-20 21:13:04 +08:00
zhongwencool 926c804314 feat: add /audit http api to filter audit log 2023-10-20 14:47:24 +08:00
zhongwencool 8b5ec2a4d5 fix: xref failed 2023-10-20 11:46:31 +08:00
lafirest 74442b0d31
Merge pull request #11766 from lafirest/feat/rbac
feat(api_key): add RBAC feature for the API key
2023-10-20 09:24:41 +08:00
Zaiming (Stone) Shi 55618dd28a
Merge pull request #11731 from zmstone/1008-expose-file-transfer-host-conf
fix(api-docs): add file-transfer config to hot-config scope
2023-10-17 11:23:16 +02:00
firest 467a2bc9b1 test(rbac): add test cases for RBAC in REST API 2023-10-17 16:23:38 +08:00
firest 26ec860d96 feat(backup): add migration mechanism when import backup data 2023-10-17 16:23:32 +08:00
firest e095de7367 feat(api_key): add RBAC feature for the API key 2023-10-16 14:10:01 +08:00
zhongwencool 1021088f5f fix: 500 error response when downloading non-existent trace files 2023-10-12 15:23:23 +08:00
zhongwencool 8e7ba16c3a fix: nothing show when run clients list command 2023-10-11 11:47:15 +08:00
Zaiming (Stone) Shi 808237364c fix(api-docs): add file-transfer config to hot-config scope 2023-10-10 10:24:10 +02:00
Ilya Averyanov 398a62031e chore: update app versions 2023-10-05 13:41:50 +03:00
Ilya Averyanov c2c56ba481 chore(auth): update tests 2023-10-05 13:41:50 +03:00
Ilya Averyanov 1eb75b43c4 chore(auth): split emqx_authn and emqx_authz apps 2023-10-05 13:41:50 +03:00
Zaiming (Stone) Shi ce5bd0a3ce Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 11:36:32 +02:00
Zaiming (Stone) Shi 922d5a9a83 fix(ldap): do not allow multi-matches to proceed
if ldap query returns more than on match
we should reject the auth request instead of picking
the first one
2023-09-28 21:20:50 +02:00
Zaiming (Stone) Shi 1a13b2ac56 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-28 13:53:03 +02:00
Ilya Averyanov 14983ec14a chore(hooks): validate hookpoints and document hook callbacks
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-26 14:27:42 +03:00
Zaiming (Stone) Shi d31bfc70fb
Merge pull request #11659 from zhongwencool/fix-listener-ssl-create-500
fix: create ssl listener return 500 crash
2023-09-26 11:38:07 +02:00
JianBo He 89e15e9134
Merge pull request #11665 from HJianBo/typo-fixes
Typo fixes
2023-09-25 17:23:40 +08:00
firest 4f4868a46c fix(sso): Disable access to `logout` endpoint by the `API key` 2023-09-25 10:31:38 +08:00
JianBo He 30862a94c6 chore: typo fixes 2023-09-24 20:19:05 +08:00
zhongwencool 0eed01abee fix: create ssl listener return 500 crash 2023-09-22 17:34:19 +08:00
firest 9e55ae240a feat(sso): add `role` into the result of login endpoints 2023-09-22 13:48:07 +08:00
Ivan Dyachkov dafd7c6085 chore: bump apps versions 2023-09-21 10:58:42 +02:00
Ivan Dyachkov 105bebc250 chore: merge release-52 into master 2023-09-21 10:22:47 +02:00
zhongwencool 2f1fa2e961 chore: unified slog message formatting to improve logging consistency 2023-09-20 18:13:00 +08:00
zhongwencool 640dd130fa fix: don't urldecode twice clientid 2023-09-15 17:36:48 +08:00
firest 45ccc66474 fix(dashbaord): fix test cases to support RBAC 2023-09-15 15:49:37 +08:00
Zaiming (Stone) Shi 7c2f87fabe test: merge broker and router boot modules 2023-09-06 21:36:16 +02:00
Zaiming (Stone) Shi e794143ae1 Merge remote-tracking branch 'origin/release-52' into 0906-sync-release-52-to-master 2023-09-06 09:08:22 +02:00
Andrew Mayorov d26995a2aa
Merge pull request #11524 from keynslug/ft/EMQX-10713/unified-route-tab
feat(router): add routing storage schema with 2 tables
2023-09-05 12:38:00 +04:00
Thales Macedo Garitezi 2cfe5ba057 fix(plugins_api): fix response types and test 2023-09-04 14:37:25 -03:00
Thales Macedo Garitezi 02c3d6a0a7 fix(data_import): rm duplicate import call and fix test 2023-09-04 14:19:59 -03:00
Ivan Dyachkov 24230a64df chore: bump app versions 2023-09-04 11:39:21 +02:00
Thales Macedo Garitezi 4dfbc859f9 fix(plugins): update plugin order on whole cluster
Fixes https://emqx.atlassian.net/browse/EMQX-10879
2023-08-30 16:39:52 -03:00
Andrew Mayorov 2da604dfa4
test(mgmt): simplify flaky `t_verify_imported_mnesia_tab_on_cluster`
And also rest of the test setup in the suite.
2023-08-30 20:24:18 +04:00
zhongwencool 501456efec
chore: more safe with iolist_to_binary
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-29 17:10:43 +08:00
zhongwencool 4245a4d8c6 fix: authz clean-cache clientid always return not_found 2023-08-29 08:17:47 +08:00
SergeTupchiy c04f770118
Merge pull request #11506 from SergeTupchiy/EMQX-10274-do-not-download-empty-trace-log
fix(emqx_trace): don't download empty trace log file
2023-08-28 22:17:00 +03:00
Serge Tupchii 0072749143 fix(emqx_trace): don't download empty trace log file
Closes: EMQX-10274
2023-08-28 19:25:55 +03:00
Thales Macedo Garitezi dec21ffc95
Merge pull request #11362 from thalesmg/ds-handle-sub-20230725
feat(ds): open iterators when handling `SUBSCRIBE` packets
2023-08-28 10:21:28 -03:00
Andrew Mayorov dcb63440bc
refactor(mgmt): avoid dealing with router tab directly
Instead, contain all this behind `emqx_router` module interface.
2023-08-28 13:52:59 +04:00
zhongwencool d8be248a3d fix: improve the suggest msg for update conf failed 2023-08-28 08:53:23 +08:00
Thales Macedo Garitezi f007b44426 fix(data_import): rm duplicate import call and fix test 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 9463e271c0 feat(ds): open iterators when handling `SUBSCRIBE` packets
Fixes https://emqx.atlassian.net/browse/EMQX-9741
2023-08-22 13:29:15 -03:00
Kjell Winblad 3ffbb7decf fix: HTTP API /api/v5/publish schema
The schema for the /api/v5/publish HTTP API endpoint was incorrect.
For 400 (Bad Request) error it cannot return a list but the incorrect
schema declared that the response could include a list.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10837
https://github.com/emqx/emqx/issues/11488
2023-08-22 15:21:20 +02:00
lafirest 5e448f5a02
Merge pull request #11446 from lafirest/refactor/calendar
refactor(calendar): refactor datetime-related code and remove redundant
2023-08-17 07:40:44 +08:00
zhongwencool 8b23ee86b3 fix: hide cpu_status if os_check is not supported 2023-08-16 20:49:28 +08:00
zhongwencool b817e03c08 fix: start os_mon application temporary 2023-08-16 20:49:27 +08:00
firest b08102269a refactor(calendar): refactor datetime-related code and remove redundant 2023-08-16 10:55:46 +08:00
firest 2fc0468e0c feat(banned): add a new API used to clear all banned data 2023-08-11 17:25:49 +08:00
JimMoen 060074e04a
fix: api total accumulate ignore `undefined` by down nodes result 2023-08-11 11:27:55 +08:00
zhongwencool aa798e531e
Merge pull request #11409 from zhongwencool/opentelemetry-metrics
feat: support opentelemetry metrics
2023-08-10 14:35:25 +08:00
zhongwencool d5fe891961 fix: don't be kill when send large payload when log is debug 2023-08-10 09:22:30 +08:00
zhongwencool 3933227636 feat: support opentelemetry metrics 2023-08-09 20:23:40 +08:00
Ilya Averyanov deaac9bd73 fix(authz): correctly identify qos of subscribe actions 2023-07-25 15:20:52 +03:00
Serge Tupchii b37920d2e5 feat: add `topic_metrics` and `slow_subs` configuration to data import/export
Fixes: EMQX-10590
2023-07-21 10:41:22 +03:00
zhongwencool 3cd223ff5b chore: add more detail msg for merging failed 2023-07-20 17:48:26 +08:00
ieQu1 57e39f42c5
Merge pull request #11294 from ieQu1/mria-fixes
Mria fixes
2023-07-19 15:10:42 +02:00
zhongwencool 0524c3b3b1 test: configs SUITE crash 2023-07-19 11:05:14 +08:00
JianBo He 439eb4d029 test: cover the node params for `get /configs` endpoint 2023-07-19 11:05:14 +08:00
JianBo He 74532a64e6 chore: update apps/emqx_management/src/emqx_mgmt_api_configs.erl 2023-07-19 11:05:14 +08:00
zhongwencool 402f75592e chore: move rpc call from emqx_management_proto to emqx_conf_proto 2023-07-19 11:05:14 +08:00
zhongwencool 59ed65787f fix: get_config return target node's node conf 2023-07-19 11:05:14 +08:00
ieQu1 3a5112829c fix(mgmt_cli): Don't print cluster status on the replicant 2023-07-18 21:45:00 +02:00
ieQu1 103b89f60c fix(mgmt_cli): Don't crash when mria is not running 2023-07-18 21:45:00 +02:00
ieQu1 293700773f refactor(mgmt_cli): Call mria directly 2023-07-18 21:45:00 +02:00
Thales Macedo Garitezi 91cd88c136 fix(api_mgmt): use `emqx_topic:join` for correct construction of topics 2023-07-18 10:19:55 -03:00
Thales Macedo Garitezi 1fdcfba629 fix(shared_sub): restore support for `$queue/` shared subscription
Partially addresses: https://emqx.atlassian.net/browse/EMQX-4589

There's still a problem with the handling of shared groups that is
_not particular to the `$queue/` syntax_ and preexistent that the same
client cannot subscribe to _different groups_ that have the same
_topic filter_.
2023-07-17 16:41:24 -03:00
Kjell Winblad f28510b3ad refactor: HTTP connector into emqx_bridge_http app 2023-07-12 14:46:43 +02:00
SergeTupchiy fce3060634
Merge pull request #11251 from SergeTupchiy/EMQX-10392-cluster-topology-api
Add cluster topology HTTP API endpoint
2023-07-11 17:45:00 +03:00
Serge Tupchii 19de10be7c feat(emqx_management): add /cluster/topology HTTP API endpoint
The endpoint shows Mria RLOG cluster topology info:
connections between core and replicant nodes.

Closes: EMQX-10392
2023-07-11 16:25:36 +03:00
JianBo He e51a4ae81e chore: avoid dialyzer warnings 2023-07-11 10:41:10 +08:00
JianBo He 15c91373fe perf(http): improve the speed of clients querying in default parameters
In https://github.com//emqx/emqx/pull/9749 we fixed the problem of
incorrect total calculation. In this PR we added an option to improve
the query speed when the query condition is empty
2023-07-11 10:41:10 +08:00
Stefan Strigler 93a9772743
Merge pull request #11211 from sstrigler/EMQX-9963-refactor-all-http-ap-is-to-respond-404-on-the-deletion-of-the-resource-is-non-exist
consistently return 404 if resource not found
2023-07-10 16:25:31 +02:00
Ilya Averyanov 12e237c36d
Merge pull request #11132 from savonarola/0531-authz-qos-retain
feat(authz): use extensible map format for actions in authz rules
2023-07-10 15:32:02 +03:00
zhongwencool 86d3984025 fix: mgmt api SUITE failed 2023-07-10 10:34:36 +08:00
zhongwencool d781346efc fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable 2023-07-10 10:34:36 +08:00
Stefan Strigler 80e4ffff75 fix(emqx_management): return 404 if plugin does not exist 2023-07-08 14:43:43 +02:00
Stefan Strigler 3fd28f9e18 fix(emqx_management): return 404 for unknown listener id 2023-07-08 14:43:43 +02:00
Ilya Averyanov 7de26a1776 feat(authz): use extensible map format for actions in authz rules
* support authorization on retain, qos fields
* refactored authz tests heavily
2023-07-07 19:40:34 +03:00
zhongwencool b5cc8fb3c3 fix: start_after_created's default value 2023-07-07 16:39:26 +08:00
JianBo He 71d1f80530 test(api): cover the accept logic for `/configs` API 2023-07-06 11:30:38 +08:00
JianBo He 966b2affc2 chore: allow handle */* or multiple values in Accept headers 2023-07-06 10:53:07 +08:00
Andrew Mayorov fe0b8cfbaf
test(conf): respect `init_config_load_done` in emqx_conf itself
This should make task of configuring various applications (emqx_conf
/ emqx) during testsuites setup much simpler.
2023-07-04 13:20:02 +02:00
Andrew Mayorov f76f3b77d7
test: add tooling to make testruns more isolated and predictable
Also showcase their usage in some of the existing testsuites.
2023-07-04 12:53:39 +02:00
zhongwencool d509c47344 feat: add conf load api 2023-07-04 14:17:11 +08:00
zhongwencool bb1ae808b4
Merge pull request #11165 from zhongwencool/hide-global-limiter-from-swagger-api
chore: hide /configs/limiter from swagger api doc
2023-06-30 11:54:01 +08:00
zhongwencool 3eefa1b2a3 chore: hide /configs/limiter from swagger api doc 2023-06-30 10:59:18 +08:00
zhongwencool 8b679cf358
Merge pull request #11137 from zhongwencool/dashboard-https-ssl-options
feat: refactor dashboard https ssl_options
2023-06-30 09:57:07 +08:00
zhongwencool 82ea9e5a04 test: dashbaord api ct failed 2023-06-29 10:49:39 +08:00
zhongwencool 30b67add05 chore: add more example for clients api 2023-06-28 10:55:55 +08:00
zhongwencool 93cbb5ebd8 fix: show the local listener's settings first 2023-06-28 10:55:55 +08:00
Zaiming (Stone) Shi 7cf8a6c892 chore: bump app vsns 2023-06-21 16:36:51 +02:00
Zaiming (Stone) Shi 5fa87091c4 Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-21 16:08:41 +02:00
ieQu1 e5803a61f1 fix(management): Don't wait for an empty shard 2023-06-20 10:08:41 +02:00
zhongwencool 6efb07f6eb fix: crash on emqx_ctl listeners 2023-06-19 18:12:33 +08:00
zhongwencool 803cd6c812
Merge pull request #11082 from emqx/release-51
sync release-51 to master
2023-06-19 15:12:37 +08:00
Andrew Mayorov 13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
2023-06-17 00:54:24 +02:00
JimMoen c4e7129cb6
fix: use mria.cluster_autoclean and mria.cluster_autoheal 2023-06-16 14:04:24 +08:00
zhongwencool fc1d78134a test: replace 'to' with 'path' 2023-06-15 19:13:51 +08:00
zhongwencool e42cc58694
Merge pull request #11050 from emqx/release-51
chore: sync release-51 back to master
2023-06-15 09:45:31 +08:00
Andrew Mayorov acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
2023-06-15 00:50:15 +03:00
zhongwencool 267053cc35
Merge pull request #11045 from zhongwencool/hide-zones-authn-in-listeners
feat: hide zone/authn in listeners and remove listeners's authn api
2023-06-14 20:32:27 +08:00
zhongwencool bd33dcb049 feat: remove zone from /clients api 2023-06-14 17:11:20 +08:00
zhongwencool 7668753f50 feat: hide zone/authn in listeners and remove /listeners/:id/authentication api 2023-06-14 15:17:12 +08:00
zhongwencool feb08b1181 chore: update 11042 changelog 2023-06-14 11:40:20 +08:00
zhongwencool 091c49277f fix: api_listener crash when setting max_connnections as string 2023-06-14 11:26:41 +08:00
Thales Macedo Garitezi 5ef7102cba fix(listener_mgmt_api): use union member selector fn for better error messages (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10247
2023-06-12 16:48:13 -03:00
Serge Tupchii 2aad6671ac fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
As we use `keep_tables` option, we don't need to modify a backup schema before importing the backup file.
With this option, `mnesia:restore/2` ignores backup schema and keeps current table schema unchanged.
2023-06-09 14:41:21 +03:00
SergeTupchiy e61b2100a3
Merge pull request #10676 from SergeTupchiy/EMQX-9203-config-backup
feat: implement configuration and user data export/import
2023-06-09 14:18:39 +03:00
Serge Tupchii e4d09d4ad4 feat: implement configuration and user data export/import CLI
Closes: EMQX-9203
2023-06-09 14:11:47 +03:00
Zaiming (Stone) Shi 7d2ea3dc76 docs: refine some desc 2023-06-08 13:58:31 +02:00
Zaiming (Stone) Shi 641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51
0607 merge master to release 51
2023-06-08 09:22:21 +02:00
JianBo He fcef456174 feat: add `live_connections` field for some HTTP APIs
i.e:
- `/monitor_current`, `/monitor_current/nodes/{node}`
- `/monitor/nodes/{node}`, `/monitor`
- `/node/{node}`, `/nodes`
2023-06-08 11:24:06 +08:00
Zaiming (Stone) Shi 641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf
feat: don't hide enable in flapping detect conf
2023-06-07 23:04:56 +02:00
Zaiming (Stone) Shi ccd2589ff2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-07 21:43:29 +02:00
JimMoen 65483e972e
refactor: use unified table define 2023-06-07 22:51:44 +08:00
zhongwencool 535870386a fix: zones update api failed 2023-06-07 21:54:32 +08:00
Andrew Mayorov 99ea9b86c2
feat(tlslib): add separate managed certfiles GC process
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
2023-06-05 22:41:07 +03:00
Zaiming (Stone) Shi 28c564d15b Merge remote-tracking branch 'origin/master' into release-51 2023-06-02 16:25:48 +02:00