Commit Graph

1035 Commits

Author SHA1 Message Date
ieQu1 e439a2e0f2
fix(sessds): Save protocol name and version in the session metadata 2024-04-17 12:31:54 +02:00
ieQu1 197a4c30be
fix(sessds): Strip unneccessary data from the durable session state 2024-04-17 01:21:52 +02:00
ieQu1 6c83bbe10b
feat(mgmt): Filter subscriptions by durability 2024-04-17 01:21:51 +02:00
Ivan Dyachkov 3ef160eed2 Merge remote-tracking branch 'upstream/release-56' into 0415-sync-release-56 2024-04-15 08:09:03 +02:00
Kjell Winblad ed5409fb6a docs(trace): add emqx_ctl documentation for the new ruleid trace 2024-04-12 11:54:14 +02:00
JianBo He 91ffc95f29
Merge pull request #12855 from JimMoen/fix-share-queue-format
fix(mgmt): $queue shared topics format in mgmt topics api
2024-04-10 10:52:48 +08:00
JimMoen 47e0f3bb1f
fix(mgmt): $queue shared topics format in mgmt topics api 2024-04-09 17:06:32 +08:00
JimMoen 6cb00cc8c7
Merge pull request #12844 from JimMoen/EMQX-12012/cpu-use-idle-two-decimal-places
fix: cpu usage and idle use two decimal places
2024-04-09 10:11:15 +08:00
Thales Macedo Garitezi ba96edb061 fix(clients api): use alternative base64 function for OTP 25
Fixes https://github.com/emqx/emqx/pull/12798#discussion_r1555524603
2024-04-08 10:23:10 -03:00
JianBo He e11c4a9c83
Merge pull request #12826 from emqx/import-source-bridges
fix: source bridges missing after restore the backup files
2024-04-08 16:02:15 +08:00
JimMoen 282cbb18be
fix: cpu usage and idle use two decimal places
- prometheus
- opentelemetry
2024-04-08 14:14:09 +08:00
ieQu1 2504b8126b
feat(ds): Pass mgmt_ds REST API calls to the application 2024-04-05 15:22:06 +02:00
ieQu1 46261440cb
feat(ds): Add a CLI for managing DB replicas 2024-04-05 15:22:06 +02:00
ieQu1 a62db08676
feat(ds): Add REST API for durable storage 2024-04-05 15:22:06 +02:00
Kjell Winblad 59a442cdb5 feat(rule trace): add support for ruleid as a trace type 2024-04-04 14:55:32 +02:00
Shawn 319ec50c0d fix: source bridges missing after restore the backup files 2024-04-03 18:26:51 +08:00
Thales Macedo Garitezi 2097e854fc feat(client mgmt api): add cursor-based list API
Fixes https://emqx.atlassian.net/browse/EMQX-12028
2024-04-02 10:55:28 -03:00
JimMoen 1a4cfc2a2d
fix(api_schema): removed metrics schema in api spec
- Followup [PR#6622](https://github.com/emqx/emqx/pull/6622).
2024-04-02 16:56:36 +08:00
Serge Tupchii ceb04ba06d fix(emqx_mgmt): do not attempt to get a stacktrace of a remote client connection process 2024-04-01 16:42:12 +03:00
Serge Tupchii 42af1f9d63 fix: handle internal timeout errors in client Mqueue/Inflight APIs 2024-03-29 23:03:35 +02:00
Serge Tupchii f5a820cb10 fix(emqx_mgmt): catch OOM shutdown exits properly when calling a client conn process
The exit reason is expected to include gen_server `Location`:
  `{{shutdown, OOMInfo}, Location}`.
2024-03-29 13:09:08 +02:00
SergeTupchiy 2e528d1dd8
Merge pull request #12802 from SergeTupchiy/EMQX-11826-prevent-left-node-from-rejoining-5.6.1
prevent a left node from rejoining the same cluster
2024-03-28 19:49:18 +02:00
Serge Tupchii 3eda182e9a fix: prevent a node from discovering and re-joining the same cluster after it has (manually) left it. 2024-03-28 18:09:27 +02:00
Serge Tupchii 1d1f595e6f fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid 2024-03-28 12:54:51 +02:00
Ivan Dyachkov db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Ivan Dyachkov f4446ec680 Merge remote-tracking branch 'upstream/release-56' into 0328-sync-release-56 2024-03-28 09:59:54 +01:00
Serge Tupchii ed0b5ac939 fix(emqx_mgmt): handle client shutdown errors in call_client
The client process may be shutdown normally during/shortly before the call.
2024-03-22 16:47:02 +02:00
Serge Tupchii 5d77e1b01e fix: return not_implemented error for mqueue/inflight msgs queries 2024-03-22 10:02:31 +02:00
JimMoen 9b085b5797
fix(api_schema): removed metrics schema in api spec
- Followup [PR#6622](https://github.com/emqx/emqx/pull/6622).
2024-03-22 10:57:14 +08:00
Serge Tupchii 40eccb10d6 fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid 2024-03-21 17:36:39 +02:00
Andrew Mayorov 7d6fde5960
fix(topics-api): respond with correct totals to paged queries 2024-03-21 16:02:16 +01:00
ieQu1 cada944350
feat(sessds): Expose relevant durable session info in the REST API 2024-03-21 10:37:04 +01:00
SergeTupchiy 27c1f1a4e4
Merge pull request #12732 from SergeTupchiy/EMQX-11861-client-mqueue-inflight-API-updates
EMQX-11861 client mqueue inflight api updates
2024-03-20 16:27:44 +02:00
Serge Tupchii cb5fdb3c79 fix: rework In-flight / Mqueue API
- use timestamp 'position' to find the next chunk of data
- add 'start' position to response meta
- sort In-flight messages by insertion time
- sort Mqueue messages by priority
2024-03-20 15:25:33 +02:00
ieQu1 783f769a98 fix(mgmt): Fix bulk kickout of durable sessions 2024-03-20 12:52:44 +01:00
ieQu1 611e4f6710
Merge pull request #12740 from ieQu1/dev/kick-durable-sessions
fix(sessds): Channel must destroy sessions that are kicked
2024-03-19 21:30:40 +01:00
Andrew Mayorov e2db038e7c
Merge pull request #12734 from keynslug/fix/EMQX-12030/topics-api
fix(api-topics): expose persistent session topics in APIs
2024-03-19 21:08:23 +01:00
ieQu1 236685f714 fix(sessds): Channel must destroy sessions that are kicked 2024-03-19 17:49:04 +01:00
Ivan Dyachkov f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
JianBo He c7db157ecd
Merge pull request #12735 from zhongwencool/fix-ban-crash
fix: don't crash when ban ip error
2024-03-19 20:57:54 +08:00
zhongwencool 111cbbbe3c fix: don't crash when ban ip error 2024-03-19 19:10:36 +08:00
Andrew Mayorov 985f1c4879
fix(api-topics): expose persistent session topics in APIs 2024-03-19 11:01:26 +01:00
Serge Tupchii b3d44125f6 feat: support multiple clientid / username Qs params in "/clients" API
Additionally, add an option to specify which Client info fields to return in the response.
2024-03-18 18:00:26 +02:00
ieQu1 a93f747afa fix(sessds): Return peername of the disconnected client in the REST 2024-03-15 13:48:53 +01:00
SergeTupchiy e35e8847b9
Merge pull request #12685 from SergeTupchiy/refactor-emqx-mgmt-call-client
Refactor emqx mgmt call client
2024-03-15 10:01:49 +02:00
zmstone d8bdb3c9aa fix(api/clients): drop expired sessions from durable storage 2024-03-14 20:57:53 +01:00
Zaiming (Stone) Shi cb77dea1e9
Merge pull request #12699 from zmstone/0313-fix-session-count-on-replicant-node
fix: do not crash on replicant node
2024-03-14 09:42:27 +01:00
zmstone ccd973d13e fix: do not crash on replicant node 2024-03-13 16:42:39 +01:00
Serge Tupchii e514c4a2ac fix(emqx_mgmt_api_clients): check that max_payload_bytes is higher than 0 2024-03-13 14:00:30 +02:00
Serge Tupchii 65be76aa06 refactor(emqx_mgmt): add call_client timeout and improve RPC error handling 2024-03-13 10:56:02 +02:00
Serge Tupchii 8be02327b2 refactor(emqx_mgmt): avoid call_client RPC to all runing nodes if global cm_registry is enabled 2024-03-12 20:58:53 +02:00
JimMoen 207f38c42a
fix(vm): cpu usage/idle handled by single worker 2024-03-12 14:14:44 +08:00
Serge Tupchii 980a4c3a4d fix(emqx_mgmt_api): use emqx_utils:bin_to_hexstr/2 2024-03-11 10:40:19 +02:00
Serge Tupchii f376aa8072 refactor(emqx_mgmt_api_clients): remove unnecessary function 2024-03-11 10:39:09 +02:00
zmstone 2cc1c563df fix(mgmt): avoid using base64:decode/2 and encode/2
they are not available in otp 25
2024-03-08 16:46:54 +01:00
SergeTupchiy 1f38813cb9
Merge pull request #12561 from SergeTupchiy/EMQX-11861-client-mqueue-inflight-API
feat: add client mqueue/inflight messages API
2024-03-07 13:20:31 +02:00
Serge Tupchii a2e761681e feat: add client mqueue/inflight messages API 2024-03-07 12:06:49 +02:00
zmstone f57f2fa1b7 chore: bump app version numbers 2024-03-06 19:37:06 +01:00
Thales Macedo Garitezi 0dd9990f24 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 09:44:30 -03:00
JimMoen 9143d5994d
feat: refactor MS SQL Server bridge to connector and action 2024-03-04 17:13:21 +08:00
Andrew Mayorov c3b044a37b
fix(api-topics): respond with `count` when it's cheap to compute 2024-02-28 11:28:33 +01:00
Andrew Mayorov bd578a799e
fix(api-topics): avoid doing full scans over router tables 2024-02-28 10:52:44 +01:00
Andrew Mayorov 9bb0868628
test(api-topics): add testcase verifying paged queries work 2024-02-28 10:52:41 +01:00
JimMoen c6b1102b2b
fix(mgmt_client): `nl` not allowed for shared-sub 2024-02-27 17:51:39 +08:00
JimMoen a76415c4f6
fix(mgmt): sub/unsub a share subscription to the client via http api
- `/clients/:clientid/subscribe`
- `/clients/:clientid/subscribe/bulk`
- `/clients/:clientid/unsubscribe`
- `/clients/:clientid/unsubscribe/bulk`
2024-02-27 16:00:45 +08:00
Zaiming (Stone) Shi 5af01c041b
Merge pull request #12559 from zmstone/0221-refactor-use-atom-fileds
refactor: use atoms for root config fields
2024-02-23 14:38:19 +01:00
Zaiming (Stone) Shi 46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
Zaiming (Stone) Shi 88b1d9ba88 refactor: use atoms for root config fields and types 2024-02-22 16:51:40 +01:00
firest f0aeaadc48 fix(api_key): enhanced bootstrap files for REST API keys 2024-02-22 18:40:24 +08:00
Thales Macedo Garitezi 03138f8345 refactor(mgmt): move persistent session counting function to mgmt module
Follow up to https://github.com/emqx/emqx/pull/12500#discussion_r1496634087
2024-02-21 09:12:56 -03:00
Thales Macedo Garitezi 529211b9ac
Merge pull request #12500 from thalesmg/ds-list-client-api-m-20240209
feat(ds): list disconnected persistent sessions in clients API
2024-02-21 09:02:27 -03:00
Thales Macedo Garitezi d469f4158e chore: bump app vsns 2024-02-20 16:53:57 -03:00
Thales Macedo Garitezi 3a4c7f60e2 feat(ds): list disconnected persistent sessions in clients API
Fixes https://emqx.atlassian.net/browse/EMQX-11540

Note that not all information provided by disconnected in-memory sessions is available to
disconnected persistent sessions, nor does all of them make sense.
2024-02-20 14:52:55 -03:00
Zaiming (Stone) Shi 2b46cbab7a
Merge pull request #12517 from zmstone/0213-hocon-multiline-string-with-indentation
0213 hocon multiline string with indentation
2024-02-16 15:12:46 +01:00
Zaiming (Stone) Shi f57f617ba3 refactor(schema): ensure roots/0 and namespace/0 for all schema modules 2024-02-16 11:35:32 +01:00
Andrew Mayorov b6d77c164e
feat(emqx-mgmt): manage mria tables explicitly during startup 2024-02-13 20:36:22 +01:00
Ilya Averyanov 90fd2b26d3 feat(banned): allow ban by clientid/username regexps, peerhost cidrs 2024-02-10 17:59:22 +03:00
Zaiming (Stone) Shi b1a05c7b59
Merge pull request #12326 from zmstone/0111-unregister-session-with-timestamp
0111 unregister session with timestamp
2024-02-02 17:00:23 +01:00
ieQu1 3d2ac97c61 feat(ds): Add a CLI interface to inspect status of DS databases 2024-02-02 10:11:01 +01:00
Zaiming (Stone) Shi f0569d8ae8 refactor: use mria:async_dirty to group dirty ops 2024-02-02 09:26:43 +01:00
ieQu1 98d1094d73 feat(sessds): Expose subscriptions in the REST API 2024-02-01 16:47:57 +01:00
Zaiming (Stone) Shi 87a2368e37 feat(metrics): add cluster_session guage 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 509ab6f35a feat(api): add /sessions_count api to count sessions 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 562a2736ae feat: add `broker` root to hot-config schema 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 55fdc83aa5 Merge remote-tracking branch 'origin/release-55' into 0130-syn-release-55-to-master 2024-01-30 12:06:43 +01:00
JianBo He 4190682a16 chore: fix the data type and example value for cluster invitation result 2024-01-30 15:29:46 +08:00
zhongwencool 186e1591df chore: document api-key path api only support bearerAuth 2024-01-25 16:01:43 +08:00
JianBo He 52a7b45d97 fix: upgrade ekka to 0.18.3 to ensure the leader correct 2024-01-25 14:47:59 +08:00
JianBo He 4c40e754f4 chore: forward the async invite to leader node 2024-01-15 11:24:05 +08:00
JianBo He 93ef6766ef chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-13 15:29:58 +08:00
JianBo He d2991dae03 chore: make dialyzer happy 2024-01-13 15:29:58 +08:00
JianBo He 81209e0ded chore: update changes 2024-01-13 15:29:58 +08:00
JianBo He 944137ad45 chore: format error message 2024-01-13 15:29:58 +08:00
JianBo He 6ff4c560e4 feat: support invite node in async mananer 2024-01-13 15:29:58 +08:00
JianBo He a08f56db42 feat(cluster): expose the timeout parameter to invite node 2024-01-13 15:29:58 +08:00
lafirest c026d3f18a
Merge pull request #12300 from lafirest/fix/clients_api
fix(api): fix `api/v5/clients ` may crash when a client authenticate with JWT
2024-01-11 19:10:54 +08:00
firest aca9942fd4 fix(api): fix `api/v5/clients ` may crash when a client authenticate with JWT 2024-01-11 16:53:02 +08:00
JianBo He da0307faa9 chore: update apps/emqx_management/src/emqx_mgmt_api_clients.erl
Co-authored-by: William Yang <mscame@gmail.com>
2024-01-10 09:52:27 +08:00
JianBo He f45298d14a chore: improve the error message if the parameter invalid 2024-01-10 09:52:27 +08:00
JianBo He 57c0a7c5ef
Merge pull request #12235 from zhongwencool/ft-config-update-api
fix: ft config update api return raw config
2024-01-03 14:07:22 +08:00
JimMoen a3f33cc8bc
chore: fix typos in code and suite 2023-12-29 09:07:51 +08:00
zhongwencool e645fa4874 fix: ft config update api return raw config 2023-12-27 21:13:47 +08:00
zhongwencool 11feab983f fix: clean rpc_commit when force_leave cluster 2023-12-26 15:25:58 +08:00
Zaiming (Stone) Shi 86a018d921 chore: bump app versions 2023-12-22 21:28:26 +01:00
Zaiming (Stone) Shi 9fdac4af0c Merge remote-tracking branch 'origin/master' into 1222-sync-e5.4.0-build.2-to-master 2023-12-22 21:27:27 +01:00
Zaiming (Stone) Shi ac6e4cf2cc fix: config update for ft 2023-12-22 19:01:14 +01:00
Zaiming (Stone) Shi 23ded313ec chore: update app versions 2023-12-22 15:29:22 +01:00
Zaiming (Stone) Shi 20543d55ef chore: bump app vsn 2023-12-22 13:13:30 +01:00
Zaiming (Stone) Shi 322b7bb7d2 chore: bump app vsn 2023-12-22 13:00:37 +01:00
Thales Macedo Garitezi 035f5f977e Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231221 2023-12-21 17:53:05 -03:00
zhongwencool 5d1a412d0c fix: redis resource_opts not working 2023-12-21 18:29:49 +08:00
zhongwencool a4fe4ceb9e chore: add example for clients api 2023-12-20 14:53:05 +08:00
Zaiming (Stone) Shi 35504bd323 refactor: move ntoa (ip address formatting) code to emqx_utils 2023-12-17 21:03:16 +01:00
zhongwencool 96d21d4dbe fix: ensure config update error text is readable 2023-12-13 17:07:28 +08:00
JimMoen 7b60928288
fix(api): endpoint `/v5/topics` with invalid topic filter 2023-12-11 15:36:50 +08:00
Zaiming (Stone) Shi 7e3ed05f87
Merge pull request #12115 from zmstone/1206-fix-dialyzer
1206 fix dialyzer
2023-12-07 12:24:47 +01:00
Zaiming (Stone) Shi 423b586c56 fix(dialyzer): fix some dialyzer issues found on otp 26 2023-12-06 20:32:49 +01:00
zhongwencool b015d08cba fix: use sync_transation when update admin/token 2023-12-06 09:04:27 +08:00
Zaiming (Stone) Shi 7f4d91d490 Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54 2023-12-01 08:32:22 +01:00
Ivan Dyachkov 9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Zaiming (Stone) Shi dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26
1129 prepare for OTP 26
2023-11-30 19:38:40 +01:00
Serge Tupchii f54bda7063 fix(emqx_mgmt_api_data_backup): add count and hasnext meta paging params 2023-11-30 19:10:12 +02:00
Zaiming (Stone) Shi 5427ebc5f1 fix: log error when failed to install plugin
the error return is a map with details.
prior to this change only the reason is returned in the api response,
now the error map is logged at error level to help troubleshooting
2023-11-30 16:49:30 +01:00
Zaiming (Stone) Shi 14644988e0 chore: change triple-quotes to single-quotes 2023-11-29 16:15:18 +01:00
Serge Tupchii 009a15c7d0 fix(emqx_mgmt_api_data_backup): validate empty file in upload API
fixes EMQX-11488
2023-11-29 12:12:42 +02:00
Serge Tupchii 89fdbf3e73 fix(emqx_mgmt_cli): remove duplicated word in the log message 2023-11-27 17:30:58 +02:00
Serge Tupchii 64ee29af81 feat(emqx_management): implement data backup API 2023-11-27 17:30:58 +02:00
zhongwencool b5a00ec6b2
Merge pull request #12023 from emqx/master
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
Andrew Mayorov 110a5a4896
feat(schema): introduce separate root for new session persistence
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
2023-11-20 23:49:14 +07:00
Andrew Mayorov d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Andrew Mayorov 6812ee9d0f
fix(mgmt): hide route selection behind router interface
Also introduce a generic _stream_ concept, mostly to deal with
iterating over 2 ETS tables at once with `ets:match_object/3`.
2023-11-14 18:36:25 +07:00
Ivan Dyachkov 28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Zaiming (Stone) Shi 86110824eb feat: upgrade hocon to 0.40.0 which supports union type display name 2023-11-10 13:41:51 +01:00
Ivan Dyachkov 0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
Ivan Dyachkov b1ab213081 chore: merge 'upstream/release-53' into 1031-sync-r53 2023-10-31 11:06:25 +01:00
Serge Tupchii 0562585c8f fix(emqx_mgmt_data_backup): implement importer modules ordering
`emqx_bridge_v2` depends on `emqx_connector`, so connectors must be imported first.
2023-10-30 19:14:58 +02:00
Serge Tupchii d1cd5dd817 fix(emqx_mgmt_data_backup): upgrade raw conf before validating and importing 2023-10-30 17:52:39 +02:00
JimMoen 3a09fdc495
refactor: check match topic before do subscriptions query 2023-10-30 16:45:29 +08:00
JimMoen 814e22feb3
fix: topics/subscripton mgmt api searching 2023-10-30 14:42:47 +08:00
JimMoen a2015f37ae
Merge pull request #10976 from JimMoen/fix-shared-sub-unsub 2023-10-27 21:22:27 +08:00
firest 4ba34f8f3e chore(rbac): fix CI errors & update change 2023-10-27 17:17:04 +08:00
JimMoen b5411da770
refactor: subscribe process to fix shared-sub 2023-10-27 09:23:43 +08:00
firest ec41479633 feat(rbac): supports setting role in API bootstrap file 2023-10-24 23:03:18 +08:00
firest e175c213a1 fix(rbac): for compatibility with old data schema, extend the existing field as extra 2023-10-24 22:28:53 +08:00
JimMoen 99fab8dc5d
chore: missing change log for duplicated apikey fixing 2023-10-24 09:44:39 +08:00
JimMoen e6576951ef
test: cleanup duplicated apikey with different name 2023-10-20 22:33:26 +08:00
JimMoen d467289bb2
fix: avoid duplicated apikey from data import 2023-10-20 21:13:04 +08:00
zhongwencool 926c804314 feat: add /audit http api to filter audit log 2023-10-20 14:47:24 +08:00
zhongwencool 8b5ec2a4d5 fix: xref failed 2023-10-20 11:46:31 +08:00