Commit Graph

10876 Commits

Author SHA1 Message Date
Ilya Averyanov 5bc67cb288
Merge pull request #12432 from ieQu1/ds_ignore_fully_replayed_streams
fix(sessds): Stream scheduler must ignore fully replayed streams
2024-01-31 09:51:49 +02:00
firest 13df7fa46e fix(dashboard): add test case & update change 2024-01-31 14:14:56 +08:00
ieQu1 e843d9fd91 fix(sessds): Stream scheduler must ignore fully replayed streams 2024-01-31 00:20:54 +01:00
Kjell Winblad 274f378c6e
fix: better delete of local_topic as suggested by @thalesmg
Co-authored-by:  Thales Macedo Garitezi <thalesmg@gmail.com>

Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-01-30 17:53:32 +01:00
lafirest c7f1e44513
Merge pull request #12424 from lafirest/fix/opentsdb
feat(opentsdb): supports more flexible tags schema
2024-01-30 22:05:30 +08:00
Zaiming (Stone) Shi 55fdc83aa5 Merge remote-tracking branch 'origin/release-55' into 0130-syn-release-55-to-master 2024-01-30 12:06:43 +01:00
JianBo He c0674913e2
Merge pull request #12421 from HJianBo/fix-import-users
fix(user_import): ensure the last record overwrites previous one
2024-01-30 18:49:35 +08:00
firest 81f96f1a68 feat(opentsdb): supports more flexible tags schema 2024-01-30 18:43:39 +08:00
SergeTupchiy 03ff6f2ddf
Merge pull request #12410 from SergeTupchiy/EMQX-11812-subscribe-performance-degraded
perf(emqx_broker): pick broker pool worker by topic/shard pair to dis…
2024-01-30 11:48:59 +02:00
Xinyu Liu db3f285054
Merge pull request #12417 from emqx/port-msg-expiry-interval-from44
feat: configurable server side message_expiry_interval
2024-01-30 17:10:32 +08:00
Shawn 206af96a33 ci: update testcases for message-expiry-interval 2024-01-30 15:46:11 +08:00
JianBo He 4190682a16 chore: fix the data type and example value for cluster invitation result 2024-01-30 15:29:46 +08:00
JianBo He aedfc8e8c0 fix(user_import): ensure the last record overwrites previous one 2024-01-30 14:14:20 +08:00
JianBo He 35c4ef2ee2
Merge pull request #12407 from zmstone/0126-script-to-update-bsl-license-convert-time
0126 script to update bsl license convert time
2024-01-30 09:54:34 +08:00
JianBo He 4688b36cdf
Merge pull request #12396 from HJianBo/import_user_2
feat(import_users): support user's password in plain text
2024-01-30 09:00:00 +08:00
Serge Tupchii 5007650bd2 perf(emqx_broker): pick broker pool worker by topic/shard pair to distribute the load more evenly.
Fixes: EMQX-11812
2024-01-29 20:57:10 +02:00
Thales Macedo Garitezi cf0465dad4
Merge pull request #12411 from thalesmg/fix-cassandra-null-r55-20240126
fix(cassandra_bridge): correctly insert null values into columns
2024-01-29 14:09:12 -03:00
Kjell Winblad b849b08dbc fix: kinesis schema problems found by @HJianBo
This commit fixes problems in schema found by @HJianBo here:

https://github.com/emqx/emqx/pull/12376#pullrequestreview-1848306286
2024-01-29 18:08:53 +01:00
ieQu1 7ee0a6aaa4
Merge pull request #12414 from ieQu1/dev/ds-readme
doc(ds): Update README
2024-01-29 18:01:03 +01:00
Zaiming (Stone) Shi 82403167c2 chore: update BSL license change date 2024-01-29 16:47:31 +01:00
Thales Macedo Garitezi 47f61ba68a fix(cassandra_bridge): correctly insert null values into columns
Fixes https://emqx.atlassian.net/browse/EMQX-11822
2024-01-29 11:32:00 -03:00
ieQu1 cb90d3c90c
Merge pull request #12415 from ieQu1/dev/remove-ds-conf-module
fix(ds): Remove unused module
2024-01-29 12:56:04 +01:00
Shawn 9f22c2c455 ci: add some sleep and retry to emqx_persistent_session_ds_SUITE 2024-01-29 18:18:18 +08:00
JianBo He d12335c4c6 chore: add tests 2024-01-29 18:10:42 +08:00
JianBo He d286b7e28c
Merge pull request #12404 from thalesmg/fix-action-metric-handler-r55-20240126
fix(resource_metrics): avoid detaching handler on crashes
2024-01-29 17:51:04 +08:00
Shawn 12da3c0986 feat: configurable server side message_expiry_interval 2024-01-29 17:29:52 +08:00
Ivan Dyachkov fa6d65887d test(retainer): fix test cases 2024-01-29 08:42:36 +01:00
Ivan Dyachkov e4b8d79444 fix(retainer): add default delivery rate in the schema 2024-01-29 08:04:58 +01:00
JianBo He ab99a17c99 chore(utils_stream): simplfy the csv reader implementation 2024-01-29 11:10:50 +08:00
JianBo He 2e35024df1 test: update eunit tests 2024-01-29 10:49:07 +08:00
JianBo He 9915c85b0b chore(authn_mnesia): use emqx_utils_stream module to imporve reusability 2024-01-29 10:49:07 +08:00
JianBo He 8fc8106819 test: cover password_type and new data format 2024-01-29 10:49:07 +08:00
JianBo He 829887630d test: refine existed test cases 2024-01-29 10:49:07 +08:00
JianBo He fed512689a chore: make elvis checking happy 2024-01-29 10:49:07 +08:00
JianBo He e65cfb836c feat(import_users): support user's password in plain text 2024-01-29 10:49:07 +08:00
ieQu1 2479e1189a fix(ds): Remove unused module 2024-01-29 00:36:13 +01:00
ieQu1 96c527541b docs(ds): Update README 2024-01-28 20:27:20 +01:00
Zaiming (Stone) Shi dbe2baa405
Merge pull request #12408 from zmstone/sync-release-55
Sync release 55
2024-01-27 19:20:58 +00:00
ieQu1 2c6a776318 fix(sessds): Stricter checks for PacketIds 2024-01-26 17:49:33 +01:00
ieQu1 eec56b0d6b fix(sessds): Improve comments 2024-01-26 17:49:33 +01:00
ieQu1 3c451c6ae6 test(sessds): Fix unstable tests 2024-01-26 17:49:33 +01:00
ieQu1 2d23212792 refactor(sessds): #ifs -> #srs 2024-01-26 17:49:33 +01:00
ieQu1 974760d331 test(sessds): Don't trap exits in the test 2024-01-26 17:49:33 +01:00
ieQu1 ebe2339810 fix(sessds): Use mria:async_dirty instead of transaction 2024-01-26 17:49:33 +01:00
ieQu1 f5b9bd30aa fix(sessds): Apply review remarks 2024-01-26 17:49:33 +01:00
ieQu1 39857626ce test(sessds): Fix failing tests 2024-01-26 17:49:33 +01:00
ieQu1 e7b03cdc59 test(sessds): Create a property-based test for the session state 2024-01-26 17:49:33 +01:00
ieQu1 893656f092 fix(sessds): Fix session garbage collection after the refactoring 2024-01-26 17:49:33 +01:00
ieQu1 963df8f941 fix(sessds): Delete the routes when the session expires 2024-01-26 17:49:33 +01:00
ieQu1 cff6c15e13 fix(sessds): Store the QoS as the MSB of the packet ID 2024-01-26 17:49:33 +01:00
ieQu1 978a3bfef3 refactor(sessds): Simplify representation of QoS tracks 2024-01-26 17:49:33 +01:00
ieQu1 1b4f69b44d refactor(sessds): Simplify data structure of ds_state pmap datatype 2024-01-26 17:49:33 +01:00
ieQu1 4f4831fe7f refactor(sessds): Factor out stream scheduler into its own module 2024-01-26 17:49:33 +01:00
ieQu1 82ef34998a feat(sessds): Index streams by a unique subid 2024-01-26 17:49:33 +01:00
ieQu1 3fb2064ea4 test(sessds): Add property-based tests for seqno generator 2024-01-26 17:49:33 +01:00
ieQu1 8e8d3af096 fix(sessds): Refactor emqx_persistent_session_ds to use CRUD module 2024-01-26 17:49:33 +01:00
ieQu1 2d08aa88d8 refactor(ds): Create a CRUD module for the persistent session 2024-01-26 17:49:33 +01:00
Thales Macedo Garitezi 6410f5a717 fix(resource_metrics): avoid detaching handler on crashes
Fixes https://emqx.atlassian.net/browse/EMQX-11821
2024-01-26 13:05:41 -03:00
Zaiming (Stone) Shi 2a40152721 Merge remote-tracking branch 'origin/release-55' into sync-release-55 2024-01-26 16:51:13 +01:00
William Yang 933c00c7ad
Merge pull request #12392 from qzhuyan/perf/william/ws-no-utf8-validate
perf: new ws listener option to disable UTF-8 validation
2024-01-26 16:01:04 +01:00
ieQu1 0933dc818e chore(ekka): Bump version to 0.18.4
Don't escalate the errors in etcd node discovery to the node level.

Fixes: #12255
2024-01-26 14:25:16 +01:00
Serge Tupchii a56a5e9c3c chore: add copyright 2024-01-26 11:01:53 +02:00
Serge Tupchii 2d693402c5 refactor: split greptimedb bridge to actions and connectors 2024-01-26 11:01:53 +02:00
William Yang 726302ef6a perf: new ws listener option to disable UTF-8 validation 2024-01-26 09:55:17 +01:00
JimMoen 2e37f7514a
fix(prom_stats): compatibility with previous api format in json mode 2024-01-26 16:01:32 +08:00
JimMoen 308cde53b9
fix(prom_stats): merge cl_consistented data into per nodes data 2024-01-26 11:39:20 +08:00
JianBo He e29a80d568
Merge pull request #12388 from qzhuyan/dev/william/per-listn-conn-cnt
feat: quic per listener conn count
2024-01-26 09:01:39 +08:00
Thales Macedo Garitezi 590746ddfa
Merge pull request #12400 from thalesmg/ds-fix-data-dir-schema-m-20240125
fix(ds): don't make data dir part of the schema
2024-01-25 15:52:11 -03:00
Thales Macedo Garitezi 8e31afe6c2 fix(ds): don't make data dir part of the schema
The data directory was ending up being persisted in the database schema.  This led to
issues when opening the DB on different nodes.
2024-01-25 14:44:06 -03:00
lafirest 0b6aee0ae9
Merge pull request #12393 from lafirest/fix/iotdb
fix(iotdb): change the `data` field to optional
2024-01-25 23:00:36 +08:00
firest c37097a150 feat(dashboard): expose the `swagger_support` option 2024-01-25 22:58:16 +08:00
Kjell Winblad 5bf37cbe0a
Merge pull request #12376 from kjellwinblad/kjell/kinesis
feat: refactor kinesis bridge to connector and action
2024-01-25 14:58:40 +01:00
Thales Macedo Garitezi b5567d9d9f
Merge pull request #12389 from thalesmg/ci-peer-cover-m-20240124
ci: start/flush cover when using peer
2024-01-25 10:00:50 -03:00
Kjell Winblad 2887a05ba3 refactor: simplify emqx_bridge_kinesis (thanks to @thalesmg's tips) 2024-01-25 12:48:31 +01:00
JimMoen 4dca1ef848
fix(prom_di): compatibility with bridge_v1 2024-01-25 19:37:15 +08:00
JimMoen 7b041683bb
fix(prom_di): action enable and status 2024-01-25 19:36:40 +08:00
JimMoen beeedd33d1
fix(prom_mria): mria_data in init_acc 2024-01-25 19:36:39 +08:00
JimMoen 58b60181d7
fix(prom_mria): move mria callback into `emqx_prometheus.erl`
- they will return in same api endpoint.
2024-01-25 19:36:38 +08:00
Kjell Winblad cfad0923cf chore: use macros for connector and action status atoms 2024-01-25 11:50:01 +01:00
firest 01f374d8e5 fix(iotdb): change the `data` field to optional 2024-01-25 18:29:41 +08:00
zhongwencool 3b0736fc67
Merge pull request #12387 from thalesmg/sync-r55-m-20240124
sync release-55 to master
2024-01-25 17:35:41 +08:00
zhongwencool 4f8accc31b
Merge branch 'master' into sync-r55-m-20240124 2024-01-25 16:09:05 +08:00
ieQu1 aeefbe9544 fix(ds): Cache database metadata in RAM 2024-01-25 16:01:44 +08:00
Thales Macedo Garitezi 846ad42a65 fix(ds): don't use env var for data dir default value
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-25 16:01:44 +08:00
Thales Macedo Garitezi 024ffe2909 test(ds): reduce flakiness 2024-01-25 16:01:44 +08:00
Thales Macedo Garitezi 9003bc5b72 perf(ds): inherit only LTS paths containing wildcards when adding a new generation
Fixes https://github.com/emqx/emqx/pull/12338#discussion_r1462139499
2024-01-25 16:01:44 +08:00
Shawn e4c683d6f8 chore: update ecql to 0.6.0 2024-01-25 16:01:44 +08:00
Shawn 5cd9f49558 ci: update generated connector name 2024-01-25 16:01:44 +08:00
Shawn 6a21766ce3 refactor: split cassandra bridges to actions and connectors 2024-01-25 16:01:44 +08:00
Shawn 497e735bf4 ci: add env vars to run cassandra tests locally 2024-01-25 16:01:43 +08:00
zhongwencool 186e1591df chore: document api-key path api only support bearerAuth 2024-01-25 16:01:43 +08:00
firest b44420c14f fix(opentsdb): Enhanced the type support for template data 2024-01-25 16:01:43 +08:00
firest 83a8822798 chore(opents): bump version && update changes 2024-01-25 16:01:43 +08:00
firest dad8a32e0b feat(opents): improve the OpentsDB bridge to v2 style 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi 878c9ee8b1 fix(ds): do not count persistent session-only routed messages as dropped
Fixes https://emqx.atlassian.net/browse/EMQX-11539
2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi d323fc7c27 feat(ps): add message gc 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi 7c0d37fdb9 feat(lts): inherit previous generation's lts when possible 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi 4a0fd756ae feat(ds): add `list_generations` and `drop_generation` APIs 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi fb39e1eacc feat(ds): allow customizing the data directory
The storage expectations for the RocksDB DB may be different from our usual data
directory.  Also, it may consume a lot more storage than other data.

This allows customizing the data directory for the builtin DS storage backend.

Note: if the cluster was already initialized using a directory path, changing that config
will have no effect.  This path is currently persisted in mnesia and used when reopening
the DB.
2024-01-25 16:01:42 +08:00
JianBo He 51d32bd620 chore: improve http connector logs format 2024-01-25 16:01:42 +08:00
Ilya Averyanov adf22f1f10 fix(mqtt_bridge): render valid messages from incomplete rule data 2024-01-25 16:01:42 +08:00
Serge Tupchii a8c6280a5e test(emqx_cm_SUITE): use one helper function: `emqx_pool:flush_async_tasks/1` 2024-01-25 16:01:42 +08:00
Serge Tupchii f52cc93d9d perf(emqx_cm): use a dedicated pool for channel cleanup
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.

Fixes: EMQX-11743
2024-01-25 16:01:42 +08:00
JianBo He 52a7b45d97 fix: upgrade ekka to 0.18.3 to ensure the leader correct 2024-01-25 14:47:59 +08:00
zhongwencool 3bc6e546e0 fix: reboot emqx_dashboard after emqx_licencse 2024-01-25 12:29:04 +08:00
Xinyu Liu 4eb0260eaf
Merge pull request #12381 from emqx/port-sql-funcs-from-emqx4
feat: port emqx/emqx-enterprise#1892, add some SQL functions
2024-01-25 11:12:58 +08:00
JianBo He ff0fd65f9d
Merge pull request #12373 from JimMoen/fix-prom-data-aggre
Follow up #12299
2024-01-25 10:37:47 +08:00
JimMoen a752119d05
fix(prom_di): use bridge_v2 metrics api 2024-01-25 09:22:22 +08:00
JimMoen 61dd362db0
fix(prom_di): api crash in `connecting`
- https://emqx.atlassian.net/browse/EMQX-11801
2024-01-25 09:22:21 +08:00
Thales Macedo Garitezi ae387d1812 ci: start/flush cover when using peer
`cover` is not automatically started by `peer`.  Without starting/flushing it, we don't
get coverage data from peer nodes.
2024-01-24 18:02:47 -03:00
William Yang 9fb6688801 fix(quic): Per listener conn count 2024-01-24 21:36:25 +01:00
Thales Macedo Garitezi a19cf253ae
Merge pull request #12359 from thalesmg/fix-connector-app-deps-r55-20240119
fix(machine_boot): inject connector/bridge apps as dependencies to `emqx_connector` app
2024-01-24 15:40:24 -03:00
ieQu1 7b5f2948fe test(ds): Fix flaky testcase 2024-01-24 19:33:30 +01:00
ieQu1 305a54f646 chore(ds): Update BPAPI version 2024-01-24 19:33:30 +01:00
ieQu1 eee221f1d0 feat(ds): Make egress batching configurable 2024-01-24 19:33:30 +01:00
ieQu1 137535a821 feat(ds): Introduce egress process for the builtin backend 2024-01-24 19:33:30 +01:00
ieQu1 ec93d8cf18
Merge pull request #12385 from ieQu1/ds-meta-ram-cache
fix(ds): Cache database metadata in RAM
2024-01-24 19:32:38 +01:00
ieQu1 9b7df302e8 fix(ds): Cache database metadata in RAM 2024-01-24 18:43:48 +01:00
Kjell Winblad fa842736d2 feat: refactor kinesis bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11461
2024-01-24 18:31:03 +01:00
Thales Macedo Garitezi 5547a40ceb fix(ds): don't use env var for data dir default value
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-24 11:39:14 -03:00
JimMoen ace1303de2
fix(prom_auth): authz metric name: `allow` and `deny`
- https://emqx.atlassian.net/browse/EMQX-11799
2024-01-24 22:38:49 +08:00
JimMoen 5e02709de0
fix(prom_auth): authz_enable key from enable status
- https://emqx.atlassian.net/browse/EMQX-11800
2024-01-24 22:38:48 +08:00
Shawn 4afba8eb94 feat: port emqx/emqx-enterprise#1892, add some SQL functions 2024-01-24 18:50:57 +08:00
JimMoen 6e3ce4d644
fix(prom_stats): emqx_stats metric type should be gauge 2024-01-24 14:11:17 +08:00
JimMoen 4a1d1b6aeb
fix: topics and retained metrics are cluster consistented 2024-01-24 14:11:16 +08:00
JimMoen 731efd8b49
fix(prom): cluster aggre/unaggre labels 2024-01-24 14:11:15 +08:00
JimMoen 497ec04359
fix(prom): missing callback attribute 2024-01-24 14:11:14 +08:00
JianBo He 947a08d1d2
Merge pull request #12375 from thalesmg/fix-actions-reset-node-metrics-r55-20240123
fix(actions_api): reset metrics on all nodes
2024-01-24 09:17:09 +08:00
Thales Macedo Garitezi 77329209a2
Merge pull request #12368 from thalesmg/ds-inherit-wildcards-only-lts-m-20240122
perf(ds): inherit only LTS paths containing wildcards when adding a new generation
2024-01-23 17:14:51 -03:00
Thales Macedo Garitezi 67c9e9c8fc fix(actions_api): reset metrics on all nodes
Fixes https://emqx.atlassian.net/browse/EMQX-11796
2024-01-23 10:40:02 -03:00
Thales Macedo Garitezi eecd7e084c test(ds): reduce flakiness 2024-01-23 09:47:03 -03:00
Thales Macedo Garitezi 1be5b6aaf3 fix(mqtt_action): use overridden `resource_opts`
Fixes https://emqx.atlassian.net/browse/EMQX-11790
2024-01-23 09:41:28 -03:00
Thales Macedo Garitezi 1eb47d0c16 perf(ds): inherit only LTS paths containing wildcards when adding a new generation
Fixes https://github.com/emqx/emqx/pull/12338#discussion_r1462139499
2024-01-23 09:20:28 -03:00
zhongwencool a8eefe808f
Merge pull request #12371 from emqx/es-server-schema
chore: es's base_url to server
2024-01-23 17:17:19 +08:00
zhongwencool 6a9cbb3d97 test: add es's update test 2024-01-23 15:51:09 +08:00
Xinyu Liu aafb683ec7
Merge pull request #12330 from emqx/bridge-v2-cassandra
refactor: split cassandra bridges to actions and connectors
2024-01-23 14:45:32 +08:00
zhongwencool 94692162b5
Merge pull request #12370 from zhongwencool/api-key-auth
chore: document api-key path api only support bearerAuth
2024-01-23 14:22:35 +08:00
Shawn 218af3fef4 chore: update ecql to 0.6.0 2024-01-23 14:14:23 +08:00
lafirest 9d2a207184
Merge pull request #12353 from lafirest/feat/optsdb
feat(opents): improve the OpentsDB bridge to v2 style
2024-01-23 13:49:33 +08:00
lafirest dc458ff0db
Merge pull request #12366 from lafirest/fix/sys
fix(sysk): fix probe testing bugs for syskeeper
2024-01-23 13:49:28 +08:00
firest 51e8c955ff fix(connector): make the connector id unique when doing probe testing 2024-01-23 13:03:12 +08:00
zhongwencool ada2785b5d chore: es's base_url to server 2024-01-23 12:20:34 +08:00
zhongwencool 0dbaaa5d94 chore: document api-key path api only support bearerAuth 2024-01-23 10:31:43 +08:00
firest 28867d07e6 fix(opentsdb): Enhanced the type support for template data 2024-01-23 10:00:56 +08:00
Thales Macedo Garitezi 609ba7e332 fix(ds): do not count persistent session-only routed messages as dropped
Fixes https://emqx.atlassian.net/browse/EMQX-11539
2024-01-22 16:58:45 -03:00
Thales Macedo Garitezi d122340c13
Merge pull request #12338 from thalesmg/ds-message-gc-20240115
feat(ds): add message GC
2024-01-22 16:57:26 -03:00
Thales Macedo Garitezi 7035b4c8b3 feat(ps): add message gc 2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi db710c4be5 feat(lts): inherit previous generation's lts when possible 2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi 75b08b525b feat(ds): add `list_generations` and `drop_generation` APIs 2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi 39090d3732
Merge pull request #12340 from thalesmg/ds-db-data-path-m-20240116
feat(ds): allow customizing the data directory
2024-01-22 13:30:59 -03:00
Thales Macedo Garitezi 3207f0ea80 fix(machine_boot): inject connector/bridge apps as dependencies to `emqx_connector` app
Fixes https://emqx.atlassian.net/browse/EMQX-11771

For the same reasons as we inject `emqx_bridge_*` applications as dependencies to
`emqx_bridge` when starting the node.  Already configured connectors are started when
`emqx_connector` application starts, and may lead to crashes and noise in the logs.  One
example is to configure a mongodb bridge and restart the node.
2024-01-22 11:13:34 -03:00
firest 3f7b913e88 chore(opents): bump version && update changes 2024-01-22 20:45:18 +08:00
firest e337e1dc40 feat(opents): improve the OpentsDB bridge to v2 style 2024-01-22 20:45:10 +08:00
firest cd90b93550 fix(sysk): fix probe testing bugs for syskeeper 2024-01-22 20:27:48 +08:00
JianBo He 2706e005ff
Merge pull request #12354 from SergeTupchiy/EMQX-11751-dashboard-import-timeout-rel55
Apply brdige post config changes in parallel
2024-01-22 17:24:55 +08:00
JimMoen 2061d75b50
docs: prometheus api `mode` field description 2024-01-22 10:39:56 +08:00
JimMoen 2263df0242
fix(prom_push_gw): use format mode `node` for prometheus push gateway 2024-01-22 10:39:55 +08:00
JimMoen b424f8ac12
feat(prom_stats): aggregated/unaggregated prometheus data 2024-01-22 10:39:54 +08:00
JimMoen 6b064dd8eb
fix(prom_stats): missing metric key
- follow https://github.com/emqx/emqx/pull/11497
2024-01-22 10:39:53 +08:00
JimMoen c6c1a7fc28
refactor(prom): prom_auth and prom_di as prom_cluster behaviour 2024-01-22 10:39:52 +08:00
JimMoen b480c5b371
fix(prom): use name `mode` and macros to put/get format mode 2024-01-22 10:39:51 +08:00
Shawn 0e1043f80c ci: update generated connector name 2024-01-22 10:34:40 +08:00
JianBo He f7ae9ab489
Merge pull request #12351 from HJianBo/improve-the-http-connector-logs
chore: improve http connector logs format
2024-01-22 10:27:35 +08:00
Shawn b32c0fb0d8 refactor: split cassandra bridges to actions and connectors 2024-01-22 10:16:46 +08:00
Shawn 29d767bd62 ci: add env vars to run cassandra tests locally 2024-01-22 10:16:46 +08:00
JimMoen 8cb12c6a74
refactor: use utils func for prom_auth and prom_di 2024-01-21 00:14:54 +08:00
JimMoen c3da792323
feat(prom_di): cluster aggregated/unaggregated metrics 2024-01-21 00:14:53 +08:00
JimMoen 9627124d67
fix(prom_di): metric `emqx_rule_enable` 2024-01-21 00:14:52 +08:00
JimMoen 5534d5e9de
fix(bpapi): make static_check happy 2024-01-21 00:14:51 +08:00
JimMoen 5914eb5ca5
fix(prom_cert): rm cacertfile expiry epoch 2024-01-21 00:14:50 +08:00
JimMoen 4fb1ff2f9d
fix(prom_api): format-mode example value 2024-01-21 00:14:49 +08:00
JimMoen 57f3efde63
feat(prom_auth): cluster metrics with different format-mode 2024-01-21 00:14:48 +08:00
JimMoen fb330f77e6
feat(prometheus): api `format_mode` parameter support
- node(default):
  The only supported format_mode for PushGateway.
  Return the current node's metrics.
  -  Without label `node_name` with content-type: `text/plain`.
  -  Without key `node_name` with content-type: `application/json`

- nodes_aggregated:
  Return all nodes metrics Arithmetic-Sum or Logical-Sum.
  See details in callback modules.
    - Logical-Sum for metrics named with `xxx_enable` or `xxx_status`.
    - Arithmetic-Sum for other metrics.
  `node_name` field:
    -  Without label `node_name` with content-type: `text/plain`.
    -  Without key `node_name` with content-type: `application/json`

- nodes_unaggregated:
  Return all nodes metrics without aggregated.
  `node_name` field:
    -  _With_ label `node_name` with content-type: `text/plain`.
    -  _With_ key `node_name` with content-type: `application/json`
2024-01-21 00:14:47 +08:00
JimMoen 94032aafb2
fix(prom_data_integration): fix metric type 2024-01-21 00:14:46 +08:00
JimMoen c3e9533260
fix: prometheus auth metrics fields and type
- rm rate fields
- fix few fields type to counter
2024-01-21 00:14:45 +08:00
JimMoen a18c4d193a
refactor: abstract function call 2024-01-21 00:14:44 +08:00
JimMoen 8f7964f435
feat(prom): data integration metrics in josn format 2024-01-21 00:14:42 +08:00
JimMoen 36f009b0c2
fix(prom): connectors specific data 2024-01-21 00:14:41 +08:00
JimMoen 76d9ace582
fix: connector_count contains bridge_v1 and bridge_v2 2024-01-21 00:14:40 +08:00
JimMoen f457def201
fix: use `id` uniformly as the label key for rules and actions 2024-01-21 00:14:39 +08:00
JimMoen 4b23930fce
fix(prometheus): license expiry and schema_registry only for ee 2024-01-21 00:14:38 +08:00
JimMoen 38a90bd273
fix(prom_auth): authn && authz data response by josn 2024-01-21 00:14:37 +08:00
JimMoen 0dca9905dd
test(prometheus): `{Registry, Collector}` tuple 2024-01-21 00:14:36 +08:00
JimMoen 7832bbc0a4
fix(prom): schema registry not in ce edition 2024-01-21 00:14:35 +08:00
JimMoen 5158395bcf
feat(prometheus): data integration prom data 2024-01-21 00:14:34 +08:00
JimMoen bf2e4d134a
refactor(prometheus): generic api response funcs 2024-01-21 00:14:33 +08:00
JimMoen e0feb580b6
feat(prometheus): auth metrics with text/plain 2024-01-21 00:14:32 +08:00
JimMoen 092159b071
feat(prometheus): cert expiry epoch in endpoint `/prometheus/stats` 2024-01-21 00:14:31 +08:00
JimMoen 558c471392
feat(prometheus): license expiry at epoch as gauge 2024-01-21 00:14:30 +08:00
JimMoen fae0cea17f
fix(license): make dialyzer happy 2024-01-21 00:14:29 +08:00
JimMoen 81ba166b49
feat(license): expiry epoch api 2024-01-21 00:14:28 +08:00
JimMoen 89128958ed
fix(dashboard): licence quota not provided to ce edition 2024-01-21 00:14:27 +08:00
JimMoen ecd0da9fde
feat(dashboard): `/monitor_current/nodes/{node}` provide `node_uptime` 2024-01-21 00:14:26 +08:00
JimMoen 7bc3a5090d
fix(dashboard): meck `emqx_retainer` in SUITE 2024-01-21 00:14:25 +08:00
JimMoen 3d6b65aced
chore: emqx_dashboard_monitor:current_rate/0 not exported anymore 2024-01-21 00:14:24 +08:00
JimMoen a35698009c
feat(dashboard): endpoint `/monitor_current` provides mor fields
- `retained_msg_count`
  Current retained_msg_count on each node and should be same on all nodes.

- `license_quota`
  Only for enterprise edition, provides the max limited connections num.
2024-01-21 00:14:23 +08:00
Serge Tupchii dc15d37dcc perf(emqx_bridge/connector): load and unload bridges/connectors in parallel
This should reduce app start/stop time, when a large number of bridges/connectors are not healthy.
2024-01-19 21:44:31 +02:00
Serge Tupchii d3a6870097 feat(emqx_utils): add pforeach/2,3 2024-01-19 21:44:31 +02:00
Serge Tupchii a6568dec75 perf(emqx_bridge/connector): apply post config bridge/connector changes in parallel
This can greatly improve the performance when many bridges/connectors are being changed,
e.g. when a backup file is being imported.

Fixes: EMQX-11751
2024-01-19 21:44:31 +02:00
Serge Tupchii 54457b7093 feat(emqx_utils): allow `infinity` timeout in `pmap/3` 2024-01-19 19:34:04 +02:00