Commit Graph

10733 Commits

Author SHA1 Message Date
ieQu1
ebe2339810 fix(sessds): Use mria:async_dirty instead of transaction 2024-01-26 17:49:33 +01:00
ieQu1
f5b9bd30aa fix(sessds): Apply review remarks 2024-01-26 17:49:33 +01:00
ieQu1
39857626ce test(sessds): Fix failing tests 2024-01-26 17:49:33 +01:00
ieQu1
e7b03cdc59 test(sessds): Create a property-based test for the session state 2024-01-26 17:49:33 +01:00
ieQu1
893656f092 fix(sessds): Fix session garbage collection after the refactoring 2024-01-26 17:49:33 +01:00
ieQu1
963df8f941 fix(sessds): Delete the routes when the session expires 2024-01-26 17:49:33 +01:00
ieQu1
cff6c15e13 fix(sessds): Store the QoS as the MSB of the packet ID 2024-01-26 17:49:33 +01:00
ieQu1
978a3bfef3 refactor(sessds): Simplify representation of QoS tracks 2024-01-26 17:49:33 +01:00
ieQu1
1b4f69b44d refactor(sessds): Simplify data structure of ds_state pmap datatype 2024-01-26 17:49:33 +01:00
ieQu1
4f4831fe7f refactor(sessds): Factor out stream scheduler into its own module 2024-01-26 17:49:33 +01:00
ieQu1
82ef34998a feat(sessds): Index streams by a unique subid 2024-01-26 17:49:33 +01:00
ieQu1
3fb2064ea4 test(sessds): Add property-based tests for seqno generator 2024-01-26 17:49:33 +01:00
ieQu1
8e8d3af096 fix(sessds): Refactor emqx_persistent_session_ds to use CRUD module 2024-01-26 17:49:33 +01:00
ieQu1
2d08aa88d8 refactor(ds): Create a CRUD module for the persistent session 2024-01-26 17:49:33 +01:00
Thales Macedo Garitezi
6410f5a717 fix(resource_metrics): avoid detaching handler on crashes
Fixes https://emqx.atlassian.net/browse/EMQX-11821
2024-01-26 13:05:41 -03:00
Zaiming (Stone) Shi
2a40152721 Merge remote-tracking branch 'origin/release-55' into sync-release-55 2024-01-26 16:51:13 +01:00
William Yang
933c00c7ad
Merge pull request #12392 from qzhuyan/perf/william/ws-no-utf8-validate
perf: new ws listener option to disable UTF-8 validation
2024-01-26 16:01:04 +01:00
ieQu1
0933dc818e chore(ekka): Bump version to 0.18.4
Don't escalate the errors in etcd node discovery to the node level.

Fixes: #12255
2024-01-26 14:25:16 +01:00
Serge Tupchii
a56a5e9c3c chore: add copyright 2024-01-26 11:01:53 +02:00
Serge Tupchii
2d693402c5 refactor: split greptimedb bridge to actions and connectors 2024-01-26 11:01:53 +02:00
William Yang
726302ef6a perf: new ws listener option to disable UTF-8 validation 2024-01-26 09:55:17 +01:00
JimMoen
2e37f7514a
fix(prom_stats): compatibility with previous api format in json mode 2024-01-26 16:01:32 +08:00
JimMoen
308cde53b9
fix(prom_stats): merge cl_consistented data into per nodes data 2024-01-26 11:39:20 +08:00
JianBo He
e29a80d568
Merge pull request #12388 from qzhuyan/dev/william/per-listn-conn-cnt
feat: quic per listener conn count
2024-01-26 09:01:39 +08:00
Thales Macedo Garitezi
590746ddfa
Merge pull request #12400 from thalesmg/ds-fix-data-dir-schema-m-20240125
fix(ds): don't make data dir part of the schema
2024-01-25 15:52:11 -03:00
Thales Macedo Garitezi
8e31afe6c2 fix(ds): don't make data dir part of the schema
The data directory was ending up being persisted in the database schema.  This led to
issues when opening the DB on different nodes.
2024-01-25 14:44:06 -03:00
lafirest
0b6aee0ae9
Merge pull request #12393 from lafirest/fix/iotdb
fix(iotdb): change the `data` field to optional
2024-01-25 23:00:36 +08:00
firest
c37097a150 feat(dashboard): expose the swagger_support option 2024-01-25 22:58:16 +08:00
Kjell Winblad
5bf37cbe0a
Merge pull request #12376 from kjellwinblad/kjell/kinesis
feat: refactor kinesis bridge to connector and action
2024-01-25 14:58:40 +01:00
Thales Macedo Garitezi
b5567d9d9f
Merge pull request #12389 from thalesmg/ci-peer-cover-m-20240124
ci: start/flush cover when using peer
2024-01-25 10:00:50 -03:00
Kjell Winblad
2887a05ba3 refactor: simplify emqx_bridge_kinesis (thanks to @thalesmg's tips) 2024-01-25 12:48:31 +01:00
JimMoen
4dca1ef848
fix(prom_di): compatibility with bridge_v1 2024-01-25 19:37:15 +08:00
JimMoen
7b041683bb
fix(prom_di): action enable and status 2024-01-25 19:36:40 +08:00
JimMoen
beeedd33d1
fix(prom_mria): mria_data in init_acc 2024-01-25 19:36:39 +08:00
JimMoen
58b60181d7
fix(prom_mria): move mria callback into emqx_prometheus.erl
- they will return in same api endpoint.
2024-01-25 19:36:38 +08:00
Kjell Winblad
cfad0923cf chore: use macros for connector and action status atoms 2024-01-25 11:50:01 +01:00
firest
01f374d8e5 fix(iotdb): change the data field to optional 2024-01-25 18:29:41 +08:00
zhongwencool
3b0736fc67
Merge pull request #12387 from thalesmg/sync-r55-m-20240124
sync release-55 to master
2024-01-25 17:35:41 +08:00
zhongwencool
4f8accc31b
Merge branch 'master' into sync-r55-m-20240124 2024-01-25 16:09:05 +08:00
ieQu1
aeefbe9544 fix(ds): Cache database metadata in RAM 2024-01-25 16:01:44 +08:00
Thales Macedo Garitezi
846ad42a65 fix(ds): don't use env var for data dir default value
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-25 16:01:44 +08:00
Thales Macedo Garitezi
024ffe2909 test(ds): reduce flakiness 2024-01-25 16:01:44 +08:00
Thales Macedo Garitezi
9003bc5b72 perf(ds): inherit only LTS paths containing wildcards when adding a new generation
Fixes https://github.com/emqx/emqx/pull/12338#discussion_r1462139499
2024-01-25 16:01:44 +08:00
Shawn
e4c683d6f8 chore: update ecql to 0.6.0 2024-01-25 16:01:44 +08:00
Shawn
5cd9f49558 ci: update generated connector name 2024-01-25 16:01:44 +08:00
Shawn
6a21766ce3 refactor: split cassandra bridges to actions and connectors 2024-01-25 16:01:44 +08:00
Shawn
497e735bf4 ci: add env vars to run cassandra tests locally 2024-01-25 16:01:43 +08:00
zhongwencool
186e1591df chore: document api-key path api only support bearerAuth 2024-01-25 16:01:43 +08:00
firest
b44420c14f fix(opentsdb): Enhanced the type support for template data 2024-01-25 16:01:43 +08:00
firest
83a8822798 chore(opents): bump version && update changes 2024-01-25 16:01:43 +08:00
firest
dad8a32e0b feat(opents): improve the OpentsDB bridge to v2 style 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi
878c9ee8b1 fix(ds): do not count persistent session-only routed messages as dropped
Fixes https://emqx.atlassian.net/browse/EMQX-11539
2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi
d323fc7c27 feat(ps): add message gc 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi
7c0d37fdb9 feat(lts): inherit previous generation's lts when possible 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi
4a0fd756ae feat(ds): add list_generations and drop_generation APIs 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi
fb39e1eacc feat(ds): allow customizing the data directory
The storage expectations for the RocksDB DB may be different from our usual data
directory.  Also, it may consume a lot more storage than other data.

This allows customizing the data directory for the builtin DS storage backend.

Note: if the cluster was already initialized using a directory path, changing that config
will have no effect.  This path is currently persisted in mnesia and used when reopening
the DB.
2024-01-25 16:01:42 +08:00
JianBo He
51d32bd620 chore: improve http connector logs format 2024-01-25 16:01:42 +08:00
Ilya Averyanov
adf22f1f10 fix(mqtt_bridge): render valid messages from incomplete rule data 2024-01-25 16:01:42 +08:00
Serge Tupchii
a8c6280a5e test(emqx_cm_SUITE): use one helper function: emqx_pool:flush_async_tasks/1 2024-01-25 16:01:42 +08:00
Serge Tupchii
f52cc93d9d perf(emqx_cm): use a dedicated pool for channel cleanup
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.

Fixes: EMQX-11743
2024-01-25 16:01:42 +08:00
JianBo He
52a7b45d97 fix: upgrade ekka to 0.18.3 to ensure the leader correct 2024-01-25 14:47:59 +08:00
zhongwencool
3bc6e546e0 fix: reboot emqx_dashboard after emqx_licencse 2024-01-25 12:29:04 +08:00
Xinyu Liu
4eb0260eaf
Merge pull request #12381 from emqx/port-sql-funcs-from-emqx4
feat: port emqx/emqx-enterprise#1892, add some SQL functions
2024-01-25 11:12:58 +08:00
JianBo He
ff0fd65f9d
Merge pull request #12373 from JimMoen/fix-prom-data-aggre
Follow up #12299
2024-01-25 10:37:47 +08:00
JimMoen
a752119d05
fix(prom_di): use bridge_v2 metrics api 2024-01-25 09:22:22 +08:00
JimMoen
61dd362db0
fix(prom_di): api crash in connecting
- https://emqx.atlassian.net/browse/EMQX-11801
2024-01-25 09:22:21 +08:00
Thales Macedo Garitezi
ae387d1812 ci: start/flush cover when using peer
`cover` is not automatically started by `peer`.  Without starting/flushing it, we don't
get coverage data from peer nodes.
2024-01-24 18:02:47 -03:00
William Yang
9fb6688801 fix(quic): Per listener conn count 2024-01-24 21:36:25 +01:00
Thales Macedo Garitezi
a19cf253ae
Merge pull request #12359 from thalesmg/fix-connector-app-deps-r55-20240119
fix(machine_boot): inject connector/bridge apps as dependencies to `emqx_connector` app
2024-01-24 15:40:24 -03:00
ieQu1
7b5f2948fe test(ds): Fix flaky testcase 2024-01-24 19:33:30 +01:00
ieQu1
305a54f646 chore(ds): Update BPAPI version 2024-01-24 19:33:30 +01:00
ieQu1
eee221f1d0 feat(ds): Make egress batching configurable 2024-01-24 19:33:30 +01:00
ieQu1
137535a821 feat(ds): Introduce egress process for the builtin backend 2024-01-24 19:33:30 +01:00
ieQu1
ec93d8cf18
Merge pull request #12385 from ieQu1/ds-meta-ram-cache
fix(ds): Cache database metadata in RAM
2024-01-24 19:32:38 +01:00
ieQu1
9b7df302e8 fix(ds): Cache database metadata in RAM 2024-01-24 18:43:48 +01:00
Kjell Winblad
fa842736d2 feat: refactor kinesis bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11461
2024-01-24 18:31:03 +01:00
Thales Macedo Garitezi
5547a40ceb fix(ds): don't use env var for data dir default value
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-24 11:39:14 -03:00
JimMoen
ace1303de2
fix(prom_auth): authz metric name: allow and deny
- https://emqx.atlassian.net/browse/EMQX-11799
2024-01-24 22:38:49 +08:00
JimMoen
5e02709de0
fix(prom_auth): authz_enable key from enable status
- https://emqx.atlassian.net/browse/EMQX-11800
2024-01-24 22:38:48 +08:00
Shawn
4afba8eb94 feat: port emqx/emqx-enterprise#1892, add some SQL functions 2024-01-24 18:50:57 +08:00
JimMoen
6e3ce4d644
fix(prom_stats): emqx_stats metric type should be gauge 2024-01-24 14:11:17 +08:00
JimMoen
4a1d1b6aeb
fix: topics and retained metrics are cluster consistented 2024-01-24 14:11:16 +08:00
JimMoen
731efd8b49
fix(prom): cluster aggre/unaggre labels 2024-01-24 14:11:15 +08:00
JimMoen
497ec04359
fix(prom): missing callback attribute 2024-01-24 14:11:14 +08:00
JianBo He
947a08d1d2
Merge pull request #12375 from thalesmg/fix-actions-reset-node-metrics-r55-20240123
fix(actions_api): reset metrics on all nodes
2024-01-24 09:17:09 +08:00
Thales Macedo Garitezi
77329209a2
Merge pull request #12368 from thalesmg/ds-inherit-wildcards-only-lts-m-20240122
perf(ds): inherit only LTS paths containing wildcards when adding a new generation
2024-01-23 17:14:51 -03:00
Thales Macedo Garitezi
67c9e9c8fc fix(actions_api): reset metrics on all nodes
Fixes https://emqx.atlassian.net/browse/EMQX-11796
2024-01-23 10:40:02 -03:00
Thales Macedo Garitezi
eecd7e084c test(ds): reduce flakiness 2024-01-23 09:47:03 -03:00
Thales Macedo Garitezi
1be5b6aaf3 fix(mqtt_action): use overridden resource_opts
Fixes https://emqx.atlassian.net/browse/EMQX-11790
2024-01-23 09:41:28 -03:00
Thales Macedo Garitezi
1eb47d0c16 perf(ds): inherit only LTS paths containing wildcards when adding a new generation
Fixes https://github.com/emqx/emqx/pull/12338#discussion_r1462139499
2024-01-23 09:20:28 -03:00
zhongwencool
a8eefe808f
Merge pull request #12371 from emqx/es-server-schema
chore: es's base_url to server
2024-01-23 17:17:19 +08:00
zhongwencool
6a9cbb3d97 test: add es's update test 2024-01-23 15:51:09 +08:00
Xinyu Liu
aafb683ec7
Merge pull request #12330 from emqx/bridge-v2-cassandra
refactor: split cassandra bridges to actions and connectors
2024-01-23 14:45:32 +08:00
zhongwencool
94692162b5
Merge pull request #12370 from zhongwencool/api-key-auth
chore: document api-key path api only support bearerAuth
2024-01-23 14:22:35 +08:00
Shawn
218af3fef4 chore: update ecql to 0.6.0 2024-01-23 14:14:23 +08:00
lafirest
9d2a207184
Merge pull request #12353 from lafirest/feat/optsdb
feat(opents): improve the OpentsDB bridge to v2 style
2024-01-23 13:49:33 +08:00
lafirest
dc458ff0db
Merge pull request #12366 from lafirest/fix/sys
fix(sysk): fix probe testing bugs for syskeeper
2024-01-23 13:49:28 +08:00
firest
51e8c955ff fix(connector): make the connector id unique when doing probe testing 2024-01-23 13:03:12 +08:00
zhongwencool
ada2785b5d chore: es's base_url to server 2024-01-23 12:20:34 +08:00
zhongwencool
0dbaaa5d94 chore: document api-key path api only support bearerAuth 2024-01-23 10:31:43 +08:00