5397402396
test(willmsg): test will delay and session expires
...
will delay > session expire
will delay < session expire
timer triggered events are handled in seq, exclude the case of
(will delay == session expire)
2024-02-22 16:39:27 +01:00
dd62280e04
fix: handle delayed willmsg, part 1
2024-02-22 16:39:27 +01:00
b76c701b1c
test(takeover): add back the delay when takeover
2024-02-22 16:39:27 +01:00
6243cf0b0c
fix(kick): defer willmsg publish when conn terminates
...
because kick means shutdown connection AND delete session
2024-02-22 16:39:27 +01:00
9da4896f57
fix(mqtt): ensure publish willmsg when session expires
2024-02-22 16:39:24 +01:00
07eec31a8a
fix: willmsg not published in takeover
2024-02-22 16:35:59 +01:00
ccf607ee2c
fix: bpapi announce's bad node name
2024-02-22 14:55:11 +08:00
985a3e3062
test: remove unused var warning
2024-02-21 18:14:46 -03:00
8e47503f7d
Merge pull request #12560 from SergeTupchiy/EMQX-11530-log-throttling-followup-fixes
...
fix: disable log throttling if primary log level is debug
2024-02-21 22:02:32 +02:00
f31e9e6a15
Merge pull request #12555 from thalesmg/ds-move-count-to-mgmt-m-20240221
...
refactor(mgmt): move persistent session counting function to mgmt module
2024-02-21 16:36:29 -03:00
ceb208f1eb
fix: disable log throttling if primary log level is debug
2024-02-21 21:19:15 +02:00
415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
...
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
25cdc2a04b
chore: bump version to 5.5.1-rc.1
2024-02-21 09:57:05 -03:00
03138f8345
refactor(mgmt): move persistent session counting function to mgmt module
...
Follow up to https://github.com/emqx/emqx/pull/12500#discussion_r1496634087
2024-02-21 09:12:56 -03:00
529211b9ac
Merge pull request #12500 from thalesmg/ds-list-client-api-m-20240209
...
feat(ds): list disconnected persistent sessions in clients API
2024-02-21 09:02:27 -03:00
de93d1c0a3
chore: add bpapi.versions
2024-02-21 10:53:22 +01:00
f7a7b7c8e0
Merge pull request #12545 from SergeTupchiy/fix-typo-tls_gc
...
chore(emqx_tls_certfile_gc): fix typo
2024-02-21 10:11:13 +02:00
65cfece0a3
chore: delete 5.5.bpapi2
...
> This particular warning won't be seen in the real life. Also, we need to start storing
bpapi files for each minor release, since the plan to use hot upgrade has been abandoned.
> ...The original procedure for BPAPI life cycle was designed under the assumption that
we'll support hot upgrade between minor releases, so we'll only ever need to keep one
BPAPI file per major release, and rewrite it whenever a new minor version is
released. We'll need to revisit it.
https://emqxio.slack.com/archives/C04H4RHKKEY/p1706862980546309
2024-02-20 17:35:28 -03:00
74dbbf6d9d
Merge branch 'release-55' into sync-r55-m-20240220
2024-02-20 16:33:34 -03:00
dd9f96211b
chore(emqx_tls_certfile_gc): fix typo
2024-02-20 21:19:36 +02:00
3a4c7f60e2
feat(ds): list disconnected persistent sessions in clients API
...
Fixes https://emqx.atlassian.net/browse/EMQX-11540
Note that not all information provided by disconnected in-memory sessions is available to
disconnected persistent sessions, nor does all of them make sense.
2024-02-20 14:52:55 -03:00
9bd0d1ba1b
feat: enable log throttling for potentially flooding log events
2024-02-20 14:20:21 +02:00
98ba300f7c
feat: implement log throttling
2024-02-20 13:23:17 +02:00
bee7fa630b
chore: bump app vsns
2024-02-19 09:05:13 -03:00
dbb6725a47
fix(config): apply config upgrade to deprecated configs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11845
Fixes https://github.com/emqx/emqx/issues/12452
2024-02-19 09:02:43 -03:00
8b0e15e402
refactor: rename 'hint' to 'cause' for MQTT fram parse failure reason
...
'reason' is maybe the wrapping field's name, so it was not used.
'hint' however, per our logging convention, is usually a free text
description for human to read.
change to 'cause' here because the field is always an atom and
it's use as shutdown counter in esockd_connection_sup
2024-02-19 09:38:35 +01:00
668d3be390
test: for better test coverage
2024-02-18 19:23:35 +01:00
22dabcb3ea
feat(mqtt): add more logging context for frame_too_large error
2024-02-18 18:33:12 +01:00
52cfeee2b1
Merge pull request #12526 from zmstone/0216-fix-flaky-tests
...
test(listeners): try to stop all listeners after each test
2024-02-16 15:33:04 +01:00
9ffda0187f
Merge pull request #12525 from zmstone/0216-docs-add-ref-to-bridge_mode
...
docs: add a reference to the origin of is_bridge and bridge_mode
2024-02-16 15:32:52 +01:00
2b46cbab7a
Merge pull request #12517 from zmstone/0213-hocon-multiline-string-with-indentation
...
0213 hocon multiline string with indentation
2024-02-16 15:12:46 +01:00
f78c30c9ff
test: reduce false warning messages in test logs
2024-02-16 14:38:51 +01:00
3fa262f9ca
test(listeners): try to stop all listeners after each test
...
- Reduce the `eaddrinuse` flakiness
- Use ?FUNCTION_NAME as listener name to avoid name clashing between
tests
- Call emqx:remove_config for adhoc listeners created for testing
tombestone is designed for default listeners
2024-02-16 14:37:06 +01:00
17a0513962
docs: add a reference to the origin of is_bridge and bridge_mode
2024-02-16 12:24:57 +01:00
a945892c52
Merge pull request #12508 from keynslug/fix/explicit-tab-mgmt
...
feat: manage mria tables explicitly during apps startup
2024-02-16 11:11:59 +01:00
b24321a8fc
feat: upgrade to hocon-0.41.0
...
hocon 0.41.0 added support for multiline string indentation.
now there is no need to escape (unless there is backslash)
quotes and line feeds etc.
2024-02-16 10:32:13 +01:00
ab2a469aff
Merge pull request #12489 from ieQu1/dev/ds-retainer
...
Durable sessions support retained messages
2024-02-15 21:59:32 +01:00
5f85105801
feat(sessds): Specialize the interval queue for positive numbers
2024-02-15 16:55:05 +01:00
3adbe65a58
refactor(sessds): Unify logic for QoS 1 and 2
2024-02-15 11:34:13 +01:00
c781240459
feat(sessds): Add support for the retainer
...
Note: this is currently not ideal. Retained messages won't be
redelivered.
2024-02-15 11:34:13 +01:00
94254ec05b
feat(sessds): Correct handling of gaps in the seqno series
2024-02-15 10:49:02 +01:00
0d0e26d6af
fix: set `info` level for potentially flooding log events
2024-02-15 11:02:03 +02:00
d0e507eba4
test(trie): fix testsuite setup
2024-02-13 20:36:22 +01:00
841fdea124
test(conf): update and simplify some testcases
2024-02-13 20:36:22 +01:00
24dfa41722
feat(cth-suite): use cheaper heuristic for schema modules
2024-02-13 20:36:22 +01:00
8f2a4f7b19
fix(cth-suite): use cheaper check for loaded applications
2024-02-13 20:36:22 +01:00
b6d77c164e
feat(emqx-mgmt): manage mria tables explicitly during startup
2024-02-13 20:36:22 +01:00
21a5751575
feat(emqx): manage mria tables explicitly during startup
2024-02-12 19:17:09 +01:00
90fd2b26d3
feat(banned): allow ban by clientid/username regexps, peerhost cidrs
2024-02-10 17:59:22 +03:00
a74c828e19
test: add test case to cover Receive-Maximum in CONNACK
2024-02-09 10:42:48 +01:00
cf22692c74
fix(emqx_channel): return Receive-Maximum in CONNACK when no error
2024-02-08 17:05:16 +01:00
811edb32a2
Merge pull request #12485 from ieQu1/dev/ds-fix-unsubscribe
...
fix(ds): Fix unsubscribe logic related to the replay
2024-02-07 21:38:25 +01:00
19c6d1127f
refactor(sessds): Extract subscription mgmt logic to separate module
2024-02-07 15:27:11 +01:00
3000a8f286
fix(sessds): Postpone deletion of the subscription until fully acked
2024-02-07 14:51:02 +01:00
30eb54e86b
fix(sessds): Delay unsubscribe until full ack of in-flight messages
2024-02-06 02:49:07 +01:00
a9c55f7568
feat(sessds): Consider #srs with only QoS0 messages fully acked
2024-02-06 02:49:07 +01:00
b444c82a42
feat: rabbitmq bridge v2 integration
2024-02-06 07:57:12 +08:00
280fcd8c52
Merge pull request #12437 from ieQu1/dev/optimize_make_filter
...
Optimize emqx_ds_bitmask_keymapper:make_filter function.
2024-02-05 17:32:28 +01:00
c7888ad1f1
Merge pull request #12475 from ieQu1/dev/lean-stream
...
Use a more compact data structure to represent streams
2024-02-05 13:55:24 +01:00
e1ff276d76
Merge pull request #12471 from thalesmg/fix-config-upgrade-m-20240202
...
fix(config): apply config upgrade to deprecated configs
2024-02-05 09:05:12 -03:00
698ba3f271
fix(ds): Optimize emqx_ds_bitmask_keymapper:make_filter
...
This optimization makes idle polling faster
2024-02-05 10:54:19 +01:00
2e2f3cb2aa
fix(sessds): Avoid stream hash collisions
2024-02-03 21:15:54 +01:00
2e56810ea2
refactor(ds): Use a simple improper list to represent the streams
2024-02-03 21:15:54 +01:00
76401a302a
fix(config): apply config upgrade to deprecated configs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11845
Fixes https://github.com/emqx/emqx/issues/12452
2024-02-02 15:12:08 -03:00
ed8660cb68
Merge pull request #12458 from id/0202-sync-release-55
...
sync release-55
2024-02-02 17:20:38 +01:00
b1a05c7b59
Merge pull request #12326 from zmstone/0111-unregister-session-with-timestamp
...
0111 unregister session with timestamp
2024-02-02 17:00:23 +01:00
84b6d7d720
fix: remove 5.5.bpapi2 file
2024-02-02 10:38:50 +01:00
f0569d8ae8
refactor: use mria:async_dirty to group dirty ops
2024-02-02 09:26:43 +01:00
dd490de2e1
chore: emqx 5.5.0
2024-02-02 08:48:56 +01:00
f1c7e716ce
chore: 5.5.0-rc.2
2024-02-02 08:48:56 +01:00
dae3a94670
chore: 5.5.0-rc.1
2024-02-02 08:48:56 +01:00
98d1094d73
feat(sessds): Expose subscriptions in the REST API
2024-02-01 16:47:57 +01:00
3e518c1876
Merge pull request #12427 from zmstone/0130-limit-kafka-partitions
...
0130 limit kafka partitions
2024-02-01 14:39:52 +01:00
3ad71f75e6
chore: emqx 5.5.0
2024-02-01 11:07:02 +01:00
0d5911d942
Merge pull request #12422 from zhongwencool/upgrade-erlfmt-to-v1.3.0
...
feat: upgrade erlfmt to support maybe syntax
2024-02-01 09:34:33 +08:00
3b6fbff9a5
test: fix compile warnings
2024-01-31 19:32:57 +01:00
53c217c383
refactor: micro optimization
2024-01-31 19:25:57 +01:00
3f71a98e5e
chore: 5.5.0-rc.2
2024-01-31 17:06:54 +01:00
fe46434687
fix(sessds): fix renew stream logic
2024-01-31 18:02:29 +03:00
e9c8446d57
feat: upgrade erlfmt to support maybe syntax
2024-01-31 20:06:49 +08:00
38047108a4
test: add test coverage for emqx_cm_registry_keeper module
2024-01-31 09:50:12 +01:00
209331ad33
test: fix config test
2024-01-31 09:50:12 +01:00
87a2368e37
feat(metrics): add cluster_session guage
2024-01-31 09:50:12 +01:00
509ab6f35a
feat(api): add /sessions_count api to count sessions
2024-01-31 09:50:12 +01:00
e9318752e6
feat: store session unregistration timestamp in emqx_cm_registry table
2024-01-31 09:50:12 +01:00
5bc67cb288
Merge pull request #12432 from ieQu1/ds_ignore_fully_replayed_streams
...
fix(sessds): Stream scheduler must ignore fully replayed streams
2024-01-31 09:51:49 +02:00
e843d9fd91
fix(sessds): Stream scheduler must ignore fully replayed streams
2024-01-31 00:20:54 +01:00
a2ba8fc645
chore: 5.5.0-rc.1
2024-01-30 18:58:00 +01:00
55fdc83aa5
Merge remote-tracking branch 'origin/release-55' into 0130-syn-release-55-to-master
2024-01-30 12:06:43 +01:00
206af96a33
ci: update testcases for message-expiry-interval
2024-01-30 15:46:11 +08:00
5007650bd2
perf(emqx_broker): pick broker pool worker by topic/shard pair to distribute the load more evenly.
...
Fixes: EMQX-11812
2024-01-29 20:57:10 +02:00
9f22c2c455
ci: add some sleep and retry to emqx_persistent_session_ds_SUITE
2024-01-29 18:18:18 +08:00
12da3c0986
feat: configurable server side message_expiry_interval
2024-01-29 17:29:52 +08:00
dbe2baa405
Merge pull request #12408 from zmstone/sync-release-55
...
Sync release 55
2024-01-27 19:20:58 +00:00
2c6a776318
fix(sessds): Stricter checks for PacketIds
2024-01-26 17:49:33 +01:00
eec56b0d6b
fix(sessds): Improve comments
2024-01-26 17:49:33 +01:00
3c451c6ae6
test(sessds): Fix unstable tests
2024-01-26 17:49:33 +01:00
2d23212792
refactor(sessds): #ifs -> #srs
2024-01-26 17:49:33 +01:00
974760d331
test(sessds): Don't trap exits in the test
2024-01-26 17:49:33 +01:00
ebe2339810
fix(sessds): Use mria:async_dirty instead of transaction
2024-01-26 17:49:33 +01:00
f5b9bd30aa
fix(sessds): Apply review remarks
2024-01-26 17:49:33 +01:00
39857626ce
test(sessds): Fix failing tests
2024-01-26 17:49:33 +01:00
e7b03cdc59
test(sessds): Create a property-based test for the session state
2024-01-26 17:49:33 +01:00
893656f092
fix(sessds): Fix session garbage collection after the refactoring
2024-01-26 17:49:33 +01:00
963df8f941
fix(sessds): Delete the routes when the session expires
2024-01-26 17:49:33 +01:00
cff6c15e13
fix(sessds): Store the QoS as the MSB of the packet ID
2024-01-26 17:49:33 +01:00
978a3bfef3
refactor(sessds): Simplify representation of QoS tracks
2024-01-26 17:49:33 +01:00
1b4f69b44d
refactor(sessds): Simplify data structure of ds_state pmap datatype
2024-01-26 17:49:33 +01:00
4f4831fe7f
refactor(sessds): Factor out stream scheduler into its own module
2024-01-26 17:49:33 +01:00
82ef34998a
feat(sessds): Index streams by a unique subid
2024-01-26 17:49:33 +01:00
3fb2064ea4
test(sessds): Add property-based tests for seqno generator
2024-01-26 17:49:33 +01:00
8e8d3af096
fix(sessds): Refactor emqx_persistent_session_ds to use CRUD module
2024-01-26 17:49:33 +01:00
2d08aa88d8
refactor(ds): Create a CRUD module for the persistent session
2024-01-26 17:49:33 +01:00
2a40152721
Merge remote-tracking branch 'origin/release-55' into sync-release-55
2024-01-26 16:51:13 +01:00
933c00c7ad
Merge pull request #12392 from qzhuyan/perf/william/ws-no-utf8-validate
...
perf: new ws listener option to disable UTF-8 validation
2024-01-26 16:01:04 +01:00
0933dc818e
chore(ekka): Bump version to 0.18.4
...
Don't escalate the errors in etcd node discovery to the node level.
Fixes : #12255
2024-01-26 14:25:16 +01:00
726302ef6a
perf: new ws listener option to disable UTF-8 validation
2024-01-26 09:55:17 +01:00
e29a80d568
Merge pull request #12388 from qzhuyan/dev/william/per-listn-conn-cnt
...
feat: quic per listener conn count
2024-01-26 09:01:39 +08:00
8e31afe6c2
fix(ds): don't make data dir part of the schema
...
The data directory was ending up being persisted in the database schema. This led to
issues when opening the DB on different nodes.
2024-01-25 14:44:06 -03:00
b5567d9d9f
Merge pull request #12389 from thalesmg/ci-peer-cover-m-20240124
...
ci: start/flush cover when using peer
2024-01-25 10:00:50 -03:00
4f8accc31b
Merge branch 'master' into sync-r55-m-20240124
2024-01-25 16:09:05 +08:00
846ad42a65
fix(ds): don't use env var for data dir default value
...
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-25 16:01:44 +08:00
024ffe2909
test(ds): reduce flakiness
2024-01-25 16:01:44 +08:00
186e1591df
chore: document api-key path api only support bearerAuth
2024-01-25 16:01:43 +08:00
878c9ee8b1
fix(ds): do not count persistent session-only routed messages as dropped
...
Fixes https://emqx.atlassian.net/browse/EMQX-11539
2024-01-25 16:01:43 +08:00
d323fc7c27
feat(ps): add message gc
2024-01-25 16:01:43 +08:00
4a0fd756ae
feat(ds): add `list_generations` and `drop_generation` APIs
2024-01-25 16:01:43 +08:00
fb39e1eacc
feat(ds): allow customizing the data directory
...
The storage expectations for the RocksDB DB may be different from our usual data
directory. Also, it may consume a lot more storage than other data.
This allows customizing the data directory for the builtin DS storage backend.
Note: if the cluster was already initialized using a directory path, changing that config
will have no effect. This path is currently persisted in mnesia and used when reopening
the DB.
2024-01-25 16:01:42 +08:00
a8c6280a5e
test(emqx_cm_SUITE): use one helper function: `emqx_pool:flush_async_tasks/1`
2024-01-25 16:01:42 +08:00
f52cc93d9d
perf(emqx_cm): use a dedicated pool for channel cleanup
...
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.
Fixes: EMQX-11743
2024-01-25 16:01:42 +08:00
52a7b45d97
fix: upgrade ekka to 0.18.3 to ensure the leader correct
2024-01-25 14:47:59 +08:00
ae387d1812
ci: start/flush cover when using peer
...
`cover` is not automatically started by `peer`. Without starting/flushing it, we don't
get coverage data from peer nodes.
2024-01-24 18:02:47 -03:00
9fb6688801
fix(quic): Per listener conn count
2024-01-24 21:36:25 +01:00
7b5f2948fe
test(ds): Fix flaky testcase
2024-01-24 19:33:30 +01:00
eee221f1d0
feat(ds): Make egress batching configurable
2024-01-24 19:33:30 +01:00
137535a821
feat(ds): Introduce egress process for the builtin backend
2024-01-24 19:33:30 +01:00
5547a40ceb
fix(ds): don't use env var for data dir default value
...
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-24 11:39:14 -03:00
77329209a2
Merge pull request #12368 from thalesmg/ds-inherit-wildcards-only-lts-m-20240122
...
perf(ds): inherit only LTS paths containing wildcards when adding a new generation
2024-01-23 17:14:51 -03:00
eecd7e084c
test(ds): reduce flakiness
2024-01-23 09:47:03 -03:00
94692162b5
Merge pull request #12370 from zhongwencool/api-key-auth
...
chore: document api-key path api only support bearerAuth
2024-01-23 14:22:35 +08:00
0dbaaa5d94
chore: document api-key path api only support bearerAuth
2024-01-23 10:31:43 +08:00
609ba7e332
fix(ds): do not count persistent session-only routed messages as dropped
...
Fixes https://emqx.atlassian.net/browse/EMQX-11539
2024-01-22 16:58:45 -03:00
d122340c13
Merge pull request #12338 from thalesmg/ds-message-gc-20240115
...
feat(ds): add message GC
2024-01-22 16:57:26 -03:00
7035b4c8b3
feat(ps): add message gc
2024-01-22 14:53:17 -03:00
75b08b525b
feat(ds): add `list_generations` and `drop_generation` APIs
2024-01-22 14:53:17 -03:00
6b064dd8eb
fix(prom_stats): missing metric key
...
- follow https://github.com/emqx/emqx/pull/11497
2024-01-22 10:39:53 +08:00
57f3efde63
feat(prom_auth): cluster metrics with different format-mode
2024-01-21 00:14:48 +08:00
57074015c6
feat(ds): allow customizing the data directory
...
The storage expectations for the RocksDB DB may be different from our usual data
directory. Also, it may consume a lot more storage than other data.
This allows customizing the data directory for the builtin DS storage backend.
Note: if the cluster was already initialized using a directory path, changing that config
will have no effect. This path is currently persisted in mnesia and used when reopening
the DB.
2024-01-19 13:07:24 -03:00
7d14a13b7b
Merge remote-tracking branch 'origin/release-55'
2024-01-18 14:36:30 +01:00
e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489
...
feat: refactor MQTT bridge to source, action, and connector
2024-01-18 09:28:14 -03:00
William Yang