Commit Graph

3661 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi cf22692c74 fix(emqx_channel): return Receive-Maximum in CONNACK when no error 2024-02-08 17:05:16 +01:00
ieQu1 811edb32a2
Merge pull request #12485 from ieQu1/dev/ds-fix-unsubscribe
fix(ds): Fix unsubscribe logic related to the replay
2024-02-07 21:38:25 +01:00
ieQu1 19c6d1127f
refactor(sessds): Extract subscription mgmt logic to separate module 2024-02-07 15:27:11 +01:00
ieQu1 3000a8f286
fix(sessds): Postpone deletion of the subscription until fully acked 2024-02-07 14:51:02 +01:00
ieQu1 30eb54e86b fix(sessds): Delay unsubscribe until full ack of in-flight messages 2024-02-06 02:49:07 +01:00
ieQu1 a9c55f7568 feat(sessds): Consider #srs with only QoS0 messages fully acked 2024-02-06 02:49:07 +01:00
zhongwencool b444c82a42 feat: rabbitmq bridge v2 integration 2024-02-06 07:57:12 +08:00
ieQu1 280fcd8c52
Merge pull request #12437 from ieQu1/dev/optimize_make_filter
Optimize emqx_ds_bitmask_keymapper:make_filter function.
2024-02-05 17:32:28 +01:00
ieQu1 c7888ad1f1
Merge pull request #12475 from ieQu1/dev/lean-stream
Use a more compact data structure to represent streams
2024-02-05 13:55:24 +01:00
Thales Macedo Garitezi e1ff276d76
Merge pull request #12471 from thalesmg/fix-config-upgrade-m-20240202
fix(config): apply config upgrade to deprecated configs
2024-02-05 09:05:12 -03:00
ieQu1 698ba3f271 fix(ds): Optimize emqx_ds_bitmask_keymapper:make_filter
This optimization makes idle polling faster
2024-02-05 10:54:19 +01:00
ieQu1 2e2f3cb2aa fix(sessds): Avoid stream hash collisions 2024-02-03 21:15:54 +01:00
ieQu1 2e56810ea2 refactor(ds): Use a simple improper list to represent the streams 2024-02-03 21:15:54 +01:00
Thales Macedo Garitezi 76401a302a fix(config): apply config upgrade to deprecated configs
Fixes https://emqx.atlassian.net/browse/EMQX-11845

Fixes https://github.com/emqx/emqx/issues/12452
2024-02-02 15:12:08 -03:00
Ivan Dyachkov ed8660cb68
Merge pull request #12458 from id/0202-sync-release-55
sync release-55
2024-02-02 17:20:38 +01:00
Zaiming (Stone) Shi b1a05c7b59
Merge pull request #12326 from zmstone/0111-unregister-session-with-timestamp
0111 unregister session with timestamp
2024-02-02 17:00:23 +01:00
Ivan Dyachkov 84b6d7d720 fix: remove 5.5.bpapi2 file 2024-02-02 10:38:50 +01:00
Zaiming (Stone) Shi f0569d8ae8 refactor: use mria:async_dirty to group dirty ops 2024-02-02 09:26:43 +01:00
Ivan Dyachkov dd490de2e1 chore: emqx 5.5.0 2024-02-02 08:48:56 +01:00
Ivan Dyachkov f1c7e716ce chore: 5.5.0-rc.2 2024-02-02 08:48:56 +01:00
Ivan Dyachkov dae3a94670 chore: 5.5.0-rc.1 2024-02-02 08:48:56 +01:00
ieQu1 98d1094d73 feat(sessds): Expose subscriptions in the REST API 2024-02-01 16:47:57 +01:00
Zaiming (Stone) Shi 3e518c1876
Merge pull request #12427 from zmstone/0130-limit-kafka-partitions
0130 limit kafka partitions
2024-02-01 14:39:52 +01:00
Ivan Dyachkov 3ad71f75e6 chore: emqx 5.5.0 2024-02-01 11:07:02 +01:00
JianBo He 0d5911d942
Merge pull request #12422 from zhongwencool/upgrade-erlfmt-to-v1.3.0
feat: upgrade erlfmt to support maybe syntax
2024-02-01 09:34:33 +08:00
Zaiming (Stone) Shi 3b6fbff9a5 test: fix compile warnings 2024-01-31 19:32:57 +01:00
Zaiming (Stone) Shi 53c217c383 refactor: micro optimization 2024-01-31 19:25:57 +01:00
Ivan Dyachkov 3f71a98e5e chore: 5.5.0-rc.2 2024-01-31 17:06:54 +01:00
Ilya Averyanov fe46434687 fix(sessds): fix renew stream logic 2024-01-31 18:02:29 +03:00
zhongwencool e9c8446d57 feat: upgrade erlfmt to support maybe syntax 2024-01-31 20:06:49 +08:00
Zaiming (Stone) Shi 38047108a4 test: add test coverage for emqx_cm_registry_keeper module 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 209331ad33 test: fix config test 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 87a2368e37 feat(metrics): add cluster_session guage 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 509ab6f35a feat(api): add /sessions_count api to count sessions 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi e9318752e6 feat: store session unregistration timestamp in emqx_cm_registry table 2024-01-31 09:50:12 +01:00
Ilya Averyanov 5bc67cb288
Merge pull request #12432 from ieQu1/ds_ignore_fully_replayed_streams
fix(sessds): Stream scheduler must ignore fully replayed streams
2024-01-31 09:51:49 +02:00
ieQu1 e843d9fd91 fix(sessds): Stream scheduler must ignore fully replayed streams 2024-01-31 00:20:54 +01:00
Ivan Dyachkov a2ba8fc645 chore: 5.5.0-rc.1 2024-01-30 18:58:00 +01:00
Zaiming (Stone) Shi 55fdc83aa5 Merge remote-tracking branch 'origin/release-55' into 0130-syn-release-55-to-master 2024-01-30 12:06:43 +01:00
Shawn 206af96a33 ci: update testcases for message-expiry-interval 2024-01-30 15:46:11 +08:00
Serge Tupchii 5007650bd2 perf(emqx_broker): pick broker pool worker by topic/shard pair to distribute the load more evenly.
Fixes: EMQX-11812
2024-01-29 20:57:10 +02:00
Shawn 9f22c2c455 ci: add some sleep and retry to emqx_persistent_session_ds_SUITE 2024-01-29 18:18:18 +08:00
Shawn 12da3c0986 feat: configurable server side message_expiry_interval 2024-01-29 17:29:52 +08:00
Zaiming (Stone) Shi dbe2baa405
Merge pull request #12408 from zmstone/sync-release-55
Sync release 55
2024-01-27 19:20:58 +00:00
ieQu1 2c6a776318 fix(sessds): Stricter checks for PacketIds 2024-01-26 17:49:33 +01:00
ieQu1 eec56b0d6b fix(sessds): Improve comments 2024-01-26 17:49:33 +01:00
ieQu1 3c451c6ae6 test(sessds): Fix unstable tests 2024-01-26 17:49:33 +01:00
ieQu1 2d23212792 refactor(sessds): #ifs -> #srs 2024-01-26 17:49:33 +01:00
ieQu1 974760d331 test(sessds): Don't trap exits in the test 2024-01-26 17:49:33 +01:00
ieQu1 ebe2339810 fix(sessds): Use mria:async_dirty instead of transaction 2024-01-26 17:49:33 +01:00
ieQu1 f5b9bd30aa fix(sessds): Apply review remarks 2024-01-26 17:49:33 +01:00
ieQu1 39857626ce test(sessds): Fix failing tests 2024-01-26 17:49:33 +01:00
ieQu1 e7b03cdc59 test(sessds): Create a property-based test for the session state 2024-01-26 17:49:33 +01:00
ieQu1 893656f092 fix(sessds): Fix session garbage collection after the refactoring 2024-01-26 17:49:33 +01:00
ieQu1 963df8f941 fix(sessds): Delete the routes when the session expires 2024-01-26 17:49:33 +01:00
ieQu1 cff6c15e13 fix(sessds): Store the QoS as the MSB of the packet ID 2024-01-26 17:49:33 +01:00
ieQu1 978a3bfef3 refactor(sessds): Simplify representation of QoS tracks 2024-01-26 17:49:33 +01:00
ieQu1 1b4f69b44d refactor(sessds): Simplify data structure of ds_state pmap datatype 2024-01-26 17:49:33 +01:00
ieQu1 4f4831fe7f refactor(sessds): Factor out stream scheduler into its own module 2024-01-26 17:49:33 +01:00
ieQu1 82ef34998a feat(sessds): Index streams by a unique subid 2024-01-26 17:49:33 +01:00
ieQu1 3fb2064ea4 test(sessds): Add property-based tests for seqno generator 2024-01-26 17:49:33 +01:00
ieQu1 8e8d3af096 fix(sessds): Refactor emqx_persistent_session_ds to use CRUD module 2024-01-26 17:49:33 +01:00
ieQu1 2d08aa88d8 refactor(ds): Create a CRUD module for the persistent session 2024-01-26 17:49:33 +01:00
Zaiming (Stone) Shi 2a40152721 Merge remote-tracking branch 'origin/release-55' into sync-release-55 2024-01-26 16:51:13 +01:00
William Yang 933c00c7ad
Merge pull request #12392 from qzhuyan/perf/william/ws-no-utf8-validate
perf: new ws listener option to disable UTF-8 validation
2024-01-26 16:01:04 +01:00
ieQu1 0933dc818e chore(ekka): Bump version to 0.18.4
Don't escalate the errors in etcd node discovery to the node level.

Fixes: #12255
2024-01-26 14:25:16 +01:00
William Yang 726302ef6a perf: new ws listener option to disable UTF-8 validation 2024-01-26 09:55:17 +01:00
JianBo He e29a80d568
Merge pull request #12388 from qzhuyan/dev/william/per-listn-conn-cnt
feat: quic per listener conn count
2024-01-26 09:01:39 +08:00
Thales Macedo Garitezi 8e31afe6c2 fix(ds): don't make data dir part of the schema
The data directory was ending up being persisted in the database schema.  This led to
issues when opening the DB on different nodes.
2024-01-25 14:44:06 -03:00
Thales Macedo Garitezi b5567d9d9f
Merge pull request #12389 from thalesmg/ci-peer-cover-m-20240124
ci: start/flush cover when using peer
2024-01-25 10:00:50 -03:00
zhongwencool 4f8accc31b
Merge branch 'master' into sync-r55-m-20240124 2024-01-25 16:09:05 +08:00
Thales Macedo Garitezi 846ad42a65 fix(ds): don't use env var for data dir default value
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-25 16:01:44 +08:00
Thales Macedo Garitezi 024ffe2909 test(ds): reduce flakiness 2024-01-25 16:01:44 +08:00
zhongwencool 186e1591df chore: document api-key path api only support bearerAuth 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi 878c9ee8b1 fix(ds): do not count persistent session-only routed messages as dropped
Fixes https://emqx.atlassian.net/browse/EMQX-11539
2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi d323fc7c27 feat(ps): add message gc 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi 4a0fd756ae feat(ds): add `list_generations` and `drop_generation` APIs 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi fb39e1eacc feat(ds): allow customizing the data directory
The storage expectations for the RocksDB DB may be different from our usual data
directory.  Also, it may consume a lot more storage than other data.

This allows customizing the data directory for the builtin DS storage backend.

Note: if the cluster was already initialized using a directory path, changing that config
will have no effect.  This path is currently persisted in mnesia and used when reopening
the DB.
2024-01-25 16:01:42 +08:00
Serge Tupchii a8c6280a5e test(emqx_cm_SUITE): use one helper function: `emqx_pool:flush_async_tasks/1` 2024-01-25 16:01:42 +08:00
Serge Tupchii f52cc93d9d perf(emqx_cm): use a dedicated pool for channel cleanup
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.

Fixes: EMQX-11743
2024-01-25 16:01:42 +08:00
JianBo He 52a7b45d97 fix: upgrade ekka to 0.18.3 to ensure the leader correct 2024-01-25 14:47:59 +08:00
Thales Macedo Garitezi ae387d1812 ci: start/flush cover when using peer
`cover` is not automatically started by `peer`.  Without starting/flushing it, we don't
get coverage data from peer nodes.
2024-01-24 18:02:47 -03:00
William Yang 9fb6688801 fix(quic): Per listener conn count 2024-01-24 21:36:25 +01:00
ieQu1 7b5f2948fe test(ds): Fix flaky testcase 2024-01-24 19:33:30 +01:00
ieQu1 eee221f1d0 feat(ds): Make egress batching configurable 2024-01-24 19:33:30 +01:00
ieQu1 137535a821 feat(ds): Introduce egress process for the builtin backend 2024-01-24 19:33:30 +01:00
Thales Macedo Garitezi 5547a40ceb fix(ds): don't use env var for data dir default value
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-24 11:39:14 -03:00
Thales Macedo Garitezi 77329209a2
Merge pull request #12368 from thalesmg/ds-inherit-wildcards-only-lts-m-20240122
perf(ds): inherit only LTS paths containing wildcards when adding a new generation
2024-01-23 17:14:51 -03:00
Thales Macedo Garitezi eecd7e084c test(ds): reduce flakiness 2024-01-23 09:47:03 -03:00
zhongwencool 94692162b5
Merge pull request #12370 from zhongwencool/api-key-auth
chore: document api-key path api only support bearerAuth
2024-01-23 14:22:35 +08:00
zhongwencool 0dbaaa5d94 chore: document api-key path api only support bearerAuth 2024-01-23 10:31:43 +08:00
Thales Macedo Garitezi 609ba7e332 fix(ds): do not count persistent session-only routed messages as dropped
Fixes https://emqx.atlassian.net/browse/EMQX-11539
2024-01-22 16:58:45 -03:00
Thales Macedo Garitezi d122340c13
Merge pull request #12338 from thalesmg/ds-message-gc-20240115
feat(ds): add message GC
2024-01-22 16:57:26 -03:00
Thales Macedo Garitezi 7035b4c8b3 feat(ps): add message gc 2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi 75b08b525b feat(ds): add `list_generations` and `drop_generation` APIs 2024-01-22 14:53:17 -03:00
JimMoen 6b064dd8eb
fix(prom_stats): missing metric key
- follow https://github.com/emqx/emqx/pull/11497
2024-01-22 10:39:53 +08:00
JimMoen 57f3efde63
feat(prom_auth): cluster metrics with different format-mode 2024-01-21 00:14:48 +08:00
Thales Macedo Garitezi 57074015c6 feat(ds): allow customizing the data directory
The storage expectations for the RocksDB DB may be different from our usual data
directory.  Also, it may consume a lot more storage than other data.

This allows customizing the data directory for the builtin DS storage backend.

Note: if the cluster was already initialized using a directory path, changing that config
will have no effect.  This path is currently persisted in mnesia and used when reopening
the DB.
2024-01-19 13:07:24 -03:00
Zaiming (Stone) Shi 7d14a13b7b Merge remote-tracking branch 'origin/release-55' 2024-01-18 14:36:30 +01:00
Thales Macedo Garitezi e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489
feat: refactor MQTT bridge to source, action, and connector
2024-01-18 09:28:14 -03:00
Zaiming (Stone) Shi 556092b7d0 feat(authz/prometheus): add authz cache_miss counter 2024-01-18 09:01:17 +01:00
Zaiming (Stone) Shi 85b6a3454c fix(authz): use binary() type instead of string() for cache.excludes 2024-01-18 09:01:17 +01:00
Zaiming (Stone) Shi 4fa906731d
Merge pull request #12333 from zmstone/0116-add-tags-for-action-and-connectors
feat: add 'tags' field for action and connector
2024-01-16 18:56:57 +00:00
Serge Tupchii 80e82db282 test(emqx_cm_SUITE): use one helper function: `emqx_pool:flush_async_tasks/1` 2024-01-16 19:42:37 +02:00
Serge Tupchii b472b56883 perf(emqx_cm): use a dedicated pool for channel cleanup
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.

Fixes: EMQX-11743
2024-01-16 19:08:02 +02:00
Zaiming (Stone) Shi 01d52e37c4
fix: resource tag type should be binary string
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-01-16 17:35:22 +01:00
Thales Macedo Garitezi 8f304d3456 test(bridge_v2_api): refactor suite to use CT matrix 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 6511693b2e refactor(action_api): prepare for `/sources` HTTP API 2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi 996a851cf6 chore: format username as string in log formatter 2024-01-16 14:42:10 +01:00
Zaiming (Stone) Shi 541525c50f fix(authz): fix authz result logs
prior to this fix, it's always the default authz result logged at
warning level
2024-01-16 14:40:53 +01:00
Zaiming (Stone) Shi 1fe1a62fe2 test: fix already exported function warning 2024-01-16 14:00:34 +01:00
Zaiming (Stone) Shi ab66986f16 feat: add 'tags' field for action and connector 2024-01-16 13:57:48 +01:00
JianBo He b6d0365027
Merge pull request #12267 from HJianBo/new-timeout-param-for-invite
feat(cluster): supports inviting nodes to join the cluster in an asynchronous manner
2024-01-16 10:11:20 +08:00
Zaiming (Stone) Shi 58a54adbb4 Merge remote-tracking branch 'origin/master' into release-55 2024-01-15 17:58:48 +01:00
William Yang d2d3ddb72a test(quic): listener port updates 2024-01-15 11:42:45 +01:00
William Yang c77837a9ea feat(quic): support reload with new binding port 2024-01-15 11:33:25 +01:00
JianBo He 4c40e754f4 chore: forward the async invite to leader node 2024-01-15 11:24:05 +08:00
Zaiming (Stone) Shi 61417f26d4 chore: bump versions to 5.5.x 2024-01-12 13:54:35 +01:00
Andrew Mayorov b4eac25678
chore(broker): fix code style issue
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-12 13:54:23 +01:00
Andrew Mayorov 2f98f1faaf
fix(route-sync): ensure batch sync preserve idemopotency 2024-01-12 11:57:26 +01:00
Andrew Mayorov e21a3497c7
chore(route-sync): turn TODO into comment 2024-01-12 11:57:26 +01:00
Andrew Mayorov 8f4758d9d4
feat(route-sync): use the smallest possible min sync delay 2024-01-12 11:57:26 +01:00
Andrew Mayorov 2ac6cddf19
fix(route-sync): handle batch sync errors gracefully 2024-01-12 11:57:25 +01:00
Andrew Mayorov a28fc7bfa8
feat(route-sync): do not run empty batches 2024-01-12 11:57:25 +01:00
Andrew Mayorov 2dffd44985
feat(route-sync): allow to enable syncer pool only on cores/replicants
So we would able to roll it out and test more gradually.
2024-01-12 11:57:25 +01:00
Andrew Mayorov 884f784c1c
refactor(router): don't `emqx_broker_helper:monitor/1` in batches
As per out current understanding, this doesn't changes much in terms of
observability, since other nodes call `ekka:monitor(membership)` anyway,
so they will observe nodedowns without explicitly writing an entry into
the `?ROUTING_NODE` table.
2024-01-12 11:57:25 +01:00
Andrew Mayorov d6f731c4fc
fix(route-sync): use public function as mria activity target 2024-01-12 11:57:25 +01:00
Andrew Mayorov 7d037cfe91
chore(route-sync): clarify why wait indefinitely for reply 2024-01-12 11:57:24 +01:00
Andrew Mayorov 498b7a922d
chore(syncer): fix code style issues
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-12 11:57:24 +01:00
Andrew Mayorov 0b3f5f7c37
feat(syncer): allow to turn syncer pool on/off through config 2024-01-12 11:57:24 +01:00
Andrew Mayorov 38e13f2337
fix(syncer): start syncer pool before broker pool
As the latter depends on the former.
2024-01-12 11:57:24 +01:00
Andrew Mayorov a1ccf85c66
test(routesync): verify that syncer preserves consistency
Under a highly concurrent load. Be aware that this testcase is not
deterministic.
2024-01-12 11:57:24 +01:00
Andrew Mayorov 5aeff20f8b
fix(routesync): ensure causal relationships are preserved
At the cost of strict FIFO semantics though.
2024-01-12 11:57:23 +01:00
Andrew Mayorov 54f8b47455
feat(routing): add route sync process pool
Dedicated to synchronizing local state updates with the global view
of the routing state.
2024-01-12 11:57:18 +01:00
JianBo He 9e85b53c39
Merge pull request #12296 from HJianBo/fix-delayed-message-http-api-500
fix(delayed): fix http 500 error
2024-01-12 09:37:41 +08:00
Ilya Averyanov 7b0b2a0527 fix(conn): avoid storing incomplete channel info 2024-01-11 18:08:13 +03:00
JianBo He 596607d549 chore: update bpapi.versions 2024-01-11 14:56:02 +08:00
Serge Tupchii 965ce5d446 perf: use mria:match_delete/2 to cleanup routes
This must be much more network efficient since both Mria and Mnesia
need only to replicate one op, e.g.: `{MatchPattern, clear_table}`
instead of replicating one per each key to be deleted.
2024-01-10 16:23:58 +02:00
Serge Tupchii febaaefc38 chore: upgrade ekka to 0.18.1 (mria 0.8.1)
Mria 0.8.1 release implements `mria:match_delete/2` function
2024-01-10 16:23:58 +02:00
Zaiming (Stone) Shi 7c8a36fc06
Merge pull request #12289 from zmstone/0108-support-acl-cache-masks
0108 support acl cache excludes
2024-01-10 13:40:29 +00:00
Zaiming (Stone) Shi 9e8a67fd68 feat: support authz cache exclusion config
now one can configure a list of topic-filters to avoid
caching ACL check results

for example

authorization.cache.excludes = ["nocache/#"]

this means ACL check results for topics having 'nocache/' prefix
will not be cached
2024-01-10 13:52:00 +01:00
Andrew Mayorov ff2fb95e5d
fix(conf): bump to hocon 0.40.4
Which includes a bugfix for undefined sensitive values obfuscation.
2024-01-10 13:09:08 +01:00
Zaiming (Stone) Shi 98e94d8619 docs: fix typo in comments 2024-01-09 19:16:55 +01:00
William Yang 2201dec2f4 feat(quic): 3s cutoff time for fast_close 2024-01-09 17:29:28 +01:00
William Yang 2483a4ecff feat(quic): support listener config reload 2024-01-09 17:29:25 +01:00
William Yang 973f40041d chore: bump 0.0.311 2024-01-09 09:35:03 +01:00
Ivan Dyachkov 06117c3a33 Merge remote-tracking branch 'upstream/release-54' into 0105-sync-r54 2024-01-05 14:20:38 +01:00
Ivan Dyachkov d04ac08d1d chore: release 5.4.1 2024-01-05 11:48:28 +01:00
Ivan Dyachkov d4c83cf463 chore: tag 5.4.1-alpha.1 2024-01-02 14:37:45 +01:00
Andrew Mayorov a87df28dfc
Merge pull request #12243 from keynslug/fix/EMQX-11483/route-async-repl
fix(router): rely on local sync state updates to propagate routes
2023-12-29 16:31:00 +01:00
Andrew Mayorov 5c91984ad7
chore: leave comment describing need for tighter synchronization 2023-12-29 14:04:48 +01:00
JimMoen 5e100f52b8
style: erlfmt all `rebar.config` files and `bin/nodetool` 2023-12-29 09:08:03 +08:00
JimMoen a3f33cc8bc
chore: fix typos in code and suite 2023-12-29 09:07:51 +08:00
JimMoen ed10827343
docs: README spell 2023-12-29 09:07:25 +08:00
Andrew Mayorov b9627c420f
fix(router): rely on local sync state updates to propagate routes
Instead of (potentially lagging) global table state itself. Local
state should be the only source of truth anyway. Moreover,
synchronize parts of local state update logic through the broker
pool to ensure consistency.

The problem was that `emqx_router:has_route/2` check may observe a
stale route, which deletion is not yet replicated from the core node
to the local replicant node.

For example:
1. The only one subscriber per a given topic A unsubscribes from
   a replicant node.
2. The route to A is deleted by the emqx_broker.
3. Mria makes RPC to a core node, it succeeds and returns.
4. The client resubscribes or another client subscribes to the same
   topic A.
5. emqx_broker tries to add a route again:
   `emqx_router:do_add_route(Topic)`.
6. `emqx_router` checks if the route is present.
7. The stale route is present because deletion not replicated yet,
   so no route is being added.
8. Route deletion is replicated locally but it's too late:
   we already have a local subscriber without a route.

Co-Authored-By: Serge Tupchii <serge.tupchii@protonmail.com>
2023-12-28 19:16:31 +01:00
Andrew Mayorov 331bfaa535
fix(broker): avoid reclaiming per-topic counter twice 2023-12-28 17:43:50 +01:00
Andrew Mayorov a8cd609ff4
chore(broker): simplify subscribe code path 2023-12-28 17:43:42 +01:00
Zaiming (Stone) Shi 4669c4d552 fix(bridge/mqtt): ensure short clientid
Some mqtt brokers do not allow long client IDs.
To make it compatible with this limitation, this commit
tries to limit the number of bytes under 23 with a best-effort
attempt to derive it from the bridge name.
2023-12-28 11:53:27 +01:00
Zaiming (Stone) Shi e421e9ce56 Merge remote-tracking branch 'origin/release-54' into 1222-sync-e5.4.0-build.2-to-master 2023-12-23 11:22:05 +01:00
Zaiming (Stone) Shi f36b842d94 chore: prepare for e5.4.0 retag 2023-12-23 11:18:19 +01:00
Zaiming (Stone) Shi 86a018d921 chore: bump app versions 2023-12-22 21:28:26 +01:00
Zaiming (Stone) Shi 9fdac4af0c Merge remote-tracking branch 'origin/master' into 1222-sync-e5.4.0-build.2-to-master 2023-12-22 21:27:27 +01:00
Zaiming (Stone) Shi ac6e4cf2cc fix: config update for ft 2023-12-22 19:01:14 +01:00
Zaiming (Stone) Shi 733196aa83 chore: prepare for e5.4.0-build.2 2023-12-22 18:59:46 +01:00
Zaiming (Stone) Shi 23ded313ec chore: update app versions 2023-12-22 15:29:22 +01:00
Zaiming (Stone) Shi 891fd972bd Merge remote-tracking branch 'origin/release-54' into 1221-prepare-bpapi-on-otp-26 2023-12-22 15:19:25 +01:00
Zaiming (Stone) Shi 5b5d4a6f68 chore: bump release version to e5.4.0-build.1 2023-12-22 14:00:12 +01:00
Zaiming (Stone) Shi 20543d55ef chore: bump app vsn 2023-12-22 13:13:30 +01:00
Zaiming (Stone) Shi 322b7bb7d2 chore: bump app vsn 2023-12-22 13:00:37 +01:00
Zaiming (Stone) Shi ecc31217fd Merge remote-tracking branch 'origin/master' into 1221-prepare-bpapi-on-otp-26 2023-12-22 12:48:06 +01:00
Zaiming (Stone) Shi 17d3037a13 chore: add 5.4.bpapi 2023-12-22 12:47:41 +01:00
ieQu1 ec0693861a fix(static_checks): Prepare for OTP26 2023-12-22 12:02:30 +01:00
Thales Macedo Garitezi 035f5f977e Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231221 2023-12-21 17:53:05 -03:00
Zaiming (Stone) Shi 36d2929534 chore: prepare for release version 5.4.0 2023-12-21 19:51:12 +01:00
Thales Macedo Garitezi 18b69809da Merge branch 'release-54' into sync-r54-m-20231221 2023-12-21 10:05:13 -03:00
Thales Macedo Garitezi a43f49f8fa chore: prepare to tag `5.4.0-rc.1` 2023-12-20 12:43:08 -03:00
Andrew Mayorov 76c89ad372
test(gw-authz): switch to `emqx_cth_suite` + simplify testsuite
Also separate DTLS related MQTT-SN test config, so that it doesn't
leak into this test suite.
2023-12-20 14:37:49 +01:00
Andrew Mayorov 3d679f7e26
test(cth-tls): add test helpers for TLS related activities
Currently, only certificate issuing and related utilities are there.
2023-12-20 11:46:07 +01:00
Andrew Mayorov 78d3f49fe4
fix(gw): use more conservative set of DTLS options 2023-12-20 11:46:04 +01:00
Andrew Mayorov d3b32b64e1
Merge pull request #12201 from keynslug/feat/EMQX-11527/tls-hot-update
feat(listen): support hot config update of MQTT listeners
2023-12-20 09:45:12 +01:00
JianBo He cc15e3a03b
Merge pull request #12199 from zhongwencool/configs-put-bridge-v1-api
fix: support bridge v1 conf hocon-format in the put configs API.
2023-12-20 09:32:32 +08:00
Andrew Mayorov a6c26ce992
test(listen): stabilize flaky testcase 2023-12-19 23:36:52 +01:00
Andrew Mayorov baf46c9aa2
test(listen): also change CA on update and verify clients notice 2023-12-19 22:48:38 +01:00
Andrew Mayorov 7e4049620d
fix(listen): ensure limiter server state consistent with updates 2023-12-19 16:06:27 +01:00
Andrew Mayorov 4796f85dff
test(listen): verify SSL options of WSS listener are hot updateable 2023-12-19 16:06:27 +01:00
Andrew Mayorov 036048b9ab
feat(listen): support hot update of WS/WSS options as well
However, currently any transport option changes will cause listener to
restart, which slightly impacts the availability of the broker.
2023-12-19 16:06:27 +01:00
Andrew Mayorov 68bbbec556
chore: bump to esockd 5.11.1
With a typespec fix.
2023-12-19 16:06:27 +01:00
Andrew Mayorov fb2d4544b6
test(listen): verify that SSL options are hot-updateable 2023-12-19 16:06:26 +01:00
Andrew Mayorov da0f0f947e
feat(listen): support hot config update of esockd-based listeners 2023-12-19 16:06:26 +01:00
Zaiming (Stone) Shi 2b6176aa60 fix: upgrade to gen_rpc 3.3.1 which does not log cast args only arity 2023-12-19 12:46:42 +01:00
Andrew Mayorov cafd384466
chore: upgrade to esockd 5.11.0 2023-12-19 11:15:09 +01:00
zhongwencool a46b415c77 fix: support bridge v1 conf hocon-format in the put configs API. 2023-12-19 16:40:45 +08:00
Thales Macedo Garitezi cf9331a95f Merge branch 'release-54' into sync-r54-m-20231218 2023-12-18 17:21:08 -03:00
Andrew Mayorov 78f7d01b1c
test(listen): simplify test suite setup / teardown 2023-12-18 20:52:57 +01:00
Zaiming (Stone) Shi ef2cf71bea chore: bump version to 5.4.0-alpha.2 2023-12-18 13:51:21 +01:00
Zaiming (Stone) Shi a9963e043b refactor(authz): improve logging
Move authz result logging to common place.

Prior to this change, the final result is not logged when
fallback to the default authorization.no_match config value.

Aso, if the result is provided by a hook callback,
it's also not logged.

After this change, only the final result is logged.
The authz chain resutls can be traced (or logged at debug level).
2023-12-17 22:32:26 +01:00
Serge Tupchii 24db03a7df chore(emqx_opentelemetry): remove unused `event` functionality 2023-12-15 14:58:45 +02:00
Zaiming (Stone) Shi cf33b3b4b1 fix(emqx.app.src): add dependen apps in app.src 2023-12-15 10:40:33 +01:00
zhongwencool d9e179b364 fix: use microsecond precision as the primary key for audit logs to prevent duplication 2023-12-15 09:07:04 +08:00
Zaiming (Stone) Shi 406a2d7a52 test: fix shared sub test case flaky 2023-12-14 22:21:31 +01:00