Serge Tupchii
f52cc93d9d
perf(emqx_cm): use a dedicated pool for channel cleanup
...
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.
Fixes: EMQX-11743
2024-01-25 16:01:42 +08:00
JianBo He
52a7b45d97
fix: upgrade ekka to 0.18.3 to ensure the leader correct
2024-01-25 14:47:59 +08:00
Thales Macedo Garitezi
ae387d1812
ci: start/flush cover when using peer
...
`cover` is not automatically started by `peer`. Without starting/flushing it, we don't
get coverage data from peer nodes.
2024-01-24 18:02:47 -03:00
William Yang
9fb6688801
fix(quic): Per listener conn count
2024-01-24 21:36:25 +01:00
ieQu1
7b5f2948fe
test(ds): Fix flaky testcase
2024-01-24 19:33:30 +01:00
ieQu1
eee221f1d0
feat(ds): Make egress batching configurable
2024-01-24 19:33:30 +01:00
ieQu1
137535a821
feat(ds): Introduce egress process for the builtin backend
2024-01-24 19:33:30 +01:00
Thales Macedo Garitezi
5547a40ceb
fix(ds): don't use env var for data dir default value
...
Fixes https://github.com/emqx/emqx/pull/12380
2024-01-24 11:39:14 -03:00
Thales Macedo Garitezi
77329209a2
Merge pull request #12368 from thalesmg/ds-inherit-wildcards-only-lts-m-20240122
...
perf(ds): inherit only LTS paths containing wildcards when adding a new generation
2024-01-23 17:14:51 -03:00
Thales Macedo Garitezi
eecd7e084c
test(ds): reduce flakiness
2024-01-23 09:47:03 -03:00
zhongwencool
94692162b5
Merge pull request #12370 from zhongwencool/api-key-auth
...
chore: document api-key path api only support bearerAuth
2024-01-23 14:22:35 +08:00
zhongwencool
0dbaaa5d94
chore: document api-key path api only support bearerAuth
2024-01-23 10:31:43 +08:00
Thales Macedo Garitezi
609ba7e332
fix(ds): do not count persistent session-only routed messages as dropped
...
Fixes https://emqx.atlassian.net/browse/EMQX-11539
2024-01-22 16:58:45 -03:00
Thales Macedo Garitezi
d122340c13
Merge pull request #12338 from thalesmg/ds-message-gc-20240115
...
feat(ds): add message GC
2024-01-22 16:57:26 -03:00
Thales Macedo Garitezi
7035b4c8b3
feat(ps): add message gc
2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi
75b08b525b
feat(ds): add `list_generations` and `drop_generation` APIs
2024-01-22 14:53:17 -03:00
JimMoen
6b064dd8eb
fix(prom_stats): missing metric key
...
- follow https://github.com/emqx/emqx/pull/11497
2024-01-22 10:39:53 +08:00
JimMoen
57f3efde63
feat(prom_auth): cluster metrics with different format-mode
2024-01-21 00:14:48 +08:00
Thales Macedo Garitezi
57074015c6
feat(ds): allow customizing the data directory
...
The storage expectations for the RocksDB DB may be different from our usual data
directory. Also, it may consume a lot more storage than other data.
This allows customizing the data directory for the builtin DS storage backend.
Note: if the cluster was already initialized using a directory path, changing that config
will have no effect. This path is currently persisted in mnesia and used when reopening
the DB.
2024-01-19 13:07:24 -03:00
Zaiming (Stone) Shi
7d14a13b7b
Merge remote-tracking branch 'origin/release-55'
2024-01-18 14:36:30 +01:00
Thales Macedo Garitezi
e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489
...
feat: refactor MQTT bridge to source, action, and connector
2024-01-18 09:28:14 -03:00
Zaiming (Stone) Shi
556092b7d0
feat(authz/prometheus): add authz cache_miss counter
2024-01-18 09:01:17 +01:00
Zaiming (Stone) Shi
85b6a3454c
fix(authz): use binary() type instead of string() for cache.excludes
2024-01-18 09:01:17 +01:00
Zaiming (Stone) Shi
4fa906731d
Merge pull request #12333 from zmstone/0116-add-tags-for-action-and-connectors
...
feat: add 'tags' field for action and connector
2024-01-16 18:56:57 +00:00
Serge Tupchii
80e82db282
test(emqx_cm_SUITE): use one helper function: `emqx_pool:flush_async_tasks/1`
2024-01-16 19:42:37 +02:00
Serge Tupchii
b472b56883
perf(emqx_cm): use a dedicated pool for channel cleanup
...
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.
Fixes: EMQX-11743
2024-01-16 19:08:02 +02:00
Zaiming (Stone) Shi
01d52e37c4
fix: resource tag type should be binary string
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-01-16 17:35:22 +01:00
Thales Macedo Garitezi
8f304d3456
test(bridge_v2_api): refactor suite to use CT matrix
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
6511693b2e
refactor(action_api): prepare for `/sources` HTTP API
2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi
996a851cf6
chore: format username as string in log formatter
2024-01-16 14:42:10 +01:00
Zaiming (Stone) Shi
541525c50f
fix(authz): fix authz result logs
...
prior to this fix, it's always the default authz result logged at
warning level
2024-01-16 14:40:53 +01:00
Zaiming (Stone) Shi
1fe1a62fe2
test: fix already exported function warning
2024-01-16 14:00:34 +01:00
Zaiming (Stone) Shi
ab66986f16
feat: add 'tags' field for action and connector
2024-01-16 13:57:48 +01:00
JianBo He
b6d0365027
Merge pull request #12267 from HJianBo/new-timeout-param-for-invite
...
feat(cluster): supports inviting nodes to join the cluster in an asynchronous manner
2024-01-16 10:11:20 +08:00
Zaiming (Stone) Shi
58a54adbb4
Merge remote-tracking branch 'origin/master' into release-55
2024-01-15 17:58:48 +01:00
William Yang
d2d3ddb72a
test(quic): listener port updates
2024-01-15 11:42:45 +01:00
William Yang
c77837a9ea
feat(quic): support reload with new binding port
2024-01-15 11:33:25 +01:00
JianBo He
4c40e754f4
chore: forward the async invite to leader node
2024-01-15 11:24:05 +08:00
Zaiming (Stone) Shi
61417f26d4
chore: bump versions to 5.5.x
2024-01-12 13:54:35 +01:00
Andrew Mayorov
b4eac25678
chore(broker): fix code style issue
...
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-12 13:54:23 +01:00
Andrew Mayorov
2f98f1faaf
fix(route-sync): ensure batch sync preserve idemopotency
2024-01-12 11:57:26 +01:00
Andrew Mayorov
e21a3497c7
chore(route-sync): turn TODO into comment
2024-01-12 11:57:26 +01:00
Andrew Mayorov
8f4758d9d4
feat(route-sync): use the smallest possible min sync delay
2024-01-12 11:57:26 +01:00
Andrew Mayorov
2ac6cddf19
fix(route-sync): handle batch sync errors gracefully
2024-01-12 11:57:25 +01:00
Andrew Mayorov
a28fc7bfa8
feat(route-sync): do not run empty batches
2024-01-12 11:57:25 +01:00
Andrew Mayorov
2dffd44985
feat(route-sync): allow to enable syncer pool only on cores/replicants
...
So we would able to roll it out and test more gradually.
2024-01-12 11:57:25 +01:00
Andrew Mayorov
884f784c1c
refactor(router): don't `emqx_broker_helper:monitor/1` in batches
...
As per out current understanding, this doesn't changes much in terms of
observability, since other nodes call `ekka:monitor(membership)` anyway,
so they will observe nodedowns without explicitly writing an entry into
the `?ROUTING_NODE` table.
2024-01-12 11:57:25 +01:00
Andrew Mayorov
d6f731c4fc
fix(route-sync): use public function as mria activity target
2024-01-12 11:57:25 +01:00
Andrew Mayorov
7d037cfe91
chore(route-sync): clarify why wait indefinitely for reply
2024-01-12 11:57:24 +01:00
Andrew Mayorov
498b7a922d
chore(syncer): fix code style issues
...
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-12 11:57:24 +01:00
Andrew Mayorov
0b3f5f7c37
feat(syncer): allow to turn syncer pool on/off through config
2024-01-12 11:57:24 +01:00
Andrew Mayorov
38e13f2337
fix(syncer): start syncer pool before broker pool
...
As the latter depends on the former.
2024-01-12 11:57:24 +01:00
Andrew Mayorov
a1ccf85c66
test(routesync): verify that syncer preserves consistency
...
Under a highly concurrent load. Be aware that this testcase is not
deterministic.
2024-01-12 11:57:24 +01:00
Andrew Mayorov
5aeff20f8b
fix(routesync): ensure causal relationships are preserved
...
At the cost of strict FIFO semantics though.
2024-01-12 11:57:23 +01:00
Andrew Mayorov
54f8b47455
feat(routing): add route sync process pool
...
Dedicated to synchronizing local state updates with the global view
of the routing state.
2024-01-12 11:57:18 +01:00
JianBo He
9e85b53c39
Merge pull request #12296 from HJianBo/fix-delayed-message-http-api-500
...
fix(delayed): fix http 500 error
2024-01-12 09:37:41 +08:00
Ilya Averyanov
7b0b2a0527
fix(conn): avoid storing incomplete channel info
2024-01-11 18:08:13 +03:00
JianBo He
596607d549
chore: update bpapi.versions
2024-01-11 14:56:02 +08:00
Serge Tupchii
965ce5d446
perf: use mria:match_delete/2 to cleanup routes
...
This must be much more network efficient since both Mria and Mnesia
need only to replicate one op, e.g.: `{MatchPattern, clear_table}`
instead of replicating one per each key to be deleted.
2024-01-10 16:23:58 +02:00
Serge Tupchii
febaaefc38
chore: upgrade ekka to 0.18.1 (mria 0.8.1)
...
Mria 0.8.1 release implements `mria:match_delete/2` function
2024-01-10 16:23:58 +02:00
Zaiming (Stone) Shi
7c8a36fc06
Merge pull request #12289 from zmstone/0108-support-acl-cache-masks
...
0108 support acl cache excludes
2024-01-10 13:40:29 +00:00
Zaiming (Stone) Shi
9e8a67fd68
feat: support authz cache exclusion config
...
now one can configure a list of topic-filters to avoid
caching ACL check results
for example
authorization.cache.excludes = ["nocache/#"]
this means ACL check results for topics having 'nocache/' prefix
will not be cached
2024-01-10 13:52:00 +01:00
Andrew Mayorov
ff2fb95e5d
fix(conf): bump to hocon 0.40.4
...
Which includes a bugfix for undefined sensitive values obfuscation.
2024-01-10 13:09:08 +01:00
Zaiming (Stone) Shi
98e94d8619
docs: fix typo in comments
2024-01-09 19:16:55 +01:00
William Yang
2201dec2f4
feat(quic): 3s cutoff time for fast_close
2024-01-09 17:29:28 +01:00
William Yang
2483a4ecff
feat(quic): support listener config reload
2024-01-09 17:29:25 +01:00
William Yang
973f40041d
chore: bump 0.0.311
2024-01-09 09:35:03 +01:00
Ivan Dyachkov
06117c3a33
Merge remote-tracking branch 'upstream/release-54' into 0105-sync-r54
2024-01-05 14:20:38 +01:00
Ivan Dyachkov
d04ac08d1d
chore: release 5.4.1
2024-01-05 11:48:28 +01:00
Ivan Dyachkov
d4c83cf463
chore: tag 5.4.1-alpha.1
2024-01-02 14:37:45 +01:00
Andrew Mayorov
a87df28dfc
Merge pull request #12243 from keynslug/fix/EMQX-11483/route-async-repl
...
fix(router): rely on local sync state updates to propagate routes
2023-12-29 16:31:00 +01:00
Andrew Mayorov
5c91984ad7
chore: leave comment describing need for tighter synchronization
2023-12-29 14:04:48 +01:00
JimMoen
5e100f52b8
style: erlfmt all `rebar.config` files and `bin/nodetool`
2023-12-29 09:08:03 +08:00
JimMoen
a3f33cc8bc
chore: fix typos in code and suite
2023-12-29 09:07:51 +08:00
JimMoen
ed10827343
docs: README spell
2023-12-29 09:07:25 +08:00
Andrew Mayorov
b9627c420f
fix(router): rely on local sync state updates to propagate routes
...
Instead of (potentially lagging) global table state itself. Local
state should be the only source of truth anyway. Moreover,
synchronize parts of local state update logic through the broker
pool to ensure consistency.
The problem was that `emqx_router:has_route/2` check may observe a
stale route, which deletion is not yet replicated from the core node
to the local replicant node.
For example:
1. The only one subscriber per a given topic A unsubscribes from
a replicant node.
2. The route to A is deleted by the emqx_broker.
3. Mria makes RPC to a core node, it succeeds and returns.
4. The client resubscribes or another client subscribes to the same
topic A.
5. emqx_broker tries to add a route again:
`emqx_router:do_add_route(Topic)`.
6. `emqx_router` checks if the route is present.
7. The stale route is present because deletion not replicated yet,
so no route is being added.
8. Route deletion is replicated locally but it's too late:
we already have a local subscriber without a route.
Co-Authored-By: Serge Tupchii <serge.tupchii@protonmail.com>
2023-12-28 19:16:31 +01:00
Andrew Mayorov
331bfaa535
fix(broker): avoid reclaiming per-topic counter twice
2023-12-28 17:43:50 +01:00
Andrew Mayorov
a8cd609ff4
chore(broker): simplify subscribe code path
2023-12-28 17:43:42 +01:00
Zaiming (Stone) Shi
4669c4d552
fix(bridge/mqtt): ensure short clientid
...
Some mqtt brokers do not allow long client IDs.
To make it compatible with this limitation, this commit
tries to limit the number of bytes under 23 with a best-effort
attempt to derive it from the bridge name.
2023-12-28 11:53:27 +01:00
Zaiming (Stone) Shi
e421e9ce56
Merge remote-tracking branch 'origin/release-54' into 1222-sync-e5.4.0-build.2-to-master
2023-12-23 11:22:05 +01:00
Zaiming (Stone) Shi
f36b842d94
chore: prepare for e5.4.0 retag
2023-12-23 11:18:19 +01:00
Zaiming (Stone) Shi
86a018d921
chore: bump app versions
2023-12-22 21:28:26 +01:00
Zaiming (Stone) Shi
9fdac4af0c
Merge remote-tracking branch 'origin/master' into 1222-sync-e5.4.0-build.2-to-master
2023-12-22 21:27:27 +01:00
Zaiming (Stone) Shi
ac6e4cf2cc
fix: config update for ft
2023-12-22 19:01:14 +01:00
Zaiming (Stone) Shi
733196aa83
chore: prepare for e5.4.0-build.2
2023-12-22 18:59:46 +01:00
Zaiming (Stone) Shi
23ded313ec
chore: update app versions
2023-12-22 15:29:22 +01:00
Zaiming (Stone) Shi
891fd972bd
Merge remote-tracking branch 'origin/release-54' into 1221-prepare-bpapi-on-otp-26
2023-12-22 15:19:25 +01:00
Zaiming (Stone) Shi
5b5d4a6f68
chore: bump release version to e5.4.0-build.1
2023-12-22 14:00:12 +01:00
Zaiming (Stone) Shi
20543d55ef
chore: bump app vsn
2023-12-22 13:13:30 +01:00
Zaiming (Stone) Shi
322b7bb7d2
chore: bump app vsn
2023-12-22 13:00:37 +01:00
Zaiming (Stone) Shi
ecc31217fd
Merge remote-tracking branch 'origin/master' into 1221-prepare-bpapi-on-otp-26
2023-12-22 12:48:06 +01:00
Zaiming (Stone) Shi
17d3037a13
chore: add 5.4.bpapi
2023-12-22 12:47:41 +01:00
ieQu1
ec0693861a
fix(static_checks): Prepare for OTP26
2023-12-22 12:02:30 +01:00
Thales Macedo Garitezi
035f5f977e
Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231221
2023-12-21 17:53:05 -03:00
Zaiming (Stone) Shi
36d2929534
chore: prepare for release version 5.4.0
2023-12-21 19:51:12 +01:00
Thales Macedo Garitezi
18b69809da
Merge branch 'release-54' into sync-r54-m-20231221
2023-12-21 10:05:13 -03:00
Thales Macedo Garitezi
a43f49f8fa
chore: prepare to tag `5.4.0-rc.1`
2023-12-20 12:43:08 -03:00
Andrew Mayorov
76c89ad372
test(gw-authz): switch to `emqx_cth_suite` + simplify testsuite
...
Also separate DTLS related MQTT-SN test config, so that it doesn't
leak into this test suite.
2023-12-20 14:37:49 +01:00
Andrew Mayorov
3d679f7e26
test(cth-tls): add test helpers for TLS related activities
...
Currently, only certificate issuing and related utilities are there.
2023-12-20 11:46:07 +01:00
Andrew Mayorov
78d3f49fe4
fix(gw): use more conservative set of DTLS options
2023-12-20 11:46:04 +01:00
Andrew Mayorov
d3b32b64e1
Merge pull request #12201 from keynslug/feat/EMQX-11527/tls-hot-update
...
feat(listen): support hot config update of MQTT listeners
2023-12-20 09:45:12 +01:00
JianBo He
cc15e3a03b
Merge pull request #12199 from zhongwencool/configs-put-bridge-v1-api
...
fix: support bridge v1 conf hocon-format in the put configs API.
2023-12-20 09:32:32 +08:00
Andrew Mayorov
a6c26ce992
test(listen): stabilize flaky testcase
2023-12-19 23:36:52 +01:00
Andrew Mayorov
baf46c9aa2
test(listen): also change CA on update and verify clients notice
2023-12-19 22:48:38 +01:00
Andrew Mayorov
7e4049620d
fix(listen): ensure limiter server state consistent with updates
2023-12-19 16:06:27 +01:00
Andrew Mayorov
4796f85dff
test(listen): verify SSL options of WSS listener are hot updateable
2023-12-19 16:06:27 +01:00
Andrew Mayorov
036048b9ab
feat(listen): support hot update of WS/WSS options as well
...
However, currently any transport option changes will cause listener to
restart, which slightly impacts the availability of the broker.
2023-12-19 16:06:27 +01:00
Andrew Mayorov
68bbbec556
chore: bump to esockd 5.11.1
...
With a typespec fix.
2023-12-19 16:06:27 +01:00
Andrew Mayorov
fb2d4544b6
test(listen): verify that SSL options are hot-updateable
2023-12-19 16:06:26 +01:00
Andrew Mayorov
da0f0f947e
feat(listen): support hot config update of esockd-based listeners
2023-12-19 16:06:26 +01:00
Zaiming (Stone) Shi
2b6176aa60
fix: upgrade to gen_rpc 3.3.1 which does not log cast args only arity
2023-12-19 12:46:42 +01:00
Andrew Mayorov
cafd384466
chore: upgrade to esockd 5.11.0
2023-12-19 11:15:09 +01:00
zhongwencool
a46b415c77
fix: support bridge v1 conf hocon-format in the put configs API.
2023-12-19 16:40:45 +08:00
Thales Macedo Garitezi
cf9331a95f
Merge branch 'release-54' into sync-r54-m-20231218
2023-12-18 17:21:08 -03:00
Andrew Mayorov
78f7d01b1c
test(listen): simplify test suite setup / teardown
2023-12-18 20:52:57 +01:00
Zaiming (Stone) Shi
ef2cf71bea
chore: bump version to 5.4.0-alpha.2
2023-12-18 13:51:21 +01:00
Zaiming (Stone) Shi
a9963e043b
refactor(authz): improve logging
...
Move authz result logging to common place.
Prior to this change, the final result is not logged when
fallback to the default authorization.no_match config value.
Aso, if the result is provided by a hook callback,
it's also not logged.
After this change, only the final result is logged.
The authz chain resutls can be traced (or logged at debug level).
2023-12-17 22:32:26 +01:00
Serge Tupchii
24db03a7df
chore(emqx_opentelemetry): remove unused `event` functionality
2023-12-15 14:58:45 +02:00
Zaiming (Stone) Shi
cf33b3b4b1
fix(emqx.app.src): add dependen apps in app.src
2023-12-15 10:40:33 +01:00
zhongwencool
d9e179b364
fix: use microsecond precision as the primary key for audit logs to prevent duplication
2023-12-15 09:07:04 +08:00
Zaiming (Stone) Shi
406a2d7a52
test: fix shared sub test case flaky
2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
fa85c3061b
test: fix emqx_ocsp_cache_SUITE error message match pattern
2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
19051f639b
test: ensure verify_none to ssl client opts as default value
2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
778662de5d
test(crl_cache): fix ssl options
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
dca72e0052
test: fix mtls options
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
06bfd878a0
test: fix compile warnings in test suites
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
a0833282f2
fix(dialyzer): fix dialyzer warnings for emqx app standalone
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
b7531839cd
fix(dialyzer): delete stale code
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
bcae3d67c7
Merge pull request #12169 from ieQu1/dev/bpapi-separate-for-otp26
...
fix(static_checks): Prepare for OTP26
2023-12-14 20:32:45 +01:00
Zaiming (Stone) Shi
c1f2287b86
Merge remote-tracking branch 'origin/release-54'
2023-12-14 15:26:49 +01:00
ieQu1
02e2231734
fix(static_checks): Prepare for OTP26
2023-12-14 15:22:09 +01:00
Thales Macedo Garitezi
dbc8141930
Merge pull request #12125 from thalesmg/ds-cache-m-20231206
...
chore(ds): return DS key from `next` and add `update_iterator` callback
2023-12-14 10:23:25 -03:00
firest
f1bde41bf6
fix(sso): Correctly handle wrapped passwords when updating
2023-12-14 10:07:04 +08:00
firest
d278d3afb5
feat(ds): implemented the replayer scheduler
2023-12-13 23:52:02 +08:00
zhongwencool
1807df157c
fix: license watermark percent support float
2023-12-12 17:27:31 +08:00
JimMoen
6552e0523a
Merge pull request #12141 from JimMoen/fix-shared-sub-api-internal-error
...
Fix shared sub api internal error
2023-12-11 21:43:09 +08:00
Zaiming (Stone) Shi
b6c11be159
Merge pull request #12138 from zmstone/1208-fix-dialyzer
...
fix(dialyzer): batch 3
2023-12-11 12:52:56 +01:00
Zaiming (Stone) Shi
d560366c14
test: fix some compile warnings
2023-12-11 09:43:13 +01:00
JimMoen
8c25371ff3
chore: bump gitignore for emqx standalone common test SUITES
2023-12-11 15:37:36 +08:00
JimMoen
68da627b4d
feat(channel): add peerport field in ClientInfo
2023-12-11 14:11:18 +08:00
Zaiming (Stone) Shi
50f4aba5cd
fix(dialyzer): batch 3
2023-12-09 15:50:09 +01:00
Thales Macedo Garitezi
66d043becd
feat(ds): introduce `update_iterator` callback
2023-12-08 15:04:18 -03:00
Andrew Mayorov
76970c268e
fix(sessds): stop overwriting QoS0-only pubrange checkpoints
...
Through making `StreamRef` part of a record key. Otherwise with
QoS0-only streams the seqno doesn't change, thus ranges over
different streams will have the same `FirstSeqno`.
2023-12-08 14:33:47 +01:00
Andrew Mayorov
24710d0f56
fix(sessds): avoid accumulating QoS0-only ranges in memory
2023-12-08 14:31:18 +01:00
Thales Macedo Garitezi
f37c86afb9
Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231207
2023-12-07 13:30:04 -03:00
Thales Macedo Garitezi
2a6d72878f
chore(ds): return DS message key along with batch
2023-12-07 11:36:08 -03:00
William Yang
499f4d4bc5
chore(quicer): dialyzer
2023-12-07 13:52:48 +01:00
William Yang
583645d97b
chore: bump emqtt 1.10.1 and quicer 0.0.308
2023-12-07 13:52:44 +01:00
ieQu1
9b612cb0e7
Merge pull request #12119 from ieQu1/EMQX-10342
...
Make stream renew timer configurable
2023-12-07 10:42:23 +01:00
Zaiming (Stone) Shi
7560016f0e
fix(hocon): upgrade hocon from 0.40.0 to 0.40.1
...
fixed type spec, exported some public types from emqx_schema module
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi
6b553e37c0
chore: pin ekka 0.17.0 previously 0.15.16
...
change summary:
1. otp 26
2. type spec fixes
3. dropped UDP mcast clustering
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi
423b586c56
fix(dialyzer): fix some dialyzer issues found on otp 26
2023-12-06 20:32:49 +01:00
Zaiming (Stone) Shi
71f9838fd7
chore: upgrade to emqtt 1.10.0
2023-12-06 19:18:15 +01:00
Zaiming (Stone) Shi
83206daa20
refactor(emqx_cm_locker): delete dead code
2023-12-06 19:18:15 +01:00
William Yang
64401cb6cc
Merge pull request #12042 from qzhuyan/dev/william/rel54/esockd-robustnes
...
fix: bump to esockd 8.9.9 improve robustness
2023-12-06 17:39:38 +01:00
Andrew Mayorov
074dc8ce11
test(emqx): switch rest of test suite to use `emqx_cth_suite`
2023-12-06 16:47:04 +01:00
ieQu1
371ec34992
feat(sessds): Make stream renew interval configurable
2023-12-06 16:04:53 +01:00
ieQu1
c43b3eb535
fix(sessds): Add debug logs for the session garbage collection
2023-12-06 15:37:23 +01:00
Andrew Mayorov
c2c9de69b4
test(emqx): switch `emqx_crl_cache_SUITE` to use `emqx_cth_suite`
2023-12-06 14:34:41 +01:00
Andrew Mayorov
f06a1f10ef
test(emqx): switch most of testsuites to use `emqx_cth_suite`
...
For increased robustness and isolation.
2023-12-06 14:34:41 +01:00
Andrew Mayorov
d643330cb9
test(ocsp): switch test suite to use `emqx_cth_suite`
...
And simplify it slightly in the process.
2023-12-06 14:34:40 +01:00
Andrew Mayorov
7d80511b3a
test(emqx): switch select test suites to use `emqx_cth_suite`
2023-12-06 14:34:40 +01:00
Thales Macedo Garitezi
8a5b0d252d
Merge pull request #12110 from thalesmg/test-plugin-flaky-r54-20231205
...
test(cth_peer): use an exclusive current dir for each peer
2023-12-06 10:16:13 -03:00
William Yang
c4956d99d1
fix: bump to esockd 8.9.9 improve listener robustness
2023-12-06 13:55:13 +01:00
Thales Macedo Garitezi
b82189fb4d
test(cth_peer): use an exclusive current dir for each peer
2023-12-06 09:35:13 -03:00
Andrew Mayorov
43813e29c2
chore(emqx): drop legacy session persistence impl leftovers
2023-12-06 10:53:57 +01:00
SergeTupchiy
28ff53e99c
Merge pull request #11984 from SergeTupchiy/EMQX-10535-openetelmetry-tracing-new
...
Integrate OpenTelmetry tracing
2023-12-06 11:45:41 +02:00
Andrew Mayorov
3798060543
Merge pull request #12109 from keynslug/test/emqx-cth-suite
...
test(emqx): switch select test suites to use `emqx_cth_suite`
2023-12-06 09:48:53 +01:00
Andrew Mayorov
83bea2254d
test(ocsp): switch test suite to use `emqx_cth_suite`
...
And simplify it slightly in the process.
2023-12-05 19:16:37 +01:00
Andrew Mayorov
6f5228e991
test(emqx): switch select test suites to use `emqx_cth_suite`
2023-12-05 19:16:37 +01:00
Serge Tupchii
938508b270
refactor(emqx_opentelemetry): pass only channel info to trace functions
2023-12-05 19:01:08 +02:00
Thales Macedo Garitezi
deb3fcd606
Merge remote-tracking branch 'origin/master' into sync-m-r54-20231205
2023-12-05 13:25:45 -03:00
Serge Tupchii
85441fda0f
test(emqx_opentelemetry): add trace test suite
2023-12-05 18:18:31 +02:00
Serge Tupchii
8d3f98eff2
fix(emqx_opentelemetry): fix dialyzer warnings
2023-12-05 18:18:31 +02:00
Serge Tupchii
7fdc650448
feat: integrate OpenTelemetry traces
2023-12-05 18:11:00 +02:00
Andrew Mayorov
286d483a3a
Merge pull request #12055 from keynslug/ft/EMQX-11474/subopts
...
fix(sessds): respect subscription options when publishing
2023-12-05 15:45:25 +01:00
Andrew Mayorov
92c4b29a4c
feat(topic): match empty topic levels more loosely
...
So that the result of `emqx_topic:tokens/1` would be perfectly
matchable with the result of `emqx_topic:words/1` of a topic filter
with empty levels.
2023-12-04 13:38:55 +03:00
Andrew Mayorov
b5c485cd55
test(sessds): add empty level topic subscription testcase
2023-12-04 13:38:42 +03:00
JimMoen
cb992fcf55
Merge pull request #12059 from JimMoen/1130-fix-vm-time-warp
...
fix: vm.args use `multi_time_warp`
2023-12-04 11:56:08 +08:00
Andrew Mayorov
627c58b07d
fix(sessds): handle `expire_awaiting_rel` common timer with no-op
...
Otherwise, the channel crashes when the timer is triggered.
2023-12-02 15:00:30 +03:00
Andrew Mayorov
47bc747323
fix(sessds): drop everything related to session on CleanStart
2023-12-02 15:00:30 +03:00
Andrew Mayorov
fd26e690b8
refactor(sessds): move parts of message processing to replayer
...
To simplify the processing flow, reducing the number of back-and-forth
between the session and the replayer.
2023-12-02 15:00:29 +03:00
Andrew Mayorov
6255ee0833
fix(sessmem): avoid defining unsafe defaults in `#session`
2023-12-02 14:38:04 +03:00
Andrew Mayorov
b5ad0f9815
fix(test): subscribe with QoS 2 to receive QoS 2 message
2023-12-02 14:38:04 +03:00
Andrew Mayorov
1948719eec
test(sessds): unskip testcase related to QoS2 handling
2023-12-02 14:38:04 +03:00
Andrew Mayorov
3265d2f2aa
fix(sessds): respect subscription options when publishing
2023-12-02 14:38:03 +03:00
Andrew Mayorov
508346f095
refactor(topicidx): split persistent term stuff off gbt-based index
2023-12-02 14:36:21 +03:00
Andrew Mayorov
88103c5f0e
refactor(session): pass config to `SessionImpl:open/3` as well
...
* Anticipate that connection info may change during session takeover.
* Avoid persisting session conf as part of persistent session state.
2023-12-02 14:36:21 +03:00
Andrew Mayorov
6eba082992
Merge pull request #12075 from keynslug/fix/EMQX-11474/function-clause
...
fix(channel): expect there's no session yet on disconnect
2023-12-02 14:32:50 +03:00
Zaiming (Stone) Shi
55621fbb86
Merge pull request #12074 from zmstone/1201-user-peer-module-for-all
...
test: replace 'slave' and 'ct_slave' with 'peer'
2023-12-01 21:35:40 +01:00
Zaiming (Stone) Shi
6e30718818
test: allow join cluster later
2023-12-01 20:33:39 +01:00
ieQu1
3d2e95fe3e
Merge pull request #12081 from ieQu1/dev/gen-rpc-3.3.0
...
chore(gen_rpc): Bump version to 3.3.0
2023-12-01 17:57:33 +01:00
ieQu1
476c300ecf
Merge pull request #12070 from ieQu1/dev/ds-min-max-batch-size
...
Various minor fixes
2023-12-01 17:57:15 +01:00
ieQu1
d3b7478d70
chore(gen_rpc): Bump version to 3.3.0
2023-12-01 17:10:12 +01:00
Zaiming (Stone) Shi
ab7d57e1c5
test: delete debug level logging
2023-12-01 17:07:14 +01:00
Ivan Dyachkov
0e27d031a8
chore: 5.4.0-alpha.1
2023-12-01 16:54:03 +01:00
Zaiming (Stone) Shi
6a9cc20d4d
test: wait for all nodes clustered
2023-12-01 16:37:39 +01:00
Andrew Mayorov
6b17920fea
fix(channel): expect there's no session yet on disconnect
2023-12-01 15:29:05 +03:00
Andrew Mayorov
cce474ce98
fix(session): include session term in stacktrace
2023-12-01 15:29:05 +03:00
ieQu1
e1ec560639
test(emqx): Fix flaky emqx_takeover_SUITE:t_takeover testcase
...
This testcase uses QoS1, so it must account for possible duplication
of the messages.
2023-12-01 10:06:18 +01:00
Zaiming (Stone) Shi
7f4d91d490
Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54
2023-12-01 08:32:22 +01:00
ieQu1
0e625d814a
feat(sessds): Make batch size configurable
...
This change affects flow control. It allows to configure maximum size
of a batch, as well as fetch threshold.
2023-12-01 08:27:05 +01:00
ieQu1
4717e56fb6
fix(sessds): Schedule poll immediately upon receiving an ack
...
This commit affects the flow control, and improves the throughput by
removing a delay between freeing up space in the in-flight window and
polling new messages.
2023-12-01 08:27:05 +01:00
ieQu1
38800c0260
refactor(sessds): Store timers in the session
2023-12-01 08:27:05 +01:00
Zaiming (Stone) Shi
22f7cc1622
test: replace 'slave' and 'ct_slave' with 'peer'
2023-12-01 08:07:09 +01:00
JimMoen
ee25d9bd9e
fix: vm.args use `multi_time_warp`
...
See also:
https://www.erlang.org/doc/apps/erts/time_correction#multi-time-warp-mode
2023-12-01 14:52:31 +08:00
ieQu1
69f1ca43c3
fix(sessds): Create a timer even if it's present in the map
2023-12-01 06:11:13 +01:00
ieQu1
1897e5c31b
refactor(session): Use common naming conventions
2023-12-01 06:11:08 +01:00
ieQu1
d9e7544070
refactor(sessds): Introduce macros for the timers
2023-12-01 06:11:08 +01:00
ieQu1
0d245acdc1
Merge pull request #12066 from keynslug/fix/sessds-tc-flaky
...
test(sessds): wait client disconnect propagates to broker
2023-12-01 00:13:38 +01:00
Thales Macedo Garitezi
113a4ad4b1
Merge pull request #12057 from thalesmg/ds-session-gc-m-20231128
...
feat(ds): add session gc process
2023-11-30 17:51:52 -03:00
Ivan Dyachkov
9fd2fa95a8
chore: bump apps versions
2023-11-30 20:01:12 +01:00
Thales Macedo Garitezi
684d637fed
test(bridge_api): workaround strange config syncing problem
...
For some unknown reason, this test has difficulties in syncing the config correctly
between the nodes, while the equivalent in bridge_v2_api_SUITE doesn't.
2023-11-30 19:51:35 +01:00
Ivan Dyachkov
ec10c51073
Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53
2023-11-30 19:51:12 +01:00
Zaiming (Stone) Shi
dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26
...
1129 prepare for OTP 26
2023-11-30 19:38:40 +01:00
Andrew Mayorov
ec471c0557
test(sessds): wait client disconnect propagates to broker
2023-11-30 20:14:57 +03:00
Ivan Dyachkov
6dd92c382f
chore: 5.3.2 release
2023-11-30 16:18:44 +01:00
Thales Macedo Garitezi
cf6cb3e4ad
chore: set low importance to config
2023-11-30 11:55:57 -03:00
Thales Macedo Garitezi
1ab009f081
refactor: rename supervisor
2023-11-30 11:55:48 -03:00
Zaiming (Stone) Shi
eed253af82
test: implement a new node restart helper func
2023-11-30 15:11:38 +01:00
Thales Macedo Garitezi
880f5e8f89
feat(ds): add session gc process
...
Fixes https://emqx.atlassian.net/browse/EMQX-9744
2023-11-30 09:38:59 -03:00
Zaiming (Stone) Shi
cf72c04fdd
test: fix peer node stop and plugin SUITE typo
2023-11-30 09:53:36 +01:00
Zaiming (Stone) Shi
f2db4cc7fc
chore: upgrade to gen_rpc 3.2.2
2023-11-30 09:42:44 +01:00
Zaiming (Stone) Shi
640b0df319
test: do not add -master erl flag for peer nodes
2023-11-30 08:49:57 +01:00
Zaiming (Stone) Shi
4ecfe2be30
test: use peer module for slave and ct_slave
2023-11-29 21:45:53 +01:00
Ivan Dyachkov
7f5433f6dd
chore: 5.3.2-rc.1
2023-11-29 18:20:15 +01:00
Thales Macedo Garitezi
fdafe2493e
Merge pull request #12041 from thalesmg/ds-estimate-last-alive-m-20231128
...
fix(ds_session): take conservative estimate of `last_alive_at` when bumping
2023-11-29 13:43:59 -03:00
Zaiming (Stone) Shi
6f35f25163
chore: upgrade esockd to 5.9.8 for OTP 26
2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
1a563b4f65
chore: fix 0.0 match for OTP 26
2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
89cdfbca63
fix(emqx_vm): trim new-line in otp version string
2023-11-29 15:32:25 +01:00
Andrew Mayorov
ccef91437d
Merge pull request #12029 from keynslug/ft/EMQX-11049/qos2
...
feat(sessds): provide QoS2 message replay support
2023-11-29 14:10:16 +03:00
Andrew Mayorov
d8691f1d64
refactor(sessds): rename marker → committed offset
...
For better clarity.
2023-11-29 13:04:42 +03:00
William Yang
64f6220cf5
feat(quic): bump to quicer-0.0.303 and emqtt 1.9.7
2023-11-28 14:43:47 +01:00
Thales Macedo Garitezi
f5c4fb5860
fix(ds_session): take conservative estimate of `last_alive_at` when bumping
...
Addresses https://github.com/emqx/emqx/pull/12024#discussion_r1407432154
2023-11-28 09:58:03 -03:00
Ivan Dyachkov
2e11ab6a16
chore: 5.3.2-alpha.2
2023-11-28 13:46:28 +01:00
Zaiming (Stone) Shi
1b1cea24f3
Merge remote-tracking branch 'origin/release-54' into 1128-sync-release-54
2023-11-28 11:43:30 +01:00
Andrew Mayorov
34c9c022d0
chore(replayer): add comment describing what "until" means
2023-11-28 11:47:54 +03:00
JianBo He
c8b5c51bbc
chore: fix failed test cases
2023-11-28 09:53:46 +08:00
JianBo He
dc99651690
test(bridge): ensure almost test cases passed
2023-11-28 09:53:46 +08:00
Serge Tupchii
64ee29af81
feat(emqx_management): implement data backup API
2023-11-27 17:30:58 +02:00
Andrew Mayorov
86685bdce2
feat(sessds): use integer tags for pubrange types
2023-11-27 18:27:15 +03:00
Andrew Mayorov
bb05281adb
refactor(sessds): add dedicated `#ds_pubrange.tracks` field
...
This slightly simplifies the replayer code.
2023-11-27 18:24:57 +03:00
Thales Macedo Garitezi
d88deb9ceb
feat(ds): add session timer to bump last alive at timestamp
2023-11-27 12:19:52 -03:00
Thales Macedo Garitezi
09c4e40511
refactor(ds): rename `disconnected_at` to `last_alive_at`, add more assertions
2023-11-27 11:48:44 -03:00
Andrew Mayorov
923898eadf
chore(chan): leave a TODO note for PUBREC handler
2023-11-27 09:55:46 +03:00
Andrew Mayorov
41973ee1fa
fix(sessds): please dialyzer with well-typed dummy msgs
2023-11-27 09:55:46 +03:00
Andrew Mayorov
46475fac66
feat(sessds): provide QoS2 message replay support
2023-11-27 09:55:45 +03:00
Andrew Mayorov
ce59cb71bb
chore: bump emqtt to 1.9.6
2023-11-27 09:55:45 +03:00
zhongwencool
b5a00ec6b2
Merge pull request #12023 from emqx/master
...
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
ieQu1
bd7a84fe3e
revert(ds): Don't duplicate the clean start in session_ds
2023-11-26 19:18:59 +01:00
Thales Macedo Garitezi
839f9dbedb
feat(ds): session expiry
...
Fixes https://emqx.atlassian.net/browse/EMQX-11048
2023-11-24 16:31:54 -03:00
ieQu1
2d4b9a7b9d
Merge pull request #12019 from ieQu1/ds-clean-start
...
Discard sessions when client connects with clean start = true
2023-11-24 13:35:39 +01:00
ieQu1
e616e0746a
feat(ds): Implement a function for dumping persistent session state
2023-11-24 04:39:46 +01:00
ieQu1
8dfcb69e52
feat(ds): Discard session when client connects with CleanStart=1
2023-11-24 04:17:28 +01:00
ieQu1
e8d18b0e09
Merge pull request #12014 from ieQu1/ds-qos0
...
Support QoS0 messages in persistent_session_ds
2023-11-23 23:43:18 +01:00
ieQu1
a158f25a40
fix(ds): Fix return type of emqx_persistent_session_ds:publish
2023-11-23 22:16:38 +01:00
ieQu1
c5bb86db67
feat(ds): Support QoS 0
2023-11-23 22:16:38 +01:00
ieQu1
1ced8786fd
feat(ds): Make session poll interval configurable
2023-11-23 17:10:03 +01:00
ieQu1
82e74d0201
feat(ds): Add a flag that forces all sessions to become durable
2023-11-23 17:10:03 +01:00
Thales Macedo Garitezi
6c9417efe0
Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122
2023-11-22 12:02:34 -03:00
Ilya Averyanov
a1b9a14fa1
feat(ds): allow fdb implementation for durable storage
2023-11-22 14:49:38 +03:00
ieQu1
ee191803ea
Merge pull request #11990 from ieQu1/dev/ds-shards
...
Implement a prototype of durable message sharding
2023-11-22 00:57:54 +01:00
Zaiming (Stone) Shi
5ebd954b16
chore: bump release version to e5.3.2-alpha.1
2023-11-21 21:19:05 +01:00
Zaiming (Stone) Shi
8fbdcab118
Merge pull request #11987 from zmstone/1120-do-not-crash-on-einval-after-check-cache
...
fix(emqx_connection): handle socket activation error return
2023-11-21 20:31:56 +01:00
ieQu1
3165b4f645
fix(ds): Abort application startup when rocksdb is not avialable
2023-11-21 20:27:07 +01:00
ieQu1
4d47490734
chore(ds): Rebase configuration
2023-11-21 19:37:01 +01:00
ieQu1
2a1f7d946a
feat(ds): Shard messages by publisher client ID
2023-11-21 19:37:01 +01:00
ieQu1
62542e5844
feat(ds): Metadata storage for the replication layer
2023-11-21 19:37:01 +01:00
Zaiming (Stone) Shi
fa91bacdfb
Merge pull request #11975 from zmstone/1119-fix-socket-close-race-condition
...
fix(emqx_channel): do not log stale sock_close event as error
2023-11-21 16:47:03 +01:00
Thales Macedo Garitezi
068d151b14
Merge pull request #11983 from thalesmg/sync-r53-to-m-20231120
...
sync `release-53` to `master`
2023-11-21 09:02:51 -03:00
Ilya Averyanov
c6fd1e4c75
Merge pull request #11971 from savonarola/1116-no-auth-rebalance-status
...
feat(rebalance): improve rebalance usability
2023-11-21 12:10:18 +02:00
Andrew Mayorov
110a5a4896
feat(schema): introduce separate root for new session persistence
...
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
2023-11-20 23:49:14 +07:00
Zaiming (Stone) Shi
b02711af79
refactor(emqx_ws_connection): rename cache to buffer for limiter
2023-11-20 17:47:20 +01:00
Zaiming (Stone) Shi
ec19247271
refactor: rename limiter buffer related messages and var names
2023-11-20 16:55:26 +01:00
Andrew Mayorov
9684e79ee0
fix(sessds): ensure dup flag is on for replayed messages
2023-11-20 22:54:40 +07:00
Zaiming (Stone) Shi
8ec3b1db5d
fix(emqx_connection): handle socket activation error return
2023-11-20 16:51:39 +01:00
Thales Macedo Garitezi
53e796bbd0
Merge remote-tracking branch 'origin/release-53' into sync-r53-to-m-20231120
2023-11-20 11:49:52 -03:00
Andrew Mayorov
ef7cfd0202
feat(sessds): add field to `ds_pubrange` for forward-compat
2023-11-20 19:56:55 +07:00
Andrew Mayorov
d7974d835f
fix(sharesub): anticipate messages w/o redispatch header
...
For instance, `emqx_session_mem` will push almost all of the messages
that are still in the state to `emqx_shared_sub:redispatch/1`. After
this commit, the session will no longer crash during channel terminate.
2023-11-20 15:53:24 +07:00
Andrew Mayorov
7081f1951f
refactor(sessds): use `ds_pubrange` record as is
...
Instead of converting it into almost similar runtime representation.
2023-11-20 15:40:43 +07:00
Andrew Mayorov
a5ff4144fe
test(sessds): add complex testcase for session replay
2023-11-20 15:40:43 +07:00
Andrew Mayorov
1246d714c5
feat(sessds): preserve acks / ranges in mnesia for replays
2023-11-20 13:33:48 +07:00
Andrew Mayorov
5b40304d1f
chore(sessds): simplify subscriptions handling
...
There's currently no point in storing parsed topic filters in the
subscriptions table.
2023-11-20 13:33:48 +07:00
Andrew Mayorov
648b6ac63e
chore(sessds): rename iterators -> subscriptions
...
Also try to make clearer the difference between 2 flavors of topic
filter representation in use.
2023-11-20 13:33:48 +07:00
Zaiming (Stone) Shi
e73bf716ae
fix(emqx_channel): do not log stale sock_close event as error
...
In some cases, EMQX may decide to close socket and mark connection
at 'disconnected' state, for example, when DISCONNECTE packet is
received, or, when failed to write data to socket.
However, by the time EMQX decided to close the socket, the socket
might have already been closed by peer, and the `tcp_closed` envet
is already delivered to the process mailbox -- causing EMQX to
handle sock_close event at 'disconnected' state.
2023-11-19 22:27:25 +01:00
Ilya Averyanov
e93e9ed108
feat(rebalance): improve rebalance usability
...
* make availability API endpoint public
* allow connections during wait_health_check interval
* make availability status calculation more consistent and lightweight
* refactor test to get rid of some mocks and to use cth
2023-11-18 21:29:16 +03:00
Thales Macedo Garitezi
90571b7d8e
test: fix noise about undefined unofficial callbacks
2023-11-16 13:16:02 -03:00
Andrew Mayorov
656d575e2d
ci(router): fix flaky testcase
2023-11-16 17:53:20 +07:00
Andrew Mayorov
d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
...
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
Andrew Mayorov
98f947f4f3
ci(router): fix flaky testcase
2023-11-15 22:28:52 +07:00
Andrew Mayorov
8919b08207
fix(utils): rename `emqx_utils_stream:take/2` to `consume/2`
...
Which is more neutral and harder to confuse with a destructive `take` in
collections.
2023-11-15 17:20:40 +07:00
Thales Macedo Garitezi
423fb32171
test: fix noise about undefined unofficial callbacks
2023-11-14 19:48:39 +01:00
Ivan Dyachkov
7c0e345d3a
Merge remote-tracking branch 'upstream/release-54'
2023-11-14 19:38:21 +01:00
ieQu1
124c0e2dba
fix(ds): Respect receive_maximum from the connection info
2023-11-14 16:06:19 +01:00
Ivan Dyachkov
bbf00c2f71
Merge pull request #11946 from id/1114-sync-r53
...
sync release-53 to master
2023-11-14 13:38:24 +01:00
Thales Macedo Garitezi
01a2a3b1c0
Merge pull request #11930 from thalesmg/ds-session-discard-m-20231110
...
feat(ds): implement session discard
2023-11-14 09:30:34 -03:00
Andrew Mayorov
6812ee9d0f
fix(mgmt): hide route selection behind router interface
...
Also introduce a generic _stream_ concept, mostly to deal with
iterating over 2 ETS tables at once with `ets:match_object/3`.
2023-11-14 18:36:25 +07:00
Ivan Dyachkov
28a577ad09
chore: bump apps versions
2023-11-14 11:02:26 +01:00
Ivan Dyachkov
1c57993c91
Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53
2023-11-14 10:44:35 +01:00
Ivan Dyachkov
a9cf8fe78c
chore: 5.3.1
2023-11-14 08:44:41 +01:00
Zaiming (Stone) Shi
9b9908780c
Merge pull request #11926 from kjellwinblad/kjell/fix_bridge_v2_to_action_cluster_hocon/EMQX-11353
...
fix: reflect bridge V1 upgreade in cluster.hocon
2023-11-13 14:18:57 +01:00
Thales Macedo Garitezi
45dad2ed3a
feat(ds): implement session discard
...
Fixes https://emqx.atlassian.net/browse/EMQX-9739
Fixes some issues to ensure the session is discarded when the client connects with
`clean_start = true`, and added some cleanup to subscriptions/routes/iterators/streams.
> There is an API that session garbage collector can use to perform cleaning
We already have `emqx_session:destroy/1`, which could serve as an API for a periodic
session GC to use.
2023-11-13 09:06:22 -03:00
Kjell Winblad
26fc9ed89d
fix: log warning message after suggestion from @zmstone
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-11-13 12:53:09 +01:00
Andrew Mayorov
03843c6071
feat(router): switch to v2 routing store by default
2023-11-13 17:43:30 +07:00
Kjell Winblad
5838568d5d
fix: hard coded call to schema module
...
The user can configure the configuration loader and schema module so we
should not hard code a call to the default schema module.
2023-11-13 11:41:40 +01:00
Zaiming (Stone) Shi
65efa2672e
test(schema): delete stale reference to emqx_schema:file() type
2023-11-13 09:28:26 +01:00
JimMoen
867d5aaea6
chore: rm dbg funcs in SUITE
2023-11-12 21:44:23 +08:00
JimMoen
523bb17fc8
test: mark proper types gateway connection modules need be fixed
2023-11-12 21:43:10 +08:00
Zaiming (Stone) Shi
021f7e6b49
refactor(schema): comma_separated_list is list(string()) not list(any())
2023-11-12 11:37:58 +01:00
Zaiming (Stone) Shi
8be718b22f
refactor(emqx_schema): delete unused type bar_separated_list
2023-11-12 11:29:33 +01:00
ieQu1
5170267fef
Merge pull request #11924 from ieQu1/dev/emqx-ds-behavior
...
feat(ds): Introduce emqx_ds behavior
2023-11-10 15:23:28 +01:00
Kjell Winblad
6ebf0d1bdd
fix: reflect bridge V1 upgrade in override configurations
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11353
2023-11-10 14:52:29 +01:00
Zaiming (Stone) Shi
86110824eb
feat: upgrade hocon to 0.40.0 which supports union type display name
2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
8a4fba431e
refactor(emqx_limiter_schema): use typerefl alias
2023-11-10 13:41:51 +01:00
ieQu1
8dc8237331
feat(ds): Introduce emqx_ds behavior
2023-11-10 12:16:39 +01:00
ieQu1
38d004e06e
Merge pull request #11913 from ieQu1/dev/fix-hooks-comment
...
chore(hooks): Fix comment
2023-11-10 03:15:54 +01:00
ieQu1
ced7be61f0
Merge pull request #11917 from ieQu1/dev/fix-packet-id-to-seqno-trans
...
fix(ds): Fix packet id -> sequence number translation
2023-11-09 22:57:16 +01:00
Thales Macedo Garitezi
7a237f6a3a
Merge branch 'master' into sync-m-to-r54-20231109
2023-11-09 18:49:52 -03:00
Thales Macedo Garitezi
371a49304d
Merge branch 'release-53' into sync-r53-to-m-20231109
2023-11-09 17:19:23 -03:00
ieQu1
5aa9d026df
fix(ds): Apply review remarks
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-09 21:16:59 +01:00
ieQu1
b812db1e3c
fix(ds): Fix packet id -> sequence number translation
2023-11-09 20:12:39 +01:00
Thales Macedo Garitezi
2d539463cd
test: fix flaky test
...
```
Testing lib.emqx.emqx_cm_SUITE: *** FAILED test case 24 of 31 ***
%%% emqx_cm_SUITE ==> t_stepdown_session_takeover_begin_timeout: FAILED
%%% emqx_cm_SUITE ==>
Failure/Error: ?assertEqual([<0.6635.8>,<0.6520.8>], lists : sort ( emqx_cm : lookup_channels ( ClientId ) ))
expected: [<0.6635.8>,<0.6520.8>]
got: [<0.6520.8>,<0.6635.8>]
line: 309
```
2023-11-09 15:32:52 -03:00
Thales Macedo Garitezi
ee87f90b0d
Merge pull request #11906 from thalesmg/ds-avoid-records-m-20231108
...
chore(ds): avoid using records in persistence / rpc
2023-11-09 14:25:27 -03:00
ieQu1
42a9f0eb6b
Merge pull request #11914 from ieQu1/dev/speed-up-poll
...
fix(ds): Speed up polling for the new messages
2023-11-09 16:40:06 +01:00
Thales Macedo Garitezi
42536bb5f7
chore(ds): avoid using records in persistence / rpc
...
Fixes https://emqx.atlassian.net/browse/EMQX-11279
2023-11-09 12:00:25 -03:00
ieQu1
cf4a46a78b
fix(ds): Speed up polling for the new messages
...
Poll immediately if the previous poll returned non-empty result
2023-11-09 14:22:32 +01:00
ieQu1
3350bda446
chore(hooks): Fix comment
2023-11-09 14:05:13 +01:00
firest
2b97800c8b
fix(limiter): simplify the descriptions for fields of the limiter
2023-11-09 18:00:09 +08:00
William Yang
f7a2359329
test(quic): fix flaky
2023-11-09 10:56:16 +01:00
Ivan Dyachkov
0bf5deaed9
chore: merge master into release-54
2023-11-09 08:32:26 +01:00
Thales Macedo Garitezi
8540566eba
chore: prepare to tag `e5.3.1-alpha.5`
2023-11-08 13:02:19 -03:00
ieQu1
788698f157
Merge pull request #11720 from ieQu1/dev/refactor-persistent-session
...
Refactor emqx_durable storage application and introduce learned topic structure storage
2023-11-08 14:27:40 +01:00
Ilya Averyanov
d061d64c70
fix(plugins): fix backward compatibility
2023-11-08 12:36:09 +03:00
Ivan Dyachkov
0c91bec98d
chore: merge 'upstream/release-53'
2023-11-08 09:24:38 +01:00
ieQu1
a1cdbaa76d
fix(ds): Address code review remarks
2023-11-08 04:31:27 +01:00
ieQu1
7cb0322856
fix(emqx): Move bpapi and emqx message record to emqx_utils app
2023-11-08 03:04:19 +01:00
ieQu1
74cb43f8b1
fix(ds): Add unique ID to the key
2023-11-08 03:04:19 +01:00
Andrew Mayorov
46d8301bc0
feat(emqx): expose timestamp function in `emqx_message`
...
So that the code that relies on it would not need to guess clock source
and precision.
2023-11-08 03:04:19 +01:00
ieQu1
8e5dda40be
fix(ds): Fix static checks
2023-11-08 03:04:19 +01:00
ieQu1
99329e1243
refactor(ds): Address review remarks
2023-11-08 03:04:19 +01:00
ieQu1
2de79dd9ac
feat(ds): Replay QoS1 messages
2023-11-08 03:04:19 +01:00
ieQu1
465e8a90dd
revert(ds): Remove change from the old protocol file
2023-11-08 03:04:19 +01:00
ieQu1
ef46c09caf
feat(ds): Implement ratchet function for bitmask keymapper
2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
903b3863d1
chore(ps_ds): make persistent session module use new `emqx_ds` APIs
2023-11-08 03:04:19 +01:00
ieQu1
2972bf14ee
refactor(ds): Implement create_generation gen_rpc storage layer call
2023-11-08 03:04:19 +01:00
ieQu1
c6a721a7eb
refactor(ds): Passthrough open_db and get_channels to storage layer
2023-11-08 03:04:19 +01:00
ieQu1
59d01dc823
refactor(ds): Implement emqx_ds:open_db
2023-11-08 03:04:19 +01:00
ieQu1
7095cb8583
refactor(ds): Refactor storage layer
2023-11-08 03:04:19 +01:00
ieQu1
c91df2f5cd
refactor(ds): Create a prototype of replication layer
2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
7ab57824dc
chore(ds): change return type of `storage_layer:next/{1,2}`
...
Part of https://emqx.atlassian.net/browse/EMQX-10942
The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.
From:
```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```
To:
```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.
-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
2023-11-08 03:04:19 +01:00
Ilya Averyanov
75704513d8
fix(plugins): fix backward compatibility
2023-11-07 19:06:37 +03:00
Thales Macedo Garitezi
c4c9d7ceb3
chore: prepara to tag `e5.3.1-alpha.4`
2023-11-07 13:04:46 -03:00
Zaiming (Stone) Shi
6af6309301
chore: upgrade to hocon 0.39.19
2023-11-06 19:16:19 +01:00
Zaiming (Stone) Shi
afe6f79a66
feat(bridge): add description field to bridge and connector
2023-11-03 06:53:42 +01:00
Andrew Mayorov
910e81bc41
Merge pull request #10442 from keynslug/ft/EMQX-9257/placeholder
...
feat(tpl): split `emqx_placeholder` into a couple of modules
2023-11-02 22:50:05 +07:00
Andrew Mayorov
0538a77700
feat(tpl): use `emqx_connector_template` in `emqx_authn`, `emqx_authz`
...
This slightly changes semantics: now the attempt to create authenticator
with illegal bindings in templates will fail, instead of treating them
as literals. The runtime behaviour on the other hand should be the same.
2023-11-02 17:11:10 +07:00
Zaiming (Stone) Shi
73dd2f0ffd
chore: bump version to e5.3.1-alpha.3
2023-11-02 10:36:41 +01:00
Andrew Mayorov
7092c75597
Merge pull request #11809 from keynslug/ft/EMQX-10808/file-secrets
...
feat(mqttbridge): support file-sourced secrets as passwords
2023-11-01 00:29:30 +07:00
Thales Macedo Garitezi
b420b53075
fix(bridges_v2): check bridge name length before attempting atom conversion
...
Fixes https://emqx.atlassian.net/browse/EMQX-11289
2023-10-31 13:21:26 -03:00
Andrew Mayorov
d278486416
fix(secret): dedicate a specific loader module for file secrets
...
To make code employing `emqx_secret` easier to follow.
2023-10-31 19:26:04 +07:00
Ivan Dyachkov
b1ab213081
chore: merge 'upstream/release-53' into 1031-sync-r53
2023-10-31 11:06:25 +01:00
JianBo He
21e4f918aa
Merge pull request #11833 from JimMoen/fix-shared-sub-topic-or-subscription-searching
...
Fix shared sub topic or subscription searching
2023-10-31 08:16:36 +08:00
Thales Macedo Garitezi
a60b96c5fd
test(kafka_producer): minor adjustments to test suite
...
- Use `emqx_cth_suite`.
- Use `query_mode` matrix value when setting up bridge in a couple test cases.
2023-10-30 12:13:40 -03:00
Zaiming (Stone) Shi
f463eff02d
chore: e5.3.1-alpha.2
2023-10-30 14:49:08 +01:00
Zaiming (Stone) Shi
0656b6be3c
refactor(emqx_config): use dynamic callback to upgrade raw config
2023-10-30 14:49:08 +01:00
Kjell Winblad
9dc3a169b3
feat: split bridges into a connector part and a bridge part
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.
Issues:
https://emqx.atlassian.net/browse/EMQX-10805
2023-10-30 14:48:47 +01:00
zhongwencool
e896957803
Merge pull request #11824 from zhongwencool/audit-log-reviewed
...
refactor: audit log review refactor
2023-10-30 17:51:15 +08:00
JimMoen
814e22feb3
fix: topics/subscripton mgmt api searching
2023-10-30 14:42:47 +08:00
zhongwencool
0634ff61c0
fix: dialyzer warning
2023-10-30 09:57:30 +08:00
JimMoen
a2015f37ae
Merge pull request #10976 from JimMoen/fix-shared-sub-unsub
2023-10-27 21:22:27 +08:00
JimMoen
d563121284
refactor: move ?REDISPATCH_TO macro to emqx_mqtt.hrl
2023-10-27 17:54:14 +08:00
JimMoen
3b5cc912e7
fix: add `redispatch_to` header to all msgs when deliver shared-sub
...
- to find correct SubOpts for shared-sub dispatch
- use previous key `redispatch_to` to ensure rolling upgrade compatibility
2023-10-27 16:15:23 +08:00
zhongwencool
995948f0e8
refactor: remove seq from audit record
2023-10-27 14:07:19 +08:00
JimMoen
afec6fa2f6
fix: TopicFilter may modified by `client.subscribe` hook
2023-10-27 09:23:44 +08:00
JimMoen
802a36c670
fix: find SubOpts by shared_record, not deliver topic
2023-10-27 09:23:44 +08:00
JimMoen
0ca725ff25
fix: deprecated `share` in `subopts`
2023-10-27 09:23:43 +08:00
JimMoen
b5411da770
refactor: subscribe process to fix shared-sub
2023-10-27 09:23:43 +08:00
Andrew Mayorov
44b4205561
fix(secret): do not treat missing file secrets as config error
...
They are intended to be used mostly in the context of resources, which
have their own feedback mechanism: statuses, retries, etc.
Also turn the error into a throw exception, so that it can be
interpreted as a regular error condition, for example by the resource
manager.
2023-10-26 14:37:14 +07:00
Zaiming (Stone) Shi
76a8b0efd4
test: fix a flaky test case emqx_crl_cache_SUITE:t_revoked
2023-10-25 16:08:31 +02:00
firest
7a8a5926ab
fix(rbac): adjust the role names
2023-10-24 19:01:12 +08:00
Andrew Mayorov
1c2f9321d1
feat(emqx): add file-sourced generic secrets
...
These secrets follow the same `emqx_secret` convention of 0-arity
functions. Also provide a simple HOCON schema module for use in
application schemas.
2023-10-24 16:24:51 +07:00
Ilya Averyanov
edde661da3
fix(authn): fix pbkdf2 option validation
2023-10-23 10:26:11 +03:00
William Yang
a9b42af8b3
Merge pull request #11796 from qzhuyan/fix/william/gen_rpc-driver
...
fix(gen_rpc): ensure client/server use same driver
2023-10-23 08:24:17 +02:00
JimMoen
d467289bb2
fix: avoid duplicated apikey from data import
2023-10-20 21:13:04 +08:00
William Yang
dd5807b797
fix(gen_rpc): ensure client/server use same driver
...
Also bump to gen_rpc 3.2.1
2023-10-20 12:17:49 +02:00
zhongwencool
ef692596f7
fix: don't crash when 401 and 403 unauthorize
2023-10-20 15:22:30 +08:00
JimMoen
f381961108
fix: macro EMQX_RELEASE_EDITION when `emqx` run as standalnoe app
2023-10-20 14:47:24 +08:00
zhongwencool
befc484544
chore: add changelog entry and fix xref warning
2023-10-20 14:47:24 +08:00
zhongwencool
141061c1e2
feat: add max_filter_size and ignore_high_frequency_request config
2023-10-20 14:47:24 +08:00
zhongwencool
926c804314
feat: add /audit http api to filter audit log
2023-10-20 14:47:24 +08:00
lafirest
74442b0d31
Merge pull request #11766 from lafirest/feat/rbac
...
feat(api_key): add RBAC feature for the API key
2023-10-20 09:24:41 +08:00
firest
26ec860d96
feat(backup): add migration mechanism when import backup data
2023-10-17 16:23:32 +08:00
firest
db3915d472
fix(rbac): update changes && fix CI errors
2023-10-17 13:44:37 +08:00
firest
e095de7367
feat(api_key): add RBAC feature for the API key
2023-10-16 14:10:01 +08:00
Zaiming (Stone) Shi
6eb3bb7cff
Merge remote-tracking branch 'origin/release-53' into 1114-sync-release-53
2023-10-14 10:16:38 +02:00
Andrew Mayorov
083e2da347
chore(bpapicheck): make some failures more user-friendly
2023-10-13 15:42:08 +07:00
Andrew Mayorov
0b9ac24c1e
fix(bpapicheck): stop depending on git index
2023-10-13 15:40:49 +07:00
William Yang
a69c53455f
fix: bump to quicer 0.0.202
...
bring in urgent fixs from msquic 2.2.3
2023-10-11 13:12:32 +02:00
Zaiming (Stone) Shi
a2e86c67db
Merge pull request #11733 from keynslug/fix/EMQX-10827/sesson-takeover
...
fix(cm): bring back pre-v5.3.0 compat in `takeover_session_begin/1`
2023-10-10 19:03:29 +02:00
Ivan Dyachkov
bfb2218392
chore: fix release version
2023-10-10 14:50:49 +02:00
Zaiming (Stone) Shi
143673b703
chore: bump release version to 5.3.1-alpha.1
2023-10-10 14:35:46 +02:00
Andrew Mayorov
522302fee1
fix(cm): bring back pre-v5.3.0 compat in `takeover_session_begin/1`
...
Which was accidentally broken in bf164175
.
2023-10-10 01:34:10 +07:00
Zaiming (Stone) Shi
1e93d2f1fc
chore: upgrade to ekka 0.15.16 gen_rpc 3.2.0
...
* ekka 0.15.16 supports 'inet6_tls' as ekka.proto_dist
* gen_rpc 3.2.0 supports true | false as gen_rpc.ipv6_only
2023-10-09 20:03:21 +02:00
Ilya Averyanov
5dff36474d
chore(auth): get rid of hardcoded schema modules in auth
2023-10-05 13:41:50 +03:00
Ilya Averyanov
c2c56ba481
chore(auth): update tests
2023-10-05 13:41:50 +03:00
Ilya Averyanov
1eb75b43c4
chore(auth): split emqx_authn and emqx_authz apps
2023-10-05 13:41:50 +03:00
Zaiming (Stone) Shi
ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53
...
0928 sync release 53
2023-10-02 21:57:32 +02:00
Thales Macedo Garitezi
2358d67908
refactor: move session stuff from `emqx_ds` to `emqx_persistent_session_ds`
...
Part of https://emqx.atlassian.net/browse/EMQX-10942
2023-09-29 18:00:24 -03:00
Zaiming (Stone) Shi
bce8fd2fbc
chore: bump app versions
2023-09-29 18:56:52 +02:00
Zaiming (Stone) Shi
ed5b456d62
Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53
2023-09-29 18:24:44 +02:00
Zaiming (Stone) Shi
a852400fb3
chore: add 5.3.bpapi
2023-09-29 16:34:04 +02:00
Zaiming (Stone) Shi
fe01aaff4f
chore: prepare for e5.3.0 promotion
2023-09-29 13:32:57 +02:00
Zaiming (Stone) Shi
ce5bd0a3ce
Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53
2023-09-29 11:36:32 +02:00
Zaiming (Stone) Shi
c64e599e81
docs: document how to retrieve peercert
2023-09-29 10:33:57 +02:00
Zaiming (Stone) Shi
1177a32310
chore: bump version to 5.3.0-rc.2
2023-09-29 09:49:27 +02:00
Zaiming (Stone) Shi
b267fc2588
chore: bump release version to 5.3.0
2023-09-28 21:22:33 +02:00
Zaiming (Stone) Shi
1a13b2ac56
Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53
2023-09-28 13:53:03 +02:00
Zaiming (Stone) Shi
7ddcd71412
docs(ssl_dist.conf): update doc link
2023-09-28 11:16:18 +02:00
ieQu1
b77e5e880a
Merge pull request #11697 from ieQu1/dev/gen-rpc-3.1.1
...
chore(gen_rpc): Bump version to 3.1.1
2023-09-28 10:03:36 +02:00
JianBo He
6d2adfc259
chore(audit): emit the message field
2023-09-28 11:04:42 +08:00
JianBo He
859b122cdd
chore: format codes
2023-09-28 10:25:37 +08:00
JianBo He
dd9938114c
chore(audit): add from field
2023-09-28 10:25:37 +08:00
lafirest
b0d86eecd6
Merge pull request #11691 from lafirest/fix/sso_ssl
...
fix(sso): support for SSL update && ensure update is atomic
2023-09-28 10:22:34 +08:00
ieQu1
0aa3ccdd65
feat(gen_rpc): Add schema for the TLS versions and cipher suites
2023-09-28 00:36:59 +02:00
Zaiming (Stone) Shi
afdda107af
fix(logger): json format log encode binary list as string array
2023-09-27 23:40:01 +02:00
Zaiming (Stone) Shi
6f7a4344dc
fix: do not gc sso saml SP singing keys
2023-09-27 23:19:39 +02:00
Zaiming (Stone) Shi
c8cbbff044
fix(logger): no need for special handling of empty string
...
when formating json logs, there is no need to handle empty strings
special, already covered by unicode handling
2023-09-27 21:26:47 +02:00
ieQu1
3c37f19105
chore(gen_rpc): Bump version to 3.1.1
2023-09-27 21:09:00 +02:00
Zaiming (Stone) Shi
6e8c73258f
test: refactor emqx_rpc unit tests
2023-09-27 12:58:06 +02:00
Andrew Mayorov
1d0e789e4d
Merge pull request #11679 from keynslug/ft/EMQX-10942/custom-session-timers
...
feat(session): add custom session timers mechanism
2023-09-26 19:09:53 +03:00
Thales Macedo Garitezi
aae59f1efd
Merge pull request #11682 from thalesmg/fix-file-logger-type-r53-20230925
...
fix(file_logger): change file logger type depending on rotation size
2023-09-26 10:22:36 -03:00
Andrew Mayorov
bce2142df8
test(session): add custom session timers testcase
2023-09-26 16:22:06 +03:00
Zaiming (Stone) Shi
4e15edb5e4
Merge pull request #11661 from zmstone/0922-fix-json-log-formatter
...
0922 fix json log formatter
2023-09-26 14:13:41 +02:00
Thales Macedo Garitezi
fdcd73e20c
fix(file_logger): change file logger type depending on rotation size
...
Fixes https://emqx.atlassian.net/browse/EMQX-11036
From `logger_disk_log_h:open_disk_log`:
```erlang
open_disk_log(Name,File,Type,MaxNoBytes,MaxNoFiles) ->
case filelib:ensure_dir(File) of
ok ->
Size =
if Type==halt -> MaxNoBytes;
Type==wrap -> {MaxNoBytes,MaxNoFiles} %% <-------
end,
Opts = [{name, Name},
{file, File},
{size, Size},
{type, Type},
{linkto, self()},
{repair, false},
{format, external},
{notify, true},
{quiet, true},
{mode, read_write}],
case disk_log:open(Opts) of
```
Affects all file loggers (audit included):
```
% emqx_config_logger:update_log_handler/1 -> ok
iex(emqx@127.0.0.1)14> Config override: log.file.emqx_audit is updated, but failed to add handler: {handler_not_added,
{badarg,
[{size,
{infinity,
10}},
{type,
wrap},
{linkto,
<0.1952.0>},
{repair,
false},
{format,
external},
{notify,
true},
{quiet,
true},
{mode,
read_write}]}}
```
2023-09-26 09:04:44 -03:00
Ilya Averyanov
14983ec14a
chore(hooks): validate hookpoints and document hook callbacks
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-26 14:27:42 +03:00
Thales Macedo Garitezi
ff7f37ccf5
test(cth): allow defining schema to load for app
2023-09-25 13:22:41 -03:00
Andrew Mayorov
b1f144ab8b
feat(session): add custom session timers mechanism
...
That are managed exclusively by the session implementation, unlike
common session timers that are managed by the channel itself.
2023-09-25 18:19:26 +03:00
lafirest
1df8326fb8
Merge pull request #11677 from lafirest/fix/sso_cfg
...
adjust the config path for the SSO feature and improve the update logic
2023-09-25 23:00:46 +08:00
Zaiming (Stone) Shi
5e6996dc05
refactor: log formatter format mfa+line as m:f/a(line)
...
also improve json formatter when a field is iolist
2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi
6a557980e4
refactor(logger): ensure JSON log field order
...
Ensure that the log fileds in JSON format are ordered as
time, level, msg, mfal, ...
2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi
5f45ba50ff
refactor: delete log formatter config for audit log handler
...
It is using 'json' formatter, the template is useless
2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi
1fed38c248
fix(logger): write 'json' format logs as JSON
2023-09-25 16:55:22 +02:00
William Yang
9106eb92d2
Merge pull request #11675 from qzhuyan/fix/william/quic-respect-param-verify
2023-09-25 15:48:01 +02:00
firest
cfdb25b213
fix(sso): updates the SSO backend when the `[dashboard]` has updated
2023-09-25 20:30:47 +08:00
William Yang
36d3a3a524
fix: bump to quicer 0.0.201
...
do not load cacertfile if verify_none
2023-09-25 13:02:01 +02:00
William Yang
22193d273a
fix(quic): ignore undefined cacertfile
2023-09-25 12:57:55 +02:00
Ivan Dyachkov
8c93c79b44
chore: e5.3.0-alpha.2
2023-09-25 12:52:45 +02:00
Ivan Dyachkov
e1f1c64ecd
Merge branch 'release-53' into 0925-sync-r53-to-master
2023-09-25 08:12:45 +02:00
JianBo He
df94426ee3
chore: make static_check happy
2023-09-22 21:50:42 +08:00
Thales Macedo Garitezi
d6935b6a67
feat: add port scan diagnostics to mria waiting for tables checks
...
Fixes https://emqx.atlassian.net/browse/EMQX-10944
Also updates ekka -> 0.15.15, mria -> 0.6.4
How to test
===========
1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop them in order.
3. Start only the first node that was stopped in the previous step.
4. Wait until the log is printed.
Or, more easily:
1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop all but one.
3. Run `mria_mnesia:diagnosis([]).` on that node.
Example output
==============
```
Check check_open_ports should get ok but got #{msg =>
"some ports are unreachable",
results =>
#{'emqx@172.100.239.4' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
4}],
status =>
bad_ports},
'emqx@172.100.239.5' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
5}],
status =>
bad_ports}}}
```
After one node is back:
```
Check check_open_ports should get ok but got #{msg =>
"some ports are unreachable",
results =>
#{'emqx@172.100.239.4' =>
#{ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
4}],
status => ok},
'emqx@172.100.239.5' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
5}],
status =>
bad_ports}}}
```
2023-09-21 14:29:01 -03:00
Ivan Dyachkov
7cf60c5a91
chore: e5.3.0-alpha.1
2023-09-21 19:29:00 +02:00
Andrew Mayorov
acf4227fc6
test(session): fix quic testgroup in persistent session suite
...
Which broker after quicer 0.0.200 upgrade.
2023-09-21 20:52:04 +04:00
Andrew Mayorov
b563e30615
Merge pull request #11650 from keynslug/fix/simplify-takeover-suite
...
test(session): make testsuite trigger takeover logic consistently
2023-09-21 18:02:38 +04:00
zhongwencool
beea1be9f0
Merge pull request #11599 from zhongwencool/audit-log
2023-09-21 18:39:23 +08:00
Ivan Dyachkov
0a61d08a0f
Merge pull request #11651 from id/0921-sync-r52-to-master
...
sync r52 to master
2023-09-21 12:31:24 +02:00
William Yang
3a5227198e
Merge pull request #11642 from qzhuyan/dev/william/quicer-0.0.200
...
quicer 0.0.200
2023-09-21 12:20:14 +02:00
Ivan Dyachkov
dafd7c6085
chore: bump apps versions
2023-09-21 10:58:42 +02:00
zhongwencool
21bb209fb1
feat: added support for auditing API and CLI activity in logs
2023-09-21 16:35:46 +08:00
Ivan Dyachkov
105bebc250
chore: merge release-52 into master
2023-09-21 10:22:47 +02:00
Andrew Mayorov
a8f4b5bf86
test(session): make testsuite trigger takeover logic consistently
2023-09-21 11:53:40 +04:00
zhongwencool
1b29e4b189
Merge pull request #11641 from zhongwencool/slog-msg-unify
2023-09-21 07:32:39 +08:00
Andrew Mayorov
a2ddd9d5f5
fix(session): respect existing session even if expiry interval = 0
...
If the original connection had Session-Expiry-Interval > 0, and the
new connection set Session-Expiry-Interval = 0, the MQTTv5 spec says
that (supposedly) we still have to continue with the existing session
(if it hasn't expired yet).
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:54 +04:00
Andrew Mayorov
3945f08f8f
fix(sessds): try to ensure iterators are closed on destroy
2023-09-20 22:55:54 +04:00
Andrew Mayorov
21e82b9534
test(sessmem): make retry delivery testcase more involved
2023-09-20 22:55:54 +04:00
Andrew Mayorov
69889d14a3
fix(sessds): fix use of undefined types
2023-09-20 22:55:54 +04:00
Andrew Mayorov
9362ef6f73
test(sessmem): drop unnecessary nesting in testcase
...
Also get rid of sneaky binding assignment.
2023-09-20 22:55:25 +04:00
Andrew Mayorov
98706cd215
chore: ensure comments follow code style consistently
2023-09-20 22:55:25 +04:00
Andrew Mayorov
8670efbfa0
chore(chan): rename `Name` → `TimerName` for better readability
2023-09-20 22:55:25 +04:00
Andrew Mayorov
7a9916c84d
fix(sessds): convert ds iterator topics upon opening ds session
2023-09-20 22:55:25 +04:00
Andrew Mayorov
540ca6d60f
chore: drop few commented out and irrelevant pieces
2023-09-20 22:55:24 +04:00
Andrew Mayorov
9d145890cc
refactor(sessmem): pass log context as part of session event
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
Andrew Mayorov
adc29e15cc
refactor(session): make typespecsa and flow a bit more clear
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
Andrew Mayorov
e422f492ef
test(sessds): reuse and expand persistent session test suite
2023-09-20 22:55:24 +04:00
Andrew Mayorov
3383ae19a9
test(session): switch `emqx_persistent_session_SUITE` to cth tooling
2023-09-20 22:55:23 +04:00
Andrew Mayorov
7326ef550b
fix(sessds): make existing parts of persistent session impl work
2023-09-20 22:55:22 +04:00
Andrew Mayorov
e713fc38aa
feat(broker): reflect persisted messages in publish result
...
In order for callers to distinguish between silently dropped and
durably persisted message w/o matching subscribers.
2023-09-20 22:53:45 +04:00
Andrew Mayorov
f4953e719b
fix(cmproto): fix few typespecs
2023-09-20 22:53:45 +04:00
Andrew Mayorov
8af107e28d
test(ds): simplify cluster test setups
2023-09-20 22:53:45 +04:00
Andrew Mayorov
7c4f68dd3d
fix(session): make utility function names consistent
...
Before this commit behavior of `is_banned_msg/1` / `should_discard/1`
were actually the exact opposite of their names.
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:53:44 +04:00
Andrew Mayorov
2dae8020ec
refactor(cm): avoid deep indirection in `emqx_session_mem`
2023-09-20 22:53:44 +04:00
Andrew Mayorov
abeff0bc4f
chore(session): try to describe what happens after session takeover
2023-09-20 22:53:44 +04:00
Andrew Mayorov
45d44df11d
refactor(session): update eviction channel session logic
...
The changes partially reflect `emqx_channel` changes with respect to
in-memory session specific logic. The difference is that eviction
channel does not replay post-takeover, instead enqueues messages.
2023-09-20 22:53:44 +04:00
Andrew Mayorov
ab1c4c4222
refactor(session): drop `lookup/1` session API
...
Due to the fact it's not used for anything right now.
2023-09-20 22:53:44 +04:00
Andrew Mayorov
97881ff3ca
refactor(session): bring back common session timers
2023-09-20 22:53:43 +04:00
Andrew Mayorov
bf16417513
feat(session): introduce session implementation concept
2023-09-20 22:53:42 +04:00
Andrew Mayorov
04731b7ef7
test(takeover): randomize messages for random natural ordering
2023-09-20 22:52:14 +04:00
Andrew Mayorov
f022c9b1a4
feat(emqx): add `emqx_inflight:fold/3` generic function
2023-09-20 22:52:14 +04:00
Andrew Mayorov
e4866adc2f
refactor(chan): make timer names equal to messages they send
...
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
Andrew Mayorov
cfb1bf1fa4
chore(emqx): drop remnants of former session persistence impl
...
1. It is not functional anyway.
2. It blocks `emqx_session` refactoring in a few places.
2023-09-20 22:52:13 +04:00
William Yang
e43b87f18e
test(quic): update emqx_broker_SUITE
2023-09-20 20:20:18 +02:00
Thales Macedo Garitezi
2965fa6fcb
Merge pull request #11613 from thalesmg/ds-keyspace-m-20230914
...
feat(ds): introduce keyspace concept
2023-09-20 14:21:27 -03:00
William Yang
da626f6d29
feat(quic): bump to quicer 0.0.200 and emqtt 1.9.0
2023-09-20 16:43:15 +02:00
zhongwencool
f789a7504f
chore: update apps/emqx/src/emqx_quic_connection.erl
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:00:44 +08:00
Ivan Dyachkov
09aea47f40
chore: 5.2.1
2023-09-20 14:04:28 +02:00
zhongwencool
2f1fa2e961
chore: unified slog message formatting to improve logging consistency
2023-09-20 18:13:00 +08:00
Thales Macedo Garitezi
5ed5ac48ee
refactor: combine shard id and keyspace into a single value
2023-09-19 12:31:32 -03:00
Zaiming (Stone) Shi
65818740f3
Merge pull request #11622 from zmstone/0917-upgrade-gen_rpc-to-3.1.0
...
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-19 10:22:53 +02:00
Ivan Dyachkov
a69c26b86d
chore: e5.2.1-alpha.3
2023-09-19 08:17:29 +02:00
JianBo He
de1bc04843
chore: bump ekka to 0.15.14, mria to 0.6.3
...
Inclued ekka's updates:
- https://github.com/emqx/ekka/pull/214
- https://github.com/emqx/ekka/pull/215
- https://github.com/emqx/ekka/pull/216
- https://github.com/emqx/ekka/pull/217
mria's updates:
- https://github.com/emqx/mria/pull/160
- https://github.com/emqx/mria/pull/161
- https://github.com/emqx/mria/pull/162
2023-09-19 09:43:19 +08:00
Thales Macedo Garitezi
a511088fd4
refactor: address review comments
2023-09-18 17:47:56 -03:00
Zaiming (Stone) Shi
de93d71014
test: fix emqx_sahred_sub_SUITE:t_remote
...
prior to this fix, the peer ct node is actually node itself
2023-09-18 21:54:43 +02:00
Zaiming (Stone) Shi
99b60849b4
test: fix flaky test emqx_falpping_SUITE:t_conf_update_timer
2023-09-18 20:44:38 +02:00
Zaiming (Stone) Shi
d1e5b69eac
test: increase wait time
2023-09-18 17:33:09 +02:00
Zaiming (Stone) Shi
4f1ec795eb
test: delete special handling of emqtt race condition
...
the race should have been fixed in emqtt 1.8.7
2023-09-18 17:27:34 +02:00
Zaiming (Stone) Shi
46557c3462
fix(emqtt): upgrade from 1.8.6 to 1.8.7
2023-09-18 17:04:23 +02:00
Zaiming (Stone) Shi
f66d9e76fe
fix(emqx_channel): aovid logging frame errors as exceptions
2023-09-18 09:04:00 +02:00
Zaiming (Stone) Shi
abbee4bc47
fix(esockd): upgrade esockd from 5.9.6 to 5.9.7
2023-09-17 19:44:04 +02:00
Zaiming (Stone) Shi
985f0e6994
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-17 19:10:13 +02:00
Thales Macedo Garitezi
2aae3856a3
Merge pull request #11596 from thalesmg/fix-typespec-typos-m-20230912
...
fix: fix typespec typos
2023-09-15 09:24:42 -03:00
Ilya Averyanov
1959e15703
chore: simplify session eviction for node rebalance
2023-09-15 11:46:51 +02:00
Thales Macedo Garitezi
b30bcf32bd
feat(ds): introduce keyspace concept
...
Fixes https://emqx.atlassian.net/browse/EMQX-10579
This introduces the concept of "keyspaces" to our durable storage (DS) implementation, and
also refactors some places where "shard" and "keyspace" would be mixed up.
We might want to tune the storage options differently for distinct sets of topics, the
keyspaces. The keyspace is composed by one or more shards.
- Keyspaces are identified simply by binary strings.
- DS configuration is scoped by keyspaces instead of shards.
- Starting a new DS shard requires definining to which keyspace the shard belongs.
2023-09-14 16:39:00 -03:00
Ilya Averyanov
11c39c4b6a
chore: simplify session eviction for node rebalance
2023-09-14 18:28:25 +03:00
Thales Macedo Garitezi
6be0e14e8c
fix: fix typespec typos
2023-09-14 10:21:05 -03:00
Thales Macedo Garitezi
e41f7dd68c
Merge pull request #11598 from thalesmg/ds-fix-ps-router-m-20230912
...
fixes and improvements to persistent session router
2023-09-14 10:00:58 -03:00
firest
3eb46be2ae
fix(banned): fix typos
2023-09-14 14:01:59 +08:00
Thales Macedo Garitezi
803b69d878
refactor: use more consistent return types
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2023-09-13 10:49:45 -03:00
Thales Macedo Garitezi
f55f6dc779
docs(ps_session): add notes explaining rationale about operation order and consistency
2023-09-13 10:12:27 -03:00
Thales Macedo Garitezi
e081abb9e6
perf(ps_router): only check if there are any routes when deciding whether to persist message
2023-09-13 10:02:39 -03:00
Thales Macedo Garitezi
cae9ae1fab
fix(ps_router): use `disc_copies` for storing persistent session routes
2023-09-13 10:02:39 -03:00
William Yang
ea73d674b9
chore: bump version to e5.2.1-alpha.2
2023-09-13 14:55:17 +02:00
Serge Tupchii
af015bc2bb
chore(ekka): bump version to 0.15.13
2023-09-13 10:11:14 +03:00
William Yang
6ef296268e
chore: bump version to e5.2.1-alpha.1
2023-09-12 15:24:02 +02:00
Thales Macedo Garitezi
59b94788bf
refactor: address review comments
2023-09-11 16:25:34 -03:00
Thales Macedo Garitezi
64fcdb671d
test: fix a bunch of inter-suite interferences
2023-09-11 16:25:34 -03:00
Thales Macedo Garitezi
f1294736b7
feat(ds): add filter for message persistence
...
Fixes https://emqx.atlassian.net/browse/EMQX-10520
2023-09-11 16:25:34 -03:00
Thales Macedo Garitezi
7805cc8c9b
fix(session): check if session is persistent
2023-09-11 16:25:33 -03:00
Thales Macedo Garitezi
f490da71cd
chore: prepare v5.2.0-build.1
2023-09-11 16:09:17 -03:00
Thales Macedo Garitezi
d0f1c75651
chore: bump apps vsns
2023-09-11 16:06:53 -03:00
Thales Macedo Garitezi
223d47a6ac
Merge branch 'master' into sync-m-r52-20230911
2023-09-11 16:04:28 -03:00
Thales Macedo Garitezi
063913f245
fix(session): remove recently added `iterators` field from `#session{}` record
...
Fixes https://emqx.atlassian.net/browse/EMQX-10945
This could lead to `badrecord` errors being raised if a takeover were to happen during a
rolling cluster upgrade, as the old nodes could receive a record with more fields than
expected.
2023-09-11 14:46:12 -03:00
Zaiming (Stone) Shi
7010ac358c
chore: bump release version to v5.2.0
2023-09-08 16:31:52 +02:00
Andrew Mayorov
3d133801af
fix(router): use `ets:new/2` options compatible with OTP 24
...
While essentially keeping the same effect.
2023-09-08 18:04:29 +04:00
Zaiming (Stone) Shi
f3bebb0ba2
Merge remote-tracking branch 'origin/master' into 0907-sync-master-to-release-52
2023-09-07 18:51:30 +02:00
Zaiming (Stone) Shi
db2ec55aa0
chore: bump release version to official e5.2.0
2023-09-07 09:40:15 +02:00
Zaiming (Stone) Shi
7c2f87fabe
test: merge broker and router boot modules
2023-09-06 21:36:16 +02:00
Zaiming (Stone) Shi
e794143ae1
Merge remote-tracking branch 'origin/release-52' into 0906-sync-release-52-to-master
2023-09-06 09:08:22 +02:00
Thales Macedo Garitezi
954b73a9aa
Merge pull request #11533 from thalesmg/ds-unsubscribe-m-20230828
...
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
2023-09-05 09:47:17 -03:00
ieQu1
01a128878f
chore(mria): Bump mria to 0.6.1
2023-09-05 11:08:32 +02:00
Andrew Mayorov
d26995a2aa
Merge pull request #11524 from keynslug/ft/EMQX-10713/unified-route-tab
...
feat(router): add routing storage schema with 2 tables
2023-09-05 12:38:00 +04:00
Thales Macedo Garitezi
23d63f5e01
refactor: fork clientid type for `emqx_ds:session_id/0`
2023-09-04 15:38:25 -03:00
Thales Macedo Garitezi
60ae3c15c8
refactor: disregard impossible case
2023-09-04 15:38:25 -03:00
Thales Macedo Garitezi
73d4248334
test(cth_cluster): always allocate listener ports
2023-09-04 11:22:26 -03:00
Andrew Mayorov
9f0f218387
test(router): simplify cluster-related testcase in helper testsuite
2023-09-04 18:05:59 +04:00
Andrew Mayorov
7d26b7bc1a
fix(router): emit clearer error when conflicting schemas in use
2023-09-04 16:37:38 +04:00
Thales Macedo Garitezi
d33ad8a46f
feat: cluster purge (r5.2)
...
Port of https://github.com/emqx/emqx/pull/11447
targeting `release-52`
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-09-04 09:15:28 -03:00
Andrew Mayorov
893f69617a
chore(test): drop obsolete TODOs
2023-09-04 14:40:49 +04:00
Andrew Mayorov
f0a0c7d4b1
fix(cthsuite): avoid duplicate application env entries
...
Because `application:set_env/1` complains loudly if there are
duplicates.
2023-09-04 14:40:49 +04:00
Andrew Mayorov
5024304bf9
fix(router): wait for tables replicate before choosing schema vsn
2023-09-04 14:40:48 +04:00
Ivan Dyachkov
24230a64df
chore: bump app versions
2023-09-04 11:39:21 +02:00
Ivan Dyachkov
b2f57636bd
Merge remote-tracking branch 'upstream/release-51' into 0904-sync-release-51
2023-09-04 11:34:22 +02:00
Serge Tupchii
f0c75d97e1
chore: upgrade emqx_http_lib to 0.5.3
...
0.5.3 release includes a fix to parse <host>:<port> URLs using the default http scheme.
2023-09-01 19:56:07 +03:00
Andrew Mayorov
4ab5f8374b
chore(schema): mark routing storage schema readOnly
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-01 19:58:31 +04:00
Andrew Mayorov
ebf13c90ea
test(router): make sort stable in e2e routing testcase
2023-09-01 19:48:56 +04:00
Andrew Mayorov
545f1c84a6
fix(cth): do not allocate ports for `emqx` app by default
...
This causes tricky and impressively hard to track side effects down
the line. Namely, loading `emqx_schema` _after_ `emqx_conf_schema`
(as part of cluster node startup sequence) leads to a couple of
schema root rewrites, because `emqx_schema` defines similar config
roots yet slightly differently (e.g. `authorization`).
2023-09-01 17:23:34 +04:00
Andrew Mayorov
0a879bffd1
test(router): add e2e testcase for cluster-wide routing
...
Which verifies that both router storage schemas work correctly.
2023-09-01 13:49:12 +04:00
Andrew Mayorov
eb0385a28f
chore(router): separate test-only function exports
2023-09-01 13:49:11 +04:00
Zaiming (Stone) Shi
c20ba0572a
chore: bump to 5.2.0-alpha.4
2023-08-31 15:47:39 +02:00
Zaiming (Stone) Shi
876d539336
Merge remote-tracking branch 'andrew/ft/EMQX-10713/unified-route-tab' into 0831-make-use-of-new-routing-table
2023-08-31 15:46:33 +02:00
Andrew Mayorov
fb094e1d47
test(router): avoid testsuite collisions
2023-08-31 13:36:01 +04:00
Thales Macedo Garitezi
4dfbc859f9
fix(plugins): update plugin order on whole cluster
...
Fixes https://emqx.atlassian.net/browse/EMQX-10879
2023-08-30 16:39:52 -03:00
Andrew Mayorov
6b9cb06334
fix(router): add / refine some comments and log messages
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-30 19:51:31 +04:00
Thales Macedo Garitezi
922ca5e141
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
...
Fixes https://emqx.atlassian.net/browse/EMQX-9742
2023-08-30 11:14:28 -03:00
Thales Macedo Garitezi
9c6dd30f44
feat(session): store iterator ids in session record
2023-08-30 11:13:55 -03:00
Andrew Mayorov
f8923ab419
Merge pull request #11539 from keynslug/ft/cth-desc-workdir
...
feat(cthsuite): add function to determine workdir of testrun
2023-08-30 17:15:55 +04:00
lafirest
8b3a9072f9
Merge pull request #11532 from lafirest/fix/frame_parse
...
fix(frame): improve some error reasons when parsing invalid packet
2023-08-30 19:10:45 +08:00
firest
804443ba40
chore: fix typos
2023-08-30 11:22:09 +08:00
Ilya Averyanov
2ef60db147
Merge pull request #11481 from savonarola/0817-simplify-async-responses
...
chore(ft): refactor async reply mechanism
2023-08-30 01:20:09 +03:00
Ilya Averyanov
54ac4a8527
chore(ft): tidy up the code according to the review
2023-08-29 23:21:36 +03:00
Andrew Mayorov
0e770bdc95
test: switch `emqx_broker_SUITE` to use new cth tooling
2023-08-29 22:05:36 +04:00
Andrew Mayorov
3268093881
feat(cth): add module-level documenation
2023-08-29 22:05:36 +04:00
Andrew Mayorov
f57d16ba13
feat(cthsuite): add function to determine workdir of testrun
...
In a deterministic fashion, to lift the burden of undestanding where
the testrun's data should go from the test writer.
2023-08-29 21:34:01 +04:00
Andrew Mayorov
063d6200c8
feat(router): enable using 2 tables to store routes
...
Instead of a single unified table, to reap the benefits of cheap
`ets:lookup/2` per regular topic subscription route.
Change configuration option naming to reflect the change: user now has
an ability to choose _storage schema_.
2023-08-29 17:41:59 +04:00
Andrew Mayorov
270fd107b2
feat(triesearch): allow reusing filter tokenization result
...
As a kind of micro-optimization.
2023-08-29 17:31:26 +04:00
zhongwencool
4e7ba5d35f
chore: ciphers should allow empty space(comma_separated_binary)
2023-08-29 11:34:40 +08:00
firest
6722722522
fix(frame): improve some error reasons when parsing invalid packet
2023-08-29 10:53:20 +08:00
SergeTupchiy
c04f770118
Merge pull request #11506 from SergeTupchiy/EMQX-10274-do-not-download-empty-trace-log
...
fix(emqx_trace): don't download empty trace log file
2023-08-28 22:17:00 +03:00
Serge Tupchii
0072749143
fix(emqx_trace): don't download empty trace log file
...
Closes: EMQX-10274
2023-08-28 19:25:55 +03:00
Thales Macedo Garitezi
f988de4ff4
feat(purge): clear delayed messages
2023-08-28 10:23:21 -03:00
Thales Macedo Garitezi
9bded07834
feat: cluster purge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-08-28 10:23:20 -03:00
Andrew Mayorov
8d2ebdea7e
fix(router): generalize config option and make effects visible
2023-08-28 14:07:46 +04:00
Andrew Mayorov
e85789306b
chore(router): drop test-only `has_routes/1`
...
Seems that it's too much support work only for test purposes, where
`lookup_routes/1` is nearly as usable.
2023-08-28 14:07:46 +04:00
Andrew Mayorov
5d51687dbf
feat(router): add unified routing table config option
2023-08-28 14:07:45 +04:00
Andrew Mayorov
33e5e1ba57
feat(router): add unified routing table
2023-08-28 14:07:44 +04:00
Andrew Mayorov
166375a000
fix(topicidx): make `get_record/2` simpler to use in concurrent env
...
The mechanic of `emqx_topic_index` cannot really guarantee atomicity
of reading records associated with index matches, so instead it's
probably better to make the user aware of that lack of this guarantee.
2023-08-28 13:52:59 +04:00
Andrew Mayorov
84e40fb6fe
test(trie): add more involved route add + delete case
...
That shows how current local trie implementation breaks because
of lack of refcounting.
2023-08-28 13:52:59 +04:00
Andrew Mayorov
dcb63440bc
refactor(mgmt): avoid dealing with router tab directly
...
Instead, contain all this behind `emqx_router` module interface.
2023-08-28 13:52:59 +04:00
Andrew Mayorov
9eccfa0909
refactor(router): isolate cleanup logic in router module
2023-08-28 13:52:59 +04:00
Ilya Averyanov
39a48179ea
chore(emqx_channel): use macros for reply construction
2023-08-28 11:57:44 +03:00
Ilya Averyanov
b8cacd2833
chore(ft): add tests for async reply registry
2023-08-28 11:57:44 +03:00
Andrew Mayorov
b74ff10705
Merge pull request #11517 from keynslug/ft/topic-index-v3
...
feat(topicidx): iterate on trie search implementation
2023-08-28 12:24:16 +04:00
Ilya Averyanov
b0d4a22aa8
chore(ft): refactor async reply mechanism
2023-08-27 21:54:23 +03:00
Thales Macedo Garitezi
33a0048155
refactor: move logic to `ensure_iterator`
2023-08-25 15:24:53 -03:00
Thales Macedo Garitezi
f15f59650d
test: rm obselete workaround code
2023-08-25 13:49:33 -03:00
Andrew Mayorov
d5cff533e3
chore(topicidx): drop TODO comment
2023-08-25 16:44:03 +04:00
Andrew Mayorov
ca59a87d47
chore(topicidx): refine example of wildcard compare
2023-08-25 16:44:03 +04:00
Zaiming (Stone) Shi
558402a68e
chore(topic_index): add topic validation
2023-08-25 16:44:03 +04:00
Andrew Mayorov
33ed53bb6a
refactor(topicidx): simplify `compare/3` further
...
It's possible to emit seek instruction just once, on the way back
out of the `compare/3` stack.
2023-08-25 16:34:45 +04:00
Andrew Mayorov
fc37d235c7
refactor(topicidx): simplify seek instructions
...
Which also avoids comparing filter tail repeatedly when evaluating
it.
2023-08-25 15:24:06 +04:00
zhongwencool
c8af5a702c
test: remove connection_closed from test
2023-08-25 15:53:35 +08:00
zhongwencool
fc1738188e
fix: packets_connack_sent is not incremented if the ack_flag field in the CONNACK packet is non-zero
2023-08-25 15:18:34 +08:00