Commit Graph

3982 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 839f9dbedb feat(ds): session expiry
Fixes https://emqx.atlassian.net/browse/EMQX-11048
2023-11-24 16:31:54 -03:00
ieQu1 2d4b9a7b9d
Merge pull request #12019 from ieQu1/ds-clean-start
Discard sessions when client connects with clean start = true
2023-11-24 13:35:39 +01:00
ieQu1 e616e0746a feat(ds): Implement a function for dumping persistent session state 2023-11-24 04:39:46 +01:00
ieQu1 8dfcb69e52 feat(ds): Discard session when client connects with CleanStart=1 2023-11-24 04:17:28 +01:00
ieQu1 e8d18b0e09
Merge pull request #12014 from ieQu1/ds-qos0
Support QoS0 messages in persistent_session_ds
2023-11-23 23:43:18 +01:00
ieQu1 a158f25a40 fix(ds): Fix return type of emqx_persistent_session_ds:publish 2023-11-23 22:16:38 +01:00
ieQu1 c5bb86db67 feat(ds): Support QoS 0 2023-11-23 22:16:38 +01:00
ieQu1 1ced8786fd feat(ds): Make session poll interval configurable 2023-11-23 17:10:03 +01:00
ieQu1 82e74d0201 feat(ds): Add a flag that forces all sessions to become durable 2023-11-23 17:10:03 +01:00
Thales Macedo Garitezi 6c9417efe0 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122 2023-11-22 12:02:34 -03:00
Ilya Averyanov a1b9a14fa1 feat(ds): allow fdb implementation for durable storage 2023-11-22 14:49:38 +03:00
ieQu1 ee191803ea
Merge pull request #11990 from ieQu1/dev/ds-shards
Implement a prototype of durable message sharding
2023-11-22 00:57:54 +01:00
Zaiming (Stone) Shi 5ebd954b16 chore: bump release version to e5.3.2-alpha.1 2023-11-21 21:19:05 +01:00
Zaiming (Stone) Shi 8fbdcab118
Merge pull request #11987 from zmstone/1120-do-not-crash-on-einval-after-check-cache
fix(emqx_connection): handle socket activation error return
2023-11-21 20:31:56 +01:00
ieQu1 3165b4f645 fix(ds): Abort application startup when rocksdb is not avialable 2023-11-21 20:27:07 +01:00
ieQu1 4d47490734 chore(ds): Rebase configuration 2023-11-21 19:37:01 +01:00
ieQu1 2a1f7d946a feat(ds): Shard messages by publisher client ID 2023-11-21 19:37:01 +01:00
ieQu1 62542e5844 feat(ds): Metadata storage for the replication layer 2023-11-21 19:37:01 +01:00
Zaiming (Stone) Shi fa91bacdfb
Merge pull request #11975 from zmstone/1119-fix-socket-close-race-condition
fix(emqx_channel): do not log stale sock_close event as error
2023-11-21 16:47:03 +01:00
Thales Macedo Garitezi 068d151b14
Merge pull request #11983 from thalesmg/sync-r53-to-m-20231120
sync `release-53` to `master`
2023-11-21 09:02:51 -03:00
Ilya Averyanov c6fd1e4c75
Merge pull request #11971 from savonarola/1116-no-auth-rebalance-status
feat(rebalance): improve rebalance usability
2023-11-21 12:10:18 +02:00
Andrew Mayorov 110a5a4896
feat(schema): introduce separate root for new session persistence
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
2023-11-20 23:49:14 +07:00
Zaiming (Stone) Shi b02711af79 refactor(emqx_ws_connection): rename cache to buffer for limiter 2023-11-20 17:47:20 +01:00
Zaiming (Stone) Shi ec19247271 refactor: rename limiter buffer related messages and var names 2023-11-20 16:55:26 +01:00
Andrew Mayorov 9684e79ee0
fix(sessds): ensure dup flag is on for replayed messages 2023-11-20 22:54:40 +07:00
Zaiming (Stone) Shi 8ec3b1db5d fix(emqx_connection): handle socket activation error return 2023-11-20 16:51:39 +01:00
Thales Macedo Garitezi 53e796bbd0 Merge remote-tracking branch 'origin/release-53' into sync-r53-to-m-20231120 2023-11-20 11:49:52 -03:00
Andrew Mayorov ef7cfd0202
feat(sessds): add field to `ds_pubrange` for forward-compat 2023-11-20 19:56:55 +07:00
Andrew Mayorov d7974d835f
fix(sharesub): anticipate messages w/o redispatch header
For instance, `emqx_session_mem` will push almost all of the messages
that are still in the state to `emqx_shared_sub:redispatch/1`. After
this commit, the session will no longer crash during channel terminate.
2023-11-20 15:53:24 +07:00
Andrew Mayorov 7081f1951f
refactor(sessds): use `ds_pubrange` record as is
Instead of converting it into almost similar runtime representation.
2023-11-20 15:40:43 +07:00
Andrew Mayorov a5ff4144fe
test(sessds): add complex testcase for session replay 2023-11-20 15:40:43 +07:00
Andrew Mayorov 1246d714c5
feat(sessds): preserve acks / ranges in mnesia for replays 2023-11-20 13:33:48 +07:00
Andrew Mayorov 5b40304d1f
chore(sessds): simplify subscriptions handling
There's currently no point in storing parsed topic filters in the
subscriptions table.
2023-11-20 13:33:48 +07:00
Andrew Mayorov 648b6ac63e
chore(sessds): rename iterators -> subscriptions
Also try to make clearer the difference between 2 flavors of topic
filter representation in use.
2023-11-20 13:33:48 +07:00
Zaiming (Stone) Shi e73bf716ae fix(emqx_channel): do not log stale sock_close event as error
In some cases, EMQX may decide to close socket and mark connection
at 'disconnected' state, for example, when DISCONNECTE packet is
received, or, when failed to write data to socket.
However, by the time EMQX decided to close the socket, the socket
might have already been closed by peer, and the `tcp_closed` envet
is already delivered to the process mailbox -- causing EMQX to
handle sock_close event at 'disconnected' state.
2023-11-19 22:27:25 +01:00
Ilya Averyanov e93e9ed108 feat(rebalance): improve rebalance usability
* make availability API endpoint public
* allow connections during wait_health_check interval
* make availability status calculation more consistent and lightweight
* refactor test to get rid of some mocks and to use cth
2023-11-18 21:29:16 +03:00
Thales Macedo Garitezi 90571b7d8e test: fix noise about undefined unofficial callbacks 2023-11-16 13:16:02 -03:00
Andrew Mayorov 656d575e2d
ci(router): fix flaky testcase 2023-11-16 17:53:20 +07:00
Andrew Mayorov d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
Andrew Mayorov 98f947f4f3
ci(router): fix flaky testcase 2023-11-15 22:28:52 +07:00
Andrew Mayorov 8919b08207
fix(utils): rename `emqx_utils_stream:take/2` to `consume/2`
Which is more neutral and harder to confuse with a destructive `take` in
collections.
2023-11-15 17:20:40 +07:00
Thales Macedo Garitezi 423fb32171 test: fix noise about undefined unofficial callbacks 2023-11-14 19:48:39 +01:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
ieQu1 124c0e2dba fix(ds): Respect receive_maximum from the connection info 2023-11-14 16:06:19 +01:00
Ivan Dyachkov bbf00c2f71
Merge pull request #11946 from id/1114-sync-r53
sync release-53 to master
2023-11-14 13:38:24 +01:00
Thales Macedo Garitezi 01a2a3b1c0
Merge pull request #11930 from thalesmg/ds-session-discard-m-20231110
feat(ds): implement session discard
2023-11-14 09:30:34 -03:00
Andrew Mayorov 6812ee9d0f
fix(mgmt): hide route selection behind router interface
Also introduce a generic _stream_ concept, mostly to deal with
iterating over 2 ETS tables at once with `ets:match_object/3`.
2023-11-14 18:36:25 +07:00
Ivan Dyachkov 28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Ivan Dyachkov 1c57993c91 Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53 2023-11-14 10:44:35 +01:00
Ivan Dyachkov a9cf8fe78c chore: 5.3.1 2023-11-14 08:44:41 +01:00
Zaiming (Stone) Shi 9b9908780c
Merge pull request #11926 from kjellwinblad/kjell/fix_bridge_v2_to_action_cluster_hocon/EMQX-11353
fix: reflect bridge V1 upgreade in cluster.hocon
2023-11-13 14:18:57 +01:00
Thales Macedo Garitezi 45dad2ed3a feat(ds): implement session discard
Fixes https://emqx.atlassian.net/browse/EMQX-9739

Fixes some issues to ensure the session is discarded when the client connects with
`clean_start = true`, and added some cleanup to subscriptions/routes/iterators/streams.

> There is an API that session garbage collector can use to perform cleaning

We already have `emqx_session:destroy/1`, which could serve as an API for a periodic
session GC to use.
2023-11-13 09:06:22 -03:00
Kjell Winblad 26fc9ed89d
fix: log warning message after suggestion from @zmstone
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-11-13 12:53:09 +01:00
Andrew Mayorov 03843c6071
feat(router): switch to v2 routing store by default 2023-11-13 17:43:30 +07:00
Kjell Winblad 5838568d5d fix: hard coded call to schema module
The user can configure the configuration loader and schema module so we
should not hard code a call to the default schema module.
2023-11-13 11:41:40 +01:00
Zaiming (Stone) Shi 65efa2672e test(schema): delete stale reference to emqx_schema:file() type 2023-11-13 09:28:26 +01:00
JimMoen 867d5aaea6
chore: rm dbg funcs in SUITE 2023-11-12 21:44:23 +08:00
JimMoen 523bb17fc8
test: mark proper types gateway connection modules need be fixed 2023-11-12 21:43:10 +08:00
Zaiming (Stone) Shi 021f7e6b49 refactor(schema): comma_separated_list is list(string()) not list(any()) 2023-11-12 11:37:58 +01:00
Zaiming (Stone) Shi 8be718b22f refactor(emqx_schema): delete unused type bar_separated_list 2023-11-12 11:29:33 +01:00
ieQu1 5170267fef
Merge pull request #11924 from ieQu1/dev/emqx-ds-behavior
feat(ds): Introduce emqx_ds behavior
2023-11-10 15:23:28 +01:00
Kjell Winblad 6ebf0d1bdd fix: reflect bridge V1 upgrade in override configurations
Fixes:
https://emqx.atlassian.net/browse/EMQX-11353
2023-11-10 14:52:29 +01:00
Zaiming (Stone) Shi 86110824eb feat: upgrade hocon to 0.40.0 which supports union type display name 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi 8a4fba431e refactor(emqx_limiter_schema): use typerefl alias 2023-11-10 13:41:51 +01:00
ieQu1 8dc8237331 feat(ds): Introduce emqx_ds behavior 2023-11-10 12:16:39 +01:00
ieQu1 38d004e06e
Merge pull request #11913 from ieQu1/dev/fix-hooks-comment
chore(hooks): Fix comment
2023-11-10 03:15:54 +01:00
ieQu1 ced7be61f0
Merge pull request #11917 from ieQu1/dev/fix-packet-id-to-seqno-trans
fix(ds): Fix packet id -> sequence number translation
2023-11-09 22:57:16 +01:00
Thales Macedo Garitezi 7a237f6a3a Merge branch 'master' into sync-m-to-r54-20231109 2023-11-09 18:49:52 -03:00
Thales Macedo Garitezi 371a49304d Merge branch 'release-53' into sync-r53-to-m-20231109 2023-11-09 17:19:23 -03:00
ieQu1 5aa9d026df fix(ds): Apply review remarks
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-09 21:16:59 +01:00
ieQu1 b812db1e3c fix(ds): Fix packet id -> sequence number translation 2023-11-09 20:12:39 +01:00
Thales Macedo Garitezi 2d539463cd test: fix flaky test
```
 Testing lib.emqx.emqx_cm_SUITE: *** FAILED test case 24 of 31 ***
%%% emqx_cm_SUITE ==> t_stepdown_session_takeover_begin_timeout: FAILED
%%% emqx_cm_SUITE ==>
Failure/Error: ?assertEqual([<0.6635.8>,<0.6520.8>], lists : sort ( emqx_cm : lookup_channels ( ClientId ) ))
  expected: [<0.6635.8>,<0.6520.8>]
       got: [<0.6520.8>,<0.6635.8>]
      line: 309
```
2023-11-09 15:32:52 -03:00
Thales Macedo Garitezi ee87f90b0d
Merge pull request #11906 from thalesmg/ds-avoid-records-m-20231108
chore(ds): avoid using records in persistence / rpc
2023-11-09 14:25:27 -03:00
ieQu1 42a9f0eb6b
Merge pull request #11914 from ieQu1/dev/speed-up-poll
fix(ds): Speed up polling for the new messages
2023-11-09 16:40:06 +01:00
Thales Macedo Garitezi 42536bb5f7 chore(ds): avoid using records in persistence / rpc
Fixes https://emqx.atlassian.net/browse/EMQX-11279
2023-11-09 12:00:25 -03:00
ieQu1 cf4a46a78b fix(ds): Speed up polling for the new messages
Poll immediately if the previous poll returned non-empty result
2023-11-09 14:22:32 +01:00
ieQu1 3350bda446 chore(hooks): Fix comment 2023-11-09 14:05:13 +01:00
firest 2b97800c8b fix(limiter): simplify the descriptions for fields of the limiter 2023-11-09 18:00:09 +08:00
William Yang f7a2359329 test(quic): fix flaky 2023-11-09 10:56:16 +01:00
Ivan Dyachkov 0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
Thales Macedo Garitezi 8540566eba chore: prepare to tag `e5.3.1-alpha.5` 2023-11-08 13:02:19 -03:00
ieQu1 788698f157
Merge pull request #11720 from ieQu1/dev/refactor-persistent-session
Refactor emqx_durable storage application and introduce learned topic structure storage
2023-11-08 14:27:40 +01:00
Ilya Averyanov d061d64c70 fix(plugins): fix backward compatibility 2023-11-08 12:36:09 +03:00
Ivan Dyachkov 0c91bec98d chore: merge 'upstream/release-53' 2023-11-08 09:24:38 +01:00
ieQu1 a1cdbaa76d fix(ds): Address code review remarks 2023-11-08 04:31:27 +01:00
ieQu1 7cb0322856 fix(emqx): Move bpapi and emqx message record to emqx_utils app 2023-11-08 03:04:19 +01:00
ieQu1 74cb43f8b1 fix(ds): Add unique ID to the key 2023-11-08 03:04:19 +01:00
Andrew Mayorov 46d8301bc0 feat(emqx): expose timestamp function in `emqx_message`
So that the code that relies on it would not need to guess clock source
and precision.
2023-11-08 03:04:19 +01:00
ieQu1 8e5dda40be fix(ds): Fix static checks 2023-11-08 03:04:19 +01:00
ieQu1 99329e1243 refactor(ds): Address review remarks 2023-11-08 03:04:19 +01:00
ieQu1 2de79dd9ac feat(ds): Replay QoS1 messages 2023-11-08 03:04:19 +01:00
ieQu1 465e8a90dd revert(ds): Remove change from the old protocol file 2023-11-08 03:04:19 +01:00
ieQu1 ef46c09caf feat(ds): Implement ratchet function for bitmask keymapper 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi 903b3863d1 chore(ps_ds): make persistent session module use new `emqx_ds` APIs 2023-11-08 03:04:19 +01:00
ieQu1 2972bf14ee refactor(ds): Implement create_generation gen_rpc storage layer call 2023-11-08 03:04:19 +01:00
ieQu1 c6a721a7eb refactor(ds): Passthrough open_db and get_channels to storage layer 2023-11-08 03:04:19 +01:00
ieQu1 59d01dc823 refactor(ds): Implement emqx_ds:open_db 2023-11-08 03:04:19 +01:00
ieQu1 7095cb8583 refactor(ds): Refactor storage layer 2023-11-08 03:04:19 +01:00
ieQu1 c91df2f5cd refactor(ds): Create a prototype of replication layer 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi 7ab57824dc chore(ds): change return type of `storage_layer:next/{1,2}`
Part of https://emqx.atlassian.net/browse/EMQX-10942

The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.

From:

```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```

To:

```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.

-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
2023-11-08 03:04:19 +01:00
Ilya Averyanov 75704513d8 fix(plugins): fix backward compatibility 2023-11-07 19:06:37 +03:00
Thales Macedo Garitezi c4c9d7ceb3 chore: prepara to tag `e5.3.1-alpha.4` 2023-11-07 13:04:46 -03:00
Zaiming (Stone) Shi 6af6309301 chore: upgrade to hocon 0.39.19 2023-11-06 19:16:19 +01:00
Zaiming (Stone) Shi afe6f79a66 feat(bridge): add description field to bridge and connector 2023-11-03 06:53:42 +01:00
Andrew Mayorov 910e81bc41
Merge pull request #10442 from keynslug/ft/EMQX-9257/placeholder
feat(tpl): split `emqx_placeholder` into a couple of modules
2023-11-02 22:50:05 +07:00
Andrew Mayorov 0538a77700
feat(tpl): use `emqx_connector_template` in `emqx_authn`, `emqx_authz`
This slightly changes semantics: now the attempt to create authenticator
with illegal bindings in templates will fail, instead of treating them
as literals. The runtime behaviour on the other hand should be the same.
2023-11-02 17:11:10 +07:00
Zaiming (Stone) Shi 73dd2f0ffd chore: bump version to e5.3.1-alpha.3 2023-11-02 10:36:41 +01:00
Andrew Mayorov 7092c75597
Merge pull request #11809 from keynslug/ft/EMQX-10808/file-secrets
feat(mqttbridge): support file-sourced secrets as passwords
2023-11-01 00:29:30 +07:00
Thales Macedo Garitezi b420b53075 fix(bridges_v2): check bridge name length before attempting atom conversion
Fixes https://emqx.atlassian.net/browse/EMQX-11289
2023-10-31 13:21:26 -03:00
Andrew Mayorov d278486416
fix(secret): dedicate a specific loader module for file secrets
To make code employing `emqx_secret` easier to follow.
2023-10-31 19:26:04 +07:00
Ivan Dyachkov b1ab213081 chore: merge 'upstream/release-53' into 1031-sync-r53 2023-10-31 11:06:25 +01:00
JianBo He 21e4f918aa
Merge pull request #11833 from JimMoen/fix-shared-sub-topic-or-subscription-searching
Fix shared sub topic or subscription searching
2023-10-31 08:16:36 +08:00
Thales Macedo Garitezi a60b96c5fd test(kafka_producer): minor adjustments to test suite
- Use `emqx_cth_suite`.
- Use `query_mode` matrix value when setting up bridge in a couple test cases.
2023-10-30 12:13:40 -03:00
Zaiming (Stone) Shi f463eff02d chore: e5.3.1-alpha.2 2023-10-30 14:49:08 +01:00
Zaiming (Stone) Shi 0656b6be3c refactor(emqx_config): use dynamic callback to upgrade raw config 2023-10-30 14:49:08 +01:00
Kjell Winblad 9dc3a169b3 feat: split bridges into a connector part and a bridge part
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>

Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.

Issues:
https://emqx.atlassian.net/browse/EMQX-10805
2023-10-30 14:48:47 +01:00
zhongwencool e896957803
Merge pull request #11824 from zhongwencool/audit-log-reviewed
refactor: audit log review refactor
2023-10-30 17:51:15 +08:00
JimMoen 814e22feb3
fix: topics/subscripton mgmt api searching 2023-10-30 14:42:47 +08:00
zhongwencool 0634ff61c0 fix: dialyzer warning 2023-10-30 09:57:30 +08:00
JimMoen a2015f37ae
Merge pull request #10976 from JimMoen/fix-shared-sub-unsub 2023-10-27 21:22:27 +08:00
JimMoen d563121284
refactor: move ?REDISPATCH_TO macro to emqx_mqtt.hrl 2023-10-27 17:54:14 +08:00
JimMoen 3b5cc912e7
fix: add `redispatch_to` header to all msgs when deliver shared-sub
- to find correct SubOpts for shared-sub dispatch
- use previous key `redispatch_to` to ensure rolling upgrade compatibility
2023-10-27 16:15:23 +08:00
zhongwencool 995948f0e8 refactor: remove seq from audit record 2023-10-27 14:07:19 +08:00
JimMoen afec6fa2f6
fix: TopicFilter may modified by `client.subscribe` hook 2023-10-27 09:23:44 +08:00
JimMoen 802a36c670
fix: find SubOpts by shared_record, not deliver topic 2023-10-27 09:23:44 +08:00
JimMoen 0ca725ff25
fix: deprecated `share` in `subopts` 2023-10-27 09:23:43 +08:00
JimMoen b5411da770
refactor: subscribe process to fix shared-sub 2023-10-27 09:23:43 +08:00
Andrew Mayorov 44b4205561
fix(secret): do not treat missing file secrets as config error
They are intended to be used mostly in the context of resources, which
have their own feedback mechanism: statuses, retries, etc.

Also turn the error into a throw exception, so that it can be
interpreted as a regular error condition, for example by the resource
manager.
2023-10-26 14:37:14 +07:00
Zaiming (Stone) Shi 76a8b0efd4 test: fix a flaky test case emqx_crl_cache_SUITE:t_revoked 2023-10-25 16:08:31 +02:00
firest 7a8a5926ab fix(rbac): adjust the role names 2023-10-24 19:01:12 +08:00
Andrew Mayorov 1c2f9321d1
feat(emqx): add file-sourced generic secrets
These secrets follow the same `emqx_secret` convention of 0-arity
functions. Also provide a simple HOCON schema module for use in
application schemas.
2023-10-24 16:24:51 +07:00
Ilya Averyanov edde661da3 fix(authn): fix pbkdf2 option validation 2023-10-23 10:26:11 +03:00
William Yang a9b42af8b3
Merge pull request #11796 from qzhuyan/fix/william/gen_rpc-driver
fix(gen_rpc): ensure client/server use same driver
2023-10-23 08:24:17 +02:00
JimMoen d467289bb2
fix: avoid duplicated apikey from data import 2023-10-20 21:13:04 +08:00
William Yang dd5807b797 fix(gen_rpc): ensure client/server use same driver
Also bump to gen_rpc 3.2.1
2023-10-20 12:17:49 +02:00
zhongwencool ef692596f7 fix: don't crash when 401 and 403 unauthorize 2023-10-20 15:22:30 +08:00
JimMoen f381961108 fix: macro EMQX_RELEASE_EDITION when `emqx` run as standalnoe app 2023-10-20 14:47:24 +08:00
zhongwencool befc484544 chore: add changelog entry and fix xref warning 2023-10-20 14:47:24 +08:00
zhongwencool 141061c1e2 feat: add max_filter_size and ignore_high_frequency_request config 2023-10-20 14:47:24 +08:00
zhongwencool 926c804314 feat: add /audit http api to filter audit log 2023-10-20 14:47:24 +08:00
lafirest 74442b0d31
Merge pull request #11766 from lafirest/feat/rbac
feat(api_key): add RBAC feature for the API key
2023-10-20 09:24:41 +08:00
firest 26ec860d96 feat(backup): add migration mechanism when import backup data 2023-10-17 16:23:32 +08:00
firest db3915d472 fix(rbac): update changes && fix CI errors 2023-10-17 13:44:37 +08:00
firest e095de7367 feat(api_key): add RBAC feature for the API key 2023-10-16 14:10:01 +08:00
Zaiming (Stone) Shi 6eb3bb7cff Merge remote-tracking branch 'origin/release-53' into 1114-sync-release-53 2023-10-14 10:16:38 +02:00
Andrew Mayorov 083e2da347
chore(bpapicheck): make some failures more user-friendly 2023-10-13 15:42:08 +07:00
Andrew Mayorov 0b9ac24c1e
fix(bpapicheck): stop depending on git index 2023-10-13 15:40:49 +07:00
William Yang a69c53455f fix: bump to quicer 0.0.202
bring in urgent fixs from msquic 2.2.3
2023-10-11 13:12:32 +02:00
Zaiming (Stone) Shi a2e86c67db
Merge pull request #11733 from keynslug/fix/EMQX-10827/sesson-takeover
fix(cm): bring back pre-v5.3.0 compat in `takeover_session_begin/1`
2023-10-10 19:03:29 +02:00
Ivan Dyachkov bfb2218392 chore: fix release version 2023-10-10 14:50:49 +02:00
Zaiming (Stone) Shi 143673b703 chore: bump release version to 5.3.1-alpha.1 2023-10-10 14:35:46 +02:00
Andrew Mayorov 522302fee1
fix(cm): bring back pre-v5.3.0 compat in `takeover_session_begin/1`
Which was accidentally broken in bf164175.
2023-10-10 01:34:10 +07:00
Zaiming (Stone) Shi 1e93d2f1fc chore: upgrade to ekka 0.15.16 gen_rpc 3.2.0
* ekka 0.15.16 supports 'inet6_tls' as ekka.proto_dist
* gen_rpc 3.2.0 supports true | false as gen_rpc.ipv6_only
2023-10-09 20:03:21 +02:00
Ilya Averyanov 5dff36474d chore(auth): get rid of hardcoded schema modules in auth 2023-10-05 13:41:50 +03:00
Ilya Averyanov c2c56ba481 chore(auth): update tests 2023-10-05 13:41:50 +03:00
Ilya Averyanov 1eb75b43c4 chore(auth): split emqx_authn and emqx_authz apps 2023-10-05 13:41:50 +03:00
Zaiming (Stone) Shi ca8da5723a
Merge pull request #11704 from zmstone/0928-sync-release-53
0928 sync release 53
2023-10-02 21:57:32 +02:00
Thales Macedo Garitezi 2358d67908 refactor: move session stuff from `emqx_ds` to `emqx_persistent_session_ds`
Part of https://emqx.atlassian.net/browse/EMQX-10942
2023-09-29 18:00:24 -03:00
Zaiming (Stone) Shi bce8fd2fbc chore: bump app versions 2023-09-29 18:56:52 +02:00
Zaiming (Stone) Shi ed5b456d62 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 18:24:44 +02:00
Zaiming (Stone) Shi a852400fb3 chore: add 5.3.bpapi 2023-09-29 16:34:04 +02:00
Zaiming (Stone) Shi fe01aaff4f chore: prepare for e5.3.0 promotion 2023-09-29 13:32:57 +02:00
Zaiming (Stone) Shi ce5bd0a3ce Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 11:36:32 +02:00
Zaiming (Stone) Shi c64e599e81 docs: document how to retrieve peercert 2023-09-29 10:33:57 +02:00
Zaiming (Stone) Shi 1177a32310 chore: bump version to 5.3.0-rc.2 2023-09-29 09:49:27 +02:00
Zaiming (Stone) Shi b267fc2588 chore: bump release version to 5.3.0 2023-09-28 21:22:33 +02:00
Zaiming (Stone) Shi 1a13b2ac56 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-28 13:53:03 +02:00
Zaiming (Stone) Shi 7ddcd71412 docs(ssl_dist.conf): update doc link 2023-09-28 11:16:18 +02:00
ieQu1 b77e5e880a
Merge pull request #11697 from ieQu1/dev/gen-rpc-3.1.1
chore(gen_rpc): Bump version to 3.1.1
2023-09-28 10:03:36 +02:00
JianBo He 6d2adfc259 chore(audit): emit the message field 2023-09-28 11:04:42 +08:00
JianBo He 859b122cdd chore: format codes 2023-09-28 10:25:37 +08:00
JianBo He dd9938114c chore(audit): add from field 2023-09-28 10:25:37 +08:00
lafirest b0d86eecd6
Merge pull request #11691 from lafirest/fix/sso_ssl
fix(sso): support for SSL update && ensure update is atomic
2023-09-28 10:22:34 +08:00
ieQu1 0aa3ccdd65 feat(gen_rpc): Add schema for the TLS versions and cipher suites 2023-09-28 00:36:59 +02:00
Zaiming (Stone) Shi afdda107af fix(logger): json format log encode binary list as string array 2023-09-27 23:40:01 +02:00
Zaiming (Stone) Shi 6f7a4344dc fix: do not gc sso saml SP singing keys 2023-09-27 23:19:39 +02:00
Zaiming (Stone) Shi c8cbbff044 fix(logger): no need for special handling of empty string
when formating json logs, there is no need to handle empty strings
special, already covered by unicode handling
2023-09-27 21:26:47 +02:00
ieQu1 3c37f19105 chore(gen_rpc): Bump version to 3.1.1 2023-09-27 21:09:00 +02:00
Zaiming (Stone) Shi 6e8c73258f test: refactor emqx_rpc unit tests 2023-09-27 12:58:06 +02:00
Andrew Mayorov 1d0e789e4d
Merge pull request #11679 from keynslug/ft/EMQX-10942/custom-session-timers
feat(session): add custom session timers mechanism
2023-09-26 19:09:53 +03:00
Thales Macedo Garitezi aae59f1efd
Merge pull request #11682 from thalesmg/fix-file-logger-type-r53-20230925
fix(file_logger): change file logger type depending on rotation size
2023-09-26 10:22:36 -03:00
Andrew Mayorov bce2142df8
test(session): add custom session timers testcase 2023-09-26 16:22:06 +03:00
Zaiming (Stone) Shi 4e15edb5e4
Merge pull request #11661 from zmstone/0922-fix-json-log-formatter
0922 fix json log formatter
2023-09-26 14:13:41 +02:00
Thales Macedo Garitezi fdcd73e20c fix(file_logger): change file logger type depending on rotation size
Fixes https://emqx.atlassian.net/browse/EMQX-11036

From `logger_disk_log_h:open_disk_log`:

```erlang
open_disk_log(Name,File,Type,MaxNoBytes,MaxNoFiles) ->
    case filelib:ensure_dir(File) of
        ok ->
            Size =
                if Type==halt -> MaxNoBytes;
                   Type==wrap -> {MaxNoBytes,MaxNoFiles} %% <-------
                end,
            Opts = [{name,   Name},
                    {file,   File},
                    {size,   Size},
                    {type,   Type},
                    {linkto, self()},
                    {repair, false},
                    {format, external},
                    {notify, true},
                    {quiet,  true},
                    {mode,   read_write}],
            case disk_log:open(Opts) of
```

Affects all file loggers (audit included):

```
% emqx_config_logger:update_log_handler/1 -> ok
iex(emqx@127.0.0.1)14> Config override: log.file.emqx_audit is updated, but failed to add handler: {handler_not_added,
                                                                             {badarg,
                                                                              [{size,
                                                                                {infinity,
                                                                                 10}},
                                                                               {type,
                                                                                wrap},
                                                                               {linkto,
                                                                                <0.1952.0>},
                                                                               {repair,
                                                                                false},
                                                                               {format,
                                                                                external},
                                                                               {notify,
                                                                                true},
                                                                               {quiet,
                                                                                true},
                                                                               {mode,
                                                                                read_write}]}}
```
2023-09-26 09:04:44 -03:00
Ilya Averyanov 14983ec14a chore(hooks): validate hookpoints and document hook callbacks
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-26 14:27:42 +03:00
Thales Macedo Garitezi ff7f37ccf5 test(cth): allow defining schema to load for app 2023-09-25 13:22:41 -03:00
Andrew Mayorov b1f144ab8b
feat(session): add custom session timers mechanism
That are managed exclusively by the session implementation, unlike
common session timers that are managed by the channel itself.
2023-09-25 18:19:26 +03:00
lafirest 1df8326fb8
Merge pull request #11677 from lafirest/fix/sso_cfg
adjust the config path for the SSO feature and improve the update logic
2023-09-25 23:00:46 +08:00
Zaiming (Stone) Shi 5e6996dc05 refactor: log formatter format mfa+line as m:f/a(line)
also improve json formatter when a field is iolist
2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi 6a557980e4 refactor(logger): ensure JSON log field order
Ensure that the log fileds in JSON format are ordered as
time, level, msg, mfal, ...
2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi 5f45ba50ff refactor: delete log formatter config for audit log handler
It is using 'json' formatter, the template is useless
2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi 1fed38c248 fix(logger): write 'json' format logs as JSON 2023-09-25 16:55:22 +02:00
William Yang 9106eb92d2
Merge pull request #11675 from qzhuyan/fix/william/quic-respect-param-verify 2023-09-25 15:48:01 +02:00
firest cfdb25b213 fix(sso): updates the SSO backend when the `[dashboard]` has updated 2023-09-25 20:30:47 +08:00
William Yang 36d3a3a524 fix: bump to quicer 0.0.201
do not load cacertfile if verify_none
2023-09-25 13:02:01 +02:00
William Yang 22193d273a fix(quic): ignore undefined cacertfile 2023-09-25 12:57:55 +02:00
Ivan Dyachkov 8c93c79b44 chore: e5.3.0-alpha.2 2023-09-25 12:52:45 +02:00
Ivan Dyachkov e1f1c64ecd Merge branch 'release-53' into 0925-sync-r53-to-master 2023-09-25 08:12:45 +02:00
JianBo He df94426ee3
chore: make static_check happy 2023-09-22 21:50:42 +08:00
Thales Macedo Garitezi d6935b6a67 feat: add port scan diagnostics to mria waiting for tables checks
Fixes https://emqx.atlassian.net/browse/EMQX-10944

Also updates ekka -> 0.15.15, mria -> 0.6.4

How to test
===========

1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop them in order.
3. Start only the first node that was stopped in the previous step.
4. Wait until the log is printed.

Or, more easily:

1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop all but one.
3. Run `mria_mnesia:diagnosis([]).` on that node.

Example output
==============

```
   Check check_open_ports should get ok but got #{msg =>
                                                     "some ports are unreachable",
                                                 results =>
                                                     #{'emqx@172.100.239.4' =>
                                                           #{open_ports =>
                                                                 #{4370 => false,
                                                                   5370 =>
                                                                       false},
                                                             ports_to_check =>
                                                                 [4370,5370],
                                                             resolved_ips =>
                                                                 [{172,100,239,
                                                                   4}],
                                                             status =>
                                                                 bad_ports},
                                                       'emqx@172.100.239.5' =>
                                                           #{open_ports =>
                                                                 #{4370 => false,
                                                                   5370 =>
                                                                       false},
                                                             ports_to_check =>
                                                                 [4370,5370],
                                                             resolved_ips =>
                                                                 [{172,100,239,
                                                                   5}],
                                                             status =>
                                                                 bad_ports}}}
```

After one node is back:

```
   Check check_open_ports should get ok but got #{msg =>
                                                     "some ports are unreachable",
                                                 results =>
                                                     #{'emqx@172.100.239.4' =>
                                                           #{ports_to_check =>
                                                                 [4370,5370],
                                                             resolved_ips =>
                                                                 [{172,100,239,
                                                                   4}],
                                                             status => ok},
                                                       'emqx@172.100.239.5' =>
                                                           #{open_ports =>
                                                                 #{4370 => false,
                                                                   5370 =>
                                                                       false},
                                                             ports_to_check =>
                                                                 [4370,5370],
                                                             resolved_ips =>
                                                                 [{172,100,239,
                                                                   5}],
                                                             status =>
                                                                 bad_ports}}}
```
2023-09-21 14:29:01 -03:00
Ivan Dyachkov 7cf60c5a91 chore: e5.3.0-alpha.1 2023-09-21 19:29:00 +02:00
Andrew Mayorov acf4227fc6
test(session): fix quic testgroup in persistent session suite
Which broker after quicer 0.0.200 upgrade.
2023-09-21 20:52:04 +04:00
Andrew Mayorov b563e30615
Merge pull request #11650 from keynslug/fix/simplify-takeover-suite
test(session): make testsuite trigger takeover logic consistently
2023-09-21 18:02:38 +04:00
zhongwencool beea1be9f0
Merge pull request #11599 from zhongwencool/audit-log 2023-09-21 18:39:23 +08:00
Ivan Dyachkov 0a61d08a0f
Merge pull request #11651 from id/0921-sync-r52-to-master
sync r52 to master
2023-09-21 12:31:24 +02:00
William Yang 3a5227198e
Merge pull request #11642 from qzhuyan/dev/william/quicer-0.0.200
quicer 0.0.200
2023-09-21 12:20:14 +02:00
Ivan Dyachkov dafd7c6085 chore: bump apps versions 2023-09-21 10:58:42 +02:00
zhongwencool 21bb209fb1 feat: added support for auditing API and CLI activity in logs 2023-09-21 16:35:46 +08:00
Ivan Dyachkov 105bebc250 chore: merge release-52 into master 2023-09-21 10:22:47 +02:00
Andrew Mayorov a8f4b5bf86
test(session): make testsuite trigger takeover logic consistently 2023-09-21 11:53:40 +04:00
zhongwencool 1b29e4b189
Merge pull request #11641 from zhongwencool/slog-msg-unify 2023-09-21 07:32:39 +08:00
Andrew Mayorov a2ddd9d5f5
fix(session): respect existing session even if expiry interval = 0
If the original connection had Session-Expiry-Interval > 0, and the
new connection set Session-Expiry-Interval = 0, the MQTTv5 spec says
that (supposedly) we still have to continue with the existing session
(if it hasn't expired yet).

Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:54 +04:00
Andrew Mayorov 3945f08f8f
fix(sessds): try to ensure iterators are closed on destroy 2023-09-20 22:55:54 +04:00
Andrew Mayorov 21e82b9534
test(sessmem): make retry delivery testcase more involved 2023-09-20 22:55:54 +04:00
Andrew Mayorov 69889d14a3
fix(sessds): fix use of undefined types 2023-09-20 22:55:54 +04:00
Andrew Mayorov 9362ef6f73
test(sessmem): drop unnecessary nesting in testcase
Also get rid of sneaky binding assignment.
2023-09-20 22:55:25 +04:00
Andrew Mayorov 98706cd215
chore: ensure comments follow code style consistently 2023-09-20 22:55:25 +04:00
Andrew Mayorov 8670efbfa0
chore(chan): rename `Name` → `TimerName` for better readability 2023-09-20 22:55:25 +04:00
Andrew Mayorov 7a9916c84d
fix(sessds): convert ds iterator topics upon opening ds session 2023-09-20 22:55:25 +04:00
Andrew Mayorov 540ca6d60f
chore: drop few commented out and irrelevant pieces 2023-09-20 22:55:24 +04:00
Andrew Mayorov 9d145890cc
refactor(sessmem): pass log context as part of session event
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
Andrew Mayorov adc29e15cc
refactor(session): make typespecsa and flow a bit more clear
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
Andrew Mayorov e422f492ef
test(sessds): reuse and expand persistent session test suite 2023-09-20 22:55:24 +04:00
Andrew Mayorov 3383ae19a9
test(session): switch `emqx_persistent_session_SUITE` to cth tooling 2023-09-20 22:55:23 +04:00
Andrew Mayorov 7326ef550b
fix(sessds): make existing parts of persistent session impl work 2023-09-20 22:55:22 +04:00
Andrew Mayorov e713fc38aa
feat(broker): reflect persisted messages in publish result
In order for callers to distinguish between silently dropped and
durably persisted message w/o matching subscribers.
2023-09-20 22:53:45 +04:00
Andrew Mayorov f4953e719b
fix(cmproto): fix few typespecs 2023-09-20 22:53:45 +04:00
Andrew Mayorov 8af107e28d
test(ds): simplify cluster test setups 2023-09-20 22:53:45 +04:00
Andrew Mayorov 7c4f68dd3d
fix(session): make utility function names consistent
Before this commit behavior of `is_banned_msg/1` / `should_discard/1`
were actually the exact opposite of their names.

Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:53:44 +04:00
Andrew Mayorov 2dae8020ec
refactor(cm): avoid deep indirection in `emqx_session_mem` 2023-09-20 22:53:44 +04:00
Andrew Mayorov abeff0bc4f
chore(session): try to describe what happens after session takeover 2023-09-20 22:53:44 +04:00
Andrew Mayorov 45d44df11d
refactor(session): update eviction channel session logic
The changes partially reflect `emqx_channel` changes with respect to
in-memory session specific logic. The difference is that eviction
channel does not replay post-takeover, instead enqueues messages.
2023-09-20 22:53:44 +04:00
Andrew Mayorov ab1c4c4222
refactor(session): drop `lookup/1` session API
Due to the fact it's not used for anything right now.
2023-09-20 22:53:44 +04:00
Andrew Mayorov 97881ff3ca
refactor(session): bring back common session timers 2023-09-20 22:53:43 +04:00
Andrew Mayorov bf16417513
feat(session): introduce session implementation concept 2023-09-20 22:53:42 +04:00
Andrew Mayorov 04731b7ef7
test(takeover): randomize messages for random natural ordering 2023-09-20 22:52:14 +04:00
Andrew Mayorov f022c9b1a4
feat(emqx): add `emqx_inflight:fold/3` generic function 2023-09-20 22:52:14 +04:00
Andrew Mayorov e4866adc2f
refactor(chan): make timer names equal to messages they send
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
Andrew Mayorov cfb1bf1fa4
chore(emqx): drop remnants of former session persistence impl
1. It is not functional anyway.
2. It blocks `emqx_session` refactoring in a few places.
2023-09-20 22:52:13 +04:00
William Yang e43b87f18e test(quic): update emqx_broker_SUITE 2023-09-20 20:20:18 +02:00
Thales Macedo Garitezi 2965fa6fcb
Merge pull request #11613 from thalesmg/ds-keyspace-m-20230914
feat(ds): introduce keyspace concept
2023-09-20 14:21:27 -03:00
William Yang da626f6d29 feat(quic): bump to quicer 0.0.200 and emqtt 1.9.0 2023-09-20 16:43:15 +02:00
zhongwencool f789a7504f
chore: update apps/emqx/src/emqx_quic_connection.erl
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:00:44 +08:00
Ivan Dyachkov 09aea47f40 chore: 5.2.1 2023-09-20 14:04:28 +02:00
zhongwencool 2f1fa2e961 chore: unified slog message formatting to improve logging consistency 2023-09-20 18:13:00 +08:00
Thales Macedo Garitezi 5ed5ac48ee refactor: combine shard id and keyspace into a single value 2023-09-19 12:31:32 -03:00
Zaiming (Stone) Shi 65818740f3
Merge pull request #11622 from zmstone/0917-upgrade-gen_rpc-to-3.1.0
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-19 10:22:53 +02:00
Ivan Dyachkov a69c26b86d chore: e5.2.1-alpha.3 2023-09-19 08:17:29 +02:00
JianBo He de1bc04843 chore: bump ekka to 0.15.14, mria to 0.6.3
Inclued ekka's updates:
- https://github.com/emqx/ekka/pull/214
- https://github.com/emqx/ekka/pull/215
- https://github.com/emqx/ekka/pull/216
- https://github.com/emqx/ekka/pull/217

mria's updates:
- https://github.com/emqx/mria/pull/160
- https://github.com/emqx/mria/pull/161
- https://github.com/emqx/mria/pull/162
2023-09-19 09:43:19 +08:00
Thales Macedo Garitezi a511088fd4 refactor: address review comments 2023-09-18 17:47:56 -03:00
Zaiming (Stone) Shi de93d71014 test: fix emqx_sahred_sub_SUITE:t_remote
prior to this fix, the peer ct node is actually node itself
2023-09-18 21:54:43 +02:00
Zaiming (Stone) Shi 99b60849b4 test: fix flaky test emqx_falpping_SUITE:t_conf_update_timer 2023-09-18 20:44:38 +02:00
Zaiming (Stone) Shi d1e5b69eac test: increase wait time 2023-09-18 17:33:09 +02:00
Zaiming (Stone) Shi 4f1ec795eb test: delete special handling of emqtt race condition
the race should have been fixed in emqtt 1.8.7
2023-09-18 17:27:34 +02:00
Zaiming (Stone) Shi 46557c3462 fix(emqtt): upgrade from 1.8.6 to 1.8.7 2023-09-18 17:04:23 +02:00
Zaiming (Stone) Shi f66d9e76fe fix(emqx_channel): aovid logging frame errors as exceptions 2023-09-18 09:04:00 +02:00
Zaiming (Stone) Shi abbee4bc47 fix(esockd): upgrade esockd from 5.9.6 to 5.9.7 2023-09-17 19:44:04 +02:00
Zaiming (Stone) Shi 985f0e6994 chore(gen_rpc): upgrade from 2.8.1 to 3.1.0 2023-09-17 19:10:13 +02:00
Thales Macedo Garitezi 2aae3856a3
Merge pull request #11596 from thalesmg/fix-typespec-typos-m-20230912
fix: fix typespec typos
2023-09-15 09:24:42 -03:00
Ilya Averyanov 1959e15703 chore: simplify session eviction for node rebalance 2023-09-15 11:46:51 +02:00
Thales Macedo Garitezi b30bcf32bd feat(ds): introduce keyspace concept
Fixes https://emqx.atlassian.net/browse/EMQX-10579

This introduces the concept of "keyspaces" to our durable storage (DS) implementation, and
also refactors some places where "shard" and "keyspace" would be mixed up.

We might want to tune the storage options differently for distinct sets of topics, the
keyspaces.  The keyspace is composed by one or more shards.

- Keyspaces are identified simply by binary strings.
- DS configuration is scoped by keyspaces instead of shards.
- Starting a new DS shard requires definining to which keyspace the shard belongs.
2023-09-14 16:39:00 -03:00
Ilya Averyanov 11c39c4b6a chore: simplify session eviction for node rebalance 2023-09-14 18:28:25 +03:00
Thales Macedo Garitezi 6be0e14e8c fix: fix typespec typos 2023-09-14 10:21:05 -03:00
Thales Macedo Garitezi e41f7dd68c
Merge pull request #11598 from thalesmg/ds-fix-ps-router-m-20230912
fixes and improvements to persistent session router
2023-09-14 10:00:58 -03:00
firest 3eb46be2ae fix(banned): fix typos 2023-09-14 14:01:59 +08:00
Thales Macedo Garitezi 803b69d878
refactor: use more consistent return types
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2023-09-13 10:49:45 -03:00
Thales Macedo Garitezi f55f6dc779 docs(ps_session): add notes explaining rationale about operation order and consistency 2023-09-13 10:12:27 -03:00
Thales Macedo Garitezi e081abb9e6 perf(ps_router): only check if there are any routes when deciding whether to persist message 2023-09-13 10:02:39 -03:00
Thales Macedo Garitezi cae9ae1fab fix(ps_router): use `disc_copies` for storing persistent session routes 2023-09-13 10:02:39 -03:00
William Yang ea73d674b9 chore: bump version to e5.2.1-alpha.2 2023-09-13 14:55:17 +02:00
Serge Tupchii af015bc2bb chore(ekka): bump version to 0.15.13 2023-09-13 10:11:14 +03:00
William Yang 6ef296268e chore: bump version to e5.2.1-alpha.1 2023-09-12 15:24:02 +02:00
Thales Macedo Garitezi 59b94788bf refactor: address review comments 2023-09-11 16:25:34 -03:00
Thales Macedo Garitezi 64fcdb671d test: fix a bunch of inter-suite interferences 2023-09-11 16:25:34 -03:00
Thales Macedo Garitezi f1294736b7 feat(ds): add filter for message persistence
Fixes https://emqx.atlassian.net/browse/EMQX-10520
2023-09-11 16:25:34 -03:00
Thales Macedo Garitezi 7805cc8c9b fix(session): check if session is persistent 2023-09-11 16:25:33 -03:00
Thales Macedo Garitezi f490da71cd chore: prepare v5.2.0-build.1 2023-09-11 16:09:17 -03:00
Thales Macedo Garitezi d0f1c75651 chore: bump apps vsns 2023-09-11 16:06:53 -03:00
Thales Macedo Garitezi 223d47a6ac Merge branch 'master' into sync-m-r52-20230911 2023-09-11 16:04:28 -03:00
Thales Macedo Garitezi 063913f245 fix(session): remove recently added `iterators` field from `#session{}` record
Fixes https://emqx.atlassian.net/browse/EMQX-10945

This could lead to `badrecord` errors being raised if a takeover were to happen during a
rolling cluster upgrade, as the old nodes could receive a record with more fields than
expected.
2023-09-11 14:46:12 -03:00
Zaiming (Stone) Shi 7010ac358c chore: bump release version to v5.2.0 2023-09-08 16:31:52 +02:00
Andrew Mayorov 3d133801af
fix(router): use `ets:new/2` options compatible with OTP 24
While essentially keeping the same effect.
2023-09-08 18:04:29 +04:00
Zaiming (Stone) Shi f3bebb0ba2 Merge remote-tracking branch 'origin/master' into 0907-sync-master-to-release-52 2023-09-07 18:51:30 +02:00
Zaiming (Stone) Shi db2ec55aa0 chore: bump release version to official e5.2.0 2023-09-07 09:40:15 +02:00
Zaiming (Stone) Shi 7c2f87fabe test: merge broker and router boot modules 2023-09-06 21:36:16 +02:00
Zaiming (Stone) Shi e794143ae1 Merge remote-tracking branch 'origin/release-52' into 0906-sync-release-52-to-master 2023-09-06 09:08:22 +02:00
Thales Macedo Garitezi 954b73a9aa
Merge pull request #11533 from thalesmg/ds-unsubscribe-m-20230828
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
2023-09-05 09:47:17 -03:00
ieQu1 01a128878f chore(mria): Bump mria to 0.6.1 2023-09-05 11:08:32 +02:00
Andrew Mayorov d26995a2aa
Merge pull request #11524 from keynslug/ft/EMQX-10713/unified-route-tab
feat(router): add routing storage schema with 2 tables
2023-09-05 12:38:00 +04:00
Thales Macedo Garitezi 23d63f5e01 refactor: fork clientid type for `emqx_ds:session_id/0` 2023-09-04 15:38:25 -03:00
Thales Macedo Garitezi 60ae3c15c8 refactor: disregard impossible case 2023-09-04 15:38:25 -03:00
Thales Macedo Garitezi 73d4248334 test(cth_cluster): always allocate listener ports 2023-09-04 11:22:26 -03:00
Andrew Mayorov 9f0f218387
test(router): simplify cluster-related testcase in helper testsuite 2023-09-04 18:05:59 +04:00
Andrew Mayorov 7d26b7bc1a
fix(router): emit clearer error when conflicting schemas in use 2023-09-04 16:37:38 +04:00
Thales Macedo Garitezi d33ad8a46f feat: cluster purge (r5.2)
Port of https://github.com/emqx/emqx/pull/11447

targeting `release-52`

Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-09-04 09:15:28 -03:00
Andrew Mayorov 893f69617a
chore(test): drop obsolete TODOs 2023-09-04 14:40:49 +04:00
Andrew Mayorov f0a0c7d4b1
fix(cthsuite): avoid duplicate application env entries
Because `application:set_env/1` complains loudly if there are
duplicates.
2023-09-04 14:40:49 +04:00
Andrew Mayorov 5024304bf9
fix(router): wait for tables replicate before choosing schema vsn 2023-09-04 14:40:48 +04:00
Ivan Dyachkov 24230a64df chore: bump app versions 2023-09-04 11:39:21 +02:00
Ivan Dyachkov b2f57636bd Merge remote-tracking branch 'upstream/release-51' into 0904-sync-release-51 2023-09-04 11:34:22 +02:00
Serge Tupchii f0c75d97e1 chore: upgrade emqx_http_lib to 0.5.3
0.5.3 release includes a fix to parse <host>:<port> URLs using the default http scheme.
2023-09-01 19:56:07 +03:00
Andrew Mayorov 4ab5f8374b
chore(schema): mark routing storage schema readOnly
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-01 19:58:31 +04:00
Andrew Mayorov ebf13c90ea
test(router): make sort stable in e2e routing testcase 2023-09-01 19:48:56 +04:00
Andrew Mayorov 545f1c84a6
fix(cth): do not allocate ports for `emqx` app by default
This causes tricky and impressively hard to track side effects down
the line. Namely, loading `emqx_schema` _after_ `emqx_conf_schema`
(as part of cluster node startup sequence) leads to a couple of
schema root rewrites, because `emqx_schema` defines similar config
roots yet slightly differently (e.g. `authorization`).
2023-09-01 17:23:34 +04:00
Andrew Mayorov 0a879bffd1
test(router): add e2e testcase for cluster-wide routing
Which verifies that both router storage schemas work correctly.
2023-09-01 13:49:12 +04:00
Andrew Mayorov eb0385a28f
chore(router): separate test-only function exports 2023-09-01 13:49:11 +04:00
Zaiming (Stone) Shi c20ba0572a chore: bump to 5.2.0-alpha.4 2023-08-31 15:47:39 +02:00
Zaiming (Stone) Shi 876d539336 Merge remote-tracking branch 'andrew/ft/EMQX-10713/unified-route-tab' into 0831-make-use-of-new-routing-table 2023-08-31 15:46:33 +02:00
Andrew Mayorov fb094e1d47
test(router): avoid testsuite collisions 2023-08-31 13:36:01 +04:00
Thales Macedo Garitezi 4dfbc859f9 fix(plugins): update plugin order on whole cluster
Fixes https://emqx.atlassian.net/browse/EMQX-10879
2023-08-30 16:39:52 -03:00
Andrew Mayorov 6b9cb06334
fix(router): add / refine some comments and log messages
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-30 19:51:31 +04:00
Thales Macedo Garitezi 922ca5e141 feat(ds): close iterators when handling `UNSUBSCRIBE` packets
Fixes https://emqx.atlassian.net/browse/EMQX-9742
2023-08-30 11:14:28 -03:00
Thales Macedo Garitezi 9c6dd30f44 feat(session): store iterator ids in session record 2023-08-30 11:13:55 -03:00
Andrew Mayorov f8923ab419
Merge pull request #11539 from keynslug/ft/cth-desc-workdir
feat(cthsuite): add function to determine workdir of testrun
2023-08-30 17:15:55 +04:00
lafirest 8b3a9072f9
Merge pull request #11532 from lafirest/fix/frame_parse
fix(frame): improve some error reasons when parsing invalid packet
2023-08-30 19:10:45 +08:00
firest 804443ba40 chore: fix typos 2023-08-30 11:22:09 +08:00
Ilya Averyanov 2ef60db147
Merge pull request #11481 from savonarola/0817-simplify-async-responses
chore(ft): refactor async reply mechanism
2023-08-30 01:20:09 +03:00
Ilya Averyanov 54ac4a8527 chore(ft): tidy up the code according to the review 2023-08-29 23:21:36 +03:00
Andrew Mayorov 0e770bdc95
test: switch `emqx_broker_SUITE` to use new cth tooling 2023-08-29 22:05:36 +04:00
Andrew Mayorov 3268093881
feat(cth): add module-level documenation 2023-08-29 22:05:36 +04:00
Andrew Mayorov f57d16ba13
feat(cthsuite): add function to determine workdir of testrun
In a deterministic fashion, to lift the burden of undestanding where
the testrun's data should go from the test writer.
2023-08-29 21:34:01 +04:00
Andrew Mayorov 063d6200c8
feat(router): enable using 2 tables to store routes
Instead of a single unified table, to reap the benefits of cheap
`ets:lookup/2` per regular topic subscription route.

Change configuration option naming to reflect the change: user now has
an ability to choose _storage schema_.
2023-08-29 17:41:59 +04:00
Andrew Mayorov 270fd107b2
feat(triesearch): allow reusing filter tokenization result
As a kind of micro-optimization.
2023-08-29 17:31:26 +04:00
zhongwencool 4e7ba5d35f chore: ciphers should allow empty space(comma_separated_binary) 2023-08-29 11:34:40 +08:00
firest 6722722522 fix(frame): improve some error reasons when parsing invalid packet 2023-08-29 10:53:20 +08:00
SergeTupchiy c04f770118
Merge pull request #11506 from SergeTupchiy/EMQX-10274-do-not-download-empty-trace-log
fix(emqx_trace): don't download empty trace log file
2023-08-28 22:17:00 +03:00
Serge Tupchii 0072749143 fix(emqx_trace): don't download empty trace log file
Closes: EMQX-10274
2023-08-28 19:25:55 +03:00
Thales Macedo Garitezi f988de4ff4 feat(purge): clear delayed messages 2023-08-28 10:23:21 -03:00
Thales Macedo Garitezi 9bded07834 feat: cluster purge
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-08-28 10:23:20 -03:00
Andrew Mayorov 8d2ebdea7e
fix(router): generalize config option and make effects visible 2023-08-28 14:07:46 +04:00
Andrew Mayorov e85789306b
chore(router): drop test-only `has_routes/1`
Seems that it's too much support work only for test purposes, where
`lookup_routes/1` is nearly as usable.
2023-08-28 14:07:46 +04:00
Andrew Mayorov 5d51687dbf
feat(router): add unified routing table config option 2023-08-28 14:07:45 +04:00
Andrew Mayorov 33e5e1ba57
feat(router): add unified routing table 2023-08-28 14:07:44 +04:00
Andrew Mayorov 166375a000
fix(topicidx): make `get_record/2` simpler to use in concurrent env
The mechanic of `emqx_topic_index` cannot really guarantee atomicity
of reading records associated with index matches, so instead it's
probably better to make the user aware of that lack of this guarantee.
2023-08-28 13:52:59 +04:00
Andrew Mayorov 84e40fb6fe
test(trie): add more involved route add + delete case
That shows how current local trie implementation breaks because
of lack of refcounting.
2023-08-28 13:52:59 +04:00
Andrew Mayorov dcb63440bc
refactor(mgmt): avoid dealing with router tab directly
Instead, contain all this behind `emqx_router` module interface.
2023-08-28 13:52:59 +04:00
Andrew Mayorov 9eccfa0909
refactor(router): isolate cleanup logic in router module 2023-08-28 13:52:59 +04:00
Ilya Averyanov 39a48179ea chore(emqx_channel): use macros for reply construction 2023-08-28 11:57:44 +03:00
Ilya Averyanov b8cacd2833 chore(ft): add tests for async reply registry 2023-08-28 11:57:44 +03:00
Andrew Mayorov b74ff10705
Merge pull request #11517 from keynslug/ft/topic-index-v3
feat(topicidx): iterate on trie search implementation
2023-08-28 12:24:16 +04:00
Ilya Averyanov b0d4a22aa8 chore(ft): refactor async reply mechanism 2023-08-27 21:54:23 +03:00
Thales Macedo Garitezi 33a0048155 refactor: move logic to `ensure_iterator` 2023-08-25 15:24:53 -03:00
Thales Macedo Garitezi f15f59650d test: rm obselete workaround code 2023-08-25 13:49:33 -03:00
Andrew Mayorov d5cff533e3
chore(topicidx): drop TODO comment 2023-08-25 16:44:03 +04:00
Andrew Mayorov ca59a87d47
chore(topicidx): refine example of wildcard compare 2023-08-25 16:44:03 +04:00
Zaiming (Stone) Shi 558402a68e
chore(topic_index): add topic validation 2023-08-25 16:44:03 +04:00
Andrew Mayorov 33ed53bb6a
refactor(topicidx): simplify `compare/3` further
It's possible to emit seek instruction just once, on the way back
out of the `compare/3` stack.
2023-08-25 16:34:45 +04:00
Andrew Mayorov fc37d235c7
refactor(topicidx): simplify seek instructions
Which also avoids comparing filter tail repeatedly when evaluating
it.
2023-08-25 15:24:06 +04:00
zhongwencool c8af5a702c test: remove connection_closed from test 2023-08-25 15:53:35 +08:00
zhongwencool fc1738188e fix: packets_connack_sent is not incremented if the ack_flag field in the CONNACK packet is non-zero 2023-08-25 15:18:34 +08:00
Andrew Mayorov cf45e80c71
feat(topicidx): iterate on trie search implementation
This improves matching performance and decreases GC pressure on
synthetic workloads.
2023-08-25 01:47:11 +04:00
Kjell Winblad 9fe5080705 build: bump versions before release 2023-08-24 16:46:27 +02:00
Zaiming (Stone) Shi 62423b0b12 refactor(topic_index): less special handling for leading $ words 2023-08-24 13:30:10 +02:00
Zaiming (Stone) Shi a30d87e14f refactor(topic_index): remove more unnecessary next calls
also avoid using records (setelement) for recursive return values
2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi a1e6635614 refactor(topic_index): no forced ceiling entry in index table 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi f4c8c6be55 refactor(topic_index): optimize trie-search performance 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi 6b152b3cb7 test: add more debug output 2023-08-24 12:25:04 +02:00
zhongwencool 2322b27542
Merge pull request #11497 from zhongwencool/prometheus-metrics
Prometheus metrics
2023-08-24 12:06:32 +08:00
zhongwencool 8d2be27ade chore: add changlog for 11466 2023-08-24 06:46:50 +08:00
zhongwencool 84dd3b83bb fix: hocon converter not working when make_serializable is true(check_plain) 2023-08-24 06:46:47 +08:00
zhongwencool d287b736ec chore: add changelog for 11497 2023-08-23 17:02:30 +08:00
zhongwencool 6bb15541c7 feat: rename olp metrics to overload_protection metrics 2023-08-23 15:25:54 +08:00
zhongwencool b9a97923c4 feat: add some authz/connection missing metrics 2023-08-23 14:47:53 +08:00
zhongwencool ae10415fc3 feat: disabled the Erlang VM Prometheus exporter by default to improve performance and security 2023-08-23 09:03:22 +08:00
Thales Macedo Garitezi ee2897e5de test(refactor): move test to integration tests dir 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi c1f49abad2 test: fix inter-suite flakiness 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi dad27091be test: rm custom option 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 65085d012b refactor: rename fn 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi c46b8de938 test: remove unused things, refactor some functions 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi e8c73b06e1 docs: add comment about future test failure 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 6de0bbe76a test(refactor): always allocate listeners for emqx app 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi a15405a800 test: fix assertions 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi c74abe79d0 refactor: reduce arity 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 021755b82b refactor: rm iterators from DS `#session{}` record 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi e4e88ebf36 test: add scenario for node stopping midway during subscribe 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi c28c6d1b7e fix: ensure iterator is opened 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 33ddbe80ad refactor: remove persistence leakeage from emqx_cm level 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi e8d7bb9a67 refactor: rename module 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 9463e271c0 feat(ds): open iterators when handling `SUBSCRIBE` packets
Fixes https://emqx.atlassian.net/browse/EMQX-9741
2023-08-22 13:29:15 -03:00
Ilya Averyanov ed67303281
Merge pull request #11382 from savonarola/0801-auth-refactor
Auth refactor stage 1
2023-08-21 00:24:15 +03:00
Ivan Dyachkov e2b9d905d7
Merge pull request #11465 from id/0718-e5.2.0-alpha.3 2023-08-17 20:59:13 +02:00
Andrew Mayorov 75ed6aa8e7
Merge pull request #11396 from keynslug/ft/EMQX-10712/ruleeng-topic-index-vol-2
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching
2023-08-17 20:43:09 +04:00
Ivan Dyachkov a0c2fe8cc1 chore: e5.2.0-alpha.3 2023-08-17 17:55:56 +02:00
lafirest 5e448f5a02
Merge pull request #11446 from lafirest/refactor/calendar
refactor(calendar): refactor datetime-related code and remove redundant
2023-08-17 07:40:44 +08:00
Andrew Mayorov 5d79823891
perf(topicidx): preserve next key on the stack to reuse later
This should further optimize the number of `ets:next/2` calls
required for a single match query.
2023-08-17 01:13:19 +04:00
Thales Macedo Garitezi f03733ba6b
Merge pull request #11460 from thalesmg/integration-tests-dir-20230816
ci: add `integration_test` dir to `emqx` application
2023-08-16 17:47:14 -03:00
Ilya Averyanov 3943830eae chore(auth): remove unnecessary atom conversions 2023-08-16 20:50:35 +03:00
Ilya Averyanov 0dae040047 chore(auth): restore injected schema key position 2023-08-16 20:50:35 +03:00
Ilya Averyanov db31e5f0d9 chore(auth): tidy up the code 2023-08-16 20:50:35 +03:00
Ilya Averyanov 67e06b3171 chore(auth): make schema injection be more universal 2023-08-16 20:41:05 +03:00
Ilya Averyanov f00f4568dc chore(auth): simplify schema injection 2023-08-16 20:41:05 +03:00
Ilya Averyanov 8a3e8ee528 chore(auth): add config propagation tests for authn 2023-08-16 20:41:05 +03:00
Ilya Averyanov 128aa19d47 chore(auth): improve emqx_config_handler to replace adhoc config updates 2023-08-16 20:41:05 +03:00
Ilya Averyanov 8213aa42c9 chore(auth): move authn leftovers to the authn app 2023-08-16 18:52:14 +03:00
Ilya Averyanov ca8c1e3ef8 chore(auth): removed direct usages of authn from emqx core app 2023-08-16 18:52:14 +03:00
Ilya Averyanov ce2b159022 chore(auth): make pre_hook_authenticate be part of emqx_access_control domain 2023-08-16 18:52:14 +03:00
zhongwencool e6f0dead9e
Merge pull request #11445 from zhongwencool/remove-os-mon-from-windows
fix: remove os_mon application in Windows release
2023-08-16 23:30:24 +08:00
Thales Macedo Garitezi a6a023eff7 ci: add `integration_test` dir to `emqx` application
This gives us a place where to put tests that exercise multiple umbrella application,
which is more closely related to `emqx`, but in a way that doesn't affect the standalone
app tests.
2023-08-16 11:48:47 -03:00
Andrew Mayorov c84ee66d80
Merge pull request #11450 from keynslug/fix/dirty-testsuite-flapping
fix(test): switch `emqx_flapping_SUITE` to cth tooling
2023-08-16 17:20:31 +04:00
zhongwencool 78356d2625 chore: start os_mon in handle_continue/2 2023-08-16 20:54:53 +08:00
zhongwencool 8b23ee86b3 fix: hide cpu_status if os_check is not supported 2023-08-16 20:49:28 +08:00
zhongwencool b817e03c08 fix: start os_mon application temporary 2023-08-16 20:49:27 +08:00
zhongwencool 848eb7e3c3 fix: remove os_mon application in Windows release 2023-08-16 20:49:27 +08:00
zhongwencool b733adca06 fix: allow empty cacertfile pem 2023-08-16 20:15:21 +08:00
zhongwencool 1af0407c48 fix: don't crash when debug huge payload 2023-08-16 15:35:09 +08:00
firest b08102269a refactor(calendar): refactor datetime-related code and remove redundant 2023-08-16 10:55:46 +08:00
Andrew Mayorov 0b21b87296
fix(test): switch `emqx_flapping_SUITE` to cth tooling 2023-08-15 21:29:42 +04:00
Andrew Mayorov 5560e6ed4f
fix(cthsuite): also prevent `emqx` from loading default conf 2023-08-15 21:25:39 +04:00
Thales Macedo Garitezi 5d707c8b7d
Merge pull request #11403 from thalesmg/gcp-produ-attr-20230728
feat(gcp_producer): add support for defining message attributes and ordering key
2023-08-15 13:22:31 -03:00
Zaiming (Stone) Shi fa9e9f3348
Merge pull request #11442 from zmstone/0814-sync-release-52-to-master
0814 sync release 52 to master
2023-08-15 16:21:55 +02:00
Andrew Mayorov e39bbf4c49
chore(topicidx): add more descriptive comments and specs
To (hopefully) better illustrate what is happening there.
2023-08-15 16:55:48 +04:00
Thales Macedo Garitezi 82b8538041 feat(gcp_producer): add support for defining message attributes and ordering key
Fixes https://emqx.atlassian.net/browse/EMQX-10652
2023-08-14 10:33:17 -03:00
lafirest 1bda8020f5
Merge pull request #11436 from lafirest/feat/banned_clear
feat(banned): add a new API used to clear all banned data
2023-08-14 20:48:43 +08:00
Andrew Mayorov 9a249e4b01
test(topicidx): increase test coverage 2023-08-14 15:36:58 +04:00
Andrew Mayorov fd0986071c
perf(topicidx): implement fast-forwarding prefixes
This should give less `ets:next/2` calls in general and much less
when index has relatively small number of long non-wildcard topics.
2023-08-14 15:36:58 +04:00
Andrew Mayorov 0c7bdbdab4
test(topicidx): add property test
Co-Authored-By: JianBo He <heeejianbo@gmail.com>
2023-08-14 15:36:57 +04:00
Andrew Mayorov 48a50c9137
fix(topicidx): fix missing matches when 'a/b' and 'a/b/#' both exist
Thanks to @HJianBo for spotting this issue. The approach to fix it
is different though: we try to keep the "recurrency branch factor"
to a minimum, instead introducing new condition for the case when
filter does not match, but iteration with `ets:next/2` is not yet
finished for the prefix.

Co-Authored-By: JianBo He <heeejianbo@gmail.com>
2023-08-14 15:36:57 +04:00
Andrew Mayorov 6a13406363
fix(topicidx): use custom topic words to keep required ordering
Otherwise, topic with empty tokens (e.g. `a/b///c`) would have
some of their tokens ordered before `#` / `+`, because empty
token was represented as empty atom (`''`).
2023-08-14 15:36:57 +04:00
Andrew Mayorov 28bcb394d1
fix(topicidx): allow to return matches unique by record id 2023-08-14 15:36:57 +04:00
Andrew Mayorov 8feda315f6
feat(index): add topic index facility
Somewhat similar to `emqx_trie` in design and logic, yet built on
top of a single, potentially pre-existing table.
2023-08-14 15:36:54 +04:00
Zaiming (Stone) Shi 970ff28200 Merge remote-tracking branch 'origin/master' into release-52 2023-08-14 11:50:40 +02:00
zhongwencool 41180170f4 fix: update error msg 2023-08-14 15:45:57 +08:00
zhongwencool e10f9e5e9b feat: change mqtt.max_packet_size type from string to bytesize 2023-08-14 10:10:46 +08:00
lafirest b8d3dcf59a
Merge pull request #11424 from lafirest/fix/epoch_maximum_val
fix(datetime): make sure the epoch is not larger than the maximum supported value
2023-08-11 20:37:56 +08:00
firest 749c2d075f chore: update changes && bump app version 2023-08-11 17:57:27 +08:00
firest bc417a6764 fix(datetime): make sure the epoch is not larger than the maximum supported value 2023-08-11 17:57:21 +08:00
firest 2fc0468e0c feat(banned): add a new API used to clear all banned data 2023-08-11 17:25:49 +08:00
Ivan Dyachkov 7852af896c chore: e5.2.0-alpha.1 2023-08-11 11:16:30 +02:00
Ivan Dyachkov bab05b6613
Merge pull request #11433 from id/0811-ci-upgrade-push-helm-action 2023-08-11 11:03:00 +02:00
Ivan Dyachkov b6084b0001 chore: bump apps/emqx version 2023-08-11 09:19:51 +02:00
lafirest 4eaced241d
Merge pull request #11423 from lafirest/chore/limiter_utils
fix(limiter): simplify the code of the limiter
2023-08-11 15:17:42 +08:00
Ivan Dyachkov 801c93dd46 chore: v5.1.5-build.3 2023-08-10 12:56:28 +02:00
Ivan Dyachkov 6ead09b28d chore: merge 'upstream/master' 2023-08-10 12:56:00 +02:00
firest 6dbddfb089 fix(limiter): simplify the code of the limiter
move runtime code out from the schema
2023-08-10 18:20:05 +08:00
zhongwencool aa798e531e
Merge pull request #11409 from zhongwencool/opentelemetry-metrics
feat: support opentelemetry metrics
2023-08-10 14:35:25 +08:00
zhongwencool 5a4dd3a5e5 fix: truncate large payload 2023-08-10 09:35:12 +08:00
zhongwencool d5fe891961 fix: don't be kill when send large payload when log is debug 2023-08-10 09:22:30 +08:00
Thales Macedo Garitezi bb8ce68c2e docs: Generate changelog for v5.1.5-patch.2 2023-08-09 16:38:28 -03:00
Thales Macedo Garitezi 95df958f10 docs: Generate changelog for v5.1.5-patch.1 2023-08-09 13:18:09 -03:00
Thales Macedo Garitezi cce0d144f9 Merge branch 'master' into release-v515-patch1-20230809 2023-08-09 13:08:43 -03:00
zhongwencool 3933227636 feat: support opentelemetry metrics 2023-08-09 20:23:40 +08:00
lafirest bef8eddad3
Merge pull request #11405 from lafirest/fix/kal_kialo
fix(calendar): make date parse error reason more sense
2023-08-09 14:42:08 +08:00
Thales Macedo Garitezi fdfdb105f0 docs: Generate changelog for v5.1.5 2023-08-08 10:33:34 -03:00
Thales Macedo Garitezi a1ffd0263d fix(cacerts): add `cacerts` back as deprecated to allow upgrading from older versions
See https://github.com/emqx/emqx/issues/11380
2023-08-08 10:13:13 -03:00
firest 9f38f5f26a fix(calendar): make date parse error reason more sense 2023-08-08 14:28:38 +08:00
lafirest 2b03436552
Merge pull request #11392 from lafirest/feat/ldap_authz
feat(ldap-authz): integrate the LDAP authorization
2023-08-07 11:12:05 +08:00
zhongwencool 665695a977
Merge pull request #11388 from SergeTupchiy/EMQX-10703-fix-replicant-crash-when-core-terminates-abnormally
fix: increase emqx_router_sup restart intensity
2023-08-06 14:40:32 +08:00
Serge Tupchii 2b7798608d fix(emqx_router_helper): don't cleanup down node on a replicant
The cleanup on a replicant node is redundant, as Mria would delegate this delete op
to a core node (via RPC), and the core node is expected to receive the same
`nodedown` message and process it.
2023-08-04 16:55:36 +03:00
SergeTupchiy 2044326734
Merge pull request #11390 from SergeTupchiy/EMQX-10661-improve-perf-high-latency-cluster-network
EMQX-10661 improve performance on high latency cluster network
2023-08-04 16:41:08 +03:00
firest b24a9d5343 chore(ldap-authz): update apps version && changes 2023-08-04 18:14:09 +08:00
Serge Tupchii f276ea9e91 fix: increase emqx_router_sup restart intensity
The goal is to tolerate occasional crashes that can happen under relatively normal conditions
and don't seem critical to shutdown the whole app (emqx).
For example, mria write/delete call delegated from a replicant to a core node may fail,
if the core node is being stopped / restarted / not ready.

Fixes: EMQX-10703, #11310
2023-08-04 10:49:59 +03:00
firest 7055eafb91 fix(ldap): fix license date and some minor problems 2023-08-04 11:01:13 +08:00
Serge Tupchii 466fe7e009 perf: add broker_pool_size, generic_pool_size and channel_cleanup_batch_size config options
Tuning these options can improve performance if cluster interconnect network latency is high.

Fixes: EMQX-10661
2023-08-03 19:44:00 +03:00
Andrew Mayorov 39bb9d5136
chore: bump ekka to 0.15.10
Which sports mria 0.6.0 with dirty activities support.
2023-08-03 11:38:53 +04:00
Ivan Dyachkov 0f161a81c3 chore: v5.1.4 2023-08-01 09:20:46 +02:00
Thales Macedo Garitezi 7687770821 fix(tls): remove `cacerts` config for now
Fixes https://github.com/emqx/emqx/issues/11370

Related: https://github.com/emqx/emqx/pull/11371
2023-07-31 10:44:05 -03:00
Thales Macedo Garitezi a3be1529c6 fix(schema): ensure enterprise module is correctly loaded
Fixes https://emqx.atlassian.net/browse/EMQX-10654

Despite loading the application in `nodetool`, we need to invoke `:module_info()` to force
the module to be actually loaded, especially when it's called in `call_hocon` to generate
the initial configurations.  Otherwise, it'll fail to list all the bridge schemas.
2023-07-28 16:27:43 -03:00
Thales Macedo Garitezi 2ee1aa6d60
Merge pull request #11347 from thalesmg/fix-ocsp-path-encoding-20230725
fix(ocsp): URL encode request path
2023-07-28 09:01:39 -03:00
Ivan Dyachkov d51b5b739b chore: v5.1.3 2023-07-27 17:00:20 +02:00
Ivan Dyachkov 5ac01c9b85
Merge pull request #11360 from id/0727-sync-release-51-to-master 2023-07-27 16:58:09 +02:00
Thales Macedo Garitezi ee3d002fca
Merge pull request #11329 from thalesmg/aeh-bridge-20230720
feat: add Azure Event Hub Producer bridge
2023-07-27 11:18:40 -03:00
Ivan Dyachkov cbfca8c043 chore: merge master into release-51 2023-07-27 15:19:57 +02:00
Ivan Dyachkov 50a0900d92 chore: e5.1.1 2023-07-27 12:18:03 +02:00
Thales Macedo Garitezi d3d52695d5 feat(tls): automatically add `cacerts` to client opts
`public_key:cacerts_get/0` was introduced in OTP 25 and allows us to
load the system trusted CA certificates.

https://www.erlang.org/doc/man/public_key.html#cacerts_get-0
2023-07-25 17:43:33 -03:00
Thales Macedo Garitezi d0deaca00d fix(ocsp): URL encode request path
Fixes https://emqx.atlassian.net/browse/EMQX-10624
2023-07-25 17:39:38 -03:00
Serge Tupchii 77184c00ff chore(ekka): Bump version to 0.15.9 2023-07-25 19:29:18 +03:00
Ivan Dyachkov f9d3d3325b chore: e5.1.1-rc.1 2023-07-25 16:34:25 +02:00
Ilya Averyanov deaac9bd73 fix(authz): correctly identify qos of subscribe actions 2023-07-25 15:20:52 +03:00
firest 8439ce0e84 chore: update app version && reboot_lists 2023-07-24 19:08:41 +08:00
Andrew Mayorov 81793c31fc
Merge pull request #11315 from keynslug/ft/EMQX-9593/peek-mqueue-info
refactor(session): allow peeking at mqueue less intrusively
2023-07-24 09:57:05 +02:00
Ivan Dyachkov 29432009db chore: 5.1.1-alpha.4 2023-07-21 18:42:14 +02:00
ieQu1 485e495b98 chore(ekka): Bump version to 0.15.8 (mria 0.5.10) 2023-07-21 18:31:02 +02:00
Ivan Dyachkov c2c3da073f chore: v5.1.2 2023-07-21 13:26:29 +02:00
Ivan Dyachkov 243b8f5b67 chore: merge 'upstream/master' into v5.1.2 2023-07-21 13:25:46 +02:00
zhongwencool 520ae0e9d3 chore: remove zone desc from document 2023-07-21 15:51:21 +08:00
zhongwencool d3e4fda21d fix: hocon_pp quote string for special char 2023-07-21 12:31:22 +08:00
Andrew Mayorov e1e4c64a30
refactor(session): allow peeking at mqueue less intrusively 2023-07-20 21:21:30 +02:00
Andrew Mayorov f2a32e8ed2
Merge pull request #11304 from keynslug/ft/EMQX-9593/ds-session-connect
refactor(session): minimize `emqx_session` interface surface
2023-07-20 14:33:21 +02:00
SergeTupchiy 8480a26b66
Merge pull request #11065 from SergeTupchiy/EMQX-9992-fix-async-cleanup-error-logs
fix: avoid logging unnecessary errors in async cleanup functions
2023-07-20 12:20:57 +03:00
Serge Tupchii ff4119e8b3 chore(emqx_config): fix typo in the log message 2023-07-19 16:58:21 +03:00
ieQu1 57e39f42c5
Merge pull request #11294 from ieQu1/mria-fixes
Mria fixes
2023-07-19 15:10:42 +02:00
Andrew Mayorov dd31487b4a
refactor(session): drop `is_session/1` helper as useless
This is a part of effort to minimize `emqx_session` module
interface to simplify adding alternative session implementations.
2023-07-19 14:55:26 +02:00
Andrew Mayorov 0cd2351134
refactor(session): hide `no_local` logic behind enqueue / deliver
This is a part of effort to minimize `emqx_session` module
interface to simplify adding alternative session implementations.
2023-07-19 14:55:26 +02:00
Ivan Dyachkov f29a9ed9d5
Merge pull request #11301 from id/0719-sync-release-51-to-master 2023-07-19 14:21:26 +02:00
Andrew Mayorov 92166f287a
Merge pull request #11293 from keynslug/fix/EMQX-9591/phase-out-persistence
chore(ps): phase out existing session persistence mechanism
2023-07-19 10:36:46 +02:00
Ivan Dyachkov b7f0f7b32d chore: merge master into release-51 2023-07-19 09:28:05 +02:00
zhongwencool 402f75592e chore: move rpc call from emqx_management_proto to emqx_conf_proto 2023-07-19 11:05:14 +08:00
zhongwencool 8a0f7bfd99 chore: update version bpapi.versions 2023-07-19 11:05:14 +08:00
ieQu1 ce76d4ab04 chore(ekka): Bump version to 0.15.7 (mria 0.5.9) 2023-07-18 21:45:00 +02:00
Ivan Dyachkov de4cc492d4 chore: e5.1.1-alpha.2 2023-07-18 18:55:24 +02:00
Andrew Mayorov 0a00c39282
chore(ps): phase out existing session persistence mechanism
* It's not really working currently in a cluster setup.
* It's not really working well.
* It would be much easier to integrate `emqx_durable_storage` w/o
  the need to support existing solution.
2023-07-18 15:46:04 +02:00
Thales Macedo Garitezi 4a889c0994 chore: bump ekka -> 0.15.6
https://github.com/emqx/erlang-rocksdb/releases/tag/1.8.0-emqx-1
2023-07-18 09:00:29 -03:00
Thales Macedo Garitezi 1fdcfba629 fix(shared_sub): restore support for `$queue/` shared subscription
Partially addresses: https://emqx.atlassian.net/browse/EMQX-4589

There's still a problem with the handling of shared groups that is
_not particular to the `$queue/` syntax_ and preexistent that the same
client cannot subscribe to _different groups_ that have the same
_topic filter_.
2023-07-17 16:41:24 -03:00
zhongwencool f853f3d3bd
Merge pull request #11271 from zhongwencool/fix-precent-check
fix: keep percentage type's range is 0%~100%
2023-07-14 22:07:41 +08:00
Serge Tupchii 950d5edc41 fix: avoid logging unnecessary errors in async cleanup functions
Cleanup functions that access ETS tables may fail with `badarg` error during EMQX shutdown.
They are called asynchronously by `emqx_pool` workers and accessed ETS tables
may be already destroyed as their owners are shut down.
This fix catches ETS `badarg` errors before they can be caught and logged by `emqx_pool`.

Fixes: EMQX-9992
2023-07-14 14:22:43 +03:00
firest fea73cf17d fix(log): fix the `pubrel` typo in the log 2023-07-14 17:19:50 +08:00
zhongwencool 11ee6abf02 fix: keep precent range is 0%~100% 2023-07-14 16:48:50 +08:00
Thales Macedo Garitezi 06010f7ca9 Merge remote-tracking branch 'origin/release-51' into sync-r51-20230712 2023-07-12 16:47:45 -03:00
SergeTupchiy fce3060634
Merge pull request #11251 from SergeTupchiy/EMQX-10392-cluster-topology-api
Add cluster topology HTTP API endpoint
2023-07-11 17:45:00 +03:00
Serge Tupchii 19de10be7c feat(emqx_management): add /cluster/topology HTTP API endpoint
The endpoint shows Mria RLOG cluster topology info:
connections between core and replicant nodes.

Closes: EMQX-10392
2023-07-11 16:25:36 +03:00
firest 211486a946 fix(ws): fix MQTT packets may be reversed in the WS
Fix when a WS packet contains many MQTT packets, the order of MQTT packets will be reversed
2023-07-11 18:40:06 +08:00
Andrew Mayorov 21fbf79c29
Merge pull request #11191 from keynslug/ft/EMQX-9591/msg-persistence-minimal
feat(ds): hardwire emqx_durable_storage message persistence
2023-07-10 21:05:04 +02:00
Andrew Mayorov d91ab7dec9
test(ds): make `emqx_persistent_messages_SUITE` setup cleaner
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-07-10 18:12:16 +02:00
Ivan Dyachkov fb96c1e20d chore: bump to e5.1.1-alpha.1 2023-07-10 17:25:07 +02:00
Ilya Averyanov 58a83739b8
Merge pull request #10986 from savonarola/ft/EMQX-9523/config-api
feat(ft-api): provide configuration API
2023-07-10 15:32:30 +03:00
Ilya Averyanov 12e237c36d
Merge pull request #11132 from savonarola/0531-authz-qos-retain
feat(authz): use extensible map format for actions in authz rules
2023-07-10 15:32:02 +03:00
Andrew Mayorov 576eab9717
Merge pull request #11193 from keynslug/fix/emqx-ft-test-flaps
fix(ft-test): use new cth tooling in emqx_ft_storage_fs_SUITE
2023-07-10 11:55:16 +02:00
JianBo He f144bb0571
Merge pull request #11226 from zhongwencool/listener-conf-enabled-to-enable
fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable
2023-07-10 17:15:24 +08:00
zhongwencool 802a50601a chore: add comment for init_loader_done 2023-07-10 12:03:36 +08:00
zhongwencool c4ba558ee3 fix: unset config_loader in emqx's env when stop emqx app 2023-07-10 11:06:08 +08:00
zhongwencool d661b10355 fix: emqx CI failed 2023-07-10 10:34:36 +08:00
zhongwencool c0ee47dc08 chore: add changelog for 11226 pr 2023-07-10 10:34:36 +08:00
zhongwencool d781346efc fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable 2023-07-10 10:34:36 +08:00
Andrew Mayorov 030beb3e6a
fix(ds): drop debug printouts in test suite 2023-07-07 21:56:20 +02:00
Andrew Mayorov daf4e86da1
feat(ds): add `ensure_shard/2` shortcut
And use it in `emqx_persistent_session_ds:init()` backend initialization
function.
2023-07-07 21:56:20 +02:00
Ilya Averyanov 77895f2555 feat(authz): fix typos and style for Retain & QoS authz feature
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-07-07 19:40:34 +03:00
Ilya Averyanov 7de26a1776 feat(authz): use extensible map format for actions in authz rules
* support authorization on retain, qos fields
* refactored authz tests heavily
2023-07-07 19:40:34 +03:00
Ilya Averyanov e0353ab750 fix(ft-api): update FT API tests to use emqx_cth_suite 2023-07-07 19:15:42 +03:00
Andrew Mayorov 97b6c43062
fix(ft-test): use new cth tooling in `emqx_ft_storage_fs_SUITE`
In attempt to battle test flakiness.
2023-07-07 14:00:33 +02:00
Andrew Mayorov 7e76914599
feat(ds): hardwire emqx_durable_storage message persistence
Only message persistence is currently implemented, irrespectively of
whether there are persistent sessions around or not.
2023-07-07 13:46:12 +02:00
zhongwencool 352d818e3b chore: update hocon to 0.39.13 2023-07-07 16:39:26 +08:00
zhongwencool 221f6eba06 fix: bad tnx-id when rejoin cluster 2023-07-07 12:25:34 +08:00
firest a1d2b2ca51 chore: bump app versions 2023-07-07 11:17:54 +08:00
Stefan Strigler 9bcabe0a9e
Merge pull request #11184 from sstrigler/EMQX-10270-e-5-1-0-missing-value-range-validation-for-the-max-packet-size-of-mqtt-settings
fix: limit mqtt max_packet_size to 256MB
2023-07-06 10:20:24 +02:00
Ilya Averyanov 9af0030538 chore: bump to v5.1.1 2023-07-05 18:43:45 +03:00
Stefan Strigler 263890be47 fix: limit mqtt max_packet_size to 256MB 2023-07-04 16:03:11 +02:00
Andrew Mayorov 8cc7022760
Merge pull request #11120 from ft/EMQX-10329/tooling
test: add tooling to make testruns more isolated and predictable
2023-07-04 14:36:39 +02:00
zhongwencool f2c5f4932c
Merge pull request #11148 from lafirest/fix/mcall_error_report
fix(emqx_conf): don't sync the MFA to a leaved node
2023-07-04 19:56:52 +08:00
Andrew Mayorov 9ed75085b7
chore: drop dead commented out code 2023-07-04 13:20:04 +02:00
Andrew Mayorov c7ba300efc
fix(cthsuite): use correct type for "smth serializable as HOCON" 2023-07-04 13:20:04 +02:00
Andrew Mayorov d568d43fe4
test(bridge): switch `emqx_bridge_api_SUITE` to use new tooling 2023-07-04 13:20:03 +02:00
Andrew Mayorov 2c50e61cee
fix(cth): reduce amount of logging + streamline logging
Also make sure that code running on `emqx_cth_cluster` nodes don't
crash when using `ct:pal` and friends.
2023-07-04 13:20:03 +02:00
Andrew Mayorov 9bda4192e5
fix(cthsuite): disable default `acl.conf` authz source
So that users of `emqx_cth_suite` won't have to deal with providing
correct filepath each time they start `emqx_conf` and/or `emqx_authz`
up.
2023-07-04 13:20:03 +02:00
Andrew Mayorov 24a16ea247
fix(cthsuite): bump to hocon 0.39.11 with atom prettyprint fix 2023-07-04 13:20:03 +02:00
Andrew Mayorov 6a7254f55e
fix(cthcluster): match correct driver in `stop_node/2`
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2023-07-04 13:20:02 +02:00
Andrew Mayorov fe0b8cfbaf
test(conf): respect `init_config_load_done` in emqx_conf itself
This should make task of configuring various applications (emqx_conf
/ emqx) during testsuites setup much simpler.
2023-07-04 13:20:02 +02:00
Andrew Mayorov f76f3b77d7
test: add tooling to make testruns more isolated and predictable
Also showcase their usage in some of the existing testsuites.
2023-07-04 12:53:39 +02:00
zhongwencool d509c47344 feat: add conf load api 2023-07-04 14:17:11 +08:00
Andrew Mayorov 0f25df3aa8
perf(router): employ `maps:foreach/2` instead 2023-07-03 15:01:47 +02:00
Andrew Mayorov 57da71eb1d
perf(broker): avoid usorting aggregated routes in a loop 2023-07-03 15:01:33 +02:00
zhongwencool e26c30ff0c feat: refactor dashboard https ssl_options 2023-06-29 10:49:39 +08:00
firest 8e68af14ef chore: bump ekka version 2023-06-29 01:45:47 +00:00
Andrew Mayorov 30d78aaac2
Merge pull request #11152 from keynslug/test/ft-prop-aggressive
test(ft): make proptests even less aggressive
2023-06-27 17:44:54 +02:00
Andrew Mayorov ff327609db
test(ft): generate segments manually in proptests
In order to avoid spamming proptest logs with HUGE type instances,
and make proptests themselves simpler. Proper now should refuse
shrinking altogether, instances that cause prop function to timeout
do not shrink well for some reason, making proptests stuck.
2023-06-27 17:13:23 +02:00
Zaiming (Stone) Shi 9b6ed09513 refactor: call hocon_tconf:check_plain/4 directly for example conf check
the attemp to re-use emqx_hocon:check failed because
hocon_tconf:check_plain/3 and check_plain/4 behaves slightly different
for unknown root fields.

To keep the old logic unchanged, for example config checks, we call
check_plain/4 directly
2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi a069b351fd test: add test script to verify config example files 2023-06-27 14:23:18 +02:00
Serge Tupchii 116675e86f chore: bump ekka to 0.15.4 2023-06-27 13:24:49 +03:00
lafirest f2e3446e29
Merge pull request #11135 from lafirest/pref/calendar_timeoffset
perf(rule_engine): improve the time offset parser
2023-06-26 18:54:00 +08:00
lafirest 850d642ec2
Merge pull request #11108 from ieQu1/emqx-ds
Durable storage
2023-06-25 11:36:34 +08:00
firest c8c2793944 perf(rule_engine): improve the time offset parser 2023-06-24 13:05:27 +00:00
Thales Macedo Garitezi 7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
Zaiming (Stone) Shi c58a98954b Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-22 11:05:51 +02:00
ieQu1 efed7cdc2f chore(ds): behavior -> behaviour 2023-06-22 10:20:18 +02:00
Andrew Mayorov 86d787eced
chore: bump hocon to 0.39.10
Which comes with a fix for slightly more user-friendly validation error
messages.
2023-06-21 21:25:43 +02:00
Zaiming (Stone) Shi 7cf8a6c892 chore: bump app vsns 2023-06-21 16:36:51 +02:00
Zaiming (Stone) Shi 5fa87091c4 Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-21 16:08:41 +02:00
Zaiming (Stone) Shi dddccfdc5c chore: bump to v5.1.0 official cut 2023-06-20 17:09:17 +02:00
Zaiming (Stone) Shi 728656d4f6 chore: bump to e5.1.0 official cut 2023-06-20 17:08:18 +02:00
Zaiming (Stone) Shi f95787d229 test: make possible to test core/replica locally 2023-06-20 12:42:00 +02:00
zhongwencool 856de78698 chore: remove the is_list guard 2023-06-20 08:39:23 +08:00
zhongwencool 093cdab838 chore: to_integer to make sure integer is converted 2023-06-20 08:39:23 +08:00
zhongwencool 07172e42f0 test: integer CI check failed 2023-06-20 08:39:23 +08:00
zhongwencool a4be9b8281 chore: bump hocon to 0.39.9 2023-06-20 08:39:23 +08:00
zhongwencool 45963b6a95 fix: ip_port schema type crash 2023-06-20 08:39:23 +08:00
Zaiming (Stone) Shi ead511a128 chore: bump to 5.1.0-rc.1 2023-06-19 23:19:27 +02:00
ieQu1 86b78981c4
Merge pull request #11092 from ieQu1/mria-0.5.5
chore(mria): Bump version to 0.5.5
2023-06-19 23:13:36 +02:00
Thales Macedo Garitezi 01d758e4c0 fix(bpapi): fix `query_mode` type(spec)
The previous spec was wrong due to a mismatch between type declaration and usage in specs:
it would never be `dynamic`, and it was lacking the `simple_async`, `simple_sync` and
`no_queries` constructors.
2023-06-19 16:04:12 -03:00
ieQu1 f82c56ae16 chore(mria): Bump version to 0.5.5 2023-06-19 19:28:17 +02:00
JianBo He cc939d2bda
Merge pull request #11087 from zhongwencool/bump-v5.1.0.alpha.8
chore: bump to v5.1.0.alpha.8
2023-06-19 23:00:53 +08:00
JianBo He b0a6c18917 chore: bump vsn to 5.1.0-alpha.8 2023-06-19 22:59:05 +08:00
zhongwencool 1f1d61cef6
Merge pull request #11084 from zhongwencool/hide-plugins-from-cli
chore: hide plugins from conf load cli
2023-06-19 19:11:52 +08:00
zhongwencool 64612cab14 chore: bump to v5.1.0.alpha.8 2023-06-19 18:22:51 +08:00
zhongwencool a578d6fa1d chore: release e5.1.0-alpha.8 2023-06-19 17:19:03 +08:00
zhongwencool 803cd6c812
Merge pull request #11082 from emqx/release-51
sync release-51 to master
2023-06-19 15:12:37 +08:00
JimMoen 30ac6c4e48
Merge pull request #11074 from JimMoen/fix-share-sub-nl-protocol-error
fix: shared-sub with nl sub-option should cause protocol error
2023-06-19 10:39:08 +08:00
Andrew Mayorov 13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
2023-06-17 00:54:24 +02:00
JimMoen 178711f742
fix: shared-sub with nl sub-option should cause protocol error 2023-06-16 18:19:23 +08:00
Serge Tupchii b49b80d36c fix(emqx_schema): use non negative integer type for 'depth' SSL option
Closes: EMQX-10276
2023-06-16 10:59:39 +08:00
zhongwencool 9b56f116f4
Merge pull request #11062 from zhongwencool/log-file-rename 2023-06-15 20:51:52 +08:00
zhongwencool 3406a5d916
Merge pull request #11063 from zhongwencool/bump-to-alpha.7
chore: bump to e5.1.0-alpha.7
2023-06-15 19:22:37 +08:00
zhongwencool 9060a2dc05 chore: bump to e5.1.0-alpha.7 2023-06-15 19:05:13 +08:00
zhongwencool e6c2da8d19 feat: rename log.file.to to log.file.path 2023-06-15 18:24:21 +08:00
zhongwencool 84a5d0c3c1 fix: delete a default listener then recreate it, it's disabled 2023-06-15 17:19:11 +08:00
zhongwencool 0adbb0deec feat: try to save config when post_config_update failed not in init mfa status 2023-06-15 17:19:11 +08:00
zhongwencool c160301dbe fix: don't log enoent error 2023-06-15 17:06:09 +08:00
zhongwencool 7dade3a52c fix: tls_certfile_gc notice log don't print abspath 2023-06-15 17:06:09 +08:00
Serge Tupchii 64bbe21209 fix(emqx_schema): use non negative integer type for 'depth' SSL option
Closes: EMQX-10276
2023-06-15 11:22:58 +03:00
zhongwencool e42cc58694
Merge pull request #11050 from emqx/release-51
chore: sync release-51 back to master
2023-06-15 09:45:31 +08:00
Andrew Mayorov acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
2023-06-15 00:50:15 +03:00
zhongwencool 41004e86da
Merge pull request #10959 from qzhuyan/dev/william/remove-depre-quic-config
chore: remove deprecated QUIC listener config
2023-06-14 23:01:00 +08:00
Serge Tupchii 7882755114 chore: bump cowboy to 2.9.2
cowboy 2.9.2 uses ranch 1.8.1-emqx that fixes incompatible TLS options
2023-06-14 17:35:17 +03:00
Serge Tupchii 4425942030 fix(emqx_schema): move tls version gap validation to emqx_schema 2023-06-14 17:29:56 +03:00
Ivan Dyachkov 6d88cd7a20 fix(tls): fix incompatible tls options and issue with tls version gap 2023-06-14 17:29:56 +03:00
zhongwencool 267053cc35
Merge pull request #11045 from zhongwencool/hide-zones-authn-in-listeners
feat: hide zone/authn in listeners and remove listeners's authn api
2023-06-14 20:32:27 +08:00
zhongwencool 7668753f50 feat: hide zone/authn in listeners and remove /listeners/:id/authentication api 2023-06-14 15:17:12 +08:00
JianBo He 8ce0380704 chore: bump vsn to e5.1.0-alpha.6 2023-06-14 12:49:02 +08:00
zhongwencool 091c49277f fix: api_listener crash when setting max_connnections as string 2023-06-14 11:26:41 +08:00
zhongwencool 7f0f40cb58
Merge pull request #11034 from zhongwencool/mqtt-conf
feat:  move shared_subscription_strategy from broker to mqtt
2023-06-14 10:00:38 +08:00
Thales Macedo Garitezi b609792a90 fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10278

Since Pulsar client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 11:44:45 -03:00
zhongwencool 946de4a303 chore: dialyzer warning 2023-06-13 20:11:49 +08:00
zhongwencool 50041fdddf feat: hide broker rootkey and move broker.shared_subscription_strategy to mqtt.shared_subscription_strategy 2023-06-13 19:35:52 +08:00
William Yang 8fa743fa22 chore: mark deprecated QUIC listener config for 5.1 2023-06-13 06:00:57 +02:00
Thales Macedo Garitezi 5ef7102cba fix(listener_mgmt_api): use union member selector fn for better error messages (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10247
2023-06-12 16:48:13 -03:00
zhongwencool 58dc042279 chore: bump to e/v5.1.0-alpha.5 2023-06-13 00:06:54 +08:00
zhongwencool 0fc10ad239
Merge pull request #10984 from zhongwencool/reload-cli
feat: support reload cli
2023-06-12 17:21:29 +08:00
Zaiming (Stone) Shi 12b73ba1c0 Merge remote-tracking branch 'origin/master' into release-51 2023-06-12 10:35:47 +02:00
firest 947d99218b chore: upgrade emqtt to avoid sensitive data leakage in the debug log 2023-06-12 16:32:24 +08:00
zhongwencool 1550158ddd fix: update emqx_flapping process's state when emqx:update_config([flapping_detect],Conf) 2023-06-11 18:23:15 +08:00
Zaiming (Stone) Shi 654a9dcab5
Merge pull request #11006 from zmstone/0610-fix-quic-default-certs-path
fix(quic): environment variables as cert file prefix for quic listener
2023-06-11 11:52:56 +02:00
zhongwencool c8cf8d1def
Merge pull request #11009 from zhongwencool/persistent_session_store-default-value
chore: correct persistent_session_store's default value type to boolean
2023-06-11 17:26:58 +08:00
zhongwencool 072f6fac8d chore: update hocon to 0.39.8 2023-06-11 09:11:17 +08:00
zhongwencool 346b363cb5 test: config SUITE failed 2023-06-11 09:11:17 +08:00
zhongwencool 505d677685 chore: move reload_local_etc_config from emqx_conf to emqx_conf_cli 2023-06-11 09:11:17 +08:00
zhongwencool 6c6ff4ea1f chore: rename {mod} to module 2023-06-11 09:11:17 +08:00
zhongwencool cf9a207743 feat: check all config schema before loading 2023-06-11 09:11:17 +08:00
zhongwencool 21ffd958b4 feat: support authn merge_authenticators 2023-06-11 09:11:17 +08:00
zhongwencool 994db58cbf fix: load authz's default sources 2023-06-11 09:11:17 +08:00
zhongwencool c819ac27f4 feat: add conf reload cli 2023-06-11 09:11:17 +08:00
zhongwencool 599eb9fef5 chore: correct persistent_session_store's default value type to boolean 2023-06-10 19:36:05 +08:00
Zaiming (Stone) Shi 97850de524 Merge remote-tracking branch 'origin/release-51' into 0610-merge-release-51-to-master 2023-06-10 12:23:55 +02:00
Zaiming (Stone) Shi 9f135d1f2b fix(quic): environment variables as cert file prefix for quic listener 2023-06-10 12:23:20 +02:00
Serge Tupchii 87b57112df refactor: move tcp keepalive options helper to emqx_utils 2023-06-09 21:43:50 +03:00
Andrew Mayorov b930f4cc73
Merge pull request #10987 from keynslug/fix/EMQX-9257/sep-placeholder
refactor: tear `emqx_plugin_libs` application apart
2023-06-09 17:02:14 +02:00
Thales Macedo Garitezi 20a7ce64cd
Merge pull request #10919 from thalesmg/test-flakiness-20230601-a
test(pulsar_producer): attempt to fix flaky test
2023-06-09 09:30:30 -03:00
Andrew Mayorov 7d0abb6146
feat(emqx): add `emqx_topic:match_any/2` utility 2023-06-09 14:44:37 +03:00
SergeTupchiy e61b2100a3
Merge pull request #10676 from SergeTupchiy/EMQX-9203-config-backup
feat: implement configuration and user data export/import
2023-06-09 14:18:39 +03:00
Serge Tupchii e4d09d4ad4 feat: implement configuration and user data export/import CLI
Closes: EMQX-9203
2023-06-09 14:11:47 +03:00
Zaiming (Stone) Shi ee697831f6
Merge pull request #10926 from zmstone/0602-add-enable-as-an-alias-for-enabled
0602 add enable as an alias for enabled
2023-06-09 08:44:20 +02:00
Zaiming (Stone) Shi af5c6720de Merge remote-tracking branch 'origin/master' into release-51 2023-06-09 08:37:29 +02:00
Zaiming (Stone) Shi 47f826f99a chore: fix typo 2023-06-09 09:31:32 +08:00
Serge Tupchii 53ec8326b0 fix(emqx_listeners): convert authentication certs 2023-06-08 20:44:18 +03:00
Serge Tupchii 33fa053b9b test(emqx_common_test_helpers): keep EMQX ENV vars on a child node for its lifetime 2023-06-08 20:42:51 +03:00
Serge Tupchii 5f526d548a style: use double percent (%%) comments 2023-06-08 20:42:51 +03:00
Kjell Winblad 273dedf7a6
Merge pull request #10911 from kjellwinblad/kjell/bridge/atom_length_too_long/EMQX-9609
fix: friendly error message when creating bridges with too long names
2023-06-08 15:46:42 +02:00
Ivan Dyachkov abbba71191 fix(tls): issue when ssl listner is configured to use tls v1.3 only
clients could not connect due to incompatible tls options if ssl listner is configured to use tls v1.3 only
2023-06-08 15:13:22 +02:00
Zaiming (Stone) Shi b481994e74 chore: bump to version 5.1.0-alpha.4 2023-06-08 12:30:35 +02:00
Zaiming (Stone) Shi c1cf2365c2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-08 12:30:02 +02:00
Zaiming (Stone) Shi 641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51
0607 merge master to release 51
2023-06-08 09:22:21 +02:00
Zaiming (Stone) Shi 641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf
feat: don't hide enable in flapping detect conf
2023-06-07 23:04:56 +02:00
Zaiming (Stone) Shi ccd2589ff2 Merge remote-tracking branch 'origin/master' into release-51 2023-06-07 21:43:29 +02:00
Andrew Mayorov 39b0d41527
fix(bpapi): ignore dropped BPAPI module
It doesn't really breaks interoperability.
2023-06-07 21:33:47 +03:00
Zaiming (Stone) Shi aecea50564
Merge pull request #10902 from zmstone/0529-donot-copy-cluster-conf-from-newer-version
0529 donot copy cluster conf from newer version
2023-06-07 20:32:11 +02:00
Zaiming (Stone) Shi ed1ad4e684 chore(emqx_schema): add 'eanble' as an alias to 'enabled' fields 2023-06-07 20:19:48 +02:00
Thales Macedo Garitezi 105c26afb5
Merge pull request #10964 from thalesmg/fix-hocon-compact-errors-v50
fix(schema): avoid `function_clause` error when compacting errors
2023-06-07 14:00:24 -03:00
Andrew Mayorov 79d430cf03
refactor(pluglib): move `emqx_plugin_libs_proto_v1` into emqx app
And rename it to `emqx_metrics_proto_v1` in the process.
2023-06-07 18:30:12 +03:00
Zaiming (Stone) Shi 622ac4d195
fix(emqx_schema): binary for string default value 2023-06-07 17:14:10 +02:00
JimMoen 4ea3483083
perf(topic): use tail recursive to joining a topic 2023-06-07 22:59:53 +08:00
JimMoen b0e18f5e75
fix(shared-sub): validate share topic error send DISCONNECT 2023-06-07 22:59:52 +08:00
JimMoen 070e410c69
refactor: variable and func rename 2023-06-07 22:59:52 +08:00
JimMoen ea81a924f1
refactor: move topic() types def in `emqx_types.erl` 2023-06-07 22:59:52 +08:00
JimMoen 65483e972e
refactor: use unified table define 2023-06-07 22:51:44 +08:00
zhongwencool 70e800f8a4 chore: delete unuse default value and make flapping_detect middle improtance 2023-06-07 22:50:11 +08:00
zhongwencool 5659cc2b47 feat: add converter to flapping_detect 2023-06-07 21:54:32 +08:00
zhongwencool 1d33b7dbd8 chore: make static check happy 2023-06-07 21:54:32 +08:00
zhongwencool 535870386a fix: zones update api failed 2023-06-07 21:54:32 +08:00
zhongwencool d58506a7c3 feat: always update default zones 2023-06-07 21:54:32 +08:00
zhongwencool 48381d4c86 feat: refactor flapping detect conf 2023-06-07 21:54:32 +08:00
Thales Macedo Garitezi cc8631223e fix(schema): avoid `function_clause` error when compacting errors
Fixes https://emqx.atlassian.net/browse/EMQX-10168

The bridge probe API displayed the typecheck errors for the new
timeout duration types correctly, but when an user tried to create the
bridge anyway a `function_clause` error was raised when trying to
compact hocon errors:

```
09:47:19.045 [warning] [exception: :error, path: '/bridges', reason: {:case_clause, {:error, {:config_update_crashed, :function_clause}}}, stacktrace: [{:emqx_bridge_api, :create_or_update_bridge, 4, [file: '/home/thales/dev/emqx/emqx/apps/emqx_bridge/src/emqx_bridge_api.erl', line: 602]}, {:minirest_handler, :apply_callback, 3, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 111]}, {:minirest_handler, :handle, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 44]}, {:minirest_handler, :init, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 27]}, {:cowboy_handler, :execute, 2, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_handler.erl', line: 41]}, {:cowboy_stream_h, :execute, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 318]}, {:cowboy_stream_h, :request_process, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 302]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]]
```

This fixes the issue so that both APIs return more friendly error messages.
2023-06-07 10:34:58 -03:00
Paulo Zulato a4407764f3 feat(connection): configurable TCP keepalive
Fixes https://emqx.atlassian.net/browse/EMQX-9852
2023-06-07 10:23:38 -03:00
Andrew Mayorov c05f462d0a
fix(tlslib): bring back deterministic filenames for managed certs 2023-06-07 13:32:35 +03:00
William Yang c1be86c5ec
Merge pull request #10931 from qzhuyan/perf/william/save-times-of-maps-build
Perf/william/save times of maps build
2023-06-07 10:32:45 +02:00
Zaiming (Stone) Shi cb3e787cb1 Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-07 09:14:04 +02:00
Serge Tupchii bf2b5a9f24 fix(emqx_listeners): fix listener authentication create/update
Listener authentication is a list of authenticators, so each of them must be created
or updated individually.
2023-06-06 19:00:35 +03:00
Serge Tupchii 1968589f81 fix(emqx_schema): don't allow enabling `fail_if_no_peer_cert` if `verify_none` is set
Setting `fail_if_no_peer_cert = true` and `verify = verify_none` causes connection errors.

Closes: EMQX-9586
2023-06-06 17:20:22 +03:00
Thales Macedo Garitezi 2c97ee1b3f test(conf_app): fix test after common test helpers changes 2023-06-06 09:32:38 -03:00
Thales Macedo Garitezi 97a9bb484a test(pulsar_producer): attempt to fix flaky test 2023-06-06 09:32:38 -03:00
Thales Macedo Garitezi 770dd188b1
Merge pull request #10947 from emqx/release-51
Release 51 to master
2023-06-06 09:32:16 -03:00
Andrew Mayorov d5aedaac7d
fix(tlslib): append random suffix to managed certfiles
In order to lessen the chance of ambiguity in determining collectable
certfiles during GC.
2023-06-06 11:21:03 +03:00
zhongwencool ddb4a09a3d
Merge pull request #10935 from zhongwencool/listeners-conf-update
feat: update listeners from cli
2023-06-06 08:10:45 +08:00
Zaiming (Stone) Shi 3caf114f68 test: fix test case is_list(Config) function guard 2023-06-05 23:00:41 +02:00
Zaiming (Stone) Shi bb1cf9beaa Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-05 22:56:58 +02:00
Andrew Mayorov 750b7158d4
fix(tlsgc): make computing orphans safe against symlinking
Before this commit, `orphans/1` considered managed file an orphan based
on its absolute filename, which is not safe when, for example, emqx has
a symlink configured as `data_dir` while config references certfiles
through realpaths.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 31d38e18c9
refactor(tlsgc): simplify `fold_config/4` 2023-06-05 22:41:08 +03:00
Andrew Mayorov f8e8b7993b
fix(tlsgc): anticipate only binaries as string in raw config
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-05 22:41:08 +03:00
Andrew Mayorov 3c2a7dbadc
fix(tlsgc): consolidate conf keypaths knowledge in `emqx_tls_lib`
So that this GC mechanism will be easier to maintain.
2023-06-05 22:41:08 +03:00
Andrew Mayorov ec06850bd0
test(auth): drop obsolete testcase altogether
Most of the stuff that was tested by this test case is now covered
by `emqx_tls_certfile_gc_SUITE`.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 206d0472e0
fix(emqx): avoid interference in testsuites
Also remove that `?wait_async_action` in `emqx_trace_SUITE` which
consistently becomes stuck for 100 seconds. It's not clear why it
was there in the first place because listeners should start
synchronously, so `emqx_common_test_helpers:start_apps/1` won't return
until required listeners are started.
2023-06-05 22:41:08 +03:00
Andrew Mayorov 8a31e5639b
fix(emqx): ensure that standalone build works
Turns out rebar3 `path` resource has troubles with dangling
symlinks and non-ASCII filenames. 🥲
2023-06-05 22:41:07 +03:00
Andrew Mayorov 99ea9b86c2
feat(tlslib): add separate managed certfiles GC process
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
2023-06-05 22:41:07 +03:00
Andrew Mayorov 95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
2023-06-05 22:41:07 +03:00
Zaiming (Stone) Shi 67db9d6fe9 chore: bump version to 5.1.0-alpha.3 2023-06-05 20:25:48 +02:00
Thales Macedo Garitezi 75df622426
Merge pull request #10930 from thalesmg/max-int-timeout-v50
check maximum timeout values in schema (5.1)
2023-06-05 12:51:36 -03:00
Thales Macedo Garitezi 46393343e2 chore: use `timeout_duration` types for timer fields
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
Thales Macedo Garitezi 9bd37937f1 feat(types): add `timeout_duration{,_ms,_s}` types 2023-06-05 11:46:32 -03:00
某文 d0d6992a14 feat: update listeners from cli 2023-06-04 20:07:33 +08:00
Zaiming (Stone) Shi f469b31fee Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-03 11:21:22 +02:00
Zaiming (Stone) Shi 1ba07e1040 Merge remote-tracking branch 'origin/release-51' into 0529-donot-copy-cluster-conf-from-newer-version 2023-06-02 16:30:58 +02:00
Zaiming (Stone) Shi 28c564d15b Merge remote-tracking branch 'origin/master' into release-51 2023-06-02 16:25:48 +02:00
William Yang b1c188f9b1 perf(mqtt-caps): dont filter cap keys
save map builds
2023-06-02 16:24:52 +02:00
William Yang 4961aca3d0 perf(mqtt-caps): save some map builds 2023-06-02 16:19:58 +02:00
Zaiming (Stone) Shi 367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition
fix(emqx_cm): fix channel data registration race-condition
2023-06-02 14:20:18 +02:00
Thales Macedo Garitezi 2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50
fix(plugins): use `emqx:running_nodes` for multicall operations
2023-06-02 09:03:42 -03:00
Zaiming (Stone) Shi c75e9bbe0d fix(emqx_cm): fix channel data registration race-condition
when clustered, there are chances the a mqtt client process
get killed (e.g. holding the channel registeration lock for too long),
if the channel data inserts happen before casting out the message
for channel process monitoring, there is a chance for the
stale message left in the ets tables indefinitely.

this commit changes the order of the non-atomic operations:
it casts out the monitor request message before inserting
channel data.
2023-06-02 11:43:35 +02:00
zhongwencool dd85c981cd
Merge pull request #10783 from zhongwencool/improve-authn
feat: load changes to the authentication configuration from command line
2023-06-02 15:58:13 +08:00
zhongwencool 9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve
feat: support emqx_config:update([authrization],Conf) update
2023-06-02 15:43:09 +08:00
Andrew Mayorov 76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum
fix(ft): respect checksum in `fin` packets
2023-06-01 23:49:55 +03:00
Thales Macedo Garitezi 64258120d7 fix(plugins): use `emqx:running_nodes` for multicall operations
Fixes https://emqx.atlassian.net/browse/EMQX-10079
2023-06-01 14:00:43 -03:00
某文 2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
某文 c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
某文 94b86f01a5 chore: reformat SUITE case 2023-06-01 22:32:31 +08:00
zhongwencool 21df321d89
Merge branch 'master' into improve-authn 2023-06-01 22:24:59 +08:00
Kjell Winblad 75ff76a16b fix: friendly error message when creating bridges with too long names
This commit makes the error message and log entry that appear when one
tries to create a bridge with a name the exceeds 255 bytes (the max
length for atoms) more friendly and easier to understand.

An even better fix would be to not store bridge names as atoms but this
probably requires a more substantial change.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9609
2023-06-01 15:30:06 +02:00
William Yang c1d9bc8033 perf(emqx_mqtt_caps): direct read mqtt from zone conf 2023-06-01 14:54:44 +02:00
William Yang 923913e15c feat(config): merge with global defaults when put new zone 2023-06-01 14:54:44 +02:00
William Yang a2f8e87389 test(config): default zone and global defaults 2023-06-01 14:54:14 +02:00
William Yang e693c6ec54 test: fix emqx_takeover_SUITE 2023-06-01 14:52:33 +02:00
William Yang 8c26ee75db test: remove hardcoded configs
test with schema defaults
2023-06-01 14:52:33 +02:00
William Yang 4416122cf3 test: fix longname/shortname 2023-06-01 14:52:33 +02:00
William Yang f621bf1f8e test(config): update emqx_client_SUITE 2023-06-01 14:52:33 +02:00
William Yang 98d6ed4251 test: fix emqx_session_SUITE 2023-06-01 14:52:33 +02:00
William Yang 88f9782db0 perf(config): read zone conf once 2023-06-01 14:52:30 +02:00
Zhongwen Deng 7eea693422 fix: reorder authn when updating 2023-06-01 17:24:39 +08:00
Zhongwen Deng 4f5d5216bb test: bad authn handler callback module in SUITE 2023-06-01 17:24:39 +08:00
Zhongwen Deng 8f12d81878 feat: support emqx_ctl conf command 2023-06-01 17:24:39 +08:00
Zaiming (Stone) Shi feecd53436 chore: bump to version 5.1.0-alpha.2 for both ce and ee 2023-06-01 10:13:56 +02:00
Zaiming (Stone) Shi 15ffa98bc5 chore: fix a typo in test case function name 2023-06-01 08:18:26 +02:00
Zaiming (Stone) Shi cb2be1811e docs: fix function doc annotation 2023-06-01 08:17:36 +02:00
Thales Macedo Garitezi f83f112189 fix(config): enforce root key name to be a binary (raw config)
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299

```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
    exception: error
    key_path: [listeners,tcp,default,authentication]
    module: emqx_schema
    msg: change_config_crashed
    reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
                               <<"authentication">>]}
    stacktrace: [{emqx_utils_maps,deep_get,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
                      {line,49}]},
                 {emqx_config_handler,return_change_result,2,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,456}]},
                 {emqx_config_handler,check_and_save_configs,7,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,284}]},
                 {emqx_config_handler,handle_update_request,4,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,195}]},
                 {emqx_config_handler,handle_call,3,
                     [{file,
                          "/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
                      {line,124}]},
                 {gen_server,try_handle_call,4,
                     [{file,"gen_server.erl"},{line,1149}]},
                 {gen_server,handle_msg,6,
                     [{file,"gen_server.erl"},{line,1178}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,240}]}]
    update_req: {{update,
                     {create_authenticator,'tcp:default',
                         #{backend => built_in_database,enable => true,
                           mechanism => password_based}}},
                 #{rawconf_with_defaults => true}}
```

Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:

```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
 #{<<"ssl">> =>
       #{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```

... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
2023-05-31 18:03:58 -03:00
Thales Macedo Garitezi b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e
more attempts to fix more test flakiness
2023-05-31 16:07:18 -03:00
Zaiming (Stone) Shi 5146de5b1c feat: add a backup copies for cluster.hocon 2023-05-31 20:34:25 +02:00
Zaiming (Stone) Shi 39d6f612ca feat(emqx_release): add new APIs to parse/compare release versions 2023-05-31 20:34:25 +02:00
Andrew Mayorov aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard
test(dashboard): fix inter-suite test flakiness
2023-05-31 20:28:39 +03:00
Andrew Mayorov f90ce9bbfe
test(dashboard): fix inter-suite test flakiness 2023-05-31 19:09:37 +03:00
Andrew Mayorov 0293b54211
fix(ft): respect checksum in `fin` packets 2023-05-31 18:50:38 +03:00
Thales Macedo Garitezi 189e2c87be test(channel): fix flaky test
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```

The problem was that the whole generated message was compared for
equality when receiving the will message, but the timestamp may be
different at publishing time...
2023-05-31 12:42:24 -03:00
JianBo He a3a6480f00
Merge pull request #10880 from HJianBo/batch-kickout-clients
feat: support kickout clients in batch
2023-05-31 22:36:35 +08:00
Thales Macedo Garitezi 77c9eda036 test: rm stray `clear_screen` calls 2023-05-31 11:07:44 -03:00
Thales Macedo Garitezi 0d539e91d1 test(pulsar_producer): attempt to stabilize flaky test
https://github.com/emqx/emqx/actions/runs/5125166433/jobs/9218613872?pr=10886#step:7:679
```
  =CRITICAL REPORT==== 30-May-2023::19:38:58.003170 ===
  Run stage failed: error:{badmatch,
                              {timeout,
                                  [#{msg => pulsar_producer_bridge_started,
                                     '~meta' =>
                                         #{gl => <97891.472.0>,
                                           location =>
                                               #Fun<emqx_bridge_pulsar_impl_producer.11.109752493>,
                                           node => 'autocluster_node1@127.0.0.1',
                                           pid => <97891.787.0>,
                                           time => -576460611692219}}]}}
  Stacktrace: [{emqx_bridge_pulsar_impl_producer_SUITE,'-t_cluster/1-fun-10-',
                   6,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1073}]},
               {emqx_bridge_pulsar_impl_producer_SUITE,t_cluster,1,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1064}]}]
```
2023-05-31 10:19:55 -03:00
zhongwencool 26c2ab0bef
Merge pull request #10894 from zhongwencool/crl-cache-conf-hot-update
feat: update crl_cache conf at runtime
2023-05-31 17:59:09 +08:00
William Yang 2250ed7052
Merge pull request #10823 from qzhuyan/dev/william/meck-olp
test: use meck in olp tests
2023-05-31 11:01:56 +02:00
Zhongwen Deng 38e57f511c feat: update crl_cache conf at runtime 2023-05-31 16:18:03 +08:00
JianBo He afcd9fcb5e chore: update bpapi.versions 2023-05-31 11:29:55 +08:00
zhongwencool 6202cefd5a
Merge pull request #10870 from zhongwencool/stop-dashboard-listener-waiting
fix: flaky test when stopping dashboard listener
2023-05-31 10:54:12 +08:00
firest 79d08e4d26 fix(limiter): temporarily hide the node-level settings 2023-05-31 09:43:14 +08:00
firest 55809f801c test: fix test case error 2023-05-31 09:43:14 +08:00
firest be2dfa51df fix(limiter): improve the default configuration 2023-05-31 09:43:14 +08:00
Zhongwen Deng 5690469896 fix: flaky test when stopping dashboard listener 2023-05-31 07:53:10 +08:00
Thales Macedo Garitezi 299a67656e
Merge pull request #10886 from thalesmg/test-flakiness-20230530-d
test(quic_multistreams): attempt to fix flaky tests
2023-05-30 17:34:23 -03:00
Thales Macedo Garitezi 4f1c55f3fb
Merge pull request #10885 from thalesmg/test-flakiness-20230530-c
test(olp): fix inter-suite flakiness
2023-05-30 16:09:17 -03:00
Thales Macedo Garitezi 596f05f981 test(channel): attempt to reduce test flakiness
https://github.com/emqx/emqx/actions/runs/5124277038/jobs/9217362541?pr=10886#step:5:1582
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```
2023-05-30 15:48:48 -03:00
Thales Macedo Garitezi c43e5b8406
Merge pull request #10883 from thalesmg/test-flakiness-20230530-b
test(exclusive_sub): fix test flakiness
2023-05-30 15:46:14 -03:00
Thales Macedo Garitezi 98a99d47cb test(shared_sub): attempt to reduce test flakiness
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9215439237#step:8:565
```
%%% emqx_shared_sub_SUITE ==> t_local_fallback: FAILED
%%% emqx_shared_sub_SUITE ==> {{badmatch,<<"not yet?">>},
 [{emqx_shared_sub_SUITE,t_local_fallback,1,
                         [{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_shared_sub_SUITE.erl"},
                          {line,669}]},
```
2023-05-30 14:03:14 -03:00
Thales Macedo Garitezi fa9afcd7e4 test(quic_multistreams): attempt to fix flaky tests
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9214950882#step:8:1757
```
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
test_server:ts_tc failed on line 1782
Reason: {test_case_failed,Client <0.6075.5> did not die: stacktrace: {current_stacktrace,
                                            [{gen_statem,loop_receive,3,
                                              [{file,"gen_statem.erl"},
                                               {line,1281}]},
                                             {proc_lib,init_p_do_apply,3,
                                              [{file,"proc_lib.erl"},
                                               {line,240}]}]}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Testing lib.emqx.emqx_quic_multistreams_SUITE: *** FAILED test case 335 of 364 ***
%%% emqx_quic_multistreams_SUITE ==> shutdown.abort_recv_shutdown.ctrl_stream_shutdown.t_multi_streams_remote_shutdown: FAILED
%%% emqx_quic_multistreams_SUITE ==> {test_case_failed,"Client <0.6075.5> did not die: stacktrace: {current_stacktrace,\n                                            [{gen_statem,loop_receive,3,\n                                              [{file,\"gen_statem.erl\"},\n                                               {line,1281}]},\n                                             {proc_lib,init_p_do_apply,3,\n                                              [{file,\"proc_lib.erl\"},\n                                               {line,240}]}]}"}
```

```
  %%% emqx_quic_multistreams_SUITE ==> mstream.profiles.profile_max_throughput.pub_qos2.sub_qos2.qos.t_multi_streams_unsub: FAILED
  %%% emqx_quic_multistreams_SUITE ==> {{case_clause,
       {ok,#{packet_id => 8,properties => #{},reason_code => 0,
             reason_code_name => success,
             via =>
                 {quic,#Ref<0.686875463.2755788802.201649>,
                     #Ref<0.686875463.2755788804.193527>}}}},
   [{emqx_quic_multistreams_SUITE,t_multi_streams_unsub,1,
        [{file,
             "/home/thales/dev/emqx/emqx/apps/emqx/test/emqx_quic_multistreams_SUITE.erl"},
         {line,1094}]},
    {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
    {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1292}]},
    {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}
```
2023-05-30 13:56:05 -03:00
Thales Macedo Garitezi 37b4a44e23 test(olp): fix inter-suite flakiness
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9212645120?pr=10883#step:8:2538
```
   emqx_olp_SUITE:t_overload_cooldown_conn failed on line 96
  Reason: {assertMatch,[{module,...},{...}|...]}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_olp_SUITE: *** FAILED test case 3 of 4 ***
  %%% emqx_olp_SUITE ==> t_overload_cooldown_conn: FAILED
  %%% emqx_olp_SUITE ==>
  Failure/Error: ?assertMatch({ ok , _Pid }, emqtt : connect ( C ))
    expected: = { ok , _Pid }
         got: {error,econnrefused}
        line: 96
  %%% emqx_olp_SUITE ==> t_overloaded_conn: OK
  Testing lib.emqx.emqx_olp_SUITE: TEST COMPLETE, 3 ok, 1 failed of 4 test cases
```

```sh
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_olp_SUITE.erl
```
2023-05-30 13:39:52 -03:00
William Yang f2e54c9a6e test: use meck in olp tests
avoid flaky test results by using mecks.
2023-05-30 17:57:09 +02:00
Thales Macedo Garitezi a28580248a test(exclusive_sub): fix test flakiness
https://github.com/emqx/emqx/actions/runs/5115608275/jobs/9208594481?pr=10864#step:8:735

```
  Testing lib.emqx.emqx_access_control_SUITE: TEST COMPLETE, 4 ok, 0 failed of 4 test cases

  Testing lib.emqx.emqx_exclusive_sub_SUITE: Starting test, 5 test cases

  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  Error detected: {'EXIT',{shutdown,econnrefused}}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 1 of 5 ***
  %%% emqx_exclusive_sub_SUITE ==> t_allow_normal_sub: FAILED
  %%% emqx_exclusive_sub_SUITE ==> {'EXIT',{shutdown,econnrefused}}

  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  emqx_exclusive_sub_SUITE:t_clean_session failed on line 128
  Reason: {badmatch,{error,econnrefused}}
  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

  Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 2 of 5 ***
  %%% emqx_exclusive_sub_SUITE ==> t_clean_session: FAILED
```

To reproduce:

```
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_exclusive_sub_SUITE.erl
```
2023-05-30 11:23:17 -03:00
Thales Macedo Garitezi cb7929a2fc test(broker): attempt to reduce flakiness 2023-05-30 11:12:18 -03:00
Thales Macedo Garitezi 26992e9d98 test(banned): attempt to improve flaky test 2023-05-30 11:08:31 -03:00
Thales Macedo Garitezi 6b5b9fc8b8
Merge pull request #10864 from thalesmg/fix-flaky-channel-test-20230529
test(channel): fix flaky test
2023-05-30 10:18:47 -03:00
Zaiming (Stone) Shi d2ce7c22fd chore: bump app vsn, core app 'emqx' is now at 5.1 2023-05-30 11:32:18 +02:00
Zaiming (Stone) Shi 1e8ed89db8 Merge remote-tracking branch 'origin/master' into release-51 2023-05-30 11:29:44 +02:00
Zaiming (Stone) Shi 91cdc69976
Merge pull request #10867 from zmstone/0530-merge-release-50-to-master
0530 merge release 50 to master
2023-05-30 09:54:57 +02:00
Zaiming (Stone) Shi 367de2a2eb docs: update bpapi readme 2023-05-30 08:29:43 +02:00
Zaiming (Stone) Shi 941939f7d7 build: finalize 5.0.bpapi 2023-05-30 08:28:45 +02:00
Zaiming (Stone) Shi 3b8cba6ba0 chore: bump rlease versions 2023-05-30 08:28:45 +02:00
Zaiming (Stone) Shi 747eb6fcff Merge remote-tracking branches 'origin/release-50' and 'origin/master' into release-51 2023-05-30 07:54:34 +02:00
JimMoen 909d348570
chore: fix spec typo 2023-05-30 11:31:37 +08:00
Thales Macedo Garitezi 1aff46b7c9 test(channel): fix flaky test 2023-05-29 19:04:48 -03:00
Thales Macedo Garitezi 67e182e0c9
Merge pull request #10813 from thalesmg/refactor-kafka-on-stop-v50
feat(kafka): ensure allocated resources are removed on failures
2023-05-29 16:49:29 -03:00
Thales Macedo Garitezi 79f6861499 chore: prepare to tag v5.0.26 2023-05-29 09:17:59 -03:00
Zaiming (Stone) Shi 8e6dc29637 chore: bump release version to v5.0.26-alpha.1 2023-05-29 08:57:28 +02:00
Zaiming (Stone) Shi 25319c3151
Merge pull request #10839 from zmstone/0526-merge-release-50-to-master
0526 merge release 50 to master
2023-05-29 08:56:20 +02:00
Zaiming (Stone) Shi c2450d230d
Merge pull request #10605 from zmstone/0503-no-telemetry-app-for-ee
0503 refactoring: no telemetry app for ee
2023-05-27 10:57:15 +02:00
Thales Macedo Garitezi d27f593309 test: fix flaky authn test 2023-05-26 16:44:00 -03:00
Zaiming (Stone) Shi fe81e9521a
Merge pull request #10600 from zmstone/0504-delete-statsd-all-together
0504 delete statsd all together
2023-05-26 16:29:59 +02:00
Zaiming (Stone) Shi 36e268c933 chore: bump app versions 2023-05-26 16:05:37 +02:00
Zaiming (Stone) Shi cc5b4d3748 Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test 2023-05-26 15:58:16 +02:00
Zaiming (Stone) Shi 6e2cde8224 refactor: delete emqx_statsd all together 2023-05-26 15:17:42 +02:00
Andrew Mayorov 45b128946a
Merge pull request #10825 from fix/EMQX-9991/bugs
fix(ft): avoid leaking secrets
2023-05-26 16:08:14 +03:00
Zaiming (Stone) Shi 7eb22e27c4 chore: bump release version to e5.0.4 2023-05-26 14:34:01 +02:00
Zhongwen Deng 9b6da3ff10 chore: bump hocon to 0.39.7 2023-05-26 19:00:58 +08:00
Zhongwen Deng df4a0afdf5 fix: zones api default value 2023-05-26 18:59:38 +08:00
Zaiming (Stone) Shi 2d986140df chore: bump version to e5.0.4-rc.2 2023-05-26 12:01:10 +02:00
Andrew Mayorov 48858dee33
fix(s3): wrap S3 secrets during config load 2023-05-26 12:59:21 +03:00
JimMoen a3b77563ba
fix: make static check happy 2023-05-26 10:34:15 +08:00
Paulo Zulato ea86f4442b fix: avoid error 500 when node is re-joining cluster
Fixes https://emqx.atlassian.net/browse/EMQX-9899
2023-05-25 13:32:28 -03:00
Zaiming (Stone) Shi e8a4df96bc Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-25 16:43:20 +02:00
ieQu1 2b79196a03
Merge pull request #10809 from ieQu1/ekka-0.15.2
chore(ekka): Bump version to 0.15.2
2023-05-24 18:13:04 +02:00
zhongwencool 14a6b36899
Merge pull request #10803 from JimMoen/merge-release-50
Merge release-50 into master
2023-05-24 23:08:46 +08:00
ieQu1 3d3dacfcf6 chore(ekka): Bump version to 0.15.2 2023-05-24 15:43:49 +02:00
JimMoen 28015597ee
Merge remote-tracking branch 'emqx/release-50' into merge-release-50 2023-05-24 19:34:12 +08:00
Zaiming (Stone) Shi 2fdf4b5dac
Merge pull request #10744 from savonarola/0518-fix-eviction
fix(evacuation): handle expire interval correctly
2023-05-24 09:20:53 +02:00
Thales Macedo Garitezi 8e6da40a31 test: fix flaky ocsp test setup (v5.0) 2023-05-23 17:52:02 -03:00
Zaiming (Stone) Shi b8b1779627 Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-23 21:42:07 +02:00
Zaiming (Stone) Shi 2fa5b511bf chore: hide stale config 2023-05-23 14:20:49 +02:00
Zaiming (Stone) Shi 3bb1f7ab2b Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-23 13:03:55 +02:00
某文 ed1268ed30 Merge branch 'release-50' into 0522-merge-release-50-to-master 2023-05-23 07:56:35 +08:00
Zaiming (Stone) Shi 68f4ae70b2 chore: bump ee release version to e5.0.4-alpha.2 2023-05-22 22:41:19 +02:00
Zaiming (Stone) Shi 732a7be187 Merge remote-tracking branch 'origin/release-50' 2023-05-22 17:46:54 +02:00
Zaiming (Stone) Shi 1f44dd4579
Merge pull request #10756 from zmstone/0519-refactor-move-lib-ee-emqx_ee_conf-to-apps-emqx_enterprise
0519 refactor move lib ee emqx ee conf to apps emqx enterprise
2023-05-22 16:56:51 +02:00
Andrew Mayorov 3f882ddcc6
Merge pull request #10751 from ci/ct-flap-listener-api
fix(listen): wait until port is free when stopping ranch listeners
2023-05-22 17:00:45 +03:00
lafirest 88e4078885
Merge pull request #10747 from lafirest/fix/rule_funs_time_offset
fix: port the `emqx_calendar` from v4.4
2023-05-22 20:52:21 +08:00
Zaiming (Stone) Shi 40e8d5d039 refactor: rename lib-ee/emqx_ee_conf to apps/emqx_enterprise 2023-05-22 14:51:27 +02:00
Andrew Mayorov 23542d1262
fix(listen): wait until port is free when stopping ranch listeners
Simple `cowboy:stop_listener/1` will not close the listening socket
explicitly, it was the source of test flaps before this fix.
2023-05-22 12:29:40 +03:00
William Yang e824e1db39
Merge pull request #10698 from qzhuyan/perf/william/force-atom-conf-path
perf(config): enforcing atom key  path in hotcode path
2023-05-22 09:38:09 +02:00
某文 082214d039 feat: add authz file rule validator 2023-05-22 14:25:05 +08:00
firest 56a6b699ac fix: port the `emqx_calendar` from v4.4 2023-05-22 09:55:31 +08:00
某文 7c2aac64bc fix: bad cert file path in dashboard https listener 2023-05-21 07:41:46 +08:00
Andrew Mayorov 9aaa0b6fe1
Merge pull request #9927 from emqx/file-transfer
feat: implement file transfer over mqtt
2023-05-19 18:35:13 +03:00
某文 6cb9efd7d3 feat: add authz file rule validator 2023-05-19 18:41:59 +08:00
Ilya Averyanov 9c48b016a9 fix(evacuation): handle expire interval correctly 2023-05-18 22:34:54 +03:00
Thales Macedo Garitezi 09ea2e2224 fix(bridge_api): don't crash when formatting empty/unknown bridge metrics
Fixes https://emqx.atlassian.net/browse/EMQX-9872
2023-05-18 14:25:13 -03:00
Thales Macedo Garitezi 56634e5238
Merge pull request #10725 from thalesmg/bump-gproc-091-r50
chore: bump gproc -> 0.9.0.1 (r5.0)
2023-05-18 10:38:06 -03:00
zhongwencool ba43a0e30f
Merge pull request #10734 from thalesmg/merge-r50-into-v50-a
merge `release-50` into `master`
2023-05-18 14:24:53 +08:00
zhongwencool bf5ee41009
Merge pull request #10702 from zhongwencool/keepalive-backoff-rename
feat: deprecated keepalive_backoff, introduce keepalive_multiplier
2023-05-18 10:00:17 +08:00
某文 d4d25d2660 chore: don't lost previous's statval 2023-05-18 09:49:11 +08:00
Thales Macedo Garitezi 447b76464b Merge branch 'release-50' into merge-r50-into-v50-a 2023-05-17 14:50:18 -03:00
William Yang 727ad59995 chore(test): improve coverage 2023-05-17 16:58:49 +02:00
Thales Macedo Garitezi 060efd6964 chore: bump gproc -> 0.9.0.1 (r5.0)
Includes this fix: https://github.com/uwiger/gproc/pull/193
2023-05-17 11:05:44 -03:00
Zhongwen Deng e26ce5816e feat: hide keepalive_backoff, introduce keepalive_multiplier 2023-05-17 21:25:10 +08:00
William Yang cdf42760fa chore: little change avoid atom leak 2023-05-17 11:01:03 +02:00
Ilya Averyanov b71955e368 fix(ft): bump application versions 2023-05-17 00:29:15 +03:00
Ilya Averyanov 1a8cf0e392 Merge branch 'release-50' into file-transfer
* release-50:
  fix(pulsar): use a binary duration as default `health_check_interval`
  docs: add changelog entry
  docs: clarify description of bridge username and password
  chore: bump to v5.0.25
  fix(limiter): adjust type for compatibility
  fix(limiter): fix that update node-level limiter config will not working
  chore: upgrade dashboard to v1.2.4-1 for ce
  chore: upgarde rulesql to 0.1.6 to fix invaid utf8 input
  chore: add changelog for 10659
  fix: crash when sysmon.os.mem_check_interval = disabled
  chore: bump influxdb version && update changes
  refactor(influxdb): move influxdb bridge into its own app
  chore: add listener default changelog
  fix: ocsp cache SUITE failed
  fix: ensure atom key for emqx_config:get
  fix: only fill cerf_file default in server side
  fix: authn init is empty
  fix: bad listeners default ssl_options
2023-05-17 00:27:55 +03:00
Andrew Mayorov 3cd95f40e5
test(chan): verify hooks receive peercert until connected 2023-05-16 19:35:56 +03:00
Andrew Mayorov 967b2e72e0
test(emqx): remove `peercert` from clientinfo fixtures
According to typespec, there's no place for `peercert` in
`clientinfo()`.
2023-05-16 19:35:55 +03:00
Andrew Mayorov b341a04955
fix(chan): postpone trimming conninfo until `connected` hooks run
Some users expect to get the peer certificate in `connected` hooks, but
the `conninfo` was trimmed before `connected` hooks run.
2023-05-16 19:35:55 +03:00
Andrew Mayorov 6f8f21106b
fix(assert): use unpredictable binding names in macros
Also translate macro vars to ALLCAPS for consistency.
2023-05-16 16:46:05 +03:00
Andrew Mayorov be3a0ce974
fix(maybe): correct `apply/2` typespec
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-05-16 16:26:26 +03:00
Andrew Mayorov 6813ea8e7a
test(client): dedicate separate testcase to peercert cleaning 2023-05-16 13:25:33 +03:00
firest 255f616d26 chore: bump emqx app version 2023-05-16 16:36:36 +08:00
William Yang 1c746ed289 perf(config): enforcing atom key path in hotcode path 2023-05-16 09:57:04 +02:00
firest 50e7de9db2 fix(limiter): a bucket with an infinity rate shouldn't be added to limiter server 2023-05-16 13:30:41 +08:00
lafirest 67ada52808
Merge pull request #10682 from lafirest/fix/will_msg_timestamp
fix: update the will message timestamp when it is ready to publish
2023-05-12 22:25:32 +08:00
某文 5ef2a603a1 chore: bump to v5.0.25 2023-05-12 20:27:36 +08:00
firest 95a67f390f fix(limiter): adjust type for compatibility 2023-05-12 18:41:41 +08:00
zhongwencool a953b951fe
Merge branch 'master' into sync-release-50-to-master 2023-05-12 18:01:58 +08:00
firest a38b270b03 fix(limiter): fix that update node-level limiter config will not working 2023-05-12 17:39:49 +08:00
Zhongwen Deng 90549abec7 fix: crash when sysmon.os.mem_check_interval = disabled 2023-05-12 14:42:23 +08:00
firest dcd4640a57 fix: update the will message timestamp when it is ready to publish 2023-05-12 14:28:26 +08:00
Thales Macedo Garitezi c4f2dba9c5
Merge pull request #10643 from thalesmg/test-sanity-checks-v50
test: perform sanity checks when starting apps
2023-05-11 14:00:22 -03:00
某文 91f97f6c29 fix: ocsp cache SUITE failed 2023-05-11 21:26:23 +08:00
Ilya Averyanov 5b5d7ceac5 Merge branch 'release-50' into file-transfer
* release-50:
  fix(limiter): fix an error when setting `max_conn_rate` in a listener
  chore: bump erlcloud dependencies vsns
  chore: rename dynamo template files
  refactor(dynamo): move dynamo bridge into its own app
  chore: update changes && bump app versions
  fix: issues with the RabbitMQ config
  refactor(pgsql): move pgsql && matrix && timescale bridges into their own app
  fix: the iotdb password field so it has the password format
  chore: update changes
  refactor(tdengine): move tdengine bridge into its own app
  feat: deprecate listeners's authn http api
2023-05-11 16:15:38 +05:00
Zhongwen Deng 3d41449fde fix: only fill cerf_file default in server side 2023-05-11 17:33:18 +08:00
SergeTupchiy 0617a9b11c
Merge pull request #10653 from SergeTupchiy/convert-gateway-authn-certs
fix(emqx_gateway): convert and clear authentication certificates
2023-05-11 11:35:55 +03:00
Zhongwen Deng d9f9e951ec fix: bad listeners default ssl_options 2023-05-11 16:25:25 +08:00
zhongwencool a736b633b0
Merge pull request #10607 from zhongwencool/log-conf-refactor
feat: simplify log configuration
2023-05-11 16:12:27 +08:00
Zaiming (Stone) Shi b3e35dac1c
Merge pull request #10584 from paulozulato/debug-ssl-handshake
feat: add log level configuration to ssl communication
2023-05-11 08:59:13 +02:00
firest b7126257a5 fix(limiter): fix an error when setting `max_conn_rate` in a listener 2023-05-11 13:52:13 +08:00
zhongwencool 7e720ca260
Merge pull request #10641 from thalesmg/bump-gproc-091-v50
chore: bump gproc -> 0.9.0.1
2023-05-10 16:16:26 +08:00
Ilya Averyanov 8d9b785bd7 Merge branch 'release-50' into file-transfer
* release-50:
  chore(rebalance): fix app metadata
  chore(rebalance): move apps from lib-ee, add READMEs
  docs: refine zh tr
  docs: delete APL header from ee file
  docs: delete zh changelog
  chore(rebalance): review fixes
  chore(rebalance): rebase and review fixes
  feat(rebalance): port apps from 4.x
2023-05-10 11:55:23 +05:00
Zhongwen Deng b80227a02a fix: crash when sysmon.os.mem_check_interval = disabled 2023-05-10 14:49:34 +08:00
Ilya Averyanov 7fa166f034 Merge branch 'release-50' into file-transfer
* release-50: (73 commits)
  feat: add RabbitMQ bridge
  docs: improve rule engine labels and descriptions
  chore: bump version && update changes
  refactor(rocketmq): move rocketmq bridge into its own app
  test: dashboard_listener_test crash
  chore: bump chart versions
  chore: bump ee version to e5.0.4-alpha.1
  test: fix inter-suite flakiness
  build: compatibility to make 4.4+
  feat: add IotDB bridge
  ci: ensure git safe dir in build_packages
  ci: ensure git safe dir
  test: check_oom's max_mailbox_size
  feat: rename max_message_queue_len to max_mailbox_size
  fix(buffer_worker): fix inflight count when updating inflight item
  chore: prepare for v5.0.25-rc.1 release
  docs: add change log entry
  fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
  chore: `MQTT X` -> `MQTTX`
  chore: make sure brod_gssapi app is included in relese package
  ...
2023-05-09 23:30:22 +05:00
Ilya Averyanov e683d28973 chore(rebalance): rebase and review fixes 2023-05-09 20:51:24 +05:00
Ilya Averyanov 609f7bd8fd feat(rebalance): port apps from 4.x 2023-05-09 20:51:22 +05:00
Serge Tupchii b98a7168c4 fix(emqx_gateway): convert and clear authentication certificates 2023-05-09 17:32:44 +03:00
Thales Macedo Garitezi 5643c12930 chore: bump gproc -> 0.9.0.1
Includes fix: https://github.com/uwiger/gproc/pull/193

Prior to the fix, when using the `random` pool strategy, one of the
workers receives about double the load of other workers, which
decreases throughput of bridges like webhook.
2023-05-09 11:27:25 -03:00
William Yang 328add3a02
Merge pull request #10563 from qzhuyan/fix/william/no_local_filter_all
fix(mqtt): drop all local messages in session deliver
2023-05-09 15:29:12 +02:00
Kjell Winblad 70cf1533db feat: add RabbitMQ bridge 2023-05-09 14:32:26 +02:00
Zhongwen Deng 428cc45c69 fix: filter disable file handler 2023-05-09 17:50:06 +08:00
lafirest b94290db58
Merge pull request #10625 from lafirest/refactor/limiter_cfg
refactor(limiter): simplify limiter configuration
2023-05-09 11:40:05 +08:00