Commit Graph

790 Commits

Author SHA1 Message Date
JianBo He 66ff982170 chore: update changes 2023-12-15 11:44:28 +08:00
Zaiming (Stone) Shi 1b8d2304cb docs: add release note 2023-12-14 22:24:56 +01:00
Shawn 68173976d1 chore: add change logs for 12158 2023-12-13 09:48:38 +08:00
zhongwencool 45c4fb33a8 fix: occasionally return stale view when updating configurations on different nodes concurrently 2023-12-08 09:56:31 +08:00
zhongwencool 253ea6093a
Merge pull request #12111 from zhongwencool/fix-token-not-found
fix: use sync_transaction when update admin/token
2023-12-07 09:56:20 +08:00
SergeTupchiy 28ff53e99c
Merge pull request #11984 from SergeTupchiy/EMQX-10535-openetelmetry-tracing-new
Integrate OpenTelmetry tracing
2023-12-06 11:45:41 +02:00
Zaiming (Stone) Shi 1df239f216 docs: Add changelog for pr 12112 (drop udp mcast clustering) 2023-12-06 08:45:42 +01:00
zhongwencool b015d08cba fix: use sync_transation when update admin/token 2023-12-06 09:04:27 +08:00
Thales Macedo Garitezi deb3fcd606 Merge remote-tracking branch 'origin/master' into sync-m-r54-20231205 2023-12-05 13:25:45 -03:00
Serge Tupchii 7fdc650448 feat: integrate OpenTelemetry traces 2023-12-05 18:11:00 +02:00
SergeTupchiy 7b9d12f20e
Merge pull request #12048 from SergeTupchiy/fix-coap_sub-opts
Fix COAP sub opts
2023-12-05 10:27:04 +02:00
Serge Tupchii ac618d331f chore: upgrade grpc-erl to 0.6.12
grpc-erl 0.6.12 doesn't start dependent apps lazily anymore, as it was not deadlock safe.
2023-12-04 12:17:58 +02:00
ieQu1 0dbff6d7ce docs: Add a changelog entry for emqx_persistent_session_ds 2023-12-04 05:46:58 +01:00
ieQu1 3d2e95fe3e
Merge pull request #12081 from ieQu1/dev/gen-rpc-3.3.0
chore(gen_rpc): Bump version to 3.3.0
2023-12-01 17:57:33 +01:00
ieQu1 d3b7478d70 chore(gen_rpc): Bump version to 3.3.0 2023-12-01 17:10:12 +01:00
Ivan Dyachkov ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
zhongwencool 8ec3857f4d
Merge pull request #12046 from savonarola/1128-fix-redis-54
[5.4] Fix Redis connector
2023-11-29 16:06:23 +08:00
Serge Tupchii 3530ed66d5 fix(emqx_gateway_coap): take subscribe options from uri_query
Closes #12031
2023-11-28 20:02:21 +02:00
Ilya Averyanov e5508105c7 fix(redis): start and load eredis app 2023-11-28 18:51:53 +03:00
Ilya Averyanov 55218e2df2 fix(redis): start and load eredis app 2023-11-28 18:51:19 +03:00
William Yang 64f6220cf5 feat(quic): bump to quicer-0.0.303 and emqtt 1.9.7 2023-11-28 14:43:47 +01:00
Serge Tupchii 64ee29af81 feat(emqx_management): implement data backup API 2023-11-27 17:30:58 +02:00
zhongwencool b5a00ec6b2
Merge pull request #12023 from emqx/master
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
Thales Macedo Garitezi 6c9417efe0 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122 2023-11-22 12:02:34 -03:00
Serge Tupchii d9f95cdc56 feat: integrate OpenTelemetry log handler 2023-11-22 13:59:30 +02:00
Ivan Dyachkov c074ed43dd chore: add changelog 2023-11-22 09:58:51 +01:00
Zaiming (Stone) Shi 8fbdcab118
Merge pull request #11987 from zmstone/1120-do-not-crash-on-einval-after-check-cache
fix(emqx_connection): handle socket activation error return
2023-11-21 20:31:56 +01:00
Zaiming (Stone) Shi 3909e0cc08 docs: add changelog for PR 11975 2023-11-20 18:06:27 +01:00
Zaiming (Stone) Shi 7f078295c1 docs: add changelog for PR 11987 2023-11-20 17:44:39 +01:00
Ilya Averyanov 58437cd35a fix(mongodb): fix deadlock while stopping mongodb resource 2023-11-17 12:34:36 +03:00
zhongwencool c4eb9f86e7 chore: add refactor prometheus refactor changelog 2023-11-17 08:23:51 +08:00
Ilya Averyanov 4c54efd8b1 fix(auth): fix deadlock while stopping mongodb resource 2023-11-16 19:29:08 +03:00
Andrew Mayorov d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Andrew Mayorov 11c7024d9d
chore: add changelog entry 2023-11-14 16:05:53 +07:00
Andrew Mayorov dc3e818e84
chore: add changelog entry
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-11-13 19:29:30 +07:00
Thales Macedo Garitezi 371a49304d Merge branch 'release-53' into sync-r53-to-m-20231109 2023-11-09 17:19:23 -03:00
Ivan Dyachkov 0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
Zaiming (Stone) Shi f95058a3e7
Merge pull request #11897 from zmstone/1107-fix-cluster-conf-sync-wait-loop
1107 fix cluster conf sync wait loop
2023-11-08 23:37:09 +01:00
Zaiming (Stone) Shi f9e9748cec fix(cluster-rpc): boot from local config if table loaded
When EMQX boots up, it tries to get latest config from peer (core type)
nodes, if none of the nodes are replying, the node will decide
to boot with local config (and replay the committed changes) if
the commit table is loaded from disk locally (an indication of the
data being latest), otherwise it will sleep for 1-2 seconds and
retry.

This lead to a race condition, e.g. in a two nodes cluster:

1. node1 boots up
2. node2 boots up and copy mnesia table from node1
3. node1 restart before node2 can sync cluster.hocon from it
4. node1 boots up and copy mnesia table from node2

Now that both node1 and node2 has the mnesia `load_node` pointing
to each other (i.e. not a local disk load).

Prior to this fix, the nodes would wait for each other in a dead loop.

This commit fixes the issue by allowing node to boot
with local config if it does not have a lagging.
2023-11-08 15:06:47 +01:00
Ilya Averyanov d061d64c70 fix(plugins): fix backward compatibility 2023-11-08 12:36:09 +03:00
Ivan Dyachkov 0c91bec98d chore: merge 'upstream/release-53' 2023-11-08 09:24:38 +01:00
Ilya Averyanov 75704513d8 fix(plugins): fix backward compatibility 2023-11-07 19:06:37 +03:00
Zaiming (Stone) Shi c38725facd fix(shell): do not repeat print shell warnings 2023-11-01 14:44:18 +01:00
Ivan Dyachkov b1ab213081 chore: merge 'upstream/release-53' into 1031-sync-r53 2023-10-31 11:06:25 +01:00
Ivan Dyachkov 58042af1fc chore: add separate section for breaking changes in changelog
also remove support for Chinese language in changelog
2023-10-30 20:48:32 +01:00
Kjell Winblad 290c433e8c
Merge pull request #11565 from kjellwinblad/kjell/upgrade/jq/v0.3.11
fix: upgrade jq library
2023-10-30 16:47:58 +01:00
Ilya Averyanov 3f6c09b195
Merge pull request #11780 from savonarola/1017-fix-pbkdf2-validation
fix(authn): fix pbkdf2 option validation
2023-10-30 16:37:37 +02:00
JimMoen 9732e31395
chore: highlight breaking changes 2023-10-27 09:23:44 +08:00
JimMoen b5411da770
refactor: subscribe process to fix shared-sub 2023-10-27 09:23:43 +08:00
Serge Tupchii 9b0a7b9c63 chore: upgrade opentelemtry to v1.3.1-emqx
v1.3.1-emqx fixes metrics timestamp issue #11802.
2023-10-26 10:17:29 +03:00
JimMoen a4ab744d4e
Merge pull request #11805 from HJianBo/typo-fixes
chore: typo fixes
2023-10-25 09:16:23 +08:00
William Yang 6346e0d28a fix(gen_rpc): ssl client port align with server port 2023-10-24 22:33:52 +02:00
JimMoen 99fab8dc5d
chore: missing change log for duplicated apikey fixing 2023-10-24 09:44:39 +08:00
JianBo He ec7b669b79 chore: typo fixes 2023-10-24 08:51:52 +08:00
Stefan Strigler 4e0e755b28 fix: return 404 if built_in_database not configured as auth source 2023-10-23 16:26:41 +02:00
Ilya Averyanov edde661da3 fix(authn): fix pbkdf2 option validation 2023-10-23 10:26:11 +03:00
William Yang a9b42af8b3
Merge pull request #11796 from qzhuyan/fix/william/gen_rpc-driver
fix(gen_rpc): ensure client/server use same driver
2023-10-23 08:24:17 +02:00
William Yang cf20927bac docs: add changelog 2023-10-20 13:28:29 +02:00
Ilya Averyanov 0b4600c293 feat(auth): improve redis command parsing and validation 2023-10-19 23:01:35 +03:00
Thales Macedo Garitezi eda82caf65
Merge pull request #11791 from thalesmg/fix-coap-heartbeat-m-20231018
fix(coap): increase received packet counter for keepalive
2023-10-19 13:12:41 -03:00
Ilya Averyanov 8d82c30b00
Merge pull request #11771 from savonarola/1015-validate-bcrypt-schema-in-api
feat(authn): allow authn providers to define a separate schama for API
2023-10-19 15:34:34 +03:00
zhongwencool 9f197a6915
Merge pull request #11785 from zhongwencool/rbac-change-pwd 2023-10-19 09:02:15 +08:00
Thales Macedo Garitezi 5b9866f630 fix(coap): increase received packet counter for keepalive
Fixes https://emqx.atlassian.net/browse/EMQX-11193
Fixes https://github.com/emqx/emqx/issues/11779
2023-10-18 16:35:33 -03:00
Zaiming (Stone) Shi ad09ca9d6d refactor(nodetool): only add libs when necessary 2023-10-18 14:08:35 +02:00
zhongwencool 81e10c6748 chore: add changelog for 11785 2023-10-18 18:46:13 +08:00
Ilya Averyanov 6354f3b04f feat(authn): allow authn providers to define a separate schama for API 2023-10-17 13:19:11 +03:00
Zaiming (Stone) Shi dfa5909392
Merge pull request #11752 from zmstone/1011-change-default-rpc-module-for-mria
refactor: change mria default rpc module from 'gen_rpc' to 'rpc'
2023-10-17 10:21:14 +02:00
Zaiming (Stone) Shi 162b901be6
Merge pull request #11763 from zmstone/1114-sync-release-53
Merge release-53 back to master
2023-10-16 21:42:02 +02:00
JianBo He 16cc816bd3
Merge pull request #11757 from zhongwencool/trace-api-return-500
fix: 500 error response when downloading non-existent trace files
2023-10-16 13:40:18 +08:00
Zaiming (Stone) Shi 6eb3bb7cff Merge remote-tracking branch 'origin/release-53' into 1114-sync-release-53 2023-10-14 10:16:38 +02:00
Zaiming (Stone) Shi 7c022c2c6a refactor: change mria default rpc module from 'gen_rpc' to 'rpc'
Erlang distribution seems to outperform gen_rpc (unless gen_rpc clients
are scaled up, but this is not easy to achive for shard transport as it
may reorder events).
2023-10-14 10:13:31 +02:00
Ilya Averyanov bde8800f2e fix(mnesia authz): destroy authz records on mnesia authz destroy 2023-10-13 18:29:45 +03:00
firest 3e658b3da9 chore: update changes 2023-10-12 17:15:47 +08:00
zhongwencool 9b2a5e6c3d chore: add changelog for 11757 2023-10-12 15:25:36 +08:00
William Yang a69c53455f fix: bump to quicer 0.0.202
bring in urgent fixs from msquic 2.2.3
2023-10-11 13:12:32 +02:00
Zaiming (Stone) Shi d651764ff4 docs: add changelog for pull request 11734 2023-10-10 10:05:14 +02:00
Zaiming (Stone) Shi ed5b456d62 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-29 18:24:44 +02:00
Zaiming (Stone) Shi f69702b82e docs: delete typo-named changelog file 2023-09-29 14:03:30 +02:00
Zaiming (Stone) Shi 1a13b2ac56 Merge remote-tracking branch 'origin/release-53' into 0928-sync-release-53 2023-09-28 13:53:03 +02:00
ieQu1 b77e5e880a
Merge pull request #11697 from ieQu1/dev/gen-rpc-3.1.1
chore(gen_rpc): Bump version to 3.1.1
2023-09-28 10:03:36 +02:00
ieQu1 0aa3ccdd65 feat(gen_rpc): Add schema for the TLS versions and cipher suites 2023-09-28 00:36:59 +02:00
ieQu1 3c37f19105 chore(gen_rpc): Bump version to 3.1.1 2023-09-27 21:09:00 +02:00
firest ffd9a35d69 chore: Add missing changelog 2023-09-26 23:33:47 +08:00
Thales Macedo Garitezi aae59f1efd
Merge pull request #11682 from thalesmg/fix-file-logger-type-r53-20230925
fix(file_logger): change file logger type depending on rotation size
2023-09-26 10:22:36 -03:00
Thales Macedo Garitezi fdcd73e20c fix(file_logger): change file logger type depending on rotation size
Fixes https://emqx.atlassian.net/browse/EMQX-11036

From `logger_disk_log_h:open_disk_log`:

```erlang
open_disk_log(Name,File,Type,MaxNoBytes,MaxNoFiles) ->
    case filelib:ensure_dir(File) of
        ok ->
            Size =
                if Type==halt -> MaxNoBytes;
                   Type==wrap -> {MaxNoBytes,MaxNoFiles} %% <-------
                end,
            Opts = [{name,   Name},
                    {file,   File},
                    {size,   Size},
                    {type,   Type},
                    {linkto, self()},
                    {repair, false},
                    {format, external},
                    {notify, true},
                    {quiet,  true},
                    {mode,   read_write}],
            case disk_log:open(Opts) of
```

Affects all file loggers (audit included):

```
% emqx_config_logger:update_log_handler/1 -> ok
iex(emqx@127.0.0.1)14> Config override: log.file.emqx_audit is updated, but failed to add handler: {handler_not_added,
                                                                             {badarg,
                                                                              [{size,
                                                                                {infinity,
                                                                                 10}},
                                                                               {type,
                                                                                wrap},
                                                                               {linkto,
                                                                                <0.1952.0>},
                                                                               {repair,
                                                                                false},
                                                                               {format,
                                                                                external},
                                                                               {notify,
                                                                                true},
                                                                               {quiet,
                                                                                true},
                                                                               {mode,
                                                                                read_write}]}}
```
2023-09-26 09:04:44 -03:00
Zaiming (Stone) Shi a9df99746a docs: add changelog for PR 11661 2023-09-25 16:55:22 +02:00
Andrew Mayorov ddf6cdc0a2
chore: add changelog 2023-09-25 14:57:42 +03:00
Thales Macedo Garitezi d6935b6a67 feat: add port scan diagnostics to mria waiting for tables checks
Fixes https://emqx.atlassian.net/browse/EMQX-10944

Also updates ekka -> 0.15.15, mria -> 0.6.4

How to test
===========

1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop them in order.
3. Start only the first node that was stopped in the previous step.
4. Wait until the log is printed.

Or, more easily:

1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop all but one.
3. Run `mria_mnesia:diagnosis([]).` on that node.

Example output
==============

```
   Check check_open_ports should get ok but got #{msg =>
                                                     "some ports are unreachable",
                                                 results =>
                                                     #{'emqx@172.100.239.4' =>
                                                           #{open_ports =>
                                                                 #{4370 => false,
                                                                   5370 =>
                                                                       false},
                                                             ports_to_check =>
                                                                 [4370,5370],
                                                             resolved_ips =>
                                                                 [{172,100,239,
                                                                   4}],
                                                             status =>
                                                                 bad_ports},
                                                       'emqx@172.100.239.5' =>
                                                           #{open_ports =>
                                                                 #{4370 => false,
                                                                   5370 =>
                                                                       false},
                                                             ports_to_check =>
                                                                 [4370,5370],
                                                             resolved_ips =>
                                                                 [{172,100,239,
                                                                   5}],
                                                             status =>
                                                                 bad_ports}}}
```

After one node is back:

```
   Check check_open_ports should get ok but got #{msg =>
                                                     "some ports are unreachable",
                                                 results =>
                                                     #{'emqx@172.100.239.4' =>
                                                           #{ports_to_check =>
                                                                 [4370,5370],
                                                             resolved_ips =>
                                                                 [{172,100,239,
                                                                   4}],
                                                             status => ok},
                                                       'emqx@172.100.239.5' =>
                                                           #{open_ports =>
                                                                 #{4370 => false,
                                                                   5370 =>
                                                                       false},
                                                             ports_to_check =>
                                                                 [4370,5370],
                                                             resolved_ips =>
                                                                 [{172,100,239,
                                                                   5}],
                                                             status =>
                                                                 bad_ports}}}
```
2023-09-21 14:29:01 -03:00
Ivan Dyachkov 0a61d08a0f
Merge pull request #11651 from id/0921-sync-r52-to-master
sync r52 to master
2023-09-21 12:31:24 +02:00
William Yang 3a5227198e
Merge pull request #11642 from qzhuyan/dev/william/quicer-0.0.200
quicer 0.0.200
2023-09-21 12:20:14 +02:00
Ivan Dyachkov 105bebc250 chore: merge release-52 into master 2023-09-21 10:22:47 +02:00
zhongwencool 9521e11d39
Merge pull request #11584 from zhongwencool/telemetry-error-on-windows
fix: telemetry report error on windows
2023-09-21 09:08:55 +08:00
William Yang da626f6d29 feat(quic): bump to quicer 0.0.200 and emqtt 1.9.0 2023-09-20 16:43:15 +02:00
Zaiming (Stone) Shi 65818740f3
Merge pull request #11622 from zmstone/0917-upgrade-gen_rpc-to-3.1.0
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-19 10:22:53 +02:00
Zaiming (Stone) Shi abbee4bc47 fix(esockd): upgrade esockd from 5.9.6 to 5.9.7 2023-09-17 19:44:04 +02:00
Zaiming (Stone) Shi 1960dbd970 docs: add changelogs for pr 11622 2023-09-17 19:13:22 +02:00
Zaiming (Stone) Shi a9682c2989
Merge pull request #11605 from zmstone/0913-lower-severity-level-for-CMD-override-warnings
0913 lower severity level for cmd override warnings
2023-09-16 11:32:07 +02:00
Zaiming (Stone) Shi 062854eff7 docs: add changelog 2023-09-15 16:07:23 +02:00
Zaiming (Stone) Shi 2bf583087b docs: add changelog for pr 11605 2023-09-13 20:55:42 +02:00