Commit Graph

5766 Commits

Author SHA1 Message Date
Stefan Strigler 7bdb029745 refactor: use POST not PUT for `/users/{name}/change_pwd` 2022-12-13 11:08:54 +01:00
Thales Macedo Garitezi 464d0a5057 refactor(test): use a linked janitor for test teardown 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi b9bc82f87a feat(gcp_pubsub): add `local_topic` config
Given the implicit convention that an egress bridge containing the
`local_topic` config will forward messages without the need for a rule
action, this was added to avoid needing a rule action.
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi a095867358 test(refactor): add ids to namespace `on_exit` callbacks 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi 697b3ecf2d docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi 1cd91a24e9 feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0) 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi fffce9316c test(fix): avoid trying to connect to system_monitor db in tests
When `system_monitor`'s application is loaded, it sets a default
hostname:

```erlang
1> application:get_env(system_monitor, db_hostname).
undefined
2> application:load(system_monitor).
ok
3> application:get_env(system_monitor, db_hostname).
{ok,"localhost"}
```

And that makes tests crash when somehow it gets loaded.

By having a semantic `enable` field, we could avoid starting this
application by checking if the hostname is filled or not.

```
=ERROR REPORT==== 1-Dec-2022::13:57:20.855070 ===
** Generic server <0.7646.3> terminating
** Last message in was {command,epgsql_cmd_connect,
                                #{codecs => [],database => "postgres",
                                  host => [],
                                  password =>
                                      #Fun<epgsql_cmd_connect.0.29916615>,
                                  port => 5432,timeout => 5000,
                                  username => "system_monitor"}}
** When Server state == {state,undefined,undefined,<<>>,undefined,on_message,
                               undefined,
                               {[],[]},
                               undefined,undefined,undefined,undefined,[],
                               information_redacted,[],undefined,undefined,
                               undefined,undefined,undefined,#{}}
** Reason for termination ==
** {badarg,
       [{gen_tcp,connect_0,4,[{file,"gen_tcp.erl"},{line,207}]},
        {epgsql_cmd_connect,open_socket,2,
            [{file,
                 "/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl"},
             {line,94}]},
        {epgsql_cmd_connect,execute,2,
            [{file,
                 "/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl"},
             {line,54}]},
        {epgsql_sock,command_exec,4,
            [{file,"/emqx/_build/default/lib/epgsql/src/epgsql_sock.erl"},
             {line,338}]},
        {gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},
        {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},
        {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
** Client system_monitor_pg stacktrace
** [{gen,do_call,4,[{file,"gen.erl"},{line,214}]},
    {gen_server,call,3,[{file,"gen_server.erl"},{line,243}]},
    {epgsql,call_connect,2,
            [{file,"/emqx/_build/default/lib/epgsql/src/epgsql.erl"},
             {line,203}]},
    {system_monitor_pg,connect,0,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,151}]},
    {system_monitor_pg,initialize,0,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,142}]},
    {system_monitor_pg,handle_info,2,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,92}]},
    {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,695}]},
    {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,771}]},
    {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]

=CRASH REPORT==== 1-Dec-2022::13:57:20.855697 ===
  crasher:
    initial call: epgsql_sock:init/1
    pid: <0.7646.3>
    registered_name: []
    exception exit: badarg
      in function  gen_tcp:connect_0/4 (gen_tcp.erl, line 207)
      in call from epgsql_cmd_connect:open_socket/2 (/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl, line 94)
      in call from epgsql_cmd_connect:execute/2 (/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl, line 54)
      in call from epgsql_sock:command_exec/4 (/emqx/_build/default/lib/epgsql/src/epgsql_sock.erl, line 338)
      in call from gen_server:try_handle_call/4 (gen_server.erl, line 721)
      in call from gen_server:handle_msg/6 (gen_server.erl, line 750)
    ancestors: [system_monitor_pg,system_monitor2_sup,system_monitor_sup,
                  <0.4297.3>]
    message_queue_len: 0
    messages: []
    links: [<0.4303.3>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1598
    stack_size: 29
    reductions: 734
  neighbours:
```
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi c69022f3c4 test(ci): clean config after test
Trying to fix this error for profile `emqx`:

```
=CRASH REPORT==== 30-Nov-2022::13:25:46.763989 ===
  crasher:
    initial call: application_master:init/4
    pid: <0.9682.1>
    registered_name: []
    exception exit: {bad_return,
                        {{emqx_conf_app,start,[normal,[]]},
                         {emqx_conf_schema,
                             [#{kind => validation_error,path => "bridges",
                                reason => unknown_fields,
                                unknown => <<"influxdb_api_v1">>,
                                unmatched => <<"mqtt,webhook">>}]}}}
      in function  application_master:init/4 (application_master.erl, line 142)
    ancestors: [<0.9681.1>]
    message_queue_len: 1
    messages: [{'EXIT',<0.9683.1>,normal}]
    links: [<0.9681.1>,<0.44.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 29
    reductions: 195
  neighbours:

=INFO REPORT==== 30-Nov-2022::13:25:46.777895 ===
    application: emqx_conf
    exited: {bad_return,
                {{emqx_conf_app,start,[normal,[]]},
                 {emqx_conf_schema,
                     [#{kind => validation_error,path => "bridges",
                        reason => unknown_fields,
                        unknown => <<"influxdb_api_v1">>,
                        unmatched => <<"mqtt,webhook">>}]}}}
    type: temporary

%%% emqx_plugins_SUITE ==> init_per_suite: FAILED
%%% emqx_plugins_SUITE ==> {{failed_to_start_app,emqx_conf,
     {emqx_conf,
         {bad_return,
             {{emqx_conf_app,start,[normal,[]]},
              {emqx_conf_schema,
                  [#{kind => validation_error,path => "bridges",
                     reason => unknown_fields,
                     unknown => <<"influxdb_api_v1">>,
                     unmatched => <<"mqtt,webhook">>}]}}}}},
 [{emqx_common_test_helpers,start_app,4,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
       {line,227}]},
  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
  {emqx_plugins_SUITE,init_per_suite,1,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
       {line,34}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1380}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}

%%% emqx_plugins_SUITE ==> t_bad_info_json: SKIPPED
%%% emqx_plugins_SUITE ==> {tc_auto_skip,
    {failed,
        {emqx_plugins_SUITE,init_per_suite,
            {'EXIT',
                {{failed_to_start_app,emqx_conf,
                     {emqx_conf,
                         {bad_return,
                             {{emqx_conf_app,start,[normal,[]]},
                              {emqx_conf_schema,
                                  [#{kind => validation_error,
                                     path => "bridges",
                                     reason => unknown_fields,
                                     unknown => <<"influxdb_api_v1">>,
                                     unmatched => <<"mqtt,webhook">>}]}}}}},
                 [{emqx_common_test_helpers,start_app,4,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
                       {line,227}]},
                  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
                  {emqx_plugins_SUITE,init_per_suite,1,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
                       {line,34}]},
                  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
                  {test_server,run_test_case_eval1,6,
                      [{file,"test_server.erl"},{line,1380}]},
                  {test_server,run_test_case_eval,9,
                      [{file,"test_server.erl"},{line,1224}]}]}}}}}
```
2022-12-12 17:18:13 -03:00
Thales Macedo Garitezi 34e9056779 refactor: fix typo in variable name
Might confuse people to think it's related to `replayq`.
2022-12-12 17:17:51 -03:00
Zaiming (Stone) Shi f8685e3a58
Merge pull request #9437 from zhongwencool/authn-env-failed
fix: set authentication array from ENV failed
2022-12-12 19:12:27 +01:00
Zhongwen Deng e57a1f2c8b fix: set authentication array from ENV failed 2022-12-12 14:18:30 +01:00
Erik Timan 0242d5f360 test: rename return_body to return_all in emqx_mgmt api test util 2022-12-12 13:29:11 +01:00
Erik Timan 8009f0a3a8 test: api returns for lwm2m client posts
Add test coverage for HTTP 204 returns from lwm2m client API
posts. This includes a small refactoring of the request functionality
in emqx_mgmt_api_test_util.
2022-12-12 13:28:24 +01:00
Erik Timan a8c9d02871 fix(emqx_gateway): return 204 for lwm2m client posts
Change the return for lwm2m client API posts from 200 to 204 since
they contain no content.
2022-12-12 13:28:24 +01:00
Stefan Strigler 6b22a074f0 refactor: move `/mqtt/sys_topics` to generic `/configs/sys_topics` 2022-12-12 13:25:42 +01:00
Zaiming (Stone) Shi c2eabfe5f7
Merge pull request #9523 from qzhuyan/fix/william/bridges-ingress-and-egress
Fix/william/bridges ingress and egress
2022-12-12 09:36:08 +01:00
Zaiming (Stone) Shi 61da68ff40 fix(emqx_schema): handshake_timeout is common for all listeners 2022-12-09 16:47:19 +01:00
William Yang 6b6bb09d4a fix: ingress only bridge causes egress bridge traffic stop 2022-12-09 15:36:20 +01:00
Stefan Strigler 85f54220b0
Merge pull request #9494 from sstrigler/EMQX-7992-post-trace-must-respond-with-409-on-duplicate-entry
fix: return 409 in case of duplicate
2022-12-09 15:10:23 +01:00
Zaiming (Stone) Shi 3084d1263b
Merge pull request #9519 from zmstone/1209-chore-merge-dev/ee5.0-to-release-50
1209 chore merge dev/ee5.0 to release 50
2022-12-09 14:29:13 +01:00
Stefan Strigler 2a27d2e781 fix: return 409 in case of duplicate 2022-12-09 13:33:34 +01:00
Zaiming (Stone) Shi 6a4fb1241b Merge remote-tracking branch 'origin/release-50' into 1209-chore-merge-dev/ee5.0-to-release-50 2022-12-09 12:54:16 +01:00
Stefan Strigler 1472cfefd0 fix: flip order of checks to return 404 before 400 2022-12-09 12:53:04 +01:00
Stefan Strigler 1f7c02ecf5 fix: return 400 if node in query doesn't look like a known node 2022-12-09 12:53:02 +01:00
Ilya Averyanov a26964291d
Merge pull request #9496 from savonarola/fix-auth-chain
Fix `emqx_authentication` hook cooperation with other hooks
2022-12-09 14:48:18 +03:00
Ilya Averyanov 7ae3ecfccb chore(retainer): optimize index writes 2022-12-09 14:42:43 +03:00
Ilya Averyanov cd0ae62995 fix(auth): fix emqx_authenticator cooperation with other 'client.authenticate' hooks 2022-12-09 13:57:48 +03:00
Zaiming (Stone) Shi 40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
lafirest 3705d5bc73
Merge pull request #9505 from lafirest/fix/psk_r50
fix(psk): Add more PSK ciphers support
2022-12-09 00:00:52 +08:00
Zaiming (Stone) Shi e1f09d9899 Merge remote-tracking branch 'origin/release-50' into 1208-merge-release-50-back-to-master 2022-12-08 14:27:01 +01:00
firest dac8845b5d test(psk): add PSK ciphers test case 2022-12-08 11:41:57 +08:00
firest 21a908167d fix(psk): Add more PSK ciphers support 2022-12-08 11:41:42 +08:00
Zaiming (Stone) Shi dc14cd450d test: render config for emqx_conf only for ee bridge tests 2022-12-07 18:27:06 +01:00
Zaiming (Stone) Shi c9d84b2fda test: do not load emqx_conf app when it's not found
also remove stale code in emqx_common_test_helpers
2022-12-07 16:49:54 +01:00
Zaiming (Stone) Shi d89dbb7caa Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-07 16:24:13 +01:00
Zaiming (Stone) Shi f3b069a0d9 test: fix config load for lib-ee tests 2022-12-07 15:50:50 +01:00
Zaiming (Stone) Shi b897d64b75 test: fix emqx_frame_SUITE 2022-12-07 12:20:03 +01:00
Ilya Averyanov 6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi 31098d6c67 test: fix bridge api tests, metrics is now a separate api 2022-12-06 19:03:32 +01:00
Zaiming (Stone) Shi bd65e8aad6 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:42:50 +01:00
Zaiming (Stone) Shi 1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Stefan Strigler 0b324da7cb refactor: move metrics out of /rules(/:id) to /rules/:id/metrics 2022-12-06 15:23:28 +01:00
Zaiming (Stone) Shi 380b75d4a9
Merge pull request #9422 from emqx/1124-refactor-avoid-creating-atom-at-run
refactor: use static function references
2022-12-06 14:24:05 +01:00
Zaiming (Stone) Shi cca3421308 refactor: use static function references 2022-12-06 09:40:03 +01:00
lafirest 3e29e37691
Merge pull request #9477 from lafirest/fix/fuzzing_errors_v50
Fix/fuzzing errors v50
2022-12-06 08:22:46 +08:00
Zaiming (Stone) Shi 817554c7d7 refactor(emqx_frame): better error handling 2022-12-06 01:48:52 +08:00
Zaiming (Stone) Shi 16aeba24b7
Merge pull request #9472 from zmstone/1203-merge-release-50-to-master
1203 merge release 50 to master
2022-12-05 16:39:51 +01:00
firest 72669080a1 fix(frame): fix potential parse errors found by fuzzing test 2022-12-05 23:16:07 +08:00
Stefan Strigler 0be5c8e84b fix: Revert "refactor: return `204` instead of `200` for `PUT /authenticator/:id`"
This reverts commit 3bda905e92.
2022-12-05 15:52:16 +01:00
Stefan Strigler 2443500884 fix: return 404 for /telemetry/data in case it's disabled 2022-12-05 12:41:27 +01:00
Stefan Strigler 1ba66fc722 docs: /trace returns list not single trace 2022-12-05 11:50:52 +01:00
Stefan Strigler f8dd8d652d docs: add swagger error codes for /trace/:name/[log|download] 2022-12-05 11:50:52 +01:00
Stefan Strigler 9100b13db6 docs: fix schema for /trace/:name/log 2022-12-05 11:50:52 +01:00
Zhongwen Deng 64ab4ca025 test: add copy data dir test case 2022-12-05 16:25:28 +08:00
Stefan Strigler 3bda905e92 refactor: return `204` instead of `200` for `PUT /authenticator/:id` 2022-12-05 09:19:55 +01:00
Zhongwen Deng 9b37837c9c fix: don't save cert/authz zip on disk when emqx start 2022-12-05 16:12:21 +08:00
Zaiming (Stone) Shi 9fd1864d17 Merge remote-tracking branch 'origin/release-50' 2022-12-03 12:58:42 +01:00
Stefan Strigler 5b4feca3cb
Merge pull request #9419 from sstrigler/EMQX-7863-post-authentication-id
refactor: use PUT on `/position` to move authenticator
2022-12-02 14:07:59 +01:00
Erik Timan 485c4ddcab
Merge pull request #9393 from olcai/add-mysql-bridge-test
test: add basic mysql bridge test (ee5.0)
2022-12-02 09:43:22 +01:00
lafirest 4cc13c4aa5
Merge pull request #9133 from lafirest/fix/limiter_api
fix(limiter): add `limiter` back to the configs API
2022-12-02 10:06:03 +08:00
Zaiming (Stone) Shi eb017ab034 test: add test case to cover mqtt bridge config upgrade 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi 9ea22d062d refactor: make all bridges optional (required = false) 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi 7adb539ae1 chore: pin hocon 0.31.2 2022-12-01 16:50:10 +01:00
Erik Timan 28fc2b9ee9 fix(mgmt_api): add missing error codes in schema 2022-12-01 12:20:11 +01:00
Erik Timan 5461311edc test: more EE mysql bridge tests 2022-12-01 10:27:51 +01:00
Erik Timan 96bff1d32e test: improve basic tests for EE mysql bridge 2022-12-01 10:27:51 +01:00
firest 88a0c77707 fix(limiter): add `limiter` back to the configs API 2022-12-01 15:07:15 +08:00
Zaiming (Stone) Shi 9a6901987f feat: add mqtt bridge config upgrade converter 2022-11-30 21:01:59 +01:00
Zaiming (Stone) Shi 983e904858 fix(connector): fix ssl clear 2022-11-30 17:03:41 +01:00
firest 81586a4c8d chore: bump connector version 2022-11-30 23:14:54 +08:00
firest 51ace31f30 fix(connector): fix previous incorrectly fixes
There was an error in the try_clear_certs2/3 clause that the argument isn't the ssl field but including it.
2022-11-30 22:14:44 +08:00
zhongwencool 5a2265a286
Merge pull request #9407 from v-zhuravlev/master
Update grafana dashboards
2022-11-30 17:03:39 +08:00
Stefan Strigler b124e64399 refactor: use PUT on `/position` to move authenticator 2022-11-29 17:28:37 +01:00
lafirest e6e559710b
Merge pull request #9432 from lafirest/fix/exhook_ssl_management
fix(exhook): fix obsolete SSL files aren't deleted after the Exhook config update
2022-11-29 22:32:10 +08:00
Zaiming (Stone) Shi 311e4cd583
Merge pull request #9440 from zmstone/1128-chore-sync-release-branch-back-to-master
1128 chore sync release branch back to master
2022-11-29 11:47:01 +01:00
firest 606389edd2 chore: update changes && bump version 2022-11-29 10:41:21 +08:00
firest f9bc700498 fix(exhook): fix obsolete SSL files aren't deleted after the Exhook config update 2022-11-29 10:40:08 +08:00
Zaiming (Stone) Shi b398617614 chore: bump app versions 2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi 77341e7a3c chore: bump app versions 2022-11-28 21:06:33 +01:00
Zaiming (Stone) Shi 9f3a9efbad Merge remote-tracking branch 'origin/release-v50' 2022-11-28 21:02:39 +01:00
Zaiming (Stone) Shi 7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
Erik Timan 2582c1713e fix: emqx_topic:join/1 instead of filename:join/1
filename:join/1 doesn't work on Windows for joining a topic. We now use
emqx_topic:join/1 instead.
2022-11-28 11:00:18 +01:00
Erik Timan 95faf56077 feat: remove $queue in favor of $shared
This commit removes support for setting shared subscriptions with the
non-standard $queue feature. Shared subscriptions is now part of the MQTT
spec (using $shared) and we will only support that from now on.
2022-11-28 11:00:18 +01:00
Zaiming (Stone) Shi 05719daff2 chore: bump version to v5.0.11 2022-11-27 11:15:28 +01:00
Zaiming (Stone) Shi 970b3eb1a0 chore: bump to version v5.0.11-rc.1 2022-11-26 15:51:38 +01:00
Vitaly Zhuravlev 5c9fff43a2 chore: add proper units for rates on the dashboard 2022-11-25 19:12:36 +03:00
Vitaly Zhuravlev 5af1d6662f chore: update grafana dashboards 2022-11-25 19:12:25 +03:00
Zhongwen Deng 65a1f90f58 Merge branch 'global-gc-interval' of https://github.com/zhongwencool/emqx into global-gc-interval 2022-11-25 16:39:02 +08:00
Zhongwen Deng 28bede0377 chore: update i18n en description 2022-11-25 16:38:40 +08:00
zhongwencool 2fbd6c9690
Merge branch 'master' into global-gc-interval 2022-11-25 12:06:16 +08:00
lafirest 345fa65241
Merge pull request #9416 from lafirest/fix/emqx_ctl
fix(ctl): fix unsafe `list_to_atom`
2022-11-25 09:58:07 +08:00
Zhongwen Deng 59ac5320b1 chore: update emqx_conf/emqx_machine version 2022-11-25 07:33:08 +08:00
Zaiming (Stone) Shi 6ee475d9b1 fix(emqx_authz_api_mnesia): return the right matchers 2022-11-24 20:32:00 +01:00
JianBo He dbdb78d38a chore: clarify the case when count returns zero 2022-11-24 20:28:50 +01:00
JianBo He edb35c08a8 chore: refactor ms2qs function type 2022-11-24 20:14:33 +01:00
JianBo He 9786a6c267 refactor(mgmt): convert fuzzy filter func to named func 2022-11-24 20:14:33 +01:00
JianBo He ffb3f2d2d2 chore: change emqx_live_connection tab type to ordered_set
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-11-24 20:14:33 +01:00
JianBo He 6d9e1e0d7a test(mgmt): cover emqx_mgmt_api:cluster_query 2022-11-24 20:14:33 +01:00
JianBo He 36de83a50a feat(cm): change emqx_channel_info to ordered_set table 2022-11-24 20:14:33 +01:00
JianBo He b9c5a5f822 fix(delayed): return correct node name 2022-11-24 20:14:33 +01:00