Commit Graph

12414 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
464d0a5057 refactor(test): use a linked janitor for test teardown 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
b66d2904be docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
b9bc82f87a feat(gcp_pubsub): add local_topic config
Given the implicit convention that an egress bridge containing the
`local_topic` config will forward messages without the need for a rule
action, this was added to avoid needing a rule action.
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
ac048dbafa docs(gcp_pubsub): describe where to get credentials 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
a095867358 test(refactor): add ids to namespace on_exit callbacks 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
697b3ecf2d docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
1cd91a24e9 feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0) 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
82be9d878d test(flaky): avoid inter-suite flakiness 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
fffce9316c test(fix): avoid trying to connect to system_monitor db in tests
When `system_monitor`'s application is loaded, it sets a default
hostname:

```erlang
1> application:get_env(system_monitor, db_hostname).
undefined
2> application:load(system_monitor).
ok
3> application:get_env(system_monitor, db_hostname).
{ok,"localhost"}
```

And that makes tests crash when somehow it gets loaded.

By having a semantic `enable` field, we could avoid starting this
application by checking if the hostname is filled or not.

```
=ERROR REPORT==== 1-Dec-2022::13:57:20.855070 ===
** Generic server <0.7646.3> terminating
** Last message in was {command,epgsql_cmd_connect,
                                #{codecs => [],database => "postgres",
                                  host => [],
                                  password =>
                                      #Fun<epgsql_cmd_connect.0.29916615>,
                                  port => 5432,timeout => 5000,
                                  username => "system_monitor"}}
** When Server state == {state,undefined,undefined,<<>>,undefined,on_message,
                               undefined,
                               {[],[]},
                               undefined,undefined,undefined,undefined,[],
                               information_redacted,[],undefined,undefined,
                               undefined,undefined,undefined,#{}}
** Reason for termination ==
** {badarg,
       [{gen_tcp,connect_0,4,[{file,"gen_tcp.erl"},{line,207}]},
        {epgsql_cmd_connect,open_socket,2,
            [{file,
                 "/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl"},
             {line,94}]},
        {epgsql_cmd_connect,execute,2,
            [{file,
                 "/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl"},
             {line,54}]},
        {epgsql_sock,command_exec,4,
            [{file,"/emqx/_build/default/lib/epgsql/src/epgsql_sock.erl"},
             {line,338}]},
        {gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},
        {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},
        {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
** Client system_monitor_pg stacktrace
** [{gen,do_call,4,[{file,"gen.erl"},{line,214}]},
    {gen_server,call,3,[{file,"gen_server.erl"},{line,243}]},
    {epgsql,call_connect,2,
            [{file,"/emqx/_build/default/lib/epgsql/src/epgsql.erl"},
             {line,203}]},
    {system_monitor_pg,connect,0,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,151}]},
    {system_monitor_pg,initialize,0,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,142}]},
    {system_monitor_pg,handle_info,2,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,92}]},
    {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,695}]},
    {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,771}]},
    {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]

=CRASH REPORT==== 1-Dec-2022::13:57:20.855697 ===
  crasher:
    initial call: epgsql_sock:init/1
    pid: <0.7646.3>
    registered_name: []
    exception exit: badarg
      in function  gen_tcp:connect_0/4 (gen_tcp.erl, line 207)
      in call from epgsql_cmd_connect:open_socket/2 (/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl, line 94)
      in call from epgsql_cmd_connect:execute/2 (/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl, line 54)
      in call from epgsql_sock:command_exec/4 (/emqx/_build/default/lib/epgsql/src/epgsql_sock.erl, line 338)
      in call from gen_server:try_handle_call/4 (gen_server.erl, line 721)
      in call from gen_server:handle_msg/6 (gen_server.erl, line 750)
    ancestors: [system_monitor_pg,system_monitor2_sup,system_monitor_sup,
                  <0.4297.3>]
    message_queue_len: 0
    messages: []
    links: [<0.4303.3>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1598
    stack_size: 29
    reductions: 734
  neighbours:
```
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
cdae451a84 ci(fix): ensure directory for i18n output exists
```
Run make eunit
escript: exception error: no match of right hand side value {error,enoent}
  in function  'merge-i18n_escript__escript__1669__814803__620426__2':main/1 (/__w/emqx/emqx/source/scripts/merge-i18n.escript, line 13)
  in call from escript:run/2 (escript.erl, line 750)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1
  in call from init:do_boot/3
make: *** [Makefile:225: conf-segs] Error 127
```
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
c69022f3c4 test(ci): clean config after test
Trying to fix this error for profile `emqx`:

```
=CRASH REPORT==== 30-Nov-2022::13:25:46.763989 ===
  crasher:
    initial call: application_master:init/4
    pid: <0.9682.1>
    registered_name: []
    exception exit: {bad_return,
                        {{emqx_conf_app,start,[normal,[]]},
                         {emqx_conf_schema,
                             [#{kind => validation_error,path => "bridges",
                                reason => unknown_fields,
                                unknown => <<"influxdb_api_v1">>,
                                unmatched => <<"mqtt,webhook">>}]}}}
      in function  application_master:init/4 (application_master.erl, line 142)
    ancestors: [<0.9681.1>]
    message_queue_len: 1
    messages: [{'EXIT',<0.9683.1>,normal}]
    links: [<0.9681.1>,<0.44.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 29
    reductions: 195
  neighbours:

=INFO REPORT==== 30-Nov-2022::13:25:46.777895 ===
    application: emqx_conf
    exited: {bad_return,
                {{emqx_conf_app,start,[normal,[]]},
                 {emqx_conf_schema,
                     [#{kind => validation_error,path => "bridges",
                        reason => unknown_fields,
                        unknown => <<"influxdb_api_v1">>,
                        unmatched => <<"mqtt,webhook">>}]}}}
    type: temporary

%%% emqx_plugins_SUITE ==> init_per_suite: FAILED
%%% emqx_plugins_SUITE ==> {{failed_to_start_app,emqx_conf,
     {emqx_conf,
         {bad_return,
             {{emqx_conf_app,start,[normal,[]]},
              {emqx_conf_schema,
                  [#{kind => validation_error,path => "bridges",
                     reason => unknown_fields,
                     unknown => <<"influxdb_api_v1">>,
                     unmatched => <<"mqtt,webhook">>}]}}}}},
 [{emqx_common_test_helpers,start_app,4,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
       {line,227}]},
  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
  {emqx_plugins_SUITE,init_per_suite,1,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
       {line,34}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1380}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}

%%% emqx_plugins_SUITE ==> t_bad_info_json: SKIPPED
%%% emqx_plugins_SUITE ==> {tc_auto_skip,
    {failed,
        {emqx_plugins_SUITE,init_per_suite,
            {'EXIT',
                {{failed_to_start_app,emqx_conf,
                     {emqx_conf,
                         {bad_return,
                             {{emqx_conf_app,start,[normal,[]]},
                              {emqx_conf_schema,
                                  [#{kind => validation_error,
                                     path => "bridges",
                                     reason => unknown_fields,
                                     unknown => <<"influxdb_api_v1">>,
                                     unmatched => <<"mqtt,webhook">>}]}}}}},
                 [{emqx_common_test_helpers,start_app,4,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
                       {line,227}]},
                  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
                  {emqx_plugins_SUITE,init_per_suite,1,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
                       {line,34}]},
                  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
                  {test_server,run_test_case_eval1,6,
                      [{file,"test_server.erl"},{line,1380}]},
                  {test_server,run_test_case_eval,9,
                      [{file,"test_server.erl"},{line,1224}]}]}}}}}
```
2022-12-12 17:18:13 -03:00
Thales Macedo Garitezi
34e9056779 refactor: fix typo in variable name
Might confuse people to think it's related to `replayq`.
2022-12-12 17:17:51 -03:00
Zaiming (Stone) Shi
061fe144b8
Merge pull request #9516 from emqx/1209-remove-allgood-checks-in-ci
ci: remove allgood checks
2022-12-09 10:21:24 +01:00
Zaiming (Stone) Shi
fd1921c279
Merge pull request #9512 from zmstone/1208-merge-release-50-back-to-master
1208 merge release 50 back to master
2022-12-09 10:18:55 +01:00
Zaiming (Stone) Shi
84824c374f ci: remove allgood checks 2022-12-09 09:24:14 +01:00
Zaiming (Stone) Shi
241feb04e3 chore: delete stale files 2022-12-08 14:28:04 +01:00
Zaiming (Stone) Shi
e1f09d9899 Merge remote-tracking branch 'origin/release-50' into 1208-merge-release-50-back-to-master 2022-12-08 14:27:01 +01:00
Zaiming (Stone) Shi
9da12a0814
Merge pull request #9482 from emqx/1206-chore-merge-ee50-to-release-50
Merge ee50 to release-50
2022-12-08 14:12:33 +01:00
Ivan Dyachkov
316505b314
Merge pull request #9495 from id/chore-fix-warngings-in-bin-emqx
fix warnings in bin emqx
2022-12-07 21:24:24 +01:00
Zaiming (Stone) Shi
dc14cd450d test: render config for emqx_conf only for ee bridge tests 2022-12-07 18:27:06 +01:00
Zaiming (Stone) Shi
c87c9e886e ci: rename build step for docker-ct 2022-12-07 17:34:17 +01:00
Zaiming (Stone) Shi
1c8a9079b9 ci: remove undefined matrix from artifact name 2022-12-07 17:27:59 +01:00
Zaiming (Stone) Shi
c9d84b2fda test: do not load emqx_conf app when it's not found
also remove stale code in emqx_common_test_helpers
2022-12-07 16:49:54 +01:00
Zaiming (Stone) Shi
d89dbb7caa Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-07 16:24:13 +01:00
Zaiming (Stone) Shi
f3b069a0d9 test: fix config load for lib-ee tests 2022-12-07 15:50:50 +01:00
Ivan Dyachkov
ec826f03c9 chore: colorize warning and error logs 2022-12-07 14:42:09 +01:00
Ivan Dyachkov
9e1b3fb47e chore: remove exclamation marks around warngings in bin/emqx 2022-12-07 14:04:25 +01:00
Ivan Dyachkov
04d3385229 chore: ensure default stat on macos when resolving running root dir 2022-12-07 14:02:35 +01:00
Zaiming (Stone) Shi
6ae289e33b
Merge pull request #9484 from zmstone/1206-fix-tests
test: fix emqx_frame_SUITE
2022-12-07 13:54:50 +01:00
Zaiming (Stone) Shi
b897d64b75 test: fix emqx_frame_SUITE 2022-12-07 12:20:03 +01:00
Zaiming (Stone) Shi
31098d6c67 test: fix bridge api tests, metrics is now a separate api 2022-12-06 19:03:32 +01:00
Zaiming (Stone) Shi
bd65e8aad6 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:42:50 +01:00
Zaiming (Stone) Shi
1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Stefan Strigler
fa80c4dfcb
Merge pull request #9461 from sstrigler/EMQX-7988-move-metrics-out-of-get-rules-response-to-separate-api
refactor: move metrics out of /rules(/:id) to /rules/:id/metrics
2022-12-06 16:33:31 +01:00
Zaiming (Stone) Shi
64effacaa7 chore: pin dashboard v1.1.3-sync-code 2022-12-06 16:31:16 +01:00
Stefan Strigler
0b324da7cb refactor: move metrics out of /rules(/:id) to /rules/:id/metrics 2022-12-06 15:23:28 +01:00
Zaiming (Stone) Shi
380b75d4a9
Merge pull request #9422 from emqx/1124-refactor-avoid-creating-atom-at-run
refactor: use static function references
2022-12-06 14:24:05 +01:00
Zaiming (Stone) Shi
cca3421308 refactor: use static function references 2022-12-06 09:40:03 +01:00
lafirest
3e29e37691
Merge pull request #9477 from lafirest/fix/fuzzing_errors_v50
Fix/fuzzing errors v50
2022-12-06 08:22:46 +08:00
Zaiming (Stone) Shi
817554c7d7 refactor(emqx_frame): better error handling 2022-12-06 01:48:52 +08:00
Stefan Strigler
57c0c9db79
Merge pull request #9479 from sstrigler/EMQX-7982-revert
fix: Revert "refactor: return `204` instead of `200` for `PUT /authen…
2022-12-05 16:43:09 +01:00
Zaiming (Stone) Shi
16aeba24b7
Merge pull request #9472 from zmstone/1203-merge-release-50-to-master
1203 merge release 50 to master
2022-12-05 16:39:51 +01:00
firest
72669080a1 fix(frame): fix potential parse errors found by fuzzing test 2022-12-05 23:16:07 +08:00
Stefan Strigler
0be5c8e84b fix: Revert "refactor: return 204 instead of 200 for PUT /authenticator/:id"
This reverts commit 3bda905e92.
2022-12-05 15:52:16 +01:00
William Yang
4a9dd09f04
Merge pull request #9433 from qzhuyan/ci/william/runnable-ct-in-fork
ci: make test runnable without self-hosted in fork repo
2022-12-05 14:18:05 +01:00
Stefan Strigler
546addccc5
Merge pull request #9464 from sstrigler/EMQX-7984-get-telemetry-data-must-respond-with-404-when-telemetry-is-disabled
fix: return 404 for /telemetry/data in case it's disabled
2022-12-05 13:16:40 +01:00
Stefan Strigler
2443500884 fix: return 404 for /telemetry/data in case it's disabled 2022-12-05 12:41:27 +01:00
Stefan Strigler
cfd792add4
Merge pull request #9468 from sstrigler/EMQX-7993-get-trace-name-log-actual-response-is-inconsistent-with-docs
Various doc fixes for /trace API
2022-12-05 12:39:37 +01:00
William Yang
4c493affda ci: make test runnable without self-hosted in fork repo 2022-12-05 12:16:46 +01:00
Stefan Strigler
5820bae7be chore: add zn translation 2022-12-05 11:50:52 +01:00