yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,722 Commits 393 Branches 684 Tags 74 MiB
Commit Graph

235 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 82be9d878d test(flaky): avoid inter-suite flakiness 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi c69022f3c4 test(ci): clean config after test 
Trying to fix this error for profile `emqx`:

```
=CRASH REPORT==== 30-Nov-2022::13:25:46.763989 ===
  crasher:
    initial call: application_master:init/4
    pid: <0.9682.1>
    registered_name: []
    exception exit: {bad_return,
                        {{emqx_conf_app,start,[normal,[]]},
                         {emqx_conf_schema,
                             [#{kind => validation_error,path => "bridges",
                                reason => unknown_fields,
                                unknown => <<"influxdb_api_v1">>,
                                unmatched => <<"mqtt,webhook">>}]}}}
      in function  application_master:init/4 (application_master.erl, line 142)
    ancestors: [<0.9681.1>]
    message_queue_len: 1
    messages: [{'EXIT',<0.9683.1>,normal}]
    links: [<0.9681.1>,<0.44.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 29
    reductions: 195
  neighbours:

=INFO REPORT==== 30-Nov-2022::13:25:46.777895 ===
    application: emqx_conf
    exited: {bad_return,
                {{emqx_conf_app,start,[normal,[]]},
                 {emqx_conf_schema,
                     [#{kind => validation_error,path => "bridges",
                        reason => unknown_fields,
                        unknown => <<"influxdb_api_v1">>,
                        unmatched => <<"mqtt,webhook">>}]}}}
    type: temporary

%%% emqx_plugins_SUITE ==> init_per_suite: FAILED
%%% emqx_plugins_SUITE ==> {{failed_to_start_app,emqx_conf,
     {emqx_conf,
         {bad_return,
             {{emqx_conf_app,start,[normal,[]]},
              {emqx_conf_schema,
                  [#{kind => validation_error,path => "bridges",
                     reason => unknown_fields,
                     unknown => <<"influxdb_api_v1">>,
                     unmatched => <<"mqtt,webhook">>}]}}}}},
 [{emqx_common_test_helpers,start_app,4,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
       {line,227}]},
  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
  {emqx_plugins_SUITE,init_per_suite,1,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
       {line,34}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1380}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}

%%% emqx_plugins_SUITE ==> t_bad_info_json: SKIPPED
%%% emqx_plugins_SUITE ==> {tc_auto_skip,
    {failed,
        {emqx_plugins_SUITE,init_per_suite,
            {'EXIT',
                {{failed_to_start_app,emqx_conf,
                     {emqx_conf,
                         {bad_return,
                             {{emqx_conf_app,start,[normal,[]]},
                              {emqx_conf_schema,
                                  [#{kind => validation_error,
                                     path => "bridges",
                                     reason => unknown_fields,
                                     unknown => <<"influxdb_api_v1">>,
                                     unmatched => <<"mqtt,webhook">>}]}}}}},
                 [{emqx_common_test_helpers,start_app,4,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
                       {line,227}]},
                  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
                  {emqx_plugins_SUITE,init_per_suite,1,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
                       {line,34}]},
                  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
                  {test_server,run_test_case_eval1,6,
                      [{file,"test_server.erl"},{line,1380}]},
                  {test_server,run_test_case_eval,9,
                      [{file,"test_server.erl"},{line,1224}]}]}}}}}
```
 2022-12-12 17:18:13 -03:00
Thales Macedo Garitezi 34e9056779 refactor: fix typo in variable name 
Might confuse people to think it's related to `replayq`.
 2022-12-12 17:17:51 -03:00
Zaiming (Stone) Shi 40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
Zaiming (Stone) Shi dc14cd450d test: render config for emqx_conf only for ee bridge tests 2022-12-07 18:27:06 +01:00
Ilya Averyanov 6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Erik Timan 485c4ddcab
Merge pull request #9393 from olcai/add-mysql-bridge-test 
test: add basic mysql bridge test (ee5.0)
 2022-12-02 09:43:22 +01:00
Zaiming (Stone) Shi 9ea22d062d refactor: make all bridges optional (required = false) 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi 7adb539ae1 chore: pin hocon 0.31.2 2022-12-01 16:50:10 +01:00
Erik Timan 499a32ce36 test: remove unnecessary async tests in EE mysql bridge 
The async test cases is not needed since the mysql connector uses the
always_sync callback mode.
 2022-12-01 16:38:03 +01:00
Erik Timan 5461311edc test: more EE mysql bridge tests 2022-12-01 10:27:51 +01:00
Erik Timan eb62192838 test: expand EE mysql bridge test with toxiproxy 2022-12-01 10:27:51 +01:00
Erik Timan 96bff1d32e test: improve basic tests for EE mysql bridge 2022-12-01 10:27:51 +01:00
Erik Timan 3e679ceb57 test: add basic tests for EE mysql bridge 2022-12-01 10:27:51 +01:00
Zaiming (Stone) Shi fa90638de2 chore: add description to apps 2022-11-30 17:03:41 +01:00
Zaiming (Stone) Shi 9382399728 chore: add an empty file emqx_license.conf to make ci happy 2022-11-30 12:52:26 +01:00
Zaiming (Stone) Shi 83fe35b148 Merge remote-tracking branch 'origin/dev/ee5.0' into 1116-sync-v5.0.10-to-ee50 2022-11-18 19:23:31 +01:00
Zaiming (Stone) Shi 7305923d12 fix: bridge name parser should not leak atom 2022-11-16 18:25:42 +01:00
Zaiming (Stone) Shi c940b901f5 chore: fix app versions 2022-11-16 16:26:43 +01:00
Thales Macedo Garitezi 0f9cc0d93f test(refactor): stop snabbkaffe on every test 2022-11-11 10:03:30 -03:00
Thales Macedo Garitezi 04588148b7 test(influxdb): increase influxdb bridge/connector coverage (ee5.0) 2022-11-07 15:15:49 -03:00
Thales Macedo Garitezi ee4c723fcb refactor: simplify wolff telemetry handler id 2022-10-17 16:26:04 -03:00
Thales Macedo Garitezi 1b2b629cdd feat: emit telemetry events for all resource worker metrics 2022-10-13 15:32:04 -03:00
Thales Macedo Garitezi 1ad3b5df17 fix: uninstall telemetry handler on resource stop, use unique id 2022-10-13 10:29:53 -03:00
Thales Macedo Garitezi 4475289ce4 feat: use upstream newly tagged 1.7.0 wolff 2022-10-11 09:47:40 -03:00
Thales Macedo Garitezi 98500313eb fix(kafka): some fixes for kafka producer 
- MQTT topic should be a binary
- use correct gauge functions from `wolff_metrics`.
- don't double increment success counter for kafka action
- adds a few more metrics assertions
 2022-10-10 17:11:29 -03:00
Kjell Winblad 57270fb8fc feat: add support for counters and gauges to the Kafka Bridge 
This commit adds support for counters and gauges to the Kafka Brige.
The Kafka bridge uses [Wolff](https://github.com/kafka4beam/wolff) for
the  Kafka connection. Wolff does its own batching and does not use the
batching functionality in `emqx_resource_worker` that is used by other
bridge types. Therefore, the counter events have to be generated by
Wolff. We have added
[telemetry](https://github.com/beam-telemetry/telemetry) events to Wolff
that we hook into to change counters and gauges for the Kafka bridge. The
counter called `matched` does not depend on specific functionality of
any bridge type so the updates of this counter is moved higher up in the
call chain then previously so that it also gets updated for Kafka
bridges.
 2022-10-10 14:40:57 -03:00
Shawn dfe14be8b2 chore: bump app vsns 2022-09-26 18:23:28 +08:00
Shawn 4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
Zaiming (Stone) Shi f6ac4c3a76
Merge pull request #8798 from zmstone/0815-feat-add-kafka-connector 
feat: Add Kafka connector
 2022-09-24 22:57:50 +02:00
Kjell Winblad a3c88b40a0 test: changes to make Kafka container run in GitHub action 2022-09-23 14:33:41 +02:00
Kjell Winblad 8e514680d8 test: fix bad binary pattern 2022-09-23 13:54:11 +02:00
Zaiming (Stone) Shi 7b601bf970 chore: delete bad parse_bridge function clause 2022-09-23 11:53:02 +02:00
Kjell Winblad adc67b165b test: test cases for Kafka bridge REST API 2022-09-23 10:09:07 +02:00
JimMoen 6788881642 fix: hide influxdb udp config 2022-09-20 14:46:59 +08:00
JimMoen 03495d8d36 fix: influxdb server string support scheme prefix 2022-09-20 14:46:59 +08:00
Kjell Winblad ac2922fc4c test: Kafka bridge cases for all combinations of SASL and SSL 2022-09-16 16:44:12 +02:00
JimMoen 0336b935a0
Merge pull request #8901 from JimMoen/fix-bridge-api 
- for connectors using ecpool, parse connect error `Reason` when start pool failed.
- Refine InfluxDB bridge desc.
 2022-09-16 10:22:06 +08:00
Kjell Winblad be7a8c11a8 test: make bridge name unique in tests 2022-09-15 16:21:32 +02:00
Kjell Winblad 4dc26eeba7 fix: use different instance id in Kafka auth test 2022-09-15 07:33:07 +02:00
JimMoen 54a9c8d201 chore: refine influxdb bridge description 2022-09-15 09:36:52 +08:00
Thales Macedo Garitezi deff429579 chore: unify default license references in fn 2022-09-14 17:26:48 -03:00
Thales Macedo Garitezi 741eb4ff51
docs: apply suggestions from code review 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2022-09-14 17:08:45 -03:00
Thales Macedo Garitezi 21f89fd83e chore(license): delete default config and add info to i18n 2022-09-14 17:00:21 -03:00
Kjell Winblad 5820b028cb feat: add test case for Kerberos Kafka authentication 2022-09-14 17:03:37 +02:00
Thales Macedo Garitezi 8cefdb00b8 chore(license): add default key as default value in schema 2022-09-14 09:45:01 -03:00
Thales Macedo Garitezi d4416dc581 chore: drop support for license files (ee5.0) 
Related issue: EMQX-7247

EMQX ee5.0 is to drop support for both the old license format (x509)
and for files containing the new format.
 2022-09-14 09:44:59 -03:00
Shawn f41adb0997 refactor: change some default values of resource_opts 2022-09-14 15:18:07 +08:00
Zaiming (Stone) Shi e45c99bf79 fix: kafka bridge schema 2022-09-13 20:02:48 +02:00
Kjell Winblad f0e03086a6 test: add test cases for Kafka SASL auth mechanisms plain and scram 2022-09-13 19:46:56 +02:00
Zaiming (Stone) Shi 0c1595be02 feat: Add Kafka connector 2022-09-13 19:46:56 +02:00
Zaiming (Stone) Shi befc4acced Merge remote-tracking branch 'origin/master' into merge-master-to-ee50-a 2022-09-06 20:31:38 +02:00
Xinyu Liu 23b6ff399d
Merge pull request #8844 from thalesmg/mongodb-bridge 
feat: add mongodb bridge (e5.0)
 2022-09-03 12:27:01 +08:00
Xinyu Liu 7fed4faac3
Merge pull request #8859 from terry-xiaoyu/refactor_counters_for_resources 
Some improvements and bug fixes to emqx_resources
 2022-09-03 12:16:49 +08:00
Thales Macedo Garitezi f1048babd8 test: refactor to use hocon and schema 2022-09-02 12:04:37 -03:00
Shawn b45f3de8db refactor(resource): rename metrics batched,queued -> batching,queuing 2022-09-02 12:41:14 +08:00
Thales Macedo Garitezi 3d4afd65df feat: add mongodb bridge (e5.0) 2022-09-01 14:47:14 -03:00
Zaiming (Stone) Shi 5a3fb4a011 Merge remote-tracking branch 'origin/master' into dev/ee5.0 2022-08-31 17:11:15 +02:00
Shawn ba1f5eecd3 fix: update the swagger for new resource metrics 2022-08-31 11:14:36 +08:00
Shawn e0a6a61d73 fix: return error on start hstreamdb crash 2022-08-30 10:19:40 +08:00
Zaiming (Stone) Shi 2eb078ff3b chore: bump changed app versions 2022-08-26 14:32:28 +02:00
Shawn 6b0ccfbc43 refactor: rename the error return resource_down -> recoverable_error 2022-08-26 17:11:12 +08:00
Shawn a896aa8b27 fix: incorrect replayq dir for the emqx_resource 2022-08-25 16:06:18 +08:00
Shawn 9327c0f51b fix(mysql_bridge): export the query_mode option to the APIs 2022-08-25 11:38:50 +08:00
JimMoen a481e8e2ee fix(influxdb): refine influxdb bridge example 2022-08-24 14:44:53 +08:00
JimMoen eb21a37145
Merge pull request #8785 from JimMoen/feat-influxdb-config 
Remove influxdb connector config layer. See #8773
 2022-08-24 10:56:26 +08:00
Shawn 4ac6f12252 refactor: configs and APIs for mysql bridge 2022-08-24 10:08:03 +08:00
JimMoen ca6533395c feat: influxdb bridge structure fits new style 2022-08-24 09:59:22 +08:00
Shawn aea8c77b49 refactor: new config structure for mqtt bridge 2022-08-22 18:24:59 +08:00
lafirest 4ad04b646f
Merge pull request #8759 from lafirest/feat/mysql_batch_query 
feat(bridge): add `on_batch_query` on emqx_connector_mysql
 2022-08-22 17:22:39 +08:00
firest 2471580c47 fix(bridge): make spellcheck happy 2022-08-22 15:58:11 +08:00
JimMoen 2648362c62 fix(bridge): password for bridge/db format as `password` for dashboard 2022-08-19 15:54:19 +08:00
JimMoen 06363e63d9 fix(influxdb): connector use a fallbacke `pool_size` for influxdb client 2022-08-19 15:54:19 +08:00
firest dce47aac17 fix(bridge): don't export internal fields 2022-08-19 15:31:37 +08:00
firest 8d8afd1688 feat(bridge): add `on_batch_query` on emqx_connector_mysql 2022-08-19 13:16:22 +08:00
Shawn de3a325953 fix: revert the changes in connector mysql 2022-08-16 09:06:13 +08:00
Thales Macedo Garitezi 7a31897f8e docs: add manual newlines 2022-08-15 16:55:53 -03:00
Thales Macedo Garitezi 34c1c61343
docs: apply suggestions from code review 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2022-08-15 16:37:50 -03:00
Thales Macedo Garitezi 900c6ec43e docs: add missing descriptions for license schema fields 2022-08-15 14:20:39 -03:00
Xinyu Liu 2898966439
Merge branch 'dev/ee5.0' into resource_opts 2022-08-15 21:43:22 +08:00
Shawn 19d85d485b refactor(resource): add resource_opts level into config structure 2022-08-15 21:40:10 +08:00
JimMoen d0e923590e fix: write influxdb line with undefined value 2022-08-15 14:02:17 +08:00
JimMoen 68946f1f6c feat: influxdb support `async`/`batch_async` query 2022-08-15 14:02:17 +08:00
JimMoen b01ae8ece6 chore: refine influxdb bridge/connector i18n 2022-08-15 14:00:14 +08:00
Shawn b3162fe5ff fix: conflicts between master and ee5.0 2022-08-14 22:24:09 +08:00
JimMoen 594d071c05 feat(influxdb): add async callback 2022-08-12 18:26:47 +08:00
JimMoen 441d8c9d57 chore: bump `influxdb-client-erl` vsn 2022-08-12 18:26:42 +08:00
JimMoen fa5e8f1422 chore: refine i18n label 2022-08-12 16:39:03 +08:00
lafirest 404e79b7d7
Merge pull request #8687 from lafirest/fix/bridge_mysql 
fix(bridge): replace prepare_statement by sql_template
 2022-08-12 10:12:34 +08:00
JimMoen 0c1285cdab fix: bridge creation opts refs 2022-08-12 10:07:54 +08:00
JimMoen db3e4f0b90 fix: flatten Points list 2022-08-11 20:58:56 +08:00
JimMoen 3a76a50382 fix: syntax error and compile error 2022-08-11 20:58:43 +08:00
Shawn 2872f0b668 fix(bridges): support create resources with options 2022-08-11 19:11:44 +08:00
JimMoen 0f6c371760 feat(influxdb): influxdb connector add `on_batch_query/3` callback 2022-08-11 18:12:41 +08:00
firest dc80499399 fix(bridge): add sql_template field format 2022-08-11 15:58:48 +08:00
firest 7581082fcb fix(bridge): replace prepare_statement by sql_template 2022-08-11 13:22:16 +08:00
JimMoen 223b84017e fix(influxdb): api schema `write_syntax` using raw type `string()` 2022-08-11 11:36:08 +08:00
JimMoen 0090a3ee93 fix(influxdb): fix illegal `wirte_syntax` example 2022-08-11 11:26:53 +08:00
Shawn 145ff66a9a fix: issues found by dialyzer and elvis 2022-08-10 00:45:26 +08:00
JimMoen 9d17bf31f8 fix: InfluxDB api v1 not support uint type 2022-08-09 14:28:54 +08:00