yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,781 Commits 393 Branches 684 Tags 74 MiB
Commit Graph

356 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 0b43ae621d ci: dump docker-compose log if failed to run ct 2022-12-29 09:23:11 +01:00
Thales Macedo Garitezi 35dc75b7ed feat(mqtt): add option to customize clientid prefix for egress bridges 
https://emqx.atlassian.net/browse/EMQX-8445

Currently the bridge client’s client ID is prefixed with the resource
ID.

Sometimes it’s useful for users to have control of this prefix,
e.g. prefix based ACL rules in the target broker.
 2022-12-23 09:50:26 -03:00
Zaiming (Stone) Shi c085ffa0fe refactor: default mqtt bridgge buffer pool size down to 4 2022-12-19 23:59:46 +01:00
Zaiming (Stone) Shi 9e3da5b661 chore: bump app versions 2022-12-14 20:07:41 +01:00
Zaiming (Stone) Shi 56066a03b5 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 20:04:20 +01:00
Zaiming (Stone) Shi 2d7099e3ae refactor(emqx_bridge_resource): rename a variable 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi 42c58e2a91 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 15:29:13 +01:00
Thales Macedo Garitezi b9bc82f87a feat(gcp_pubsub): add `local_topic` config 
Given the implicit convention that an egress bridge containing the
`local_topic` config will forward messages without the need for a rule
action, this was added to avoid needing a rule action.
 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi c69022f3c4 test(ci): clean config after test 
Trying to fix this error for profile `emqx`:

```
=CRASH REPORT==== 30-Nov-2022::13:25:46.763989 ===
  crasher:
    initial call: application_master:init/4
    pid: <0.9682.1>
    registered_name: []
    exception exit: {bad_return,
                        {{emqx_conf_app,start,[normal,[]]},
                         {emqx_conf_schema,
                             [#{kind => validation_error,path => "bridges",
                                reason => unknown_fields,
                                unknown => <<"influxdb_api_v1">>,
                                unmatched => <<"mqtt,webhook">>}]}}}
      in function  application_master:init/4 (application_master.erl, line 142)
    ancestors: [<0.9681.1>]
    message_queue_len: 1
    messages: [{'EXIT',<0.9683.1>,normal}]
    links: [<0.9681.1>,<0.44.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 29
    reductions: 195
  neighbours:

=INFO REPORT==== 30-Nov-2022::13:25:46.777895 ===
    application: emqx_conf
    exited: {bad_return,
                {{emqx_conf_app,start,[normal,[]]},
                 {emqx_conf_schema,
                     [#{kind => validation_error,path => "bridges",
                        reason => unknown_fields,
                        unknown => <<"influxdb_api_v1">>,
                        unmatched => <<"mqtt,webhook">>}]}}}
    type: temporary

%%% emqx_plugins_SUITE ==> init_per_suite: FAILED
%%% emqx_plugins_SUITE ==> {{failed_to_start_app,emqx_conf,
     {emqx_conf,
         {bad_return,
             {{emqx_conf_app,start,[normal,[]]},
              {emqx_conf_schema,
                  [#{kind => validation_error,path => "bridges",
                     reason => unknown_fields,
                     unknown => <<"influxdb_api_v1">>,
                     unmatched => <<"mqtt,webhook">>}]}}}}},
 [{emqx_common_test_helpers,start_app,4,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
       {line,227}]},
  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
  {emqx_plugins_SUITE,init_per_suite,1,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
       {line,34}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1380}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}

%%% emqx_plugins_SUITE ==> t_bad_info_json: SKIPPED
%%% emqx_plugins_SUITE ==> {tc_auto_skip,
    {failed,
        {emqx_plugins_SUITE,init_per_suite,
            {'EXIT',
                {{failed_to_start_app,emqx_conf,
                     {emqx_conf,
                         {bad_return,
                             {{emqx_conf_app,start,[normal,[]]},
                              {emqx_conf_schema,
                                  [#{kind => validation_error,
                                     path => "bridges",
                                     reason => unknown_fields,
                                     unknown => <<"influxdb_api_v1">>,
                                     unmatched => <<"mqtt,webhook">>}]}}}}},
                 [{emqx_common_test_helpers,start_app,4,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
                       {line,227}]},
                  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
                  {emqx_plugins_SUITE,init_per_suite,1,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
                       {line,34}]},
                  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
                  {test_server,run_test_case_eval1,6,
                      [{file,"test_server.erl"},{line,1380}]},
                  {test_server,run_test_case_eval,9,
                      [{file,"test_server.erl"},{line,1224}]}]}}}}}
```
 2022-12-12 17:18:13 -03:00
William Yang 6b6bb09d4a fix: ingress only bridge causes egress bridge traffic stop 2022-12-09 15:36:20 +01:00
Zaiming (Stone) Shi 40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
Ilya Averyanov 6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi 31098d6c67 test: fix bridge api tests, metrics is now a separate api 2022-12-06 19:03:32 +01:00
Zaiming (Stone) Shi eb017ab034 test: add test case to cover mqtt bridge config upgrade 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi 9ea22d062d refactor: make all bridges optional (required = false) 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi 9a6901987f feat: add mqtt bridge config upgrade converter 2022-11-30 21:01:59 +01:00
Zaiming (Stone) Shi 983e904858 fix(connector): fix ssl clear 2022-11-30 17:03:41 +01:00
Zaiming (Stone) Shi b398617614 chore: bump app versions 2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi 7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
firest 6dc5078a96 fix: fix obsolete SSL files aren't deleted after the bridge configuration update 2022-11-23 23:49:43 +01:00
firest 1719414c04 chore: update app versions 2022-11-23 11:42:49 +08:00
firest 19405114e2 fix: fix newly found unsafe `binary_to_atom` 2022-11-23 11:07:20 +08:00
Zaiming (Stone) Shi e5ced07665 fix: add no_return function spec to make dialyzer happy 2022-11-16 19:07:51 +01:00
Zaiming (Stone) Shi 7305923d12 fix: bridge name parser should not leak atom 2022-11-16 18:25:42 +01:00
Zaiming (Stone) Shi c940b901f5 chore: fix app versions 2022-11-16 16:26:43 +01:00
Zaiming (Stone) Shi 09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
firest 6c52d5de1f fix(bridge): Ensure that the node name is known 2022-11-01 15:36:10 +08:00
firest 73b4ac9f65 fix(bridge): keep bridge name type as binary don't convert it to atom 
After investigation, it was confirmed that there was no need to convert the bridge name to atom
 2022-11-01 11:24:24 +08:00
Zaiming (Stone) Shi c157392452 docs: fix self-closing html tag, change </br> to <br/> 2022-10-27 13:57:18 +02:00
Thales Macedo Garitezi 2d01726b22 fix: account calls when resource is not connected as matched 2022-10-13 15:32:04 -03:00
Thales Macedo Garitezi f0ff32c031 test: fix tests after counter changes 2022-10-11 17:45:48 -03:00
Shawn dfe14be8b2 chore: bump app vsns 2022-09-26 18:23:28 +08:00
Shawn 4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
Zaiming (Stone) Shi f6ac4c3a76
Merge pull request #8798 from zmstone/0815-feat-add-kafka-connector 
feat: Add Kafka connector
 2022-09-24 22:57:50 +02:00
Shawn d24441b204 fix(bridges): some issues found in code review 2022-09-23 18:32:39 +08:00
Zaiming (Stone) Shi 7b601bf970 chore: delete bad parse_bridge function clause 2022-09-23 11:53:02 +02:00
Kjell Winblad adc67b165b test: test cases for Kafka bridge REST API 2022-09-23 10:09:07 +02:00
JimMoen 6788881642 fix: hide influxdb udp config 2022-09-20 14:46:59 +08:00
Xinyu Liu cc327629c3
Merge pull request #8992 from terry-xiaoyu/fast_resume_from_inflight_full 
Fast resume from inflight full
 2022-09-18 17:49:04 +08:00
Shawn 9aa7e826cb refactor(resource): fast resume resource worker if inflight msgs are ACKed 2022-09-17 00:34:30 +08:00
JimMoen 0336b935a0
Merge pull request #8901 from JimMoen/fix-bridge-api 
- for connectors using ecpool, parse connect error `Reason` when start pool failed.
- Refine InfluxDB bridge desc.
 2022-09-16 10:22:06 +08:00
Shawn d5d3972ff5 chore: add test cases for MQTT Bridge reconnecting 2022-09-15 10:19:33 +08:00
JimMoen 0390a5e547 fix(bridge): mysql bridge error case 2022-09-15 09:36:52 +08:00
Shawn f41adb0997 refactor: change some default values of resource_opts 2022-09-14 15:18:07 +08:00
JimMoen f018b8ab49 fix(bridge/authz/exhook): no need to drop invalid certs 2022-09-14 14:08:55 +08:00
Zaiming (Stone) Shi 0c1595be02 feat: Add Kafka connector 2022-09-13 19:46:56 +02:00
Shawn b9ae4ea276 refactor: rename some metrics for emqx_resource 2022-09-13 14:04:25 +08:00
Zaiming (Stone) Shi 81ec4c4403
Merge pull request #8906 from zmstone/merge-master-to-ee50-a 
Merge master to ee50
 2022-09-07 11:19:41 +02:00
Zaiming (Stone) Shi befc4acced Merge remote-tracking branch 'origin/master' into merge-master-to-ee50-a 2022-09-06 20:31:38 +02:00
Shawn 60a90858f8 feat: check dependent actions before removing the bridges 2022-09-06 15:34:23 +08:00
Xinyu Liu 23b6ff399d
Merge pull request #8844 from thalesmg/mongodb-bridge 
feat: add mongodb bridge (e5.0)
 2022-09-03 12:27:01 +08:00
Xinyu Liu 7fed4faac3
Merge pull request #8859 from terry-xiaoyu/refactor_counters_for_resources 
Some improvements and bug fixes to emqx_resources
 2022-09-03 12:16:49 +08:00
JimMoen d533e35df0 chore: update app vsn 2022-09-02 22:26:21 +08:00
JimMoen 2693fc263d fix(bridge): show file paths for SSL certs in dashboard for bridge 2022-09-02 14:12:03 +08:00
Shawn 83f21b4c65 refactor(resource): remove metrics 'sent.exception' 2022-09-02 12:46:53 +08:00
Shawn b45f3de8db refactor(resource): rename metrics batched,queued -> batching,queuing 2022-09-02 12:41:14 +08:00
Thales Macedo Garitezi 3d4afd65df feat: add mongodb bridge (e5.0) 2022-09-01 14:47:14 -03:00
Zaiming (Stone) Shi 8a5a9ca2bf
Merge pull request #8854 from zmstone/0831-chore-merge-main-to-dev-ee50 
0831 chore merge main to dev ee50
 2022-09-01 11:35:23 +01:00
JianBo He c1afb34a86 test: fix failed tests 2022-09-01 11:34:31 +08:00
JianBo He a5ac5b6f3a
Merge branch 'dev/ee5.0' into refactor-mqtt-bridge 2022-09-01 09:30:37 +08:00
Zaiming (Stone) Shi dac178cbaf chore: ensure version bumps 2022-08-31 17:23:47 +02:00
Shawn ba1f5eecd3 fix: update the swagger for new resource metrics 2022-08-31 11:14:36 +08:00
Shawn 73e19d84ee feat: use the new metrics to bridge APIs 2022-08-30 23:47:58 +08:00
Shawn 65dfa63324 fix: update the counters for data bridges 2022-08-30 12:28:01 +08:00
JianBo He a6eff81163 chore: update emqtt to 1.7.0-rc.1 2022-08-30 09:50:36 +08:00
JianBo He 0aa10702db feat(bridge): support async mode resource options 2022-08-30 09:34:56 +08:00
JimMoen ca6533395c feat: influxdb bridge structure fits new style 2022-08-24 09:59:22 +08:00
JimMoen 43c964c87e fix(docs): ee bridge api docs generation 
Use try catch. Because function was unexported before called.
 2022-08-24 09:52:38 +08:00
Shawn 45352206a3 fix(auth): remove emqx_connector from testcases of authz/authn 2022-08-22 20:20:45 +08:00
Shawn 55c18c0b5f fix(bridges): update the test cases for new config structure 2022-08-22 18:24:59 +08:00
Shawn aea8c77b49 refactor: new config structure for mqtt bridge 2022-08-22 18:24:59 +08:00
Shawn 9e35032d78 fix: make resume_interval defaults to health_check_interval 2022-08-16 10:09:02 +08:00
Xinyu Liu 2898966439
Merge branch 'dev/ee5.0' into resource_opts 2022-08-15 21:43:22 +08:00
Shawn 19d85d485b refactor(resource): add resource_opts level into config structure 2022-08-15 21:40:10 +08:00
Shawn b3162fe5ff fix: conflicts between master and ee5.0 2022-08-14 22:24:09 +08:00
Shawn 3cb79e811f fix: remove the extra fields max_retries 2022-08-12 18:05:03 +08:00
Shawn 0cdf4b47f1 feat: add more resource creation opts 2022-08-12 13:47:45 +08:00
Xinyu Liu a05cc20db7
Merge pull request #8693 from JimMoen/feat-influxdb-batch-query 
Feat influxdb batch query
 2022-08-12 00:22:23 +08:00
JimMoen 83746daad5 fix: update bridge config badmap error 2022-08-11 23:15:53 +08:00
JimMoen 458dab53c5 fix: undefined_functions dialyzer warning 2022-08-11 22:55:22 +08:00
JimMoen ed796acb95 fix: fetch resource options after reboot 2022-08-11 22:11:28 +08:00
Shawn 88388b0c54 feat: http connector support async sending 2022-08-11 21:44:22 +08:00
JimMoen 3a76a50382 fix: syntax error and compile error 2022-08-11 20:58:43 +08:00
Shawn 2872f0b668 fix(bridges): support create resources with options 2022-08-11 19:11:44 +08:00
Shawn 6203a01320 feat: add inflight window to emqx_resource 2022-08-11 08:36:35 +08:00
Shawn 0377d3cf61 fix: update existing testcases for new emqx_resource 2022-08-10 00:34:35 +08:00
JimMoen 2e336fbc01 fix: bridge and connector namespace contains DB name 
Contains Detailed DB name for identifier hocon schema refs.
 2022-08-07 19:37:44 +08:00
DDDHuang 55b9684541 fix: influxdb bridge api available 2022-08-01 10:28:07 +08:00
DDDHuang 3fe76446bc Merge branch 'master' into dev/ee5.0 2022-08-01 10:14:21 +08:00
DDDHuang a4992ef1b5 fix: hstreamdb connector conf & api 2022-07-28 16:47:30 +08:00
Shawn 23eeb21b13 fix(webhook): the 'max_retries' opt means request retry times 2022-07-27 17:06:14 +08:00
DDDHuang 9ae7c62656 fix: exs deps & bad suites 2022-07-27 16:24:13 +08:00
DDDHuang be6de4aad0 fix: ee bridge & connector remove app mod 2022-07-27 11:38:45 +08:00
DDDHuang fa4bc921ac fix: hstream db connector & bridge, TODO: SUITE 2022-07-27 11:38:45 +08:00
DDDHuang 98b36c4681 fix: hstream db connector , TODO: start apps 2022-07-27 11:38:45 +08:00
DDDHuang f00a7417bb feat: hstream bridge & connector 2022-07-27 11:38:45 +08:00
Shawn 6685a3c5a8 fix: remove the 'headers' field from the rule events 2022-06-17 15:57:10 +08:00
Shawn d6ef2f7502 refactor: graceful recreate resources 2022-06-17 05:29:18 +08:00
Xinyu Liu 3b00b16abe
Merge pull request #8221 from terry-xiaoyu/fix_delayed_module_disbled_after_emqx_stop 
feat: fix the hook priorities
 2022-06-16 09:42:42 +08:00
Zaiming (Stone) Shi 738a551550 refactor: best-effort json for hocon validation errors 2022-06-15 15:01:15 +02:00
Shawn 39b1b20506 feat: fix the hook priorities 2022-06-15 19:03:40 +08:00
Shawn dd099137f0 fix: catch errors when load bridges 2022-06-13 22:31:58 +08:00
Shawn 9a0fdc4b11 feat: support configure connector configs directly in bridges 2022-06-10 23:06:20 +08:00
Shawn 69fba6958b refactor: remove the emqx_bridge_monitor module 2022-06-01 15:31:48 +08:00
Shawn 88ca25c60c fix(resource): fast return when starting a unavailable resource 2022-06-01 08:24:53 +08:00
JianBo He efc0ca2b62
Merge branch 'master' into changes-enable_pipelining-type 2022-05-31 20:49:10 +08:00
JianBo He f0ddac63cb chore: change the enable_pipelining default value 2022-05-31 19:32:53 +08:00
Zhongwen Deng d61b44ef97 fix: ct failed again 2022-05-31 19:20:27 +08:00
Zhongwen Deng 8aa60cc0a5 feat: generate a minimized emqx.conf 2022-05-31 19:20:27 +08:00
Shawn 1054c364ad refactor(resource): improve health check and alarm it if resource down 2022-05-31 01:40:40 +08:00
Shawn a157539710 refactor(rule): rename http_bridge -> webhook 2022-05-25 18:56:12 +08:00
Shawn 71de9616d3 refactor(rule): rename outputs -> actions 2022-05-25 17:02:37 +08:00
Shawn d89d692d3e refactor: move some APIs to new module emqx_bridge_resource 2022-05-18 15:12:34 +08:00
DDDHuang 8b0adf69d2
Merge pull request #7926 from DDDHuang/rule_api_errormsg 
fix: better error message for rule engine
 2022-05-12 19:07:47 +08:00
DDDHuang 2de69c97ba fix: ruleengine & connector & bridge api, better error message 2022-05-12 16:51:36 +08:00
Thales Macedo Garitezi 47ed9cbc76
fix(telemetry): count bridges referenced in `from` rule clauses 2022-05-11 15:15:17 -03:00
Chris Hicks 841acb7828
Merge pull request #7783 from emqx/EMQX-4199-introduce-a-new-emqx-resource-manager-module 
feat: isolate resource manager processes
 2022-05-09 18:33:01 +02:00
Chris 0b3e30e813 feat: isolate resource manager processes 2022-05-09 13:24:34 +02:00
Shawn 9a033af2e8 fix: restart bridges via HTTP APIs to wrong node 2022-05-05 18:47:19 +08:00
Shawn 1ce8a8c886 fix: testing connector removes ssl cert files that are in use 2022-05-05 14:55:40 +08:00
zhongwencool c7241f2abc
Merge pull request #7813 from DDDHuang/bridge_api_bad_format 
fix: bridge api response bad node name format
 2022-04-29 09:12:22 +08:00
Zhongwen Deng 03d2a1ebb6 chore: reformat emqx_bridge_api 2022-04-29 09:08:45 +08:00
Shawn 6edb453ce9 fix: the field 'enable' is missing for GET /bridges/:id 2022-04-28 18:13:19 +08:00
Shawn e18c2a6de2 fix: convert ssl certfiles for dry-run creating bridges 2022-04-28 16:17:55 +08:00
DDDHuang a26b0f79f2 fix: bridge api response bad node name format 2022-04-28 16:06:57 +08:00
Zaiming (Stone) Shi 02c3f87b31 style: reformat all remaining apps 2022-04-27 15:51:18 +02:00
Shawn 94e24c2621 refactor: move ssl file handling from resources to bridges 2022-04-27 11:59:15 +08:00
Thales Macedo Garitezi f8f97d39d3
Merge pull request #7766 from thalesmg/bugfix-telemetry-bridge 
fix: avoid crashing telemetry if bridge app is not ready
 2022-04-26 09:03:29 -03:00
Thales Macedo Garitezi ada3b03f6e
fix(telemetry): do not crash if bridge app is not ready yet 2022-04-25 11:23:13 -03:00
EMQ-YangM eb3870b538 fix: when enable is false, restart and stop cannot be performed. 2022-04-25 18:48:12 +08:00
JimMoen e9843aa225 fix(doc): html line break use `</br>` 2022-04-24 18:56:21 +08:00
Zhongwen Deng 5223c3ee61 feat: obfuscate sensitive values default_password 2022-04-24 09:41:29 +08:00
Yang Miao 4a43a10296
Merge pull request #7701 from EMQ-YangM/improve_docs 
fix: improve document quality
 2022-04-22 16:52:09 +08:00
EMQ-YangM f00854aa9d fix(emqx_bridge): add required fields 2022-04-22 14:22:22 +08:00
EMQ-YangM 739fe9b71e feat: emqx_bridge i18n support 2022-04-22 10:53:06 +08:00
EMQ-YangM 5d9df74be7 test: fix required fields 2022-04-21 18:47:02 +08:00
Shawn 19630e9a99 feat: save ssl cert files for data bridges 2022-04-21 09:00:06 +08:00
Shawn 5f74db6d88 fix: treat all returns except {error, Reason} as ok 2022-04-18 22:24:39 +08:00
EMQ-YangM a7ee4f9fea fix: fix bridge resource Id 2022-04-12 14:10:35 +08:00
EMQ-YangM 5aeb3c2dc2 feat: impl bridge reset_metrics api 2022-04-11 10:25:48 +08:00
Thales Macedo Garitezi 2a7b777a2a
fix(test): fix inter-suite test coupling 2022-04-07 12:21:15 -03:00
Thales Macedo Garitezi 911e02f626
feat(bridge): export basic usage info for telemetry 2022-04-06 10:20:23 -03:00
ieQu1 99b3f2f83e docs(schema): Document missing records and fields 2022-04-04 23:58:43 +02:00
Ilya Averyanov 0728b1e3f4 chore(emqx_modules): add emqx_topic_metrics_api tests 2022-03-30 21:18:27 +03:00
Zaiming (Stone) Shi 5458490bc3 chore: add namespace to bridge schema 2022-03-23 21:12:55 +01:00
DDDHuang 74e6fa5a94 fix: generate bridge api response code 
fix: generate connector api response code SUITE

fix: generate exhook api response code
 2022-03-21 14:12:19 +08:00
Shawn 1d023b541f refactor(connnector): rename waiting_connect_complete -> wait_for_resource_ready 
Rename the option to wait_for_resource_ready and defaults to 5s.
 2022-03-10 10:46:57 +08:00
Shawn 50d0a3271b fix(test): add testcases for enable/disbale bridges 2022-03-09 18:49:30 +08:00
Shawn 36e068d00d feat(bridge): add APIs for restart/stop bridges on one node 2022-03-09 18:42:19 +08:00
EMQ-YangM 583624fb8d fix(emqx_authn): fix test suite 2022-03-09 13:53:05 +08:00