Commit Graph

506 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 447b76464b Merge branch 'release-50' into merge-r50-into-v50-a 2023-05-17 14:50:18 -03:00
Zhongwen Deng e26ce5816e feat: hide keepalive_backoff, introduce keepalive_multiplier 2023-05-17 21:25:10 +08:00
Kjell Winblad 397dce0d78
Merge pull request #10654 from kjellwinblad/kjell/refactor/clickhouse_bridge/EMQX-9538
refactor: the clickhouse bridge to use the new bridge app structure
2023-05-17 11:19:33 +02:00
Kjell Winblad bce43978a2 refactor: the clickhouse bridge to use the new bridge app structure
Fixes:
https://emqx.atlassian.net/browse/EMQX-9538
2023-05-17 10:12:43 +02:00
Kjell Winblad ab8b8ccaad docs: make sure there is a summary for all API endpoints
Many HTTP API endpoints did not have a summary when looking at the API
docs at "http://emqx_host_name:18083/api-docs". This has been fixed by
making sure there is a summary corresponding to all endpoints.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9767
2023-05-17 09:58:56 +02:00
Ilya Averyanov 1a8cf0e392 Merge branch 'release-50' into file-transfer
* release-50:
  fix(pulsar): use a binary duration as default `health_check_interval`
  docs: add changelog entry
  docs: clarify description of bridge username and password
  chore: bump to v5.0.25
  fix(limiter): adjust type for compatibility
  fix(limiter): fix that update node-level limiter config will not working
  chore: upgrade dashboard to v1.2.4-1 for ce
  chore: upgarde rulesql to 0.1.6 to fix invaid utf8 input
  chore: add changelog for 10659
  fix: crash when sysmon.os.mem_check_interval = disabled
  chore: bump influxdb version && update changes
  refactor(influxdb): move influxdb bridge into its own app
  chore: add listener default changelog
  fix: ocsp cache SUITE failed
  fix: ensure atom key for emqx_config:get
  fix: only fill cerf_file default in server side
  fix: authn init is empty
  fix: bad listeners default ssl_options
2023-05-17 00:27:55 +03:00
Zaiming (Stone) Shi 9b9731b943
Merge pull request #10473 from zmstone/0421-refactor-delete-unused-schema-field
0421 refactor delete unused schema field
2023-05-16 22:22:03 +02:00
Andrew Mayorov 91ce1c2a90
fix(ft-schema): make description more natural
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-05-16 16:27:33 +03:00
Kjell Winblad 46f05056de docs: clarify description of bridge username and password
Fixes:
https://emqx.atlassian.net/browse/EMQX-9613
2023-05-15 16:29:52 +02:00
zhongwencool a953b951fe
Merge branch 'master' into sync-release-50-to-master 2023-05-12 18:01:58 +08:00
firest 6497bcae46 refactor(influxdb): move influxdb bridge into its own app 2023-05-12 11:48:49 +08:00
Ilya Averyanov 5b5d7ceac5 Merge branch 'release-50' into file-transfer
* release-50:
  fix(limiter): fix an error when setting `max_conn_rate` in a listener
  chore: bump erlcloud dependencies vsns
  chore: rename dynamo template files
  refactor(dynamo): move dynamo bridge into its own app
  chore: update changes && bump app versions
  fix: issues with the RabbitMQ config
  refactor(pgsql): move pgsql && matrix && timescale bridges into their own app
  fix: the iotdb password field so it has the password format
  chore: update changes
  refactor(tdengine): move tdengine bridge into its own app
  feat: deprecate listeners's authn http api
2023-05-11 16:15:38 +05:00
Zaiming (Stone) Shi 7566c25199 refactor: delete unused connector field 'base_url' 2023-05-11 09:04:55 +02:00
Zaiming (Stone) Shi b3e35dac1c
Merge pull request #10584 from paulozulato/debug-ssl-handshake
feat: add log level configuration to ssl communication
2023-05-11 08:59:13 +02:00
JimMoen fbd516d740
Merge pull request #10622 from JimMoen/refactor-dynamo-bridge
refactor(dynamo): move dynamo bridge into its own app
2023-05-11 11:00:50 +08:00
JimMoen 8914e006c2
refactor(dynamo): move dynamo bridge into its own app 2023-05-10 18:49:21 +08:00
firest 066ed5c6ec refactor(pgsql): move pgsql && matrix && timescale bridges into their own app 2023-05-10 16:51:33 +08:00
lafirest 594ceaf9e6
Merge pull request #10650 from lafirest/refactor/tdengine_dir
refactor(tdengine): move tdengine bridge into its own app
2023-05-10 16:42:35 +08:00
Ilya Averyanov 8d9b785bd7 Merge branch 'release-50' into file-transfer
* release-50:
  chore(rebalance): fix app metadata
  chore(rebalance): move apps from lib-ee, add READMEs
  docs: refine zh tr
  docs: delete APL header from ee file
  docs: delete zh changelog
  chore(rebalance): review fixes
  chore(rebalance): rebase and review fixes
  feat(rebalance): port apps from 4.x
2023-05-10 11:55:23 +05:00
firest 0f979b3d24 refactor(tdengine): move tdengine bridge into its own app 2023-05-10 14:12:57 +08:00
Andrew Mayorov 5e5f854ce1 feat(ft-conf): simplify schema of storage / exporter backends
Assumption is this changes will make `emqx_ft` config schema
user-friendlier and also more future-proof.
2023-05-09 23:32:16 +05:00
Ilya Averyanov 7fa166f034 Merge branch 'release-50' into file-transfer
* release-50: (73 commits)
  feat: add RabbitMQ bridge
  docs: improve rule engine labels and descriptions
  chore: bump version && update changes
  refactor(rocketmq): move rocketmq bridge into its own app
  test: dashboard_listener_test crash
  chore: bump chart versions
  chore: bump ee version to e5.0.4-alpha.1
  test: fix inter-suite flakiness
  build: compatibility to make 4.4+
  feat: add IotDB bridge
  ci: ensure git safe dir in build_packages
  ci: ensure git safe dir
  test: check_oom's max_mailbox_size
  feat: rename max_message_queue_len to max_mailbox_size
  fix(buffer_worker): fix inflight count when updating inflight item
  chore: prepare for v5.0.25-rc.1 release
  docs: add change log entry
  fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
  chore: `MQTT X` -> `MQTTX`
  chore: make sure brod_gssapi app is included in relese package
  ...
2023-05-09 23:30:22 +05:00
Zaiming (Stone) Shi 3ff04d51bc docs: refine zh tr 2023-05-09 20:51:24 +05:00
Ilya Averyanov ffed8a132b chore(rebalance): review fixes
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-05-09 20:51:24 +05:00
Ilya Averyanov e683d28973 chore(rebalance): rebase and review fixes 2023-05-09 20:51:24 +05:00
Kjell Winblad 70cf1533db feat: add RabbitMQ bridge 2023-05-09 14:32:26 +02:00
Kjell Winblad 4d1499b747
Merge pull request #10582 from kjellwinblad/kjell/fix/max_overflow_desc/EMQX-9714
docs: better max_overflow field label and description for MongoDB bridge
2023-05-09 12:26:26 +02:00
Kjell Winblad b0c647dd44
Merge pull request #10649 from kjellwinblad/kjell/fix/unknown_desc/EMQX-9645
docs: improve rule engine labels and descriptions
2023-05-09 12:25:01 +02:00
Kjell Winblad 9af8375773 docs: improve rule engine labels and descriptions
Fixes:
https://emqx.atlassian.net/browse/EMQX-9645
2023-05-09 11:44:30 +02:00
firest a119100a0e refactor(rocketmq): move rocketmq bridge into its own app 2023-05-09 15:57:56 +08:00
lafirest b94290db58
Merge pull request #10625 from lafirest/refactor/limiter_cfg
refactor(limiter): simplify limiter configuration
2023-05-09 11:40:05 +08:00
Stefan Strigler 1ade7ce9a3
Merge pull request #10560 from sstrigler/EMQX-8390-5-0-x-io-tdb-bridge-port-conf-and-api-to-5-0
IoTDB Bridge
2023-05-08 20:27:01 +02:00
Paulo Zulato 77176787ca feat: add log level configuration to ssl communication
Fixes https://emqx.atlassian.net/browse/EMQX-9781
2023-05-08 15:11:49 -03:00
Stefan Strigler d231e708cb feat: add IotDB bridge 2023-05-08 16:19:01 +02:00
Zhongwen Deng eaa129d0d7 feat: rename max_message_queue_len to max_mailbox_size 2023-05-08 20:27:52 +08:00
firest d914d1ee1d refactor(limiter): simplify limiter configuration 2023-05-08 11:45:49 +08:00
JimMoen d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app 2023-05-06 10:22:05 +08:00
Ilya Averyanov dd3471bc22 Merge branch 'master' into file-transfer
* master: (194 commits)
  fix(limiter): update change && fix deprecated version
  chore: update changes
  perf(limiter): simplify the memory represent of limiter configuration
  ci(perf test): update tf variable name and set job timeout
  ci: fix artifact name in scheduled packages workflow
  fix: build_packages_cron.yaml workflow
  ci: move scheduled builds to a separate workflow
  build: check mnesia compatibility when generating mria config
  docs: fix a typo in api doc description
  feat(./dev): use command style and added 'ctl' command
  test: fix delayed-pubish test case flakyness
  refactor: remove raw_with_default config load option
  chore: add changelog for trace timestrap
  feat: increase the time precision of trace logs to microseconds
  chore: make sure topic_metrics/rewrite's default is []
  docs: Update changes/ce/perf-10417.en.md
  chore: bump `snabbkaffe` to 1.0.8
  ci: run static checks in separate jobs
  chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
  chore: remove unused mqtt cap 'subscription_identifiers'
  ...
2023-05-05 16:50:18 +03:00
Ilya Averyanov cff7788b2e chore(ft): add s3 exporter tests 2023-05-05 11:55:09 +03:00
Zaiming (Stone) Shi 232c5926f2 docs: fix a typo in api doc description 2023-05-04 10:50:14 +02:00
Zaiming (Stone) Shi 856129984b refactor: remove raw_with_default config load option
This option was previously only in tests to avoid
emqx_conf app overwriting previously set configs with default values.

After a03f2dd64b, the issue for
test cases had been resolved.
This commit is to get rid of the option all together
2023-05-04 09:48:28 +02:00
Kjell Winblad b7c7d400e2 docs: better max_overflow field label and description for MongoDB bridge
The new label and description for the max_overflow field clarifies the
purpose of the field, which is to manage the maximum number of
additional workers created when all workers in the pool are busy,
allowing more concurrent connections to the MongoDB server during
workload spikes.

Fixes part of:
https://emqx.atlassian.net/browse/EMQX-9714
2023-05-03 17:49:08 +02:00
Zaiming (Stone) Shi d5f5f35787 Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master 2023-05-02 18:08:04 +02:00
Zaiming (Stone) Shi 4d705817d8 refactor(log): move default values to schema 2023-04-29 22:10:19 +02:00
Andrew Mayorov ed3756ea09
feat(ft-api): add paging support through cursors 2023-04-28 13:49:15 +03:00
Andrew Mayorov 811e449357
feat(ft-conf): provide global killswitch 2023-04-28 10:54:21 +03:00
Paulo Zulato dd90b2f498 feat(oracle): Oracle Database integration 2023-04-27 13:07:20 -03:00
Zaiming (Stone) Shi ed7a8659d2 feat: add a json format support for the /status API 2023-04-26 14:41:33 +02:00
Zhongwen Deng f0cd5c98c7 chore: split i18n with script 2023-04-26 13:57:15 +08:00
zhongwencool 9d893b49eb
Merge branch 'master' into sync-release-50-to-master 2023-04-26 10:54:46 +08:00
firest 3bb50a5751 fix(rocketmq): fix that the update of ACL info not working 2023-04-25 16:55:03 +08:00
Thales Macedo Garitezi cb149ac345 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi 631863d843 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi ad4be08bb2 feat: implement Pulsar Producer bridge (e5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-8398
2023-04-24 10:28:26 -03:00
Zaiming (Stone) Shi 366aa53c80
Merge pull request #10462 from zmstone/0430-EMQX-8434-shared-dispatch-ack-deprecated
chore: Hide config shared_dispatch_ack_enabled
2023-04-24 09:42:09 +02:00
lafirest ac54fa0ba5
Merge pull request #10482 from lafirest/chore/rocketmq_template
chore: add examples for RocketMQ template
2023-04-23 18:10:35 +08:00
firest 5ab0887687 chore: add examples for RocketMQ template 2023-04-23 15:17:51 +08:00
firest 5ad5d7ee8d fix(opents): adjust code structure 2023-04-23 14:28:14 +08:00
firest 932a327952 chore: make spellcheck and xref happy 2023-04-23 11:22:56 +08:00
firest 0b46acda87 test(opents): add test cases for OpenTSDB 2023-04-23 11:03:20 +08:00
firest 5074825075 feat(opents): OpenTSDB integration 2023-04-23 09:56:24 +08:00
Ilya Averyanov b951de4c6e feat(ft): convert hocon files to the new format 2023-04-21 19:14:24 +03:00
Ilya Averyanov 0211bcf030 Merge branch 'master' into file-transfer
* master: (279 commits)
  chore: shorten ct/run.sh script
  chore: rename cassandra_impl to cassandra_connector
  chore: fix mix.exs checking
  refactor(cassandra): move cassandra bridge into its own app
  chore: apply review suggestions
  chore: update changes/ce/fix-10449.en.md
  test: add a test for authn {}
  chore: add changlog for authn_http validation
  fix: always check authn_http's header and ssl_option
  chore: apply suggestions from code review
  fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
  fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
  perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
  ci: simplify find-apps.sh for ee apps
  perf(emqx_resource): don't reactivate alarms on reoccurring errors
  ci: check if Elixir files are formatted in pre-commit hook
  fix(dynamo): fix field name errors
  chore: remove *_collector for prometheus api's example
  chore: make plugins config to low level
  chore: re-split dynamo i18n file
  ...
2023-04-21 17:37:17 +03:00
Zaiming (Stone) Shi 701a1f65f9 chore: Hide config shared_dispatch_ack_enabled 2023-04-21 13:59:55 +02:00
JianBo He b270623c46 chore: rename cassandra_impl to cassandra_connector 2023-04-21 16:49:25 +08:00
firest 7704995279 fix(rocketmq): expose the driver parameter `sync_timeout` into the RocketMQ bridge configuration 2023-04-21 15:10:25 +08:00
JianBo He 5cc28a7b45 chore: fix mix.exs checking 2023-04-21 14:46:09 +08:00
JianBo He bdce32e713 refactor(cassandra): move cassandra bridge into its own app 2023-04-21 14:45:38 +08:00
JianBo He 6532cfb337
Merge pull request #10452 from thalesmg/refactor-gcp-pubsub-bridge-v50
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
2023-04-21 14:42:00 +08:00
Zaiming (Stone) Shi adc422d063 chore: re-split dynamo i18n file 2023-04-20 11:11:35 +02:00
Zaiming (Stone) Shi 415830a0a2 Merge remote-tracking branch 'origin/release-50' into 0420-sync-release-50-to-masteer 2023-04-20 11:03:39 +02:00
Thales Macedo Garitezi a9bd91fcff refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
Fixes https://emqx.atlassian.net/browse/EMQX-9536

Note: since GCP PubSub is not shared by any authn/authz backend,
there's no need to separate its connector into another app.
2023-04-19 13:24:32 -03:00
firest e89f4d4565 fix(dynamo): fix terminology erros
- Changed `username` to `aws_access_key_id`
- Changed `password` to `aws_secret_access_key`
2023-04-19 16:26:42 +08:00
firest bc353b0a06 fix(dynamo): change `database` to `table` in the schema of the DynamoDB bridge
there is no term like `database` in DynamoDB, the correct concept should be `table`
2023-04-19 15:56:29 +08:00
Zaiming (Stone) Shi 48d8381a6d docs: update doc refs 2023-04-18 09:48:28 +02:00
Thales Macedo Garitezi 9d15247dd5 feat(schema_registry): add support for protobuf schemas
Fixes https://emqx.atlassian.net/browse/EMQX-9470
2023-04-17 13:04:54 -03:00
Zaiming (Stone) Shi b9a43ead39 refactor: Split i18n files 2023-04-17 13:35:33 +02:00
firest 02f8d073f8 test(limiter): fix test errors and make spellcheck happy 2023-04-17 10:06:43 +08:00
firest 55376144ce fix(limiter): simplify the configuration of the limiter 2023-04-17 10:06:36 +08:00
Thales Macedo Garitezi 3373a63137
docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-14 13:21:30 -03:00
Thales Macedo Garitezi e073bc90bc refactor(buffer_worker): rename `s/queue/buffer/g` 2023-04-14 11:37:19 -03:00
Thales Macedo Garitezi dd38122797 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-14 11:37:19 -03:00
Thales Macedo Garitezi 14ed4a7ada feat(buffer_worker): set default queue mode to `memory_only`
Fixes https://emqx.atlassian.net/browse/EMQX-9367

For better user experience and performance for the average bridge, we
should change the default queue mode to `memory_only`, as was the
behavior of most bridges in e4.x.  This leads to better performance
when message rate is high enough and the remote resource is not
keeping up with EMQX.

Also, we set the default segment size to equal max queue bytes.
2023-04-14 11:37:19 -03:00
JimMoen 06b1ec862e
fix: hocon label field 2023-04-14 12:43:29 +08:00
JimMoen 35ab1c0190
style: make hocon style check and spellcheck happy
- with more human readable i18n style check format
2023-04-14 10:02:47 +08:00
JimMoen 5841969877
feat: implement Microsoft SQL Server bridge (e5.0) 2023-04-14 10:02:45 +08:00
Ilya Averyanov 92ff2b7d6e feat(ft): make timeouts configurable 2023-04-14 01:58:14 +03:00
Thales Macedo Garitezi 9acfe00498
Merge pull request #10347 from thalesmg/refactor-kafka-bridge-dirs-v50
refactor(kafka_bridge): move kafka bridge into its own app
2023-04-13 13:26:36 -03:00
JianBo He 993be31998
chore: update rel/i18n/emqx_conf_schema.hocon
Co-authored-by: LenaLenaPan <120552185+LenaLenaPan@users.noreply.github.com>
2023-04-13 18:12:17 +08:00
JianBo He b57b964083 chore(i18n): improve config desc 2023-04-13 11:15:36 +08:00
Thales Macedo Garitezi 871ee90b3e refactor(kafka_bridge): move kafka bridge into its own app
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-12 13:54:45 -03:00
Ivan Dyachkov bdffa925db chore: merge upstream/master release-50 2023-04-12 15:30:20 +02:00
Stefan Strigler f668ad7b9e
Merge pull request #10336 from sstrigler/EMQX-8507-rule-engine-need-a-new-api-to-crud-rule-engines-setting-configs
feat: add `/rule_engine` API endpoint
2023-04-12 11:44:42 +02:00
Thales Macedo Garitezi 914184697e
Merge pull request #10337 from thalesmg/schema-registry-v50
feat: implement schema registry for 5.0 (avro)
2023-04-11 16:46:27 -03:00
Stefan Strigler e282d361f8
style: fix Chinese translations
Co-authored-by: LenaLenaPan <120552185+LenaLenaPan@users.noreply.github.com>
2023-04-11 09:34:51 +02:00
Andrew Mayorov 37a520d797
fix(i18n): relocate i18n files to rel/i18n 2023-04-10 16:02:34 +03:00
Thales Macedo Garitezi 33100ecca6 feat: implement schema registry for 5.0 (avro)
Part of https://emqx.atlassian.net/browse/EMQX-9251

This ports part of the Schema Registry app from 4.x to 5.0.  Here,
only support for Avro is added.  Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
JianBo He 702ecbcb6e Merge remote-tracking branch 'upstream/master' into refactor-gw-dir 2023-04-06 10:44:59 +08:00
Stefan Strigler 576d1524bb style: fix wording of i18n files 2023-04-05 16:29:17 +02:00
Stefan Strigler aea870f319 feat: add `/rule_engine` API endpoint 2023-04-05 15:19:31 +02:00
Zaiming (Stone) Shi a9bf633e03
Merge pull request #10320 from zmstone/0403-sync-release-50-back-to-master
0403 sync release 50 back to master
2023-04-04 23:31:24 +02:00
Zaiming (Stone) Shi 429b3d9efd Merge remote-tracking branch 'origin/master' into 0403-sync-release-50-back-to-master 2023-04-03 20:32:33 +02:00
Thales Macedo Garitezi f3ffc02bff feat(bridges): enable async query mode for all bridges with buffer workers
Fixes https://emqx.atlassian.net/browse/EMQX-9130

Since buffer workers always support async calls ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.

For bridges that currently only allow sync query modes, we should
allow them to be configured with async.  That means basically all
bridge types except Kafka Producer.
2023-04-03 14:49:51 -03:00
Zaiming (Stone) Shi 5f6d318cf0 fix(i18n): fix missing docs for gateway configs 2023-04-03 13:12:24 +02:00
Zaiming (Stone) Shi c17de6c415 docs: fix i18n desc style 2023-04-03 13:12:24 +02:00
Zaiming (Stone) Shi 36000abf51 refactor: relocate i18n files for apps/emqx 2023-04-03 13:12:24 +02:00