Commit Graph

596 Commits

Author SHA1 Message Date
JimMoen
583ccfaafd
test(hstreamdb): create stream and wirte data 2023-07-07 12:25:36 +08:00
JimMoen
1587f038a5
feat: hstreamdb bridge with batch query 2023-07-07 12:25:36 +08:00
zhongwencool
82ea9e5a04 test: dashbaord api ct failed 2023-06-29 10:49:39 +08:00
zhongwencool
e26c30ff0c feat: refactor dashboard https ssl_options 2023-06-29 10:49:39 +08:00
Zaiming (Stone) Shi
41de679557 ci: run example config check in ci 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi
a069b351fd test: add test script to verify config example files 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi
a757afcb7e
Merge pull request #11122 from zmstone/0622-ci-split-emqx-app-suites
ci: split apps/emqx common test suites into chunks
2023-06-22 18:17:00 +02:00
Thales Macedo Garitezi
7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
Zaiming (Stone) Shi
d6cd56558a ci: split apps/emqx common test suites into chunks 2023-06-22 10:35:00 +02:00
Zaiming (Stone) Shi
f95787d229 test: make possible to test core/replica locally 2023-06-20 12:42:00 +02:00
Thales Macedo Garitezi
b442910ff1 feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
Fixes https://emqx.atlassian.net/browse/EMQX-10281
2023-06-19 16:04:12 -03:00
zhongwencool
3331af4757 feat: make gateway and slow_subs low level and hide rule_engine 2023-06-13 11:47:38 +08:00
Thales Macedo Garitezi
54985c1d9d ci(run.sh): fix cookie generation
Sometimes, the generated cookie could be interpret by erlang as a
flag (e.g.: `+8w7uIsgUeG80c/ZgUDD9g==`) if not escaped properly,
causing the start up to fail.
2023-06-09 14:09:07 -03:00
Zaiming (Stone) Shi
7d2ea3dc76 docs: refine some desc 2023-06-08 13:58:31 +02:00
Zaiming (Stone) Shi
085ffb617f chore: add 'mcast' to spellcheck dict 2023-06-07 12:05:24 +02:00
JimMoen
bed1efd99f
build: bump dashboard tag v1.2.6-beta.1 and e1.1.0-beta.2
The dashboard asset zip file will contain version after these two tags.
2023-06-07 10:26:36 +08:00
Zaiming (Stone) Shi
0b87ea4cdc
Merge pull request #10929 from zmstone/0602-ci-pin-otp-25.3.2-1-elixir-1.14.5
ci: upgrade to otp 25.3.2-1 and elixir 1.14.5
2023-06-02 21:46:08 +02:00
Thales Macedo Garitezi
33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
2023-06-02 16:20:36 -03:00
Zaiming (Stone) Shi
9c1f77d733 ci: upgrade to otp 25.3.2-1 and elixir 1.14.5 2023-06-02 18:01:09 +02:00
Ivan Dyachkov
1761e669f0
Merge pull request #10916 from id/0601-add-ui-tests-in-ci
ci: add ui(dashboard) tests based on pytest and selenium
2023-06-02 09:07:41 +02:00
Ivan Dyachkov
8cdb1458c7 ci: add ui(dashboard) tests based on pytest and selenium 2023-06-01 20:01:24 +02:00
Thales Macedo Garitezi
99796224d8 refactor(resource): rename request_timeout -> request_ttl
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 13:01:53 -03:00
Thales Macedo Garitezi
10425eb925 feat(resource): deprecate auto_restart_interval in favor of health_check_interval
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
Thales Macedo Garitezi
cd2e21e89f ci: check if given schema.json exists before running spellcheck
Prints helpful message when schema.json does not exist, as otherwise
the "file" will be mounted as an empty directory inside the
spellchecker container.
2023-05-31 09:59:00 -03:00
Zaiming (Stone) Shi
74582ad36e test(conf-test): use major version number as prefix to find old config 2023-05-30 09:59:57 +02:00
Zaiming (Stone) Shi
65506eb15e build: remove changelog base compare
all changes in the future will have a compare base
2023-05-30 09:32:42 +02:00
Zaiming (Stone) Shi
5aaac86e28 ci: prepare for 5.1 release 2023-05-30 08:03:42 +02:00
Zaiming (Stone) Shi
d014b80834 build: download zh translation from emqx-i18n.git 2023-05-29 14:22:23 +02:00
Zaiming (Stone) Shi
c2450d230d
Merge pull request #10605 from zmstone/0503-no-telemetry-app-for-ee
0503 refactoring: no telemetry app for ee
2023-05-27 10:57:15 +02:00
Zaiming (Stone) Shi
6e2cde8224 refactor: delete emqx_statsd all together 2023-05-26 15:17:42 +02:00
Zaiming (Stone) Shi
ee122155f1 docs: fix spellcheck for telemetry app 2023-05-26 08:44:19 +02:00
Zaiming (Stone) Shi
b8b1779627 Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-23 21:42:07 +02:00
Zaiming (Stone) Shi
3bb1f7ab2b Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee 2023-05-23 13:03:55 +02:00
Andrew Mayorov
9aaa0b6fe1
Merge pull request #9927 from emqx/file-transfer
feat: implement file transfer over mqtt
2023-05-19 18:35:13 +03:00
JianBo He
73f1421d1f ci: prefixed config files wit v/e 2023-05-17 17:52:08 +08:00
JianBo He
a05156df9a chore: tests v5.0.20~25 confs 2023-05-17 17:52:08 +08:00
JianBo He
90d1a0096c chore: make spellcheck happy 2023-05-17 17:52:08 +08:00
JianBo He
07e46592a8 chore(ci): add conf tests 2023-05-17 17:52:08 +08:00
Ilya Averyanov
7fa166f034 Merge branch 'release-50' into file-transfer
* release-50: (73 commits)
  feat: add RabbitMQ bridge
  docs: improve rule engine labels and descriptions
  chore: bump version && update changes
  refactor(rocketmq): move rocketmq bridge into its own app
  test: dashboard_listener_test crash
  chore: bump chart versions
  chore: bump ee version to e5.0.4-alpha.1
  test: fix inter-suite flakiness
  build: compatibility to make 4.4+
  feat: add IotDB bridge
  ci: ensure git safe dir in build_packages
  ci: ensure git safe dir
  test: check_oom's max_mailbox_size
  feat: rename max_message_queue_len to max_mailbox_size
  fix(buffer_worker): fix inflight count when updating inflight item
  chore: prepare for v5.0.25-rc.1 release
  docs: add change log entry
  fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
  chore: `MQTT X` -> `MQTTX`
  chore: make sure brod_gssapi app is included in relese package
  ...
2023-05-09 23:30:22 +05:00
Kjell Winblad
70cf1533db feat: add RabbitMQ bridge 2023-05-09 14:32:26 +02:00
Stefan Strigler
d231e708cb feat: add IotDB bridge 2023-05-08 16:19:01 +02:00
Zaiming (Stone) Shi
7c1b453285 refactor: stop creating emqx-enterprise.conf 2023-05-08 15:16:27 +02:00
Zaiming (Stone) Shi
db1e986469
Merge pull request #10619 from emqx/0505-ci-no-need-for-source-zip
0505 ci no need for source zip
2023-05-06 10:16:07 +02:00
JimMoen
bcd3e51ac3
Merge pull request #10621 from JimMoen/fix-bin-grep-dash-escaping
fix: avoid stray `\` warning for grep 3.8+
2023-05-06 13:51:27 +08:00
JimMoen
5a25d53fba
fix: avoid stray \ warning for grep 3.8+
Only the first `-` in ERE need escaping to ensure it's not a command
option for `grep`.

The fix also tested on BSD grep 2.6 and works well.

More details:
https://lists.gnu.org/archive/html/info-gnu/2022-09/msg00001.html

> Regular expressions with stray backslashes now cause warnings, as
  their unspecified behavior can lead to unexpected results.
  For example, '\a' and 'a' are not always equivalent
  <https://bugs.gnu.org/39678>.
2023-05-06 12:42:53 +08:00
JimMoen
d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app 2023-05-06 10:22:05 +08:00
Zaiming (Stone) Shi
22815d4cec ci(buildx): use a special builder value to indicate host build 2023-05-05 22:20:34 +02:00
Ilya Averyanov
dd3471bc22 Merge branch 'master' into file-transfer
* master: (194 commits)
  fix(limiter): update change && fix deprecated version
  chore: update changes
  perf(limiter): simplify the memory represent of limiter configuration
  ci(perf test): update tf variable name and set job timeout
  ci: fix artifact name in scheduled packages workflow
  fix: build_packages_cron.yaml workflow
  ci: move scheduled builds to a separate workflow
  build: check mnesia compatibility when generating mria config
  docs: fix a typo in api doc description
  feat(./dev): use command style and added 'ctl' command
  test: fix delayed-pubish test case flakyness
  refactor: remove raw_with_default config load option
  chore: add changelog for trace timestrap
  feat: increase the time precision of trace logs to microseconds
  chore: make sure topic_metrics/rewrite's default is []
  docs: Update changes/ce/perf-10417.en.md
  chore: bump `snabbkaffe` to 1.0.8
  ci: run static checks in separate jobs
  chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
  chore: remove unused mqtt cap 'subscription_identifiers'
  ...
2023-05-05 16:50:18 +03:00
William Yang
caf8d43d24 chore(git-hook): don't fail commit hook when no mix 2023-05-05 11:02:58 +02:00
Zaiming (Stone) Shi
d5f5f35787 Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master 2023-05-02 18:08:04 +02:00
Zaiming (Stone) Shi
4d60b0da1d
Merge pull request #10459 from zmstone/0420-delete-old-code
0420 delete old code
2023-05-02 13:27:51 +02:00
Paulo Zulato
dd90b2f498 feat(oracle): Oracle Database integration 2023-04-27 13:07:20 -03:00
Zaiming (Stone) Shi
0bd30e039f test: simplify swagger json check script 2023-04-26 14:41:33 +02:00
Zaiming (Stone) Shi
48e68b7c77 test: add smoke test to cover schemas api 2023-04-26 14:41:33 +02:00
Zaiming (Stone) Shi
1c4f4037a5 test(ct/run.sh): remove the trailing / in app name 2023-04-26 13:41:42 +02:00
zhongwencool
9d893b49eb
Merge branch 'master' into sync-release-50-to-master 2023-04-26 10:54:46 +08:00
Thales Macedo Garitezi
79cf5cad19
Merge pull request #10378 from thalesmg/pulsar-producer-e50
feat: implement Pulsar Producer bridge (e5.0)
2023-04-25 18:01:43 -03:00
Thales Macedo Garitezi
56b884ab17 style: change docker compose file name 2023-04-25 14:28:00 -03:00
ieQu1
e618c66822
Merge pull request #10515 from ieQu1/gen-erlang-ls-conf
ci: Add a script to generate erlang_ls config
2023-04-25 18:56:35 +02:00
ieQu1
d6e46dcadb ci: Add a script to generate erlang_ls config 2023-04-25 15:48:22 +02:00
Thales Macedo Garitezi
ad4be08bb2 feat: implement Pulsar Producer bridge (e5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-8398
2023-04-24 10:28:26 -03:00
JimMoen
4e7472090b
fix: refine default sql and driver name for mssql bridge 2023-04-23 15:47:18 +08:00
firest
932a327952 chore: make spellcheck and xref happy 2023-04-23 11:22:56 +08:00
firest
540518eac3 chore: add README for OpenTSDB bridge 2023-04-23 11:22:48 +08:00
firest
0b46acda87 test(opents): add test cases for OpenTSDB 2023-04-23 11:03:20 +08:00
Zaiming (Stone) Shi
a6d72b178b chore: delete old script
split-i19n-files.escript is no longer needed
2023-04-22 09:07:15 +02:00
Zaiming (Stone) Shi
ceafc52ad6 refactor: use emqx_utils_ets for ets table creation 2023-04-22 09:07:15 +02:00
Ilya Averyanov
0211bcf030 Merge branch 'master' into file-transfer
* master: (279 commits)
  chore: shorten ct/run.sh script
  chore: rename cassandra_impl to cassandra_connector
  chore: fix mix.exs checking
  refactor(cassandra): move cassandra bridge into its own app
  chore: apply review suggestions
  chore: update changes/ce/fix-10449.en.md
  test: add a test for authn {}
  chore: add changlog for authn_http validation
  fix: always check authn_http's header and ssl_option
  chore: apply suggestions from code review
  fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
  fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
  perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
  ci: simplify find-apps.sh for ee apps
  perf(emqx_resource): don't reactivate alarms on reoccurring errors
  ci: check if Elixir files are formatted in pre-commit hook
  fix(dynamo): fix field name errors
  chore: remove *_collector for prometheus api's example
  chore: make plugins config to low level
  chore: re-split dynamo i18n file
  ...
2023-04-21 17:37:17 +03:00
JianBo He
34afa16236
Merge pull request #10447 from HJianBo/refactor-cassa-dir
refactor: move cassandra bridge into its own app
2023-04-21 20:44:51 +08:00
JianBo He
895963c0f6 chore: shorten ct/run.sh script 2023-04-21 20:00:26 +08:00
Zaiming (Stone) Shi
fbc744419f
Merge pull request #10453 from thalesmg/mix-fmt-pre-commit-hook
ci: check if Elixir files are formatted in pre-commit hook
2023-04-21 13:51:31 +02:00
JianBo He
bdce32e713 refactor(cassandra): move cassandra bridge into its own app 2023-04-21 14:45:38 +08:00
JianBo He
6532cfb337
Merge pull request #10452 from thalesmg/refactor-gcp-pubsub-bridge-v50
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
2023-04-21 14:42:00 +08:00
Thales Macedo Garitezi
662206fd33 ci: simplify find-apps.sh for ee apps 2023-04-20 10:53:12 -03:00
Thales Macedo Garitezi
a55017ffaa ci: check if Elixir files are formatted in pre-commit hook 2023-04-20 09:32:18 -03:00
Zaiming (Stone) Shi
adc422d063 chore: re-split dynamo i18n file 2023-04-20 11:11:35 +02:00
Thales Macedo Garitezi
a9bd91fcff refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
Fixes https://emqx.atlassian.net/browse/EMQX-9536

Note: since GCP PubSub is not shared by any authn/authz backend,
there's no need to separate its connector into another app.
2023-04-19 13:24:32 -03:00
Ivan Dyachkov
dc78ecb41c chore: merge upstream/master 2023-04-18 17:33:32 +02:00
Zaiming (Stone) Shi
18974a8e11 refactor: make schema dump and swagger spec work with split desc files 2023-04-17 13:58:11 +02:00
Ivan Dyachkov
9712aad7a8
Merge pull request #10413 from id/0416-e5.0.3-code-freeze
0416 e5.0.3 code freeze
2023-04-17 13:51:58 +02:00
Zaiming (Stone) Shi
9b7800aa8c refactor: merge into per-language i18n files 2023-04-17 13:36:10 +02:00
Zaiming (Stone) Shi
b63b880116 refactor: update i18n style check script to work with new layout 2023-04-17 13:36:10 +02:00
Zaiming (Stone) Shi
334058eeec build: add a script to split en and zh descriptions 2023-04-17 13:35:10 +02:00
Thales Macedo Garitezi
e073bc90bc refactor(buffer_worker): rename s/queue/buffer/g 2023-04-14 11:37:19 -03:00
JimMoen
35ab1c0190
style: make hocon style check and spellcheck happy
- with more human readable i18n style check format
2023-04-14 10:02:47 +08:00
JimMoen
f8c0aa49a2
style: make elvis and shellcheck happy 2023-04-14 10:02:47 +08:00
JimMoen
57505cdfb3
ci: unmount odbcinst.ini file 2023-04-14 10:02:47 +08:00
JimMoen
d7ad07f1dc
ci: install odbc driver for ms sqlserver bridge 2023-04-14 10:02:46 +08:00
JimMoen
59433b8b33
ci: install msodbcsql17 drier data bridge 2023-04-14 10:02:46 +08:00
JimMoen
c366267b0f
test: MS SQL Server data bridge 2023-04-14 10:02:46 +08:00
ieQu1
3605e4209a ci(cut): Don't generate Chinese changelog 2023-04-13 13:22:13 +02:00
Thales Macedo Garitezi
871ee90b3e refactor(kafka_bridge): move kafka bridge into its own app
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-12 13:54:45 -03:00
Thales Macedo Garitezi
914184697e
Merge pull request #10337 from thalesmg/schema-registry-v50
feat: implement schema registry for 5.0 (avro)
2023-04-11 16:46:27 -03:00
Ilya Averyanov
31b441a46e feat(s3): add S3 client application 2023-04-07 17:25:22 +03:00
Thales Macedo Garitezi
33100ecca6 feat: implement schema registry for 5.0 (avro)
Part of https://emqx.atlassian.net/browse/EMQX-9251

This ports part of the Schema Registry app from 4.x to 5.0.  Here,
only support for Avro is added.  Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
Zaiming (Stone) Shi
68c15ffd48 Merge remote-tracking branch 'origin/release-50' into 0403-sync-release-50-back-to-master 2023-04-04 16:42:58 +02:00
Zaiming (Stone) Shi
974b180da8 build: fix buildx.sh with git config 2023-04-04 12:39:33 +02:00
Zaiming (Stone) Shi
5f6d318cf0 fix(i18n): fix missing docs for gateway configs 2023-04-03 13:12:24 +02:00
Zaiming (Stone) Shi
c17de6c415 docs: fix i18n desc style 2023-04-03 13:12:24 +02:00
Zaiming (Stone) Shi
7ec9b9a408 refactor(merge-i18n.escript): merge files in rel/i18n 2023-04-03 13:12:24 +02:00