Commit Graph

70 Commits

Author SHA1 Message Date
Ilya Averyanov 7fa166f034 Merge branch 'release-50' into file-transfer
* release-50: (73 commits)
  feat: add RabbitMQ bridge
  docs: improve rule engine labels and descriptions
  chore: bump version && update changes
  refactor(rocketmq): move rocketmq bridge into its own app
  test: dashboard_listener_test crash
  chore: bump chart versions
  chore: bump ee version to e5.0.4-alpha.1
  test: fix inter-suite flakiness
  build: compatibility to make 4.4+
  feat: add IotDB bridge
  ci: ensure git safe dir in build_packages
  ci: ensure git safe dir
  test: check_oom's max_mailbox_size
  feat: rename max_message_queue_len to max_mailbox_size
  fix(buffer_worker): fix inflight count when updating inflight item
  chore: prepare for v5.0.25-rc.1 release
  docs: add change log entry
  fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
  chore: `MQTT X` -> `MQTTX`
  chore: make sure brod_gssapi app is included in relese package
  ...
2023-05-09 23:30:22 +05:00
Kjell Winblad 70cf1533db feat: add RabbitMQ bridge 2023-05-09 14:32:26 +02:00
Stefan Strigler d231e708cb feat: add IotDB bridge 2023-05-08 16:19:01 +02:00
JimMoen d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app 2023-05-06 10:22:05 +08:00
Ilya Averyanov dd3471bc22 Merge branch 'master' into file-transfer
* master: (194 commits)
  fix(limiter): update change && fix deprecated version
  chore: update changes
  perf(limiter): simplify the memory represent of limiter configuration
  ci(perf test): update tf variable name and set job timeout
  ci: fix artifact name in scheduled packages workflow
  fix: build_packages_cron.yaml workflow
  ci: move scheduled builds to a separate workflow
  build: check mnesia compatibility when generating mria config
  docs: fix a typo in api doc description
  feat(./dev): use command style and added 'ctl' command
  test: fix delayed-pubish test case flakyness
  refactor: remove raw_with_default config load option
  chore: add changelog for trace timestrap
  feat: increase the time precision of trace logs to microseconds
  chore: make sure topic_metrics/rewrite's default is []
  docs: Update changes/ce/perf-10417.en.md
  chore: bump `snabbkaffe` to 1.0.8
  ci: run static checks in separate jobs
  chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
  chore: remove unused mqtt cap 'subscription_identifiers'
  ...
2023-05-05 16:50:18 +03:00
Zaiming (Stone) Shi d5f5f35787 Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master 2023-05-02 18:08:04 +02:00
Paulo Zulato dd90b2f498 feat(oracle): Oracle Database integration 2023-04-27 13:07:20 -03:00
Zaiming (Stone) Shi 1c4f4037a5 test(ct/run.sh): remove the trailing / in app name 2023-04-26 13:41:42 +02:00
zhongwencool 9d893b49eb
Merge branch 'master' into sync-release-50-to-master 2023-04-26 10:54:46 +08:00
Thales Macedo Garitezi 56b884ab17 style: change docker compose file name 2023-04-25 14:28:00 -03:00
Thales Macedo Garitezi ad4be08bb2 feat: implement Pulsar Producer bridge (e5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-8398
2023-04-24 10:28:26 -03:00
JimMoen 4e7472090b
fix: refine default sql and driver name for mssql bridge 2023-04-23 15:47:18 +08:00
firest 540518eac3 chore: add README for OpenTSDB bridge 2023-04-23 11:22:48 +08:00
firest 0b46acda87 test(opents): add test cases for OpenTSDB 2023-04-23 11:03:20 +08:00
Ilya Averyanov 0211bcf030 Merge branch 'master' into file-transfer
* master: (279 commits)
  chore: shorten ct/run.sh script
  chore: rename cassandra_impl to cassandra_connector
  chore: fix mix.exs checking
  refactor(cassandra): move cassandra bridge into its own app
  chore: apply review suggestions
  chore: update changes/ce/fix-10449.en.md
  test: add a test for authn {}
  chore: add changlog for authn_http validation
  fix: always check authn_http's header and ssl_option
  chore: apply suggestions from code review
  fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
  fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
  perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
  ci: simplify find-apps.sh for ee apps
  perf(emqx_resource): don't reactivate alarms on reoccurring errors
  ci: check if Elixir files are formatted in pre-commit hook
  fix(dynamo): fix field name errors
  chore: remove *_collector for prometheus api's example
  chore: make plugins config to low level
  chore: re-split dynamo i18n file
  ...
2023-04-21 17:37:17 +03:00
JianBo He 895963c0f6 chore: shorten ct/run.sh script 2023-04-21 20:00:26 +08:00
JianBo He bdce32e713 refactor(cassandra): move cassandra bridge into its own app 2023-04-21 14:45:38 +08:00
JimMoen d7ad07f1dc
ci: install odbc driver for ms sqlserver bridge 2023-04-14 10:02:46 +08:00
JimMoen c366267b0f
test: MS SQL Server data bridge 2023-04-14 10:02:46 +08:00
Thales Macedo Garitezi 871ee90b3e refactor(kafka_bridge): move kafka bridge into its own app
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-12 13:54:45 -03:00
Ilya Averyanov 31b441a46e feat(s3): add S3 client application 2023-04-07 17:25:22 +03:00
JianBo He 8cbbc9f271 Merge remote-tracking branch 'upstream/master' into cassa 2023-03-23 11:53:17 +08:00
firest 4ad3579966 test(bridges): add test suite for RocketMQ 2023-03-22 10:36:58 +08:00
JianBo He 539ec2f774 chore(bridge): cover username/password auth for cassandra bridges 2023-03-21 13:55:53 +08:00
JianBo He 12942b676d Merge remote-tracking branch 'upstream/master' into cassa 2023-03-20 09:50:27 +08:00
JianBo He c0a216a740 feat(bridge): support cassandra bridge 2023-03-17 11:34:48 +08:00
Thales Macedo Garitezi 8b1fa50413 ci(fix): fix shellsheck warning 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi e4f058ce2a ci(fix): create user inside container so `emqx console` works 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 9d6af17f69 ci(fix): create and give permissions to mix directories 2023-03-16 13:43:01 -03:00
Ivan Dyachkov a59827cc6a chore(run.sh): prefer docker compose plugin over docker-compose 2023-03-16 13:43:01 -03:00
Ivan Dyachkov 2fe341d152 chore(run.sh): fix permissions on secrets directory 2023-03-16 13:43:01 -03:00
Ivan Dyachkov 5eaaa83b82 chore: simplify run.sh
- get rid of sudo
- do not change permissions of existing files
- use our own docker image to generate certs to make it working on arm
- bump kafka docker image version to have access to multiplatofrm one
2023-03-16 13:43:01 -03:00
firest 984dd3446d test(bridges): add test suite for DynamoDB 2023-03-08 11:13:51 +08:00
Kjell Winblad 96318bda1d build: run run.sh tests without being asked for sudo password
This commit makes it possible to run the scripts/ct/run.sh script
without being asked for sudo password if one instead wants to use the
run docker container with root user trick to reset owner of files.

One simply has to set the environment variable
EMQX_TEST_DO_NOT_RUN_SUDO.
2023-03-02 12:22:11 +01:00
Kjell Winblad 67acdf0888 feat: add clickhouse database bridge
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
2023-03-02 12:22:11 +01:00
Ivan Dyachkov cac0d2f2ae fix(scripts): typo in run.sh 2023-02-27 23:16:30 +01:00
Zaiming (Stone) Shi 4e70374e2a test: make docker-compose work 2023-02-22 18:28:23 +01:00
Ivan Dyachkov d5174c1555 ci: make sure we use latest compose plugin
https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command
2023-02-20 18:19:04 +01:00
firest 0420e9acb5 test(bridges): add test cases for TDEngine 2023-02-14 22:04:29 +08:00
Kjell Winblad e487247731 build: make sure run.sh with EE app is used with EE profile
If one runs a test suite for an EE app using `scripts/ct/run.sh` without
setting the EE profile, the test suite seems to just get stuck. This
commit ensures that this will not happen and can hopefully save someone
some time.

Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-10 20:14:14 +01:00
Zaiming (Stone) Shi 5fdf7fd24c fix(kafka): use async callback to bump success counters
some telemetry events from wolff are discarded:

* dropped:
    this is double counted in wolff,
    we now only subscribe to the dropped_queue_full event
* retried_failed:
    it has different meanings in wolff,
    in wolff, it means it's the 2nd (or onward) produce attempt
    in EMQX, it means it's eventually failed after some retries

* retried_success
    since we are going to handle the success counters in callbac
    this having this reported from wolff will only make things
    harder to understand

* failed
    wolff never fails (unelss drop which is a different counter)
2023-01-24 21:12:36 +01:00
Zaiming (Stone) Shi 3891aeb5fc ci(ct/run.sh): best-effort file ownership restore 2023-01-18 10:52:29 +01:00
Zaiming (Stone) Shi c3635f537a ci: wait for redis in emqx_ee_bridge_redis_SUITE 2023-01-09 14:39:56 +01:00
Zaiming (Stone) Shi a23fdcab6f chore: delete unused var in ct/run.sh 2023-01-05 22:51:31 +01:00
Zaiming (Stone) Shi 5f12cdff6c ci: should not skip tests in github actions 2023-01-05 22:47:08 +01:00
Thales Macedo Garitezi fd360ac6c0 feat(buffer_worker): refactor buffer/resource workers to always use queue
This makes the buffer/resource workers always use `replayq` for
queuing, along with collecting multiple requests in a single call.
This is done to avoid long message queues for the buffer workers and
rely on `replayq`'s capabilities of offloading to disk and detecting
overflow.

Also, this deprecates the `enable_batch` and `enable_queue` resource
creation options, as: i) queuing is now always enables; ii) batch_size
> 1 <=> batch_enabled.  The corresponding metric
`dropped.queue_not_enabled` is dropped, along with `batching`.  The
batching is too ephemeral, especially considering a default batch time
of 20 ms, and is not shown in the dashboard, so it was removed.
2023-01-05 10:15:09 -03:00
Thales Macedo Garitezi 0014dfef3b chore(docker): add option to stop local testing containers (v5.0) 2023-01-03 10:01:45 -03:00
Zaiming (Stone) Shi e69e016e54 ci: show timestamp from docker compose logs 2022-12-29 11:46:56 +01:00
Zaiming (Stone) Shi 0b43ae621d ci: dump docker-compose log if failed to run ct 2022-12-29 09:23:11 +01:00
Andrew Mayorov 2e1cdd039e
ci: include OTP-25 in some workflows
* Use OTP-25 exclusively for Elixir builds
* Include OTP-25 in slim package builds
* Include OTP-25 in test workflow matrices
2022-12-16 13:45:04 +03:00