Commit Graph

49 Commits

Author SHA1 Message Date
JianBo He
8cbbc9f271 Merge remote-tracking branch 'upstream/master' into cassa 2023-03-23 11:53:17 +08:00
firest
4ad3579966 test(bridges): add test suite for RocketMQ 2023-03-22 10:36:58 +08:00
JianBo He
539ec2f774 chore(bridge): cover username/password auth for cassandra bridges 2023-03-21 13:55:53 +08:00
JianBo He
12942b676d Merge remote-tracking branch 'upstream/master' into cassa 2023-03-20 09:50:27 +08:00
JianBo He
c0a216a740 feat(bridge): support cassandra bridge 2023-03-17 11:34:48 +08:00
Thales Macedo Garitezi
8b1fa50413 ci(fix): fix shellsheck warning 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
e4f058ce2a ci(fix): create user inside container so emqx console works 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi
9d6af17f69 ci(fix): create and give permissions to mix directories 2023-03-16 13:43:01 -03:00
Ivan Dyachkov
a59827cc6a chore(run.sh): prefer docker compose plugin over docker-compose 2023-03-16 13:43:01 -03:00
Ivan Dyachkov
2fe341d152 chore(run.sh): fix permissions on secrets directory 2023-03-16 13:43:01 -03:00
Ivan Dyachkov
5eaaa83b82 chore: simplify run.sh
- get rid of sudo
- do not change permissions of existing files
- use our own docker image to generate certs to make it working on arm
- bump kafka docker image version to have access to multiplatofrm one
2023-03-16 13:43:01 -03:00
firest
984dd3446d test(bridges): add test suite for DynamoDB 2023-03-08 11:13:51 +08:00
Kjell Winblad
96318bda1d build: run run.sh tests without being asked for sudo password
This commit makes it possible to run the scripts/ct/run.sh script
without being asked for sudo password if one instead wants to use the
run docker container with root user trick to reset owner of files.

One simply has to set the environment variable
EMQX_TEST_DO_NOT_RUN_SUDO.
2023-03-02 12:22:11 +01:00
Kjell Winblad
67acdf0888 feat: add clickhouse database bridge
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
2023-03-02 12:22:11 +01:00
Ivan Dyachkov
cac0d2f2ae fix(scripts): typo in run.sh 2023-02-27 23:16:30 +01:00
Zaiming (Stone) Shi
4e70374e2a test: make docker-compose work 2023-02-22 18:28:23 +01:00
Ivan Dyachkov
d5174c1555 ci: make sure we use latest compose plugin
https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command
2023-02-20 18:19:04 +01:00
firest
0420e9acb5 test(bridges): add test cases for TDEngine 2023-02-14 22:04:29 +08:00
Kjell Winblad
e487247731 build: make sure run.sh with EE app is used with EE profile
If one runs a test suite for an EE app using `scripts/ct/run.sh` without
setting the EE profile, the test suite seems to just get stuck. This
commit ensures that this will not happen and can hopefully save someone
some time.

Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-10 20:14:14 +01:00
Zaiming (Stone) Shi
5fdf7fd24c fix(kafka): use async callback to bump success counters
some telemetry events from wolff are discarded:

* dropped:
    this is double counted in wolff,
    we now only subscribe to the dropped_queue_full event
* retried_failed:
    it has different meanings in wolff,
    in wolff, it means it's the 2nd (or onward) produce attempt
    in EMQX, it means it's eventually failed after some retries

* retried_success
    since we are going to handle the success counters in callbac
    this having this reported from wolff will only make things
    harder to understand

* failed
    wolff never fails (unelss drop which is a different counter)
2023-01-24 21:12:36 +01:00
Zaiming (Stone) Shi
3891aeb5fc ci(ct/run.sh): best-effort file ownership restore 2023-01-18 10:52:29 +01:00
Zaiming (Stone) Shi
c3635f537a ci: wait for redis in emqx_ee_bridge_redis_SUITE 2023-01-09 14:39:56 +01:00
Zaiming (Stone) Shi
a23fdcab6f chore: delete unused var in ct/run.sh 2023-01-05 22:51:31 +01:00
Zaiming (Stone) Shi
5f12cdff6c ci: should not skip tests in github actions 2023-01-05 22:47:08 +01:00
Thales Macedo Garitezi
fd360ac6c0 feat(buffer_worker): refactor buffer/resource workers to always use queue
This makes the buffer/resource workers always use `replayq` for
queuing, along with collecting multiple requests in a single call.
This is done to avoid long message queues for the buffer workers and
rely on `replayq`'s capabilities of offloading to disk and detecting
overflow.

Also, this deprecates the `enable_batch` and `enable_queue` resource
creation options, as: i) queuing is now always enables; ii) batch_size
> 1 <=> batch_enabled.  The corresponding metric
`dropped.queue_not_enabled` is dropped, along with `batching`.  The
batching is too ephemeral, especially considering a default batch time
of 20 ms, and is not shown in the dashboard, so it was removed.
2023-01-05 10:15:09 -03:00
Thales Macedo Garitezi
0014dfef3b chore(docker): add option to stop local testing containers (v5.0) 2023-01-03 10:01:45 -03:00
Zaiming (Stone) Shi
e69e016e54 ci: show timestamp from docker compose logs 2022-12-29 11:46:56 +01:00
Zaiming (Stone) Shi
0b43ae621d ci: dump docker-compose log if failed to run ct 2022-12-29 09:23:11 +01:00
Andrew Mayorov
2e1cdd039e
ci: include OTP-25 in some workflows
* Use OTP-25 exclusively for Elixir builds
* Include OTP-25 in slim package builds
* Include OTP-25 in test workflow matrices
2022-12-16 13:45:04 +03:00
Ilya Averyanov
6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Erik Timan
eb62192838 test: expand EE mysql bridge test with toxiproxy 2022-12-01 10:27:51 +01:00
Zaiming (Stone) Shi
1084dd1723 ci: add --remove-orphans option to docker-compose up command 2022-11-30 17:03:41 +01:00
Zaiming (Stone) Shi
cc9e5b1a56 ci(scripts/ct/run.sh): fix undefined vars when runing ct for non-docker 2022-11-30 14:03:24 +01:00
Zaiming (Stone) Shi
83fe35b148 Merge remote-tracking branch 'origin/dev/ee5.0' into 1116-sync-v5.0.10-to-ee50 2022-11-18 19:23:31 +01:00
Zaiming (Stone) Shi
c6b32c2773 ci: ensure PROFILE for ct runs 2022-11-18 12:05:52 +01:00
Zaiming (Stone) Shi
95f3df9a10 ci: ensure docker-compose up erlang container with root when test Kafka 2022-11-18 00:12:48 +01:00
Zaiming (Stone) Shi
09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
Thales Macedo Garitezi
b673bef027 style: indent script 2022-11-11 10:02:02 -03:00
Ivan Dyachkov
c1255188db chore: manually initialize .erlang.cookie for ct_docker 2022-11-09 10:12:13 +01:00
Ivan Dyachkov
7bbd6353aa chore: fix /.cache and /.erlang.cookie in erlang container 2022-11-09 09:30:32 +01:00
firest
9b999f070d chore: make the ./cache to be writeable 2022-11-09 15:39:45 +08:00
Ivan Dyachkov
d90b3760b1 ci(ct/run.sh): use host uid and gid in erlang container for ct 2022-11-09 09:53:03 +08:00
Thales Macedo Garitezi
04588148b7 test(influxdb): increase influxdb bridge/connector coverage (ee5.0) 2022-11-07 15:15:49 -03:00
Kjell Winblad
5ec4b0a6ca fix: fix entrypoint in docker compose for Kafka bridge test 2022-09-23 15:58:22 +02:00
Kjell Winblad
ac37c5d58a test: github actions debug printouts 2022-09-23 15:02:01 +02:00
Kjell Winblad
f0e03086a6 test: add test cases for Kafka SASL auth mechanisms plain and scram 2022-09-13 19:46:56 +02:00
Zaiming (Stone) Shi
0c1595be02 feat: Add Kafka connector 2022-09-13 19:46:56 +02:00
Zaiming (Stone) Shi
befc4acced Merge remote-tracking branch 'origin/master' into merge-master-to-ee50-a 2022-09-06 20:31:38 +02:00
Zaiming (Stone) Shi
f785da075b ci: only start required docker for integration tests 2022-09-06 19:25:53 +02:00