Commit Graph

505 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 5bdb85b04f build: add a script to find previous release version tag 2023-03-09 18:34:26 +01:00
Zaiming (Stone) Shi e4ce9e1111 chore: make possible to debug format-changelog.sh 2023-03-09 18:34:26 +01:00
Zaiming (Stone) Shi 9bba9b47c7 chore: allow untracked files to exist when cutting release 2023-03-09 18:34:26 +01:00
Zaiming (Stone) Shi b0e6b290ff chore: allow no changelog diff when cutting a release 2023-03-09 18:34:26 +01:00
Zaiming (Stone) Shi 16f6789c9c chore: update script print info 2023-03-09 18:34:26 +01:00
Zaiming (Stone) Shi fe27604010 Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master 2023-03-08 16:46:45 +01:00
firest 984dd3446d test(bridges): add test suite for DynamoDB 2023-03-08 11:13:51 +08:00
Ivan Dyachkov ea68a75725 chore: reenable scripts/apps-version-check.sh 2023-03-03 13:56:21 +01:00
Kjell Winblad c24a84e370 docs: fix spelling 2023-03-03 12:06:18 +01:00
Kjell Winblad 6a87b119b0 docs: add word to spellcheck dict 2023-03-03 08:32:43 +01:00
Kjell Winblad 96318bda1d build: run run.sh tests without being asked for sudo password
This commit makes it possible to run the scripts/ct/run.sh script
without being asked for sudo password if one instead wants to use the
run docker container with root user trick to reset owner of files.

One simply has to set the environment variable
EMQX_TEST_DO_NOT_RUN_SUDO.
2023-03-02 12:22:11 +01:00
Kjell Winblad 67acdf0888 feat: add clickhouse database bridge
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
2023-03-02 12:22:11 +01:00
Ivan Dyachkov 6be9967d2d chore: add a reminder to push the tags 2023-03-01 20:51:12 +01:00
Ivan Dyachkov 7a154e4d58 chore: docker-latest tag can be create automatically in cut.sh 2023-03-01 20:51:03 +01:00
Zaiming (Stone) Shi 083330ad80 Merge remote-tracking branch 'origin/master' into 0301-merge-release-50-to-master 2023-03-01 08:53:03 +01:00
Zaiming (Stone) Shi 8d63c6c08d
Merge pull request #10041 from zmstone/0228-add-influx-line-protocol-desc
0228 add influx line protocol desc
2023-03-01 08:44:26 +01:00
Zaiming (Stone) Shi 8d00c003a9 test: fix test script 2023-02-28 21:06:17 +01:00
Zaiming (Stone) Shi 24f476e35f test: add README to influxdb test script 2023-02-28 19:38:43 +01:00
Zaiming (Stone) Shi 2682a1ee28 test: use static balancing for dashboard backend in haproxy 2023-02-28 11:56:46 +01:00
Ivan Dyachkov cac0d2f2ae fix(scripts): typo in run.sh 2023-02-27 23:16:30 +01:00
Zaiming (Stone) Shi 79bf77c2f1
Merge pull request #10020 from zmstone/0222-fix-bridge-async-mode-counters
fix(bridge): fix dropped counter and inflight gauge
2023-02-24 19:51:24 +01:00
Zaiming (Stone) Shi e9787c3ee3
Merge pull request #9966 from zmstone/0213-port-blackbox-cover-report-from-v4
feat: add new module emqx_cover.erl
2023-02-23 09:44:11 +01:00
Zaiming (Stone) Shi 3458aee296
Merge branch 'master' into 0213-port-blackbox-cover-report-from-v4 2023-02-22 22:21:44 +01:00
Zaiming (Stone) Shi 4e70374e2a test: make docker-compose work 2023-02-22 18:28:23 +01:00
Zaiming (Stone) Shi 1715b9a480 refactor: make format-changelog.sh easier to run for alpha/beta releases 2023-02-21 20:43:48 +01:00
Zaiming (Stone) Shi 7538a672b2 refactor: move changelog formatter to scripts/rel sub dir 2023-02-21 20:43:48 +01:00
Zaiming (Stone) Shi 97d08553c1 test: fix test script to use provided boot script 2023-02-21 09:09:51 +01:00
Ivan Dyachkov 67f88f27fd
Merge pull request #10005 from id/ci-try-to-fix-flaky-kafka
ci: fix flaky kafka tests
2023-02-20 23:36:43 +01:00
Ivan Dyachkov d5174c1555 ci: make sure we use latest compose plugin
https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command
2023-02-20 18:19:04 +01:00
William Yang ebd0fb74a3 test(quic): by default, bind to port not IPv4 2023-02-20 10:43:24 +01:00
William Yang fef0a9375c chore(quic): make spell check happy 2023-02-20 10:43:17 +01:00
Ivan Dyachkov 739455d844 ci: test docker images and packages in the same way 2023-02-17 17:42:46 +01:00
Zaiming (Stone) Shi 92c3cd6882
Merge pull request #9982 from zmstone/0216-fix-bin-emqx-read-vm-args
0216 allow starting two nodes from the same boot script
2023-02-17 14:42:02 +01:00
Zaiming (Stone) Shi fea310cf93 test: use different loop back address in script which starts two nodes 2023-02-17 13:22:15 +01:00
Andrew Mayorov c7f535abc7
chore: bump OTP-24 image version to 24.3.4.2-2 2023-02-17 11:16:26 +03:00
Zaiming (Stone) Shi 8fcce53a0d test: add script to run two nodes in host from the same build 2023-02-16 14:35:50 +01:00
lafirest 8a822c664c
Merge pull request #9971 from lafirest/refactor/format_changes
chore: refactor the format-changelog script
2023-02-16 20:35:59 +08:00
firest 6599c44213 chore: refactor the format-changelog script 2023-02-16 15:25:31 +08:00
firest 0420e9acb5 test(bridges): add test cases for TDEngine 2023-02-14 22:04:29 +08:00
firest 530b21268a feat(bridges): add TDengine 2023-02-14 22:04:23 +08:00
Zaiming (Stone) Shi f738415b63 ci: allow minor version bump in app vsns 2023-02-13 22:31:18 +01:00
Zaiming (Stone) Shi 3bf8a7ede7
Merge pull request #9960 from zmstone/0213-merge-release-50-to-master
0213 merge release 50 to master
2023-02-13 13:38:01 +01:00
Kjell Winblad 80cd018f28
Merge pull request #9934 from kjellwinblad/kjell/fix/run-sh-ee-app-with-ee-profile
build: make sure run.sh with EE app is used with EE profile
2023-02-13 13:37:24 +01:00
Zaiming (Stone) Shi 7947d83cb9 ci: fix change log format script 2023-02-13 08:57:54 +01:00
Kjell Winblad e487247731 build: make sure run.sh with EE app is used with EE profile
If one runs a test suite for an EE app using `scripts/ct/run.sh` without
setting the EE profile, the test suite seems to just get stuck. This
commit ensures that this will not happen and can hopefully save someone
some time.

Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-10 20:14:14 +01:00
Zaiming (Stone) Shi dfcc59afcf chore: add changelog 2023-02-10 19:40:37 +01:00
Zaiming (Stone) Shi 157c919ba1 ci: add i18n style check script 2023-02-09 11:41:52 +01:00
Zaiming (Stone) Shi ccb0c7ebaa Merge remote-tracking branch 'origin/release-50' into 0202-merge-release-50-back-to-master 2023-02-03 16:12:11 +01:00
Zaiming (Stone) Shi c95d201331 ci: no relup for now 2023-02-03 09:40:28 +01:00
Zaiming (Stone) Shi 6362476508 feat: disable telemetry for ee by default 2023-02-03 09:29:39 +01:00
Zaiming (Stone) Shi 90f23ffc78 ci: no relup support for now 2023-02-02 21:43:15 +01:00
Zaiming (Stone) Shi 0c80c31c9e test: delete log override from influxdb bridge test script 2023-01-31 20:43:59 +01:00
Zaiming (Stone) Shi 5fdf7fd24c fix(kafka): use async callback to bump success counters
some telemetry events from wolff are discarded:

* dropped:
    this is double counted in wolff,
    we now only subscribe to the dropped_queue_full event
* retried_failed:
    it has different meanings in wolff,
    in wolff, it means it's the 2nd (or onward) produce attempt
    in EMQX, it means it's eventually failed after some retries

* retried_success
    since we are going to handle the success counters in callbac
    this having this reported from wolff will only make things
    harder to understand

* failed
    wolff never fails (unelss drop which is a different counter)
2023-01-24 21:12:36 +01:00
Erik Timan 424d6ed34b test: improve start.sh script for influxdb tests 2023-01-24 14:42:52 +01:00
Erik Timan 21e933e9bb test: add rule to influx-bridge.conf and add newlines 2023-01-24 14:37:15 +01:00
Erik Timan fbfd4a2565 test: use EMQX_IMAGE_TAG for influx test script 2023-01-24 14:13:03 +01:00
Erik Timan 9fc586f670 test: add influx docker test certs 2023-01-24 13:58:37 +01:00
Erik Timan 502b826396 test: add influxdb docker test files 2023-01-24 13:58:37 +01:00
Zaiming (Stone) Shi e8910c9748 chore: do not gnerate change log when first release or alpha, beta, rc 2023-01-18 17:36:38 +01:00
Zaiming (Stone) Shi 941216b6d0 chore: disable hot-upgrade for now 2023-01-18 17:25:05 +01:00
Zaiming (Stone) Shi 3891aeb5fc ci(ct/run.sh): best-effort file ownership restore 2023-01-18 10:52:29 +01:00
Ivan Dyachkov 49e1f51765 ci: simplify build_and_push_docker_images workflow 2023-01-12 09:45:17 +01:00
Zaiming (Stone) Shi c3635f537a ci: wait for redis in emqx_ee_bridge_redis_SUITE 2023-01-09 14:39:56 +01:00
Zaiming (Stone) Shi a23fdcab6f chore: delete unused var in ct/run.sh 2023-01-05 22:51:31 +01:00
Zaiming (Stone) Shi 5f12cdff6c ci: should not skip tests in github actions 2023-01-05 22:47:08 +01:00
Thales Macedo Garitezi fd360ac6c0 feat(buffer_worker): refactor buffer/resource workers to always use queue
This makes the buffer/resource workers always use `replayq` for
queuing, along with collecting multiple requests in a single call.
This is done to avoid long message queues for the buffer workers and
rely on `replayq`'s capabilities of offloading to disk and detecting
overflow.

Also, this deprecates the `enable_batch` and `enable_queue` resource
creation options, as: i) queuing is now always enables; ii) batch_size
> 1 <=> batch_enabled.  The corresponding metric
`dropped.queue_not_enabled` is dropped, along with `batching`.  The
batching is too ephemeral, especially considering a default batch time
of 20 ms, and is not shown in the dashboard, so it was removed.
2023-01-05 10:15:09 -03:00
ieQu1 e33409132e ci: Add generate_changlelog script to scripts/rel/cut.sh 2023-01-05 13:28:14 +01:00
ieQu1 6278d48bd7 ci: Change how the changelog is created 2023-01-05 13:28:12 +01:00
Thales Macedo Garitezi 0014dfef3b chore(docker): add option to stop local testing containers (v5.0) 2023-01-03 10:01:45 -03:00
ieQu1 ff145ecc43
feat(replay): Add metadata storage and supervision tree 2023-01-03 14:08:40 +03:00
Thales Macedo Garitezi 7e02eac3bc
Merge pull request #9619 from thalesmg/refactor-gauges-v50
refactor(metrics): use absolute gauge values rather than deltas (v5.0)
2023-01-02 10:56:47 -03:00
Zaiming (Stone) Shi 0ce1ca89b7 refactor: use string type for server and servers 2022-12-30 14:20:23 +01:00
firest 081d8accd3 feat(bom): import rebar_sbom for generating bom files 2022-12-30 00:00:14 +08:00
Zaiming (Stone) Shi e69e016e54 ci: show timestamp from docker compose logs 2022-12-29 11:46:56 +01:00
Zaiming (Stone) Shi 0b43ae621d ci: dump docker-compose log if failed to run ct 2022-12-29 09:23:11 +01:00
Thales Macedo Garitezi a0e11f75d9 refactor(docs): use var for output filepath 2022-12-27 10:30:14 -03:00
Ivan Dyachkov ab474323a4 build(macos): do not automatically lock keychain 2022-12-27 09:08:57 +01:00
Ivan Dyachkov a1af5742e2 chore: use unique name for temp macos keychain 2022-12-22 15:34:28 +01:00
Andrew Mayorov 501f0fc8d5
ci: reduce test matrix size
Test emqx profile under OTP-25 only. Build test matrix programmatically
for that purpose.
2022-12-18 23:12:22 +03:00
Andrew Mayorov 30aa7d7a7e
ci: bump elvis version to `1.1.0-emqx-2` 2022-12-16 13:45:05 +03:00
Andrew Mayorov 2e1cdd039e
ci: include OTP-25 in some workflows
* Use OTP-25 exclusively for Elixir builds
* Include OTP-25 in slim package builds
* Include OTP-25 in test workflow matrices
2022-12-16 13:45:04 +03:00
Andrew Mayorov 202d2b07ff
chore: switch to `emqx/rebar3` 3.19.0-emqx-1
Which should be safe to run on OTP-25.
2022-12-16 13:45:04 +03:00
Zaiming (Stone) Shi 1dd12e6d3c Merge remote-tracking branch 'origin/release-50' 2022-12-15 09:11:50 +01:00
Zaiming (Stone) Shi f704c4c96b ci: no need to set python2 for yum for el8 2022-12-14 22:50:07 +01:00
Zaiming (Stone) Shi 56066a03b5 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 20:04:20 +01:00
Ivan Dyachkov d8b1809cfc chore: fix distro detection for amazon linux 2 2022-12-14 19:09:14 +01:00
Zaiming (Stone) Shi 42c58e2a91 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 15:29:13 +01:00
Thales Macedo Garitezi f827062f0b
Merge pull request #9449 from thalesmg/gcp-pubsub-ee50
feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0)
2022-12-14 10:31:12 -03:00
Ivan Dyachkov 4f0cf167ec chore: switch to python2 to use yum in tests 2022-12-13 09:04:57 +01:00
Thales Macedo Garitezi cdae451a84 ci(fix): ensure directory for i18n output exists
```
Run make eunit
escript: exception error: no match of right hand side value {error,enoent}
  in function  'merge-i18n_escript__escript__1669__814803__620426__2':main/1 (/__w/emqx/emqx/source/scripts/merge-i18n.escript, line 13)
  in call from escript:run/2 (escript.erl, line 750)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_em/1
  in call from init:do_boot/3
make: *** [Makefile:225: conf-segs] Error 127
```
2022-12-12 17:18:19 -03:00
Ivan Dyachkov 93789fb872 ci: fix logs artefact name 2022-12-09 16:13:53 +01:00
Zaiming (Stone) Shi 40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
Zaiming (Stone) Shi f3b069a0d9 test: fix config load for lib-ee tests 2022-12-07 15:50:50 +01:00
Ilya Averyanov 6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi 1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Zaiming (Stone) Shi f91d97d3c9
Merge pull request #9428 from zmstone/1127-build-add-release-help-scripts
build: add release helper scripts
2022-12-03 12:53:49 +01:00
Erik Timan eb62192838 test: expand EE mysql bridge test with toxiproxy 2022-12-01 10:27:51 +01:00
Zaiming (Stone) Shi 1084dd1723 ci: add --remove-orphans option to docker-compose up command 2022-11-30 17:03:41 +01:00
Zaiming (Stone) Shi cc9e5b1a56 ci(scripts/ct/run.sh): fix undefined vars when runing ct for non-docker 2022-11-30 14:03:24 +01:00
Zaiming (Stone) Shi 7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
Zaiming (Stone) Shi 729cd8fda4 build: add release helper scripts 2022-11-27 22:57:14 +01:00
Ivan Dyachkov 262e53eed2 chore: ensure macos keychain does not exist when signing binaries 2022-11-20 16:38:55 +01:00
Zaiming (Stone) Shi e4a4956c19 ci: fix find-apps.sh default printout 2022-11-18 21:18:14 +01:00
Zaiming (Stone) Shi 83fe35b148 Merge remote-tracking branch 'origin/dev/ee5.0' into 1116-sync-v5.0.10-to-ee50 2022-11-18 19:23:31 +01:00
Zaiming (Stone) Shi 06cb7aeb48 ci: make ci matrix from find-apps.sh 2022-11-18 19:06:51 +01:00
Zaiming (Stone) Shi c6b32c2773 ci: ensure PROFILE for ct runs 2022-11-18 12:05:52 +01:00
Zaiming (Stone) Shi 95f3df9a10 ci: ensure docker-compose up erlang container with root when test Kafka 2022-11-18 00:12:48 +01:00
Zaiming (Stone) Shi c940b901f5 chore: fix app versions 2022-11-16 16:26:43 +01:00
Zaiming (Stone) Shi 09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
Ivan Dyachkov 9d1d273a1b chore: sign extra files on macos when packaging ee 2022-11-16 15:59:40 +01:00
Thales Macedo Garitezi b673bef027 style: indent script 2022-11-11 10:02:02 -03:00
Ivan Dyachkov c1255188db chore: manually initialize .erlang.cookie for ct_docker 2022-11-09 10:12:13 +01:00
Ivan Dyachkov 7bbd6353aa chore: fix /.cache and /.erlang.cookie in erlang container 2022-11-09 09:30:32 +01:00
firest 9b999f070d chore: make the ./cache to be writeable 2022-11-09 15:39:45 +08:00
Ivan Dyachkov d90b3760b1 ci(ct/run.sh): use host uid and gid in erlang container for ct 2022-11-09 09:53:03 +08:00
Zaiming (Stone) Shi 8faa9d8358 chore: make nl check script happy 2022-11-08 21:36:03 +01:00
Thales Macedo Garitezi 04588148b7 test(influxdb): increase influxdb bridge/connector coverage (ee5.0) 2022-11-07 15:15:49 -03:00
Ivan Dyachkov 881a91a788 ci: build elixir images for ce only 2022-11-01 15:57:23 +01:00
Ivan Dyachkov 5654778bd9 build: check if quicer binary present before trying to sign it 2022-10-07 09:50:25 +02:00
Zaiming (Stone) Shi d49675cde5 chore: supprot DNS discovery test for 4.x 2022-09-28 21:31:18 +02:00
Zaiming (Stone) Shi c3f6c302ec
Merge pull request #8998 from zmstone/0918-build-force-fetch-tags-when-update-appup
chore: force fetch upstream tags before appup updates
2022-09-27 21:45:02 +02:00
Shawn 4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
Zaiming (Stone) Shi f6ac4c3a76
Merge pull request #8798 from zmstone/0815-feat-add-kafka-connector
feat: Add Kafka connector
2022-09-24 22:57:50 +02:00
Kjell Winblad 5ec4b0a6ca fix: fix entrypoint in docker compose for Kafka bridge test 2022-09-23 15:58:22 +02:00
Kjell Winblad ac37c5d58a test: github actions debug printouts 2022-09-23 15:02:01 +02:00
JimMoen 7aa99b26b9 chore: add git commit hook for git-blame-ignore-revs
ignore redis log files
2022-09-23 12:37:07 +08:00
Kjell Winblad 9ea0147a8c feat(rule engine jq function): config for changing implementation
This commit adds a rule engine configuration option for changing the
implementation module used for the rule engine function jq. The two
options are `jq_port` (uses Erlang port programs to interact with jq)
and `jq_nif` (uses an Erlang NIF library to interact with jq).

Thanks to @terry-xiaoyu (Xinyu Liu <506895667@qq.com>) for Chinese
translations
2022-09-22 09:02:30 +02:00
Zaiming (Stone) Shi 595f19cddd chore: force fetch upstream tags before appup updates 2022-09-18 11:35:39 +02:00
Ivan Dyachkov 501667dc1f chore: skip signing macos binaries when dev cert of pw are not set 2022-09-15 10:27:09 +02:00
zhongwencool b2732dc3ff
Merge pull request #8950 from zhongwencool/rename-emqx-enterprise.conf
feat: don't include emqx-entriprise.conf in opensource
2022-09-14 11:05:28 +08:00
Zhongwen Deng 499ed3ca51 chore: don't generate enterprise.conf when not emqx-enterprise profile 2022-09-14 10:13:58 +08:00
Kjell Winblad f0e03086a6 test: add test cases for Kafka SASL auth mechanisms plain and scram 2022-09-13 19:46:56 +02:00
Zaiming (Stone) Shi 0c1595be02 feat: Add Kafka connector 2022-09-13 19:46:56 +02:00
Ivan Dyachkov cb5be71165
Merge pull request #8897 from emqx/build-sign-macos-binaries
build: sign binary files in macos package
2022-09-13 16:31:32 +02:00
Zhongwen Deng 6a470dc3ac feat: don't include emqx-entriprise.conf in opensource 2022-09-13 18:53:56 +08:00
Zaiming (Stone) Shi 4534966e36 Merge remote-tracking branch 'origin/master' into dev/ee5.0 2022-09-13 12:16:54 +02:00
Ivan Dyachkov 7cfca31aaf build: sign binary files in macos package 2022-09-12 21:50:00 +02:00
Zaiming (Stone) Shi d6bd1555ec refactor: move spellcheck dictionary to this repo
prior to this change, the EMQX spellcheck dictionary
is baked into the docker image, which makes adding
new words to the dict more complicated long process:
you'd have to send a separate PR to the docker image
repo and tag a new docker image and update the dokcer
iamge tag in emqx.git
2022-09-12 20:02:16 +02:00
Zaiming (Stone) Shi c9b6dfd7f9
Merge pull request #8905 from zmstone/0906-warn-default-erlang-cookie
refactor(cookie): Warning message when boot with default Erlang cookie
2022-09-07 08:06:55 +02:00
Zaiming (Stone) Shi befc4acced Merge remote-tracking branch 'origin/master' into merge-master-to-ee50-a 2022-09-06 20:31:38 +02:00
Zaiming (Stone) Shi f785da075b ci: only start required docker for integration tests 2022-09-06 19:25:53 +02:00
Zaiming (Stone) Shi ac840e318e refactor(cookie): Warning message when boot with default Erlang cookie 2022-09-06 18:58:35 +02:00
Thales Macedo Garitezi 3d4afd65df feat: add mongodb bridge (e5.0) 2022-09-01 14:47:14 -03:00
Zaiming (Stone) Shi dac178cbaf chore: ensure version bumps 2022-08-31 17:23:47 +02:00
Shawn b3162fe5ff fix: conflicts between master and ee5.0 2022-08-14 22:24:09 +08:00
Zaiming (Stone) Shi c82a05a0e7 refactor(Makefile): support download enterprise dashboard download 2022-08-12 14:36:31 +02:00
JimMoen d51ca2672a fix: spellcheck and update checker repo version 2022-08-09 11:50:05 +08:00
firest 966b508c6f chore: fix status test error 2022-08-09 11:22:53 +08:00
Thales Macedo Garitezi b19e8fb3cd feat(license): add HTTP API for license 2022-08-02 09:31:59 -03:00
DDDHuang 3fe76446bc Merge branch 'master' into dev/ee5.0 2022-08-01 10:14:21 +08:00