Commit Graph

316 Commits

Author SHA1 Message Date
Ivan Dyachkov 7c917334dc ci: enforce stricter condition on ssl_cert_gen dependency 2023-02-20 18:16:20 +01:00
Andrew Mayorov c7f535abc7
chore: bump OTP-24 image version to 24.3.4.2-2 2023-02-17 11:16:26 +03:00
Thales Macedo Garitezi 6bf385793c
Merge pull request #9978 from thalesmg/fix-psql-tls-v50
fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0)
2023-02-16 09:20:32 -03:00
Ilya Averyanov c684730bbe chore(ci): remove legacy helper Makefile 2023-02-16 11:30:56 +02:00
Thales Macedo Garitezi dbda504f2c fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0)
Fixes https://github.com/emqx/emqx/issues/9907

At v5.0.14, we changed the `ssl` option for the Postgres connector
from `true` to `required`, but there was another transformation in
`conn_opts/2` that led to an incorrect configuration.  This change
ended up preventing users from connecting to Postgres with their
previous configurations after upgrading EMQX.
2023-02-15 14:06:33 -03:00
firest 0420e9acb5 test(bridges): add test cases for TDEngine 2023-02-14 22:04:29 +08:00
Ivan Dyachkov 202f6d0181 ci: stop building alpine docker image 2023-02-09 14:19:35 +01:00
Ilya Averyanov fce1e74c3d fix(connector): fix redis cluster resource recovery 2023-01-31 16:55:05 +02:00
Ilya Averyanov f6fbbf3ee3 chore(bridges): reduce Redis bridge flakyness 2023-01-18 14:34:11 +02:00
firest ea405fe55d test(bridges): add test case for the PostgreSQL backend 2023-01-06 11:56:28 +08:00
Zaiming (Stone) Shi 6d7e6e6ea7 ci: delay Kafka topic creation 2022-12-29 14:30:30 +01:00
Andrew Mayorov 2e1cdd039e
ci: include OTP-25 in some workflows
* Use OTP-25 exclusively for Elixir builds
* Include OTP-25 in slim package builds
* Include OTP-25 in test workflow matrices
2022-12-16 13:45:04 +03:00
Zaiming (Stone) Shi 40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
Zaiming (Stone) Shi f3b069a0d9 test: fix config load for lib-ee tests 2022-12-07 15:50:50 +01:00
Ilya Averyanov 6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Erik Timan eb62192838 test: expand EE mysql bridge test with toxiproxy 2022-12-01 10:27:51 +01:00
Zaiming (Stone) Shi 83fe35b148 Merge remote-tracking branch 'origin/dev/ee5.0' into 1116-sync-v5.0.10-to-ee50 2022-11-18 19:23:31 +01:00
Zaiming (Stone) Shi 09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
Ivan Dyachkov 7bbd6353aa chore: fix /.cache and /.erlang.cookie in erlang container 2022-11-09 09:30:32 +01:00
Ivan Dyachkov d90b3760b1 ci(ct/run.sh): use host uid and gid in erlang container for ct 2022-11-09 09:53:03 +08:00
Thales Macedo Garitezi 04588148b7 test(influxdb): increase influxdb bridge/connector coverage (ee5.0) 2022-11-07 15:15:49 -03:00
Shawn 4135910b42 chore: merge master into dev/ee5.0 2022-09-26 09:52:33 +08:00
Kjell Winblad 5ec4b0a6ca fix: fix entrypoint in docker compose for Kafka bridge test 2022-09-23 15:58:22 +02:00
Kjell Winblad ac37c5d58a test: github actions debug printouts 2022-09-23 15:02:01 +02:00
Kjell Winblad a3c88b40a0 test: changes to make Kafka container run in GitHub action 2022-09-23 14:33:41 +02:00
Kjell Winblad c4f7d385b5 test: fix true not allowed as docker compose env var value 2022-09-23 11:47:12 +02:00
Kjell Winblad 9f3e38aeb0 test: fix error in script detected by spellcheck script 2022-09-23 10:37:30 +02:00
Zaiming (Stone) Shi ba34326010 ci(kafka): fix shellcheck errors 2022-09-23 08:58:51 +02:00
JimMoen 7aa99b26b9 chore: add git commit hook for git-blame-ignore-revs
ignore redis log files
2022-09-23 12:37:07 +08:00
Kjell Winblad ac2922fc4c test: Kafka bridge cases for all combinations of SASL and SSL 2022-09-16 16:44:12 +02:00
Kjell Winblad 5820b028cb feat: add test case for Kerberos Kafka authentication 2022-09-14 17:03:37 +02:00
Kjell Winblad f0e03086a6 test: add test cases for Kafka SASL auth mechanisms plain and scram 2022-09-13 19:46:56 +02:00
Zaiming (Stone) Shi 0c1595be02 feat: Add Kafka connector 2022-09-13 19:46:56 +02:00
Thales Macedo Garitezi 3d4afd65df feat: add mongodb bridge (e5.0) 2022-09-01 14:47:14 -03:00
Zhongwen Deng b08991cb44 chore: upgrade minirest to 1.3.7 2022-08-17 23:31:27 +08:00
Zhongwen Deng abcdad39e5 chore: bump fvt_test's minirest to latest(1.3.6) 2022-08-03 16:29:06 +08:00
Zaiming (Stone) Shi 387a1d822a
Merge pull request #8410 from zmstone/0704-add-haproxy-to-cluster-example-secript
0704 add haproxy to cluster example script
2022-07-19 19:27:05 +01:00
JimMoen ab17fd80e7 ci: common test add redis-sentinel 2022-07-13 10:37:16 +08:00
Zaiming (Stone) Shi a95474669e chore: remove duplicated haproxy config section
the mgmt and dashbaord endpoints are both merged 18083 now
no need to declare twice
2022-07-04 17:57:13 +02:00
Zaiming (Stone) Shi 8b7fc490ae refactor: run relup test nodes in individual docker containers
When running EMQX in lux shell inside a docker container
the node won't boot due to load_failed for redbug modules
reason is still unknown
2022-06-29 07:10:45 +02:00
Zaiming (Stone) Shi 184b337e84 chore: add a script to run relup lux test locally 2022-06-28 10:54:07 +02:00
Zaiming (Stone) Shi f7a2112a5a ci: define package name instead of globing 2022-06-27 22:22:35 +02:00
Zaiming (Stone) Shi 67710472f5 ci: debug boot in relup test 2022-06-27 20:49:20 +02:00
Zaiming (Stone) Shi d3f8ebd10f ci: fix relup test workflow and upgrade builder to 5.0-17
builder 5.0-17 has emqtt-bench and lux included
so we do not need to rebuild the software for each ci job
2022-06-27 12:06:13 +02:00
Zaiming (Stone) Shi 0d6c4e0996 Merge remote-tracking branch 'xinyu/fix_relup_testcases' into 0623-fix-relup 2022-06-24 06:00:16 +02:00
Zaiming (Stone) Shi bcefec1935 ci: fix relup test 2022-06-23 21:59:43 +02:00
Shawn 9ec88198cb fix: remove one_more_emqx.sh from the relup.lux 2022-06-23 09:14:57 +08:00
Shawn 0d980904e0 fix: tmp - test relup 2022-06-22 22:16:03 +08:00
Shawn efffc450d0 fix: update relup.lux 2022-06-22 19:21:40 +08:00
Shawn 5d81f159d6 fix(CI): syntax errors in run_relup_tests.yaml 2022-06-22 14:53:34 +08:00
Shawn 08207c086c fix: update integration testcases for relup 2022-06-22 11:56:57 +08:00
Shawn f18eab402a fix(CI): don't use any authz sources when testing 2022-06-10 14:21:20 +08:00
Zaiming (Stone) Shi fc4d9e4dcd ci: use emqx-builder 5.0-16
Fixed openssl compatibility issue
2022-06-06 10:54:26 +02:00
Thales Macedo Garitezi d0c5de1da6
chore(config): move db configs under cluster/node keys 2022-05-26 14:02:22 -03:00
Shawn eea9bd90b4 fix: remove the extra level of 'emqx' dir from .tar.gz 2022-05-25 02:00:01 +08:00
Shawn 5d65bdaa47 chore: ping emqx-builder to 5.0-15 2022-05-16 18:10:42 +08:00
Zaiming (Stone) Shi e95a14843d ci: delete otp 23 from ci matrix 2022-05-10 16:34:53 +02:00
Kjell Winblad 74c33cd4e5 feat(rule_engine): add jq function to the rule engine
This commit adds a function to the rule engine that alows users
to transform text or JSON objects using [jq filter programs][1].

[jq][1] is a command line tool that can be used to transform
and filter JSON text using jq's built-in language. The rule engine
function that is added with this commit uses the
[Erlang jq NIF library][2] that wraps the jq C library in an
Erlang NIF function.

[1]: https://stedolan.github.io/jq/
[2]: https://github.com/emqx/jq
2022-05-04 17:09:16 +02:00
Zaiming (Stone) Shi b49df4599a build: use latest builder images (alpine 3.15.1)
OpenSSL-1.1.1n
2022-03-25 07:38:39 +01:00
xiangfangyang-tech a1c76f474b chore(autotest): add gitactions for new autotest scripts 2022-03-10 15:59:09 +08:00
Thales Macedo Garitezi a85d636a68
feat(rlog): use rlog as the default db backend 2022-03-08 13:34:05 -03:00
Zaiming (Stone) Shi e6495f4300 ci: upgrade emqx-builder to 5.0-8:1.13.3-24.2.1-1 2022-03-02 21:57:26 +01:00
Thales Macedo Garitezi 8cac5f6b39
chore(ci): update emqx-builder image
Uses Elixir 1.13.3 and OTP 24.2.1-1.
2022-02-23 15:00:07 -03:00
Zaiming (Stone) Shi 1aa9abdb4d chore: relocate tests.sh 2022-02-21 12:22:53 +01:00
Thales Macedo Garitezi 1b3b2c0d03
feat: add eval-erl command to `bin/emqx`
This allows Erlang expressions to be evaluated easily even on nodes
running Elixir.
2022-02-04 16:53:27 -03:00
Thales Macedo Garitezi 6d557bc0ca
ci(pytest): pin pytest version at 6.2.5
After a major version release (7.0.0), our current FVT tests all
failed due to some change in the lib.  Our pytest version in CI was
not pinned, and picked this update up.
2022-02-04 10:19:29 -03:00
Thales Macedo Garitezi 506dad169a
chore(elixir_otp): bump emqx-builder images 2022-01-26 16:59:50 -03:00
Thales Macedo Garitezi 9c68f4bd11
fix(tgz): replace zip with tgz after rebase 2022-01-26 16:59:49 -03:00
Thales Macedo Garitezi 3ec7d9ae24
chore(ci): use new tagged image with elixir 2022-01-26 16:59:49 -03:00
Thales Macedo Garitezi 59f0d5f537
chore(mix): use flag instead of elixirpkg to denote package type
Also, add Makefile target to build zip files with Elixir.
2022-01-26 16:59:49 -03:00
Thales Macedo Garitezi 5a7943f583
chore(ci): grep `emqx ctl listeners` to check if node is listening 2022-01-26 16:59:49 -03:00
Thales Macedo Garitezi ba6d4e4a19
fix(ci): raspbian arch is arm, not arm64 2022-01-26 16:59:49 -03:00
Thales Macedo Garitezi e874ec2b36
chore(images): use new image in more CI workflows 2022-01-26 16:59:49 -03:00
Thales Macedo Garitezi bc498a8041
ci(mix): add fvt tests and docker images with elixir 2022-01-26 16:59:48 -03:00
Thales Macedo Garitezi 6dd11665bb
ci(mix): build emqx packages with elixir 2022-01-26 16:59:48 -03:00
Zaiming (Stone) Shi b44b090d9e ci: get package version per profile 2022-01-26 14:25:59 +01:00
JimMoen d8f49f8a5f
Merge pull request #6732 from JimMoen/telemetry-api-refactor
refactor(api): telemetry api swagger spec
2022-01-14 09:36:54 +08:00
Zaiming (Stone) Shi ecd7ea0a2f ci: change to build and test with .tar.gz instead of .zip packages 2022-01-13 18:54:01 +01:00
JimMoen 82e3565344 chore: major mode file variable for Emacs 2022-01-13 15:53:47 +08:00
Thales Macedo Garitezi b9494662ae
fix(ci): fix shellcheck file pattern
Since we started to use the portable shebang style for our scripts, we
need to update the pattern used to find scripts for Shellcheck to
check.
2022-01-05 11:34:04 -03:00
Zaiming (Stone) Shi 3ee4d69988 ci: fix rpm install package name
prior to this fix, package name 'emqx' was hard coded
this commit fixes it to use EMQX_NAME variable
because it can be 'emqx-edge' or 'emqx-enterprise'
2022-01-04 17:28:53 +01:00
Zaiming (Stone) Shi 77652b9858 ci: refactor package build & test jobs 2022-01-04 17:28:53 +01:00
Ilya Averyanov 7208669946 chore(authn): reduce number of pre-generated certificates, refactor
tests
2021-12-30 13:49:14 +03:00
Ilya Averyanov a7ca6cb39f chore(authn): test MongoDB authn via ssl connection 2021-12-29 16:24:52 +03:00
Ilya Averyanov e2e2c98679 chore(authn): test Mysql authn via ssl connection 2021-12-27 11:16:31 +03:00
Ilya Averyanov 6de89d1207 chore(authn): test PostgreSQL authn via ssl connection 2021-12-27 11:16:31 +03:00
Ilya Averyanov 520629d6e2 chore(authn): test Redis authn via ssl connection 2021-12-27 11:16:31 +03:00
Ilya Averyanov 6ad71a483e chore(authz): test Redis backend with real Redis 2021-12-17 14:18:54 +03:00
William Yang 4402327da0 ci(deb): remove system V test 2021-12-02 22:46:22 +01:00
zhouzb caf1784a90 build: update otp version 2021-12-02 15:25:11 +08:00
Ilya Averyanov 390575eafb chore(authn): add MongoDB backend tests 2021-11-29 21:17:56 +03:00
Ilya Averyanov 8cc0b43de7 chore(authn): add MySQL & PostgreSQL backend tests 2021-11-23 11:52:51 +03:00
Ilya Averyanov 071c2c99e8 refactor(authn resources): add `emqx_resource` and `emqx_authn` tests 2021-11-22 21:08:04 +03:00
Zaiming Shi a705bc1fc7 build: change package name scheme
`${PROFILE}-${PKG_VSN}-otp${OTP_VSN}-${SYSTEM}-${ARCH}.zip`
2021-11-20 20:44:48 +01:00
Thales Macedo Garitezi 315a06214f
test(rlog): reenable RLOG FVT tests
The RLOG DB backend tests in FVT were temporarily disabled due to some
paho tests being specially flaky in CI.

In particular, those tests had the common pattern of subscribing to a
topic, immediately publishing to that topic and then waiting for the
response.  When in CI and using RLOG, there seems to be more delays in
replication of data, and often this pattern would fail in the
constraint testing enviroment.
2021-11-18 18:03:30 -03:00
zhanghongtong 670281a0fa ci: change localhost to 127.0.0.1 for support new curl version 2021-11-18 15:30:41 +08:00
Thales Macedo Garitezi 030e4857ec
docs(issue): mark solution as TODO and link related issue
https://github.com/emqx/emqx/issues/6094
2021-11-08 15:36:54 -03:00
Thales Macedo Garitezi f8fc67b313
fix(lag): target only replica if rlog core+replicant
there seems to be race conditions related to some tests with sessions
hitting the core and the replicant alternately and rlog.

for intance, if there is some delay in this replication, a new
connection made to the replica with a just-created session in the core
may not have been replicated to the replicant, resulting in a test
failure if it expects the session to be present.

since such replication lags are inherent to the core-replicant
topology, we can try to target only the replicant to avoid seeing this
inconsistent view of the system during the tests.
2021-11-08 14:18:07 -03:00
Thales Macedo Garitezi c60feaaad2
test(fvt): extend functional verification tests to use replicant node
This parameterizes the Functional Verification Tests (FVTs) that run
in CI to use a replication log (RLOG) role of "replicant" for one of
the nodes.  With this addition, our FVTs may explore more scenarios
with data replication.
2021-11-07 17:02:57 -03:00
zhanghongtong f817ba0075 chore(CI): rename image 2021-10-22 13:40:01 +08:00
zhanghongtong 7732796331 chore(CI): change otp version for docker compose file
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-10-22 13:40:01 +08:00
Zaiming Shi a591f00c5b chore(ci): increase wait time for emqx stop when running on arm 2021-09-29 14:21:30 +02:00
Zaiming Shi ad46208fa5 chore(ci): dump debug log if failed to stop 2021-09-29 12:56:09 +02:00
JimMoen deac54c847 chore(for editor): Add comments for Emacs major mode. 2021-09-16 15:43:30 +08:00
William Yang 6e7d3d05e4 ci: install openssl11 for centos7 amd64 only 2021-09-07 09:49:09 +08:00
zhanghongtong cff15dfc44 chore(CI): fix env error for test wrokflows 2021-09-03 15:44:38 +08:00
William Yang 2426482ae1 ci: install openssl11 as deps. 2021-09-02 08:29:02 +02:00
DDDHuang 86231f795d
refactor: close managment http 8081 (#5564) 2021-08-27 13:46:39 +08:00
Shawn 092f29fecd refactor(CI): update the ENVs for new listener,zone configs 2021-08-26 09:11:30 +08:00
Shawn 5177ba02d6 fix(config): update ENVs for logger 2021-08-23 18:25:45 +08:00
Zaiming (Stone) Shi 64ddb13d8e
Merge pull request #5450 from zmstone/chore-update-otp-version-for-ci
chore: update otp version in CI
2021-08-11 14:02:54 +02:00
Zaiming Shi ae8c3cf779 chore(ci): make use of haproxy mqtt sticky session 2021-08-11 10:39:40 +08:00
Zaiming Shi 1d755f1272 test(ci): create certificate chain in tmp dir
the haproxy container boot script has no permission writing to
the mounted dir.
2021-08-11 10:39:40 +08:00
Zaiming Shi 1a4f7f190d test(ci): update haproxy version to 2.4 2021-08-11 10:39:40 +08:00
Zaiming Shi bf94051d56 chore: update otp version in CI 2021-08-10 15:56:22 +02:00
Zaiming Shi cd92f73ca1 test: add debug print 2021-08-09 22:09:11 +02:00
x1001100011 0fe3083cd7 build: add erlang 24.0.1 for ci 2021-08-09 11:12:25 +02:00
Zaiming (Stone) Shi 672f94e118
Merge pull request #5430 from zmstone/refactor-simplify-ci-scripts
refactor: simplify ci scripts
2021-08-09 11:09:44 +02:00
Zaiming Shi 9f3063a823 refactor: bin/emqx call fucntions for error logs 2021-08-09 10:28:04 +02:00
tigercl 40a9932167
Merge pull request #5427 from tigercl/feat/redis-authn
feat(authn redis): support authn with redis
2021-08-09 11:11:37 +08:00
Zaiming Shi c473f9990a test: run package test uder emqx user
no need to su before start
2021-08-08 11:14:26 +02:00
zhouzb 3761db0525 feat(authn redis): support authn with redis 2021-08-06 14:05:31 +08:00
Zaiming Shi bc23ff5e47 feat: add graceful shutdown
prior to this cahnge emqx node shutdown is done by init:stop
which might have undesired stop order of the applications

in this change, emqx_machine_terminator is added to stop apps
in defined order and then terminate the node in infinite loop
2021-08-05 14:11:58 +02:00
zhanghongtong cf20fc6db7 chore(CI): easier build packages
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-08-04 22:20:54 +08:00
zhanghongtong 318beef503 chore(relup tests): check rules matched
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-07-22 10:57:45 +08:00
William Yang de387c4c8e chore: gitignore erlang_ls.config 2021-07-21 17:51:49 +08:00
William Yang 7521d09079 feat(quic): toggle QUIC listener and fix ci test 2021-07-21 17:51:49 +08:00
Shawn f8f7493352 fix(test): set RETRY_INTERVAL and MAX_TOPIC_ALIAS for paho test 2021-07-17 17:36:01 +08:00
Shawn e3cfc922b2 fix(docker): set console log to debug when testing 2021-07-17 15:49:50 +08:00
Shawn c9cf8b66e7 fix(docker): cannot set log_level using os env 2021-07-17 15:25:18 +08:00
Shawn 72d3b2db1b fix(docker): update the env names in scripts 2021-07-17 14:25:07 +08:00
Shawn 4d279e47d8 chore(merge): merge code from master 2021-07-17 01:20:57 +08:00
zhanghongtong 69f06b3631 chore(CI): add time sleep for relup test 2021-07-16 23:17:51 +08:00
zhanghongtong 9239d3a840 chore(CI): update emqx cluster docker compose file 2021-07-16 23:17:51 +08:00
Shawn 7d66760c1e fix(hocon): start emqx failed using os env configs 2021-07-15 11:55:31 +08:00
DDDHuang d04e1c6f54
refactor: clients api; status api; adapter minirest v1
The serious influence:
authn:
api
authz:
api; api test suit
dashboard:
all closed
lwm2m:
api;
modules:
api(api_topic_metrics, modules_api); test suit(emqx_modules_SUITE)
prometheus:
api
retainer:
api; api test suit
rule_engine:
api: api test suit
telemetry:
api
2021-07-12 10:05:51 +08:00
DDDHuang c11a8c6db6 refactor: clients api; status api; adapter minirest v1
The serious influence:

authn:
    api
authz:
    api; api test suit
dashboard:
    all closed
lwm2m:
    api;
modules:
    api(api_topic_metrics, modules_api); test suit(emqx_modules_SUITE)
prometheus:
    api
retainer:
    api; api test suit
rule_engine:
    api: api test suit
telemetry:
    api
2021-07-09 18:09:07 +08:00
William Yang 793aa951e3
Merge pull request #5170 from qzhuyan/dev/william/ci-fix-build-package
ci(quic): conditionally quicer build
2021-07-08 17:34:47 +02:00
William Yang 2d2be45c0d ci(build-package): fix env setting before start emqx 2021-07-07 13:41:47 +02:00
William Yang 1b014b492f ci(build-package): disable quicer listener for deb test 2021-07-07 13:41:47 +02:00
William Yang 5fcdaa6afe ci(build-packages): fix test zip 2021-07-07 13:41:47 +02:00
William Yang 9516e5047a ci(build-packages): start emqx as user emqx 2021-07-07 13:41:47 +02:00
William Yang 16eb5da440 ci: disable quic listener on arm 2021-07-07 13:41:47 +02:00
William Yang 7a3330856d ci(build-packages): Don't start quic on ARM platform 2021-07-07 13:41:47 +02:00
Rory Z 53df218e6a feat(connector): mongo support replica set 2021-07-06 18:07:06 +08:00
Rory Z fd0a211629 chore(authz): mongo connector support ssl 2021-07-05 17:57:03 +08:00
Rory Z 694f3bd67f feat(authz): support mongo single 2021-07-05 17:57:03 +08:00
Turtle 918a26e921 feat(conf): merge all conf to emqx.conf 2021-07-02 17:01:16 +08:00
zhanghongtong 2b082f9cf9 chore(connector): update connector ssl schema 2021-06-29 18:22:44 +08:00
Zaiming Shi c7fe49c200 test(ci): add plugin list status check after relup new vsn install 2021-06-28 11:42:12 +08:00