Commit Graph

13958 Commits

Author SHA1 Message Date
Ilya Averyanov 39f22585d0 chore(mix): fix mix builds 2023-03-19 01:39:56 +02:00
Zaiming (Stone) Shi d5ea435e86
Merge pull request #10158 from Kinplemelon/kinple/for-upgrade-dashboard
chore: upgrade dashboard to e1.0.5-beta.1 for ee
2023-03-17 13:44:32 +01:00
Thales Macedo Garitezi cf1cce011b
Merge pull request #9564 from thalesmg/kafka-source-ee50
feat: implement kafka consumer (ee5.0)
2023-03-17 09:04:53 -03:00
Zaiming (Stone) Shi 1bf72c4d2e
Merge pull request #10155 from zmstone/0316-build-schema-dump-per-lang
build: generate per-lang schema dump
2023-03-17 11:45:35 +01:00
zhongwencool 6d495223d0
Merge pull request #10157 from zhongwencool/fix-listener-create-without-limiter
fix: newly created listeners have no limiter restrictions
2023-03-17 17:25:08 +08:00
Kinplemelon c47c9feff1 chore: upgrade dashboard to e1.0.5-beta.1 for ee 2023-03-17 16:23:47 +08:00
Zhongwen Deng 3c1254d873 fix: newly created listeners have no limiter restrictions 2023-03-17 14:32:42 +08:00
Zaiming (Stone) Shi ad1deedd0e build: generate per-lang schema dump 2023-03-16 21:25:21 +01:00
Thales Macedo Garitezi 966276127e test: trying to make tests more stable 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 3954b7bde2 fix(bridges): function clause when a non-ingress bridge coexists with an egress bridge
This was not caught by our tests because we always test bridge types
in isolation.  So, if the config only contains ingress-only bridges,
the `on_message_publish` hook is never installed.

In a real system, if there are bridges of mixed types in the config,
the hook might be installed, and `emqx_bridge:get_matched_bridge_id`
would crash when iterating over the ingress bridges.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 0587e0d3f3 refactor(kafka_consumer): change the generated consumer group id
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 41b8d47696 test(kafka_consumer): add more clusterized tests 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 947e014132 test: improve cluster node helpers
* Add option to start autocluster.  This is useful for scenarios where
a cluster is already running and has some configurations set (via
config handler/cluster rpc) and later another node joins.

* Improve mnesia data directory isolation for nodes.  A new dir is set
so that we may avoid intra and inter-suite flakiness due to dirty
tables and schema.

* The janitor is now called synchronously.  This ensures the cleanup
is done before the test pid dies.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi d464e2aad5 refactor: rename test resource prefix
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 46c3305ad8 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 19f9c35662 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi fc5dfa108a fix(kafka_consumer): rename `force_utf8` to `none`
We actually enforce the key/value to be a valid UTF-8 string when
using `emqx_json:encode`, if we do encode using that, which is
template-dependent.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 37a35b22df docs(kafka): improve kafka consumer/producer API examples 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 66eb4ef069 test: fix inter-suite flakiness 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 53979b6261 feat(kafka_consumer): support multiple topic mappings, payload
templates and key/value encodings

Added after feedback from the product team.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi f31f15e44e chore(kafka_producer): make schema changes more backwards compatible
This will still require fixes to the frontend.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 8b1fa50413 ci(fix): fix shellsheck warning 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi e4f058ce2a ci(fix): create user inside container so `emqx console` works 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 9d6af17f69 ci(fix): create and give permissions to mix directories 2023-03-16 13:43:01 -03:00
Ivan Dyachkov a59827cc6a chore(run.sh): prefer docker compose plugin over docker-compose 2023-03-16 13:43:01 -03:00
Ivan Dyachkov 2fe341d152 chore(run.sh): fix permissions on secrets directory 2023-03-16 13:43:01 -03:00
Ivan Dyachkov 5eaaa83b82 chore: simplify run.sh
- get rid of sudo
- do not change permissions of existing files
- use our own docker image to generate certs to make it working on arm
- bump kafka docker image version to have access to multiplatofrm one
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi c182a4053e fix(kafka_consumer): avoid leaking atoms in bridge probe API 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 03342923b9 fix(bridge): use the same dry run prefix
Kafka Producer and Consumer bridges rely on this prefix for detecting
a dry run and avoid leaking atoms.  At some point, this prefix was
changed, effectively disabling the check in Kafka Producer.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 27dfd98f46 fix(kafka): fix api methods schemas and bridge type enum 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi fecc08bb8f refactor(kafka_consumer): apply review suggestions 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 2a3aef92bb refactor(kafka_consumer): redact error info 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 9037314aea docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1b78f22d92 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1f31a87974 fix(bridge): improve macro var usage
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi e1fdd041b3 feat(kafka_consumer): add `offset_commit_interval_seconds` kafka parameter 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1d5fe14a30 test: remove sleeps 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 65c15b3fae refactor(kafka_consumer): move subscriber startup logic to separate fn 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 969fa03ecc feat: implement kafka consumer 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 094e4a2eeb chore: fix typespec 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1051df7af8 test: add kafka to toxiproxy 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi b9e92173cf test: improve cluster helper
1) Make each node have its own isolated data dir to avoid false
negatives.
2) Allow parameterizing the peer module.
3) Fix cluster RPC after a node joins the cluster.
2023-03-16 13:42:59 -03:00
Thales Macedo Garitezi 561c25f0e3 feat: update snabbkaffe -> 1.0.7 2023-03-16 13:42:38 -03:00
Thales Macedo Garitezi 5ab5236ad3 test: fix flaky test 2023-03-16 13:42:38 -03:00
Thales Macedo Garitezi 51a0b93868 test: ensure configs are up to date before running suites 2023-03-16 13:42:38 -03:00
Thales Macedo Garitezi 91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII
feat: add ocsp stapling support to mqtt ssl listener (5.0)
2023-03-16 13:10:48 -03:00
Ivan Dyachkov ef8f764154
Merge pull request #10150 from id/ci-do-not-post-enterprise-releases-to-emqx.io
ci: do not post updates about enterprise releases to emqx.io
2023-03-16 13:43:40 +00:00
JimMoen 1688cb7df2
Merge pull request #10152 from JimMoen/refine-changes
chore: refine changes for merged PRs
2023-03-16 21:11:52 +08:00
Thales Macedo Garitezi 75dad647a9
docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 09:57:01 -03:00
Thales Macedo Garitezi a614bdc94a chore(ocsp): catch unexpected error when fetching ocsp response 2023-03-16 09:55:52 -03:00