Commit Graph

15300 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 0587e0d3f3 refactor(kafka_consumer): change the generated consumer group id
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 41b8d47696 test(kafka_consumer): add more clusterized tests 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 947e014132 test: improve cluster node helpers
* Add option to start autocluster.  This is useful for scenarios where
a cluster is already running and has some configurations set (via
config handler/cluster rpc) and later another node joins.

* Improve mnesia data directory isolation for nodes.  A new dir is set
so that we may avoid intra and inter-suite flakiness due to dirty
tables and schema.

* The janitor is now called synchronously.  This ensures the cleanup
is done before the test pid dies.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi d464e2aad5 refactor: rename test resource prefix
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 46c3305ad8 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 19f9c35662 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi fc5dfa108a fix(kafka_consumer): rename `force_utf8` to `none`
We actually enforce the key/value to be a valid UTF-8 string when
using `emqx_json:encode`, if we do encode using that, which is
template-dependent.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 37a35b22df docs(kafka): improve kafka consumer/producer API examples 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 66eb4ef069 test: fix inter-suite flakiness 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 53979b6261 feat(kafka_consumer): support multiple topic mappings, payload
templates and key/value encodings

Added after feedback from the product team.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi f31f15e44e chore(kafka_producer): make schema changes more backwards compatible
This will still require fixes to the frontend.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 8b1fa50413 ci(fix): fix shellsheck warning 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi e4f058ce2a ci(fix): create user inside container so `emqx console` works 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 9d6af17f69 ci(fix): create and give permissions to mix directories 2023-03-16 13:43:01 -03:00
Ivan Dyachkov a59827cc6a chore(run.sh): prefer docker compose plugin over docker-compose 2023-03-16 13:43:01 -03:00
Ivan Dyachkov 2fe341d152 chore(run.sh): fix permissions on secrets directory 2023-03-16 13:43:01 -03:00
Ivan Dyachkov 5eaaa83b82 chore: simplify run.sh
- get rid of sudo
- do not change permissions of existing files
- use our own docker image to generate certs to make it working on arm
- bump kafka docker image version to have access to multiplatofrm one
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi c182a4053e fix(kafka_consumer): avoid leaking atoms in bridge probe API 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 03342923b9 fix(bridge): use the same dry run prefix
Kafka Producer and Consumer bridges rely on this prefix for detecting
a dry run and avoid leaking atoms.  At some point, this prefix was
changed, effectively disabling the check in Kafka Producer.
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 27dfd98f46 fix(kafka): fix api methods schemas and bridge type enum 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi fecc08bb8f refactor(kafka_consumer): apply review suggestions 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 2a3aef92bb refactor(kafka_consumer): redact error info 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 9037314aea docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1b78f22d92 docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1f31a87974 fix(bridge): improve macro var usage
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi e1fdd041b3 feat(kafka_consumer): add `offset_commit_interval_seconds` kafka parameter 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1d5fe14a30 test: remove sleeps 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 65c15b3fae refactor(kafka_consumer): move subscriber startup logic to separate fn 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 969fa03ecc feat: implement kafka consumer 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 094e4a2eeb chore: fix typespec 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi 1051df7af8 test: add kafka to toxiproxy 2023-03-16 13:43:01 -03:00
Thales Macedo Garitezi b9e92173cf test: improve cluster helper
1) Make each node have its own isolated data dir to avoid false
negatives.
2) Allow parameterizing the peer module.
3) Fix cluster RPC after a node joins the cluster.
2023-03-16 13:42:59 -03:00
Thales Macedo Garitezi 561c25f0e3 feat: update snabbkaffe -> 1.0.7 2023-03-16 13:42:38 -03:00
Thales Macedo Garitezi 5ab5236ad3 test: fix flaky test 2023-03-16 13:42:38 -03:00
Thales Macedo Garitezi 51a0b93868 test: ensure configs are up to date before running suites 2023-03-16 13:42:38 -03:00
Thales Macedo Garitezi 91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII
feat: add ocsp stapling support to mqtt ssl listener (5.0)
2023-03-16 13:10:48 -03:00
Ivan Dyachkov ef8f764154
Merge pull request #10150 from id/ci-do-not-post-enterprise-releases-to-emqx.io
ci: do not post updates about enterprise releases to emqx.io
2023-03-16 13:43:40 +00:00
JimMoen 1688cb7df2
Merge pull request #10152 from JimMoen/refine-changes
chore: refine changes for merged PRs
2023-03-16 21:11:52 +08:00
Thales Macedo Garitezi 75dad647a9
docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 09:57:01 -03:00
Thales Macedo Garitezi a614bdc94a chore(ocsp): catch unexpected error when fetching ocsp response 2023-03-16 09:55:52 -03:00
Thales Macedo Garitezi d1f58d6e2d refactor: replace macro by simple function 2023-03-16 09:32:40 -03:00
Thales Macedo Garitezi e5645a7b21 refactor: rename macros 2023-03-16 09:18:16 -03:00
Kjell Winblad 884dddec79
Merge pull request #10136 from kjellwinblad/kjell/fix/default_value_ui
fix: schema default value so that it shows up correctly in UI
2023-03-16 12:20:08 +01:00
Ivan Dyachkov 17b52ee32a
Merge pull request #10149 from id/chore-use-default-pr-template
chore: use single default pr template
2023-03-16 10:27:06 +00:00
JimMoen afd29b69c4
chore: refine i18n doc punctuation 2023-03-16 18:22:29 +08:00
JimMoen d147299e23
chore: refine changes for merged PRs 2023-03-16 18:22:29 +08:00
Ivan Dyachkov cfae0baf03 ci: do not post updates about enterprise releases to emqx.io 2023-03-16 09:32:13 +01:00
Ivan Dyachkov abbe5be3eb chore: use single default pr template 2023-03-16 09:24:50 +01:00
Ivan Dyachkov e3d4deb8a0
Merge pull request #10147 from id/chore-minor-improvements
Chore minor improvements
2023-03-16 08:18:43 +00:00
Kinplemelon 651f72eabc docs: correct the chinese label for the `socket_nodelay` field 2023-03-16 11:39:29 +08:00