Commit Graph

16280 Commits

Author SHA1 Message Date
Ilya Averyanov bc0a15afd7 feat(ft): fix test for CI 2023-04-07 17:25:21 +03:00
Ilya Averyanov 16fa55e19a feat(ft): add API tests 2023-04-07 17:25:21 +03:00
Andrew Mayorov 6d9f780313 chore: remove unused code 2023-04-07 17:25:21 +03:00
Andrew Mayorov f896fefa59 feat(ft): make storage backend fully async-aware
Introduce an ad-hoc concept of tasks that need to be kicked off
manually. Rework filesystem backend to accomodate for this change.
Adapt responder logic for that "kickoff" protocol.
2023-04-07 17:25:21 +03:00
Andrew Mayorov 2cdf486bf4 feat(ft): simplify responder mechanism
Make responder short-lived process responsible for a single task, and
manage them with supervisor + gproc.
2023-04-07 17:25:21 +03:00
Ilya Averyanov f6a0598f27 feat(ft): add file transfer tests 2023-04-07 17:25:21 +03:00
Ilya Averyanov 2e889f4ac7 feat(ft): add emqx_ft tests and fixes 2023-04-07 17:25:21 +03:00
Ilya Averyanov 1d48a97fd2 feat(ft): fix remote reader handling of gen_rpc errors 2023-04-07 17:25:21 +03:00
Ilya Averyanov c44fe92ef1 feat(ft): add assembler tests 2023-04-07 17:25:21 +03:00
Ilya Averyanov eae3dc7b4b feat(ft): update BPAPI 2023-04-07 17:25:21 +03:00
Ilya Averyanov bcfa22f343 fix(ft): use correct supervison strategy for emqx_ft_sup 2023-04-07 17:25:21 +03:00
Ilya Averyanov 9e4a37a398 fix(ft): fix typespecs 2023-04-07 17:25:21 +03:00
Ilya Averyanov 836ec213c9 feat(ft): add responder tests 2023-04-07 17:25:21 +03:00
Ilya Averyanov 8038a3fd4a feat(ft): add tests for remote reader 2023-04-07 17:25:21 +03:00
Ilya Averyanov b7d0bad970 feat(ft): improve remote reader 2023-04-07 17:25:21 +03:00
Ilya Averyanov 0aefd4a8c7 feat(ft): add streaming of file content when downloading 2023-04-07 17:25:21 +03:00
Ilya Averyanov 197ce32669 feat(ft): add proxy module for emqx_ft_storage_fs 2023-04-07 17:25:21 +03:00
Ilya Averyanov 04e5378bda feat(ft): add API 2023-04-07 17:25:21 +03:00
Andrew Mayorov 92670bfe3d feat(ft): add fs storage bpapi and use it in assembler 2023-04-07 17:25:21 +03:00
Andrew Mayorov 7ed06b0a2a feat(ft-fs): allow to list all transfers in storage
This is rather simplistic and thus, temporary solution.
2023-04-07 17:25:21 +03:00
Andrew Mayorov 429eeaf029 feat(ft-fs): make `list` / `read` more generic
And usable in wider contexts as a consequence, for example querying and
fetching resulting files from remote nodes.
2023-04-07 17:25:21 +03:00
Andrew Mayorov 1308fa0e6b fix(ft-fs): put fragments into separate directories
In order to avoid potential filename collisions.
2023-04-07 17:25:21 +03:00
Andrew Mayorov 8298236908 refactor(ft): bring back userdata to filemeta schema 2023-04-07 17:25:21 +03:00
Ilya Averyanov b4a42a447c feat(ft): removed replicated data 2023-04-07 17:25:21 +03:00
Ilya Averyanov 72e3eee6c9 feat(ft): add config & backend behaviour 2023-04-07 17:25:21 +03:00
Andrew Mayorov 97b831a160 fix(ft-fs): add missing `read_segment/5` + fix atomic write 2023-04-07 17:25:21 +03:00
Andrew Mayorov 14b2a1013b fix(ft-asm): follow proper `segment` fragment type 2023-04-07 17:25:21 +03:00
Andrew Mayorov 1fedae8a16 fix(ft-asm): ensure module follows statem behaviour 2023-04-07 17:25:21 +03:00
Andrew Mayorov 7b77e96ab9 test(ft): add some basic assembler tests 2023-04-07 17:25:21 +03:00
Ilya Averyanov cbff2e2309 feat(ft): improve robustness of asynchronous acks
* add auto ack after timeout
* add fin file transfer packet registration to avoid
duplication and multiple acks
2023-04-07 17:25:21 +03:00
Ilya Averyanov d36ca18bff feat(ft): tie file transfer frontend and backend together 2023-04-07 17:25:21 +03:00
Andrew Mayorov 81e04ce93a feat(ft): introduce simple filesystem storage backend + assembler 2023-04-07 17:25:21 +03:00
Ilya Averyanov aaaef30be6 feat(ft): add file transfer app and bootstrap replicated ft data structure 2023-04-07 17:25:21 +03:00
Ilya Averyanov d976943f99 feat(ft): add basic hooks 2023-04-07 17:25:21 +03:00
Kinplemelon ffa8f21d2e chore: upgrade dashboard to v1.2.1 for ce 2023-04-07 18:02:04 +08:00
JianBo He 502cc2b8b8 chore: fix common tests 2023-04-07 15:34:16 +08:00
Thales Macedo Garitezi 698951e90a
Merge pull request #10346 from thalesmg/release-50
chore: bump version to e5.0.2-rc.6
2023-04-06 18:19:39 -03:00
Thales Macedo Garitezi 7c05304ff4 chore: bump version to e5.0.2-rc.6 2023-04-06 16:53:55 -03:00
Thales Macedo Garitezi 67ff058d46
Merge pull request #10345 from thalesmg/fix-kafka-async-return-action-rv50
fix(rule_action): fix metrics for bridges returning `async_return`
2023-04-06 16:52:32 -03:00
Thales Macedo Garitezi 33100ecca6 feat: implement schema registry for 5.0 (avro)
Part of https://emqx.atlassian.net/browse/EMQX-9251

This ports part of the Schema Registry app from 4.x to 5.0.  Here,
only support for Avro is added.  Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
Thales Macedo Garitezi 4c24b08244 fix(rule_action): fix metrics for bridges returning `async_return`
Kafka Producer, when called asynchronously, will return
`{async_return, {ok, pid()}}`, which currently counts as an unknown failure.
2023-04-06 16:00:01 -03:00
Zaiming (Stone) Shi cf71f9148e
Merge pull request #10287 from thalesmg/fix-kprod-zh-desc-mqtt-topic-rv50
docs(kafka_producer): update ZH translation of field (rv5.0)
2023-04-06 16:13:41 +02:00
Zaiming (Stone) Shi 11a7770d16 chore: bump version to e5.0.2-rc.5 2023-04-06 11:57:39 +02:00
JimMoen f71e576bdd
Merge pull request #10341 from JimMoen/fix-undef-fun
fix: make emqx_api_lib compatible
2023-04-06 17:47:40 +08:00
JimMoen bbb3fdb49e
fix: make emqx_api_lib compatible 2023-04-06 17:03:39 +08:00
Kjell Winblad 1938882f16
Merge pull request #10318 from kjellwinblad/kjell/feat/rule_engine_from_clause_support_both_string_types
feat(rule engine sql): enable both ' and " strings in FROM clause
2023-04-06 09:19:44 +02:00
JimMoen ab807fba51
Merge pull request #10329 from kjellwinblad/kjell/EMQX-9138-e-5-0-2-ensure-that-the-error-messages-returned-are-consistent
fix: create consistent interface 'with_node' for API access
2023-04-06 14:47:42 +08:00
JianBo He 702ecbcb6e Merge remote-tracking branch 'upstream/master' into refactor-gw-dir 2023-04-06 10:44:59 +08:00
SergeTupchiy 0a73866d27
Merge pull request #10338 from SergeTupchiy/refactor-rule-engine-runtime-ok-result
refactor(rule_engine): test 'ok' results in one dedicated function
2023-04-05 22:40:40 +03:00
Stefan Strigler b799af1f71 fix: don't create virtual type "rule_engine_api" 2023-04-05 16:46:19 +02:00