yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,300 Commits 393 Branches 684 Tags 74 MiB
Commit Graph

121 Commits

Author SHA1 Message Date
Andrew Mayorov c95ef71fb5
chore(ft): provide more details in README.md 2023-05-17 12:49:20 +03:00
Andrew Mayorov e3b822c1a0
chore: remove empty header 
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
 2023-05-16 16:45:41 +03:00
Andrew Mayorov a7595ff468
chore: fixup copyright years 2023-05-16 16:45:07 +03:00
Andrew Mayorov 6b688d6646
fix(ft): anticipate repeated `kickoff`s + fix testcase 2023-05-10 17:35:53 +03:00
Ilya Averyanov 42f5433aaf
Merge pull request #10655 from savonarola/0510-idempotent-fin 
Make FT fin command idempotent
 2023-05-10 11:39:43 +05:00
Ilya Averyanov 244188982b fix(ft): add missing translations 2023-05-09 23:32:17 +05:00
Ilya Averyanov 45875bfee5 fix(ft): update s3 exporter tests to the new config 2023-05-09 23:32:17 +05:00
Ilya Averyanov 9381c895bb fix(ft): fix dialyzer issues 2023-05-09 23:32:16 +05:00
Andrew Mayorov 079b8e9476 fix(ft): silence warnings when some root is not yet `mkdir`ed 2023-05-09 23:32:16 +05:00
Andrew Mayorov 5e5f854ce1 feat(ft-conf): simplify schema of storage / exporter backends 
Assumption is this changes will make `emqx_ft` config schema
user-friendlier and also more future-proof.
 2023-05-09 23:32:16 +05:00
Ilya Averyanov 8ac881a140 chore(ft): handle multiple/concurrent fins more gracefully 2023-05-09 23:31:39 +05:00
Ilya Averyanov cff7788b2e chore(ft): add s3 exporter tests 2023-05-05 11:55:09 +03:00
Andrew Mayorov bd9f129bb8
test(ft-api): also run testcases under cluster setup 2023-04-28 13:50:23 +03:00
Andrew Mayorov a9866fede4
feat(ft-api): support paging in S3 storage exporter 2023-04-28 13:49:17 +03:00
Andrew Mayorov 75cceffa06
fix(ft-test): rename testcases for consistency 2023-04-28 13:49:17 +03:00
Andrew Mayorov ed3756ea09
feat(ft-api): add paging support through cursors 2023-04-28 13:49:15 +03:00
Andrew Mayorov 573bb22ada
feat(ft-fs): introduce fs iterators concept + forward seeks 
In order to support paging over filesystem contents, to serve REST
APIs effectively.
 2023-04-28 13:47:50 +03:00
Andrew Mayorov 811e449357
feat(ft-conf): provide global killswitch 2023-04-28 10:54:21 +03:00
Andrew Mayorov a420c92d28
fix(ft): fix typing issue 2023-04-24 21:51:54 +03:00
Andrew Mayorov 5efd590ca4
feat(ft): properly propagate config updates 
Ensure that:
* Storage config might be removed.
* Local FS GC process is set up when Local FS storage is configured.
* Local FS GC process gets its timer reset on config updates.
* Storage / exporter gets chosen based on `type` only.
* Exporter config updates propagated as before.

Also employ `emqx_ft_schema:translate/1` instead of duplicating
defaults where applicable.
 2023-04-24 21:51:54 +03:00
Ilya Averyanov 69c4ba2a62 feat(ft): use new utils application 2023-04-22 22:50:19 +03:00
Andrew Mayorov 04523b3f81
fix(ft): restrict max filename length in transfers 
Reject transfers with too long filenames right away, during `init`
handling, to avoid having to deal with filesystem errors later.
 2023-04-19 18:48:40 +03:00
Ilya Averyanov 92ff2b7d6e feat(ft): make timeouts configurable 2023-04-14 01:58:14 +03:00
Andrew Mayorov 37a520d797
fix(i18n): relocate i18n files to rel/i18n 2023-04-10 16:02:34 +03:00
Andrew Mayorov 8daa38ef06
feat(ft-s3): store metadata in ASCII-safe format 
Also ensure consistent encoding and decoding filenames throughout
the `emqx_ft` application.
 2023-04-07 22:20:20 +03:00
Ilya Averyanov b2fd2dcbc1 fix(ft): make configs user friendlier 2023-04-07 17:36:47 +03:00
Ilya Averyanov 7eeba32619 fix(s3): fix typings 2023-04-07 17:36:47 +03:00
Ilya Averyanov 52f3189779 fix(ft): fix default ft config 2023-04-07 17:36:47 +03:00
Ilya Averyanov bd7250cb13 fix(s3): fix hash pool type 2023-04-07 17:36:47 +03:00
Ilya Averyanov 4a144044b7 fix(ft): set default ft config in tests 2023-04-07 17:36:47 +03:00
Ilya Averyanov 918bdcae7d fix(ft): fix schema descriptions 2023-04-07 17:36:47 +03:00
Ilya Averyanov 8b13d249eb fix(ft): fix detection of ee apps in mix 2023-04-07 17:36:47 +03:00
Ilya Averyanov 7bcb60a84a fix(ft): fix config update tests 2023-04-07 17:36:47 +03:00
Ilya Averyanov f6461fe287 fix(ft): fix typings 2023-04-07 17:36:47 +03:00
Ilya Averyanov 2f1970adbc fix(ft): make ee schema aware of ee fields 2023-04-07 17:36:47 +03:00
Ilya Averyanov be99242e32 fix(ft-s3): fix review comments 2023-04-07 17:36:47 +03:00
Ilya Averyanov d7a85242de fix(ft-s3): fix s3 listing 2023-04-07 17:36:47 +03:00
Ilya Averyanov 07ae50a54b fix(ft-s3): fix release settings 2023-04-07 17:25:23 +03:00
Ilya Averyanov 3ffa01e160 feat(ft-s3): fix logging level 2023-04-07 17:25:23 +03:00
Ilya Averyanov ca22f11161 feat(ft-s3): fix logging 2023-04-07 17:25:23 +03:00
Ilya Averyanov 9bb72ee020 feat(ft-s3): fix behaviour specification 2023-04-07 17:25:23 +03:00
Ilya Averyanov 43f9737420 feat(ft-s3): integrate list API 2023-04-07 17:25:23 +03:00
Ilya Averyanov 5ac3543a76 feat(ft-s3): integrate exporter configs 2023-04-07 17:25:22 +03:00
Ilya Averyanov 8361223648 feat(ft-s3): extract checksum verification 2023-04-07 17:25:22 +03:00
Ilya Averyanov 818a5cacf2 feat(ft-s3): add initial integration 2023-04-07 17:25:22 +03:00
Andrew Mayorov 28d87ca62d fix(ft-fs): move default config to the backend impl 2023-04-07 17:25:22 +03:00
Andrew Mayorov 258fabbf8b fix(fs-exp): reply with error when listing failed everywhere 2023-04-07 17:25:22 +03:00
Andrew Mayorov 11edfc1c6a fix(ft): comment out some clauses as "unreachable" 2023-04-07 17:25:22 +03:00
Andrew Mayorov bef5cc9c0f fix(fs-fold): avoid folding through symlinked directories 
Also a testsuite that verifies multilevel fold behaviour.
 2023-04-07 17:25:22 +03:00
Andrew Mayorov 6ad7ce55d2 fix(ft-conf): separate local segments storage settings 
To make things less ambiguous.
 2023-04-07 17:25:22 +03:00