Commit Graph

14429 Commits

Author SHA1 Message Date
Ilya Averyanov
92ff2b7d6e feat(ft): make timeouts configurable 2023-04-14 01:58:14 +03:00
Ilya Averyanov
e22c1c01ec feat(s3): make acl optional 2023-04-13 19:47:12 +03:00
Ilya Averyanov
a99eb082d1
Merge pull request #10350 from keynslug/fix/EMQX-9486/sigv4-s3-urls
fix(ft-s3): use AWS4 signed urls for S3 export URIs
2023-04-10 23:51:04 +03:00
Andrew Mayorov
37a520d797
fix(i18n): relocate i18n files to rel/i18n 2023-04-10 16:02:34 +03:00
Andrew Mayorov
9224b339be
fix(dep): upgrade rulesql back to 0.1.5 2023-04-07 22:20:20 +03:00
Andrew Mayorov
8daa38ef06
feat(ft-s3): store metadata in ASCII-safe format
Also ensure consistent encoding and decoding filenames throughout
the `emqx_ft` application.
2023-04-07 22:20:20 +03:00
Andrew Mayorov
0d86ef8d3a
fix(ft-s3): use AWS4 signed urls for S3 export URIs
This will ensure that the S3 export URIs are valid in all AWS regions.
2023-04-07 22:15:01 +03:00
Ilya Averyanov
b2fd2dcbc1 fix(ft): make configs user friendlier 2023-04-07 17:36:47 +03:00
Ilya Averyanov
7eeba32619 fix(s3): fix typings 2023-04-07 17:36:47 +03:00
Ilya Averyanov
6fbc7dc822 fix(ft): remove s3 roots from ee schema 2023-04-07 17:36:47 +03:00
Ilya Averyanov
d21daae7ae fix(ft): fix dynamic function calls 2023-04-07 17:36:47 +03:00
Ilya Averyanov
52f3189779 fix(ft): fix default ft config 2023-04-07 17:36:47 +03:00
Ilya Averyanov
eae2478678 fix(ft): add descriptions for missing ft schema fields 2023-04-07 17:36:47 +03:00
Ilya Averyanov
bd7250cb13 fix(s3): fix hash pool type 2023-04-07 17:36:47 +03:00
Ilya Averyanov
4a144044b7 fix(ft): set default ft config in tests 2023-04-07 17:36:47 +03:00
Ilya Averyanov
918bdcae7d fix(ft): fix schema descriptions 2023-04-07 17:36:47 +03:00
Ilya Averyanov
8b13d249eb fix(ft): fix detection of ee apps in mix 2023-04-07 17:36:47 +03:00
Ilya Averyanov
7bcb60a84a fix(ft): fix config update tests 2023-04-07 17:36:47 +03:00
Ilya Averyanov
6e50b168a0 fix(ft): add empty strings for zh translations 2023-04-07 17:36:47 +03:00
Ilya Averyanov
f6461fe287 fix(ft): fix typings 2023-04-07 17:36:47 +03:00
Ilya Averyanov
2f1970adbc fix(ft): make ee schema aware of ee fields 2023-04-07 17:36:47 +03:00
Ilya Averyanov
18aa39f892 fix(ft): add emqx_ft_schema to ee schemas 2023-04-07 17:36:47 +03:00
Ilya Averyanov
d333187c5a docs(s3): update readme 2023-04-07 17:36:47 +03:00
Ilya Averyanov
820e06d756 fix(ft-s3): make controllable use of max_retries and request_timeout arguments 2023-04-07 17:36:47 +03:00
Ilya Averyanov
be99242e32 fix(ft-s3): fix review comments 2023-04-07 17:36:47 +03:00
Ilya Averyanov
d7a85242de fix(ft-s3): fix s3 listing 2023-04-07 17:36:47 +03:00
Ilya Averyanov
7d13862da5 fix(ft-s3): use uploader specific headers only for object creation 2023-04-07 17:36:47 +03:00
Ilya Averyanov
c7865e5eae fix(ft-s3): fix dependency dependencies 2023-04-07 17:36:45 +03:00
Ilya Averyanov
4fbabe5a76 fix(ft-s3): fix atom and variable naming 2023-04-07 17:27:01 +03:00
Ilya Averyanov
71965d90e4 fix(ft-s3): synchronize erlcloud version across apps 2023-04-07 17:26:53 +03:00
Ilya Averyanov
07ae50a54b fix(ft-s3): fix release settings 2023-04-07 17:25:23 +03:00
Ilya Averyanov
3ffa01e160 feat(ft-s3): fix logging level 2023-04-07 17:25:23 +03:00
Ilya Averyanov
ca22f11161 feat(ft-s3): fix logging 2023-04-07 17:25:23 +03:00
Ilya Averyanov
9bb72ee020 feat(ft-s3): fix behaviour specification 2023-04-07 17:25:23 +03:00
Ilya Averyanov
43f9737420 feat(ft-s3): integrate list API 2023-04-07 17:25:23 +03:00
Ilya Averyanov
5ac3543a76 feat(ft-s3): integrate exporter configs 2023-04-07 17:25:22 +03:00
Ilya Averyanov
8361223648 feat(ft-s3): extract checksum verification 2023-04-07 17:25:22 +03:00
Ilya Averyanov
818a5cacf2 feat(ft-s3): add initial integration 2023-04-07 17:25:22 +03:00
Ilya Averyanov
31b441a46e feat(s3): add S3 client application 2023-04-07 17:25:22 +03:00
Andrew Mayorov
28d87ca62d fix(ft-fs): move default config to the backend impl 2023-04-07 17:25:22 +03:00
Andrew Mayorov
258fabbf8b fix(fs-exp): reply with error when listing failed everywhere 2023-04-07 17:25:22 +03:00
Andrew Mayorov
11edfc1c6a fix(ft): comment out some clauses as "unreachable" 2023-04-07 17:25:22 +03:00
Andrew Mayorov
bef5cc9c0f fix(fs-fold): avoid folding through symlinked directories
Also a testsuite that verifies multilevel fold behaviour.
2023-04-07 17:25:22 +03:00
Andrew Mayorov
6ad7ce55d2 fix(ft-conf): separate local segments storage settings
To make things less ambiguous.
2023-04-07 17:25:22 +03:00
Andrew Mayorov
45e3b62dc4 refactor(ft): prefer plain files over exports in generic APIs
So there's no more `exports` magically becomings `files` in the
REST API which was slightly confusing.
2023-04-07 17:25:22 +03:00
Andrew Mayorov
c24c7eca34 fix(ft-test): unbreak testcase by inhibiting local fs storage GC 2023-04-07 17:25:22 +03:00
Andrew Mayorov
23cd78b8d6 refactor(ft-asm): turn state data record into a map
Which should be more future-proof.
2023-04-07 17:25:22 +03:00
Andrew Mayorov
b6b044f429 feat(ft): move out exporter concept into dedicated modules 2023-04-07 17:25:22 +03:00
Andrew Mayorov
54e54cc63d feat(api): provide utility to qualify API URIs 2023-04-07 17:25:22 +03:00
Andrew Mayorov
64f15f1fdb fix(ft-gc): ensure directories of complete transfers are GCed 2023-04-07 17:25:22 +03:00