d2ce7c22fd
chore: bump app vsn, core app 'emqx' is now at 5.1
2023-05-30 11:32:18 +02:00
8569fac6eb
fix: make static_check happy
2023-05-30 14:00:20 +08:00
909d348570
chore: fix spec typo
2023-05-30 11:31:37 +08:00
daa7eebc0f
Merge pull request #10843 from keynslug/fix/ft-prop-zerodiv
...
test(ft): ensure no division by zero in coverage proptest
2023-05-29 10:23:52 +03:00
98ce7e8bbd
test(ft): ensure no division by zero in coverage proptest
2023-05-26 21:53:01 +03:00
36e268c933
chore: bump app versions
2023-05-26 16:05:37 +02:00
3d428a2e0e
fix(ft-s3): rely on asynchronous upload abort
...
So that upload aborts would not block assemblers. This should not
affect the expected behavior since S3 API usually allows having
concurrent uploads, plus we couldn't really guarantee successful aborts
anyway.
2023-05-26 12:59:20 +03:00
cc3275b389
fix(ft): use conservative timeouts for S3 uploader calls
2023-05-26 12:59:20 +03:00
d36c2c2928
fix(ft-gc): ensure GC of already complete transfers
2023-05-26 12:59:20 +03:00
f4047d3946
test(ft): add testcase for nasty tranfer filenames
...
* Emoji
* Chinese characters
2023-05-23 20:46:47 +03:00
2dbf84479c
fix(ft): handle wider class of jiffy decode errors
...
With malformed cursors in the File listing API.
Fixes EMQX-9965
2023-05-23 20:00:00 +03:00
cb14a3e08b
fix(ft): handle empty filepath in fs exporter API
...
Fixes EMQX-9973
2023-05-23 19:58:05 +03:00
d22541e8b3
fix(ft): correct mistyped option in README
2023-05-22 15:32:08 +03:00
966f748551
test(ft): make proptest less aggressive
...
So that the chance of getting huge coverage and as a result, high
chance of running out of memory, is reduced.
2023-05-22 10:45:27 +03:00
c95ef71fb5
chore(ft): provide more details in README.md
2023-05-17 12:49:20 +03:00
e3b822c1a0
chore: remove empty header
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-05-16 16:45:41 +03:00
a7595ff468
chore: fixup copyright years
2023-05-16 16:45:07 +03:00
6b688d6646
fix(ft): anticipate repeated `kickoff`s + fix testcase
2023-05-10 17:35:53 +03:00
42f5433aaf
Merge pull request #10655 from savonarola/0510-idempotent-fin
...
Make FT fin command idempotent
2023-05-10 11:39:43 +05:00
244188982b
fix(ft): add missing translations
2023-05-09 23:32:17 +05:00
45875bfee5
fix(ft): update s3 exporter tests to the new config
2023-05-09 23:32:17 +05:00
9381c895bb
fix(ft): fix dialyzer issues
2023-05-09 23:32:16 +05:00
079b8e9476
fix(ft): silence warnings when some root is not yet `mkdir`ed
2023-05-09 23:32:16 +05:00
5e5f854ce1
feat(ft-conf): simplify schema of storage / exporter backends
...
Assumption is this changes will make `emqx_ft` config schema
user-friendlier and also more future-proof.
2023-05-09 23:32:16 +05:00
8ac881a140
chore(ft): handle multiple/concurrent fins more gracefully
2023-05-09 23:31:39 +05:00
cff7788b2e
chore(ft): add s3 exporter tests
2023-05-05 11:55:09 +03:00
bd9f129bb8
test(ft-api): also run testcases under cluster setup
2023-04-28 13:50:23 +03:00
a9866fede4
feat(ft-api): support paging in S3 storage exporter
2023-04-28 13:49:17 +03:00
75cceffa06
fix(ft-test): rename testcases for consistency
2023-04-28 13:49:17 +03:00
ed3756ea09
feat(ft-api): add paging support through cursors
2023-04-28 13:49:15 +03:00
573bb22ada
feat(ft-fs): introduce fs iterators concept + forward seeks
...
In order to support paging over filesystem contents, to serve REST
APIs effectively.
2023-04-28 13:47:50 +03:00
811e449357
feat(ft-conf): provide global killswitch
2023-04-28 10:54:21 +03:00
a420c92d28
fix(ft): fix typing issue
2023-04-24 21:51:54 +03:00
5efd590ca4
feat(ft): properly propagate config updates
...
Ensure that:
* Storage config might be removed.
* Local FS GC process is set up when Local FS storage is configured.
* Local FS GC process gets its timer reset on config updates.
* Storage / exporter gets chosen based on `type` only.
* Exporter config updates propagated as before.
Also employ `emqx_ft_schema:translate/1` instead of duplicating
defaults where applicable.
2023-04-24 21:51:54 +03:00
69c4ba2a62
feat(ft): use new utils application
2023-04-22 22:50:19 +03:00
04523b3f81
fix(ft): restrict max filename length in transfers
...
Reject transfers with too long filenames right away, during `init`
handling, to avoid having to deal with filesystem errors later.
2023-04-19 18:48:40 +03:00
92ff2b7d6e
feat(ft): make timeouts configurable
2023-04-14 01:58:14 +03:00
37a520d797
fix(i18n): relocate i18n files to rel/i18n
2023-04-10 16:02:34 +03:00
8daa38ef06
feat(ft-s3): store metadata in ASCII-safe format
...
Also ensure consistent encoding and decoding filenames throughout
the `emqx_ft` application.
2023-04-07 22:20:20 +03:00
b2fd2dcbc1
fix(ft): make configs user friendlier
2023-04-07 17:36:47 +03:00
7eeba32619
fix(s3): fix typings
2023-04-07 17:36:47 +03:00
52f3189779
fix(ft): fix default ft config
2023-04-07 17:36:47 +03:00
bd7250cb13
fix(s3): fix hash pool type
2023-04-07 17:36:47 +03:00
4a144044b7
fix(ft): set default ft config in tests
2023-04-07 17:36:47 +03:00
918bdcae7d
fix(ft): fix schema descriptions
2023-04-07 17:36:47 +03:00
8b13d249eb
fix(ft): fix detection of ee apps in mix
2023-04-07 17:36:47 +03:00
7bcb60a84a
fix(ft): fix config update tests
2023-04-07 17:36:47 +03:00
f6461fe287
fix(ft): fix typings
2023-04-07 17:36:47 +03:00
2f1970adbc
fix(ft): make ee schema aware of ee fields
2023-04-07 17:36:47 +03:00
be99242e32
fix(ft-s3): fix review comments
2023-04-07 17:36:47 +03:00
Zaiming (Stone) Shi