Zaiming (Stone) Shi
d2ce7c22fd
chore: bump app vsn, core app 'emqx' is now at 5.1
2023-05-30 11:32:18 +02:00
JimMoen
8569fac6eb
fix: make static_check happy
2023-05-30 14:00:20 +08:00
JimMoen
909d348570
chore: fix spec typo
2023-05-30 11:31:37 +08:00
Zaiming (Stone) Shi
36e268c933
chore: bump app versions
2023-05-26 16:05:37 +02:00
Andrew Mayorov
3d428a2e0e
fix(ft-s3): rely on asynchronous upload abort
...
So that upload aborts would not block assemblers. This should not
affect the expected behavior since S3 API usually allows having
concurrent uploads, plus we couldn't really guarantee successful aborts
anyway.
2023-05-26 12:59:20 +03:00
Andrew Mayorov
cc3275b389
fix(ft): use conservative timeouts for S3 uploader calls
2023-05-26 12:59:20 +03:00
Andrew Mayorov
d36c2c2928
fix(ft-gc): ensure GC of already complete transfers
2023-05-26 12:59:20 +03:00
Andrew Mayorov
2dbf84479c
fix(ft): handle wider class of jiffy decode errors
...
With malformed cursors in the File listing API.
Fixes EMQX-9965
2023-05-23 20:00:00 +03:00
Andrew Mayorov
cb14a3e08b
fix(ft): handle empty filepath in fs exporter API
...
Fixes EMQX-9973
2023-05-23 19:58:05 +03:00
Andrew Mayorov
a7595ff468
chore: fixup copyright years
2023-05-16 16:45:07 +03:00
Andrew Mayorov
6b688d6646
fix(ft): anticipate repeated kickoffs + fix testcase
2023-05-10 17:35:53 +03:00
Ilya Averyanov
42f5433aaf
Merge pull request #10655 from savonarola/0510-idempotent-fin
...
Make FT fin command idempotent
2023-05-10 11:39:43 +05:00
Ilya Averyanov
244188982b
fix(ft): add missing translations
2023-05-09 23:32:17 +05:00
Ilya Averyanov
9381c895bb
fix(ft): fix dialyzer issues
2023-05-09 23:32:16 +05:00
Andrew Mayorov
079b8e9476
fix(ft): silence warnings when some root is not yet mkdired
2023-05-09 23:32:16 +05:00
Andrew Mayorov
5e5f854ce1
feat(ft-conf): simplify schema of storage / exporter backends
...
Assumption is this changes will make `emqx_ft` config schema
user-friendlier and also more future-proof.
2023-05-09 23:32:16 +05:00
Ilya Averyanov
8ac881a140
chore(ft): handle multiple/concurrent fins more gracefully
2023-05-09 23:31:39 +05:00
Ilya Averyanov
cff7788b2e
chore(ft): add s3 exporter tests
2023-05-05 11:55:09 +03:00
Andrew Mayorov
a9866fede4
feat(ft-api): support paging in S3 storage exporter
2023-04-28 13:49:17 +03:00
Andrew Mayorov
ed3756ea09
feat(ft-api): add paging support through cursors
2023-04-28 13:49:15 +03:00
Andrew Mayorov
573bb22ada
feat(ft-fs): introduce fs iterators concept + forward seeks
...
In order to support paging over filesystem contents, to serve REST
APIs effectively.
2023-04-28 13:47:50 +03:00
Andrew Mayorov
811e449357
feat(ft-conf): provide global killswitch
2023-04-28 10:54:21 +03:00
Andrew Mayorov
a420c92d28
fix(ft): fix typing issue
2023-04-24 21:51:54 +03:00
Andrew Mayorov
5efd590ca4
feat(ft): properly propagate config updates
...
Ensure that:
* Storage config might be removed.
* Local FS GC process is set up when Local FS storage is configured.
* Local FS GC process gets its timer reset on config updates.
* Storage / exporter gets chosen based on `type` only.
* Exporter config updates propagated as before.
Also employ `emqx_ft_schema:translate/1` instead of duplicating
defaults where applicable.
2023-04-24 21:51:54 +03:00
Ilya Averyanov
69c4ba2a62
feat(ft): use new utils application
2023-04-22 22:50:19 +03:00
Andrew Mayorov
04523b3f81
fix(ft): restrict max filename length in transfers
...
Reject transfers with too long filenames right away, during `init`
handling, to avoid having to deal with filesystem errors later.
2023-04-19 18:48:40 +03:00
Ilya Averyanov
92ff2b7d6e
feat(ft): make timeouts configurable
2023-04-14 01:58:14 +03:00
Andrew Mayorov
8daa38ef06
feat(ft-s3): store metadata in ASCII-safe format
...
Also ensure consistent encoding and decoding filenames throughout
the `emqx_ft` application.
2023-04-07 22:20:20 +03:00
Ilya Averyanov
b2fd2dcbc1
fix(ft): make configs user friendlier
2023-04-07 17:36:47 +03:00
Ilya Averyanov
7eeba32619
fix(s3): fix typings
2023-04-07 17:36:47 +03:00
Ilya Averyanov
bd7250cb13
fix(s3): fix hash pool type
2023-04-07 17:36:47 +03:00
Ilya Averyanov
918bdcae7d
fix(ft): fix schema descriptions
2023-04-07 17:36:47 +03:00
Ilya Averyanov
f6461fe287
fix(ft): fix typings
2023-04-07 17:36:47 +03:00
Ilya Averyanov
2f1970adbc
fix(ft): make ee schema aware of ee fields
2023-04-07 17:36:47 +03:00
Ilya Averyanov
be99242e32
fix(ft-s3): fix review comments
2023-04-07 17:36:47 +03:00
Ilya Averyanov
d7a85242de
fix(ft-s3): fix s3 listing
2023-04-07 17:36:47 +03:00
Ilya Averyanov
07ae50a54b
fix(ft-s3): fix release settings
2023-04-07 17:25:23 +03:00
Ilya Averyanov
3ffa01e160
feat(ft-s3): fix logging level
2023-04-07 17:25:23 +03:00
Ilya Averyanov
ca22f11161
feat(ft-s3): fix logging
2023-04-07 17:25:23 +03:00
Ilya Averyanov
9bb72ee020
feat(ft-s3): fix behaviour specification
2023-04-07 17:25:23 +03:00
Ilya Averyanov
43f9737420
feat(ft-s3): integrate list API
2023-04-07 17:25:23 +03:00
Ilya Averyanov
5ac3543a76
feat(ft-s3): integrate exporter configs
2023-04-07 17:25:22 +03:00
Ilya Averyanov
8361223648
feat(ft-s3): extract checksum verification
2023-04-07 17:25:22 +03:00
Ilya Averyanov
818a5cacf2
feat(ft-s3): add initial integration
2023-04-07 17:25:22 +03:00
Andrew Mayorov
28d87ca62d
fix(ft-fs): move default config to the backend impl
2023-04-07 17:25:22 +03:00
Andrew Mayorov
258fabbf8b
fix(fs-exp): reply with error when listing failed everywhere
2023-04-07 17:25:22 +03:00
Andrew Mayorov
11edfc1c6a
fix(ft): comment out some clauses as "unreachable"
2023-04-07 17:25:22 +03:00
Andrew Mayorov
bef5cc9c0f
fix(fs-fold): avoid folding through symlinked directories
...
Also a testsuite that verifies multilevel fold behaviour.
2023-04-07 17:25:22 +03:00
Andrew Mayorov
6ad7ce55d2
fix(ft-conf): separate local segments storage settings
...
To make things less ambiguous.
2023-04-07 17:25:22 +03:00
Andrew Mayorov
45e3b62dc4
refactor(ft): prefer plain files over exports in generic APIs
...
So there's no more `exports` magically becomings `files` in the
REST API which was slightly confusing.
2023-04-07 17:25:22 +03:00
