yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,956 Commits 393 Branches 684 Tags 74 MiB
ce4e49cd9e
Commit Graph

141 Commits

Author SHA1 Message Date
Andrew Mayorov
dcd59e4f1b
fix(ft): set more conservative filename length limit 
Otherwise, local fs exporter will have a hard time preserving the
filemeta, because its filename is even 13 bytes longer.
 2023-06-06 10:40:20 +03:00
Andrew Mayorov
69b98c1830
fix(ft): ensure temp filenames are under 255 bytes long 2023-06-06 10:38:31 +03:00
Thales Macedo Garitezi
46393343e2 chore: use timeout_duration types for timer fields 
Fixes https://emqx.atlassian.net/browse/EMQX-10020
 2023-06-05 11:46:38 -03:00
zhongwencool
9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve 
feat: support emqx_config:update([authrization],Conf) update
 2023-06-02 15:43:09 +08:00
某文
2e5401f3cb fix: delete rule_engine failed 2023-06-02 00:30:07 +08:00
Andrew Mayorov
0293b54211
fix(ft): respect checksum in fin packets 2023-05-31 18:50:38 +03:00
Zaiming (Stone) Shi
d2ce7c22fd chore: bump app vsn, core app 'emqx' is now at 5.1 2023-05-30 11:32:18 +02:00
JimMoen
8569fac6eb
fix: make static_check happy 2023-05-30 14:00:20 +08:00
JimMoen
909d348570
chore: fix spec typo 2023-05-30 11:31:37 +08:00
Andrew Mayorov
daa7eebc0f
Merge pull request #10843 from keynslug/fix/ft-prop-zerodiv 
test(ft): ensure no division by zero in coverage proptest
 2023-05-29 10:23:52 +03:00
Andrew Mayorov
98ce7e8bbd
test(ft): ensure no division by zero in coverage proptest 2023-05-26 21:53:01 +03:00
Zaiming (Stone) Shi
36e268c933 chore: bump app versions 2023-05-26 16:05:37 +02:00
Andrew Mayorov
3d428a2e0e
fix(ft-s3): rely on asynchronous upload abort 
So that upload aborts would not block assemblers. This should not
affect the expected behavior since S3 API usually allows having
concurrent uploads, plus we couldn't really guarantee successful aborts
anyway.
 2023-05-26 12:59:20 +03:00
Andrew Mayorov
cc3275b389
fix(ft): use conservative timeouts for S3 uploader calls 2023-05-26 12:59:20 +03:00
Andrew Mayorov
d36c2c2928
fix(ft-gc): ensure GC of already complete transfers 2023-05-26 12:59:20 +03:00
Andrew Mayorov
f4047d3946
test(ft): add testcase for nasty tranfer filenames 
* Emoji
* Chinese characters
 2023-05-23 20:46:47 +03:00
Andrew Mayorov
2dbf84479c
fix(ft): handle wider class of jiffy decode errors 
With malformed cursors in the File listing API.

Fixes EMQX-9965
 2023-05-23 20:00:00 +03:00
Andrew Mayorov
cb14a3e08b
fix(ft): handle empty filepath in fs exporter API 
Fixes EMQX-9973
 2023-05-23 19:58:05 +03:00
Andrew Mayorov
d22541e8b3
fix(ft): correct mistyped option in README 2023-05-22 15:32:08 +03:00
Andrew Mayorov
966f748551
test(ft): make proptest less aggressive 
So that the chance of getting huge coverage and as a result, high
chance of running out of memory, is reduced.
 2023-05-22 10:45:27 +03:00
Andrew Mayorov
c95ef71fb5
chore(ft): provide more details in README.md 2023-05-17 12:49:20 +03:00
Andrew Mayorov
e3b822c1a0
chore: remove empty header 
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
 2023-05-16 16:45:41 +03:00
Andrew Mayorov
a7595ff468
chore: fixup copyright years 2023-05-16 16:45:07 +03:00
Andrew Mayorov
6b688d6646
fix(ft): anticipate repeated kickoffs + fix testcase 2023-05-10 17:35:53 +03:00
Ilya Averyanov
42f5433aaf
Merge pull request #10655 from savonarola/0510-idempotent-fin 
Make FT fin command idempotent
 2023-05-10 11:39:43 +05:00
Ilya Averyanov
244188982b fix(ft): add missing translations 2023-05-09 23:32:17 +05:00
Ilya Averyanov
45875bfee5 fix(ft): update s3 exporter tests to the new config 2023-05-09 23:32:17 +05:00
Ilya Averyanov
9381c895bb fix(ft): fix dialyzer issues 2023-05-09 23:32:16 +05:00
Andrew Mayorov
079b8e9476 fix(ft): silence warnings when some root is not yet mkdired 2023-05-09 23:32:16 +05:00
Andrew Mayorov
5e5f854ce1 feat(ft-conf): simplify schema of storage / exporter backends 
Assumption is this changes will make `emqx_ft` config schema
user-friendlier and also more future-proof.
 2023-05-09 23:32:16 +05:00
Ilya Averyanov
8ac881a140 chore(ft): handle multiple/concurrent fins more gracefully 2023-05-09 23:31:39 +05:00
Ilya Averyanov
cff7788b2e chore(ft): add s3 exporter tests 2023-05-05 11:55:09 +03:00
Andrew Mayorov
bd9f129bb8
test(ft-api): also run testcases under cluster setup 2023-04-28 13:50:23 +03:00
Andrew Mayorov
a9866fede4
feat(ft-api): support paging in S3 storage exporter 2023-04-28 13:49:17 +03:00
Andrew Mayorov
75cceffa06
fix(ft-test): rename testcases for consistency 2023-04-28 13:49:17 +03:00
Andrew Mayorov
ed3756ea09
feat(ft-api): add paging support through cursors 2023-04-28 13:49:15 +03:00
Andrew Mayorov
573bb22ada
feat(ft-fs): introduce fs iterators concept + forward seeks 
In order to support paging over filesystem contents, to serve REST
APIs effectively.
 2023-04-28 13:47:50 +03:00
Andrew Mayorov
811e449357
feat(ft-conf): provide global killswitch 2023-04-28 10:54:21 +03:00
Andrew Mayorov
a420c92d28
fix(ft): fix typing issue 2023-04-24 21:51:54 +03:00
Andrew Mayorov
5efd590ca4
feat(ft): properly propagate config updates 
Ensure that:
* Storage config might be removed.
* Local FS GC process is set up when Local FS storage is configured.
* Local FS GC process gets its timer reset on config updates.
* Storage / exporter gets chosen based on `type` only.
* Exporter config updates propagated as before.

Also employ `emqx_ft_schema:translate/1` instead of duplicating
defaults where applicable.
 2023-04-24 21:51:54 +03:00
Ilya Averyanov
69c4ba2a62 feat(ft): use new utils application 2023-04-22 22:50:19 +03:00
Andrew Mayorov
04523b3f81
fix(ft): restrict max filename length in transfers 
Reject transfers with too long filenames right away, during `init`
handling, to avoid having to deal with filesystem errors later.
 2023-04-19 18:48:40 +03:00
Ilya Averyanov
92ff2b7d6e feat(ft): make timeouts configurable 2023-04-14 01:58:14 +03:00
Andrew Mayorov
37a520d797
fix(i18n): relocate i18n files to rel/i18n 2023-04-10 16:02:34 +03:00
Andrew Mayorov
8daa38ef06
feat(ft-s3): store metadata in ASCII-safe format 
Also ensure consistent encoding and decoding filenames throughout
the `emqx_ft` application.
 2023-04-07 22:20:20 +03:00
Ilya Averyanov
b2fd2dcbc1 fix(ft): make configs user friendlier 2023-04-07 17:36:47 +03:00
Ilya Averyanov
7eeba32619 fix(s3): fix typings 2023-04-07 17:36:47 +03:00
Ilya Averyanov
52f3189779 fix(ft): fix default ft config 2023-04-07 17:36:47 +03:00
Ilya Averyanov
bd7250cb13 fix(s3): fix hash pool type 2023-04-07 17:36:47 +03:00
Ilya Averyanov
4a144044b7 fix(ft): set default ft config in tests 2023-04-07 17:36:47 +03:00