Andrew Mayorov
28f6eb0200
fix(s3-bridge): report inconsistent min/max_part_size option
2024-05-21 18:49:01 +02:00
Andrew Mayorov
f6e5eea4f7
feat(s3-aggreg): handle delivery shutdowns gracefully
2024-04-30 10:48:15 +02:00
Andrew Mayorov
339036045d
feat(s3-aggreg): support custom and default S3 object HTTP headers
...
I.e. configured container decides default `Content-Type` header.
2024-04-30 10:48:15 +02:00
Andrew Mayorov
b91ff97170
feat(s3): separate streaming upload logic into dedicated module
...
And use it in `emqx_s3_uploader`, while also turning it into a simple
gen_server.
2024-04-30 10:48:12 +02:00
zmstone
5a4bfff9e5
refactor: add template_str type
2024-04-17 20:34:33 +02:00
zmstone
51c8173174
feat(bridge): add is_template flag to bridge config fields
2024-04-17 13:10:55 +02:00
Zaiming (Stone) Shi
46877e979b
chore: update copyright-year
2024-02-23 08:21:06 +01:00
Andrew Mayorov
69c7f858af
fix(s3): try to obtain credentials with default lhttpc
...
Otherwise `emqx_s3_client` will try to do it later by contacting the S3
endpoint instead of the metadata server.
2024-02-22 16:56:13 +01:00
Andrew Mayorov
8f66bd9ddf
fix(s3-client): make log levels saner
2024-02-09 12:03:41 +01:00
Andrew Mayorov
4ff04ab1f3
feat(s3): separate concepts to make app reusable in bridges
2024-02-09 12:03:41 +01:00
Andrew Mayorov
a5266f68ec
feat(s3): switch schema to use secrets with loader support
...
This will make applications using `emqx_s3` follow the same
conventions as bridges and support loading secrets from files
at runtime.
2024-02-08 22:43:20 +01:00
zhongwencool
e9c8446d57
feat: upgrade erlfmt to support maybe syntax
2024-01-31 20:06:49 +08:00
Andrew Mayorov
18dc9b2516
fix(s3): remove emqx_bridge_http from runtime dependencies
...
Otherwise it's impossible to test `emqx_s3` in isolation.
2023-12-28 14:13:22 +01:00
Andrew Mayorov
1d9374125c
chore: bump emqx_s3 to 5.0.13
2023-12-28 14:12:33 +01:00
Andrew Mayorov
33ef964582
fix(s3): handle extra configured headers correctly
...
* Header names in config become atoms.
(Probably because of blanket `unsafe_atom_key_map/1` in `emqx_config`)
* Header names should be normalized to lowercase.
(Otherwise signature will be incorrect)
2023-12-28 13:55:05 +01:00
Zaiming (Stone) Shi
423b586c56
fix(dialyzer): fix some dialyzer issues found on otp 26
2023-12-06 20:32:49 +01:00
Ivan Dyachkov
28a577ad09
chore: bump apps versions
2023-11-14 11:02:26 +01:00
Zaiming (Stone) Shi
a2700771c9
refactor(emqx_s3_schema): make type readable
2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
abcb3166f6
fix: use binary string for raw default values
2023-10-24 14:46:01 +02:00
Kjell Winblad
f28510b3ad
refactor: HTTP connector into emqx_bridge_http app
2023-07-12 14:46:43 +02:00
Ilya Averyanov
fde506838a
fix(ft-api): implement import config behaviour
2023-07-07 19:15:42 +03:00
Ilya Averyanov
fe691e8330
fix(ft-api): fix swagger schema dump for ft schema
2023-07-07 19:15:42 +03:00
Ilya Averyanov
a2eb658cd9
feat(ft-api): do cleanup certs explicitly
2023-07-07 19:15:42 +03:00
Andrew Mayorov
34793c5ed0
feat(ft-conf): preprocess TLS configuration on updates
...
Specifically, reify keys and certificates into files in the file system
and update the configuration to point to those files.
2023-07-07 19:15:42 +03:00
Thales Macedo Garitezi
46393343e2
chore: use timeout_duration types for timer fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
Zaiming (Stone) Shi
36e268c933
chore: bump app versions
2023-05-26 16:05:37 +02:00
Andrew Mayorov
48858dee33
fix(s3): wrap S3 secrets during config load
2023-05-26 12:59:21 +03:00
Andrew Mayorov
3d428a2e0e
fix(ft-s3): rely on asynchronous upload abort
...
So that upload aborts would not block assemblers. This should not
affect the expected behavior since S3 API usually allows having
concurrent uploads, plus we couldn't really guarantee successful aborts
anyway.
2023-05-26 12:59:20 +03:00
Andrew Mayorov
75cf562c90
fix(ft): tune some logging levels
2023-05-26 12:59:20 +03:00
Andrew Mayorov
3278158c66
fix(ft): disable ipv6_probe by default for S3 clients
...
It's often demonstrates degraded performance on common setups, e.g.
containers with virtualized network interfaces.
2023-05-26 12:59:20 +03:00
Andrew Mayorov
a79cf681f7
fix(s3): drop default from acl config field
2023-04-28 10:54:21 +03:00
Ilya Averyanov
69c4ba2a62
feat(ft): use new utils application
2023-04-22 22:50:19 +03:00
Andrew Mayorov
f06300cbed
fix(s3): mark S3 secrets as sensitive in schema
2023-04-19 19:05:12 +03:00
Ilya Averyanov
e22c1c01ec
feat(s3): make acl optional
2023-04-13 19:47:12 +03:00
Andrew Mayorov
0d86ef8d3a
fix(ft-s3): use AWS4 signed urls for S3 export URIs
...
This will ensure that the S3 export URIs are valid in all AWS regions.
2023-04-07 22:15:01 +03:00
Ilya Averyanov
b2fd2dcbc1
fix(ft): make configs user friendlier
2023-04-07 17:36:47 +03:00
Ilya Averyanov
7eeba32619
fix(s3): fix typings
2023-04-07 17:36:47 +03:00
Ilya Averyanov
eae2478678
fix(ft): add descriptions for missing ft schema fields
2023-04-07 17:36:47 +03:00
Ilya Averyanov
bd7250cb13
fix(s3): fix hash pool type
2023-04-07 17:36:47 +03:00
Ilya Averyanov
f6461fe287
fix(ft): fix typings
2023-04-07 17:36:47 +03:00
Ilya Averyanov
820e06d756
fix(ft-s3): make controllable use of max_retries and request_timeout arguments
2023-04-07 17:36:47 +03:00
Ilya Averyanov
be99242e32
fix(ft-s3): fix review comments
2023-04-07 17:36:47 +03:00
Ilya Averyanov
d7a85242de
fix(ft-s3): fix s3 listing
2023-04-07 17:36:47 +03:00
Ilya Averyanov
7d13862da5
fix(ft-s3): use uploader specific headers only for object creation
2023-04-07 17:36:47 +03:00
Ilya Averyanov
4fbabe5a76
fix(ft-s3): fix atom and variable naming
2023-04-07 17:27:01 +03:00
Ilya Averyanov
ca22f11161
feat(ft-s3): fix logging
2023-04-07 17:25:23 +03:00
Ilya Averyanov
43f9737420
feat(ft-s3): integrate list API
2023-04-07 17:25:23 +03:00
Ilya Averyanov
818a5cacf2
feat(ft-s3): add initial integration
2023-04-07 17:25:22 +03:00
Ilya Averyanov
31b441a46e
feat(s3): add S3 client application
2023-04-07 17:25:22 +03:00
