Commit Graph

63 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
Andrew Mayorov 69c7f858af
fix(s3): try to obtain credentials with default lhttpc
Otherwise `emqx_s3_client` will try to do it later by contacting the S3
endpoint instead of the metadata server.
2024-02-22 16:56:13 +01:00
Andrew Mayorov 0edc5b5992
docs(s3): fix README example 2024-02-12 10:44:18 +01:00
Andrew Mayorov 8f66bd9ddf
fix(s3-client): make log levels saner 2024-02-09 12:03:41 +01:00
Andrew Mayorov 4ff04ab1f3
feat(s3): separate concepts to make app reusable in bridges 2024-02-09 12:03:41 +01:00
Andrew Mayorov a5266f68ec
feat(s3): switch schema to use secrets with loader support
This will make applications using `emqx_s3` follow the same
conventions as bridges and support loading secrets from files
at runtime.
2024-02-08 22:43:20 +01:00
zhongwencool e9c8446d57 feat: upgrade erlfmt to support maybe syntax 2024-01-31 20:06:49 +08:00
Zaiming (Stone) Shi 82403167c2 chore: update BSL license change date 2024-01-29 16:47:31 +01:00
Andrew Mayorov 803529968e
test(s3): fix httpc ssl opts 2024-01-05 19:35:13 +01:00
Ivan Dyachkov 06117c3a33 Merge remote-tracking branch 'upstream/release-54' into 0105-sync-r54 2024-01-05 14:20:38 +01:00
JimMoen 5e100f52b8
style: erlfmt all `rebar.config` files and `bin/nodetool` 2023-12-29 09:08:03 +08:00
Andrew Mayorov 18dc9b2516
fix(s3): remove `emqx_bridge_http` from runtime dependencies
Otherwise it's impossible to test `emqx_s3` in isolation.
2023-12-28 14:13:22 +01:00
Andrew Mayorov 1d9374125c
chore: bump `emqx_s3` to 5.0.13 2023-12-28 14:12:33 +01:00
Andrew Mayorov 33ef964582
fix(s3): handle extra configured headers correctly
* Header names in config become atoms.
  (Probably because of blanket `unsafe_atom_key_map/1` in `emqx_config`)
* Header names should be normalized to lowercase.
  (Otherwise signature will be incorrect)
2023-12-28 13:55:05 +01:00
Zaiming (Stone) Shi 8a425d09bc test(s3): fix httpc ssl opts 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi 89a61bc3bc chore: pin erlcloud version 3.7.0.3
- upgraded from 3.7.0-emqx-2
- 3.7.0.3 is a dependency upgrade:
  - lhttpc upgraded from 1.6.2 to 1.6.3
  - lhttpc 1.6.3 fixed otp 26 compatibility
2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi 25d17ae68d test(ft/s3_uploader): wrong test case was put to wrong group 2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi 423b586c56 fix(dialyzer): fix some dialyzer issues found on otp 26 2023-12-06 20:32:49 +01:00
Ivan Dyachkov 28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Zaiming (Stone) Shi a2700771c9 refactor(emqx_s3_schema): make type readable 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi abcb3166f6 fix: use binary string for raw default values 2023-10-24 14:46:01 +02:00
Paulo Zulato 484519dcf9 feat(kinesis): implement Amazon Kinesis Producer bridge
Fixes https://emqx.atlassian.net/browse/EMQX-10474
Fixes https://emqx.atlassian.net/browse/EMQX-10475
2023-07-17 11:51:42 -03:00
Ivan Dyachkov 6180e775f4 test: fix mocking and application start 2023-07-12 14:47:52 +02:00
Kjell Winblad f28510b3ad refactor: HTTP connector into emqx_bridge_http app 2023-07-12 14:46:43 +02:00
Ilya Averyanov fde506838a fix(ft-api): implement import config behaviour 2023-07-07 19:15:42 +03:00
Ilya Averyanov fe691e8330 fix(ft-api): fix swagger schema dump for ft schema 2023-07-07 19:15:42 +03:00
Ilya Averyanov a2eb658cd9 feat(ft-api): do cleanup certs explicitly 2023-07-07 19:15:42 +03:00
Andrew Mayorov 34793c5ed0 feat(ft-conf): preprocess TLS configuration on updates
Specifically, reify keys and certificates into files in the file system
and update the configuration to point to those files.
2023-07-07 19:15:42 +03:00
Ilya Averyanov 7382c6e6e1
Merge pull request #11103 from savonarola/0620-update-erlcloud
chore(deps): update erlcloud and remove unnecessary overrides
2023-06-21 15:07:45 +03:00
Ilya Averyanov 54d96488a0 chore(deps): update erlcloud and remove unnecessary overrides 2023-06-20 16:46:23 +03:00
zhongwencool 07172e42f0 test: integer CI check failed 2023-06-20 08:39:23 +08:00
Thales Macedo Garitezi 46393343e2 chore: use `timeout_duration` types for timer fields
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
Zaiming (Stone) Shi 36e268c933 chore: bump app versions 2023-05-26 16:05:37 +02:00
Andrew Mayorov 48858dee33
fix(s3): wrap S3 secrets during config load 2023-05-26 12:59:21 +03:00
Andrew Mayorov 3d428a2e0e
fix(ft-s3): rely on asynchronous upload abort
So that upload aborts would not block assemblers. This should not
affect the expected behavior since S3 API usually allows having
concurrent uploads, plus we couldn't really guarantee successful aborts
anyway.
2023-05-26 12:59:20 +03:00
Andrew Mayorov 75cf562c90
fix(ft): tune some logging levels 2023-05-26 12:59:20 +03:00
Andrew Mayorov 3278158c66
fix(ft): disable `ipv6_probe` by default for S3 clients
It's often demonstrates degraded performance on common setups, e.g.
containers with virtualized network interfaces.
2023-05-26 12:59:20 +03:00
Ilya Averyanov 8d9b785bd7 Merge branch 'release-50' into file-transfer
* release-50:
  chore(rebalance): fix app metadata
  chore(rebalance): move apps from lib-ee, add READMEs
  docs: refine zh tr
  docs: delete APL header from ee file
  docs: delete zh changelog
  chore(rebalance): review fixes
  chore(rebalance): rebase and review fixes
  feat(rebalance): port apps from 4.x
2023-05-10 11:55:23 +05:00
Ilya Averyanov cff7788b2e chore(ft): add s3 exporter tests 2023-05-05 11:55:09 +03:00
Andrew Mayorov a79cf681f7
fix(s3): drop default from `acl` config field 2023-04-28 10:54:21 +03:00
Ilya Averyanov 69c4ba2a62 feat(ft): use new utils application 2023-04-22 22:50:19 +03:00
Andrew Mayorov f06300cbed
fix(s3): mark S3 secrets as `sensitive` in schema 2023-04-19 19:05:12 +03:00
Ilya Averyanov e22c1c01ec feat(s3): make acl optional 2023-04-13 19:47:12 +03:00
Andrew Mayorov 37a520d797
fix(i18n): relocate i18n files to rel/i18n 2023-04-10 16:02:34 +03:00
Andrew Mayorov 0d86ef8d3a
fix(ft-s3): use AWS4 signed urls for S3 export URIs
This will ensure that the S3 export URIs are valid in all AWS regions.
2023-04-07 22:15:01 +03:00
Ilya Averyanov b2fd2dcbc1 fix(ft): make configs user friendlier 2023-04-07 17:36:47 +03:00
Ilya Averyanov 7eeba32619 fix(s3): fix typings 2023-04-07 17:36:47 +03:00
Ilya Averyanov eae2478678 fix(ft): add descriptions for missing ft schema fields 2023-04-07 17:36:47 +03:00
Ilya Averyanov bd7250cb13 fix(s3): fix hash pool type 2023-04-07 17:36:47 +03:00
Ilya Averyanov 918bdcae7d fix(ft): fix schema descriptions 2023-04-07 17:36:47 +03:00