Ivan Dyachkov
606d829256
Merge remote-tracking branch 'upstream/release-57' into 20240724-sync-release-57
2024-07-24 10:28:00 +02:00
zhongwencool
e7d07ea17c
feat: add resource_type to emqx_resource behaviour
2024-07-23 18:24:28 +08:00
Andrew Mayorov
d7d5eb2c52
test(bridge-s3): correct assertions to get rid of flakiness
2024-07-03 16:40:34 +02:00
Thales Macedo Garitezi
5532f40d83
Merge branch 'release-57' into 20240702-m-sync-r57-mix-umbrella
2024-07-02 11:52:36 -03:00
Thales Macedo Garitezi
19f3b030f9
chore: preparing to run common tests / eunit with mix
2024-07-02 09:45:45 -03:00
Thales Macedo Garitezi
b74828d7ea
perf: "mixify" project to improve release speed
...
Fixes https://emqx.atlassian.net/browse/EMQX-12527
2024-07-02 09:39:48 -03:00
Ivan Dyachkov
e28750b522
Merge remote-tracking branch 'upstream/release-57' into 0701-sync-release-57
2024-07-01 13:52:14 +02:00
Thales Macedo Garitezi
795d280861
fix(connector api): handle bad tls config file conversion errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-12581
2024-06-28 14:31:31 -03:00
zmstone
82e7b75a02
chore: bump app versions
2024-06-26 22:47:18 +02:00
zmstone
98f70ea8d8
Merge remote-tracking branch 'origin/release-572' into release-57
2024-06-26 22:28:08 +02:00
Andrew Mayorov
da214be5a1
test(bridge-s3): adapt testcase to different CI environment
2024-06-25 18:35:53 +02:00
Andrew Mayorov
486a041adf
feat(bridge-s3): also map credentials / aggreg upload errors
2024-06-25 16:51:07 +02:00
Andrew Mayorov
fb9afd8313
feat(bridge-s3): beautify posix write errors
2024-06-25 16:51:07 +02:00
Andrew Mayorov
f3ffbd4710
feat(bridge-s3): provide more meaningful error details in status
2024-06-25 16:51:06 +02:00
Andrew Mayorov
98e4ea6fde
feat(bridge-s3): make validation errors more readable
...
And also turn them into schema-level validations, instead of
bridge-level error conditions.
2024-06-25 16:51:06 +02:00
zmstone
5d1608f34b
chore: minimize oss/platform diff
2024-06-24 15:30:03 +02:00
Thales Macedo Garitezi
db572d35a7
Merge remote-tracking branch 'origin/release-57' into sync-r57-m-20240611
2024-06-11 15:34:54 -03:00
Andrew Mayorov
29fc30ea69
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 13:21:41 +02:00
zmstone
45dd7816d7
Merge remote-tracking branch 'origin/release-57'
2024-06-09 10:11:15 +02:00
Andrew Mayorov
8a85d7cb5a
fix(bridge-s3): pass SSL options through convert_certs/2
2024-06-06 19:37:38 +02:00
Thales Macedo Garitezi
9a1d3ea04f
chore: bump app vsns
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
af99829a21
refactor: move common templating logic to emqx_connector_aggreg_buffer_ctx
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
60d24c6ad5
refactor: simplify resource_opts definition
2024-05-31 11:07:44 -03:00
Andrew Mayorov
5c2a68076f
fix(bridge-v2): report descriptive error on invalid update request
...
Before this commit, generic validation errors were reported as union
mismatches of _all_ of the bridge schemas. After this commit, specific
schema is chosen before validation.
2024-05-27 14:37:10 +02:00
JianBo He
adaf238131
fix(s3): fix the convert function if the mode is missing
2024-05-24 16:02:06 +08:00
Kjell Winblad
89b47e8ffc
fix(s3 tracing): do not format records as IO data in aggregate mode
2024-05-23 12:41:30 +02:00
Kjell Winblad
e10c87b825
fix: add action rendered trace for s3 in aggregated mode
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12429
2024-05-23 10:25:27 +02:00
Andrew Mayorov
8016e9adf4
fix(s3-bridge): restore backward config compatibility
2024-05-22 17:59:52 +02:00
Andrew Mayorov
28f6eb0200
fix(s3-bridge): report inconsistent min/max_part_size option
2024-05-21 18:49:01 +02:00
Andrew Mayorov
385ac268d7
chore(s3-bridge): decrease default aggreg interval to 30 minutes
2024-05-17 14:49:18 +02:00
Andrew Mayorov
c348f759af
chore(s3-bridge): decrease default aggreg limit to 100K records
2024-05-17 14:19:59 +02:00
Andrew Mayorov
39c2433afb
fix(s3-bridge): update config typespecs
2024-05-17 09:47:16 +02:00
Andrew Mayorov
5b2c2b336c
feat(s3-bridge): meld 2 separate actions into a single one
...
Discriminated through `mode` parameter. Also rename "simple" action to
a "direct" upload mode.
2024-05-17 09:34:21 +02:00
Andrew Mayorov
c10a48f4be
fix(s3-aggreg): disambiguate action schema namespace
...
Otherwise schemas for `s3` and `s3_aggregated_upload` collide when
projected into OpenAPI schema.
2024-05-14 10:49:21 +02:00
Thales Macedo Garitezi
3a29696a48
fix(aggregator): refactor supervision tree
...
Instead of using a aggregator supervisor with a fixed local name, we should allow
specifying different names so each action app will spawn its own aggregator supervisor.
2024-05-10 09:06:12 -03:00
Thales Macedo Garitezi
7b80a9aa44
fix(aggregator): namespace aggregator ids with action type
...
Otherwise, actions of different types but same names will clash when starting the
aggregator supervision tree.
2024-05-09 16:02:11 -03:00
Thales Macedo Garitezi
32493e395c
refactor: avoid passing record down to template rendering function
2024-05-03 17:43:30 -03:00
Thales Macedo Garitezi
157e2c2535
refactor: make api more generic
2024-05-03 17:25:02 -03:00
Thales Macedo Garitezi
a6508e2fca
refactor: use better names
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2024-05-03 17:04:09 -03:00
Thales Macedo Garitezi
d61f4078e2
refactor: extract blob aggregator logic to separate application
...
This will allow other bridges to share the same aggregation logic as S3 aggregated action.
2024-05-03 14:35:58 -03:00
Kjell Winblad
5bfe31b691
fix: issues found during PR review (thanks @thalesmg and @zmstone)
...
* Simpler handling of true and false in best effort JSON formatter
* inet:ntoa/1 to format IP addresses
* Made a record for lazy formatted trace values and formatter to improve
maintainability
* Added callback to format return value from connector
* Extended test case to check that the format return value callback
works
* Added handling of "lazy" trace entry data to the text formatter. Do we
need to handle this data in the normal log formatters as well?
2024-05-03 15:04:57 +02:00
Kjell Winblad
ea7633c484
fix: rule trace formatting, republish and console stop after rendering
...
* Better rule trace formatting for many trace entries
* The republish and console actions have got working stop after
rendering functionality
2024-05-03 15:02:36 +02:00
Andrew Mayorov
6d3add3646
fix(s3-aggreg): do not handle {error, closed} on buffer write
...
Because it's not really something `file:write/2` is supposed to return.
2024-04-30 10:48:16 +02:00
Andrew Mayorov
4bea938ef2
fix(s3-csv): clarify naming of CSV container concepts
...
Co-Authored-By: Ilya Averyanov <av@rubybox.dev>
2024-04-30 10:48:16 +02:00
Andrew Mayorov
83366cbed0
fix(s3-aggreg): ensure action works in Rule SQL contexts
2024-04-30 10:48:16 +02:00
Andrew Mayorov
a1a313d992
fix(s3-aggeg): apply CSV column order setting consistently
...
Otherwise, columns that are part of column order could appear and
disappear from consecutive uploads, depending on if they are part
of the very first buffered event or not.
2024-04-30 10:48:16 +02:00
Andrew Mayorov
f6e5eea4f7
feat(s3-aggreg): handle delivery shutdowns gracefully
2024-04-30 10:48:15 +02:00
Andrew Mayorov
339036045d
feat(s3-aggreg): support custom and default S3 object HTTP headers
...
I.e. configured container decides default `Content-Type` header.
2024-04-30 10:48:15 +02:00
Andrew Mayorov
5b15b2d641
docs(s3-bridge): document aggregated upload action schema
2024-04-30 10:48:15 +02:00
Andrew Mayorov
ccbcc0c4e3
feat(s3-bridge): implement aggregated upload action
2024-04-30 10:48:15 +02:00
