3ed385201c
Merge pull request #12947 from savonarola/0426-auth-timeout
...
feat(authn): add connection expire based on authn data
2024-05-03 11:32:18 +03:00
3064a1cbae
Merge pull request #12957 from id/0430-adjust-build-scripts
...
adjust build scripts
2024-05-02 17:05:06 +02:00
607bf0fe44
Merge pull request #12952 from thalesmg/fix-dont-list-mv-rule-event-r57-20240429
...
fix(rule events): don't list validation failure event in CE API
2024-05-02 09:20:53 -03:00
035b8480b0
chore(emqx_conf): extract data dirs to sync to a macro
2024-05-02 10:25:53 +02:00
7a44746492
fix(rule events): don't list validation failures in CE
...
Fixes https://emqx.atlassian.net/browse/EMQX-12284
2024-04-30 14:41:09 -03:00
874f1f2428
Merge pull request #12950 from thalesmg/fix-mv-duplicated-topics-r57-20240429
...
fix(message validation): validate duplicated topics
2024-04-30 13:34:58 -03:00
dd6566f3c5
Merge pull request #12948 from thalesmg/fix-http-bridge-header-update-obfuscate-r57-20240429
...
fix(http connector): deobfuscate sensitive headers
2024-04-30 13:34:15 -03:00
e4154dd472
feat(authn): use correct time resolution for setting channel expire in JWT authn
2024-04-30 19:01:16 +03:00
80d724c504
feat(authn): add connection expire based on authn data
2024-04-30 17:04:55 +03:00
279169105b
Merge pull request #12934 from keynslug/feat/EMQX-12204/aggreg-s3-bridge
...
feat(s3-bridge): implement aggregated upload action
2024-04-30 13:50:19 +02:00
1974ec15ec
fix(client_attrs): fix client_attrs extraction loop
2024-04-30 12:12:35 +02:00
a88862ebe0
Merge pull request #12951 from zmstone/0429-fix-description-schema
...
fix(schema): description should be in binary() type
2024-04-30 12:03:55 +02:00
6d3add3646
fix(s3-aggreg): do not handle `{error, closed}` on buffer write
...
Because it's not really something `file:write/2` is supposed to return.
2024-04-30 10:48:16 +02:00
4bea938ef2
fix(s3-csv): clarify naming of CSV container concepts
...
Co-Authored-By: Ilya Averyanov <av@rubybox.dev>
2024-04-30 10:48:16 +02:00
83366cbed0
fix(s3-aggreg): ensure action works in Rule SQL contexts
2024-04-30 10:48:16 +02:00
a1a313d992
fix(s3-aggeg): apply CSV column order setting consistently
...
Otherwise, columns that are part of column order could appear and
disappear from consecutive uploads, depending on if they are part
of the very first buffered event or not.
2024-04-30 10:48:16 +02:00
f6e5eea4f7
feat(s3-aggreg): handle delivery shutdowns gracefully
2024-04-30 10:48:15 +02:00
339036045d
feat(s3-aggreg): support custom and default S3 object HTTP headers
...
I.e. configured container decides default `Content-Type` header.
2024-04-30 10:48:15 +02:00
5b15b2d641
docs(s3-bridge): document aggregated upload action schema
2024-04-30 10:48:15 +02:00
ccbcc0c4e3
feat(s3-bridge): implement aggregated upload action
2024-04-30 10:48:15 +02:00
b91ff97170
feat(s3): separate streaming upload logic into dedicated module
...
And use it in `emqx_s3_uploader`, while also turning it into a simple
gen_server.
2024-04-30 10:48:12 +02:00
c8d6976b14
feat: add conditions to variform expressions
...
- refactored `coalesce` function to allow lazy evaluation
- added `iif(Cond, IfExpr, EleseExpr)` to allow simple conditions
2024-04-30 07:28:27 +02:00
9917293fd0
fix(schema): description should be in binary() type
2024-04-29 21:44:55 +02:00
ffedce014f
fix(message validation): validate duplicated topics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12254
2024-04-29 15:44:03 -03:00
a847389159
fix(http connector): deobfuscate sensitive headers
...
Fixes https://emqx.atlassian.net/browse/EMQX-12213
2024-04-29 15:20:46 -03:00
5cf92dcb73
refactor: use `spawn_link` instead of `spawn_monitor`
...
This should cover the case when the resource manager is brutally killed.
2024-04-29 15:20:32 -03:00
475077c798
fix(resource): account for ongoing channel health checks, update data and reply immediately when receiving an update
2024-04-29 15:20:32 -03:00
79526d539a
fix(resource manager): clean up any running health checks when terminating
...
Fixes https://github.com/emqx/emqx/pull/12812#discussion_r1555564254
2024-04-29 15:20:32 -03:00
7a05a4754f
docs: expose zone config in schema doc
2024-04-29 13:57:18 +02:00
a41652ec31
Merge pull request #12940 from zmstone/0427-catch-hocon-syntax-error
...
feat(mgmt): add ignore_readonly to configs API
2024-04-29 13:22:12 +02:00
1db932df21
chore(mgmt): PUT /configs?ignore_readonly=true, lower log to info level
2024-04-29 10:20:47 +02:00
07cbdc6e90
feat(mgmt): add ignore_readonly qeury-string to PUT /configs API
2024-04-28 14:45:53 +02:00
10625eacac
chore: upgrade to hocon-0.42.2
...
hocon pretty-print quotes more string values
if a string has '.' or '-', or if it starts with a digit 0-9,
then it's quoted.
see details here: https://github.com/emqx/hocon/pull/293
2024-04-28 14:05:30 +02:00
d6c203b4fd
fix(dynamo): fixed the keys checking for Dynamo
2024-04-28 16:59:01 +08:00
5f4215b333
fix(mgmt): avoid 500 error when hocon syntax error
2024-04-27 21:12:42 +02:00
43ac4f5dfe
fix: make bpapi check happy
2024-04-26 21:17:01 +08:00
5ff4e76904
refactor: rename plugins config api functions
2024-04-26 20:51:20 +08:00
00cab33fde
fix: plugin's internal config api
2024-04-26 18:10:55 +08:00
11389bc086
fix: i18n file renamed
2024-04-26 11:12:24 +08:00
28e8131984
refactor: avoid make when do serde
2024-04-26 10:37:15 +08:00
e5bd747b32
refactor: read avsc file when make serde
2024-04-26 10:25:19 +08:00
670ddae57c
chore: fix typo
2024-04-26 10:25:18 +08:00
c884dfb451
test: make eunit happy
2024-04-26 10:25:17 +08:00
1869f6fd0a
fix: enusre plugin installed when do config operation
2024-04-26 10:25:16 +08:00
b0aa3bb70f
fix(plugin): get plugin config api
2024-04-26 10:25:15 +08:00
f343cd2021
fix: get plugin config in json
2024-04-26 10:25:14 +08:00
d2ecccc2ff
fix: call json encoder/decoder for plugin config
2024-04-26 10:25:12 +08:00
c180b6a417
fix(api): plugin api docs
2024-04-26 10:25:11 +08:00
1f00ce789f
fix(plugin): gen_server call timeout infinity
2024-04-26 10:25:10 +08:00
c0429ca333
fix(plugin): refine schema serde log
2024-04-26 10:25:09 +08:00
27d1f91cac
refactor: refine function name
2024-04-26 10:25:08 +08:00
2686a66b14
fix: make eunit happy
2024-04-26 10:25:07 +08:00
d2e0c09f2e
fix: make static check happy
2024-04-26 10:25:06 +08:00
d06f410fd5
fix(plugins): read avsc file
2024-04-26 10:25:05 +08:00
8db5e51592
feat: plugin config with avro schema and apis
2024-04-26 10:25:04 +08:00
71cdcc860a
fix(plugin): plugin's mgmt api schema codes
2024-04-26 10:25:03 +08:00
6af651cf2f
Merge pull request #12931 from zmstone/0425-fix-acl.conf-topic-template-render-fialure-handling
...
0425 fix acl.conf topic template render fialure handling
2024-04-25 20:30:02 +02:00
01923147a2
fix(variform and authz): do not initialize empty client_attrs field
...
when client_attrs_init expression renders to empty string,
do not initialize the attribute.
also fixed an ACL error: a template render failure for a topic
would stop the ACL checks for the following topics if more
than one topic is configured.
2024-04-25 17:32:07 +02:00
d30b52f0f9
docs: refine acl.conf comments
2024-04-25 17:32:07 +02:00
15594b4db6
fix(HTTP connector): retry on 503 Service Unavailable response
...
Previously, if an HTTP request received a 503 (Service Unavailable)
status, it was marked as a failure without retrying. This has now been
fixed so that the request is retried a configurable number of times.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12217
https://github.com/emqx/emqx/issues/12869 (partly)
2024-04-25 17:05:18 +02:00
ff09f14191
fix(http connector): remove sensitive info from headers lazily
...
In production code we don't need to redact the headers for a trace that
will never appear anywhere so we can improve performance by doing
removal of sensitive information lazily.
2024-04-25 13:27:14 +02:00
ef9884cf47
refactor(rule trace): templates rendered trace to increase code reuse
...
* The code for passing the trace context to a sub process has been
improved to increase code reuse. This code is used when the action
templates are rendered in a sub process.
* A macro has also been added for the error term that is thrown when the
action shall be stopped after the templates has been rendered. This is
also done to reduce code duplication and to reduce the risk of
introducing bugs due to typos.
* Fix incorrect type spec
Thanks to @zmstone for suggesting these improvements in comments to a PR
(https://github.com/emqx/emqx/pull/12916 ).
2024-04-25 13:15:36 +02:00
0dbaef4316
feat: add stop after render and after render trace to tdengine action
2024-04-25 11:51:46 +02:00
d6ceeb3b30
feat: add stop after render and after render trace to rabbitmq action
2024-04-25 11:51:46 +02:00
11d9d30fc0
feat: add stop after render and after render trace to syskeeper action
2024-04-25 11:51:46 +02:00
a0b2357abb
feat: add stop after render and after render trace to sqlserver action
2024-04-25 11:51:46 +02:00
03e3ac19a9
feat: add stop after render and after render trace to s3 action
2024-04-25 11:51:46 +02:00
22c7224267
feat: add stop after render and after render trace to rocketmq action
2024-04-25 11:51:46 +02:00
2abc1b1141
feat: add stop after render and after render trace to redis action
2024-04-25 11:51:46 +02:00
74fac80e7e
feat: add stop after render and after render trace to pulsar action
2024-04-25 11:51:46 +02:00
9c37c99b62
feat: add stop after render and after render trace to oracle action
2024-04-25 11:51:46 +02:00
d27f05fa60
feat: add stop after render and after render trace to influxdb action
2024-04-25 11:51:46 +02:00
279ad186f7
feat: add stop after render and after render trace to kinesis action
2024-04-25 11:51:46 +02:00
beedc72be4
feat: add stop after render and after render trace to mongodb action
2024-04-25 11:51:46 +02:00
e2b35ea242
feat: add stop after render and after render trace to opents action
2024-04-25 11:51:46 +02:00
9d6655bc30
feat: add stop after render and after render trace to hstreamdb action
2024-04-25 11:51:46 +02:00
7c7590fbc8
feat: add stop after render and after render trace to greptimedb action
2024-04-25 11:51:46 +02:00
7922d5d422
feat: add stop after render and after render trace to gcp action
2024-04-25 11:51:46 +02:00
a2dd8f5aee
feat: add stop after render and after render trace to cassandra action
2024-04-25 11:51:46 +02:00
810aa68b02
feat: add stop after render and after render trace to dynamo action
2024-04-25 11:51:46 +02:00
120b35ac75
feat: add stop after render and after render trace to mysql action
2024-04-25 11:51:46 +02:00
b2811f96b2
refactor(rule trace): simplify function for setting trace meta data
...
This commit simplifies a function to set trace meta data in line with a
suggestion from @zmstone:
https://github.com/emqx/emqx/pull/12912#discussion_r1576053856
2024-04-25 11:51:46 +02:00
7ad354f412
feat: add stop after render and after render trace to clickhouse action
2024-04-25 11:51:46 +02:00
32c27f1711
feat: add stop after render and after render trace to kafka action
2024-04-25 11:51:46 +02:00
b02ed4e6ec
feat: add stop after render and after render trace to pgsql action
2024-04-25 11:51:46 +02:00
1f676ce035
feat: add stop after render and after render trace to mqtt action
2024-04-25 11:51:46 +02:00
5ca90ccced
fix: improve structure of log trace entries for HTTP action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12025
2024-04-25 11:51:46 +02:00
a67773e973
Merge remote-tracking branch 'origin/master' into release-57
2024-04-25 10:56:52 +02:00
2cbf4dc789
Merge pull request #12923 from zhongwencool/authn-mnesia
...
chore: provided more specific error for wrong import method
2024-04-25 10:28:09 +08:00
2a2da3766e
chore: apply suggestions from code review
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-04-25 09:16:53 +08:00
f2fa35efdf
Merge remote-tracking branch 'origin/master' into 0424-merge-latest-master-to-release-57
2024-04-24 22:02:56 +02:00
02a0faa1d6
refactor: avoid evaluating Data more than once in SLOG_THROTTE macro
2024-04-24 16:26:34 +03:00
b66d7a688e
Merge remote-tracking branch 'origin/release-57' into release-57
2024-04-24 13:09:06 +02:00
4f12d45436
chore: provided more specific error for wrong import method
2024-04-24 15:37:35 +08:00
712008bc1d
Merge pull request #12909 from zhongwencool/esockd-bump-version
...
fix: bump escokd to 5.11.2 to handle udp_error/closed message
2024-04-24 09:15:39 +08:00
5804b70c1f
Merge pull request #12918 from ieQu1/dev/ds-readme1
...
docs(ds): Update README with CLI and REST API endpoints
2024-04-23 17:02:01 +02:00
e4c3283c9c
docs(ds): Update README with CLI and REST API endpoints
2024-04-23 16:28:35 +02:00
e1eed30b5d
Merge pull request #12912 from kjellwinblad/kjell/rule_trigger_time/EMQX-12025
...
feat(rule tracing): add rule trigger time meta data field
2024-04-23 15:00:05 +02:00
0d1c13661f
Merge pull request #12895 from lafirest/fix/dyndb
...
fix(dynamo): Added missing keys for DynamoDB
2024-04-22 23:03:00 +08:00
a81ca359fc
Merge pull request #12899 from lafirest/fix/rocmq
...
feat(rocketmq): add support for namespace and key dispatch strategy
2024-04-22 21:20:44 +08:00
e9d498dde2
feat(rule tracing): add rule trigger time meta data field
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12025
2024-04-22 13:51:48 +02:00
6049b4966f
fix: bump escokd to 5.11.2 to handle udp_error/closed message
2024-04-22 16:41:09 +08:00
f9eda1883f
Merge pull request #12903 from ieQu1/dev/ds-egress-flush-condition
...
fix(ds): Fix egress flush condition
2024-04-22 10:17:22 +02:00
c5c4bb987f
Merge pull request #12902 from HJianBo/stomp
...
Pass the Content-type of MQTT message to the Stomp message
2024-04-22 16:02:03 +08:00
b723e87e54
Merge remote-tracking branch 'upstream/release-56' into 0422-sync-release-56
2024-04-22 08:12:05 +02:00
d5cdc07eab
feat(rocketmq): add support for namespace and key dispatch strategy
2024-04-22 13:14:11 +08:00
46f339dfab
chore: updte change log
2024-04-22 13:12:01 +08:00
a1e85e3c59
fix(dynamo): Added missing keys for DynamoDB
2024-04-22 13:09:34 +08:00
6e99f01ecd
Merge pull request #12863 from kjellwinblad/kjell/new_trace_log_formatter/EMQX-12025
...
feat(emqx_trace): add JSON trace log entry formatter
2024-04-22 06:25:18 +02:00
d85df14b85
Merge pull request #12892 from HJianBo/fix-gateway-related-issues
...
fix(ocpp): avoid an error log in handling downstream messages
2024-04-22 11:25:26 +08:00
69bdcd2f24
Merge pull request #12898 from emqx/iotdb-1.3.0
...
feat: support iotdb 1.3.0
2024-04-22 10:56:06 +08:00
4c76a2574d
fix(ds): Fix egress flush condition
2024-04-21 21:51:31 +02:00
307cd79be2
Merge pull request #12897 from zmstone/0418-support-dynamic-license
...
feat(license): add business-critical customer type
2024-04-20 09:03:58 +02:00
5520e54147
chore: add tests
2024-04-20 08:59:36 +08:00
aaf7953968
chore: apply suggestions from code review
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-04-20 08:36:59 +08:00
95f3e49edb
fix(stomp): pass the Content-Type from the MQTT message
2024-04-20 08:33:55 +08:00
43f8346c00
fix(dssnap): ensure idempotent write of empty chunks
2024-04-19 18:52:33 +02:00
bac5100635
Merge pull request #12874 from ieQu1/dev/EMQX-12030-subscriptions-api
...
API for durable subscriptions
2024-04-19 17:11:53 +02:00
2890bc2619
fix(tracing): remove internal extra field from the trace config
...
This commit removes the internal extra field from the trace config
structure exposed to the user via the HTTP API.
2024-04-19 17:05:34 +02:00
9e46c18443
fix(emqx_trace_SUITE:t_base_create_delete): broken test case
2024-04-19 15:41:50 +02:00
ec83fbe3dc
feat(license): add business critical customer type
2024-04-19 14:19:39 +02:00
ede7246882
fix(sessds): Avoid double-enriching transient messages
2024-04-19 13:39:04 +02:00
b7a4536f47
Merge pull request #12871 from savonarola/0412-fix-rebalance-app-dependencies
...
fix(rebalance): fix start order of rebalance applications
2024-04-19 13:27:41 +03:00
a8b1224225
Merge pull request #12893 from zmstone/0416-add-is-template-flag-to-dashboard-schema
...
0416 add `is_template` flag to dashboard schema
2024-04-19 10:41:43 +02:00
c163956d08
fix(trace formatter): remove record field to enable rolling upgrade
2024-04-19 09:52:20 +02:00
b163a87386
feat: support batch_size on iotdb
2024-04-19 14:52:56 +08:00
c0521fd250
feat: support iotdb 1.3.0
2024-04-19 08:54:18 +08:00
f1e6565ddd
refactor(sessds): Move all subscription logic to the subs module
2024-04-19 01:20:29 +02:00
d12966db5b
test: Avoid dumping raw snabbkaffe traces to the console
2024-04-19 01:20:29 +02:00
180130d684
feat(sessds): List persistent subscriptions in the REST API
2024-04-19 01:20:29 +02:00
09b414f368
test: add necessary application to test suites
2024-04-18 17:39:52 +02:00
285bfa9367
fix: improve rendering of action_template_rendered trace
2024-04-18 17:00:47 +02:00
7be18730e8
test(emqx_rule_engine_api_rule_apply_SUITE): remove unnecessary code
2024-04-18 15:16:18 +02:00
f5b0439724
fix(emqx_rule_engine_api_rule_apply_SUITE): flaky test case
2024-04-18 14:28:28 +02:00
8f1486f6d3
fix: clean up trace messages to make it easier to interpret
...
This commit removes some redundant trace messages and renames some to
make it easier to interpret what is happening for the user.
2024-04-18 14:28:01 +02:00
5b38d592f0
feat(http): add `is_template` as HTTP headers field property
...
is_template was designed to be type property.
however for HTTP headers, it's a map() type,
instead of creating a new type for it, it's easier to just
add it as a field property.
2024-04-18 13:16:29 +02:00
ede4eeae9f
fix(http_bridge): path is template field
2024-04-18 13:04:56 +02:00
ca56e7e8d7
fix(kafka): headers are template fields
2024-04-18 13:04:36 +02:00
3232ab5ea3
fix(rule tracing): unset trace meta data in try-after-end
...
We wrap the reset of the process trace meta data in the after clause of
a try-after-end expression to be sure we never get any lingering
incorrect meta data.
2024-04-18 11:28:50 +02:00
aa388adba9
fix(json trace format): format client_ids and rule_ids as lists
2024-04-18 11:27:39 +02:00
10957e7d79
fix: change name of the default trace log format from plain to text
2024-04-18 10:42:51 +02:00
ef9f8a8fdf
fix: unreachable clause found by dialyzer
2024-04-18 10:28:37 +02:00
6c0ee8bb01
test(emqx_trace_SUITE): fix failure due to new field
2024-04-18 10:28:37 +02:00
968dc2ccda
fix(json trace format): use best_effort_json instead of new encoder
2024-04-18 10:28:37 +02:00
aa39835242
feat(emqx_cli): add parameter for trace format (plain or json)
2024-04-18 10:28:29 +02:00
6fd7a06c5d
fix: problems reported by dialyzer
2024-04-18 10:20:18 +02:00
a6558740e8
feat(emqx_trace): add JSON trace log entry formatter
...
This commit makes it possible to select the JSON trace log entry
formatter when crating a trace pattern. This will make it easier for the
dashboard and automatic tests to parse the log entries.
Fixes: EMQX-12025 (partly)
2024-04-18 10:20:11 +02:00
ab763fe665
test: fix test case flakyness
2024-04-18 09:32:05 +02:00
2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
...
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
d7ebecddb4
fix(ocpp): return correct current_connections number of listenrs http api
2024-04-18 14:54:12 +08:00
e93bd314bc
chore: 5.6.1
2024-04-18 08:03:56 +02:00
75632bb2cd
chore(gw): update listener examples
2024-04-18 13:58:31 +08:00
b16b9d8fcc
refactor: made code more readable thanks to suggestion from @zmstone
2024-04-18 06:51:30 +02:00
55941000c0
test: make test case more stable, less flaky
2024-04-17 20:42:03 +02:00
6ab2b004ed
fix(resource_manager): update cache after channel add
2024-04-17 20:34:33 +02:00
c96ae8dd23
fix: return 503 if bridge bpapi call timeout
2024-04-17 20:34:33 +02:00
5a4bfff9e5
refactor: add template_str type
2024-04-17 20:34:33 +02:00
5c014f4c29
test: fix test cases
2024-04-17 20:34:33 +02:00
95891db29a
test: clean up created resources in the end of test case
2024-04-17 18:43:19 +02:00
6cf29ba688
fix: clean up traces to make them easier to parse and understand
2024-04-17 18:20:17 +02:00
cf56050759
feat: avoid mixing request with and without the stop_after_render flag
...
Previously a batch of requests that was sent to a connector could
contain both requests with the stop_after_rendering flag and requests
without this flag. When this happened a warning message was generated and
the stop_after_render flags for the batch would be ignored. This commit
fixes so that a mixed batch is never created so there is no longer any
need for a warning message or ignoring flags.
2024-04-17 16:21:21 +02:00
d49e98bc4b
test: fix dashboard schema validation
2024-04-17 14:09:48 +02:00
f9f14f9758
refactor(emqx_conf): raise exception at higher level for more context
2024-04-17 13:10:55 +02:00
51c8173174
feat(bridge): add is_template flag to bridge config fields
2024-04-17 13:10:55 +02:00
2468243dfd
chore: update changes
2024-04-17 18:52:48 +08:00
124c5047d0
feat(sessds): Add API for getting session data from the cold storage
2024-04-17 12:31:54 +02:00
38a2e8add9
fix(sessds): Return the number of subscriptions for offline sessions
2024-04-17 12:31:54 +02:00
e439a2e0f2
fix(sessds): Save protocol name and version in the session metadata
2024-04-17 12:31:54 +02:00
18196ec19c
fix(ocpp): avoid an error log in handling dnsteeam messages
2024-04-17 17:48:19 +08:00
c61c3157c6
Merge pull request #12888 from HJianBo/data-backup-for-license
...
fix(license): license file is not taking effect after importing backup files
2024-04-17 16:31:16 +08:00
1dfd9115cd
Merge pull request #12880 from JimMoen/EMQX-12147/influx-write-syntax-tag-type
...
fix(influx): literal number values in tag set
2024-04-17 10:23:57 +08:00
197a4c30be
fix(sessds): Strip unneccessary data from the durable session state
2024-04-17 01:21:52 +02:00
93bb840365
docs(ds): Update README
2024-04-17 01:21:52 +02:00
87ffaf89e5
refactor(sessds_state): Use macros for map keys
2024-04-17 01:21:52 +02:00
113a990482
feat(sessds): Support max subscriptions
2024-04-17 01:21:51 +02:00
6c897c26ae
fix(sessds): Commit session on unsubscribe
2024-04-17 01:21:51 +02:00
6c83bbe10b
feat(mgmt): Filter subscriptions by durability
2024-04-17 01:21:51 +02:00
b30ddc206e
fix(sessds): Immutable subscriptions
...
This commit fixes two issues:
- Behavior of overlapping subscriptions has been aligned with the
in-memory session.
- Fixed handling of replays when subscription changes (either by
client or EMQX configuration)
2024-04-17 01:21:51 +02:00
3e0c649e8e
feat(sessds): Store awaiting rel
2024-04-17 01:21:51 +02:00
df458b98d7
refactor(dashboard_schema): no need to translate labels
...
the trans_label implementation was ugly, it compares an anonymous
function to check if the label should be translated.
since we have stopped generating i18n message ids for dashboard
schema, this entire function is now stale, so this function
is deleted.
2024-04-16 17:54:34 +02:00
2bd72aab44
chore: bump dashboard schema version to 0.2.0
2024-04-16 16:58:45 +02:00
4d38a8fd44
Merge pull request #12872 from zmstone/0412-support-variform-for-client_attrs
...
0412 use variform for client_attrs
2024-04-16 13:29:56 +02:00
6b8111c066
fix(license): license file is not taking effect after importing backup files
2024-04-16 19:07:17 +08:00
c645cfa5d6
fix(sessds): Graceful handling of shared subscription error
2024-04-16 12:37:01 +02:00
f3bb28c6af
chore: 5.6.1-rc.2
2024-04-16 09:47:32 +02:00
5d7b2e2ce6
fix(dsrepl): attempt leadership transfer on terminate
...
In addition to on removal. The reasoning is basically the same: try to
avoid situations when log entries are replicated (or will be considered
replicated when the new leader is elected) but the leader terminates
before replying to the client.
To be clear: this is a stupid solution. Something much more robust is
needed.
2024-04-15 22:05:24 +02:00
a0ffe5e7ae
chore: 5.6.1-rc.1
2024-04-15 22:05:05 +02:00
f481871792
fix(rocketmq action): we need one producer group per channel and topic
...
We need one producer group per channel and topic because we can have
several topics per channel due to templating.
2024-04-15 20:52:08 +02:00
1fe92bddd0
fix(rocketmq action): make sure that topic template is respected
2024-04-15 20:37:23 +02:00
7f9a311988
fix(rocketmq action): all actions used only one topic
...
This commit makes sure that a set of rocketmq workers are started for
each topic. Before this commit all actions for a rocketmq connector used
the same workers which all were bound to a single topic so all messages
got delivered to that topic regardless of the action configuration.
We should have automatic tests to check that the messages actually go to
different topics but this needs to go into another PR since we have to
deliver the fix fast and the rocketmq library does not support reading
from topics.
Fixes:
https://emqx.atlassian.net/browse/EEC-1006
2024-04-15 20:08:56 +02:00
f80d078de3
feat(variform): Add more functions
2024-04-15 16:56:52 +02:00
89f42f1171
fix(dsrepl): make placeholder shard process permanent under supervisor
2024-04-15 16:43:52 +02:00
c4d1360b96
fix(dsrepl): trigger election for new ra servers unconditionallly
...
Otherwise we might end up in a situation when there's no member online
yet at the time of the election trigger, and the election will never
happen.
2024-04-15 16:42:29 +02:00
084e920c6e
test(influx): literal values or variable in tag set
2024-04-15 17:47:53 +08:00
8ed397d4fa
fix(influx): literal number values in tag set
2024-04-15 17:47:52 +08:00
3ef160eed2
Merge remote-tracking branch 'upstream/release-56' into 0415-sync-release-56
2024-04-15 08:09:03 +02:00
b76b6fbe63
feat(variform): initialize client_attrs with variform
...
Moved regular expression extraction as a variform function.
2024-04-14 10:13:24 +02:00
da5b01aa46
refactor(client_attr): allow more than one initial extraction
2024-04-13 01:00:25 +02:00
2577224bc7
fix(swagger): do not generate dummy descriptions
2024-04-13 01:00:25 +02:00
06b07460e8
Merge pull request #12866 from zmstone/0411-refactor-resove-TODOs
...
refactor: resolve some old TODOs
2024-04-13 00:10:47 +02:00
b4198185bc
fix(http connector): redact sensitive information from headers
2024-04-12 15:27:03 +02:00
6777f04780
fix: iolist_to_binrary -> unicode:characters_to_binary
...
It is not always safe to use iolist_to_binrary on the output of an
io_lib:format call with the ~ts placeholder.
2024-04-12 15:21:25 +02:00
500d4fedda
fix(rebalance): fix start order of rebalance applications
2024-04-12 15:58:57 +03:00
ed5409fb6a
docs(trace): add emqx_ctl documentation for the new ruleid trace
2024-04-12 11:54:14 +02:00
f444c6fc32
fix: pass stop_action_after_render=true in trace meta data
...
Even if there is no trace we still need to pass
stop_action_after_render=true in the trace meta data so that the action
will be stopped.
2024-04-12 11:38:01 +02:00
aa950f97fa
test: fix tests with missing application
2024-04-12 11:23:19 +02:00
9998940aa2
fix(trace): several improvements thanks to comments from @zmstone
2024-04-11 16:48:43 +02:00
b27fc0da26
test(emqx_machine): ensure node is down before testing open ports
2024-04-11 15:24:41 +02:00
dfd13b4ab5
test: fix dashboard schema json test cases
2024-04-11 15:12:01 +02:00
afc87ddc9e
refactor: do not generate i18n msgid in dashboard schema json
...
dashboard has its own mind
now i18n is only used to generate docs:
1. runtime swagger spec at /api-docs
2. build-time config schema
both result in the resolved i18n text, but not msgid
2024-04-11 14:09:22 +02:00
3b7cade671
chore: 5.6.1-beta.1
2024-04-11 13:49:40 +02:00
eac25194e5
refactor: resolve some old TODOs
...
move dashboard schema generation code to the right module
2024-04-11 13:41:34 +02:00
e7a4210943
Merge pull request #12858 from zmstone/0410-fix-variform-number-handling
...
fix(variform): allow numbers to be numbers
2024-04-11 11:24:55 +02:00
834bddadad
test: delete flaky test for now
2024-04-11 09:39:17 +02:00
e6330dddec
fix(variform): allow numbers to be numbers
2024-04-10 11:57:45 +02:00
2fea651d1d
Merge pull request #12851 from zmstone/0327-feat-add-emqx_variform
...
emqx_variform for string substitution and transform
2024-04-10 10:18:40 +02:00
91ffc95f29
Merge pull request #12855 from JimMoen/fix-share-queue-format
...
fix(mgmt): $queue shared topics format in mgmt topics api
2024-04-10 10:52:48 +08:00
5f8986209e
Merge pull request #12842 from lafirest/fix/iotdb
...
fix(iotdb): correctly handle undefined value of bool type
2024-04-10 09:49:16 +08:00
fae9005f87
test(emqx_mgmt_cli): test that replicants do not join a left core node
2024-04-09 19:06:13 +03:00
55179ccfed
chore: update ekka to 0.19.3
...
Included updates:
- https://github.com/emqx/mria/pull/178
2024-04-09 19:04:38 +03:00
31142df5cf
fix: default value of stop_action_after_template_rendering to true
...
This commit changes the default value for the
stop_action_after_template_rendering option of the apply rule HTTP API
endpoint so that it is true instead of false.
2024-04-09 16:20:37 +02:00
7bcd553786
test: move test case with rule_engine dep from emqx to emqx_rule_engine
2024-04-09 14:08:46 +02:00
3898950017
test: fix test by adding dependency
2024-04-09 11:38:35 +02:00
958748cf7f
test: fix inter test suite problem
2024-04-09 11:10:39 +02:00
03a9c46ca7
fix(sys_topic): format shared topics
2024-04-09 17:06:36 +08:00
47e0f3bb1f
fix(mgmt): $queue shared topics format in mgmt topics api
2024-04-09 17:06:32 +08:00
8bbfa9ca8b
Merge pull request #12839 from keynslug/feat/EMQX-12110/rebalancing-impl
...
feat(dsrepl): implement membership changes and rebalancing
2024-04-09 10:46:31 +02:00
53b78086ed
chore: fix xref checks
2024-04-09 09:34:05 +02:00
8381132919
chore: bump emqx_conf to 0.1.36
2024-04-09 14:44:11 +08:00
a1495689c0
fix: clean self node's cluster commit when leave cluster
2024-04-09 14:13:25 +08:00
6cb00cc8c7
Merge pull request #12844 from JimMoen/EMQX-12012/cpu-use-idle-two-decimal-places
...
fix: cpu usage and idle use two decimal places
2024-04-09 10:11:15 +08:00
d12e907209
fix(dsrepl): correctly handle ra membership change command results
...
Before this change, results similar to `{error, {no_more_servers_to_try,
[{error, nodedown}, {error, not_member}]}}` were considered retryable
failures, which is incorrect.
2024-04-08 22:44:34 +02:00
3223797ae5
fix(dsrepl): attempt leadership transfer before server removal
...
This should make it much less likely to hit weird edge cases that lead
to duplicate Raft log entries because of client retries upon receiving
`shutdown` from the leader being removed.
2024-04-08 22:43:58 +02:00
1e95bd4da6
test(dsrepl): test unresponsive nodes removal / node restarts
2024-04-08 21:27:56 +02:00
41677eb785
refactor: make elvis happy
2024-04-08 21:25:58 +02:00
bf12efac6d
fix(variform): add basic tests
2024-04-08 21:08:43 +02:00
7a836317ac
fix(dsrepl): trigger unfinished shard transition upon startup
...
Also provide a trivial API to trigger them by hand.
2024-04-08 16:12:42 +02:00
75bb7f5cdc
fix(dsrepl): retry only `{add, Site}` crashed membership transitions
...
To minimize the potential negative impact of removal transitions that
crash for some unknown and unusual reasons.
2024-04-08 16:04:33 +02:00
9628a00a82
docs(emqx_rule_api apply rule): fix doc strings
2024-04-08 15:34:29 +02:00
ba96edb061
fix(clients api): use alternative base64 function for OTP 25
...
Fixes https://github.com/emqx/emqx/pull/12798#discussion_r1555524603
2024-04-08 10:23:10 -03:00
600526a0e4
test(emqx_bridge_http_SUITE): test case after name change
2024-04-08 14:54:33 +02:00
02ee873094
docs(emqx_rule_api_schema): fix type spec
2024-04-08 13:42:15 +02:00
4c0cc079c2
fix(dsrepl): apply unnecessary rebalancing transitions cleanly
2024-04-08 13:25:45 +02:00
dcde30c38a
test(dsrepl): add two more testcases for rebalancing
2024-04-08 13:22:31 +02:00
b57725f996
test(resource_SUITE): do test case fixes needed due to rule tracing work
2024-04-08 12:41:35 +02:00
79440064fe
style: fix problems reported by elvis
2024-04-08 11:03:55 +02:00
e11c4a9c83
Merge pull request #12826 from emqx/import-source-bridges
...
fix: source bridges missing after restore the backup files
2024-04-08 16:02:15 +08:00
3e87d4bf9f
Merge pull request #12772 from killme2008/feature/upgrade-greptimedb-ingester-ci
...
feat: update greptimedb client lib and ci version
2024-04-08 09:02:03 +02:00
282cbb18be
fix: cpu usage and idle use two decimal places
...
- prometheus
- opentelemetry
2024-04-08 14:14:09 +08:00
2ace9bb893
chore(dsrepl): sprinkle few comments and typespecs for exports
2024-04-07 22:51:56 +02:00
ecaad348a7
chore(dsrepl): update few outdated comments / TODOs
2024-04-07 22:51:56 +02:00
6293efb995
fix(dsrepl): retry crashed membership transitions
2024-04-07 22:51:56 +02:00
826ce5806d
fix(dsrepl): ensure that new member UID matches server's UID
...
Before that change, UIDs supplied in the `ra:add_member/3` were not
the same as those servers were using. This haven't caused any issues
for some reason, but it's better to ensure that UIDs are the same.
2024-04-07 22:31:24 +02:00
e89dc32c90
ci: run emqx_management both with ee and ce profile
2024-04-07 18:33:52 +08:00
f40e47365c
fix(iotdb): correctly handle undefined value of bool type
2024-04-07 17:26:15 +08:00
1c81c79a2c
chore: add testcase for importing retained msgs and sources
2024-04-07 17:24:26 +08:00
9d1a69aaa9
fix: cannot import retained messages
2024-04-07 17:03:14 +08:00
5479932190
feat(apply rule test): make option to stop action after render work
...
This commit makes the apply rule HTTP API option to stop an action work
for the HTTP action, and adds infrastructure that makes it easy to add
this functionality to other actions.
2024-04-06 17:21:12 +02:00
ef705c2285
feat: add apply rule API, clientid/ruleid tracing for rule and connector
...
This commit adds:
* Support for forwarding the rule id and client id to the connector so
that events such as template rendered successfully can be traced.
* HTTP API for for applying/activating a rule with the given context
2024-04-06 17:20:47 +02:00
04ba2aaf8a
Merge pull request #12830 from thalesmg/async-channel-hc-m-20240404
...
feat(resource): non-blocking channel health checks
2024-04-05 14:24:09 -03:00
556ffc78c9
feat(dsrepl): implement membership changes and rebalancing
2024-04-05 18:57:28 +02:00
d6058b7f51
feat(dsrepl): allow to subscribe to DB metadata changes
...
Currently, only shard metadata changes are announced to the
subscribers.
2024-04-05 17:40:55 +02:00
a07295d3bc
fix(ds): address shards in the supervisor properly
2024-04-05 17:40:38 +02:00
3f7b14c861
Merge pull request #12833 from ieQu1/dev/ds-cluster-api
...
feat(ds): Add REST API for durable storage
2024-04-05 17:33:20 +02:00
2504b8126b
feat(ds): Pass mgmt_ds REST API calls to the application
2024-04-05 15:22:06 +02:00
46261440cb
feat(ds): Add a CLI for managing DB replicas
2024-04-05 15:22:06 +02:00
a62db08676
feat(ds): Add REST API for durable storage
2024-04-05 15:22:06 +02:00
d09787d1a6
fix(ds): Fix return types in replication_layer_meta
2024-04-05 15:22:06 +02:00
be47fe49ad
chore: bump ecql version to 0.7.0
...
PR: https://github.com/emqx/ecql/pull/13
No functional changes, just switch gen_fsm to gen_statem.
2024-04-05 13:31:33 +02:00
70396e9766
Merge pull request #12825 from keynslug/feat/EMQX-12110/repl-meta-api
...
feat(dsrepl): add APIs to manage DB replication sites
2024-04-04 22:32:03 +02:00
df6c5b35fe
feat(dsrepl): add more primitive operations to modify DB sites
2024-04-04 21:22:49 +02:00
bb8ffee18c
feat(dsrepl): add API to get current DB replication sites
2024-04-04 21:22:02 +02:00
ad52f7838e
feat(dsrepl): add APIs to manage DB replication sites
2024-04-04 21:22:01 +02:00
60cad74286
feat(resource): non-blocking channel health checks
...
Fixes https://emqx.atlassian.net/browse/EMQX-12015
Continuation of https://github.com/emqx/emqx/pull/12812
2024-04-04 16:13:30 -03:00
8d58b40f33
Merge pull request #12831 from thalesmg/ds-checkpoint-clean-m-20240404
...
feat(ds): clear all checkpoints when (re)starting storage layer
2024-04-04 15:41:50 -03:00
217b35bce5
Merge pull request #12798 from thalesmg/ds-client-api-v2-m-20240327
...
feat(client mgmt api): add cursor-based list API
2024-04-04 15:10:49 -03:00
c57c36adb2
feat(ds): clear all checkpoints when (re)starting storage layer
...
Fixes https://emqx.atlassian.net/browse/EMQX-12143
2024-04-04 14:05:52 -03:00
59a442cdb5
feat(rule trace): add support for ruleid as a trace type
2024-04-04 14:55:32 +02:00
069cd4fbb4
Merge pull request #12812 from thalesmg/async-res-manager-health-check-m-20240328
...
feat(resource manager): perform non-blocking health checks
2024-04-04 09:07:02 -03:00
0e79b543cf
refactor: move variform to emqx_utils
2024-04-04 11:10:56 +02:00
f37ed3a40a
fix(ds): Limit the number of retries in egress to 0
2024-04-03 16:38:49 +02:00
319ec50c0d
fix: source bridges missing after restore the backup files
2024-04-03 18:26:51 +08:00
2bbfada7af
fix(ds): Make async batches truly async
2024-04-03 11:57:47 +02:00
92ca90c0ca
fix(ds): Improve egress logging
2024-04-03 11:57:47 +02:00
ae5935e7f7
test(ds): Attempt to stabilize metrics_worker tests in CI
2024-04-02 19:14:10 +02:00
4382971443
fix(ds): Preserve errors in the egress
2024-04-02 16:47:43 +02:00
94ca7ad0f8
feat(ds): Report counters for LTS storage layout
2024-04-02 16:47:43 +02:00
f14c253dea
fix(prometheus): Don't add DS metrics when feature is disabled
2024-04-02 16:47:43 +02:00
b379f331de
fix(sessds): Handle errors when storing messages
2024-04-02 16:47:41 +02:00
f41e538526
feat(sessds): Observe next time
2024-04-02 16:45:52 +02:00
b9ad241658
feat(sessds): Add metrics for the number of persisted messages
2024-04-02 16:45:52 +02:00
75b092bf0e
fix(ds): Actually retry sending batch
2024-04-02 16:45:49 +02:00
0de255cac8
feat(ds): Report egress flush time
2024-04-02 16:25:04 +02:00
044f3d4ef5
fix(ds): Don't reverse entries in the atomic batch
2024-04-02 16:25:04 +02:00
606f2a88cd
feat(ds): Add egress metrics
2024-04-02 16:25:04 +02:00
c9de336234
feat(ds): Add metrics worker to the builtin db supervision tree
2024-04-02 16:25:04 +02:00
d8204021dc
refactor(metrics): Move metrics worker to emqx_utils application
2024-04-02 16:25:04 +02:00
2097e854fc
feat(client mgmt api): add cursor-based list API
...
Fixes https://emqx.atlassian.net/browse/EMQX-12028
2024-04-02 10:55:28 -03:00
778e897f1f
chore(dsrepl): describe snapshot ownership and few shortcomings
2024-04-02 13:48:51 +02:00
c666c65c6a
test(ds): factor out storage iteration into helper module
2024-04-02 13:48:51 +02:00
Ilia Averianov