Commit Graph

12500 Commits

Author SHA1 Message Date
zhongwencool a49e5877ec chore(debug): use redbug for default trace tools 2024-06-17 09:28:23 +08:00
zmstone 8daaf7b727
Merge pull request #13233 from zmstone/0611-expose-regex_extract-func
fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-16 16:16:48 +02:00
zmstone 2b7f3a597f
Merge pull request #13245 from qzhuyan/merge/william/sync-rel57-to-master
sync release57 to master
2024-06-15 09:13:55 +02:00
zhongwencool d433fc7c30
Merge pull request #13236 from zhongwencool/bug-schema-validation
fix: some minor bug fixes
2024-06-15 06:40:28 +08:00
Thales Macedo Garitezi 198ccd320b fix: add `emqx_s3` to `reboot_lists.eterm`
Otherwise, it's unconditionally started as a permanent application instead of being
controlled by `emqx_machine`.

```sh
ͳ rg start_boot _build/emqx-enterprise/rel/emqx/releases/5.7.0-g4e6be8a6/start.script | rg emqx_s3
     {apply,{application,start_boot,[emqx_s3,permanent]}},
```
2024-06-14 15:06:18 -03:00
Thales Macedo Garitezi 6e702cd16c fix(postgres): fix usage by authn/authz modules when prepared statements are disabled
Fixes https://emqx.atlassian.net/browse/EMQX-12550
2024-06-14 12:19:46 -03:00
zmstone 626aae6edf chore: fix bad conflict resolution 2024-06-14 16:57:53 +02:00
zhongwencool d95f17fe77 chore: revert ULOG/ELOG 2024-06-14 22:11:27 +08:00
Thales Macedo Garitezi fd5e844cc1 fix(postgres): correctly handle batch queries with disabled prepared statements
Fixes https://emqx.atlassian.net/browse/EMQX-12549
2024-06-14 11:10:21 -03:00
zhongwencool f713f13b2c test: generate dispatch.eterm in dashboard test 2024-06-14 21:52:58 +08:00
Thales Macedo Garitezi 9b3c806ba7 feat(message transformation): implement dryrun endpoint
Follow up to https://github.com/emqx/emqx/pull/13199
2024-06-14 10:19:36 -03:00
zhongwencool a6e3a09118 feat: make the dashboard restart quicker 2024-06-14 18:22:34 +08:00
JimMoen 0f0e7d18db
Merge pull request #13261 from JimMoen/fix-typo
chore: fix typo
2024-06-14 16:55:35 +08:00
JimMoen 6823c79ae0
chore: fix typo 2024-06-14 16:04:11 +08:00
William Yang b86d631744 test: fix tc t_handle_outing_non_utf8_topic 2024-06-14 08:17:19 +02:00
William Yang d39b8caff9 Merge branch 'release-57' into merge/william/sync-rel57-to-master 2024-06-14 08:17:09 +02:00
lafirest afd4b46f72
Merge pull request #13225 from lafirest/fix/mysql_redact
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-14 09:19:21 +08:00
Thales Macedo Garitezi 01635722e9
docs: improve descriptions
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2024-06-13 18:34:44 -03:00
Thales Macedo Garitezi 52e6c88941 feat: enforce singleton discovery strategy when using DS `builtin_local` backend
Fixes https://emqx.atlassian.net/browse/EMQX-12493
2024-06-13 17:04:23 -03:00
Thales Macedo Garitezi cba5c7bb45 refactor: rm dead code 2024-06-13 16:57:34 -03:00
Thales Macedo Garitezi 4d876f2af2 chore: bump ekka -> 0.19.4
https://github.com/emqx/ekka/pull/236
2024-06-13 16:35:58 -03:00
Thales Macedo Garitezi 3adf64e637
Merge pull request #13240 from thalesmg/ds-cluster-purge-m-20240612
feat: support purging durable sessions during cluster purge
2024-06-13 09:30:18 -03:00
Thales Macedo Garitezi 6ccf1dcbf9
Merge pull request #13238 from thalesmg/fix-authz-http-content-type-handling-r57-20240612
fix(http authz): handle unknown content types in responses
2024-06-13 09:10:43 -03:00
Thales Macedo Garitezi 62cf277cd7
Merge pull request #13235 from thalesmg/fix-template-types-r57-20240611
chore: change types of mysql and mongodb fields to `template()`
2024-06-13 09:08:40 -03:00
Thales Macedo Garitezi c81a3ebc0a
Merge pull request #13234 from thalesmg/fix-clients-api-client-attrs-union-r57-20240611
fix(client mgmt api): allow projecting `client_attrs` from client fields
2024-06-13 09:08:29 -03:00
William Yang 1664ea4ad4 Revert: TLS partial-chain and keyUsage #12955 #12977
This reverts commit 28b17a2562.
This reverts commit 01467246fc.
This reverts commit c3f8ba5762.
This reverts commit 1a4a4bb3a5.
This reverts commit fb30207ef3.
This reverts commit 337c230e79.
This reverts commit 3a674f44f1.
This reverts commit 70ffd77f99.
This reverts commit 03b0935564.
This reverts commit 650cf4b27e.
This reverts commit 43ad665dcf.
This reverts commit a29a43e5fc.
This reverts commit 4e9c1ec0c9.
This reverts commit 8eb463c58d.
This reverts commit 90430fa66d.
This reverts commit eb1ab9adfe.
This reverts commit 8bc3a86f63.
This reverts commit fa4357ce89.
This reverts commit 0b95a08d32.
2024-06-13 13:46:45 +02:00
William Yang 723144199e
Merge pull request #13211 from qzhuyan/fix/william/rel57/ee-tls
Fix/william/rel57/ee tls
2024-06-13 13:19:44 +02:00
Andrew Mayorov 0f53b33417
chore(s3): make schema fields with defaults optional 2024-06-13 11:57:36 +02:00
zhongwencool 0eee2edd28 test: fix api_config SUITE failed 2024-06-13 14:55:24 +08:00
zhongwencool 2b1fa1711b fix: observer command crash when can't find object code 2024-06-13 14:55:23 +08:00
zhongwencool 623845d07d fix: don't allow set active_n to negative int 2024-06-13 14:55:23 +08:00
zhongwencool 825bfe4717 fix: load bad configs return unknown msg 2024-06-13 14:55:23 +08:00
zhongwencool 5d3f464ec3 fix: ws/wss's max_frame_size should > 0 2024-06-13 14:55:23 +08:00
zhongwencool 800a79adde test: test failed with_mfa 2024-06-13 09:50:52 +08:00
Thales Macedo Garitezi 2c264d9a4b fix(http authz): handle unknown content types in responses
Fixes https://emqx.atlassian.net/browse/EMQX-12530
2024-06-12 14:31:12 -03:00
zhongwencool c72b455a83 feat: add hidden option to record MFA and line in the log is very useful for debug. 2024-06-12 22:51:45 +08:00
Thales Macedo Garitezi d0e6f22a79 feat: support purging durable sessions during cluster purge
Fixes https://emqx.atlassian.net/browse/EMQX-12405
2024-06-12 11:19:53 -03:00
JimMoen daeb10151b
chore: fix typos 2024-06-12 19:06:34 +08:00
JimMoen cde4cb1358
fix: metrics macro with description 2024-06-12 19:06:33 +08:00
JimMoen 756797b258
refactor: gen metric schema with desc from macros 2024-06-12 19:04:39 +08:00
JimMoen 263d2dbae2
fix: inc metric `'client.auth.anonymous'` when client anonymous 2024-06-12 19:04:39 +08:00
JimMoen 9a78a6f640
refactor: mv metrics macros in hrl file 2024-06-12 19:04:29 +08:00
JimMoen cbac4019b3
fix: rm havn't used counter idx 2024-06-12 19:01:50 +08:00
Thales Macedo Garitezi 46ab3be1f4 chore: change types of mysql and mongodb fields to `template()`
Fixes https://emqx.atlassian.net/browse/EMQX-12395
2024-06-11 17:25:53 -03:00
Thales Macedo Garitezi 0874768c1d fix(client mgmt api): allow projecting `client_attrs` from client fields
Fixes https://emqx.atlassian.net/browse/EMQX-12511
2024-06-11 17:14:33 -03:00
zmstone 8a283b1cc0
Merge pull request #13180 from zmstone/0604-optimize-tcp-ssl-connection-data-send-performance
0604 optimize tcp ssl connection data send performance
2024-06-11 22:06:57 +02:00
zmstone 09f91159c9 fix(emqx_rule_funcs): expose regex_extract function to rule engine
previoulsy only available in variform expressions,
now made available for rule-engine
2024-06-11 21:31:13 +02:00
zmstone 51810c6919
Merge pull request #13229 from zmstone/0611-auth-with-raw-cert
feat(authn): support `${cert_pem}` placeholder
2024-06-11 21:16:01 +02:00
Thales Macedo Garitezi db572d35a7 Merge remote-tracking branch 'origin/release-57' into sync-r57-m-20240611 2024-06-11 15:34:54 -03:00
zmstone 751f7a24e9 feat(authn): support ${cert_pem} placeholder 2024-06-11 19:54:24 +02:00
Thales Macedo Garitezi 337009c3a0 fix: declare `emqx_schema_registry` as a dependency of `emqx_schema_validation` 2024-06-11 14:03:27 -03:00
Thales Macedo Garitezi 0f9c3b4cea feat(schema validation): check references schema names and types before changing config
Fixes https://emqx.atlassian.net/browse/EMQX-12368
2024-06-11 14:03:26 -03:00
Thales Macedo Garitezi 03b226248a feat(schema registry): add check for inner types
Currently, only `protobuf` has any.
2024-06-11 13:50:11 -03:00
Thales Macedo Garitezi c70e8252fe fix: declare `emqx_schema_registry` as a dependency of `emqx_message_transformation` 2024-06-11 13:35:02 -03:00
Thales Macedo Garitezi b2d716909f
Merge pull request #13200 from thalesmg/update-erlazure-no-gen-server-m-20240606
chore: refactor azure blob connector to use new `erlazure` without `gen_server`
2024-06-11 13:30:14 -03:00
Thales Macedo Garitezi 686bcc8a48
Merge pull request #13207 from thalesmg/republish-metrics-m-20240607
fix(rule actions): check `republish` action result for metrics
2024-06-11 13:29:58 -03:00
Thales Macedo Garitezi 2c6aa80b3c
Merge pull request #13221 from thalesmg/refactor-sv-topic-index-r57-20240610
refactor(schema validation): module organization and position indexing
2024-06-11 13:29:48 -03:00
Thales Macedo Garitezi 83ff07ade0
Merge pull request #13199 from thalesmg/message-transformation-r57-20240604
feat: implement message transformation
2024-06-11 13:29:25 -03:00
Thales Macedo Garitezi ab5329be02
Merge pull request #13220 from thalesmg/refactor-channel-hc-opt-r57-20240610
fix: remove another redundant health check
2024-06-11 12:22:43 -03:00
Andrew Mayorov 2f0eb3560b
Merge pull request #13227 from keynslug/fix/EMQX-12533/bad-tpl
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 15:39:12 +03:00
Andrew Mayorov d4fb812352
fix(aggreg): anticipate and handle delivery startup errors 2024-06-11 13:21:41 +02:00
Andrew Mayorov 29fc30ea69
fix(bridge-s3): validate aggreg key template before adding a channel 2024-06-11 13:21:41 +02:00
Andrew Mayorov fb0da9848c
feat(tpl): add separate `placeholders/1` function
The purpose is to have a clearer view of placeholders used in a
template, without going the usual `render(Template, #{})` route that is
actually subtly misleading: it won't mention that `${}` / `${.}`
placeholder has been used.

Also unify handling of `${}` / `${.}` in a couple of places.
2024-06-11 11:42:43 +02:00
Kjell Winblad 9d22089104
Merge pull request #13173 from kjellwinblad/kjell/pgsql_timestamp_encode_etc/EEC-1036
fix(pgsql connector): handle prepared statement already exists
2024-06-11 10:53:30 +02:00
William Yang 44258204bd chore: move tls_certs test helper to apps/emqx 2024-06-11 10:06:18 +02:00
firest e64f60b73f fix(auth): redact sensitive data for the authn/authz APIs 2024-06-11 16:05:11 +08:00
zhongwencool b657dc537c
Merge pull request #13156 from savonarola/0529-fix-dashboard-monitor
fix(dashboard_monitor): correctly merge old and new sample data
2024-06-11 10:33:15 +08:00
Thales Macedo Garitezi 05ebb17cd6 refactor: index positions rather than names
Addresses https://github.com/emqx/emqx/pull/13199#discussion_r1633096025
2024-06-10 15:54:37 -03:00
Thales Macedo Garitezi 5eff4a7544 refactor(schema validation): improve api and module organization
To conform to comment from another PR:
https://github.com/emqx/emqx/pull/13199#discussion_r1632658742

... since schema validation is the "inspiration" for message transformation.
2024-06-10 15:46:28 -03:00
Thales Macedo Garitezi d93c8540c8 feat: pass along peername down to transformation context 2024-06-10 15:35:35 -03:00
Thales Macedo Garitezi 6e0ef893f4 feat: pass along client attributes down to message transformation context 2024-06-10 15:27:04 -03:00
Thales Macedo Garitezi 3c501e4f2a fix: remove another redundant health check
Fixes https://emqx.atlassian.net/browse/EMQX-12521
2024-06-10 15:13:34 -03:00
Thales Macedo Garitezi 1a20a40218
Merge pull request #13209 from thalesmg/rm-redundant-hc-r57-20240607
chore: remove redundant health check call
2024-06-10 14:58:48 -03:00
Thales Macedo Garitezi e586178479 chore: refactor azure blob connector to use new `erlazure` without `gen_server`
https://github.com/dkataskin/erlazure/pull/43 removes unnecessary usage of `gen_server`
from the driver.
2024-06-10 13:42:28 -03:00
Thales Macedo Garitezi 5629fe60c1 refactor: index positions rather than names 2024-06-10 13:33:22 -03:00
Thales Macedo Garitezi 8c7a7cf9db refactor: apply review remarks 2024-06-10 11:17:13 -03:00
Thales Macedo Garitezi e54cf2f218 refactor: use option to skip rendering as string 2024-06-10 10:49:39 -03:00
Thales Macedo Garitezi d34558954d refactor: improve api and module organization 2024-06-10 10:40:53 -03:00
Thales Macedo Garitezi 4aa1c3842f fix: declare `emqx` as a dependency in `.app.src` 2024-06-10 09:53:29 -03:00
Thales Macedo Garitezi aa416cc0ad docs: fix stale comments and dates 2024-06-10 09:53:02 -03:00
zmstone 7a09b0470d chore: cosmetic changes 2024-06-10 14:22:25 +02:00
Kjell Winblad a885f0b41a test(emqx_bridge_pgsql_SUITE): call test janitor 2024-06-10 13:45:27 +02:00
William Yang a1aa9a4375 fix(ee): emqx no longer deps on emqx_auth_ext 2024-06-10 10:32:01 +02:00
zmstone 4347f3de3e fix(bridge/mqtt): respect client ID prefix 2024-06-10 10:27:45 +02:00
William Yang 76cfc309a9 feat(tls): partial_chain not required 2024-06-10 10:25:44 +02:00
William Yang 7c37bf9965 feat(tls): ee only: TLS partial_chain and Keyusage 2024-06-10 10:24:48 +02:00
William Yang e60380d205 chore: fix nit 2024-06-10 09:55:12 +02:00
William Yang e9b813d8ef chore: fix test 2024-06-10 09:54:54 +02:00
William Yang 1739bc0c24 feat(partial_chain): gateway support 2024-06-10 09:54:44 +02:00
zmstone 45dd7816d7 Merge remote-tracking branch 'origin/release-57' 2024-06-09 10:11:15 +02:00
William Yang abbf2ef62f chore(TLS-chain-test): update for OTP 26 2024-06-08 00:16:06 +02:00
William Yang 221b748b0f test(partial_chain): update tcs for OTP-25 2024-06-08 00:16:01 +02:00
William Yang 463d1a1875 fix(test): tls_verify_partial_chain 2024-06-08 00:15:54 +02:00
William Yang 788cdbc6dd fix(listener): remove partial_chain in wss opts 2024-06-08 00:15:51 +02:00
William Yang 3fc99315e0 chore: happy elvis 2024-06-08 00:15:45 +02:00
William Yang c5dccdf526 feat(tls): update schema for TLS keyusage 2024-06-08 00:15:43 +02:00
William Yang aa25e3badd fix(tls): undefined keyusage 2024-06-08 00:15:39 +02:00
William Yang 9e5cfea8c0 test(tls): verify peer keyusage 2024-06-08 00:15:36 +02:00
William Yang c5657029ab feat(config): partial_chain 2024-06-08 00:15:34 +02:00
William Yang f7ff9496e6 test: port listener tls partial_chain 2024-06-08 00:15:31 +02:00
William Yang 1ce13242a8 feat(tls): port partial_chain, part 1 2024-06-08 00:15:28 +02:00
Kjell Winblad c13631102e fix(PgSQL connector): typo in function name 2024-06-07 19:40:18 +02:00
Andrew Mayorov b2f027c974
Merge pull request #13197 from keynslug/fix/EMQX-12522/conv-ssl
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-07 19:45:53 +03:00
Thales Macedo Garitezi 39615e1cb6 fix(rule actions): check `republish` action result for metrics
Fixes https://emqx.atlassian.net/browse/EMQX-12328
2024-06-07 11:17:43 -03:00
Thales Macedo Garitezi 28a3c77e88 chore: remove redundant health check call
Since `emqx_resource_manager:channel_health_check` already does a base resource health
check, the call was repeated.
2024-06-07 11:07:12 -03:00
Kjell Winblad e63dcc84b0 fix: unused variable and better error message 2024-06-07 12:20:42 +02:00
Kjell Winblad a3f3f96781 test(emqx_bridge_v2_pgsql_SUITE): fix broken test case
We have to delete the connector and action created by the previous step
of the test case so we don't get a name conflict.
2024-06-07 12:03:34 +02:00
JimMoen 78a196e215
Merge pull request #13189 from JimMoen/fix-apply-sqlserver-template
fix(utils_sql): improve insert sql regular expression
2024-06-07 13:50:19 +08:00
lafirest eb215726d5
Merge pull request #13195 from lafirest/fix/typos
fix: fix typos
2024-06-07 09:04:16 +08:00
Thales Macedo Garitezi ef36350bf6 feat: implement message transformation
Fixes https://emqx.atlassian.net/browse/EMQX-12517
2024-06-06 17:51:08 -03:00
Andrew Mayorov 8a85d7cb5a
fix(bridge-s3): pass SSL options through `convert_certs/2` 2024-06-06 19:37:38 +02:00
Andrew Mayorov c17c788438
feat(connector): allow delegate `pre_config_update` to impls 2024-06-06 19:37:34 +02:00
zmstone ebf17c8143 Merge remote-tracking branch 'origin/release-57' into 0606-merge-release-57-to-master 2024-06-06 17:43:36 +02:00
firest 3ae26c8a54 chore: update changes 2024-06-06 18:13:24 +08:00
firest 5532c7b0a6 fix(authz_mnesia): add a soft limit in the API for the length of ACL rules 2024-06-06 16:22:53 +08:00
zmstone 2525bd0c16 perf: eliminate the loop-back fake async inet_reply message
Now that EMQX is on OTP 26 by default, the fake inet_reply
message when the send call returns ok only adds overhead.

For TCP (gen_tcp module), OTP 26 no longer supports direct port_command
so we had to swith to gen_tcp:send, then fake an async reply:
see https://github.com/emqx/esockd/pull/181

For TLS (ssl module), it has never supported async send anyway.

This commit switches to call Transport:send/2 directly,
this should result in one less loop-back message for each message.

For OTP 25 on which EMQX can still be compiled, one will have to
suffer the performance penalty for gen_tcp:send function to
do a non-optimized 'receive' for the send result.
2024-06-06 09:35:51 +02:00
zmstone 5c3f5d8085 perf: do not call inet getstat before each and every send
In a stress test environment, when alarm is enabled, there were
a lot of long_schedule warnings with the stacktrace pointing to
congestion alarm based inet:getstat function.
When alarm is disabled, the one single client is able to
hold 40,000 QoS 1 messages per second (inflight=32) throughput.
When alarm is enabled, its mqueue overflows.

This commit removes the call before each and every data send,
so to rely on emit_stats timer to trigger the congestion check.
2024-06-06 09:35:51 +02:00
firest d4118c6e8e fix: fix typos 2024-06-06 11:19:40 +08:00
JimMoen dff31b293c
perf(utils_sql): pre-build regexp 2024-06-06 11:01:38 +08:00
zmstone c41be7ee22 chore: fix dialyzer warning 2024-06-05 23:37:21 +02:00
Ilia Averianov dc2e6d1695
Merge pull request #13163 from savonarola/0523-ds-shared-subs-dispatch
Inject shared subcription handling into durable session
2024-06-05 23:59:50 +03:00
Serge Tupchii 159da912c2 feat(emqx_dashboard_swagger): apply field validator in request body check 2024-06-05 19:35:04 +03:00
JimMoen 144264e0d8
test: add utils_sql test cases 2024-06-06 00:30:18 +08:00
JimMoen 627b8c45d2
fix(utils_sql): improve insert sql regular expression 2024-06-06 00:30:17 +08:00
Ilya Averyanov b075b7120c feat(sessds): implement dispatchig between CE/EE shared sub agents 2024-06-05 19:17:58 +03:00
Ilya Averyanov 60556abc4f fix(dashboard_monitor): correctly merge old and new sample data 2024-06-05 18:49:52 +03:00
Kjell Winblad 2956e849eb fix(pgsql connector): better msg when failing to remove statement 2024-06-05 16:58:14 +02:00
Kjell Winblad 336089f8a7 fix: bug found by dialyzer and make test case cleaner 2024-06-05 15:53:02 +02:00
Thales Macedo Garitezi ae0379f974
Merge pull request #13181 from thalesmg/force-stop-connector-r57-20240604
fix(resource manager): force kill process if stuck when stopping/removing
2024-06-05 09:08:30 -03:00
lafirest d79193fde4
Merge pull request #13177 from lafirest/sync-authn-com
feat: sync `client.check_authn_complete` to release-57
2024-06-05 19:51:10 +08:00
Thales Macedo Garitezi 5eab221f7c
Merge pull request #13175 from thalesmg/disable-prepared-statements-postgres-r57-20240603
feat(postgres): add `disable_prepared_statements` option
2024-06-04 12:29:53 -03:00
Thales Macedo Garitezi f17aefe3d7 fix(action/source api): improve returned error message on timeout 2024-06-04 12:04:55 -03:00
zmstone cda6d5f636
Merge pull request #13172 from zmstone/0603-add-redis-arg-formatter-to-rule-engine-funcs
feat(redis): add a rule function to help formatting redis args
2024-06-04 16:56:14 +02:00
Thales Macedo Garitezi 3013189cd7 fix(resource manager): force kill process if stuck when stopping/removing
Fixes https://emqx.atlassian.net/browse/EMQX-12357
2024-06-04 11:38:24 -03:00
Ilya Averyanov 9cdfbb0845 feat(sessds): make ds session be aware of shared subscriptions 2024-06-04 17:15:30 +03:00
Thales Macedo Garitezi c07bc68e6f feat(postgres): add `disable_prepared_statements` option
Fixes https://emqx.atlassian.net/browse/EMQX-12496

Some Postgres connections, such ones made to [PGBouncer](https://www.pgbouncer.org/) or
[Supabase in Transaction Mode](https://supabase.com/), do not support some session
features like prepared statements.
2024-06-04 10:11:25 -03:00
firest 87384cae32 fix(events): call `client.check_authn_complete` even if authentication fails 2024-06-04 17:55:39 +08:00
firest 92d26ff27b chore: update change 2024-06-04 17:55:30 +08:00
firest e4e53844d5 feat(events): add new hook && event `client.check_authn_complete` 2024-06-04 17:55:21 +08:00
Ilia Averianov dc95218c68
Merge pull request #13144 from savonarola/0528-increase-bridge-buffer-drop-log-level
chore(bridges): increase log level for bridge buffer overflow reports
2024-06-04 12:37:50 +03:00
Kjell Winblad 572ca6433e fix(pgsql connector): improvements due to suggestions from @thalesmg
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-06-04 10:11:45 +02:00
zhongwencool a8a67a2ac9
Merge pull request #13151 from zhongwencool/authz-trace-log
chore: make authz's logs easier to understand
2024-06-04 11:12:09 +08:00
zmstone aa7ce1f641 fix(bridge/redis): add test case for map_to_redis_hset_args 2024-06-03 23:05:43 +02:00
Kjell Winblad f2ccfff803 fix(pgsql connector): handle prepared statement already exists
In a user's log file it was found that that the pgsql driver can end up
in a situation where the prepared statement for a channel/action is not
properly removed before a channel with the same name as the prepared
statement is added to the connector. This commit handles this by
attempting to remove the old prepared statement if one already exists
when adding channel.

Related issue:
https://emqx.atlassian.net/browse/EEC-1036
2024-06-03 16:14:26 +02:00
zmstone e3ed7b59dd feat(redis): add a rule function to help formatting redis args
The new function named 'map_to_redis_hset_args' can be used to format
a map's key-value pairs into redis HSET (or HMSET) arg list.

This new function is dedicated for redis to avoid abuse for other
data integrations.
2024-06-03 14:09:15 +02:00
Thales Macedo Garitezi c5547543e0
Merge pull request #13069 from thalesmg/azure-blob-storage-action-m-20240508
feat: implement azure blob storage action
2024-06-03 08:56:37 -03:00
Ilia Averianov 193978af90
Merge pull request #13168 from savonarola/0530-fix-mgmt-test
chore(tests): improve test stability
2024-06-03 14:44:12 +03:00
zhongwencool fda365a87b chore: make authz's logs easier to understand 2024-06-03 18:01:57 +08:00
JimMoen 21624bc865
fix: get plugin app vsn from field `rel_apps` 2024-06-03 15:26:14 +08:00
zmstone b6ff67d712
Merge pull request #13093 from zmstone/0522-upgrade-to-brod-3.18
chore: upgrade to brod 3.18.0
2024-06-02 15:06:40 +02:00
JimMoen 8d22a5a3ae
Merge pull request #13167 from JimMoen/fix-rule-events-i18n-todo
fix: missing rule event i18n in zh_cn
2024-06-02 18:23:20 +08:00
zmstone 9637b14543 fix: upgrade to brod 3.18.0 to fix unnecessary rebalance 2024-06-01 10:57:24 +02:00
Ilya Averyanov 57741a41e9 chore(tests): improve test stability 2024-05-31 22:27:31 +03:00
zmstone fb07f40bea test: fix default value of retry_interval in test case 2024-05-31 17:53:12 +02:00
zmstone 20c92407c7 feat: change message retry_interval default value to infinity
Previous Setting: The default value for `retry_interval` was 30 seconds.

New Default: The default `retry_interval` has been changed to 'infinity'.
With this update, EMQX will not automatically retry message deliveries
by default.

Compliance and Compatibility: Aligning with MQTT specification
standards, in-session message delivery retries are not typically
compliant. We recognize that some users depend on this feature, so the
option to configure retries remains available for backward
compatibility.
2024-05-31 17:53:12 +02:00
zmstone 37ce8d93a1
Merge pull request #13164 from zmstone/0530-http-acl-compatibility-with-v4
0530 http acl compatibility with v4
2024-05-31 16:28:30 +02:00
Thales Macedo Garitezi 347971e24a fix: check `max_block_size` for direct transfers 2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi 4859cebd9f refactor: move `min_block_size` one level up 2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi 9a1d3ea04f chore: bump app vsns 2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi 616f14ae53 fix: avoid uploading blocks too frequently and splitting large buffers
Fixes https://github.com/emqx/emqx/pull/13069#discussion_r1613706747
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi af99829a21 refactor: move common templating logic to `emqx_connector_aggreg_buffer_ctx` 2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi c916c83c7c feat: implement azure blob storage action
Fixes https://emqx.atlassian.net/browse/EMQX-12280
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi 60d24c6ad5 refactor: simplify resource_opts definition 2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi 729441d1ce refactor(connector aggregator): move shared schemas to app 2024-05-31 11:07:44 -03:00
Ilya Averyanov db713cb656 chore(bridges): increase log level for bridge buffer overflow reports
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2024-05-31 16:35:18 +03:00
Thales Macedo Garitezi c289c7d065
Merge pull request #13148 from thalesmg/EMQX-12251-fix-conn-api-timeout-case-clause-r571-20240528
fix(connector api): handle `timeout` when waiting for connector status
2024-05-31 10:07:35 -03:00
zmstone 6fe8a09e97 fix(authz/http): rename PH_ACCESS to VAR_ACCESS 2024-05-31 14:49:54 +02:00
JimMoen 45e2c81ebc
fix: missing rule event i18n in zh_cn 2024-05-31 17:51:47 +08:00
Ivan Dyachkov 29ad07ce29 Merge remote-tracking branch 'upstream/release-57' into 0531-sync-release-57 2024-05-31 07:32:26 +02:00
zmstone ca32cf8d29 Merge branch 'emqx-12487-fix-authz-http-content-type' into 0530-http-acl-compatibility-with-v4 2024-05-30 22:30:20 +02:00
zmstone a00d9f17b1 chore: upgrade kafka producer lib wolff to 1.10.4
wolff-1.10.4 handles message_too_large error code from Kafka.
It tries to split the too-large batch into single-call batches
(for EMQX, one call batch is always one message), and retry.
In case a single-call batch is too large, EMQX should increment
a failure counter.
2024-05-30 15:40:10 +02:00
zmstone 460081a22e
Merge pull request #13131 from zmstone/0527-handle-unicode-in-template-strings
0527 handle unicode in template strings
2024-05-30 15:37:43 +02:00
Ilya Averyanov 97f9c81e19 feat(auth): add legacy ${access} placeholder 2024-05-30 15:47:48 +03:00
Ivan Dyachkov 0c4da98b52 chore: update deps 2024-05-30 11:53:00 +02:00
Ivan Dyachkov 1129c18330 fix(authz_http): fix content-type header in http request 2024-05-30 11:36:13 +02:00
zhongwencool 8ab7e75e64
Merge pull request #13139 from zhongwencool/influxdb-client-failed-msg-port
chore: improved InfluxDB connection error messaging
2024-05-30 10:57:21 +08:00
zmstone 532812f6f6 test: assert results 2024-05-29 22:10:34 +02:00
zmstone 9fe29bf5f7 chore: bump kafka bridge app vsn 2024-05-29 21:57:14 +02:00
zmstone 78a6100346 chore: fix app vsn bumps
only bug fixes so far
2024-05-29 21:56:22 +02:00
zmstone 1db9d54d17 fix: handle unicode in data integration template strings
The bug was introduced in 5.7.0 (#12893)
2024-05-29 21:53:07 +02:00
zmstone c54d25de98
Merge pull request #13070 from zmstone/0518-improve-kafka-connection-error-logs
0518 improve kafka connection error logs
2024-05-29 16:24:01 +02:00
Kjell Winblad 5017ef82a3
Merge pull request #13130 from kjellwinblad/kjell/fix_redis_trace_format/EMQX-12428
fix(redis trace): add separators in redis batch action trace
2024-05-29 15:59:32 +02:00
Kjell Winblad 6810591911
Merge pull request #13147 from kjellwinblad/kjell/fix_function_clause_when_decoding_protobuf/EMQX-12453
fix: make protobuf schema decode error reported to user less cryptic
2024-05-29 15:56:27 +02:00
Kjell Winblad 3769bbb7f4
Merge pull request #13140 from kjellwinblad/kjell/republish_trace_crash/EMQX-12474
fix: trace formatter should not crash when given a non-binary payload
2024-05-29 15:54:02 +02:00
Kjell Winblad 9b089a4e5a fix: use explain instead of msg as error explanation and add test 2024-05-29 10:20:15 +02:00
Kjell Winblad 479889f162
Merge pull request #13136 from kjellwinblad/kjell/fix_oracle_trace_format/EMQX-12433
fix(oracle action trace): parameters should not be rendered as IO Data
2024-05-29 09:12:38 +02:00
zhongwencool 7d9333a9d7 chore: bump influxdb app version 2024-05-29 11:13:17 +08:00
zhongwencool 00d74826ec chore: improved InfluxDB connection error messaging 2024-05-29 11:13:17 +08:00
Thales Macedo Garitezi 802361dbd0 fix(connector api): handle `timeout` when waiting for connector status
Fixes https://emqx.atlassian.net/browse/EMQX-12251
2024-05-28 14:19:32 -03:00
Kjell Winblad 82800faadf fix: make protobuf schema decode error reported to user less cryptic
Before this commit the user would see a cryptic warning log including
function_clause when decoding a message failed. This commit improves
this by removing function_caluse as well as some other cryptic words
from the message and adding a description describing what went wrong.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12453
2024-05-28 18:07:14 +02:00
Kjell Winblad 3151a3123e chore: bump version of emqx_redis_bridge app 2024-05-28 17:23:35 +02:00
Kjell Winblad 3f1fe413ee refactor: simplify redis trace formatting 2024-05-28 17:22:58 +02:00
Kjell Winblad cbdde7165e fix(redis trace): add separators in redis batch action trace
Logger will transform data that looks like IO data into a string which
made redis batch traces look like the spaces had been removed
from the strings. To prevent this, we render the batched commands into
binary string separated by spaces and semicolon. The components of a
single command is separated by spaces and the commands in a batch are
separated by semicolons.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12428
2024-05-28 17:22:58 +02:00
Kjell Winblad 5e3de1a4e3 fix(emqx_logger_jsonfmt:best_effort_json_test): test cases 2024-05-28 17:15:32 +02:00
zmstone 062ab31ecf Merge remote-tracking branch 'origin/release-57' into 0527-port-back-diverged-modules 2024-05-28 14:35:25 +02:00
zmstone 40080f5e7d
Merge pull request #13141 from zmstone/0528-test-fix-dashboard-config-test-with-stable-base-config
test: fix test case to have deterministic base config
2024-05-28 14:32:55 +02:00
Andrew Mayorov 9ae4de10ed
Merge pull request #13127 from keynslug/fix/EMQX-12439/update-leaner-error
fix(bridge-v2): report descriptive error on invalid update request
2024-05-28 13:11:41 +03:00
zmstone e9cc88cb95 test: fix test case to have deterministic base config 2024-05-28 12:03:01 +02:00
Kjell Winblad 84bc6ba28c fix: trace formatter should not crash when given a non-binary payload
Fixes:
https://emqx.atlassian.net/browse/EMQX-12474
2024-05-28 11:13:51 +02:00
Kjell Winblad e55bda6fe6 chore: bump app versions
Bump app version as required to pass the check which is done by the
following script:

./scripts/apps-version-check.sh
2024-05-28 11:01:00 +02:00