Thales Macedo Garitezi
05ebb17cd6
refactor: index positions rather than names
...
Addresses https://github.com/emqx/emqx/pull/13199#discussion_r1633096025
2024-06-10 15:54:37 -03:00
Thales Macedo Garitezi
5eff4a7544
refactor(schema validation): improve api and module organization
...
To conform to comment from another PR:
https://github.com/emqx/emqx/pull/13199#discussion_r1632658742
... since schema validation is the "inspiration" for message transformation.
2024-06-10 15:46:28 -03:00
Thales Macedo Garitezi
d93c8540c8
feat: pass along peername down to transformation context
2024-06-10 15:35:35 -03:00
Thales Macedo Garitezi
6e0ef893f4
feat: pass along client attributes down to message transformation context
2024-06-10 15:27:04 -03:00
Thales Macedo Garitezi
3c501e4f2a
fix: remove another redundant health check
...
Fixes https://emqx.atlassian.net/browse/EMQX-12521
2024-06-10 15:13:34 -03:00
Thales Macedo Garitezi
1a20a40218
Merge pull request #13209 from thalesmg/rm-redundant-hc-r57-20240607
...
chore: remove redundant health check call
2024-06-10 14:58:48 -03:00
Thales Macedo Garitezi
5629fe60c1
refactor: index positions rather than names
2024-06-10 13:33:22 -03:00
Thales Macedo Garitezi
8c7a7cf9db
refactor: apply review remarks
2024-06-10 11:17:13 -03:00
Thales Macedo Garitezi
e54cf2f218
refactor: use option to skip rendering as string
2024-06-10 10:49:39 -03:00
Thales Macedo Garitezi
d34558954d
refactor: improve api and module organization
2024-06-10 10:40:53 -03:00
Thales Macedo Garitezi
4aa1c3842f
fix: declare emqx as a dependency in .app.src
2024-06-10 09:53:29 -03:00
Thales Macedo Garitezi
aa416cc0ad
docs: fix stale comments and dates
2024-06-10 09:53:02 -03:00
zmstone
7a09b0470d
chore: cosmetic changes
2024-06-10 14:22:25 +02:00
Kjell Winblad
a885f0b41a
test(emqx_bridge_pgsql_SUITE): call test janitor
2024-06-10 13:45:27 +02:00
William Yang
a1aa9a4375
fix(ee): emqx no longer deps on emqx_auth_ext
2024-06-10 10:32:01 +02:00
zmstone
4347f3de3e
fix(bridge/mqtt): respect client ID prefix
2024-06-10 10:27:45 +02:00
William Yang
76cfc309a9
feat(tls): partial_chain not required
2024-06-10 10:25:44 +02:00
William Yang
7c37bf9965
feat(tls): ee only: TLS partial_chain and Keyusage
2024-06-10 10:24:48 +02:00
William Yang
e60380d205
chore: fix nit
2024-06-10 09:55:12 +02:00
William Yang
e9b813d8ef
chore: fix test
2024-06-10 09:54:54 +02:00
William Yang
1739bc0c24
feat(partial_chain): gateway support
2024-06-10 09:54:44 +02:00
William Yang
abbf2ef62f
chore(TLS-chain-test): update for OTP 26
2024-06-08 00:16:06 +02:00
William Yang
221b748b0f
test(partial_chain): update tcs for OTP-25
2024-06-08 00:16:01 +02:00
William Yang
463d1a1875
fix(test): tls_verify_partial_chain
2024-06-08 00:15:54 +02:00
William Yang
788cdbc6dd
fix(listener): remove partial_chain in wss opts
2024-06-08 00:15:51 +02:00
William Yang
3fc99315e0
chore: happy elvis
2024-06-08 00:15:45 +02:00
William Yang
c5dccdf526
feat(tls): update schema for TLS keyusage
2024-06-08 00:15:43 +02:00
William Yang
aa25e3badd
fix(tls): undefined keyusage
2024-06-08 00:15:39 +02:00
William Yang
9e5cfea8c0
test(tls): verify peer keyusage
2024-06-08 00:15:36 +02:00
William Yang
c5657029ab
feat(config): partial_chain
2024-06-08 00:15:34 +02:00
William Yang
f7ff9496e6
test: port listener tls partial_chain
2024-06-08 00:15:31 +02:00
William Yang
1ce13242a8
feat(tls): port partial_chain, part 1
2024-06-08 00:15:28 +02:00
Kjell Winblad
c13631102e
fix(PgSQL connector): typo in function name
2024-06-07 19:40:18 +02:00
Andrew Mayorov
b2f027c974
Merge pull request #13197 from keynslug/fix/EMQX-12522/conv-ssl
...
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-07 19:45:53 +03:00
Thales Macedo Garitezi
28a3c77e88
chore: remove redundant health check call
...
Since `emqx_resource_manager:channel_health_check` already does a base resource health
check, the call was repeated.
2024-06-07 11:07:12 -03:00
Kjell Winblad
e63dcc84b0
fix: unused variable and better error message
2024-06-07 12:20:42 +02:00
JimMoen
78a196e215
Merge pull request #13189 from JimMoen/fix-apply-sqlserver-template
...
fix(utils_sql): improve insert sql regular expression
2024-06-07 13:50:19 +08:00
lafirest
eb215726d5
Merge pull request #13195 from lafirest/fix/typos
...
fix: fix typos
2024-06-07 09:04:16 +08:00
Thales Macedo Garitezi
ef36350bf6
feat: implement message transformation
...
Fixes https://emqx.atlassian.net/browse/EMQX-12517
2024-06-06 17:51:08 -03:00
Andrew Mayorov
8a85d7cb5a
fix(bridge-s3): pass SSL options through convert_certs/2
2024-06-06 19:37:38 +02:00
Andrew Mayorov
c17c788438
feat(connector): allow delegate pre_config_update to impls
2024-06-06 19:37:34 +02:00
zmstone
2525bd0c16
perf: eliminate the loop-back fake async inet_reply message
...
Now that EMQX is on OTP 26 by default, the fake inet_reply
message when the send call returns ok only adds overhead.
For TCP (gen_tcp module), OTP 26 no longer supports direct port_command
so we had to swith to gen_tcp:send, then fake an async reply:
see https://github.com/emqx/esockd/pull/181
For TLS (ssl module), it has never supported async send anyway.
This commit switches to call Transport:send/2 directly,
this should result in one less loop-back message for each message.
For OTP 25 on which EMQX can still be compiled, one will have to
suffer the performance penalty for gen_tcp:send function to
do a non-optimized 'receive' for the send result.
2024-06-06 09:35:51 +02:00
zmstone
5c3f5d8085
perf: do not call inet getstat before each and every send
...
In a stress test environment, when alarm is enabled, there were
a lot of long_schedule warnings with the stacktrace pointing to
congestion alarm based inet:getstat function.
When alarm is disabled, the one single client is able to
hold 40,000 QoS 1 messages per second (inflight=32) throughput.
When alarm is enabled, its mqueue overflows.
This commit removes the call before each and every data send,
so to rely on emit_stats timer to trigger the congestion check.
2024-06-06 09:35:51 +02:00
firest
d4118c6e8e
fix: fix typos
2024-06-06 11:19:40 +08:00
JimMoen
dff31b293c
perf(utils_sql): pre-build regexp
2024-06-06 11:01:38 +08:00
zmstone
c41be7ee22
chore: fix dialyzer warning
2024-06-05 23:37:21 +02:00
JimMoen
144264e0d8
test: add utils_sql test cases
2024-06-06 00:30:18 +08:00
JimMoen
627b8c45d2
fix(utils_sql): improve insert sql regular expression
2024-06-06 00:30:17 +08:00
Ilya Averyanov
60556abc4f
fix(dashboard_monitor): correctly merge old and new sample data
2024-06-05 18:49:52 +03:00
Kjell Winblad
2956e849eb
fix(pgsql connector): better msg when failing to remove statement
2024-06-05 16:58:14 +02:00
Kjell Winblad
336089f8a7
fix: bug found by dialyzer and make test case cleaner
2024-06-05 15:53:02 +02:00
Thales Macedo Garitezi
ae0379f974
Merge pull request #13181 from thalesmg/force-stop-connector-r57-20240604
...
fix(resource manager): force kill process if stuck when stopping/removing
2024-06-05 09:08:30 -03:00
lafirest
d79193fde4
Merge pull request #13177 from lafirest/sync-authn-com
...
feat: sync `client.check_authn_complete` to release-57
2024-06-05 19:51:10 +08:00
Thales Macedo Garitezi
5eab221f7c
Merge pull request #13175 from thalesmg/disable-prepared-statements-postgres-r57-20240603
...
feat(postgres): add `disable_prepared_statements` option
2024-06-04 12:29:53 -03:00
Thales Macedo Garitezi
f17aefe3d7
fix(action/source api): improve returned error message on timeout
2024-06-04 12:04:55 -03:00
zmstone
cda6d5f636
Merge pull request #13172 from zmstone/0603-add-redis-arg-formatter-to-rule-engine-funcs
...
feat(redis): add a rule function to help formatting redis args
2024-06-04 16:56:14 +02:00
Thales Macedo Garitezi
3013189cd7
fix(resource manager): force kill process if stuck when stopping/removing
...
Fixes https://emqx.atlassian.net/browse/EMQX-12357
2024-06-04 11:38:24 -03:00
Thales Macedo Garitezi
c07bc68e6f
feat(postgres): add disable_prepared_statements option
...
Fixes https://emqx.atlassian.net/browse/EMQX-12496
Some Postgres connections, such ones made to [PGBouncer](https://www.pgbouncer.org/ ) or
[Supabase in Transaction Mode](https://supabase.com/ ), do not support some session
features like prepared statements.
2024-06-04 10:11:25 -03:00
firest
87384cae32
fix(events): call client.check_authn_complete even if authentication fails
2024-06-04 17:55:39 +08:00
firest
92d26ff27b
chore: update change
2024-06-04 17:55:30 +08:00
firest
e4e53844d5
feat(events): add new hook && event client.check_authn_complete
2024-06-04 17:55:21 +08:00
Kjell Winblad
572ca6433e
fix(pgsql connector): improvements due to suggestions from @thalesmg
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-06-04 10:11:45 +02:00
zmstone
aa7ce1f641
fix(bridge/redis): add test case for map_to_redis_hset_args
2024-06-03 23:05:43 +02:00
Kjell Winblad
f2ccfff803
fix(pgsql connector): handle prepared statement already exists
...
In a user's log file it was found that that the pgsql driver can end up
in a situation where the prepared statement for a channel/action is not
properly removed before a channel with the same name as the prepared
statement is added to the connector. This commit handles this by
attempting to remove the old prepared statement if one already exists
when adding channel.
Related issue:
https://emqx.atlassian.net/browse/EEC-1036
2024-06-03 16:14:26 +02:00
zmstone
e3ed7b59dd
feat(redis): add a rule function to help formatting redis args
...
The new function named 'map_to_redis_hset_args' can be used to format
a map's key-value pairs into redis HSET (or HMSET) arg list.
This new function is dedicated for redis to avoid abuse for other
data integrations.
2024-06-03 14:09:15 +02:00
JimMoen
21624bc865
fix: get plugin app vsn from field rel_apps
2024-06-03 15:26:14 +08:00
zmstone
b6ff67d712
Merge pull request #13093 from zmstone/0522-upgrade-to-brod-3.18
...
chore: upgrade to brod 3.18.0
2024-06-02 15:06:40 +02:00
JimMoen
8d22a5a3ae
Merge pull request #13167 from JimMoen/fix-rule-events-i18n-todo
...
fix: missing rule event i18n in zh_cn
2024-06-02 18:23:20 +08:00
zmstone
9637b14543
fix: upgrade to brod 3.18.0 to fix unnecessary rebalance
2024-06-01 10:57:24 +02:00
zmstone
37ce8d93a1
Merge pull request #13164 from zmstone/0530-http-acl-compatibility-with-v4
...
0530 http acl compatibility with v4
2024-05-31 16:28:30 +02:00
Thales Macedo Garitezi
c289c7d065
Merge pull request #13148 from thalesmg/EMQX-12251-fix-conn-api-timeout-case-clause-r571-20240528
...
fix(connector api): handle `timeout` when waiting for connector status
2024-05-31 10:07:35 -03:00
zmstone
6fe8a09e97
fix(authz/http): rename PH_ACCESS to VAR_ACCESS
2024-05-31 14:49:54 +02:00
JimMoen
45e2c81ebc
fix: missing rule event i18n in zh_cn
2024-05-31 17:51:47 +08:00
zmstone
ca32cf8d29
Merge branch 'emqx-12487-fix-authz-http-content-type' into 0530-http-acl-compatibility-with-v4
2024-05-30 22:30:20 +02:00
zmstone
a00d9f17b1
chore: upgrade kafka producer lib wolff to 1.10.4
...
wolff-1.10.4 handles message_too_large error code from Kafka.
It tries to split the too-large batch into single-call batches
(for EMQX, one call batch is always one message), and retry.
In case a single-call batch is too large, EMQX should increment
a failure counter.
2024-05-30 15:40:10 +02:00
zmstone
460081a22e
Merge pull request #13131 from zmstone/0527-handle-unicode-in-template-strings
...
0527 handle unicode in template strings
2024-05-30 15:37:43 +02:00
Ilya Averyanov
97f9c81e19
feat(auth): add legacy ${access} placeholder
2024-05-30 15:47:48 +03:00
Ivan Dyachkov
0c4da98b52
chore: update deps
2024-05-30 11:53:00 +02:00
Ivan Dyachkov
1129c18330
fix(authz_http): fix content-type header in http request
2024-05-30 11:36:13 +02:00
zhongwencool
8ab7e75e64
Merge pull request #13139 from zhongwencool/influxdb-client-failed-msg-port
...
chore: improved InfluxDB connection error messaging
2024-05-30 10:57:21 +08:00
zmstone
532812f6f6
test: assert results
2024-05-29 22:10:34 +02:00
zmstone
9fe29bf5f7
chore: bump kafka bridge app vsn
2024-05-29 21:57:14 +02:00
zmstone
78a6100346
chore: fix app vsn bumps
...
only bug fixes so far
2024-05-29 21:56:22 +02:00
zmstone
1db9d54d17
fix: handle unicode in data integration template strings
...
The bug was introduced in 5.7.0 (#12893 )
2024-05-29 21:53:07 +02:00
zmstone
c54d25de98
Merge pull request #13070 from zmstone/0518-improve-kafka-connection-error-logs
...
0518 improve kafka connection error logs
2024-05-29 16:24:01 +02:00
Kjell Winblad
5017ef82a3
Merge pull request #13130 from kjellwinblad/kjell/fix_redis_trace_format/EMQX-12428
...
fix(redis trace): add separators in redis batch action trace
2024-05-29 15:59:32 +02:00
Kjell Winblad
6810591911
Merge pull request #13147 from kjellwinblad/kjell/fix_function_clause_when_decoding_protobuf/EMQX-12453
...
fix: make protobuf schema decode error reported to user less cryptic
2024-05-29 15:56:27 +02:00
Kjell Winblad
3769bbb7f4
Merge pull request #13140 from kjellwinblad/kjell/republish_trace_crash/EMQX-12474
...
fix: trace formatter should not crash when given a non-binary payload
2024-05-29 15:54:02 +02:00
Kjell Winblad
9b089a4e5a
fix: use explain instead of msg as error explanation and add test
2024-05-29 10:20:15 +02:00
Kjell Winblad
479889f162
Merge pull request #13136 from kjellwinblad/kjell/fix_oracle_trace_format/EMQX-12433
...
fix(oracle action trace): parameters should not be rendered as IO Data
2024-05-29 09:12:38 +02:00
zhongwencool
7d9333a9d7
chore: bump influxdb app version
2024-05-29 11:13:17 +08:00
zhongwencool
00d74826ec
chore: improved InfluxDB connection error messaging
2024-05-29 11:13:17 +08:00
Thales Macedo Garitezi
802361dbd0
fix(connector api): handle timeout when waiting for connector status
...
Fixes https://emqx.atlassian.net/browse/EMQX-12251
2024-05-28 14:19:32 -03:00
Kjell Winblad
82800faadf
fix: make protobuf schema decode error reported to user less cryptic
...
Before this commit the user would see a cryptic warning log including
function_clause when decoding a message failed. This commit improves
this by removing function_caluse as well as some other cryptic words
from the message and adding a description describing what went wrong.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12453
2024-05-28 18:07:14 +02:00
Kjell Winblad
3151a3123e
chore: bump version of emqx_redis_bridge app
2024-05-28 17:23:35 +02:00
Kjell Winblad
3f1fe413ee
refactor: simplify redis trace formatting
2024-05-28 17:22:58 +02:00
Kjell Winblad
cbdde7165e
fix(redis trace): add separators in redis batch action trace
...
Logger will transform data that looks like IO data into a string which
made redis batch traces look like the spaces had been removed
from the strings. To prevent this, we render the batched commands into
binary string separated by spaces and semicolon. The components of a
single command is separated by spaces and the commands in a batch are
separated by semicolons.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12428
2024-05-28 17:22:58 +02:00
Kjell Winblad
5e3de1a4e3
fix(emqx_logger_jsonfmt:best_effort_json_test): test cases
2024-05-28 17:15:32 +02:00
zmstone
40080f5e7d
Merge pull request #13141 from zmstone/0528-test-fix-dashboard-config-test-with-stable-base-config
...
test: fix test case to have deterministic base config
2024-05-28 14:32:55 +02:00
Andrew Mayorov
9ae4de10ed
Merge pull request #13127 from keynslug/fix/EMQX-12439/update-leaner-error
...
fix(bridge-v2): report descriptive error on invalid update request
2024-05-28 13:11:41 +03:00
