Andrew Mayorov
7b8f466adf
feat(topic): avoid `lists:reverse` when intersecting
2024-06-17 12:23:52 +03:00
Andrew Mayorov
24be189728
fix(topic): respect special topic rules when intersecting
2024-06-17 12:23:51 +03:00
Andrew Mayorov
de1ac131f7
test(cluster-link): fix test suite setup
2024-06-17 12:23:51 +03:00
Serge Tupchii
58eaf07627
fix(clusterlink): valide config to disallow duplicated cluster links
2024-06-17 12:23:51 +03:00
Andrew Mayorov
54d51d0982
test(cluster-link): draft basic integration test suite
2024-06-17 12:23:51 +03:00
Andrew Mayorov
e9c24090d4
fix(cluster-link): avoid starting ps syncer if persistence disabled
2024-06-17 12:23:51 +03:00
Andrew Mayorov
036c7e8492
test(cluster-link): adapt extrouter testsuite to new APIs
2024-06-17 12:23:51 +03:00
Andrew Mayorov
45eda4f3b9
fix(cluster-link): adapt heartbeat / reincarnation handling to new API
2024-06-17 12:23:51 +03:00
Andrew Mayorov
43d114546c
feat(cluster-link): preserve replication actor state in pdict
2024-06-17 12:23:51 +03:00
Andrew Mayorov
5771a41a32
fix(cluster-link): ensure replication actor bootstraps do heartbeats
2024-06-17 12:23:51 +03:00
Andrew Mayorov
d4b449c6e1
feat(cluster-link): implement replication actor heartbeats
2024-06-17 12:23:51 +03:00
Serge Tupchii
faa4420e1f
fix(clusterlink): improve actor error handling
...
Add status and error reason to the actor state, report alarms.
2024-06-17 12:23:51 +03:00
Serge Tupchii
21711c6e0d
fix(clusterlink): communicate bootstrap requirements via actor handshake
...
`session_present` flag is not reliable to decide whether bootstrap is needed if durable sessions is enabled.
In this case, the client session may survive cluster restart while all the external routes are lost, as they are not persistent.
2024-06-17 12:23:51 +03:00
Serge Tupchii
d5e82cdfac
refactor(clusterlink): avoid unnecessary `maybe_` external_broker CB names
2024-06-17 12:23:51 +03:00
Serge Tupchii
d578ac3f9e
fix(clusterlink): match correct timer ref in router actor
2024-06-17 12:23:51 +03:00
Serge Tupchii
b1aeb35370
feat(clusterlink): implement actor init handshake
2024-06-17 12:23:51 +03:00
Serge Tupchii
ac19cf89df
chore(clusterlink): remove code related to the rejected coordinator-based implementation
2024-06-17 12:23:51 +03:00
Serge Tupchii
e26e7acaa1
refactor(clusterlink): use `emqx_bridge_mqtt_lib:clientid_base/1` to construct routesync client id
2024-06-17 12:23:51 +03:00
Serge Tupchii
e7305c62ee
feat(clusterlink): replicate shared subscription and persistent session routes
2024-06-17 12:23:51 +03:00
Serge Tupchii
f036b641eb
feat(clusterlink): integrate node local routes replication and message forwarding
2024-06-17 12:23:51 +03:00
Serge Tupchii
7df91d852c
feat(clusterlink): integrate node local syncer/actor implementation
...
- support multiple cluster links in extrouter
- apply extrouter ops on 'message.publish' hook
- fix several minor bugs
2024-06-17 12:23:51 +03:00
Andrew Mayorov
cbd01ae818
feat(clusterlink): add node-local route sync actor implementation
2024-06-17 12:23:51 +03:00
Andrew Mayorov
7b95273218
feat(routesync): make syncer a bit more generic and reusable
2024-06-17 12:23:51 +03:00
Andrew Mayorov
5bd9ee5c7f
feat(utils): add `emqx_utils_ets:keyfoldl/3` function
...
Designed to be used with `bag` / `duplicate_bag` tables.
2024-06-17 12:23:51 +03:00
Andrew Mayorov
4097585f5d
fix(clusterlink): ensure extrouter works on replicants
...
This is sort of a quick fix to make things safe, but it will likely be
a subject to the same drawbacks as the regular router in high-latency
deployments: reduced throughput.
2024-06-17 12:23:51 +03:00
Andrew Mayorov
a53524c826
fix(cth-cluster): fix occasional case clauses during cluster bootup
2024-06-17 12:23:51 +03:00
Andrew Mayorov
2dd99c5a08
feat(clusterlink): add facility to reconstruct remote routing table
2024-06-17 12:23:51 +03:00
Serge Tupchii
f08342c704
feat: cluster link prototype WIP
2024-06-17 12:23:48 +03:00
lafirest
98a54994c0
Merge pull request #13196 from lafirest/fix/rules_len
...
fix(authz_mnesia): add a soft limit in the API for the length of ACL rules
2024-06-17 15:47:59 +08:00
zhongwencool
17809a5803
chore: fix dialyzer warning
2024-06-17 14:07:30 +08:00
firest
1716852057
fix(authz_mnesia): improve field names and changes
2024-06-17 10:09:25 +08:00
zhongwencool
a49e5877ec
chore(debug): use redbug for default trace tools
2024-06-17 09:28:23 +08:00
zmstone
8daaf7b727
Merge pull request #13233 from zmstone/0611-expose-regex_extract-func
...
fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-16 16:16:48 +02:00
zmstone
2b7f3a597f
Merge pull request #13245 from qzhuyan/merge/william/sync-rel57-to-master
...
sync release57 to master
2024-06-15 09:13:55 +02:00
zhongwencool
d433fc7c30
Merge pull request #13236 from zhongwencool/bug-schema-validation
...
fix: some minor bug fixes
2024-06-15 06:40:28 +08:00
Thales Macedo Garitezi
198ccd320b
fix: add `emqx_s3` to `reboot_lists.eterm`
...
Otherwise, it's unconditionally started as a permanent application instead of being
controlled by `emqx_machine`.
```sh
ͳ rg start_boot _build/emqx-enterprise/rel/emqx/releases/5.7.0-g4e6be8a6/start.script | rg emqx_s3
{apply,{application,start_boot,[emqx_s3,permanent]}},
```
2024-06-14 15:06:18 -03:00
Thales Macedo Garitezi
6e702cd16c
fix(postgres): fix usage by authn/authz modules when prepared statements are disabled
...
Fixes https://emqx.atlassian.net/browse/EMQX-12550
2024-06-14 12:19:46 -03:00
zmstone
626aae6edf
chore: fix bad conflict resolution
2024-06-14 16:57:53 +02:00
zhongwencool
d95f17fe77
chore: revert ULOG/ELOG
2024-06-14 22:11:27 +08:00
Thales Macedo Garitezi
fd5e844cc1
fix(postgres): correctly handle batch queries with disabled prepared statements
...
Fixes https://emqx.atlassian.net/browse/EMQX-12549
2024-06-14 11:10:21 -03:00
zhongwencool
f713f13b2c
test: generate dispatch.eterm in dashboard test
2024-06-14 21:52:58 +08:00
Thales Macedo Garitezi
9b3c806ba7
feat(message transformation): implement dryrun endpoint
...
Follow up to https://github.com/emqx/emqx/pull/13199
2024-06-14 10:19:36 -03:00
zhongwencool
a6e3a09118
feat: make the dashboard restart quicker
2024-06-14 18:22:34 +08:00
JimMoen
0f0e7d18db
Merge pull request #13261 from JimMoen/fix-typo
...
chore: fix typo
2024-06-14 16:55:35 +08:00
JimMoen
6823c79ae0
chore: fix typo
2024-06-14 16:04:11 +08:00
William Yang
b86d631744
test: fix tc t_handle_outing_non_utf8_topic
2024-06-14 08:17:19 +02:00
William Yang
d39b8caff9
Merge branch 'release-57' into merge/william/sync-rel57-to-master
2024-06-14 08:17:09 +02:00
lafirest
afd4b46f72
Merge pull request #13225 from lafirest/fix/mysql_redact
...
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-14 09:19:21 +08:00
Thales Macedo Garitezi
01635722e9
docs: improve descriptions
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2024-06-13 18:34:44 -03:00
Thales Macedo Garitezi
52e6c88941
feat: enforce singleton discovery strategy when using DS `builtin_local` backend
...
Fixes https://emqx.atlassian.net/browse/EMQX-12493
2024-06-13 17:04:23 -03:00
Thales Macedo Garitezi
cba5c7bb45
refactor: rm dead code
2024-06-13 16:57:34 -03:00
Thales Macedo Garitezi
4d876f2af2
chore: bump ekka -> 0.19.4
...
https://github.com/emqx/ekka/pull/236
2024-06-13 16:35:58 -03:00
Thales Macedo Garitezi
3adf64e637
Merge pull request #13240 from thalesmg/ds-cluster-purge-m-20240612
...
feat: support purging durable sessions during cluster purge
2024-06-13 09:30:18 -03:00
Thales Macedo Garitezi
6ccf1dcbf9
Merge pull request #13238 from thalesmg/fix-authz-http-content-type-handling-r57-20240612
...
fix(http authz): handle unknown content types in responses
2024-06-13 09:10:43 -03:00
Thales Macedo Garitezi
62cf277cd7
Merge pull request #13235 from thalesmg/fix-template-types-r57-20240611
...
chore: change types of mysql and mongodb fields to `template()`
2024-06-13 09:08:40 -03:00
Thales Macedo Garitezi
c81a3ebc0a
Merge pull request #13234 from thalesmg/fix-clients-api-client-attrs-union-r57-20240611
...
fix(client mgmt api): allow projecting `client_attrs` from client fields
2024-06-13 09:08:29 -03:00
William Yang
1664ea4ad4
Revert: TLS partial-chain and keyUsage #12955 #12977
...
This reverts commit 28b17a2562
.
This reverts commit 01467246fc
.
This reverts commit c3f8ba5762
.
This reverts commit 1a4a4bb3a5
.
This reverts commit fb30207ef3
.
This reverts commit 337c230e79
.
This reverts commit 3a674f44f1
.
This reverts commit 70ffd77f99
.
This reverts commit 03b0935564
.
This reverts commit 650cf4b27e
.
This reverts commit 43ad665dcf
.
This reverts commit a29a43e5fc
.
This reverts commit 4e9c1ec0c9
.
This reverts commit 8eb463c58d
.
This reverts commit 90430fa66d
.
This reverts commit eb1ab9adfe
.
This reverts commit 8bc3a86f63
.
This reverts commit fa4357ce89
.
This reverts commit 0b95a08d32
.
2024-06-13 13:46:45 +02:00
William Yang
723144199e
Merge pull request #13211 from qzhuyan/fix/william/rel57/ee-tls
...
Fix/william/rel57/ee tls
2024-06-13 13:19:44 +02:00
Andrew Mayorov
0f53b33417
chore(s3): make schema fields with defaults optional
2024-06-13 11:57:36 +02:00
zhongwencool
0eee2edd28
test: fix api_config SUITE failed
2024-06-13 14:55:24 +08:00
zhongwencool
2b1fa1711b
fix: observer command crash when can't find object code
2024-06-13 14:55:23 +08:00
zhongwencool
623845d07d
fix: don't allow set active_n to negative int
2024-06-13 14:55:23 +08:00
zhongwencool
825bfe4717
fix: load bad configs return unknown msg
2024-06-13 14:55:23 +08:00
zhongwencool
5d3f464ec3
fix: ws/wss's max_frame_size should > 0
2024-06-13 14:55:23 +08:00
zhongwencool
800a79adde
test: test failed with_mfa
2024-06-13 09:50:52 +08:00
Thales Macedo Garitezi
2c264d9a4b
fix(http authz): handle unknown content types in responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-12530
2024-06-12 14:31:12 -03:00
zhongwencool
c72b455a83
feat: add hidden option to record MFA and line in the log is very useful for debug.
2024-06-12 22:51:45 +08:00
Thales Macedo Garitezi
d0e6f22a79
feat: support purging durable sessions during cluster purge
...
Fixes https://emqx.atlassian.net/browse/EMQX-12405
2024-06-12 11:19:53 -03:00
JimMoen
daeb10151b
chore: fix typos
2024-06-12 19:06:34 +08:00
JimMoen
cde4cb1358
fix: metrics macro with description
2024-06-12 19:06:33 +08:00
JimMoen
756797b258
refactor: gen metric schema with desc from macros
2024-06-12 19:04:39 +08:00
JimMoen
263d2dbae2
fix: inc metric `'client.auth.anonymous'` when client anonymous
2024-06-12 19:04:39 +08:00
JimMoen
9a78a6f640
refactor: mv metrics macros in hrl file
2024-06-12 19:04:29 +08:00
JimMoen
cbac4019b3
fix: rm havn't used counter idx
2024-06-12 19:01:50 +08:00
Thales Macedo Garitezi
46ab3be1f4
chore: change types of mysql and mongodb fields to `template()`
...
Fixes https://emqx.atlassian.net/browse/EMQX-12395
2024-06-11 17:25:53 -03:00
Thales Macedo Garitezi
0874768c1d
fix(client mgmt api): allow projecting `client_attrs` from client fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-12511
2024-06-11 17:14:33 -03:00
zmstone
8a283b1cc0
Merge pull request #13180 from zmstone/0604-optimize-tcp-ssl-connection-data-send-performance
...
0604 optimize tcp ssl connection data send performance
2024-06-11 22:06:57 +02:00
zmstone
09f91159c9
fix(emqx_rule_funcs): expose regex_extract function to rule engine
...
previoulsy only available in variform expressions,
now made available for rule-engine
2024-06-11 21:31:13 +02:00
zmstone
51810c6919
Merge pull request #13229 from zmstone/0611-auth-with-raw-cert
...
feat(authn): support `${cert_pem}` placeholder
2024-06-11 21:16:01 +02:00
Thales Macedo Garitezi
db572d35a7
Merge remote-tracking branch 'origin/release-57' into sync-r57-m-20240611
2024-06-11 15:34:54 -03:00
zmstone
751f7a24e9
feat(authn): support ${cert_pem} placeholder
2024-06-11 19:54:24 +02:00
Thales Macedo Garitezi
337009c3a0
fix: declare `emqx_schema_registry` as a dependency of `emqx_schema_validation`
2024-06-11 14:03:27 -03:00
Thales Macedo Garitezi
0f9c3b4cea
feat(schema validation): check references schema names and types before changing config
...
Fixes https://emqx.atlassian.net/browse/EMQX-12368
2024-06-11 14:03:26 -03:00
Thales Macedo Garitezi
03b226248a
feat(schema registry): add check for inner types
...
Currently, only `protobuf` has any.
2024-06-11 13:50:11 -03:00
Thales Macedo Garitezi
c70e8252fe
fix: declare `emqx_schema_registry` as a dependency of `emqx_message_transformation`
2024-06-11 13:35:02 -03:00
Thales Macedo Garitezi
b2d716909f
Merge pull request #13200 from thalesmg/update-erlazure-no-gen-server-m-20240606
...
chore: refactor azure blob connector to use new `erlazure` without `gen_server`
2024-06-11 13:30:14 -03:00
Thales Macedo Garitezi
686bcc8a48
Merge pull request #13207 from thalesmg/republish-metrics-m-20240607
...
fix(rule actions): check `republish` action result for metrics
2024-06-11 13:29:58 -03:00
Thales Macedo Garitezi
2c6aa80b3c
Merge pull request #13221 from thalesmg/refactor-sv-topic-index-r57-20240610
...
refactor(schema validation): module organization and position indexing
2024-06-11 13:29:48 -03:00
Thales Macedo Garitezi
83ff07ade0
Merge pull request #13199 from thalesmg/message-transformation-r57-20240604
...
feat: implement message transformation
2024-06-11 13:29:25 -03:00
Thales Macedo Garitezi
ab5329be02
Merge pull request #13220 from thalesmg/refactor-channel-hc-opt-r57-20240610
...
fix: remove another redundant health check
2024-06-11 12:22:43 -03:00
Andrew Mayorov
2f0eb3560b
Merge pull request #13227 from keynslug/fix/EMQX-12533/bad-tpl
...
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 15:39:12 +03:00
Andrew Mayorov
d4fb812352
fix(aggreg): anticipate and handle delivery startup errors
2024-06-11 13:21:41 +02:00
Andrew Mayorov
29fc30ea69
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 13:21:41 +02:00
Andrew Mayorov
fb0da9848c
feat(tpl): add separate `placeholders/1` function
...
The purpose is to have a clearer view of placeholders used in a
template, without going the usual `render(Template, #{})` route that is
actually subtly misleading: it won't mention that `${}` / `${.}`
placeholder has been used.
Also unify handling of `${}` / `${.}` in a couple of places.
2024-06-11 11:42:43 +02:00
Kjell Winblad
9d22089104
Merge pull request #13173 from kjellwinblad/kjell/pgsql_timestamp_encode_etc/EEC-1036
...
fix(pgsql connector): handle prepared statement already exists
2024-06-11 10:53:30 +02:00
William Yang
44258204bd
chore: move tls_certs test helper to apps/emqx
2024-06-11 10:06:18 +02:00
firest
e64f60b73f
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-11 16:05:11 +08:00
zhongwencool
b657dc537c
Merge pull request #13156 from savonarola/0529-fix-dashboard-monitor
...
fix(dashboard_monitor): correctly merge old and new sample data
2024-06-11 10:33:15 +08:00
Thales Macedo Garitezi
05ebb17cd6
refactor: index positions rather than names
...
Addresses https://github.com/emqx/emqx/pull/13199#discussion_r1633096025
2024-06-10 15:54:37 -03:00
Thales Macedo Garitezi
5eff4a7544
refactor(schema validation): improve api and module organization
...
To conform to comment from another PR:
https://github.com/emqx/emqx/pull/13199#discussion_r1632658742
... since schema validation is the "inspiration" for message transformation.
2024-06-10 15:46:28 -03:00
Thales Macedo Garitezi
d93c8540c8
feat: pass along peername down to transformation context
2024-06-10 15:35:35 -03:00
Thales Macedo Garitezi
6e0ef893f4
feat: pass along client attributes down to message transformation context
2024-06-10 15:27:04 -03:00
Thales Macedo Garitezi
3c501e4f2a
fix: remove another redundant health check
...
Fixes https://emqx.atlassian.net/browse/EMQX-12521
2024-06-10 15:13:34 -03:00
Thales Macedo Garitezi
1a20a40218
Merge pull request #13209 from thalesmg/rm-redundant-hc-r57-20240607
...
chore: remove redundant health check call
2024-06-10 14:58:48 -03:00
Thales Macedo Garitezi
e586178479
chore: refactor azure blob connector to use new `erlazure` without `gen_server`
...
https://github.com/dkataskin/erlazure/pull/43 removes unnecessary usage of `gen_server`
from the driver.
2024-06-10 13:42:28 -03:00
Thales Macedo Garitezi
5629fe60c1
refactor: index positions rather than names
2024-06-10 13:33:22 -03:00
Thales Macedo Garitezi
8c7a7cf9db
refactor: apply review remarks
2024-06-10 11:17:13 -03:00
Thales Macedo Garitezi
e54cf2f218
refactor: use option to skip rendering as string
2024-06-10 10:49:39 -03:00
Thales Macedo Garitezi
d34558954d
refactor: improve api and module organization
2024-06-10 10:40:53 -03:00
Thales Macedo Garitezi
4aa1c3842f
fix: declare `emqx` as a dependency in `.app.src`
2024-06-10 09:53:29 -03:00
Thales Macedo Garitezi
aa416cc0ad
docs: fix stale comments and dates
2024-06-10 09:53:02 -03:00
zmstone
7a09b0470d
chore: cosmetic changes
2024-06-10 14:22:25 +02:00
Kjell Winblad
a885f0b41a
test(emqx_bridge_pgsql_SUITE): call test janitor
2024-06-10 13:45:27 +02:00
William Yang
a1aa9a4375
fix(ee): emqx no longer deps on emqx_auth_ext
2024-06-10 10:32:01 +02:00
zmstone
4347f3de3e
fix(bridge/mqtt): respect client ID prefix
2024-06-10 10:27:45 +02:00
William Yang
76cfc309a9
feat(tls): partial_chain not required
2024-06-10 10:25:44 +02:00
William Yang
7c37bf9965
feat(tls): ee only: TLS partial_chain and Keyusage
2024-06-10 10:24:48 +02:00
William Yang
e60380d205
chore: fix nit
2024-06-10 09:55:12 +02:00
William Yang
e9b813d8ef
chore: fix test
2024-06-10 09:54:54 +02:00
William Yang
1739bc0c24
feat(partial_chain): gateway support
2024-06-10 09:54:44 +02:00
zmstone
45dd7816d7
Merge remote-tracking branch 'origin/release-57'
2024-06-09 10:11:15 +02:00
William Yang
abbf2ef62f
chore(TLS-chain-test): update for OTP 26
2024-06-08 00:16:06 +02:00
William Yang
221b748b0f
test(partial_chain): update tcs for OTP-25
2024-06-08 00:16:01 +02:00
William Yang
463d1a1875
fix(test): tls_verify_partial_chain
2024-06-08 00:15:54 +02:00
William Yang
788cdbc6dd
fix(listener): remove partial_chain in wss opts
2024-06-08 00:15:51 +02:00
William Yang
3fc99315e0
chore: happy elvis
2024-06-08 00:15:45 +02:00
William Yang
c5dccdf526
feat(tls): update schema for TLS keyusage
2024-06-08 00:15:43 +02:00
William Yang
aa25e3badd
fix(tls): undefined keyusage
2024-06-08 00:15:39 +02:00
William Yang
9e5cfea8c0
test(tls): verify peer keyusage
2024-06-08 00:15:36 +02:00
William Yang
c5657029ab
feat(config): partial_chain
2024-06-08 00:15:34 +02:00
William Yang
f7ff9496e6
test: port listener tls partial_chain
2024-06-08 00:15:31 +02:00
William Yang
1ce13242a8
feat(tls): port partial_chain, part 1
2024-06-08 00:15:28 +02:00
Kjell Winblad
c13631102e
fix(PgSQL connector): typo in function name
2024-06-07 19:40:18 +02:00
Andrew Mayorov
b2f027c974
Merge pull request #13197 from keynslug/fix/EMQX-12522/conv-ssl
...
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-07 19:45:53 +03:00
Thales Macedo Garitezi
39615e1cb6
fix(rule actions): check `republish` action result for metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12328
2024-06-07 11:17:43 -03:00
Thales Macedo Garitezi
28a3c77e88
chore: remove redundant health check call
...
Since `emqx_resource_manager:channel_health_check` already does a base resource health
check, the call was repeated.
2024-06-07 11:07:12 -03:00
Kjell Winblad
e63dcc84b0
fix: unused variable and better error message
2024-06-07 12:20:42 +02:00
Kjell Winblad
a3f3f96781
test(emqx_bridge_v2_pgsql_SUITE): fix broken test case
...
We have to delete the connector and action created by the previous step
of the test case so we don't get a name conflict.
2024-06-07 12:03:34 +02:00
JimMoen
78a196e215
Merge pull request #13189 from JimMoen/fix-apply-sqlserver-template
...
fix(utils_sql): improve insert sql regular expression
2024-06-07 13:50:19 +08:00
lafirest
eb215726d5
Merge pull request #13195 from lafirest/fix/typos
...
fix: fix typos
2024-06-07 09:04:16 +08:00
Thales Macedo Garitezi
ef36350bf6
feat: implement message transformation
...
Fixes https://emqx.atlassian.net/browse/EMQX-12517
2024-06-06 17:51:08 -03:00
Andrew Mayorov
8a85d7cb5a
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-06 19:37:38 +02:00
Andrew Mayorov
c17c788438
feat(connector): allow delegate `pre_config_update` to impls
2024-06-06 19:37:34 +02:00
zmstone
ebf17c8143
Merge remote-tracking branch 'origin/release-57' into 0606-merge-release-57-to-master
2024-06-06 17:43:36 +02:00
firest
3ae26c8a54
chore: update changes
2024-06-06 18:13:24 +08:00
firest
5532c7b0a6
fix(authz_mnesia): add a soft limit in the API for the length of ACL rules
2024-06-06 16:22:53 +08:00
zmstone
2525bd0c16
perf: eliminate the loop-back fake async inet_reply message
...
Now that EMQX is on OTP 26 by default, the fake inet_reply
message when the send call returns ok only adds overhead.
For TCP (gen_tcp module), OTP 26 no longer supports direct port_command
so we had to swith to gen_tcp:send, then fake an async reply:
see https://github.com/emqx/esockd/pull/181
For TLS (ssl module), it has never supported async send anyway.
This commit switches to call Transport:send/2 directly,
this should result in one less loop-back message for each message.
For OTP 25 on which EMQX can still be compiled, one will have to
suffer the performance penalty for gen_tcp:send function to
do a non-optimized 'receive' for the send result.
2024-06-06 09:35:51 +02:00
zmstone
5c3f5d8085
perf: do not call inet getstat before each and every send
...
In a stress test environment, when alarm is enabled, there were
a lot of long_schedule warnings with the stacktrace pointing to
congestion alarm based inet:getstat function.
When alarm is disabled, the one single client is able to
hold 40,000 QoS 1 messages per second (inflight=32) throughput.
When alarm is enabled, its mqueue overflows.
This commit removes the call before each and every data send,
so to rely on emit_stats timer to trigger the congestion check.
2024-06-06 09:35:51 +02:00
firest
d4118c6e8e
fix: fix typos
2024-06-06 11:19:40 +08:00
JimMoen
dff31b293c
perf(utils_sql): pre-build regexp
2024-06-06 11:01:38 +08:00
zmstone
c41be7ee22
chore: fix dialyzer warning
2024-06-05 23:37:21 +02:00
Ilia Averianov
dc2e6d1695
Merge pull request #13163 from savonarola/0523-ds-shared-subs-dispatch
...
Inject shared subcription handling into durable session
2024-06-05 23:59:50 +03:00
Serge Tupchii
159da912c2
feat(emqx_dashboard_swagger): apply field validator in request body check
2024-06-05 19:35:04 +03:00
JimMoen
144264e0d8
test: add utils_sql test cases
2024-06-06 00:30:18 +08:00
JimMoen
627b8c45d2
fix(utils_sql): improve insert sql regular expression
2024-06-06 00:30:17 +08:00
Ilya Averyanov
b075b7120c
feat(sessds): implement dispatchig between CE/EE shared sub agents
2024-06-05 19:17:58 +03:00
Ilya Averyanov
60556abc4f
fix(dashboard_monitor): correctly merge old and new sample data
2024-06-05 18:49:52 +03:00
Kjell Winblad
2956e849eb
fix(pgsql connector): better msg when failing to remove statement
2024-06-05 16:58:14 +02:00
Kjell Winblad
336089f8a7
fix: bug found by dialyzer and make test case cleaner
2024-06-05 15:53:02 +02:00
Thales Macedo Garitezi
ae0379f974
Merge pull request #13181 from thalesmg/force-stop-connector-r57-20240604
...
fix(resource manager): force kill process if stuck when stopping/removing
2024-06-05 09:08:30 -03:00
lafirest
d79193fde4
Merge pull request #13177 from lafirest/sync-authn-com
...
feat: sync `client.check_authn_complete` to release-57
2024-06-05 19:51:10 +08:00
Thales Macedo Garitezi
5eab221f7c
Merge pull request #13175 from thalesmg/disable-prepared-statements-postgres-r57-20240603
...
feat(postgres): add `disable_prepared_statements` option
2024-06-04 12:29:53 -03:00
Thales Macedo Garitezi
f17aefe3d7
fix(action/source api): improve returned error message on timeout
2024-06-04 12:04:55 -03:00
zmstone
cda6d5f636
Merge pull request #13172 from zmstone/0603-add-redis-arg-formatter-to-rule-engine-funcs
...
feat(redis): add a rule function to help formatting redis args
2024-06-04 16:56:14 +02:00
Thales Macedo Garitezi
3013189cd7
fix(resource manager): force kill process if stuck when stopping/removing
...
Fixes https://emqx.atlassian.net/browse/EMQX-12357
2024-06-04 11:38:24 -03:00
Ilya Averyanov
9cdfbb0845
feat(sessds): make ds session be aware of shared subscriptions
2024-06-04 17:15:30 +03:00
Thales Macedo Garitezi
c07bc68e6f
feat(postgres): add `disable_prepared_statements` option
...
Fixes https://emqx.atlassian.net/browse/EMQX-12496
Some Postgres connections, such ones made to [PGBouncer](https://www.pgbouncer.org/ ) or
[Supabase in Transaction Mode](https://supabase.com/ ), do not support some session
features like prepared statements.
2024-06-04 10:11:25 -03:00
firest
87384cae32
fix(events): call `client.check_authn_complete` even if authentication fails
2024-06-04 17:55:39 +08:00
firest
92d26ff27b
chore: update change
2024-06-04 17:55:30 +08:00
firest
e4e53844d5
feat(events): add new hook && event `client.check_authn_complete`
2024-06-04 17:55:21 +08:00
Ilia Averianov
dc95218c68
Merge pull request #13144 from savonarola/0528-increase-bridge-buffer-drop-log-level
...
chore(bridges): increase log level for bridge buffer overflow reports
2024-06-04 12:37:50 +03:00
Kjell Winblad
572ca6433e
fix(pgsql connector): improvements due to suggestions from @thalesmg
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-06-04 10:11:45 +02:00
zhongwencool
a8a67a2ac9
Merge pull request #13151 from zhongwencool/authz-trace-log
...
chore: make authz's logs easier to understand
2024-06-04 11:12:09 +08:00
zmstone
aa7ce1f641
fix(bridge/redis): add test case for map_to_redis_hset_args
2024-06-03 23:05:43 +02:00
Kjell Winblad
f2ccfff803
fix(pgsql connector): handle prepared statement already exists
...
In a user's log file it was found that that the pgsql driver can end up
in a situation where the prepared statement for a channel/action is not
properly removed before a channel with the same name as the prepared
statement is added to the connector. This commit handles this by
attempting to remove the old prepared statement if one already exists
when adding channel.
Related issue:
https://emqx.atlassian.net/browse/EEC-1036
2024-06-03 16:14:26 +02:00
zmstone
e3ed7b59dd
feat(redis): add a rule function to help formatting redis args
...
The new function named 'map_to_redis_hset_args' can be used to format
a map's key-value pairs into redis HSET (or HMSET) arg list.
This new function is dedicated for redis to avoid abuse for other
data integrations.
2024-06-03 14:09:15 +02:00
Thales Macedo Garitezi
c5547543e0
Merge pull request #13069 from thalesmg/azure-blob-storage-action-m-20240508
...
feat: implement azure blob storage action
2024-06-03 08:56:37 -03:00
Ilia Averianov
193978af90
Merge pull request #13168 from savonarola/0530-fix-mgmt-test
...
chore(tests): improve test stability
2024-06-03 14:44:12 +03:00
zhongwencool
fda365a87b
chore: make authz's logs easier to understand
2024-06-03 18:01:57 +08:00
JimMoen
21624bc865
fix: get plugin app vsn from field `rel_apps`
2024-06-03 15:26:14 +08:00
zmstone
b6ff67d712
Merge pull request #13093 from zmstone/0522-upgrade-to-brod-3.18
...
chore: upgrade to brod 3.18.0
2024-06-02 15:06:40 +02:00
JimMoen
8d22a5a3ae
Merge pull request #13167 from JimMoen/fix-rule-events-i18n-todo
...
fix: missing rule event i18n in zh_cn
2024-06-02 18:23:20 +08:00
zmstone
9637b14543
fix: upgrade to brod 3.18.0 to fix unnecessary rebalance
2024-06-01 10:57:24 +02:00
Ilya Averyanov
57741a41e9
chore(tests): improve test stability
2024-05-31 22:27:31 +03:00
zmstone
fb07f40bea
test: fix default value of retry_interval in test case
2024-05-31 17:53:12 +02:00
zmstone
20c92407c7
feat: change message retry_interval default value to infinity
...
Previous Setting: The default value for `retry_interval` was 30 seconds.
New Default: The default `retry_interval` has been changed to 'infinity'.
With this update, EMQX will not automatically retry message deliveries
by default.
Compliance and Compatibility: Aligning with MQTT specification
standards, in-session message delivery retries are not typically
compliant. We recognize that some users depend on this feature, so the
option to configure retries remains available for backward
compatibility.
2024-05-31 17:53:12 +02:00
zmstone
37ce8d93a1
Merge pull request #13164 from zmstone/0530-http-acl-compatibility-with-v4
...
0530 http acl compatibility with v4
2024-05-31 16:28:30 +02:00
Thales Macedo Garitezi
347971e24a
fix: check `max_block_size` for direct transfers
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
4859cebd9f
refactor: move `min_block_size` one level up
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
9a1d3ea04f
chore: bump app vsns
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
616f14ae53
fix: avoid uploading blocks too frequently and splitting large buffers
...
Fixes https://github.com/emqx/emqx/pull/13069#discussion_r1613706747
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
af99829a21
refactor: move common templating logic to `emqx_connector_aggreg_buffer_ctx`
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
c916c83c7c
feat: implement azure blob storage action
...
Fixes https://emqx.atlassian.net/browse/EMQX-12280
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
60d24c6ad5
refactor: simplify resource_opts definition
2024-05-31 11:07:44 -03:00
Thales Macedo Garitezi
729441d1ce
refactor(connector aggregator): move shared schemas to app
2024-05-31 11:07:44 -03:00
Ilya Averyanov
db713cb656
chore(bridges): increase log level for bridge buffer overflow reports
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2024-05-31 16:35:18 +03:00
Thales Macedo Garitezi
c289c7d065
Merge pull request #13148 from thalesmg/EMQX-12251-fix-conn-api-timeout-case-clause-r571-20240528
...
fix(connector api): handle `timeout` when waiting for connector status
2024-05-31 10:07:35 -03:00
zmstone
6fe8a09e97
fix(authz/http): rename PH_ACCESS to VAR_ACCESS
2024-05-31 14:49:54 +02:00
JimMoen
45e2c81ebc
fix: missing rule event i18n in zh_cn
2024-05-31 17:51:47 +08:00
Ivan Dyachkov
29ad07ce29
Merge remote-tracking branch 'upstream/release-57' into 0531-sync-release-57
2024-05-31 07:32:26 +02:00
zmstone
ca32cf8d29
Merge branch 'emqx-12487-fix-authz-http-content-type' into 0530-http-acl-compatibility-with-v4
2024-05-30 22:30:20 +02:00
zmstone
a00d9f17b1
chore: upgrade kafka producer lib wolff to 1.10.4
...
wolff-1.10.4 handles message_too_large error code from Kafka.
It tries to split the too-large batch into single-call batches
(for EMQX, one call batch is always one message), and retry.
In case a single-call batch is too large, EMQX should increment
a failure counter.
2024-05-30 15:40:10 +02:00
zmstone
460081a22e
Merge pull request #13131 from zmstone/0527-handle-unicode-in-template-strings
...
0527 handle unicode in template strings
2024-05-30 15:37:43 +02:00
Ilya Averyanov
97f9c81e19
feat(auth): add legacy ${access} placeholder
2024-05-30 15:47:48 +03:00
Ivan Dyachkov
0c4da98b52
chore: update deps
2024-05-30 11:53:00 +02:00
Ivan Dyachkov
1129c18330
fix(authz_http): fix content-type header in http request
2024-05-30 11:36:13 +02:00
zhongwencool
8ab7e75e64
Merge pull request #13139 from zhongwencool/influxdb-client-failed-msg-port
...
chore: improved InfluxDB connection error messaging
2024-05-30 10:57:21 +08:00
zmstone
532812f6f6
test: assert results
2024-05-29 22:10:34 +02:00
zmstone
9fe29bf5f7
chore: bump kafka bridge app vsn
2024-05-29 21:57:14 +02:00
zmstone
78a6100346
chore: fix app vsn bumps
...
only bug fixes so far
2024-05-29 21:56:22 +02:00
zmstone
1db9d54d17
fix: handle unicode in data integration template strings
...
The bug was introduced in 5.7.0 (#12893 )
2024-05-29 21:53:07 +02:00
zmstone
c54d25de98
Merge pull request #13070 from zmstone/0518-improve-kafka-connection-error-logs
...
0518 improve kafka connection error logs
2024-05-29 16:24:01 +02:00
Kjell Winblad
5017ef82a3
Merge pull request #13130 from kjellwinblad/kjell/fix_redis_trace_format/EMQX-12428
...
fix(redis trace): add separators in redis batch action trace
2024-05-29 15:59:32 +02:00
Kjell Winblad
6810591911
Merge pull request #13147 from kjellwinblad/kjell/fix_function_clause_when_decoding_protobuf/EMQX-12453
...
fix: make protobuf schema decode error reported to user less cryptic
2024-05-29 15:56:27 +02:00
Kjell Winblad
3769bbb7f4
Merge pull request #13140 from kjellwinblad/kjell/republish_trace_crash/EMQX-12474
...
fix: trace formatter should not crash when given a non-binary payload
2024-05-29 15:54:02 +02:00
Kjell Winblad
9b089a4e5a
fix: use explain instead of msg as error explanation and add test
2024-05-29 10:20:15 +02:00
Kjell Winblad
479889f162
Merge pull request #13136 from kjellwinblad/kjell/fix_oracle_trace_format/EMQX-12433
...
fix(oracle action trace): parameters should not be rendered as IO Data
2024-05-29 09:12:38 +02:00
zhongwencool
7d9333a9d7
chore: bump influxdb app version
2024-05-29 11:13:17 +08:00
zhongwencool
00d74826ec
chore: improved InfluxDB connection error messaging
2024-05-29 11:13:17 +08:00
Thales Macedo Garitezi
802361dbd0
fix(connector api): handle `timeout` when waiting for connector status
...
Fixes https://emqx.atlassian.net/browse/EMQX-12251
2024-05-28 14:19:32 -03:00
Kjell Winblad
82800faadf
fix: make protobuf schema decode error reported to user less cryptic
...
Before this commit the user would see a cryptic warning log including
function_clause when decoding a message failed. This commit improves
this by removing function_caluse as well as some other cryptic words
from the message and adding a description describing what went wrong.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12453
2024-05-28 18:07:14 +02:00
Kjell Winblad
3151a3123e
chore: bump version of emqx_redis_bridge app
2024-05-28 17:23:35 +02:00
Kjell Winblad
3f1fe413ee
refactor: simplify redis trace formatting
2024-05-28 17:22:58 +02:00
Kjell Winblad
cbdde7165e
fix(redis trace): add separators in redis batch action trace
...
Logger will transform data that looks like IO data into a string which
made redis batch traces look like the spaces had been removed
from the strings. To prevent this, we render the batched commands into
binary string separated by spaces and semicolon. The components of a
single command is separated by spaces and the commands in a batch are
separated by semicolons.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12428
2024-05-28 17:22:58 +02:00
Kjell Winblad
5e3de1a4e3
fix(emqx_logger_jsonfmt:best_effort_json_test): test cases
2024-05-28 17:15:32 +02:00
zmstone
062ab31ecf
Merge remote-tracking branch 'origin/release-57' into 0527-port-back-diverged-modules
2024-05-28 14:35:25 +02:00
zmstone
40080f5e7d
Merge pull request #13141 from zmstone/0528-test-fix-dashboard-config-test-with-stable-base-config
...
test: fix test case to have deterministic base config
2024-05-28 14:32:55 +02:00
Andrew Mayorov
9ae4de10ed
Merge pull request #13127 from keynslug/fix/EMQX-12439/update-leaner-error
...
fix(bridge-v2): report descriptive error on invalid update request
2024-05-28 13:11:41 +03:00
zmstone
e9cc88cb95
test: fix test case to have deterministic base config
2024-05-28 12:03:01 +02:00
Kjell Winblad
84bc6ba28c
fix: trace formatter should not crash when given a non-binary payload
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12474
2024-05-28 11:13:51 +02:00
Kjell Winblad
e55bda6fe6
chore: bump app versions
...
Bump app version as required to pass the check which is done by the
following script:
./scripts/apps-version-check.sh
2024-05-28 11:01:00 +02:00
Kjell Winblad
f65168982b
fix(oracle action trace): parameters should not be rendered as IO Data
...
This forces the parameters to the database statement to be rendered as
a JSON array in JSON traces instead of being rendered as a string when
the parameters are interpreted as IO data.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12433
2024-05-28 10:59:44 +02:00
zmstone
8f2ca71d8b
test: fix test case to have deterministic base config
2024-05-28 10:57:20 +02:00
Andrew Mayorov
c04aaad0a4
chore(bridge-v2): leave comment describing need for `nowarn_function`
2024-05-28 10:31:14 +02:00
ieQu1
8fbeca4321
chore: Version bumps
2024-05-28 00:14:01 +02:00
ieQu1
04305d004f
refactor(sessds): Extract metadata keys to a header
2024-05-27 23:36:20 +02:00
ieQu1
16d1425982
refactor(emqx): Move files related to durable session to a subdir
2024-05-27 23:31:15 +02:00
zmstone
1148091f0b
chore: restore test code
2024-05-27 23:05:21 +02:00
zmstone
96ef37d0ea
chore: port diverged modules back to oss
2024-05-27 22:40:08 +02:00
zmstone
2771a10d39
test: fix a flaky one
2024-05-27 20:59:50 +02:00
zmstone
238c207b09
chore: bump app versions
2024-05-27 20:26:52 +02:00
zmstone
41cbfcfaa1
Merge remote-tracking branch 'origin/release-57' into 0527-sync-5.7.0-to-master
2024-05-27 20:10:44 +02:00
Andrew Mayorov
663f4fd39f
chore: bump application versions
2024-05-27 17:49:16 +02:00
Andrew Mayorov
08d88ea814
feat(bridge-api): improve error messages for Update Source API
2024-05-27 17:49:16 +02:00
Andrew Mayorov
ed7c29ec26
fix(api): mention spec override in "request metadata" type
2024-05-27 16:57:45 +02:00
Andrew Mayorov
5c2a68076f
fix(bridge-v2): report descriptive error on invalid update request
...
Before this commit, generic validation errors were reported as union
mismatches of _all_ of the bridge schemas. After this commit, specific
schema is chosen before validation.
2024-05-27 14:37:10 +02:00
Ivan Dyachkov
f393fd5304
chore: release 5.7.0
2024-05-27 14:32:23 +02:00
zmstone
b2f5e50f16
Merge pull request #13120 from ieQu1/dev/ds-api-site-status
...
fix(dsapi): Check status of the node via Mria
2024-05-25 05:42:05 +02:00
zmstone
0763e29786
chore: prepare for 5.7.0-rc.2 release
2024-05-24 21:10:37 +02:00
ieQu1
73df01e934
fix(dsapi): Check status of the node via Mria
2024-05-24 20:59:07 +02:00
ieQu1
7f356aa3a8
Merge pull request #13115 from ieQu1/dev/fix-drop-generations2
...
Mitigate transient errors when dropping generations
2024-05-24 20:13:39 +02:00
ieQu1
830b62d899
fix(dsrepl): Retry sending ra commands to the leader
2024-05-24 19:09:34 +02:00
ieQu1
1ffc7d5d9e
fix(dsrepl): Treat all exceptions from storage layer as recoverable
2024-05-24 19:09:34 +02:00
ieQu1
25721bceba
fix(dsstor): Delete generation metadata before dropping it
2024-05-24 19:09:33 +02:00
ieQu1
3d9837a0b8
test(ds): Add emqx_ds_replication_SUITE:t_drop_generation
2024-05-24 19:09:33 +02:00
Ilya Averyanov
327080e70e
perf(emqx_placeholder): do not try to decode maps as json
...
fix
2024-05-24 20:08:31 +03:00
Andrew Mayorov
c30b881239
test(monitor): verify historical and current metrics are consistent
2024-05-24 14:56:10 +02:00
Andrew Mayorov
fd9655bc35
fix(monitor): unify metric names for consistency and simplicity
2024-05-24 14:55:48 +02:00
Andrew Mayorov
7b137c0aaf
fix(monitor): reflect durable subscriptions in historical metrics
...
Essentially in the same manner as they are currently reflected in the
current metrics.
2024-05-24 14:08:35 +02:00
zmstone
6ec8ae5a5f
Merge pull request #13112 from HJianBo/change-the-default-batch-size-s3-uploader
...
Fix(s3): change the batch_size to 1 for direct mode
2024-05-24 10:56:53 +02:00
Andrew Mayorov
29d203ae33
Merge pull request #13108 from keynslug/fix/ct-flaky/mgmt
...
test(mgmt): avoid inter-suite contamination by switching to cth tooling
2024-05-24 11:14:16 +03:00
JianBo He
adaf238131
fix(s3): fix the convert function if the mode is missing
2024-05-24 16:02:06 +08:00
JimMoen
5b60f6a5b2
fix: make dialyzer happy
2024-05-24 14:29:02 +08:00
JimMoen
43506495ae
fix: ensure plugin configured states after join cluster
2024-05-24 13:51:56 +08:00
JimMoen
874bcd9a8b
fix: match decode plugin config map failed case
2024-05-24 10:20:50 +08:00
zhongwencool
3aa1f86d9e
chore: is_content_type_json to validate_content_type_json
2024-05-24 10:13:24 +08:00
zhongwencool
5c759941d5
feat: return 415 when UNSUPPORTED_MEDIA_TYPE
2024-05-24 10:13:23 +08:00
zmstone
c3c6da7ec4
chore: prepare for release cut 5.7.0-rc.1
2024-05-23 21:39:15 +02:00
Zaiming (Stone) Shi
3a4b8e6c24
Merge pull request #13106 from zmstone/0523-add-customer-type
...
0523 add customer type (4=BYOC) to license schema
2024-05-23 21:30:02 +02:00
Zaiming (Stone) Shi
40288b99b2
Merge pull request #13101 from JimMoen/fix-plugin-config-map
...
fix: allow put plugin config without schema
2024-05-23 21:18:43 +02:00
ieQu1
c952e46f08
Merge remote-tracking branch 'origin/release-57'
2024-05-23 21:04:03 +02:00
Thales Macedo Garitezi
823a6bc521
Merge pull request #13107 from thalesmg/hide-clientv2-response-type-r57-20240523
...
fix(client mgmt api): hide unused return type
2024-05-23 16:03:51 -03:00
Andrew Mayorov
7878d106f6
test(mgmt): avoid inter-suite contamination by switching to cth tooling
2024-05-23 20:17:34 +02:00
Andrew Mayorov
b4c6968f8c
Merge pull request #13104 from keynslug/fix/EMQX-12415/last-resort-poll
...
fix(dsrepl): trigger "last-resort" pending transitions handler when idle
2024-05-23 21:17:03 +03:00
ieQu1
0dc507441a
Merge pull request #13105 from ieQu1/dev/sessds-renew-streams-interval
...
fix(sessds): Change the defaults to renew streams every 1 second
2024-05-23 18:23:20 +02:00
Thales Macedo Garitezi
b1e3c941fb
fix(client mgmt api): hide unused return type
...
Fixes https://emqx.atlassian.net/browse/EMQX-12427
2024-05-23 13:17:43 -03:00
Andrew Mayorov
d47bf8076a
test(mgmt): better isolate testsuite running environments
2024-05-23 17:53:13 +02:00
zmstone
c737045dff
chore: add customer type 4 (BYOC)
2024-05-23 17:30:06 +02:00
ieQu1
5584b658c9
fix(sessds): Change the defaults to renew streams every 1 second
2024-05-23 17:17:29 +02:00
Zaiming (Stone) Shi
1d99ba2a8a
Merge pull request #13102 from JimMoen/fix-ds-prom-badkey
...
fix: add ds metrics into prometheus zipper init acc
2024-05-23 16:57:06 +02:00
Thales Macedo Garitezi
118c1d2806
Merge pull request #13103 from thalesmg/fix-another-ds-counter-nit-r57-20240523
...
fix(monitor api): fix cluster metric aggregation
2024-05-23 10:57:53 -03:00
William Yang
bb3f858db2
Merge pull request #13086 from qzhuyan/dev/william/quic-lb
...
feat: quic-lb support
2024-05-23 15:31:14 +02:00
Andrew Mayorov
ba6382adae
fix(dsrepl): trigger "last-resort" pending transitions handler when idle
...
This is a hack to work around the unintended issues causing shard
allocator to become idle even when there are pending transitions.
2024-05-23 14:54:01 +02:00
Andrew Mayorov
839b143fc4
fix(sessds): consider message durability disabled by default
2024-05-23 14:48:39 +02:00
Thales Macedo Garitezi
c0e3a81c61
fix(monitor api): fix cluster metric aggregation
...
Fixes https://emqx.atlassian.net/browse/EMQX-12423
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-23 09:38:24 -03:00
Andrew Mayorov
bf326acd7b
fix(dsrepl): handle stopping non-yet-running shard supervisor
2024-05-23 14:34:27 +02:00
Kjell Winblad
0e5b02c373
Merge pull request #13100 from kjellwinblad/kjell/s3_aggregate_only_has_result_trace/EMQX-12429
...
fix: add action rendered trace for s3 in aggregated mode
2024-05-23 14:29:18 +02:00
JimMoen
a40c7d646a
fix: apply suggestions from code review, thanks @zmstone
2024-05-23 20:28:14 +08:00
JimMoen
b91ecac168
refactor(prom): unify function name to avoid multiple levels call
2024-05-23 20:11:22 +08:00
JimMoen
065411c69c
fix: add ds metrics into prometheus zipper init acc
2024-05-23 20:11:21 +08:00
Kjell Winblad
89b47e8ffc
fix(s3 tracing): do not format records as IO data in aggregate mode
2024-05-23 12:41:30 +02:00
Kjell Winblad
fb7688ab94
fix(trace): make sure that the payload encode works with nested payloads
...
This commit makes sure that the trace setting for payload encode works
even when the payload is in a nested structure or when the payload key
is a binary instead of an atom.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12424
2024-05-23 12:30:56 +02:00
JimMoen
5dc96a37e2
fix: ensure plugin avsc serde before config
2024-05-23 17:46:58 +08:00
JimMoen
25a9aa1797
fix: allow put plugin config without schema
2024-05-23 17:33:34 +08:00
JimMoen
99869821fb
fix: ensure plugin config map after fresh install
2024-05-23 17:16:16 +08:00
William Yang
94e59c8fa7
feat(quic-lb): fix test
2024-05-23 10:32:23 +02:00
William Yang
d8eecfe64f
feat(quic-lb): add configuration
2024-05-23 10:32:20 +02:00
Kjell Winblad
e10c87b825
fix: add action rendered trace for s3 in aggregated mode
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-12429
2024-05-23 10:25:27 +02:00
Andrew Mayorov
398dc97ed6
Merge pull request #13092 from keynslug/fix/dsrepl/site-autoleave
...
fix(dsrepl): properly handle transaction abort during forget site
2024-05-23 10:56:23 +03:00
ieQu1
950fa82be3
Merge pull request #13097 from ieQu1/dev/ds-replay-unrecoverable-no-test
...
feat(sessds): Handle unrecoverable errors
2024-05-23 09:49:21 +02:00
William Yang
34a69625b5
feat: quic-lb support
2024-05-23 09:17:09 +02:00
Kjell Winblad
26c988fe11
Merge pull request #13077 from kjellwinblad/kjell/fix_action_update_race_issue/EMQX-12376
...
fix: action config update would sometimes not be reflected in connector
2024-05-23 09:11:48 +02:00
Andrew Mayorov
2f6729d203
Merge pull request #13089 from keynslug/fix/EMQX-12437/restore
...
fix(s3-bridge): restore backward config compatibility
2024-05-23 10:10:01 +03:00
zmstone
b0832ecc74
test: fix a flaky one
2024-05-23 08:55:28 +02:00
JianBo He
4053356597
Merge pull request #13063 from JimMoen/improve-plugin-config-behavior
...
fix(plugin): serde not found on new-joined nodes
2024-05-23 11:24:44 +08:00
lafirest
b1076221b0
Merge pull request #13085 from lafirest/fix/rocket_namespace
...
fix(rocketmq): fix namespace error for RocketMQ
2024-05-23 10:39:30 +08:00
ieQu1
07774ab060
feat(sessds): Handle unrecoverable errors
2024-05-22 22:45:37 +02:00
Thales Macedo Garitezi
63cb12d7c2
Merge pull request #13095 from thalesmg/fix-monitor-ds-counters-again-r57-20240522
...
fix(monitor api): add ds connections and subscriptions to old counters
2024-05-22 16:43:15 -03:00
Thales Macedo Garitezi
20a767eee8
test: fix broken test
2024-05-22 16:19:47 -03:00
ieQu1
bfd01c425d
Merge pull request #13072 from ieQu1/dev/fix-drop-generation
...
Idempotent drop_generation
2024-05-22 21:13:30 +02:00
ieQu1
6eb04f90a3
fix(ds): Allow to write batches to older generations
2024-05-22 20:28:16 +02:00
Thales Macedo Garitezi
5cad4497de
fix(monitor api): add ds connections and subscriptions to old counters
...
Fixes https://emqx.atlassian.net/browse/EMQX-12423
Fixes https://emqx.atlassian.net/browse/EMQX-12267
2024-05-22 15:19:46 -03:00
Thales Macedo Garitezi
4094032649
fix(bridge v2 api): don't attempt to start disabled connector when starting action/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-12435
2024-05-22 13:33:55 -03:00
zmstone
1bc9600c58
Merge remote-tracking branch 'origin/release-57'
2024-05-22 18:28:23 +02:00
Kjell Winblad
88c96e26de
refactor: simplify the code with maps:map/2
...
Thanks @thalesmg for the suggestion
2024-05-22 18:03:43 +02:00
ieQu1
59a09fb86f
fix(ds): Apply review remarks
2024-05-22 18:01:33 +02:00
ieQu1
5c78ecba40
docs(ds): Update documentation for the storage layouts
2024-05-22 18:01:33 +02:00
ieQu1
29345aaa30
fix(ds): Fix idle event generation in bitfield_lts layout
2024-05-22 18:01:33 +02:00
ieQu1
b3ded7edce
fix(ds): Fix code review remark
2024-05-22 18:01:33 +02:00
ieQu1
60edf5e9b8
fix(ds): Move responsibility of returning end_of_stream to the CBM
2024-05-22 18:01:33 +02:00
ieQu1
0ff307e789
fix(ds): Include generation ID in the storage events
...
Make sure storage events originating from generation X are handled in
the context of the same generation.
2024-05-22 18:01:33 +02:00
ieQu1
acdae4fba3
fix(ds): Workaround for the idempotency error when dropping gens
2024-05-22 18:01:33 +02:00
ieQu1
eb7c43ee9d
fix(ds): Always store messages in the current generation
2024-05-22 18:01:33 +02:00
ieQu1
074d98a14a
test(ds): Refactor ds_SUITE
2024-05-22 18:01:33 +02:00
ieQu1
e4a73f003a
feat(ds): Implement format_status callback
...
Reduce volume of logs and crash reports from DS
2024-05-22 18:01:32 +02:00
ieQu1
1526c527d0
fix(ds): Log generation operations
2024-05-22 18:01:32 +02:00
ieQu1
aca2d9586c
fix(ds): Fix return type of drop_generation
2024-05-22 18:01:32 +02:00
ieQu1
c6fc76e335
fix(ds): Perform read operations on the leader.
2024-05-22 18:01:32 +02:00
ieQu1
4580906405
fix(ds): Use erpc instead of gen_rpc for `delete_next'
2024-05-22 18:01:32 +02:00
Andrew Mayorov
8016e9adf4
fix(s3-bridge): restore backward config compatibility
2024-05-22 17:59:52 +02:00
Kjell Winblad
4b540e3bd0
fix: do not leak action configurations in alarm messages
2024-05-22 17:43:07 +02:00
Andrew Mayorov
e6c5c1b598
chore(dsrepl): provide more information in rebalancing log messages
2024-05-22 17:24:08 +02:00
Andrew Mayorov
c355c9ad50
fix(dsrepl): properly handle transaction abort during forget site
2024-05-22 17:22:55 +02:00
Kjell Winblad
cff8b97e8a
fix: handle channel updated during health check
...
This commit fixes an issue found by CI test case
emqx_bridge_influxdb_SUITE:t_start_stop and others. While the channel
health check process is running, the channel could be removed or updated
which could cause a crash in the resource manager or non up-to-date
alarms being triggered.
2024-05-22 12:15:04 +02:00
Kjell Winblad
39d758c4d6
fix: do not return configs for channels from emqx_resource_manager
2024-05-22 12:15:04 +02:00
Kjell Winblad
917474f694
fix: action config update would sometimes not be reflected in connector
...
Before this commit the following happened sometimes:
1. action status is connected
2. action config is updated to something that should change the status to
disconnected
3. action status is still connected and the old config is being used by
the connector even though the config has been correctly updated.
The reason for this bug is that the post_config_hook runs before the
global EMQX config is updated. The post config hook is adding the new
config to the connector. Since the new config causes the action to get
status disconnected, the adding of the action to the connector is
retried when the health check runs but this time the config will be
loaded from the global config which could cause the old config to be
loaded (this happens when the global config has not had time to get
updated).
The above problem has been fixed in this commit by only reading action
configs from the global config when the connector starts/restarts and
instead store the latest configs for the actions in the connector.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12376
2024-05-22 12:15:04 +02:00
firest
f3e8037e0f
fix(rocketmq): fix namespace error for RocketMQ
2024-05-22 16:29:38 +08:00
Andrew Mayorov
a3cd3e31b1
Merge pull request #13084 from keynslug/fix/EMQX-12422/part-size
...
fix(s3-bridge): report inconsistent `min/max_part_size` option
2024-05-22 09:48:09 +02:00
JimMoen
e5f7aa9817
refactor: plguin functions and types rename
2024-05-22 11:51:54 +08:00
JimMoen
33aa61daea
fix: use hocon format as plugin config
2024-05-22 06:37:58 +08:00
JimMoen
14f2a68799
fix: bpapi spec type
2024-05-22 06:01:57 +08:00
Thales Macedo Garitezi
31a35f2a15
Merge pull request #13076 from thalesmg/fix-ds-session-count-client-api-r57-20240520
...
fix(client mgmt api): cache disconnected durable session count for `/clients` api
2024-05-21 15:34:21 -03:00
JimMoen
5abd23af5a
test: plugin refactor
2024-05-22 02:09:37 +08:00
JimMoen
87b3b214b9
fix: make static_check happy
2024-05-22 02:09:36 +08:00
JimMoen
3ce091e9d7
fix: ensure plugin tarball and extracted
2024-05-22 02:09:35 +08:00
JimMoen
1abc8cf502
fix: ensure plugin's config on boot and join cluster
2024-05-22 02:09:34 +08:00
JimMoen
677352e498
fix: ensure installed and plugin state on boot
2024-05-22 02:09:33 +08:00
JimMoen
2076681e76
fix: put plugin config with binary namevsn key
2024-05-22 02:09:32 +08:00
JimMoen
140b7ce51e
fix(plugin): schema content only provided in enterprise edition
2024-05-22 02:09:31 +08:00
JimMoen
a7f2f95318
fix: ensure avro file
2024-05-22 02:09:30 +08:00