Ilya Averyanov
b9c5911883
feat(queue): implement `find_leader_timeout` event and so the `connecting` group_sm state
2024-06-21 12:22:17 +03:00
SergeTupchiy
fb266fbf8c
Merge pull request #13300 from SergeTupchiy/EMQX-12195-cluster-link-conf-backup
...
cluster link conf backup
2024-06-21 11:42:57 +03:00
zmstone
8a99995810
docs: fix regex_extract function doc
2024-06-21 09:25:09 +02:00
JianBo He
d8e6c07ca0
chore: fix failed tests
2024-06-21 11:04:23 +08:00
JianBo He
a8af90f912
chore: add changes
2024-06-21 09:39:14 +08:00
JianBo He
08f085d823
fix: gateway app must depends on all auth apps
2024-06-21 09:39:14 +08:00
Ilya Averyanov
979fb58e50
feat(queue): use tp to trace new lease events
2024-06-20 21:12:15 +03:00
Ilya Averyanov
1205e34650
feat(queue): remove too useless header file
2024-06-20 20:48:56 +03:00
JimMoen
acc8bf3405
Merge pull request #13222 from JimMoen/fix-will-qos-retain-flag-check
...
fix: check willretain and willqos when WillFlag set to `true`
2024-06-21 01:28:23 +08:00
Ivan Dyachkov
ba4fb0d3f9
Merge pull request #13290 from kjellwinblad/kjell/format_rule_cli/EMQX-12548
...
fix: do not crash when showing rule unsing the cmd interface
2024-06-20 14:55:33 +02:00
zmstone
2a0071aa01
Merge pull request #13295 from thalesmg/test-flaky-greptime-mkII-r57-20240619
...
test(greptime): attempt to fix flaky tests (attempt without driver patch)
2024-06-20 14:49:21 +02:00
Ilya Averyanov
be175d205c
feat(queue): add JIRA task links for TODOs
2024-06-20 15:38:16 +03:00
Ilya Averyanov
a9c976b6c1
feat(queue): fix notes about the usage of agent<->leader protocol-related data structures
2024-06-20 15:24:25 +03:00
Ilya Averyanov
d0cff63ed6
feat(queue): remove unused hrl-guard
2024-06-20 15:18:47 +03:00
Ilya Averyanov
a92460d38f
feat(queue): improve hints for macro clauses
2024-06-20 15:17:08 +03:00
Serge Tupchii
ac8762be10
chore: upgrade ekka to 0.19.5 (mria 0.8.8)
...
mria 0.8.8 heals a network partition once majority of core nodes are alive.
Previously, the autoheal worked only when all core nodes were reachable.
Fixes: EMQX-10974
2024-06-20 12:53:58 +03:00
William Yang
464a0a82f0
fix(tls): move default_root_fun to ?CONST_MOD_V1
2024-06-20 11:38:52 +02:00
zmstone
7d32275ebe
chore: minimize oss/platform diff
2024-06-20 10:36:01 +02:00
Kjell Winblad
12859b8959
fix: handle yet another action format
2024-06-20 10:03:49 +02:00
JimMoen
675abd7512
test: will retain and willqos in connect flags
2024-06-20 10:32:15 +08:00
JimMoen
cf9d6943d5
fix: check willretain and willqos when WillFlag set to `true`
2024-06-20 10:32:15 +08:00
JianBo He
c307c1dfc2
chore: cut v5.7.1-alpha.1
2024-06-20 09:40:42 +08:00
Ilya Averyanov
6da10036dc
feat(queue): fix dialyzer issues
2024-06-19 22:25:53 +03:00
Ilya Averyanov
a4bbab4aa2
feat(queue): document general entity layout
2024-06-19 22:24:20 +03:00
JimMoen
ff742d926a
fix: `?SLOG` with msg field
2024-06-20 02:18:26 +08:00
Serge Tupchii
71f5eaf11e
fix(data_backup): import `cluster.links` conf
2024-06-19 20:47:06 +03:00
Serge Tupchii
ef5cf4fac3
chore(clusterlink): fix comment
2024-06-19 20:46:00 +03:00
Ilya Averyanov
2b0146663a
feat(queue): add smoke test for ds shared subs
2024-06-19 19:32:23 +03:00
Ilya Averyanov
e3c4816035
feat(queue): move group subscription state machine to its own module
2024-06-19 19:22:10 +03:00
Thales Macedo Garitezi
9559ba2f06
test: remove unnecessary lines
2024-06-19 13:22:02 -03:00
William Yang
8d04545f03
test(tls): root_fun is set even when partial_chain is false
2024-06-19 17:34:04 +02:00
William Yang
41239ae766
fix(tls): disable partial_chain in hot config
2024-06-19 17:03:05 +02:00
zmstone
021b6b3902
Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master
2024-06-19 15:06:41 +02:00
zmstone
02a6ee1ef4
Merge pull request #13279 from zmstone/0617-json-decode-payload-for-map_to_redis_hset_args
...
fix(redis): json decode if arg for map_to_redis_hset_args is string
2024-06-19 14:55:21 +02:00
zmstone
213e4785e7
Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master
2024-06-19 14:54:48 +02:00
JimMoen
a6fa3e82d9
fix: prevent use pd
2024-06-19 20:15:07 +08:00
JimMoen
b7b2a08399
fix: match plugin_not_found reason
2024-06-19 20:15:07 +08:00
JimMoen
28293284a7
fix: let crash when plugin not found in cluster
2024-06-19 20:15:07 +08:00
JimMoen
19e039e0d2
fix: mark fresh install to cp the default configuration file directly
2024-06-19 20:15:07 +08:00
Thales Macedo Garitezi
ff2e6e1434
test(greptime): revert driver patch to test ci again
2024-06-19 09:12:30 -03:00
zmstone
95d387a790
Merge pull request #13294 from zmstone/0618-per-release-edition-code-injection
...
0618 per release edition code injection
2024-06-19 11:53:57 +02:00
zmstone
f0e5721959
refactor: make dialyzer happy
2024-06-19 10:42:08 +02:00
SergeTupchiy
0d098a01ef
Merge pull request #13293 from SergeTupchiy/EMQX-12345-reindex-retained-msgs-on-import
...
reindex retained msgs on import
2024-06-19 11:23:08 +03:00
Thales Macedo Garitezi
ffec1c7fe0
fix(greptime): attempt patch for cleaning up channels when worker terminates
2024-06-18 22:16:03 -03:00
Thales Macedo Garitezi
9dd419d822
test(greptime): attempt to fix flaky tests
...
```
=WARNING REPORT==== 18-Jun-2024::12:42:04.739109 ===
reason: no_endpoints
msg: failed_to_start_greptimedb_connector
client: #{protocol => http,
pool => <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>,
cli_opts =>
[{endpoints,[{http,"toxiproxy",4001}]},
{pool_size,2},
{pool_type,random},
{auto_reconnect,1},
{gprc_options,#{sync_start => true,
connect_timeout => 5000}},
{dbname,"public"},
{auth,{basic,#{password => <<"******">>,
username => "greptime_user"}}},
{https_enabled,false}]}
connector: <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>
```
2024-06-18 18:23:22 -03:00
zmstone
54fc605cc5
refactor: move schema injection from hrl to erl
2024-06-18 22:22:53 +02:00
zmstone
a53fa977df
refactor: delete an unused macro
2024-06-18 22:22:48 +02:00
Thales Macedo Garitezi
e69ba33420
Merge pull request #13291 from thalesmg/fix-ds-mgmt-node-down-r57-20240618
...
fix(dsreplmeta): check site status when fetching shard info
2024-06-18 15:14:08 -03:00
Ilya Averyanov
bca743054b
feat(queue): implement backbones of queue agent, leader and leader registry
2024-06-18 21:03:51 +03:00
Thales Macedo Garitezi
4c54ab6379
test: remove tests per review request
2024-06-18 13:08:19 -03:00
Kjell Winblad
4a318f8f51
style: fix type problem found by dialyzer
2024-06-18 17:02:26 +02:00
Serge Tupchii
baa7996289
fix(data_backup): allow exporting `ram_copies` Mnesia tables
...
Currently, ram tables can be used for message retainer.
2024-06-18 17:31:33 +03:00
Serge Tupchii
60882a616e
fix(data_backup): re-index retained messages after importing
2024-06-18 17:31:19 +03:00
Thales Macedo Garitezi
6bde6aa711
Merge pull request #13280 from thalesmg/fix-plugin-elixir-r57-20240617
...
fix(plugins): do not stop Elixir protected apps
2024-06-18 10:41:12 -03:00
Thales Macedo Garitezi
8ce16fd7d9
fix(dsreplmeta): check site status when fetching shard info
...
Fixes https://emqx.atlassian.net/browse/EMQX-12356
2024-06-18 10:40:09 -03:00
Kjell Winblad
a4f855108c
fix: do not crash when showing rule unsing the cmd interface
...
Before the change the command line command
$ bin/emqx ctl rules show rule_0hyd
would crash if the rule had bridge actions. This has now been fixed by
adding a handler for printing bridge actions.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12548
2024-06-18 15:17:25 +02:00
Kjell Winblad
3f7723b2dc
Merge pull request #13274 from kjellwinblad/kjell/add_ssl_to_rocksdb/EMQX-12289
...
feat: add SSL support to RocketMQ connector
2024-06-18 09:38:28 +02:00
Thales Macedo Garitezi
20cffb54d4
Merge branch 'release-57' into sync-r57-m-20240617
2024-06-17 17:10:12 -03:00
Thales Macedo Garitezi
6897f0141b
fix(plugins): do not stop Elixir protected apps
...
On one hand, Elixir plugins might include Elixir itself, when targetting a non-Elixir EMQX
release. If, on the other hand, the EMQX release already includes Elixir, we shouldn't
stop Elixir nor IEx, or the running IEx shell will break.
2024-06-17 16:42:23 -03:00
Ilia Averianov
ad993437aa
Merge pull request #13278 from savonarola/0617-release-57-sync
...
Sync release-57
2024-06-17 21:25:00 +03:00
zmstone
9479c8d33b
fix(redis): json decode if arg for map_to_redis_hset_args is string
2024-06-17 20:19:53 +02:00
Andrew Mayorov
eede9f349e
Merge pull request #13276 from keynslug/fix/ds/inherited-lts-lost
...
fix(dsstore): persist inherited LTS trie
2024-06-17 20:25:31 +03:00
zmstone
ae22a64157
Merge pull request #13277 from zmstone/0617-handle-kafka-message_too_large-return
...
0617 handle kafka message too large return
2024-06-17 19:15:33 +02:00
Kjell Winblad
0cbbae9655
fix: make fixes due to comments from @thalesmg
2024-06-17 19:06:12 +02:00
Kjell Winblad
c39c544c96
feat: add SSL support to RocketMQ connector
2024-06-17 19:06:12 +02:00
Thales Macedo Garitezi
b64f0c0ca7
Merge pull request #13231 from thalesmg/mt-test-endpoint-r57-20240611
...
feat(message transformation): implement dryrun endpoint
2024-06-17 13:11:39 -03:00
Thales Macedo Garitezi
240bb09070
Merge pull request #13275 from thalesmg/test-flaky-sk-r57-20240617
...
test: attempt to fix flaky test
2024-06-17 13:01:08 -03:00
Ilya Averyanov
f8e6aab86f
Merge branch 'release-57' into 0617-release-57-sync
...
* release-57:
chore(auth,http): cache REs for parsing URIs
fix(auth,http): improve URI handling
chore: revert ULOG/ELOG
test: generate dispatch.eterm in dashboard test
docs: refine change log
feat: make the dashboard restart quicker
chore: fix typo
fix(http authz): handle unknown content types in responses
chore: change types of mysql and mongodb fields to `template()`
fix(client mgmt api): allow projecting `client_attrs` from client fields
fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-17 18:53:39 +03:00
Andrew Mayorov
82588fbc35
chore(dsstore): refine module and callback typespecs
2024-06-17 17:53:39 +02:00
Andrew Mayorov
05d97397d3
fix(dsstore): use correct comparison when walking over generations
2024-06-17 17:53:39 +02:00
Andrew Mayorov
22009bcc58
test(dsstore): verify that inherited TLS trie is persisted
2024-06-17 17:53:38 +02:00
Andrew Mayorov
68f6556856
fix(dsstore): persist inherited LTS trie
...
Before this commit, inherited trie was actually only kept in memory
cache.
Also simplify storage backend behaviour around inheriting previous
generation's legacy.
2024-06-17 17:53:38 +02:00
SergeTupchiy
830266b4d5
Merge pull request #13126 from SergeTupchiy/EMQX-11967-cross-cluster-route-replication-syncer-actor
...
cluster linking
2024-06-17 18:52:46 +03:00
zmstone
3ac4ddcbe3
fix(kafka): handle message_too_large
...
bump 'failed' counter
2024-06-17 17:27:25 +02:00
zmstone
00dfdc22cf
chore: pin Kafka client wolff 0.10.5
2024-06-17 17:27:25 +02:00
zhongwencool
5ab4b321a0
Merge pull request #13267 from zhongwencool/use-redbug-for-trace
...
chore(debug): use redbug for default trace tools
2024-06-17 22:10:18 +08:00
Thales Macedo Garitezi
fb492e3dc5
Merge pull request #13264 from thalesmg/fix-postgres-disabled-prepared-r57-20240614
...
fix(postgres): authn/authz/batch requests when prepared statements are disabled
2024-06-17 11:05:05 -03:00
Thales Macedo Garitezi
5665a4917c
Merge pull request #13266 from thalesmg/fix-s3-release-app-r57-20240614
...
fix: add `emqx_s3` to `reboot_lists.eterm`
2024-06-17 10:48:12 -03:00
Thales Macedo Garitezi
ec4f462684
refactor: apply review requests
2024-06-17 10:39:32 -03:00
Thales Macedo Garitezi
1464094967
test: attempt to fix flaky test
...
```
%%% emqx_persistent_session_ds_SUITE ==> t_session_discard_persistent_to_persistent: FAILED
%%% emqx_persistent_session_ds_SUITE ==> {noproc,{gen_server,call,[snabbkaffe_collector,flush_trace,infinity]}}
```
2024-06-17 10:34:57 -03:00
Thales Macedo Garitezi
b71212b022
refactor: apply review comments
2024-06-17 09:40:33 -03:00
Thales Macedo Garitezi
f5eb3e7471
Merge pull request #13250 from thalesmg/ds-singleton-m-20240613
...
feat: enforce singleton discovery strategy when using DS `builtin_local` backend
2024-06-17 09:34:05 -03:00
Ilya Averyanov
19c9f0d76f
chore(auth,http): cache REs for parsing URIs
2024-06-17 14:42:29 +03:00
Ilya Averyanov
f7ac829f28
fix(auth,http): improve URI handling
2024-06-17 14:42:29 +03:00
Serge Tupchii
a905a6048c
chore(clusterlink): rename link `upstream` field to `name`
2024-06-17 12:24:51 +03:00
Serge Tupchii
a95a08efd3
test(clusterlink): add more test cases
2024-06-17 12:24:51 +03:00
Serge Tupchii
44c37571cc
fix(clusterlink): ignore not_registered error
2024-06-17 12:24:51 +03:00
Serge Tupchii
00f912928f
fix: fix emqx_external_broker:forward/1 spec
2024-06-17 12:24:51 +03:00
Andrew Mayorov
ed16ff07df
refactor(broker): simplify external broker behaviour
2024-06-17 12:24:51 +03:00
Serge Tupchii
d282c61120
feat(clusterlink): update only necessary resources when a link config is changed
2024-06-17 12:24:50 +03:00
Serge Tupchii
ff16521d4f
fix(clusterlink): add schema descriptions, fix dialyzer warnings, add resource_opts
2024-06-17 12:23:52 +03:00
Serge Tupchii
94e81ba812
feat(clusterlink): implement actor config handler
2024-06-17 12:23:52 +03:00
Serge Tupchii
c871b37453
fix(clusterlink): add link topics schema validator
2024-06-17 12:23:52 +03:00
Andrew Mayorov
780a0bf807
fix(cluster-link): clear exit signal of failed-to-connect client
2024-06-17 12:23:52 +03:00
Andrew Mayorov
d0df4de2a3
test(cluster-link): add e2e replication actor GC testcase
2024-06-17 12:23:52 +03:00
Andrew Mayorov
c4840b30d2
fix(cluster-link): deduplicate routes down to dest cluster
2024-06-17 12:23:52 +03:00
Andrew Mayorov
ede35df24a
fix(cluster-link): cancel heartbeats on client down
2024-06-17 12:23:52 +03:00
Andrew Mayorov
e0604e3af6
fix(cluster-link): anticipate clients may occasionally retry
2024-06-17 12:23:52 +03:00
Andrew Mayorov
0219b8bd4d
feat(cluster-link): add simple replication actor GC process
2024-06-17 12:23:52 +03:00
Andrew Mayorov
7fccb5dbc9
test(topic): add more `intersection/2` testcases
2024-06-17 12:23:52 +03:00
Andrew Mayorov
7b8f466adf
feat(topic): avoid `lists:reverse` when intersecting
2024-06-17 12:23:52 +03:00
Andrew Mayorov
24be189728
fix(topic): respect special topic rules when intersecting
2024-06-17 12:23:51 +03:00
Andrew Mayorov
de1ac131f7
test(cluster-link): fix test suite setup
2024-06-17 12:23:51 +03:00
Serge Tupchii
58eaf07627
fix(clusterlink): valide config to disallow duplicated cluster links
2024-06-17 12:23:51 +03:00
Andrew Mayorov
54d51d0982
test(cluster-link): draft basic integration test suite
2024-06-17 12:23:51 +03:00
Andrew Mayorov
e9c24090d4
fix(cluster-link): avoid starting ps syncer if persistence disabled
2024-06-17 12:23:51 +03:00
Andrew Mayorov
036c7e8492
test(cluster-link): adapt extrouter testsuite to new APIs
2024-06-17 12:23:51 +03:00
Andrew Mayorov
45eda4f3b9
fix(cluster-link): adapt heartbeat / reincarnation handling to new API
2024-06-17 12:23:51 +03:00
Andrew Mayorov
43d114546c
feat(cluster-link): preserve replication actor state in pdict
2024-06-17 12:23:51 +03:00
Andrew Mayorov
5771a41a32
fix(cluster-link): ensure replication actor bootstraps do heartbeats
2024-06-17 12:23:51 +03:00
Andrew Mayorov
d4b449c6e1
feat(cluster-link): implement replication actor heartbeats
2024-06-17 12:23:51 +03:00
Serge Tupchii
faa4420e1f
fix(clusterlink): improve actor error handling
...
Add status and error reason to the actor state, report alarms.
2024-06-17 12:23:51 +03:00
Serge Tupchii
21711c6e0d
fix(clusterlink): communicate bootstrap requirements via actor handshake
...
`session_present` flag is not reliable to decide whether bootstrap is needed if durable sessions is enabled.
In this case, the client session may survive cluster restart while all the external routes are lost, as they are not persistent.
2024-06-17 12:23:51 +03:00
Serge Tupchii
d5e82cdfac
refactor(clusterlink): avoid unnecessary `maybe_` external_broker CB names
2024-06-17 12:23:51 +03:00
Serge Tupchii
d578ac3f9e
fix(clusterlink): match correct timer ref in router actor
2024-06-17 12:23:51 +03:00
Serge Tupchii
b1aeb35370
feat(clusterlink): implement actor init handshake
2024-06-17 12:23:51 +03:00
Serge Tupchii
ac19cf89df
chore(clusterlink): remove code related to the rejected coordinator-based implementation
2024-06-17 12:23:51 +03:00
Serge Tupchii
e26e7acaa1
refactor(clusterlink): use `emqx_bridge_mqtt_lib:clientid_base/1` to construct routesync client id
2024-06-17 12:23:51 +03:00
Serge Tupchii
e7305c62ee
feat(clusterlink): replicate shared subscription and persistent session routes
2024-06-17 12:23:51 +03:00
Serge Tupchii
f036b641eb
feat(clusterlink): integrate node local routes replication and message forwarding
2024-06-17 12:23:51 +03:00
Serge Tupchii
7df91d852c
feat(clusterlink): integrate node local syncer/actor implementation
...
- support multiple cluster links in extrouter
- apply extrouter ops on 'message.publish' hook
- fix several minor bugs
2024-06-17 12:23:51 +03:00
Andrew Mayorov
cbd01ae818
feat(clusterlink): add node-local route sync actor implementation
2024-06-17 12:23:51 +03:00
Andrew Mayorov
7b95273218
feat(routesync): make syncer a bit more generic and reusable
2024-06-17 12:23:51 +03:00
Andrew Mayorov
5bd9ee5c7f
feat(utils): add `emqx_utils_ets:keyfoldl/3` function
...
Designed to be used with `bag` / `duplicate_bag` tables.
2024-06-17 12:23:51 +03:00
Andrew Mayorov
4097585f5d
fix(clusterlink): ensure extrouter works on replicants
...
This is sort of a quick fix to make things safe, but it will likely be
a subject to the same drawbacks as the regular router in high-latency
deployments: reduced throughput.
2024-06-17 12:23:51 +03:00
Andrew Mayorov
a53524c826
fix(cth-cluster): fix occasional case clauses during cluster bootup
2024-06-17 12:23:51 +03:00
Andrew Mayorov
2dd99c5a08
feat(clusterlink): add facility to reconstruct remote routing table
2024-06-17 12:23:51 +03:00
Serge Tupchii
f08342c704
feat: cluster link prototype WIP
2024-06-17 12:23:48 +03:00
lafirest
98a54994c0
Merge pull request #13196 from lafirest/fix/rules_len
...
fix(authz_mnesia): add a soft limit in the API for the length of ACL rules
2024-06-17 15:47:59 +08:00
zhongwencool
17809a5803
chore: fix dialyzer warning
2024-06-17 14:07:30 +08:00
firest
1716852057
fix(authz_mnesia): improve field names and changes
2024-06-17 10:09:25 +08:00
zhongwencool
a49e5877ec
chore(debug): use redbug for default trace tools
2024-06-17 09:28:23 +08:00
zmstone
8daaf7b727
Merge pull request #13233 from zmstone/0611-expose-regex_extract-func
...
fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-16 16:16:48 +02:00
zmstone
2b7f3a597f
Merge pull request #13245 from qzhuyan/merge/william/sync-rel57-to-master
...
sync release57 to master
2024-06-15 09:13:55 +02:00
zhongwencool
d433fc7c30
Merge pull request #13236 from zhongwencool/bug-schema-validation
...
fix: some minor bug fixes
2024-06-15 06:40:28 +08:00
Thales Macedo Garitezi
198ccd320b
fix: add `emqx_s3` to `reboot_lists.eterm`
...
Otherwise, it's unconditionally started as a permanent application instead of being
controlled by `emqx_machine`.
```sh
ͳ rg start_boot _build/emqx-enterprise/rel/emqx/releases/5.7.0-g4e6be8a6/start.script | rg emqx_s3
{apply,{application,start_boot,[emqx_s3,permanent]}},
```
2024-06-14 15:06:18 -03:00
Thales Macedo Garitezi
6e702cd16c
fix(postgres): fix usage by authn/authz modules when prepared statements are disabled
...
Fixes https://emqx.atlassian.net/browse/EMQX-12550
2024-06-14 12:19:46 -03:00
zmstone
626aae6edf
chore: fix bad conflict resolution
2024-06-14 16:57:53 +02:00
zhongwencool
d95f17fe77
chore: revert ULOG/ELOG
2024-06-14 22:11:27 +08:00
Thales Macedo Garitezi
fd5e844cc1
fix(postgres): correctly handle batch queries with disabled prepared statements
...
Fixes https://emqx.atlassian.net/browse/EMQX-12549
2024-06-14 11:10:21 -03:00
zhongwencool
f713f13b2c
test: generate dispatch.eterm in dashboard test
2024-06-14 21:52:58 +08:00
Thales Macedo Garitezi
9b3c806ba7
feat(message transformation): implement dryrun endpoint
...
Follow up to https://github.com/emqx/emqx/pull/13199
2024-06-14 10:19:36 -03:00
zhongwencool
a6e3a09118
feat: make the dashboard restart quicker
2024-06-14 18:22:34 +08:00
JimMoen
0f0e7d18db
Merge pull request #13261 from JimMoen/fix-typo
...
chore: fix typo
2024-06-14 16:55:35 +08:00
JimMoen
6823c79ae0
chore: fix typo
2024-06-14 16:04:11 +08:00
William Yang
b86d631744
test: fix tc t_handle_outing_non_utf8_topic
2024-06-14 08:17:19 +02:00
William Yang
d39b8caff9
Merge branch 'release-57' into merge/william/sync-rel57-to-master
2024-06-14 08:17:09 +02:00
lafirest
afd4b46f72
Merge pull request #13225 from lafirest/fix/mysql_redact
...
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-14 09:19:21 +08:00
Thales Macedo Garitezi
01635722e9
docs: improve descriptions
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2024-06-13 18:34:44 -03:00
Thales Macedo Garitezi
52e6c88941
feat: enforce singleton discovery strategy when using DS `builtin_local` backend
...
Fixes https://emqx.atlassian.net/browse/EMQX-12493
2024-06-13 17:04:23 -03:00
Thales Macedo Garitezi
cba5c7bb45
refactor: rm dead code
2024-06-13 16:57:34 -03:00
Thales Macedo Garitezi
4d876f2af2
chore: bump ekka -> 0.19.4
...
https://github.com/emqx/ekka/pull/236
2024-06-13 16:35:58 -03:00
Thales Macedo Garitezi
3adf64e637
Merge pull request #13240 from thalesmg/ds-cluster-purge-m-20240612
...
feat: support purging durable sessions during cluster purge
2024-06-13 09:30:18 -03:00
Thales Macedo Garitezi
6ccf1dcbf9
Merge pull request #13238 from thalesmg/fix-authz-http-content-type-handling-r57-20240612
...
fix(http authz): handle unknown content types in responses
2024-06-13 09:10:43 -03:00
Thales Macedo Garitezi
62cf277cd7
Merge pull request #13235 from thalesmg/fix-template-types-r57-20240611
...
chore: change types of mysql and mongodb fields to `template()`
2024-06-13 09:08:40 -03:00
Thales Macedo Garitezi
c81a3ebc0a
Merge pull request #13234 from thalesmg/fix-clients-api-client-attrs-union-r57-20240611
...
fix(client mgmt api): allow projecting `client_attrs` from client fields
2024-06-13 09:08:29 -03:00
William Yang
1664ea4ad4
Revert: TLS partial-chain and keyUsage #12955 #12977
...
This reverts commit 28b17a2562
.
This reverts commit 01467246fc
.
This reverts commit c3f8ba5762
.
This reverts commit 1a4a4bb3a5
.
This reverts commit fb30207ef3
.
This reverts commit 337c230e79
.
This reverts commit 3a674f44f1
.
This reverts commit 70ffd77f99
.
This reverts commit 03b0935564
.
This reverts commit 650cf4b27e
.
This reverts commit 43ad665dcf
.
This reverts commit a29a43e5fc
.
This reverts commit 4e9c1ec0c9
.
This reverts commit 8eb463c58d
.
This reverts commit 90430fa66d
.
This reverts commit eb1ab9adfe
.
This reverts commit 8bc3a86f63
.
This reverts commit fa4357ce89
.
This reverts commit 0b95a08d32
.
2024-06-13 13:46:45 +02:00
William Yang
723144199e
Merge pull request #13211 from qzhuyan/fix/william/rel57/ee-tls
...
Fix/william/rel57/ee tls
2024-06-13 13:19:44 +02:00
Andrew Mayorov
0f53b33417
chore(s3): make schema fields with defaults optional
2024-06-13 11:57:36 +02:00
zhongwencool
0eee2edd28
test: fix api_config SUITE failed
2024-06-13 14:55:24 +08:00
zhongwencool
2b1fa1711b
fix: observer command crash when can't find object code
2024-06-13 14:55:23 +08:00
zhongwencool
623845d07d
fix: don't allow set active_n to negative int
2024-06-13 14:55:23 +08:00
zhongwencool
825bfe4717
fix: load bad configs return unknown msg
2024-06-13 14:55:23 +08:00
zhongwencool
5d3f464ec3
fix: ws/wss's max_frame_size should > 0
2024-06-13 14:55:23 +08:00
zhongwencool
800a79adde
test: test failed with_mfa
2024-06-13 09:50:52 +08:00
Thales Macedo Garitezi
2c264d9a4b
fix(http authz): handle unknown content types in responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-12530
2024-06-12 14:31:12 -03:00
zhongwencool
c72b455a83
feat: add hidden option to record MFA and line in the log is very useful for debug.
2024-06-12 22:51:45 +08:00
Thales Macedo Garitezi
d0e6f22a79
feat: support purging durable sessions during cluster purge
...
Fixes https://emqx.atlassian.net/browse/EMQX-12405
2024-06-12 11:19:53 -03:00
JimMoen
daeb10151b
chore: fix typos
2024-06-12 19:06:34 +08:00
JimMoen
cde4cb1358
fix: metrics macro with description
2024-06-12 19:06:33 +08:00
JimMoen
756797b258
refactor: gen metric schema with desc from macros
2024-06-12 19:04:39 +08:00
JimMoen
263d2dbae2
fix: inc metric `'client.auth.anonymous'` when client anonymous
2024-06-12 19:04:39 +08:00
JimMoen
9a78a6f640
refactor: mv metrics macros in hrl file
2024-06-12 19:04:29 +08:00
JimMoen
cbac4019b3
fix: rm havn't used counter idx
2024-06-12 19:01:50 +08:00
Thales Macedo Garitezi
46ab3be1f4
chore: change types of mysql and mongodb fields to `template()`
...
Fixes https://emqx.atlassian.net/browse/EMQX-12395
2024-06-11 17:25:53 -03:00
Thales Macedo Garitezi
0874768c1d
fix(client mgmt api): allow projecting `client_attrs` from client fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-12511
2024-06-11 17:14:33 -03:00
zmstone
8a283b1cc0
Merge pull request #13180 from zmstone/0604-optimize-tcp-ssl-connection-data-send-performance
...
0604 optimize tcp ssl connection data send performance
2024-06-11 22:06:57 +02:00
zmstone
09f91159c9
fix(emqx_rule_funcs): expose regex_extract function to rule engine
...
previoulsy only available in variform expressions,
now made available for rule-engine
2024-06-11 21:31:13 +02:00
zmstone
51810c6919
Merge pull request #13229 from zmstone/0611-auth-with-raw-cert
...
feat(authn): support `${cert_pem}` placeholder
2024-06-11 21:16:01 +02:00
Thales Macedo Garitezi
db572d35a7
Merge remote-tracking branch 'origin/release-57' into sync-r57-m-20240611
2024-06-11 15:34:54 -03:00
zmstone
751f7a24e9
feat(authn): support ${cert_pem} placeholder
2024-06-11 19:54:24 +02:00
Thales Macedo Garitezi
337009c3a0
fix: declare `emqx_schema_registry` as a dependency of `emqx_schema_validation`
2024-06-11 14:03:27 -03:00
Thales Macedo Garitezi
0f9c3b4cea
feat(schema validation): check references schema names and types before changing config
...
Fixes https://emqx.atlassian.net/browse/EMQX-12368
2024-06-11 14:03:26 -03:00
Thales Macedo Garitezi
03b226248a
feat(schema registry): add check for inner types
...
Currently, only `protobuf` has any.
2024-06-11 13:50:11 -03:00
Thales Macedo Garitezi
c70e8252fe
fix: declare `emqx_schema_registry` as a dependency of `emqx_message_transformation`
2024-06-11 13:35:02 -03:00
Thales Macedo Garitezi
b2d716909f
Merge pull request #13200 from thalesmg/update-erlazure-no-gen-server-m-20240606
...
chore: refactor azure blob connector to use new `erlazure` without `gen_server`
2024-06-11 13:30:14 -03:00
Thales Macedo Garitezi
686bcc8a48
Merge pull request #13207 from thalesmg/republish-metrics-m-20240607
...
fix(rule actions): check `republish` action result for metrics
2024-06-11 13:29:58 -03:00
Thales Macedo Garitezi
2c6aa80b3c
Merge pull request #13221 from thalesmg/refactor-sv-topic-index-r57-20240610
...
refactor(schema validation): module organization and position indexing
2024-06-11 13:29:48 -03:00
Thales Macedo Garitezi
83ff07ade0
Merge pull request #13199 from thalesmg/message-transformation-r57-20240604
...
feat: implement message transformation
2024-06-11 13:29:25 -03:00
Thales Macedo Garitezi
ab5329be02
Merge pull request #13220 from thalesmg/refactor-channel-hc-opt-r57-20240610
...
fix: remove another redundant health check
2024-06-11 12:22:43 -03:00
Andrew Mayorov
2f0eb3560b
Merge pull request #13227 from keynslug/fix/EMQX-12533/bad-tpl
...
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 15:39:12 +03:00
Andrew Mayorov
d4fb812352
fix(aggreg): anticipate and handle delivery startup errors
2024-06-11 13:21:41 +02:00
Andrew Mayorov
29fc30ea69
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 13:21:41 +02:00
Andrew Mayorov
fb0da9848c
feat(tpl): add separate `placeholders/1` function
...
The purpose is to have a clearer view of placeholders used in a
template, without going the usual `render(Template, #{})` route that is
actually subtly misleading: it won't mention that `${}` / `${.}`
placeholder has been used.
Also unify handling of `${}` / `${.}` in a couple of places.
2024-06-11 11:42:43 +02:00
Kjell Winblad
9d22089104
Merge pull request #13173 from kjellwinblad/kjell/pgsql_timestamp_encode_etc/EEC-1036
...
fix(pgsql connector): handle prepared statement already exists
2024-06-11 10:53:30 +02:00
William Yang
44258204bd
chore: move tls_certs test helper to apps/emqx
2024-06-11 10:06:18 +02:00
firest
e64f60b73f
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-11 16:05:11 +08:00
zhongwencool
b657dc537c
Merge pull request #13156 from savonarola/0529-fix-dashboard-monitor
...
fix(dashboard_monitor): correctly merge old and new sample data
2024-06-11 10:33:15 +08:00
Thales Macedo Garitezi
05ebb17cd6
refactor: index positions rather than names
...
Addresses https://github.com/emqx/emqx/pull/13199#discussion_r1633096025
2024-06-10 15:54:37 -03:00
Thales Macedo Garitezi
5eff4a7544
refactor(schema validation): improve api and module organization
...
To conform to comment from another PR:
https://github.com/emqx/emqx/pull/13199#discussion_r1632658742
... since schema validation is the "inspiration" for message transformation.
2024-06-10 15:46:28 -03:00
Thales Macedo Garitezi
d93c8540c8
feat: pass along peername down to transformation context
2024-06-10 15:35:35 -03:00
Thales Macedo Garitezi
6e0ef893f4
feat: pass along client attributes down to message transformation context
2024-06-10 15:27:04 -03:00
Thales Macedo Garitezi
3c501e4f2a
fix: remove another redundant health check
...
Fixes https://emqx.atlassian.net/browse/EMQX-12521
2024-06-10 15:13:34 -03:00
Thales Macedo Garitezi
1a20a40218
Merge pull request #13209 from thalesmg/rm-redundant-hc-r57-20240607
...
chore: remove redundant health check call
2024-06-10 14:58:48 -03:00
Thales Macedo Garitezi
e586178479
chore: refactor azure blob connector to use new `erlazure` without `gen_server`
...
https://github.com/dkataskin/erlazure/pull/43 removes unnecessary usage of `gen_server`
from the driver.
2024-06-10 13:42:28 -03:00
Thales Macedo Garitezi
5629fe60c1
refactor: index positions rather than names
2024-06-10 13:33:22 -03:00
Thales Macedo Garitezi
8c7a7cf9db
refactor: apply review remarks
2024-06-10 11:17:13 -03:00
Thales Macedo Garitezi
e54cf2f218
refactor: use option to skip rendering as string
2024-06-10 10:49:39 -03:00
Thales Macedo Garitezi
d34558954d
refactor: improve api and module organization
2024-06-10 10:40:53 -03:00
Thales Macedo Garitezi
4aa1c3842f
fix: declare `emqx` as a dependency in `.app.src`
2024-06-10 09:53:29 -03:00
Thales Macedo Garitezi
aa416cc0ad
docs: fix stale comments and dates
2024-06-10 09:53:02 -03:00
zmstone
7a09b0470d
chore: cosmetic changes
2024-06-10 14:22:25 +02:00
Kjell Winblad
a885f0b41a
test(emqx_bridge_pgsql_SUITE): call test janitor
2024-06-10 13:45:27 +02:00
William Yang
a1aa9a4375
fix(ee): emqx no longer deps on emqx_auth_ext
2024-06-10 10:32:01 +02:00
zmstone
4347f3de3e
fix(bridge/mqtt): respect client ID prefix
2024-06-10 10:27:45 +02:00
William Yang
76cfc309a9
feat(tls): partial_chain not required
2024-06-10 10:25:44 +02:00
William Yang
7c37bf9965
feat(tls): ee only: TLS partial_chain and Keyusage
2024-06-10 10:24:48 +02:00
William Yang
e60380d205
chore: fix nit
2024-06-10 09:55:12 +02:00
William Yang
e9b813d8ef
chore: fix test
2024-06-10 09:54:54 +02:00
William Yang
1739bc0c24
feat(partial_chain): gateway support
2024-06-10 09:54:44 +02:00
zmstone
45dd7816d7
Merge remote-tracking branch 'origin/release-57'
2024-06-09 10:11:15 +02:00
William Yang
abbf2ef62f
chore(TLS-chain-test): update for OTP 26
2024-06-08 00:16:06 +02:00
William Yang
221b748b0f
test(partial_chain): update tcs for OTP-25
2024-06-08 00:16:01 +02:00
William Yang
463d1a1875
fix(test): tls_verify_partial_chain
2024-06-08 00:15:54 +02:00
William Yang
788cdbc6dd
fix(listener): remove partial_chain in wss opts
2024-06-08 00:15:51 +02:00
William Yang
3fc99315e0
chore: happy elvis
2024-06-08 00:15:45 +02:00
William Yang
c5dccdf526
feat(tls): update schema for TLS keyusage
2024-06-08 00:15:43 +02:00
William Yang
aa25e3badd
fix(tls): undefined keyusage
2024-06-08 00:15:39 +02:00
William Yang
9e5cfea8c0
test(tls): verify peer keyusage
2024-06-08 00:15:36 +02:00
William Yang
c5657029ab
feat(config): partial_chain
2024-06-08 00:15:34 +02:00
William Yang
f7ff9496e6
test: port listener tls partial_chain
2024-06-08 00:15:31 +02:00
William Yang
1ce13242a8
feat(tls): port partial_chain, part 1
2024-06-08 00:15:28 +02:00
Kjell Winblad
c13631102e
fix(PgSQL connector): typo in function name
2024-06-07 19:40:18 +02:00
Andrew Mayorov
b2f027c974
Merge pull request #13197 from keynslug/fix/EMQX-12522/conv-ssl
...
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-07 19:45:53 +03:00
Thales Macedo Garitezi
39615e1cb6
fix(rule actions): check `republish` action result for metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12328
2024-06-07 11:17:43 -03:00
Thales Macedo Garitezi
28a3c77e88
chore: remove redundant health check call
...
Since `emqx_resource_manager:channel_health_check` already does a base resource health
check, the call was repeated.
2024-06-07 11:07:12 -03:00
Kjell Winblad
e63dcc84b0
fix: unused variable and better error message
2024-06-07 12:20:42 +02:00
Kjell Winblad
a3f3f96781
test(emqx_bridge_v2_pgsql_SUITE): fix broken test case
...
We have to delete the connector and action created by the previous step
of the test case so we don't get a name conflict.
2024-06-07 12:03:34 +02:00
JimMoen
78a196e215
Merge pull request #13189 from JimMoen/fix-apply-sqlserver-template
...
fix(utils_sql): improve insert sql regular expression
2024-06-07 13:50:19 +08:00
lafirest
eb215726d5
Merge pull request #13195 from lafirest/fix/typos
...
fix: fix typos
2024-06-07 09:04:16 +08:00
Thales Macedo Garitezi
ef36350bf6
feat: implement message transformation
...
Fixes https://emqx.atlassian.net/browse/EMQX-12517
2024-06-06 17:51:08 -03:00
Andrew Mayorov
8a85d7cb5a
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-06 19:37:38 +02:00
Andrew Mayorov
c17c788438
feat(connector): allow delegate `pre_config_update` to impls
2024-06-06 19:37:34 +02:00
zmstone
ebf17c8143
Merge remote-tracking branch 'origin/release-57' into 0606-merge-release-57-to-master
2024-06-06 17:43:36 +02:00
firest
3ae26c8a54
chore: update changes
2024-06-06 18:13:24 +08:00
firest
5532c7b0a6
fix(authz_mnesia): add a soft limit in the API for the length of ACL rules
2024-06-06 16:22:53 +08:00
zmstone
2525bd0c16
perf: eliminate the loop-back fake async inet_reply message
...
Now that EMQX is on OTP 26 by default, the fake inet_reply
message when the send call returns ok only adds overhead.
For TCP (gen_tcp module), OTP 26 no longer supports direct port_command
so we had to swith to gen_tcp:send, then fake an async reply:
see https://github.com/emqx/esockd/pull/181
For TLS (ssl module), it has never supported async send anyway.
This commit switches to call Transport:send/2 directly,
this should result in one less loop-back message for each message.
For OTP 25 on which EMQX can still be compiled, one will have to
suffer the performance penalty for gen_tcp:send function to
do a non-optimized 'receive' for the send result.
2024-06-06 09:35:51 +02:00
zmstone
5c3f5d8085
perf: do not call inet getstat before each and every send
...
In a stress test environment, when alarm is enabled, there were
a lot of long_schedule warnings with the stacktrace pointing to
congestion alarm based inet:getstat function.
When alarm is disabled, the one single client is able to
hold 40,000 QoS 1 messages per second (inflight=32) throughput.
When alarm is enabled, its mqueue overflows.
This commit removes the call before each and every data send,
so to rely on emit_stats timer to trigger the congestion check.
2024-06-06 09:35:51 +02:00
firest
d4118c6e8e
fix: fix typos
2024-06-06 11:19:40 +08:00
JimMoen
dff31b293c
perf(utils_sql): pre-build regexp
2024-06-06 11:01:38 +08:00