ea30d50125
fix(schema registry): avoid atom leak
...
Fixes https://emqx.atlassian.net/browse/EMQX-12603
2024-07-02 14:01:21 -03:00
1d5669d008
fix(sessds): tolerate removal of nonexistent gbt entries
2024-07-02 17:15:41 +02:00
5532f40d83
Merge branch 'release-57' into 20240702-m-sync-r57-mix-umbrella
2024-07-02 11:52:36 -03:00
f64bd313aa
Merge pull request #13263 from thalesmg/mix-umbrella-m-20240612
...
perf: "mixify" project to improve release speed (part 1)
2024-07-02 11:36:09 -03:00
55298ab6f3
Merge pull request #13387 from zhongwencool/dont-override-authn-users
...
fix: don't override authn users when import_user from authn.boostrap_file
2024-07-02 22:33:16 +08:00
3f0d59300b
Merge pull request #13348 from lafirest/fix/drop_payload
...
feat(log): allows custom the encoding of payload in the log
2024-07-02 21:47:48 +08:00
a57917b66b
perf(sessds): rotate through streams with iterators when fetching
...
This avoids expensive shuffling of the whole list of fetchable streams,
which can be quite long.
2024-07-02 15:42:33 +02:00
9a4f3f88e3
feat(sessds): allow stream iteration starting from a specific key
2024-07-02 15:40:40 +02:00
dc73b957b3
feat(sessds): use trees to hold streams in session state
2024-07-02 15:39:02 +02:00
4df2e0be85
fix: bad rebase conflict resolution
2024-07-02 10:21:52 -03:00
08596f886a
feat: add default csv file in authn-built-in-db
2024-07-02 21:18:38 +08:00
8e904099c7
fix: add boostrap_file/type to post authn_api
2024-07-02 21:18:37 +08:00
4d912516c8
fix: don't override authn users when import_user from authn.boostrap_file
2024-07-02 21:18:37 +08:00
bd0c8f0204
fix: bad conflict resolution
2024-07-02 09:47:49 -03:00
89bd69eb50
fix(bridge v1 api): return correct http response code when incompatible
2024-07-02 09:46:06 -03:00
8843fcbbf4
perf: use manifest to track proto file compilation
2024-07-02 09:45:47 -03:00
19f3b030f9
chore: preparing to run common tests / eunit with mix
2024-07-02 09:45:45 -03:00
b74828d7ea
perf: "mixify" project to improve release speed
...
Fixes https://emqx.atlassian.net/browse/EMQX-12527
2024-07-02 09:39:48 -03:00
dea2bf19b1
Merge pull request #13389 from lafirest/fix/builtin_acl
...
fix: fix two minor issues of bulti-in authn/authz
2024-07-02 20:19:53 +08:00
5a0bae2318
Merge pull request #13367 from thalesmg/20240628-r57-test-flaky-authz-cache
...
test(authz cache): attempt to fix flaky test
2024-07-02 09:14:06 -03:00
24ac241727
Merge pull request #13380 from thalesmg/20240701-r57-fix-http-bridge-hc-return-value
...
fix(http action): use correct return value for channel health check
2024-07-02 09:13:53 -03:00
db07a1ebea
feat(message transformation): add more read-only fields to context
...
Fixes https://emqx.atlassian.net/browse/EMQX-12583
2024-07-02 09:13:19 -03:00
686f79c036
fix: fix two minor issues of bulti-in authn/authz
...
1. the `Derived Key Length` for `pbkdf2` should be a positive integer.
2. fix topics in the authorization rules might be parsed incorrectly
2024-07-02 19:35:48 +08:00
a46440d00a
feat(log): allows custom the encoding of payload in the log
2024-07-02 19:33:31 +08:00
7e0bcd4eda
chore: tests in otp26
2024-07-02 15:26:07 +08:00
317b29451f
test(gcp consumer): attempt to stabilize flaky test
...
```
%%% emqx_bridge_kafka_impl_consumer_SUITE ==> ssl.t_start_and_consume_ok: FAILED
%%% emqx_bridge_kafka_impl_consumer_SUITE ==> {{panic,
#{msg => "Unexpected result",
result =>
{run_stage_failed,error,
{badmatch,{{1,0},timeout}},
[{emqx_bridge_kafka_impl_consumer_SUITE,
'-t_start_and_consume_ok/1-fun-12-',4,
[{file,
"/emqx/apps/emqx_bridge_kafka/test/emqx_bridge_kafka_impl_consumer_SUITE.erl"},
{line,1184}]},
{emqx_bridge_kafka_impl_consumer_SUITE,
t_start_and_consume_ok,1,
[{file,
"/emqx/apps/emqx_bridge_kafka/test/emqx_bridge_kafka_impl_consumer_SUITE.erl"},
{line,1171}]}]}}},
[{emqx_bridge_kafka_impl_consumer_SUITE,t_start_and_consume_ok,1,
[{file,
"/emqx/apps/emqx_bridge_kafka/test/emqx_bridge_kafka_impl_consumer_SUITE.erl"},
{line,1240}]},
```
2024-07-01 17:57:17 -03:00
c04e93838f
test(eviction agent): attempt to stabilize flaky test
...
```
%%% emqx_eviction_agent_SUITE ==> t_session_serialization: FAILED
%%% emqx_eviction_agent_SUITE ==>
Failure/Error: ?assertEqual(1, emqx_eviction_agent : session_count ( ))
expected: 1
got: 2
line: 348
```
2024-07-01 16:10:32 -03:00
59084dbfbe
fix(message transformation): correctly read from user properties in context
...
Port of https://github.com/emqx/emqx/pull/13316 to release-57
Fixes https://emqx.atlassian.net/browse/EMQX-12582
2024-07-01 14:53:42 -03:00
6dbb561944
Merge pull request #13355 from thalesmg/20240627-r57-fix-connector-api-bad-ssl-config
...
fix(connector api): handle bad tls config file conversion errors
2024-07-01 14:26:09 -03:00
b14856cf1a
refactor: improve error messages
2024-07-01 13:21:17 -03:00
72664780df
fix(http action): use correct return value for channel health check
...
Fixes https://emqx.atlassian.net/browse/EMQX-12622
2024-07-01 11:17:57 -03:00
532f04da9d
Merge pull request #13373 from id/0701-sync-release-57
...
sync release-57
2024-07-01 16:02:29 +02:00
d1e9b097d1
Merge pull request #13365 from thalesmg/20240628-r57-fix-dashboard-add-default-user-concurrently
...
fix(dashboard): handle add default user race condition
2024-07-01 10:22:32 -03:00
505f568c32
chore: bump apps versions
2024-07-01 13:52:27 +02:00
e28750b522
Merge remote-tracking branch 'upstream/release-57' into 0701-sync-release-57
2024-07-01 13:52:14 +02:00
e291dcdd18
fix: default value for max_conn_rate etc should be set to infinity
...
Before this commit the default value for the fields max_conn_rate,
messages_rate and bytes_rate were not set. This is fixed by setting the
default value to infinity. This breaks the corresponding dashboard
fields (they can not be edited) so the dashboard also needs to be
updated.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12514
2024-07-01 13:16:26 +02:00
bd075caf56
Merge pull request #13358 from lafirest/fix/authn_reason
...
fix(events): fixed the reason format of the `authn_complete_event`
2024-07-01 18:46:18 +08:00
6c665037de
Merge pull request #13372 from lafirest/fix/gateway_license
...
fix: limit gateway connections with license
2024-07-01 18:46:05 +08:00
a0644d4612
chore: update app version
2024-07-01 16:32:08 +08:00
b5d507bad8
fix: limit gateway connections with license
2024-06-30 20:06:04 +08:00
ec6e862539
test(authz cache): attempt to fix flaky test
...
Hypothesis: some race condition involving the previous test case, which uses the same
clientid.
```
Testing apps.emqx.emqx_authz_cache_SUITE: *** FAILED test case 3 of 3 ***
%%% emqx_authz_cache_SUITE ==> t_drain_authz_cache: FAILED
%%% emqx_authz_cache_SUITE ==>
Failure/Error: ?assertEqual([], list_cache ( ClientPid ))
expected: []
got: [{{#{qos => 0,action_type => publish,retain => false},<<"t1">>},
{allow,1719599365019}}]
line: 72
```
2024-06-28 15:40:55 -03:00
9215b3710f
Merge pull request #13327 from thalesmg/fix-kprodu-delete-wolff-r57-20240624
...
fix(kafka and derivatives): add `alias` config to avoid clashes with same topic
2024-06-28 15:22:11 -03:00
b69f298058
fix(dashboard): handle add default user race condition
...
This can happen at least in tests, when nodes boot concurrently.
2024-06-28 15:18:23 -03:00
795d280861
fix(connector api): handle bad tls config file conversion errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-12581
2024-06-28 14:31:31 -03:00
9ede62c9b1
Merge pull request #13356 from thalesmg/20240626-m-test-sparkplug-empty-roundtrip
...
test(schema registry): add test asserting the behavior of empty message roundtrip
2024-06-28 12:08:38 -03:00
2a9c27d206
Merge pull request #13345 from thalesmg/20240626-r572-fix-validate-schema-reg-name
...
fix(schema registry api): validate schema name when creating
2024-06-28 12:08:27 -03:00
063e7657b5
Merge pull request #13344 from thalesmg/20240626-r572-multi-node-bulk-subscribe
...
fix(client mgmt api): make bulk subscribe work again in clusters
2024-06-28 12:08:16 -03:00
067beece75
chore: add count to thrown error
2024-06-28 12:06:47 -03:00
7cf0e69fdf
chore: fix failed tests
2024-06-28 14:07:59 +08:00
e8176b80a6
fix(events): fixed the reason format of the `authn_complete_event`
2024-06-28 11:09:28 +08:00
b39557f6fd
Merge pull request #13336 from zhongwencool/authn-boostrap-file
...
feat: support bootstrap_file on authentication for build-in-database
2024-06-28 09:07:53 +08:00
2c48d7e0f0
Merge pull request #13347 from zhongwencool/improve-check-oom-log
...
chore: log shutdown reason for check_oom trace log
2024-06-28 08:06:11 +08:00
79f15b1daa
test(schema registry): add test asserting the behavior of empty message roundtrip
...
Relates to https://emqx.atlassian.net/browse/EMQX-10866
2024-06-27 17:49:57 -03:00
e76e94b497
perf(mgmt): optimize bulk unsubscribe when registry is enabled
2024-06-27 10:32:55 -03:00
0b329dbf06
perf(mgmt): optimize bulk subscribe when registry is enabled
2024-06-27 10:32:55 -03:00
c49900af50
perf(mgmt): optimize clientid lookup when registry is enabled
2024-06-27 10:32:55 -03:00
7b7f44b9ac
fix(client mgmt api): make bulk subscribe work again in clusters
...
Fixes https://emqx.atlassian.net/browse/EMQX-12337
2024-06-27 09:37:55 -03:00
6f00df6452
fix(schema registry api): validate schema name when creating
...
Fixes https://emqx.atlassian.net/browse/EMQX-10958
2024-06-27 09:26:41 -03:00
ff8c2bc1d8
feat(authz): add `ignore` metric for each source type
...
Fixes https://emqx.atlassian.net/browse/EMQX-12411
2024-06-27 09:22:45 -03:00
4bd0abc93f
chore: bump app vsns
2024-06-27 09:22:06 -03:00
4c3c86e919
chore: bump wolff -> 2.0.0
2024-06-27 09:20:00 -03:00
164a507899
test(pulsar): add testcase for different producers using the same topic
2024-06-27 09:20:00 -03:00
ed5e6599d9
fix(buffer worker, kafka): send reply when async call fails immediately
...
Fixes https://emqx.atlassian.net/browse/EMQX-12585
2024-06-27 09:20:00 -03:00
6023012f8b
fix(kafka and derivatives): add `alias` config to avoid clashes with same topic
...
Fixes https://emqx.atlassian.net/browse/EMQX-12592
2024-06-27 09:20:00 -03:00
3b21c41690
feat: support to extract the client peersni field to clientinfo
2024-06-27 17:13:10 +08:00
b6e7d7566d
Merge remote-tracking branch 'origin/release-572' into release-57
2024-06-27 11:09:02 +02:00
ef28579c4a
Merge pull request #13334 from JimMoen/fix-mqtt-username-password-flag
...
fix(mqtt): check password flag to respect protocol spec
2024-06-27 16:16:53 +08:00
6a78951715
chore: log shutdown reason for check_oom trace log
2024-06-27 14:52:22 +08:00
68d747b7b9
Merge pull request #13324 from lafirest/feat/oidc-572
...
feat(sso): add OIDC support
2024-06-27 11:24:05 +08:00
3d398873f1
fix(oidc): return to dashboard when provider calls back
...
fixed a bug when updating config
2024-06-27 10:22:07 +08:00
9594b6df32
chore: warning overrided when restart authn
2024-06-27 10:16:04 +08:00
24d2534641
chore: apply review suggestion
2024-06-27 10:16:04 +08:00
5265c3cc1f
feat: support bootstrap_file on build-in-db authn
2024-06-27 10:16:04 +08:00
557a843c69
chore: minimize oss/platform diff
2024-06-26 22:48:50 +02:00
82e7b75a02
chore: bump app versions
2024-06-26 22:47:18 +02:00
98f70ea8d8
Merge remote-tracking branch 'origin/release-572' into release-57
2024-06-26 22:28:08 +02:00
d8963c836e
Merge pull request #13332 from keynslug/ft/EMQX-12571/error-mapping
...
feat(bridge-s3): provide more meaningful error details in status
2024-06-26 21:13:51 +02:00
e1de18ef10
test(dsrepl): await stable state before running testcase
2024-06-26 20:44:35 +02:00
30efa1f57e
test(dsrepl): relax crash-recover testcase to tolerate message loss
...
Which is quite an expected occasion for this kind of stress test.
2024-06-26 20:44:34 +02:00
3d296abde9
fix(dsrepl): classify ra error conditions more carefully
...
Most importantly: avoid automatic retries of `shutdown` and `nodedown`
errors as this could easily lead to Raft log entries duplication.
2024-06-26 20:44:34 +02:00
733751fadd
refactor(dsstore): keep passing `Options` to both prepare + commit
2024-06-26 20:44:34 +02:00
5b5f33c421
chore(dsstore): resurrect `prepare_batch` entry tracepoint
2024-06-26 20:44:34 +02:00
8ff48ac5ea
feat(dsrepl): rely on accumulated log size to decide when to flush
2024-06-26 20:44:34 +02:00
b6a249baa9
feat(cth-peer): add brutal `kill/1` facility
2024-06-26 20:44:34 +02:00
8db70b5bbc
test(dsrepl): add crash-restart-recover testcase
...
That verifies nothing is lost in the event of abrupt node failures.
2024-06-26 20:44:33 +02:00
ae89b61af0
feat(cth-cluster): make `restart/1` more generic
2024-06-26 20:44:33 +02:00
5fd5fc76e5
fix(dsstore): ensure backward compatibility
2024-06-26 20:44:33 +02:00
8538a5a5b6
test(dsrepl): anticipate transitionless membership changes
...
E.g. when a membership change is applied twice in a row.
2024-06-26 20:44:33 +02:00
cd0663074e
test(dsrepl): add `add_generation` events into the mix
...
They usually cause storage layer to perform flushes, and thus enable
testing `handle_flush/1` codepath in different circumstances.
2024-06-26 20:44:33 +02:00
2180cc7c26
fix(dsstore): avoid storing `cf_refs()` in the RocksDB itself
...
This is both pointless and confusing.
2024-06-26 20:44:33 +02:00
80ea2e62f7
fix(stream): ensure that `chain/1` preserves the order
2024-06-26 20:44:32 +02:00
7895e9cc45
feat(dsstore): make WAL-less mode optional
...
And make the upper layer choose when to use it.
2024-06-26 20:44:32 +02:00
0c0757b8c2
feat(dsrepl): enable WAL-less batch writes
2024-06-26 20:44:32 +02:00
2705226eb5
feat(dsrepl): release log entries occasionally
...
Also make tracepoints in `apply/3` callback implementation more
uniform.
2024-06-26 20:44:32 +02:00
b6894c18fa
chore(dsrepl): improve tracepoints usability a bit
2024-06-26 20:44:32 +02:00
263e654208
Merge pull request #13340 from thalesmg/20240626-r572-fix-docs-conf-load
...
fix: don't crash if application is already loaded
2024-06-26 15:15:49 -03:00
e81494a132
fix: don't crash if application is already loaded
...
```
Runtime terminating during boot ({{badmatch,{error,{already_loaded,wolff}}},[{emqx_conf,load,2,[{file,"emqx_conf.erl"},{line,167}]},{lists,foreach_1,2,[{file,"lists.erl"},{line,1686}]},{emqx_conf,dump_schema,2,[{file,"emqx_conf.erl"},{line,150}]},{erl_eval,do_apply,7,[{file,"erl_eval.erl"},{line,746}]},{erl_eval,expr,6,[{file,"erl_eval.erl"},{line,494}]},{erl_eval,exprs,6,[{file,"erl_eval.erl"},{line,136}]},{init,start_it,1,[]},{init,start_em,1,[]}]})
```
2024-06-26 13:41:32 -03:00
954adc71c4
test: attempt to fix flaky test
...
https://github.com/emqx/emqx/actions/runs/9662725303/job/26653594859?pr=13328#step:6:186
```
%%% emqx_resource_SUITE ==> t_expiration_retry: FAILED
%%% emqx_resource_SUITE ==> {{panic,
#{msg => "Unexpected result",
result =>
{run_stage_failed,error,
{badmatch,{ok,timeout}},
[{emqx_resource_SUITE,'-do_t_expiration_retry/0-fun-12-',0,
[{file,
"/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
{line,2569}]},
{emqx_resource_SUITE,do_t_expiration_retry,0,
[{file,
"/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
{line,2518}]}]}}},
[{emqx_resource_SUITE,do_t_expiration_retry,0,
[{file,"/__w/emqx/emqx/apps/emqx_resource/test/emqx_resource_SUITE.erl"},
{line,2594}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
{test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1302}]},
{test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1234}]}]}
```
2024-06-26 13:40:49 -03:00
ef1b6d2a55
chore: keep enterprise edition version updated as well
2024-06-26 11:40:49 +02:00
465ae507d8
chore: update release version to v5.7.1
2024-06-26 10:49:59 +02:00
af9bfc21cb
Merge pull request #13312 from zmstone/0621-fix-regex_extract-docs
...
doc: fix regex_extract function doc
2024-06-26 10:48:40 +02:00
7f73f27d56
docs: fix typo
2024-06-26 10:18:08 +02:00
4942f6f75a
feat: improve keepalive_multiplier and keepalive_check_interval
2024-06-26 15:43:48 +08:00
21c01f32ff
Merge pull request #13305 from kjellwinblad/kjell/redis_conn_always_time_out_no_username_password/EMQX-12557
...
fix: redis connector should not timeout because no username and password
2024-06-25 16:06:21 -03:00
da214be5a1
test(bridge-s3): adapt testcase to different CI environment
2024-06-25 18:35:53 +02:00
10e9fed22b
Merge pull request #13326 from keynslug/fix/ct/api-clients-flaky
...
test(api-clients): refactor suite to isolate flakiness
2024-06-25 18:26:42 +02:00
ed130fdc57
test: MQTT CONNECT flags check
2024-06-25 23:25:42 +08:00
02a9885aa5
fix(mqtt): check password flag to respect protocol spec
2024-06-25 23:25:36 +08:00
983f02ea1b
refactor: separate CONNECT flags validation funcs
2024-06-25 23:15:39 +08:00
130571b56e
fix: code improvements thanks to comments from @thalesmg
2024-06-25 12:00:36 -03:00
6190192cbc
fix: redis connector should not timeout because no username and password
...
A redis connector of type single or sentinel always got a timeout error
when doing the connector test in the dashboard if no username or
password was provided. This commit makes sure that the user instead get
an informative error message. Additionally, this commit adds more
more error information for all redis connector types.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12557
2024-06-25 12:00:36 -03:00
cc89a20ed9
Merge pull request #13318 from thalesmg/test-flaky-gcp-consu-r57-20240621
...
test(gcp consumer): stabilize flaky test
2024-06-25 11:52:20 -03:00
486a041adf
feat(bridge-s3): also map credentials / aggreg upload errors
2024-06-25 16:51:07 +02:00
fb9afd8313
feat(bridge-s3): beautify posix write errors
2024-06-25 16:51:07 +02:00
f3ffbd4710
feat(bridge-s3): provide more meaningful error details in status
2024-06-25 16:51:06 +02:00
98e4ea6fde
feat(bridge-s3): make validation errors more readable
...
And also turn them into schema-level validations, instead of
bridge-level error conditions.
2024-06-25 16:51:06 +02:00
1a497bcaf2
fix(greptime): correctly define grpc options for `grpcbox_channel`
...
Will probably fix CI flakiness.
2024-06-25 09:47:31 -03:00
c9ec5ac87b
test(gcp consumer): stabilize flaky test
...
https://github.com/emqx/emqx/actions/runs/9614788348/job/26526973635?pr=13317#step:5:1463
```
%%% emqx_bridge_gcp_pubsub_consumer_SUITE ==> t_connection_down_before_starting: FAILED
%%% emqx_bridge_gcp_pubsub_consumer_SUITE ==> {{panic,
#{msg => "Unexpected result",
result =>
{run_stage_failed,error,
{assertMatch,
[{module,emqx_bridge_gcp_pubsub_consumer_SUITE},
{line,1451},
{expression,"health_check ( Config )"},
{pattern,"{ ok , connecting }"},
{value,{ok,disconnected}}]},
[{emqx_bridge_gcp_pubsub_consumer_SUITE,
'-t_connection_down_before_starting/1-fun-11-',4,
[{file,
"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},
{line,1451}]},
{emqx_bridge_gcp_pubsub_consumer_SUITE,
t_connection_down_before_starting,1,
[{file,
"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},
{line,1427}]}]}}},
[{emqx_bridge_gcp_pubsub_consumer_SUITE,t_connection_down_before_starting,1,
[{file,
"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},
{line,1462}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
{test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1302}]},
{test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1234}]}]}
```
2024-06-25 09:36:55 -03:00
9f30da334f
Merge pull request #13248 from ieQu1/dev/EMQX-12491-local-backend
...
replace builtin DS backend with builtin_local and builtin_raft
2024-06-24 23:07:57 +02:00
d349f84f04
fix(ds): Apply remarks
2024-06-24 20:20:40 +02:00
d12b985507
test(api-clients): refactor suite to isolate flakiness
2024-06-24 16:54:20 +02:00
5d1608f34b
chore: minimize oss/platform diff
2024-06-24 15:30:03 +02:00
debf1e6cd5
Merge pull request #13316 from thalesmg/fix-mt-read-user-prop-m-20240621
...
fix(message transformation): correctly read from user properties in context
2024-06-24 09:31:48 -03:00
abc255bb02
fix(oidc): make CI happy
2024-06-24 18:11:02 +08:00
892420e2c6
feat(oidc): be more compatible with okta
2024-06-24 16:51:54 +08:00
ddb197951e
feat(oidc): implement JWKS, private_key_jwt, DPoP
2024-06-24 16:51:43 +08:00
9c0df3c0a8
feat(oidc): support the PKCE extension
2024-06-24 16:51:32 +08:00
5e2693c9b4
feat(oidc): implement session management
2024-06-24 16:51:20 +08:00
512b4b9cbb
feat(sso): add OIDC support
2024-06-24 16:50:58 +08:00
23acbb664e
Merge pull request #13258 from HJianBo/fix-boot-order-for-gws
...
fix: gateway app must depends on all auth apps
2024-06-24 16:31:27 +08:00
f9e17d6c25
Merge pull request #13269 from JimMoen/improve-plugin-behavior
...
fix: mark fresh install to cp the default configuration file directly
2024-06-24 09:17:25 +02:00
4a81c3ac18
Merge pull request #13298 from qzhuyan/fix/william/hot-conf-disable-partial-chain
...
fix(tls): disable partial_chain in hot config
2024-06-24 09:02:07 +02:00
c0472a06f1
test(ds): Set initial conditions in repl. suite explicitly
2024-06-22 02:55:05 +02:00
ec83d999bf
Merge remote-tracking branch 'origin/release-57' into 20240621-sync-r57-m
2024-06-21 14:11:08 -03:00
9a58d71378
fix(ds): Move DS backend supervision trees to their own apps
2024-06-21 18:22:04 +02:00
3d69ec496a
fix(schema): Transform config of `singleton' discovery_strategy
2024-06-21 17:16:25 +02:00
d7d878fd43
test(ds): Fix emqx standalone test profile selection
2024-06-21 17:16:25 +02:00
8aa27488b6
test: Disable certain DS-related suites in CE
2024-06-21 17:16:24 +02:00
1d3b1868fb
test(ds): Use close_db API
2024-06-21 17:16:24 +02:00
b2f7815a7f
test(ds): Don't start raft explicitly
2024-06-21 17:16:24 +02:00
3851fc189f
fix(ds): Avoid reverse dependencies from storage to repl. layer
2024-06-21 17:16:24 +02:00
bc915216a0
feat(ds): Support metrics for builtin_local backend
2024-06-21 17:16:24 +02:00
be6c5e172f
fix(ds): Disable DS management APIs for builtin_local backend
2024-06-21 17:16:24 +02:00
99c9b56cf3
feat(ds_buffer): Add `ls' function to list all local buffers
2024-06-21 17:16:24 +02:00
5a8818edf3
feat(ds): Add schema for builtin_local backend
2024-06-21 17:16:24 +02:00
4484f30021
feat(ds_schema): Add schema for builtin_local backend
2024-06-21 17:16:24 +02:00
8990b1312b
refactor(ds_schema): Rename backend builtin -> builtin_raft
2024-06-21 17:16:24 +02:00
ea48b1265d
refactor(ds_schema): Extract common builtin fields
2024-06-21 17:16:24 +02:00
abe41de19b
refactor(ds_schema): builtin_local_buffer -> builtin_buffer
2024-06-21 17:16:24 +02:00
ecb172b07e
refactor(ds): Rename egress metrics to 'buffer'
2024-06-21 17:16:24 +02:00
09c3ae795d
refactor(ds_raft): Replace egress server with common emqx_ds_buffer
2024-06-21 17:16:24 +02:00
a0fbd37e58
refactor(emqx): Use emqx_ds_backends application
2024-06-21 17:16:24 +02:00
ef09cfcd71
feat(ds): Add `emqx_ds_backends` application
2024-06-21 17:16:24 +02:00
279619fc80
feat(ds): Add `builtin_local' backend
2024-06-21 17:16:24 +02:00
a8ea0ae4e5
refactor(ds): Extract DS replication layer to a separate application
2024-06-21 17:16:24 +02:00
63f1856a2c
feat(ds): Dynamic backend registration
2024-06-21 17:16:24 +02:00
83dc8f4d77
fix(ds): Fix return values of emqx_ds_storage_layer functions
2024-06-21 17:16:23 +02:00
a18d1987a2
test(ds): Add a helper function for diffing messages
2024-06-21 17:16:23 +02:00
f1b8c356a6
Merge pull request #13299 from savonarola/0617-shared-sub-leader-poc
...
feat(queue): implement PoC version of session ↔️ shared group leader interaction
2024-06-21 16:45:17 +03:00
28c7d94bd2
fix(message transformation): correctly read from user properties in context
...
Fixes https://emqx.atlassian.net/browse/EMQX-12582
2024-06-21 09:39:22 -03:00
db28a042d5
feat(queue): handle renew_lease_timeout
2024-06-21 15:18:37 +03:00
75a524c916
test: add more debug msg to flaky cluster_rpc SUITE
2024-06-21 18:04:24 +08:00
2096755ad6
feat(queue): reorganize group sm callbacks and methods
2024-06-21 13:01:38 +03:00
b9c5911883
feat(queue): implement `find_leader_timeout` event and so the `connecting` group_sm state
2024-06-21 12:22:17 +03:00
fb266fbf8c
Merge pull request #13300 from SergeTupchiy/EMQX-12195-cluster-link-conf-backup
...
cluster link conf backup
2024-06-21 11:42:57 +03:00
8a99995810
docs: fix regex_extract function doc
2024-06-21 09:25:09 +02:00
d8e6c07ca0
chore: fix failed tests
2024-06-21 11:04:23 +08:00
a8af90f912
chore: add changes
2024-06-21 09:39:14 +08:00
08f085d823
fix: gateway app must depends on all auth apps
2024-06-21 09:39:14 +08:00
979fb58e50
feat(queue): use tp to trace new lease events
2024-06-20 21:12:15 +03:00
1205e34650
feat(queue): remove too useless header file
2024-06-20 20:48:56 +03:00
acc8bf3405
Merge pull request #13222 from JimMoen/fix-will-qos-retain-flag-check
...
fix: check willretain and willqos when WillFlag set to `true`
2024-06-21 01:28:23 +08:00
ba4fb0d3f9
Merge pull request #13290 from kjellwinblad/kjell/format_rule_cli/EMQX-12548
...
fix: do not crash when showing rule unsing the cmd interface
2024-06-20 14:55:33 +02:00
2a0071aa01
Merge pull request #13295 from thalesmg/test-flaky-greptime-mkII-r57-20240619
...
test(greptime): attempt to fix flaky tests (attempt without driver patch)
2024-06-20 14:49:21 +02:00
be175d205c
feat(queue): add JIRA task links for TODOs
2024-06-20 15:38:16 +03:00
a9c976b6c1
feat(queue): fix notes about the usage of agent<->leader protocol-related data structures
2024-06-20 15:24:25 +03:00
d0cff63ed6
feat(queue): remove unused hrl-guard
2024-06-20 15:18:47 +03:00
a92460d38f
feat(queue): improve hints for macro clauses
2024-06-20 15:17:08 +03:00
ac8762be10
chore: upgrade ekka to 0.19.5 (mria 0.8.8)
...
mria 0.8.8 heals a network partition once majority of core nodes are alive.
Previously, the autoheal worked only when all core nodes were reachable.
Fixes: EMQX-10974
2024-06-20 12:53:58 +03:00
464a0a82f0
fix(tls): move default_root_fun to ?CONST_MOD_V1
2024-06-20 11:38:52 +02:00
7d32275ebe
chore: minimize oss/platform diff
2024-06-20 10:36:01 +02:00
12859b8959
fix: handle yet another action format
2024-06-20 10:03:49 +02:00
675abd7512
test: will retain and willqos in connect flags
2024-06-20 10:32:15 +08:00
cf9d6943d5
fix: check willretain and willqos when WillFlag set to `true`
2024-06-20 10:32:15 +08:00
c307c1dfc2
chore: cut v5.7.1-alpha.1
2024-06-20 09:40:42 +08:00
6da10036dc
feat(queue): fix dialyzer issues
2024-06-19 22:25:53 +03:00
a4bbab4aa2
feat(queue): document general entity layout
2024-06-19 22:24:20 +03:00
ff742d926a
fix: `?SLOG` with msg field
2024-06-20 02:18:26 +08:00
71f5eaf11e
fix(data_backup): import `cluster.links` conf
2024-06-19 20:47:06 +03:00
ef5cf4fac3
chore(clusterlink): fix comment
2024-06-19 20:46:00 +03:00
2b0146663a
feat(queue): add smoke test for ds shared subs
2024-06-19 19:32:23 +03:00
e3c4816035
feat(queue): move group subscription state machine to its own module
2024-06-19 19:22:10 +03:00
9559ba2f06
test: remove unnecessary lines
2024-06-19 13:22:02 -03:00
8d04545f03
test(tls): root_fun is set even when partial_chain is false
2024-06-19 17:34:04 +02:00
41239ae766
fix(tls): disable partial_chain in hot config
2024-06-19 17:03:05 +02:00
021b6b3902
Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master
2024-06-19 15:06:41 +02:00
02a6ee1ef4
Merge pull request #13279 from zmstone/0617-json-decode-payload-for-map_to_redis_hset_args
...
fix(redis): json decode if arg for map_to_redis_hset_args is string
2024-06-19 14:55:21 +02:00
213e4785e7
Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master
2024-06-19 14:54:48 +02:00
a6fa3e82d9
fix: prevent use pd
2024-06-19 20:15:07 +08:00
b7b2a08399
fix: match plugin_not_found reason
2024-06-19 20:15:07 +08:00
28293284a7
fix: let crash when plugin not found in cluster
2024-06-19 20:15:07 +08:00
19e039e0d2
fix: mark fresh install to cp the default configuration file directly
2024-06-19 20:15:07 +08:00
ff2e6e1434
test(greptime): revert driver patch to test ci again
2024-06-19 09:12:30 -03:00
95d387a790
Merge pull request #13294 from zmstone/0618-per-release-edition-code-injection
...
0618 per release edition code injection
2024-06-19 11:53:57 +02:00
f0e5721959
refactor: make dialyzer happy
2024-06-19 10:42:08 +02:00
0d098a01ef
Merge pull request #13293 from SergeTupchiy/EMQX-12345-reindex-retained-msgs-on-import
...
reindex retained msgs on import
2024-06-19 11:23:08 +03:00
ffec1c7fe0
fix(greptime): attempt patch for cleaning up channels when worker terminates
2024-06-18 22:16:03 -03:00
9dd419d822
test(greptime): attempt to fix flaky tests
...
```
=WARNING REPORT==== 18-Jun-2024::12:42:04.739109 ===
reason: no_endpoints
msg: failed_to_start_greptimedb_connector
client: #{protocol => http,
pool => <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>,
cli_opts =>
[{endpoints,[{http,"toxiproxy",4001}]},
{pool_size,2},
{pool_type,random},
{auto_reconnect,1},
{gprc_options,#{sync_start => true,
connect_timeout => 5000}},
{dbname,"public"},
{auth,{basic,#{password => <<"******">>,
username => "greptime_user"}}},
{https_enabled,false}]}
connector: <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>
```
2024-06-18 18:23:22 -03:00
54fc605cc5
refactor: move schema injection from hrl to erl
2024-06-18 22:22:53 +02:00
a53fa977df
refactor: delete an unused macro
2024-06-18 22:22:48 +02:00
e69ba33420
Merge pull request #13291 from thalesmg/fix-ds-mgmt-node-down-r57-20240618
...
fix(dsreplmeta): check site status when fetching shard info
2024-06-18 15:14:08 -03:00
bca743054b
feat(queue): implement backbones of queue agent, leader and leader registry
2024-06-18 21:03:51 +03:00
4c54ab6379
test: remove tests per review request
2024-06-18 13:08:19 -03:00
4a318f8f51
style: fix type problem found by dialyzer
2024-06-18 17:02:26 +02:00
baa7996289
fix(data_backup): allow exporting `ram_copies` Mnesia tables
...
Currently, ram tables can be used for message retainer.
2024-06-18 17:31:33 +03:00
60882a616e
fix(data_backup): re-index retained messages after importing
2024-06-18 17:31:19 +03:00
6bde6aa711
Merge pull request #13280 from thalesmg/fix-plugin-elixir-r57-20240617
...
fix(plugins): do not stop Elixir protected apps
2024-06-18 10:41:12 -03:00
8ce16fd7d9
fix(dsreplmeta): check site status when fetching shard info
...
Fixes https://emqx.atlassian.net/browse/EMQX-12356
2024-06-18 10:40:09 -03:00
a4f855108c
fix: do not crash when showing rule unsing the cmd interface
...
Before the change the command line command
$ bin/emqx ctl rules show rule_0hyd
would crash if the rule had bridge actions. This has now been fixed by
adding a handler for printing bridge actions.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12548
2024-06-18 15:17:25 +02:00
3f7723b2dc
Merge pull request #13274 from kjellwinblad/kjell/add_ssl_to_rocksdb/EMQX-12289
...
feat: add SSL support to RocketMQ connector
2024-06-18 09:38:28 +02:00
20cffb54d4
Merge branch 'release-57' into sync-r57-m-20240617
2024-06-17 17:10:12 -03:00
6897f0141b
fix(plugins): do not stop Elixir protected apps
...
On one hand, Elixir plugins might include Elixir itself, when targetting a non-Elixir EMQX
release. If, on the other hand, the EMQX release already includes Elixir, we shouldn't
stop Elixir nor IEx, or the running IEx shell will break.
2024-06-17 16:42:23 -03:00
ad993437aa
Merge pull request #13278 from savonarola/0617-release-57-sync
...
Sync release-57
2024-06-17 21:25:00 +03:00
9479c8d33b
fix(redis): json decode if arg for map_to_redis_hset_args is string
2024-06-17 20:19:53 +02:00
eede9f349e
Merge pull request #13276 from keynslug/fix/ds/inherited-lts-lost
...
fix(dsstore): persist inherited LTS trie
2024-06-17 20:25:31 +03:00
ae22a64157
Merge pull request #13277 from zmstone/0617-handle-kafka-message_too_large-return
...
0617 handle kafka message too large return
2024-06-17 19:15:33 +02:00
0cbbae9655
fix: make fixes due to comments from @thalesmg
2024-06-17 19:06:12 +02:00
c39c544c96
feat: add SSL support to RocketMQ connector
2024-06-17 19:06:12 +02:00
b64f0c0ca7
Merge pull request #13231 from thalesmg/mt-test-endpoint-r57-20240611
...
feat(message transformation): implement dryrun endpoint
2024-06-17 13:11:39 -03:00
240bb09070
Merge pull request #13275 from thalesmg/test-flaky-sk-r57-20240617
...
test: attempt to fix flaky test
2024-06-17 13:01:08 -03:00
f8e6aab86f
Merge branch 'release-57' into 0617-release-57-sync
...
* release-57:
chore(auth,http): cache REs for parsing URIs
fix(auth,http): improve URI handling
chore: revert ULOG/ELOG
test: generate dispatch.eterm in dashboard test
docs: refine change log
feat: make the dashboard restart quicker
chore: fix typo
fix(http authz): handle unknown content types in responses
chore: change types of mysql and mongodb fields to `template()`
fix(client mgmt api): allow projecting `client_attrs` from client fields
fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-17 18:53:39 +03:00
82588fbc35
chore(dsstore): refine module and callback typespecs
2024-06-17 17:53:39 +02:00
05d97397d3
fix(dsstore): use correct comparison when walking over generations
2024-06-17 17:53:39 +02:00
22009bcc58
test(dsstore): verify that inherited TLS trie is persisted
2024-06-17 17:53:38 +02:00
68f6556856
fix(dsstore): persist inherited LTS trie
...
Before this commit, inherited trie was actually only kept in memory
cache.
Also simplify storage backend behaviour around inheriting previous
generation's legacy.
2024-06-17 17:53:38 +02:00
830266b4d5
Merge pull request #13126 from SergeTupchiy/EMQX-11967-cross-cluster-route-replication-syncer-actor
...
cluster linking
2024-06-17 18:52:46 +03:00
3ac4ddcbe3
fix(kafka): handle message_too_large
...
bump 'failed' counter
2024-06-17 17:27:25 +02:00
00dfdc22cf
chore: pin Kafka client wolff 0.10.5
2024-06-17 17:27:25 +02:00
5ab4b321a0
Merge pull request #13267 from zhongwencool/use-redbug-for-trace
...
chore(debug): use redbug for default trace tools
2024-06-17 22:10:18 +08:00
fb492e3dc5
Merge pull request #13264 from thalesmg/fix-postgres-disabled-prepared-r57-20240614
...
fix(postgres): authn/authz/batch requests when prepared statements are disabled
2024-06-17 11:05:05 -03:00
5665a4917c
Merge pull request #13266 from thalesmg/fix-s3-release-app-r57-20240614
...
fix: add `emqx_s3` to `reboot_lists.eterm`
2024-06-17 10:48:12 -03:00
ec4f462684
refactor: apply review requests
2024-06-17 10:39:32 -03:00
1464094967
test: attempt to fix flaky test
...
```
%%% emqx_persistent_session_ds_SUITE ==> t_session_discard_persistent_to_persistent: FAILED
%%% emqx_persistent_session_ds_SUITE ==> {noproc,{gen_server,call,[snabbkaffe_collector,flush_trace,infinity]}}
```
2024-06-17 10:34:57 -03:00
b71212b022
refactor: apply review comments
2024-06-17 09:40:33 -03:00
f5eb3e7471
Merge pull request #13250 from thalesmg/ds-singleton-m-20240613
...
feat: enforce singleton discovery strategy when using DS `builtin_local` backend
2024-06-17 09:34:05 -03:00
19c9f0d76f
chore(auth,http): cache REs for parsing URIs
2024-06-17 14:42:29 +03:00
f7ac829f28
fix(auth,http): improve URI handling
2024-06-17 14:42:29 +03:00
a905a6048c
chore(clusterlink): rename link `upstream` field to `name`
2024-06-17 12:24:51 +03:00
a95a08efd3
test(clusterlink): add more test cases
2024-06-17 12:24:51 +03:00
44c37571cc
fix(clusterlink): ignore not_registered error
2024-06-17 12:24:51 +03:00
00f912928f
fix: fix emqx_external_broker:forward/1 spec
2024-06-17 12:24:51 +03:00
ed16ff07df
refactor(broker): simplify external broker behaviour
2024-06-17 12:24:51 +03:00
d282c61120
feat(clusterlink): update only necessary resources when a link config is changed
2024-06-17 12:24:50 +03:00
ff16521d4f
fix(clusterlink): add schema descriptions, fix dialyzer warnings, add resource_opts
2024-06-17 12:23:52 +03:00
94e81ba812
feat(clusterlink): implement actor config handler
2024-06-17 12:23:52 +03:00
c871b37453
fix(clusterlink): add link topics schema validator
2024-06-17 12:23:52 +03:00
780a0bf807
fix(cluster-link): clear exit signal of failed-to-connect client
2024-06-17 12:23:52 +03:00
d0df4de2a3
test(cluster-link): add e2e replication actor GC testcase
2024-06-17 12:23:52 +03:00
c4840b30d2
fix(cluster-link): deduplicate routes down to dest cluster
2024-06-17 12:23:52 +03:00
ede35df24a
fix(cluster-link): cancel heartbeats on client down
2024-06-17 12:23:52 +03:00
e0604e3af6
fix(cluster-link): anticipate clients may occasionally retry
2024-06-17 12:23:52 +03:00
0219b8bd4d
feat(cluster-link): add simple replication actor GC process
2024-06-17 12:23:52 +03:00
7fccb5dbc9
test(topic): add more `intersection/2` testcases
2024-06-17 12:23:52 +03:00
7b8f466adf
feat(topic): avoid `lists:reverse` when intersecting
2024-06-17 12:23:52 +03:00
24be189728
fix(topic): respect special topic rules when intersecting
2024-06-17 12:23:51 +03:00
de1ac131f7
test(cluster-link): fix test suite setup
2024-06-17 12:23:51 +03:00
58eaf07627
fix(clusterlink): valide config to disallow duplicated cluster links
2024-06-17 12:23:51 +03:00
54d51d0982
test(cluster-link): draft basic integration test suite
2024-06-17 12:23:51 +03:00
e9c24090d4
fix(cluster-link): avoid starting ps syncer if persistence disabled
2024-06-17 12:23:51 +03:00
036c7e8492
test(cluster-link): adapt extrouter testsuite to new APIs
2024-06-17 12:23:51 +03:00
45eda4f3b9
fix(cluster-link): adapt heartbeat / reincarnation handling to new API
2024-06-17 12:23:51 +03:00
43d114546c
feat(cluster-link): preserve replication actor state in pdict
2024-06-17 12:23:51 +03:00
5771a41a32
fix(cluster-link): ensure replication actor bootstraps do heartbeats
2024-06-17 12:23:51 +03:00
d4b449c6e1
feat(cluster-link): implement replication actor heartbeats
2024-06-17 12:23:51 +03:00
faa4420e1f
fix(clusterlink): improve actor error handling
...
Add status and error reason to the actor state, report alarms.
2024-06-17 12:23:51 +03:00
21711c6e0d
fix(clusterlink): communicate bootstrap requirements via actor handshake
...
`session_present` flag is not reliable to decide whether bootstrap is needed if durable sessions is enabled.
In this case, the client session may survive cluster restart while all the external routes are lost, as they are not persistent.
2024-06-17 12:23:51 +03:00
d5e82cdfac
refactor(clusterlink): avoid unnecessary `maybe_` external_broker CB names
2024-06-17 12:23:51 +03:00
d578ac3f9e
fix(clusterlink): match correct timer ref in router actor
2024-06-17 12:23:51 +03:00
b1aeb35370
feat(clusterlink): implement actor init handshake
2024-06-17 12:23:51 +03:00
ac19cf89df
chore(clusterlink): remove code related to the rejected coordinator-based implementation
2024-06-17 12:23:51 +03:00
e26e7acaa1
refactor(clusterlink): use `emqx_bridge_mqtt_lib:clientid_base/1` to construct routesync client id
2024-06-17 12:23:51 +03:00
e7305c62ee
feat(clusterlink): replicate shared subscription and persistent session routes
2024-06-17 12:23:51 +03:00
f036b641eb
feat(clusterlink): integrate node local routes replication and message forwarding
2024-06-17 12:23:51 +03:00
7df91d852c
feat(clusterlink): integrate node local syncer/actor implementation
...
- support multiple cluster links in extrouter
- apply extrouter ops on 'message.publish' hook
- fix several minor bugs
2024-06-17 12:23:51 +03:00
cbd01ae818
feat(clusterlink): add node-local route sync actor implementation
2024-06-17 12:23:51 +03:00
7b95273218
feat(routesync): make syncer a bit more generic and reusable
2024-06-17 12:23:51 +03:00
5bd9ee5c7f
feat(utils): add `emqx_utils_ets:keyfoldl/3` function
...
Designed to be used with `bag` / `duplicate_bag` tables.
2024-06-17 12:23:51 +03:00
4097585f5d
fix(clusterlink): ensure extrouter works on replicants
...
This is sort of a quick fix to make things safe, but it will likely be
a subject to the same drawbacks as the regular router in high-latency
deployments: reduced throughput.
2024-06-17 12:23:51 +03:00
a53524c826
fix(cth-cluster): fix occasional case clauses during cluster bootup
2024-06-17 12:23:51 +03:00
2dd99c5a08
feat(clusterlink): add facility to reconstruct remote routing table
2024-06-17 12:23:51 +03:00
f08342c704
feat: cluster link prototype WIP
2024-06-17 12:23:48 +03:00
98a54994c0
Merge pull request #13196 from lafirest/fix/rules_len
...
fix(authz_mnesia): add a soft limit in the API for the length of ACL rules
2024-06-17 15:47:59 +08:00
17809a5803
chore: fix dialyzer warning
2024-06-17 14:07:30 +08:00
1716852057
fix(authz_mnesia): improve field names and changes
2024-06-17 10:09:25 +08:00
a49e5877ec
chore(debug): use redbug for default trace tools
2024-06-17 09:28:23 +08:00
8daaf7b727
Merge pull request #13233 from zmstone/0611-expose-regex_extract-func
...
fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-16 16:16:48 +02:00
2b7f3a597f
Merge pull request #13245 from qzhuyan/merge/william/sync-rel57-to-master
...
sync release57 to master
2024-06-15 09:13:55 +02:00
d433fc7c30
Merge pull request #13236 from zhongwencool/bug-schema-validation
...
fix: some minor bug fixes
2024-06-15 06:40:28 +08:00
198ccd320b
fix: add `emqx_s3` to `reboot_lists.eterm`
...
Otherwise, it's unconditionally started as a permanent application instead of being
controlled by `emqx_machine`.
```sh
ͳ rg start_boot _build/emqx-enterprise/rel/emqx/releases/5.7.0-g4e6be8a6/start.script | rg emqx_s3
{apply,{application,start_boot,[emqx_s3,permanent]}},
```
2024-06-14 15:06:18 -03:00
6e702cd16c
fix(postgres): fix usage by authn/authz modules when prepared statements are disabled
...
Fixes https://emqx.atlassian.net/browse/EMQX-12550
2024-06-14 12:19:46 -03:00
626aae6edf
chore: fix bad conflict resolution
2024-06-14 16:57:53 +02:00
d95f17fe77
chore: revert ULOG/ELOG
2024-06-14 22:11:27 +08:00
fd5e844cc1
fix(postgres): correctly handle batch queries with disabled prepared statements
...
Fixes https://emqx.atlassian.net/browse/EMQX-12549
2024-06-14 11:10:21 -03:00
f713f13b2c
test: generate dispatch.eterm in dashboard test
2024-06-14 21:52:58 +08:00
9b3c806ba7
feat(message transformation): implement dryrun endpoint
...
Follow up to https://github.com/emqx/emqx/pull/13199
2024-06-14 10:19:36 -03:00
a6e3a09118
feat: make the dashboard restart quicker
2024-06-14 18:22:34 +08:00
0f0e7d18db
Merge pull request #13261 from JimMoen/fix-typo
...
chore: fix typo
2024-06-14 16:55:35 +08:00
6823c79ae0
chore: fix typo
2024-06-14 16:04:11 +08:00
b86d631744
test: fix tc t_handle_outing_non_utf8_topic
2024-06-14 08:17:19 +02:00
d39b8caff9
Merge branch 'release-57' into merge/william/sync-rel57-to-master
2024-06-14 08:17:09 +02:00
afd4b46f72
Merge pull request #13225 from lafirest/fix/mysql_redact
...
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-14 09:19:21 +08:00
01635722e9
docs: improve descriptions
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2024-06-13 18:34:44 -03:00
52e6c88941
feat: enforce singleton discovery strategy when using DS `builtin_local` backend
...
Fixes https://emqx.atlassian.net/browse/EMQX-12493
2024-06-13 17:04:23 -03:00
cba5c7bb45
refactor: rm dead code
2024-06-13 16:57:34 -03:00
4d876f2af2
chore: bump ekka -> 0.19.4
...
https://github.com/emqx/ekka/pull/236
2024-06-13 16:35:58 -03:00
3adf64e637
Merge pull request #13240 from thalesmg/ds-cluster-purge-m-20240612
...
feat: support purging durable sessions during cluster purge
2024-06-13 09:30:18 -03:00
6ccf1dcbf9
Merge pull request #13238 from thalesmg/fix-authz-http-content-type-handling-r57-20240612
...
fix(http authz): handle unknown content types in responses
2024-06-13 09:10:43 -03:00
62cf277cd7
Merge pull request #13235 from thalesmg/fix-template-types-r57-20240611
...
chore: change types of mysql and mongodb fields to `template()`
2024-06-13 09:08:40 -03:00
c81a3ebc0a
Merge pull request #13234 from thalesmg/fix-clients-api-client-attrs-union-r57-20240611
...
fix(client mgmt api): allow projecting `client_attrs` from client fields
2024-06-13 09:08:29 -03:00
1664ea4ad4
Revert: TLS partial-chain and keyUsage #12955 #12977
...
This reverts commit 28b17a256201467246fcc3f8ba57621a4a4bb3a5fb30207ef3337c230e793a674f44f170ffd77f9903b0935564650cf4b27e43ad665dcfa29a43e5fc4e9c1ec0c98eb463c58d90430fa66deb1ab9adfe8bc3a86f63fa4357ce890b95a08d32
2024-06-13 13:46:45 +02:00
723144199e
Merge pull request #13211 from qzhuyan/fix/william/rel57/ee-tls
...
Fix/william/rel57/ee tls
2024-06-13 13:19:44 +02:00
0f53b33417
chore(s3): make schema fields with defaults optional
2024-06-13 11:57:36 +02:00
0eee2edd28
test: fix api_config SUITE failed
2024-06-13 14:55:24 +08:00
2b1fa1711b
fix: observer command crash when can't find object code
2024-06-13 14:55:23 +08:00
623845d07d
fix: don't allow set active_n to negative int
2024-06-13 14:55:23 +08:00
825bfe4717
fix: load bad configs return unknown msg
2024-06-13 14:55:23 +08:00
5d3f464ec3
fix: ws/wss's max_frame_size should > 0
2024-06-13 14:55:23 +08:00
800a79adde
test: test failed with_mfa
2024-06-13 09:50:52 +08:00
2c264d9a4b
fix(http authz): handle unknown content types in responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-12530
2024-06-12 14:31:12 -03:00
c72b455a83
feat: add hidden option to record MFA and line in the log is very useful for debug.
2024-06-12 22:51:45 +08:00
d0e6f22a79
feat: support purging durable sessions during cluster purge
...
Fixes https://emqx.atlassian.net/browse/EMQX-12405
2024-06-12 11:19:53 -03:00
daeb10151b
chore: fix typos
2024-06-12 19:06:34 +08:00
cde4cb1358
fix: metrics macro with description
2024-06-12 19:06:33 +08:00
756797b258
refactor: gen metric schema with desc from macros
2024-06-12 19:04:39 +08:00
263d2dbae2
fix: inc metric `'client.auth.anonymous'` when client anonymous
2024-06-12 19:04:39 +08:00
9a78a6f640
refactor: mv metrics macros in hrl file
2024-06-12 19:04:29 +08:00
cbac4019b3
fix: rm havn't used counter idx
2024-06-12 19:01:50 +08:00
46ab3be1f4
chore: change types of mysql and mongodb fields to `template()`
...
Fixes https://emqx.atlassian.net/browse/EMQX-12395
2024-06-11 17:25:53 -03:00
0874768c1d
fix(client mgmt api): allow projecting `client_attrs` from client fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-12511
2024-06-11 17:14:33 -03:00
8a283b1cc0
Merge pull request #13180 from zmstone/0604-optimize-tcp-ssl-connection-data-send-performance
...
0604 optimize tcp ssl connection data send performance
2024-06-11 22:06:57 +02:00
09f91159c9
fix(emqx_rule_funcs): expose regex_extract function to rule engine
...
previoulsy only available in variform expressions,
now made available for rule-engine
2024-06-11 21:31:13 +02:00
51810c6919
Merge pull request #13229 from zmstone/0611-auth-with-raw-cert
...
feat(authn): support `${cert_pem}` placeholder
2024-06-11 21:16:01 +02:00
db572d35a7
Merge remote-tracking branch 'origin/release-57' into sync-r57-m-20240611
2024-06-11 15:34:54 -03:00
751f7a24e9
feat(authn): support ${cert_pem} placeholder
2024-06-11 19:54:24 +02:00
337009c3a0
fix: declare `emqx_schema_registry` as a dependency of `emqx_schema_validation`
2024-06-11 14:03:27 -03:00
0f9c3b4cea
feat(schema validation): check references schema names and types before changing config
...
Fixes https://emqx.atlassian.net/browse/EMQX-12368
2024-06-11 14:03:26 -03:00
03b226248a
feat(schema registry): add check for inner types
...
Currently, only `protobuf` has any.
2024-06-11 13:50:11 -03:00
c70e8252fe
fix: declare `emqx_schema_registry` as a dependency of `emqx_message_transformation`
2024-06-11 13:35:02 -03:00
b2d716909f
Merge pull request #13200 from thalesmg/update-erlazure-no-gen-server-m-20240606
...
chore: refactor azure blob connector to use new `erlazure` without `gen_server`
2024-06-11 13:30:14 -03:00
686bcc8a48
Merge pull request #13207 from thalesmg/republish-metrics-m-20240607
...
fix(rule actions): check `republish` action result for metrics
2024-06-11 13:29:58 -03:00
2c6aa80b3c
Merge pull request #13221 from thalesmg/refactor-sv-topic-index-r57-20240610
...
refactor(schema validation): module organization and position indexing
2024-06-11 13:29:48 -03:00
83ff07ade0
Merge pull request #13199 from thalesmg/message-transformation-r57-20240604
...
feat: implement message transformation
2024-06-11 13:29:25 -03:00
ab5329be02
Merge pull request #13220 from thalesmg/refactor-channel-hc-opt-r57-20240610
...
fix: remove another redundant health check
2024-06-11 12:22:43 -03:00
2f0eb3560b
Merge pull request #13227 from keynslug/fix/EMQX-12533/bad-tpl
...
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 15:39:12 +03:00
d4fb812352
fix(aggreg): anticipate and handle delivery startup errors
2024-06-11 13:21:41 +02:00
29fc30ea69
fix(bridge-s3): validate aggreg key template before adding a channel
2024-06-11 13:21:41 +02:00
fb0da9848c
feat(tpl): add separate `placeholders/1` function
...
The purpose is to have a clearer view of placeholders used in a
template, without going the usual `render(Template, #{})` route that is
actually subtly misleading: it won't mention that `${}` / `${.}`
placeholder has been used.
Also unify handling of `${}` / `${.}` in a couple of places.
2024-06-11 11:42:43 +02:00
9d22089104
Merge pull request #13173 from kjellwinblad/kjell/pgsql_timestamp_encode_etc/EEC-1036
...
fix(pgsql connector): handle prepared statement already exists
2024-06-11 10:53:30 +02:00
44258204bd
chore: move tls_certs test helper to apps/emqx
2024-06-11 10:06:18 +02:00
e64f60b73f
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-11 16:05:11 +08:00
b657dc537c
Merge pull request #13156 from savonarola/0529-fix-dashboard-monitor
...
fix(dashboard_monitor): correctly merge old and new sample data
2024-06-11 10:33:15 +08:00
05ebb17cd6
refactor: index positions rather than names
...
Addresses https://github.com/emqx/emqx/pull/13199#discussion_r1633096025
2024-06-10 15:54:37 -03:00
5eff4a7544
refactor(schema validation): improve api and module organization
...
To conform to comment from another PR:
https://github.com/emqx/emqx/pull/13199#discussion_r1632658742
... since schema validation is the "inspiration" for message transformation.
2024-06-10 15:46:28 -03:00
d93c8540c8
feat: pass along peername down to transformation context
2024-06-10 15:35:35 -03:00
6e0ef893f4
feat: pass along client attributes down to message transformation context
2024-06-10 15:27:04 -03:00
3c501e4f2a
fix: remove another redundant health check
...
Fixes https://emqx.atlassian.net/browse/EMQX-12521
2024-06-10 15:13:34 -03:00
1a20a40218
Merge pull request #13209 from thalesmg/rm-redundant-hc-r57-20240607
...
chore: remove redundant health check call
2024-06-10 14:58:48 -03:00
e586178479
chore: refactor azure blob connector to use new `erlazure` without `gen_server`
...
https://github.com/dkataskin/erlazure/pull/43 removes unnecessary usage of `gen_server`
from the driver.
2024-06-10 13:42:28 -03:00
5629fe60c1
refactor: index positions rather than names
2024-06-10 13:33:22 -03:00
8c7a7cf9db
refactor: apply review remarks
2024-06-10 11:17:13 -03:00
e54cf2f218
refactor: use option to skip rendering as string
2024-06-10 10:49:39 -03:00
d34558954d
refactor: improve api and module organization
2024-06-10 10:40:53 -03:00
4aa1c3842f
fix: declare `emqx` as a dependency in `.app.src`
2024-06-10 09:53:29 -03:00
aa416cc0ad
docs: fix stale comments and dates
2024-06-10 09:53:02 -03:00
7a09b0470d
chore: cosmetic changes
2024-06-10 14:22:25 +02:00
a885f0b41a
test(emqx_bridge_pgsql_SUITE): call test janitor
2024-06-10 13:45:27 +02:00
a1aa9a4375
fix(ee): emqx no longer deps on emqx_auth_ext
2024-06-10 10:32:01 +02:00
4347f3de3e
fix(bridge/mqtt): respect client ID prefix
2024-06-10 10:27:45 +02:00
76cfc309a9
feat(tls): partial_chain not required
2024-06-10 10:25:44 +02:00
7c37bf9965
feat(tls): ee only: TLS partial_chain and Keyusage
2024-06-10 10:24:48 +02:00
e60380d205
chore: fix nit
2024-06-10 09:55:12 +02:00
e9b813d8ef
chore: fix test
2024-06-10 09:54:54 +02:00
1739bc0c24
feat(partial_chain): gateway support
2024-06-10 09:54:44 +02:00
45dd7816d7
Merge remote-tracking branch 'origin/release-57'
2024-06-09 10:11:15 +02:00
abbf2ef62f
chore(TLS-chain-test): update for OTP 26
2024-06-08 00:16:06 +02:00
221b748b0f
test(partial_chain): update tcs for OTP-25
2024-06-08 00:16:01 +02:00
463d1a1875
fix(test): tls_verify_partial_chain
2024-06-08 00:15:54 +02:00
788cdbc6dd
fix(listener): remove partial_chain in wss opts
2024-06-08 00:15:51 +02:00
3fc99315e0
chore: happy elvis
2024-06-08 00:15:45 +02:00
c5dccdf526
feat(tls): update schema for TLS keyusage
2024-06-08 00:15:43 +02:00
aa25e3badd
fix(tls): undefined keyusage
2024-06-08 00:15:39 +02:00
9e5cfea8c0
test(tls): verify peer keyusage
2024-06-08 00:15:36 +02:00
c5657029ab
feat(config): partial_chain
2024-06-08 00:15:34 +02:00
f7ff9496e6
test: port listener tls partial_chain
2024-06-08 00:15:31 +02:00
1ce13242a8
feat(tls): port partial_chain, part 1
2024-06-08 00:15:28 +02:00
c13631102e
fix(PgSQL connector): typo in function name
2024-06-07 19:40:18 +02:00
b2f027c974
Merge pull request #13197 from keynslug/fix/EMQX-12522/conv-ssl
...
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-07 19:45:53 +03:00
39615e1cb6
fix(rule actions): check `republish` action result for metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12328
2024-06-07 11:17:43 -03:00
28a3c77e88
chore: remove redundant health check call
...
Since `emqx_resource_manager:channel_health_check` already does a base resource health
check, the call was repeated.
2024-06-07 11:07:12 -03:00
e63dcc84b0
fix: unused variable and better error message
2024-06-07 12:20:42 +02:00
a3f3f96781
test(emqx_bridge_v2_pgsql_SUITE): fix broken test case
...
We have to delete the connector and action created by the previous step
of the test case so we don't get a name conflict.
2024-06-07 12:03:34 +02:00
78a196e215
Merge pull request #13189 from JimMoen/fix-apply-sqlserver-template
...
fix(utils_sql): improve insert sql regular expression
2024-06-07 13:50:19 +08:00
eb215726d5
Merge pull request #13195 from lafirest/fix/typos
...
fix: fix typos
2024-06-07 09:04:16 +08:00
ef36350bf6
feat: implement message transformation
...
Fixes https://emqx.atlassian.net/browse/EMQX-12517
2024-06-06 17:51:08 -03:00
8a85d7cb5a
fix(bridge-s3): pass SSL options through `convert_certs/2`
2024-06-06 19:37:38 +02:00
c17c788438
feat(connector): allow delegate `pre_config_update` to impls
2024-06-06 19:37:34 +02:00
ebf17c8143
Merge remote-tracking branch 'origin/release-57' into 0606-merge-release-57-to-master
2024-06-06 17:43:36 +02:00
3ae26c8a54
chore: update changes
2024-06-06 18:13:24 +08:00
5532c7b0a6
fix(authz_mnesia): add a soft limit in the API for the length of ACL rules
2024-06-06 16:22:53 +08:00
2525bd0c16
perf: eliminate the loop-back fake async inet_reply message
...
Now that EMQX is on OTP 26 by default, the fake inet_reply
message when the send call returns ok only adds overhead.
For TCP (gen_tcp module), OTP 26 no longer supports direct port_command
so we had to swith to gen_tcp:send, then fake an async reply:
see https://github.com/emqx/esockd/pull/181
For TLS (ssl module), it has never supported async send anyway.
This commit switches to call Transport:send/2 directly,
this should result in one less loop-back message for each message.
For OTP 25 on which EMQX can still be compiled, one will have to
suffer the performance penalty for gen_tcp:send function to
do a non-optimized 'receive' for the send result.
2024-06-06 09:35:51 +02:00
5c3f5d8085
perf: do not call inet getstat before each and every send
...
In a stress test environment, when alarm is enabled, there were
a lot of long_schedule warnings with the stacktrace pointing to
congestion alarm based inet:getstat function.
When alarm is disabled, the one single client is able to
hold 40,000 QoS 1 messages per second (inflight=32) throughput.
When alarm is enabled, its mqueue overflows.
This commit removes the call before each and every data send,
so to rely on emit_stats timer to trigger the congestion check.
2024-06-06 09:35:51 +02:00
d4118c6e8e
fix: fix typos
2024-06-06 11:19:40 +08:00
dff31b293c
perf(utils_sql): pre-build regexp
2024-06-06 11:01:38 +08:00
c41be7ee22
chore: fix dialyzer warning
2024-06-05 23:37:21 +02:00
dc2e6d1695
Merge pull request #13163 from savonarola/0523-ds-shared-subs-dispatch
...
Inject shared subcription handling into durable session
2024-06-05 23:59:50 +03:00
159da912c2
feat(emqx_dashboard_swagger): apply field validator in request body check
2024-06-05 19:35:04 +03:00
144264e0d8
test: add utils_sql test cases
2024-06-06 00:30:18 +08:00
627b8c45d2
fix(utils_sql): improve insert sql regular expression
2024-06-06 00:30:17 +08:00
b075b7120c
feat(sessds): implement dispatchig between CE/EE shared sub agents
2024-06-05 19:17:58 +03:00
60556abc4f
fix(dashboard_monitor): correctly merge old and new sample data
2024-06-05 18:49:52 +03:00
2956e849eb
fix(pgsql connector): better msg when failing to remove statement
2024-06-05 16:58:14 +02:00
336089f8a7
fix: bug found by dialyzer and make test case cleaner
2024-06-05 15:53:02 +02:00
ae0379f974
Merge pull request #13181 from thalesmg/force-stop-connector-r57-20240604
...
fix(resource manager): force kill process if stuck when stopping/removing
2024-06-05 09:08:30 -03:00
d79193fde4
Merge pull request #13177 from lafirest/sync-authn-com
...
feat: sync `client.check_authn_complete` to release-57
2024-06-05 19:51:10 +08:00
5eab221f7c
Merge pull request #13175 from thalesmg/disable-prepared-statements-postgres-r57-20240603
...
feat(postgres): add `disable_prepared_statements` option
2024-06-04 12:29:53 -03:00
f17aefe3d7
fix(action/source api): improve returned error message on timeout
2024-06-04 12:04:55 -03:00
cda6d5f636
Merge pull request #13172 from zmstone/0603-add-redis-arg-formatter-to-rule-engine-funcs
...
feat(redis): add a rule function to help formatting redis args
2024-06-04 16:56:14 +02:00
3013189cd7
fix(resource manager): force kill process if stuck when stopping/removing
...
Fixes https://emqx.atlassian.net/browse/EMQX-12357
2024-06-04 11:38:24 -03:00
9cdfbb0845
feat(sessds): make ds session be aware of shared subscriptions
2024-06-04 17:15:30 +03:00
c07bc68e6f
feat(postgres): add `disable_prepared_statements` option
...
Fixes https://emqx.atlassian.net/browse/EMQX-12496
Some Postgres connections, such ones made to [PGBouncer](https://www.pgbouncer.org/ ) or
[Supabase in Transaction Mode](https://supabase.com/ ), do not support some session
features like prepared statements.
2024-06-04 10:11:25 -03:00
87384cae32
fix(events): call `client.check_authn_complete` even if authentication fails
2024-06-04 17:55:39 +08:00
92d26ff27b
chore: update change
2024-06-04 17:55:30 +08:00
e4e53844d5
feat(events): add new hook && event `client.check_authn_complete`
2024-06-04 17:55:21 +08:00
dc95218c68
Merge pull request #13144 from savonarola/0528-increase-bridge-buffer-drop-log-level
...
chore(bridges): increase log level for bridge buffer overflow reports
2024-06-04 12:37:50 +03:00
572ca6433e
fix(pgsql connector): improvements due to suggestions from @thalesmg
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-06-04 10:11:45 +02:00
a8a67a2ac9
Merge pull request #13151 from zhongwencool/authz-trace-log
...
chore: make authz's logs easier to understand
2024-06-04 11:12:09 +08:00
aa7ce1f641
fix(bridge/redis): add test case for map_to_redis_hset_args
2024-06-03 23:05:43 +02:00
f2ccfff803
fix(pgsql connector): handle prepared statement already exists
...
In a user's log file it was found that that the pgsql driver can end up
in a situation where the prepared statement for a channel/action is not
properly removed before a channel with the same name as the prepared
statement is added to the connector. This commit handles this by
attempting to remove the old prepared statement if one already exists
when adding channel.
Related issue:
https://emqx.atlassian.net/browse/EEC-1036
2024-06-03 16:14:26 +02:00
e3ed7b59dd
feat(redis): add a rule function to help formatting redis args
...
The new function named 'map_to_redis_hset_args' can be used to format
a map's key-value pairs into redis HSET (or HMSET) arg list.
This new function is dedicated for redis to avoid abuse for other
data integrations.
2024-06-03 14:09:15 +02:00
c5547543e0
Merge pull request #13069 from thalesmg/azure-blob-storage-action-m-20240508
...
feat: implement azure blob storage action
2024-06-03 08:56:37 -03:00
193978af90
Merge pull request #13168 from savonarola/0530-fix-mgmt-test
...
chore(tests): improve test stability
2024-06-03 14:44:12 +03:00
fda365a87b
chore: make authz's logs easier to understand
2024-06-03 18:01:57 +08:00
21624bc865
fix: get plugin app vsn from field `rel_apps`
2024-06-03 15:26:14 +08:00
b6ff67d712
Merge pull request #13093 from zmstone/0522-upgrade-to-brod-3.18
...
chore: upgrade to brod 3.18.0
2024-06-02 15:06:40 +02:00
8d22a5a3ae
Merge pull request #13167 from JimMoen/fix-rule-events-i18n-todo
...
fix: missing rule event i18n in zh_cn
2024-06-02 18:23:20 +08:00
9637b14543
fix: upgrade to brod 3.18.0 to fix unnecessary rebalance
2024-06-01 10:57:24 +02:00
57741a41e9
chore(tests): improve test stability
2024-05-31 22:27:31 +03:00
fb07f40bea
test: fix default value of retry_interval in test case
2024-05-31 17:53:12 +02:00
20c92407c7
feat: change message retry_interval default value to infinity
...
Previous Setting: The default value for `retry_interval` was 30 seconds.
New Default: The default `retry_interval` has been changed to 'infinity'.
With this update, EMQX will not automatically retry message deliveries
by default.
Compliance and Compatibility: Aligning with MQTT specification
standards, in-session message delivery retries are not typically
compliant. We recognize that some users depend on this feature, so the
option to configure retries remains available for backward
compatibility.
2024-05-31 17:53:12 +02:00
37ce8d93a1
Merge pull request #13164 from zmstone/0530-http-acl-compatibility-with-v4
...
0530 http acl compatibility with v4
2024-05-31 16:28:30 +02:00
Thales Macedo Garitezi