Commit Graph

10340 Commits

Author SHA1 Message Date
Serge Tupchii ea6f5764bb test: remove debug message 2023-12-12 12:05:46 +02:00
zhongwencool 1807df157c fix: license watermark percent support float 2023-12-12 17:27:31 +08:00
Stefan Strigler 71011ad084
Merge pull request #12142 from sstrigler/EMQX-11487-e-5-4-0-mongodb-test-connection-failed
mongodb test connection failed
2023-12-12 09:05:04 +01:00
zhongwencool e2e7f96515 fix: incorrect behavior in audit logs during stop and start of EMQX 2023-12-12 14:29:39 +08:00
zhongwencool 363055a32e fix: add descriptions fields to bridge_v1 2023-12-12 10:18:18 +08:00
zhongwencool cd1365d753 fix: don't require path in http action(fill default path) 2023-12-12 10:15:53 +08:00
Thales Macedo Garitezi b80c9b0863 fix(actions_schema): remove redundant `resource_opts` subfields for actions
Buffer workers don't use those fields.
2023-12-11 16:58:43 -03:00
Thales Macedo Garitezi 30719d286a fix(http_bridge_schema): use correct `resource_opts` subfields for connector and action 2023-12-11 16:58:43 -03:00
Thales Macedo Garitezi 593283df93 fix(redis_bridge_schema): use correct `resource_opts` subfields for connector and action 2023-12-11 16:58:43 -03:00
Thales Macedo Garitezi 4fc14fb4ba
Merge pull request #12143 from thalesmg/fix-reply-to-async-r54-20231211
fix(actions): increment rule statistics even if channel is not installed
2023-12-11 15:35:35 -03:00
Thales Macedo Garitezi f7adaa8911 fix(postgres_schema): fix usages of `resource_opts` in connector and action schemas
Fixes https://emqx.atlassian.net/browse/EMQX-11569
2023-12-11 15:03:20 -03:00
Thales Macedo Garitezi fbbb55633d fix(connector_schema): remove `query_mode` from `resource_opts`
The connector query mode is inferred during its creation, and later it must be overridden
by an action, anyway.
2023-12-11 14:13:40 -03:00
Thales Macedo Garitezi b4a5c141ad fix(actions): use action query mode instead of connector's query mode 2023-12-11 14:10:51 -03:00
Thales Macedo Garitezi 2495f59c91 fix(actions): increment rule statistics even if channel is not installed
Fixes new bug posted after https://emqx.atlassian.net/browse/EMQX-11494 was already fixed.

Also reduces the usage of error throwing for flow control a bit.
2023-12-11 14:01:30 -03:00
Stefan Strigler a7c344ba29 fix(emqx_bridge): remove unused 2023-12-11 14:58:09 +01:00
Stefan Strigler 8e9cb6a928 style(emqx_bridge_mysql): minor cleanup 2023-12-11 14:54:10 +01:00
Stefan Strigler 31a7301f6f refactor(emqx_bridge): be more generic in to_bridge_v1_config 2023-12-11 14:50:38 +01:00
JimMoen 6552e0523a
Merge pull request #12141 from JimMoen/fix-shared-sub-api-internal-error
Fix shared sub api internal error
2023-12-11 21:43:09 +08:00
Stefan Strigler be31486983 fix(emqx_utils): use deep_merge in unindent 2023-12-11 14:40:24 +01:00
Stefan Strigler 6143b3a2be fix(emqx_bridge_mongodb): use correct config when calling downgrade_type
Also limit result in mongodb ..._to_bridge_v1_config to not include more fields
than what's in the schema (most notably `description`).
2023-12-11 14:08:45 +01:00
Stefan Strigler 5d7ae4b980
Merge pull request #12132 from sstrigler/EMQX-11154-bridge-v-2-my-sql-support
feat(emqx_bridge_mysql): port to shared connectors
2023-12-11 13:56:34 +01:00
Zaiming (Stone) Shi b6c11be159
Merge pull request #12138 from zmstone/1208-fix-dialyzer
fix(dialyzer): batch 3
2023-12-11 12:52:56 +01:00
zhongwencool 6ce2d96ae3
Merge pull request #12139 from zhongwencool/prometheus-api-get
fix: only return recommend setting when get prometheus api
2023-12-11 17:29:36 +08:00
zhongwencool 64c843088a
Merge pull request #12140 from zhongwencool/redis-schema-example
chore: update redis bridge example
2023-12-11 17:29:10 +08:00
lafirest 11622031c5
Merge pull request #12133 from lafirest/fix/ldap
fix(authn_ldap): Improve the type inference of method union
2023-12-11 17:05:51 +08:00
JianBo He 5b70baf624
Merge pull request #12121 from zhongwencool/cluster-rpc
fix: occasionally return stale view when updating configurations
2023-12-11 16:59:17 +08:00
Zaiming (Stone) Shi d560366c14 test: fix some compile warnings 2023-12-11 09:43:13 +01:00
JimMoen 8c25371ff3
chore: bump gitignore for emqx standalone common test SUITES 2023-12-11 15:37:36 +08:00
JimMoen 7b60928288
fix(api): endpoint `/v5/topics` with invalid topic filter 2023-12-11 15:36:50 +08:00
zhongwencool 0595db3911 chore: update redis bridge example 2023-12-11 14:55:04 +08:00
zhongwencool 525a9e0adf fix: only return recommend setting when get prometheus api 2023-12-11 14:49:32 +08:00
JimMoen 47901c9fed
fix(exhook): `client.authorize` hook always uses real-topic
See: emqx_channel:do_check_sub_authzs/3, line: 1895
2023-12-11 14:11:18 +08:00
JimMoen cfe3b2dcee
feat(exhook): subopts in on_client_subscribe/on_client_unsubscribe 2023-12-11 14:11:18 +08:00
JimMoen 46201a8796
feat(exhook): provide the `peerport` field
- both in `ConnInfo` and `ClientInfo`
2023-12-11 14:11:18 +08:00
JimMoen 68da627b4d
feat(channel): add peerport field in ClientInfo 2023-12-11 14:11:18 +08:00
JianBo He b34e66bdf7
Merge pull request #12134 from lafirest/fix/gbt
fix(gbt): Add unimplemented command call
2023-12-11 13:46:07 +08:00
Zaiming (Stone) Shi 50f4aba5cd fix(dialyzer): batch 3 2023-12-09 15:50:09 +01:00
SergeTupchiy 107efda78f
Merge pull request #12136 from SergeTupchiy/upgrade-otel-1.4.7
Upgrade otel 1.4.7
2023-12-08 22:08:01 +02:00
Zaiming (Stone) Shi 5ad46ce077
Merge pull request #12129 from zmstone/1208-default-license-conn-25
refactor: update default enterprise license
2023-12-08 21:01:04 +01:00
Zaiming (Stone) Shi 1212ccc519
Merge pull request #12137 from zmstone/1207-fix-dialyzer
fix(dialyzer): batch 2
2023-12-08 20:31:17 +01:00
Thales Macedo Garitezi 66d043becd feat(ds): introduce `update_iterator` callback 2023-12-08 15:04:18 -03:00
Zaiming (Stone) Shi ddbb8560fa fix(dialyzer): batch 2 2023-12-08 17:59:55 +01:00
Serge Tupchii b5de58df2d chore: upgrade opentelemetry lib to 1.4.7-emqx 2023-12-08 18:44:19 +02:00
Stefan Strigler 71607aa2ad feat(emqx_bridge_mysql): port to shared connectors 2023-12-08 16:15:18 +01:00
Andrew Mayorov 76970c268e
fix(sessds): stop overwriting QoS0-only pubrange checkpoints
Through making `StreamRef` part of a record key. Otherwise with
QoS0-only streams the seqno doesn't change, thus ranges over
different streams will have the same `FirstSeqno`.
2023-12-08 14:33:47 +01:00
Andrew Mayorov 24710d0f56
fix(sessds): avoid accumulating QoS0-only ranges in memory 2023-12-08 14:31:18 +01:00
firest 1a8b59045b fix(gbt): Add unimplemented command call 2023-12-08 20:55:20 +08:00
firest f4f45cf634 fix(authn_ldap): Improve the type inference of method union 2023-12-08 19:26:29 +08:00
Kjell Winblad 0c80c74554 fix: remove unnecessary prepare_statement from pgsql action 2023-12-08 09:59:19 +01:00
Zaiming (Stone) Shi 41679808fe chore: add version number in license apply link 2023-12-08 08:35:58 +01:00
Zaiming (Stone) Shi e5cf9e40ac refactor: update default enterprise license 2023-12-08 08:25:20 +01:00
firest e57d796df5 fix(ldap): Adjust logger level for authentication failures 2023-12-08 12:47:32 +08:00
zhongwencool 45c4fb33a8 fix: occasionally return stale view when updating configurations on different nodes concurrently 2023-12-08 09:56:31 +08:00
Thales Macedo Garitezi f37c86afb9 Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231207 2023-12-07 13:30:04 -03:00
Thales Macedo Garitezi 2a6d72878f chore(ds): return DS message key along with batch 2023-12-07 11:36:08 -03:00
JimMoen 17e6703ba2
Merge pull request #12120 from qzhuyan/dev/william/bump-new-quicer
chore: bump emqtt 1.10.1 and quicer 0.0.307
2023-12-07 22:10:43 +08:00
William Yang 499f4d4bc5 chore(quicer): dialyzer 2023-12-07 13:52:48 +01:00
William Yang 583645d97b chore: bump emqtt 1.10.1 and quicer 0.0.308 2023-12-07 13:52:44 +01:00
Thales Macedo Garitezi c45ed7b888 fix(connectors_api): fill default raw config values before returning
Fixes https://emqx.atlassian.net/browse/EMQX-11562
2023-12-07 09:44:48 -03:00
Thales Macedo Garitezi c274450c5e
Merge pull request #12118 from thalesmg/fix-missing-type-name-api-actions-r54-20231206
fix(actions_api): add missing fields to API spec and returned data
2023-12-07 09:34:29 -03:00
Zaiming (Stone) Shi 7e3ed05f87
Merge pull request #12115 from zmstone/1206-fix-dialyzer
1206 fix dialyzer
2023-12-07 12:24:47 +01:00
JianBo He bd985e52d7
Merge pull request #12072 from killme2008/feature/greptimedb-async-write
feat: implements async query mode for GreptimeDB data bridge
2023-12-07 19:21:44 +08:00
ieQu1 9b612cb0e7
Merge pull request #12119 from ieQu1/EMQX-10342
Make stream renew timer configurable
2023-12-07 10:42:23 +01:00
zhongwencool 253ea6093a
Merge pull request #12111 from zhongwencool/fix-token-not-found
fix: use sync_transaction when update admin/token
2023-12-07 09:56:20 +08:00
zhongwencool 850c6027d4
Merge pull request #12113 from zhongwencool/log-fix
fix: audit log format
2023-12-07 09:56:01 +08:00
Zaiming (Stone) Shi 7560016f0e fix(hocon): upgrade hocon from 0.40.0 to 0.40.1
fixed type spec, exported some public types from emqx_schema module
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi 33a7282cdd fix(dialyzer): only include eunit when TEST is defined 2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi 9a6056ff2f refactor(emqx_machine): move system_monitor to included_applications
system_monitor has to be found in application (or included_applications)
otherwise dialyzer fails with Unknown function

Adding to 'applications' would cause a deadlock because one
cannot call application:ensure_all_started in the application/2
callback.
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi 6b553e37c0 chore: pin ekka 0.17.0 previously 0.15.16
change summary:
1. otp 26
2. type spec fixes
3. dropped UDP mcast clustering
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi 423b586c56 fix(dialyzer): fix some dialyzer issues found on otp 26 2023-12-06 20:32:49 +01:00
Zaiming (Stone) Shi 6affda8194 refactor(emqx_gateway_cm): no need to keep locker pid in state
For two reasons
1. Every other places computes the process register name from gwname
2. The ekka_locker api spec only accepts atom but not pid
2023-12-06 19:20:24 +01:00
Zaiming (Stone) Shi 71f9838fd7 chore: upgrade to emqtt 1.10.0 2023-12-06 19:18:15 +01:00
Zaiming (Stone) Shi 83206daa20 refactor(emqx_cm_locker): delete dead code 2023-12-06 19:18:15 +01:00
Zaiming (Stone) Shi de61d9d609 chore: upgrade to jiffy from 1.0.5 to 1.0.6
1.0.5 has an unexported type which causes dialyzer to fail on otp 26
2023-12-06 19:18:15 +01:00
William Yang 64401cb6cc
Merge pull request #12042 from qzhuyan/dev/william/rel54/esockd-robustnes
fix: bump to esockd 8.9.9  improve robustness
2023-12-06 17:39:38 +01:00
Andrew Mayorov 074dc8ce11
test(emqx): switch rest of test suite to use `emqx_cth_suite` 2023-12-06 16:47:04 +01:00
ieQu1 371ec34992 feat(sessds): Make stream renew interval configurable 2023-12-06 16:04:53 +01:00
ieQu1 c43b3eb535 fix(sessds): Add debug logs for the session garbage collection 2023-12-06 15:37:23 +01:00
Thales Macedo Garitezi fdeedc360e fix(actions_api): add missing fields to `POST` API schema 2023-12-06 11:13:09 -03:00
Thales Macedo Garitezi 2757b95850 fix(actions_api): fill raw config defaults before returning
Fixes https://emqx.atlassian.net/browse/EMQX-11541
2023-12-06 11:13:09 -03:00
Andrew Mayorov c2c9de69b4
test(emqx): switch `emqx_crl_cache_SUITE` to use `emqx_cth_suite` 2023-12-06 14:34:41 +01:00
Andrew Mayorov f06a1f10ef
test(emqx): switch most of testsuites to use `emqx_cth_suite`
For increased robustness and isolation.
2023-12-06 14:34:41 +01:00
Andrew Mayorov d643330cb9
test(ocsp): switch test suite to use `emqx_cth_suite`
And simplify it slightly in the process.
2023-12-06 14:34:40 +01:00
Andrew Mayorov 7d80511b3a
test(emqx): switch select test suites to use `emqx_cth_suite` 2023-12-06 14:34:40 +01:00
Thales Macedo Garitezi 8a5b0d252d
Merge pull request #12110 from thalesmg/test-plugin-flaky-r54-20231205
test(cth_peer): use an exclusive current dir for each peer
2023-12-06 10:16:13 -03:00
Thales Macedo Garitezi 17b6143c30
Merge pull request #12102 from thalesmg/nit-postgres-log-m-20231205
chore(postgres): prettify logged errors
2023-12-06 10:10:02 -03:00
William Yang c4956d99d1 fix: bump to esockd 8.9.9 improve listener robustness 2023-12-06 13:55:13 +01:00
Thales Macedo Garitezi b82189fb4d test(cth_peer): use an exclusive current dir for each peer 2023-12-06 09:35:13 -03:00
Andrew Mayorov 43813e29c2
chore(emqx): drop legacy session persistence impl leftovers 2023-12-06 10:53:57 +01:00
SergeTupchiy 28ff53e99c
Merge pull request #11984 from SergeTupchiy/EMQX-10535-openetelmetry-tracing-new
Integrate OpenTelmetry tracing
2023-12-06 11:45:41 +02:00
Zaiming (Stone) Shi d945a26c06
Merge pull request #12112 from zmstone/1206-refactor-delete-mcast-cluster-strategy
refactor: delete UDP multicast clustering strategy
2023-12-06 10:01:26 +01:00
zhongwencool 8965aa2a21 fix: audit log format 2023-12-06 16:58:04 +08:00
Andrew Mayorov 3798060543
Merge pull request #12109 from keynslug/test/emqx-cth-suite
test(emqx): switch select test suites to use `emqx_cth_suite`
2023-12-06 09:48:53 +01:00
Zaiming (Stone) Shi d0732aa2a6 refactor: delete UDP multicast clustering strategy 2023-12-06 08:39:08 +01:00
zhongwencool b015d08cba fix: use sync_transation when update admin/token 2023-12-06 09:04:27 +08:00
Thales Macedo Garitezi e03b8fd80e chore(postgres): prettify logged errors
Fixes https://emqx.atlassian.net/browse/EMQX-11490
2023-12-05 15:55:18 -03:00
Andrew Mayorov 83bea2254d
test(ocsp): switch test suite to use `emqx_cth_suite`
And simplify it slightly in the process.
2023-12-05 19:16:37 +01:00
Andrew Mayorov 6f5228e991
test(emqx): switch select test suites to use `emqx_cth_suite` 2023-12-05 19:16:37 +01:00
Thales Macedo Garitezi 99399c6e95
Merge pull request #12098 from thalesmg/revert-downgrade-bridge-id-m-20231205
chore: stop downgrading action id in rule API responses
2023-12-05 14:26:23 -03:00
Thales Macedo Garitezi f489de8860
Merge pull request #12100 from thalesmg/fix-password-action-probe-m-20231205
fix(bridges/actions api): correctly deobfuscate passwords when probing
2023-12-05 14:25:32 -03:00
Thales Macedo Garitezi 2579f8ea92
Merge pull request #12096 from thalesmg/fix-delete-deps-rules-m-20231204
fix(api): return list of dependent rule ids when trying to delete bridge/action
2023-12-05 14:24:50 -03:00
Serge Tupchii 938508b270 refactor(emqx_opentelemetry): pass only channel info to trace functions 2023-12-05 19:01:08 +02:00
Ivan Dyachkov f5e184ec24
Merge pull request #12101 from id/1205-scheduled-build-of-release54
ci: run scheduled packages build of release-54
2023-12-05 17:37:38 +01:00
Thales Macedo Garitezi f30c97f190 chore: bump app vsn 2023-12-05 13:26:02 -03:00
Thales Macedo Garitezi deb3fcd606 Merge remote-tracking branch 'origin/master' into sync-m-r54-20231205 2023-12-05 13:25:45 -03:00
Serge Tupchii 195a23ae27 chore(emqx_opentelemetry): bump opentelemetry to v1.4.6-emqx 2023-12-05 18:18:31 +02:00
Serge Tupchii 85441fda0f test(emqx_opentelemetry): add trace test suite 2023-12-05 18:18:31 +02:00
Serge Tupchii c8e69357cc feat(emqx_opentelemetry): use one global exporter config for all otel signals 2023-12-05 18:18:31 +02:00
Serge Tupchii 2a3f6b749c fix(emqx_opentelemetry): avoid using `application:ensure_all_started/1` for better deadlock safety 2023-12-05 18:18:31 +02:00
Serge Tupchii 8d3f98eff2 fix(emqx_opentelemetry): fix dialyzer warnings 2023-12-05 18:18:31 +02:00
Serge Tupchii 60da6427ca fix: move openetelmetry deps to emqx_openetelmetry app
This is to fix elixir build, as mix can't find included opentelemetry hrl files.
2023-12-05 18:18:31 +02:00
ieQu1 2a50ec3f7d
Merge pull request #12104 from ieQu1/dev/gcp-logs
fix(bridge_gcp): Don't print config to the logs
2023-12-05 17:15:11 +01:00
Serge Tupchii 7fdc650448 feat: integrate OpenTelemetry traces 2023-12-05 18:11:00 +02:00
Thales Macedo Garitezi e42f4155d3 fix(bridges/actions api): correctly deobfuscate passwords when probing
Fixes https://emqx.atlassian.net/browse/EMQX-11533
2023-12-05 11:49:50 -03:00
ieQu1 b2ca59dfa7 fix(bridge_gcp): Don't print config to the logs 2023-12-05 15:48:53 +01:00
Thales Macedo Garitezi e5a4be11f5 chore: stop downgrading action id in rule API responses
Fixes https://emqx.atlassian.net/browse/EMQX-11405
2023-12-05 11:48:46 -03:00
Thales Macedo Garitezi 1b68fbff05 fix(api): return list of dependent rule ids when trying to delete bridge/action
Fixes https://emqx.atlassian.net/browse/EMQX-11523
2023-12-05 11:47:26 -03:00
Andrew Mayorov 286d483a3a
Merge pull request #12055 from keynslug/ft/EMQX-11474/subopts
fix(sessds): respect subscription options when publishing
2023-12-05 15:45:25 +01:00
Ivan Dyachkov e995b3948e chore: bump emqx_gateway_coap version 2023-12-05 15:17:53 +01:00
Thales Macedo Garitezi b6e8f5a37c
Merge pull request #12095 from thalesmg/fix-rule-test-api-m-20231204
fix(rule_api): fix sql test when testing against events
2023-12-05 08:57:54 -03:00
SergeTupchiy 7b9d12f20e
Merge pull request #12048 from SergeTupchiy/fix-coap_sub-opts
Fix COAP sub opts
2023-12-05 10:27:04 +02:00
Thales Macedo Garitezi c1c7d1246f
Merge pull request #12084 from thalesmg/refactor-resource-status-state-m-20231201
refactor(resource_manager): use macros and better differentiate status from state
2023-12-04 17:55:27 -03:00
Thales Macedo Garitezi d594b38ceb
Merge pull request #12083 from thalesmg/fix-connector-bws-m-20231201
fix(connector): don't start buffer workers for the connector itself
2023-12-04 16:14:15 -03:00
Thales Macedo Garitezi 984e2ccc74
Merge pull request #12082 from thalesmg/test-ensure-res-opts-conn-m-20231201
add missing `resource_opts` fields to connectors schemas
2023-12-04 16:13:32 -03:00
Thales Macedo Garitezi cea857cb8e fix(rule_api): fix sql test when testing against events
Fixes https://emqx.atlassian.net/browse/EMQX-11449
2023-12-04 15:24:09 -03:00
Thales Macedo Garitezi b2cd8aaab7 refactor: use single clause
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi ac2c99f996 test(connector): add test to ensure connector schema has `resource_opts`
https://emqx.atlassian.net/browse/EMQX-11509
2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi bb04a2d89e fix(bridge_redis): add `resource_opts` to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi 87a13aa316 fix(bridge_mongo): add `resource_opts` to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi 7a1288e802 fix(bridge_postgres): add `resource_opts` to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi 3078de6e97 fix(bridge_syskeeper): add `resource_opts` to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi 5bdfac52b8 fix(bridge_kafka_producer): add missing `resource_opts` sub-fields to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi 626bcc1c62 fix(bridge_http): add `resource_opts` to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi 10b1a8bbda feat(connector): add some `resource_opts` schema facilities 2023-12-04 10:02:05 -03:00
Dennis Zhuang 9920841deb chore: upgrade greptimedb client to v0.1.6 to fix the failed test 2023-12-04 20:30:33 +08:00
Dennis Zhuang 1d6ea685ef fix: another try to fix t_write_failure 2023-12-04 20:30:33 +08:00
Dennis Zhuang 97592e2e40 fix: test case t_write_failure 2023-12-04 20:30:33 +08:00
Dennis Zhuang fbeb56e475 fix: test case t_write_failure 2023-12-04 20:30:33 +08:00
Dennis Zhuang 547acef01d feat: upgrade greptimedb client to v0.1.5 2023-12-04 20:30:33 +08:00
Dennis Zhuang a694f3c92d fix: tests 2023-12-04 20:30:33 +08:00
Dennis Zhuang 8bc874c261 feat: supports async mode for greptimedb data bridge 2023-12-04 20:30:33 +08:00
Dennis Zhuang 90d0081df4 feat: upgrade greptimedb and client 2023-12-04 20:30:33 +08:00
Andrew Mayorov 130a5a5442
fix(ds): pass topics to `emqx_topic:words/1` before feeding LTS tree
So that empty levels in topics will be properly mapped into `''` atoms.
2023-12-04 13:39:01 +03:00
Andrew Mayorov 92c4b29a4c
feat(topic): match empty topic levels more loosely
So that the result of `emqx_topic:tokens/1` would be perfectly
matchable with the result of `emqx_topic:words/1` of a topic filter
with empty levels.
2023-12-04 13:38:55 +03:00
Andrew Mayorov b5c485cd55
test(sessds): add empty level topic subscription testcase 2023-12-04 13:38:42 +03:00
Serge Tupchii e455433694 test(emqx_gateway): fix test failures caused by grpc-erl upgrade 2023-12-04 12:17:58 +02:00
JimMoen cb992fcf55
Merge pull request #12059 from JimMoen/1130-fix-vm-time-warp
fix: vm.args use `multi_time_warp`
2023-12-04 11:56:08 +08:00
Andrew Mayorov 627c58b07d
fix(sessds): handle `expire_awaiting_rel` common timer with no-op
Otherwise, the channel crashes when the timer is triggered.
2023-12-02 15:00:30 +03:00
Andrew Mayorov 47bc747323
fix(sessds): drop everything related to session on CleanStart 2023-12-02 15:00:30 +03:00
Andrew Mayorov ee9a98f0a4
chore: bump emqx_gateway_mqttsn to 0.1.7 2023-12-02 15:00:29 +03:00
Andrew Mayorov fd26e690b8
refactor(sessds): move parts of message processing to replayer
To simplify the processing flow, reducing the number of back-and-forth
between the session and the replayer.
2023-12-02 15:00:29 +03:00
Andrew Mayorov 29ec73847a
fix(utils): make `flattermap/2` results less variative 2023-12-02 14:38:04 +03:00
Andrew Mayorov 6255ee0833
fix(sessmem): avoid defining unsafe defaults in `#session` 2023-12-02 14:38:04 +03:00
Andrew Mayorov b5ad0f9815
fix(test): subscribe with QoS 2 to receive QoS 2 message 2023-12-02 14:38:04 +03:00
Andrew Mayorov 1948719eec
test(sessds): unskip testcase related to QoS2 handling 2023-12-02 14:38:04 +03:00
Andrew Mayorov 3265d2f2aa
fix(sessds): respect subscription options when publishing 2023-12-02 14:38:03 +03:00
Andrew Mayorov b5f39f89e3
feat(utils): add `flattermap/2` as slightly more generic `flatmap/2` 2023-12-02 14:36:21 +03:00
Andrew Mayorov 508346f095
refactor(topicidx): split persistent term stuff off gbt-based index 2023-12-02 14:36:21 +03:00
Andrew Mayorov 88103c5f0e
refactor(session): pass config to `SessionImpl:open/3` as well
* Anticipate that connection info may change during session takeover.
* Avoid persisting session conf as part of persistent session state.
2023-12-02 14:36:21 +03:00
Andrew Mayorov 6eba082992
Merge pull request #12075 from keynslug/fix/EMQX-11474/function-clause
fix(channel): expect there's no session yet on disconnect
2023-12-02 14:32:50 +03:00
Thales Macedo Garitezi dc5e3b939c refactor(resource_manager): use macros and better differentiate status from state
Internally in `emqx_resource_manager`, there seems to be many points where the
`gen_statem` states are conflated with resource status, since their names coincide.  While
that works for now, introducing a new `gen_statem` state, an internal state, shouldn't
necessarily imply a new, externally facing resource status.

Here we also introduce the usage of some macros to avoid the pitfalls of making a typo in
a state/status name.
2023-12-01 18:23:05 -03:00
Zaiming (Stone) Shi 55621fbb86
Merge pull request #12074 from zmstone/1201-user-peer-module-for-all
test: replace 'slave' and 'ct_slave' with 'peer'
2023-12-01 21:35:40 +01:00
Thales Macedo Garitezi 29ae45c39d fix(connector): don't start buffer workers for the connector itself
Fixes https://emqx.atlassian.net/browse/EMQX-11448
2023-12-01 17:20:38 -03:00
Zaiming (Stone) Shi 6e30718818 test: allow join cluster later 2023-12-01 20:33:39 +01:00
ieQu1 3d2e95fe3e
Merge pull request #12081 from ieQu1/dev/gen-rpc-3.3.0
chore(gen_rpc): Bump version to 3.3.0
2023-12-01 17:57:33 +01:00
ieQu1 476c300ecf
Merge pull request #12070 from ieQu1/dev/ds-min-max-batch-size
Various minor fixes
2023-12-01 17:57:15 +01:00
ieQu1 d3b7478d70 chore(gen_rpc): Bump version to 3.3.0 2023-12-01 17:10:12 +01:00
Zaiming (Stone) Shi ab7d57e1c5 test: delete debug level logging 2023-12-01 17:07:14 +01:00
Ivan Dyachkov 0e27d031a8 chore: 5.4.0-alpha.1 2023-12-01 16:54:03 +01:00
Zaiming (Stone) Shi 6a9cc20d4d test: wait for all nodes clustered 2023-12-01 16:37:39 +01:00
Zaiming (Stone) Shi 956b25cc4c
Merge pull request #12071 from zmstone/1201-sync-release-54
1201 sync release 54
2023-12-01 14:38:47 +01:00
Thales Macedo Garitezi cb60880bab
Merge pull request #12067 from thalesmg/fix-kafka-check-client-m-20231130
fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
2023-12-01 09:55:12 -03:00
Andrew Mayorov 6b17920fea
fix(channel): expect there's no session yet on disconnect 2023-12-01 15:29:05 +03:00
Andrew Mayorov cce474ce98
fix(session): include session term in stacktrace 2023-12-01 15:29:05 +03:00
Zaiming (Stone) Shi 1a78e7ae79 fix(connector): ensure webhook bridge convert to http connector 2023-12-01 11:56:37 +01:00
Zaiming (Stone) Shi 90b7486624 fix(redis): ensure schema namespace 2023-12-01 11:37:06 +01:00
ieQu1 e1ec560639 test(emqx): Fix flaky emqx_takeover_SUITE:t_takeover testcase
This testcase uses QoS1, so it must account for possible duplication
of the messages.
2023-12-01 10:06:18 +01:00
Zaiming (Stone) Shi 60d70b22dc chore: bump app vsn for emqx_auth_redis 2023-12-01 08:37:37 +01:00
Zaiming (Stone) Shi 7f4d91d490 Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54 2023-12-01 08:32:22 +01:00
ieQu1 e238602533 fix(ds): Update README 2023-12-01 08:27:05 +01:00
ieQu1 0ae618d010 fix(ds): Use emqx_rpc for calls that work with large binaries 2023-12-01 08:27:05 +01:00
ieQu1 0e625d814a feat(sessds): Make batch size configurable
This change affects flow control. It allows to configure maximum size
of a batch, as well as fetch threshold.
2023-12-01 08:27:05 +01:00
ieQu1 4717e56fb6 fix(sessds): Schedule poll immediately upon receiving an ack
This commit affects the flow control, and improves the throughput by
removing a delay between freeing up space in the in-flight window and
polling new messages.
2023-12-01 08:27:05 +01:00
ieQu1 38800c0260 refactor(sessds): Store timers in the session 2023-12-01 08:27:05 +01:00
Zaiming (Stone) Shi 22f7cc1622 test: replace 'slave' and 'ct_slave' with 'peer' 2023-12-01 08:07:09 +01:00
JimMoen ee25d9bd9e
fix: vm.args use `multi_time_warp`
See also:
https://www.erlang.org/doc/apps/erts/time_correction#multi-time-warp-mode
2023-12-01 14:52:31 +08:00
ieQu1 69f1ca43c3 fix(sessds): Create a timer even if it's present in the map 2023-12-01 06:11:13 +01:00
ieQu1 1897e5c31b refactor(session): Use common naming conventions 2023-12-01 06:11:08 +01:00
ieQu1 d9e7544070 refactor(sessds): Introduce macros for the timers 2023-12-01 06:11:08 +01:00
ieQu1 0d245acdc1
Merge pull request #12066 from keynslug/fix/sessds-tc-flaky
test(sessds): wait client disconnect propagates to broker
2023-12-01 00:13:38 +01:00
Thales Macedo Garitezi 113a4ad4b1
Merge pull request #12057 from thalesmg/ds-session-gc-m-20231128
feat(ds): add session gc process
2023-11-30 17:51:52 -03:00
Thales Macedo Garitezi 0388e1c1c4 fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
Fixes https://emqx.atlassian.net/browse/EMQX-11494
2023-11-30 17:51:32 -03:00
Ivan Dyachkov 9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Thales Macedo Garitezi 684d637fed test(bridge_api): workaround strange config syncing problem
For some unknown reason, this test has difficulties in syncing the config correctly
between the nodes, while the equivalent in bridge_v2_api_SUITE doesn't.
2023-11-30 19:51:35 +01:00
Ivan Dyachkov ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
Zaiming (Stone) Shi dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26
1129 prepare for OTP 26
2023-11-30 19:38:40 +01:00
Andrew Mayorov ec471c0557
test(sessds): wait client disconnect propagates to broker 2023-11-30 20:14:57 +03:00
Serge Tupchii f54bda7063 fix(emqx_mgmt_api_data_backup): add count and hasnext meta paging params 2023-11-30 19:10:12 +02:00
Zaiming (Stone) Shi 81e75cf068 test: fix bulk-kick test case flakyness 2023-11-30 17:30:25 +01:00
Zaiming (Stone) Shi 5427ebc5f1 fix: log error when failed to install plugin
the error return is a map with details.
prior to this change only the reason is returned in the api response,
now the error map is logged at error level to help troubleshooting
2023-11-30 16:49:30 +01:00
Ivan Dyachkov 6dd92c382f chore: 5.3.2 release 2023-11-30 16:18:44 +01:00
Thales Macedo Garitezi cf6cb3e4ad chore: set low importance to config 2023-11-30 11:55:57 -03:00
Thales Macedo Garitezi 1ab009f081 refactor: rename supervisor 2023-11-30 11:55:48 -03:00
Zaiming (Stone) Shi eed253af82 test: implement a new node restart helper func 2023-11-30 15:11:38 +01:00
Thales Macedo Garitezi 880f5e8f89 feat(ds): add session gc process
Fixes https://emqx.atlassian.net/browse/EMQX-9744
2023-11-30 09:38:59 -03:00
Zaiming (Stone) Shi cf72c04fdd test: fix peer node stop and plugin SUITE typo 2023-11-30 09:53:36 +01:00
Zaiming (Stone) Shi f2db4cc7fc chore: upgrade to gen_rpc 3.2.2 2023-11-30 09:42:44 +01:00
Zaiming (Stone) Shi 640b0df319 test: do not add -master erl flag for peer nodes 2023-11-30 08:49:57 +01:00
zhongwencool 773bd8ecb2
Merge pull request #12047 from thalesmg/test-more-flaky-tests-r53-20231128
test(flaky): more adjustments
2023-11-30 09:33:55 +08:00
Zaiming (Stone) Shi 4ecfe2be30 test: use peer module for slave and ct_slave 2023-11-29 21:45:53 +01:00
Thales Macedo Garitezi 62b763a8f8 test(gcp_pubsub_consumer): even more adjustments 2023-11-29 15:12:47 -03:00
Ivan Dyachkov 7f5433f6dd chore: 5.3.2-rc.1 2023-11-29 18:20:15 +01:00
Thales Macedo Garitezi fdafe2493e
Merge pull request #12041 from thalesmg/ds-estimate-last-alive-m-20231128
fix(ds_session): take conservative estimate of `last_alive_at` when bumping
2023-11-29 13:43:59 -03:00
Zaiming (Stone) Shi 6f35f25163 chore: upgrade esockd to 5.9.8 for OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi 1a563b4f65 chore: fix 0.0 match for OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi 14644988e0 chore: change triple-quotes to single-quotes 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi e6eb97e104 chore: upgrade dependency brod_gssapi to work with OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi 41194cacc8
Merge pull request #12030 from zhongwencool/redis-connector-v2-540
Redis connector v2 540
2023-11-29 16:13:09 +01:00
Kjell Winblad a750fbbc52
Merge pull request #12051 from HJianBo/fix-http-bridge-bugs
Fix http bridge bugs
2023-11-29 15:48:51 +01:00
Zaiming (Stone) Shi 89cdfbca63 fix(emqx_vm): trim new-line in otp version string 2023-11-29 15:32:25 +01:00
Andrew Mayorov ccef91437d
Merge pull request #12029 from keynslug/ft/EMQX-11049/qos2
feat(sessds): provide QoS2 message replay support
2023-11-29 14:10:16 +03:00
Serge Tupchii 009a15c7d0 fix(emqx_mgmt_api_data_backup): validate empty file in upload API
fixes EMQX-11488
2023-11-29 12:12:42 +02:00
Andrew Mayorov d8691f1d64
refactor(sessds): rename marker → committed offset
For better clarity.
2023-11-29 13:04:42 +03:00
zhongwencool 0aec2f7605 feat: redis bridge v2 2023-11-29 16:06:46 +08:00
zhongwencool 8ec3857f4d
Merge pull request #12046 from savonarola/1128-fix-redis-54
[5.4] Fix Redis connector
2023-11-29 16:06:23 +08:00
William Yang e90e843ff3
Merge pull request #12040 from qzhuyan/dev/william/quic-0.0.300
feat(quic): bump to quicer-0.0.303
2023-11-29 08:48:47 +01:00
JianBo He d2e5f302a8 chore(http): add description for bridges v1 2023-11-29 13:47:59 +08:00
JianBo He 72bc046063 chore: avoid unnecessary default values being filled when querying via the v1 api.
see: https://emqx.atlassian.net/browse/EMQX-11482
2023-11-29 12:01:59 +08:00
JianBo He d7bf8e97d2 chore: add tests case 2023-11-29 11:18:51 +08:00
Serge Tupchii b0d4ba183e test: use `emqx_cth_suite` in `emqx_coap_api_SUITE`
`emqx_coap_api_SUITE` was failing with `emqx_authn_schema_not_injected`
(missing authn schema mods in persistent_term) if it was run standalone.
2023-11-28 20:02:21 +02:00
Serge Tupchii 3530ed66d5 fix(emqx_gateway_coap): take subscribe options from uri_query
Closes #12031
2023-11-28 20:02:21 +02:00
Thales Macedo Garitezi 095e7c4ecb test(flaky): more adjustments 2023-11-28 13:41:31 -03:00
Thales Macedo Garitezi f6f61f4353
Merge pull request #12036 from thalesmg/test-more-flaky-tests-r54-20231127
test: attempting to stabilize more flaky tests
2023-11-28 13:40:49 -03:00
Ilya Averyanov e5508105c7 fix(redis): start and load eredis app 2023-11-28 18:51:53 +03:00
Ilya Averyanov 55218e2df2 fix(redis): start and load eredis app 2023-11-28 18:51:19 +03:00
Zaiming (Stone) Shi d93fdd8e9b
Merge pull request #12039 from zmstone/1128-sync-release-54
Sync release-54 to master
2023-11-28 15:52:49 +01:00
William Yang 64f6220cf5 feat(quic): bump to quicer-0.0.303 and emqtt 1.9.7 2023-11-28 14:43:47 +01:00
ieQu1 8a065ba6d0
Merge pull request #12027 from ieQu1/dev/ds-meta-persist-shard
fix(ds): Persist shard table
2023-11-28 14:33:57 +01:00
Thales Macedo Garitezi f5c4fb5860 fix(ds_session): take conservative estimate of `last_alive_at` when bumping
Addresses https://github.com/emqx/emqx/pull/12024#discussion_r1407432154
2023-11-28 09:58:03 -03:00
Thales Macedo Garitezi bd40b5c553 test(flaky): more adjustments 2023-11-28 09:48:40 -03:00
Ivan Dyachkov 2e11ab6a16 chore: 5.3.2-alpha.2 2023-11-28 13:46:28 +01:00
ieQu1 05e47254e2 fix(ds): Fixes related to the shards table 2023-11-28 12:59:09 +01:00
Zaiming (Stone) Shi 6c85e62d26 fix(schema): add namespaces 2023-11-28 12:07:42 +01:00
Zaiming (Stone) Shi 1b1cea24f3 Merge remote-tracking branch 'origin/release-54' into 1128-sync-release-54 2023-11-28 11:43:30 +01:00
Andrew Mayorov 34c9c022d0
chore(replayer): add comment describing what "until" means 2023-11-28 11:47:54 +03:00
JianBo He bd6e9503e6 fix(http): compose the url and path in correctly format 2023-11-28 16:15:46 +08:00
zhongwencool 6e6e68c902
Merge pull request #12037 from zhongwencool/license-watermark-check
fix: validate error when set license's watermark to 100%
2023-11-28 16:04:39 +08:00
zhongwencool 29bcdb9a4a fix: validate error when set license's watermark to 100% 2023-11-28 13:16:16 +08:00
JianBo He 891ecc179d chore: fix flaky tests 2023-11-28 10:34:44 +08:00
JianBo He c8b5c51bbc chore: fix failed test cases 2023-11-28 09:53:46 +08:00