yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,094 Commits 393 Branches 684 Tags 74 MiB
6e6b4dd44a
Commit Graph

10147 Commits

Author SHA1 Message Date
Andrew Mayorov
92c4b29a4c
feat(topic): match empty topic levels more loosely 
So that the result of `emqx_topic:tokens/1` would be perfectly
matchable with the result of `emqx_topic:words/1` of a topic filter
with empty levels.
 2023-12-04 13:38:55 +03:00
Andrew Mayorov
b5c485cd55
test(sessds): add empty level topic subscription testcase 2023-12-04 13:38:42 +03:00
Serge Tupchii
e455433694 test(emqx_gateway): fix test failures caused by grpc-erl upgrade 2023-12-04 12:17:58 +02:00
JimMoen
cb992fcf55
Merge pull request #12059 from JimMoen/1130-fix-vm-time-warp 
fix: vm.args use `multi_time_warp`
 2023-12-04 11:56:08 +08:00
Andrew Mayorov
627c58b07d
fix(sessds): handle expire_awaiting_rel common timer with no-op 
Otherwise, the channel crashes when the timer is triggered.
 2023-12-02 15:00:30 +03:00
Andrew Mayorov
47bc747323
fix(sessds): drop everything related to session on CleanStart 2023-12-02 15:00:30 +03:00
Andrew Mayorov
ee9a98f0a4
chore: bump emqx_gateway_mqttsn to 0.1.7 2023-12-02 15:00:29 +03:00
Andrew Mayorov
fd26e690b8
refactor(sessds): move parts of message processing to replayer 
To simplify the processing flow, reducing the number of back-and-forth
between the session and the replayer.
 2023-12-02 15:00:29 +03:00
Andrew Mayorov
29ec73847a
fix(utils): make flattermap/2 results less variative 2023-12-02 14:38:04 +03:00
Andrew Mayorov
6255ee0833
fix(sessmem): avoid defining unsafe defaults in #session 2023-12-02 14:38:04 +03:00
Andrew Mayorov
b5ad0f9815
fix(test): subscribe with QoS 2 to receive QoS 2 message 2023-12-02 14:38:04 +03:00
Andrew Mayorov
1948719eec
test(sessds): unskip testcase related to QoS2 handling 2023-12-02 14:38:04 +03:00
Andrew Mayorov
3265d2f2aa
fix(sessds): respect subscription options when publishing 2023-12-02 14:38:03 +03:00
Andrew Mayorov
b5f39f89e3
feat(utils): add flattermap/2 as slightly more generic flatmap/2 2023-12-02 14:36:21 +03:00
Andrew Mayorov
508346f095
refactor(topicidx): split persistent term stuff off gbt-based index 2023-12-02 14:36:21 +03:00
Andrew Mayorov
88103c5f0e
refactor(session): pass config to SessionImpl:open/3 as well 
* Anticipate that connection info may change during session takeover.
* Avoid persisting session conf as part of persistent session state.
 2023-12-02 14:36:21 +03:00
Andrew Mayorov
6eba082992
Merge pull request #12075 from keynslug/fix/EMQX-11474/function-clause 
fix(channel): expect there's no session yet on disconnect
 2023-12-02 14:32:50 +03:00
Thales Macedo Garitezi
dc5e3b939c refactor(resource_manager): use macros and better differentiate status from state 
Internally in `emqx_resource_manager`, there seems to be many points where the
`gen_statem` states are conflated with resource status, since their names coincide.  While
that works for now, introducing a new `gen_statem` state, an internal state, shouldn't
necessarily imply a new, externally facing resource status.

Here we also introduce the usage of some macros to avoid the pitfalls of making a typo in
a state/status name.
 2023-12-01 18:23:05 -03:00
Zaiming (Stone) Shi
55621fbb86
Merge pull request #12074 from zmstone/1201-user-peer-module-for-all 
test: replace 'slave' and 'ct_slave' with 'peer'
 2023-12-01 21:35:40 +01:00
Thales Macedo Garitezi
29ae45c39d fix(connector): don't start buffer workers for the connector itself 
Fixes https://emqx.atlassian.net/browse/EMQX-11448
 2023-12-01 17:20:38 -03:00
Zaiming (Stone) Shi
6e30718818 test: allow join cluster later 2023-12-01 20:33:39 +01:00
ieQu1
3d2e95fe3e
Merge pull request #12081 from ieQu1/dev/gen-rpc-3.3.0 
chore(gen_rpc): Bump version to 3.3.0
 2023-12-01 17:57:33 +01:00
ieQu1
476c300ecf
Merge pull request #12070 from ieQu1/dev/ds-min-max-batch-size 
Various minor fixes
 2023-12-01 17:57:15 +01:00
ieQu1
d3b7478d70 chore(gen_rpc): Bump version to 3.3.0 2023-12-01 17:10:12 +01:00
Zaiming (Stone) Shi
ab7d57e1c5 test: delete debug level logging 2023-12-01 17:07:14 +01:00
Ivan Dyachkov
0e27d031a8 chore: 5.4.0-alpha.1 2023-12-01 16:54:03 +01:00
Zaiming (Stone) Shi
6a9cc20d4d test: wait for all nodes clustered 2023-12-01 16:37:39 +01:00
Zaiming (Stone) Shi
956b25cc4c
Merge pull request #12071 from zmstone/1201-sync-release-54 
1201 sync release 54
 2023-12-01 14:38:47 +01:00
Thales Macedo Garitezi
cb60880bab
Merge pull request #12067 from thalesmg/fix-kafka-check-client-m-20231130 
fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
 2023-12-01 09:55:12 -03:00
Andrew Mayorov
6b17920fea
fix(channel): expect there's no session yet on disconnect 2023-12-01 15:29:05 +03:00
Andrew Mayorov
cce474ce98
fix(session): include session term in stacktrace 2023-12-01 15:29:05 +03:00
Zaiming (Stone) Shi
1a78e7ae79 fix(connector): ensure webhook bridge convert to http connector 2023-12-01 11:56:37 +01:00
Zaiming (Stone) Shi
90b7486624 fix(redis): ensure schema namespace 2023-12-01 11:37:06 +01:00
ieQu1
e1ec560639 test(emqx): Fix flaky emqx_takeover_SUITE:t_takeover testcase 
This testcase uses QoS1, so it must account for possible duplication
of the messages.
 2023-12-01 10:06:18 +01:00
Zaiming (Stone) Shi
60d70b22dc chore: bump app vsn for emqx_auth_redis 2023-12-01 08:37:37 +01:00
Zaiming (Stone) Shi
7f4d91d490 Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54 2023-12-01 08:32:22 +01:00
ieQu1
e238602533 fix(ds): Update README 2023-12-01 08:27:05 +01:00
ieQu1
0ae618d010 fix(ds): Use emqx_rpc for calls that work with large binaries 2023-12-01 08:27:05 +01:00
ieQu1
0e625d814a feat(sessds): Make batch size configurable 
This change affects flow control. It allows to configure maximum size
of a batch, as well as fetch threshold.
 2023-12-01 08:27:05 +01:00
ieQu1
4717e56fb6 fix(sessds): Schedule poll immediately upon receiving an ack 
This commit affects the flow control, and improves the throughput by
removing a delay between freeing up space in the in-flight window and
polling new messages.
 2023-12-01 08:27:05 +01:00
ieQu1
38800c0260 refactor(sessds): Store timers in the session 2023-12-01 08:27:05 +01:00
Zaiming (Stone) Shi
22f7cc1622 test: replace 'slave' and 'ct_slave' with 'peer' 2023-12-01 08:07:09 +01:00
JimMoen
ee25d9bd9e
fix: vm.args use multi_time_warp 
See also:
https://www.erlang.org/doc/apps/erts/time_correction#multi-time-warp-mode
 2023-12-01 14:52:31 +08:00
ieQu1
69f1ca43c3 fix(sessds): Create a timer even if it's present in the map 2023-12-01 06:11:13 +01:00
ieQu1
1897e5c31b refactor(session): Use common naming conventions 2023-12-01 06:11:08 +01:00
ieQu1
d9e7544070 refactor(sessds): Introduce macros for the timers 2023-12-01 06:11:08 +01:00
ieQu1
0d245acdc1
Merge pull request #12066 from keynslug/fix/sessds-tc-flaky 
test(sessds): wait client disconnect propagates to broker
 2023-12-01 00:13:38 +01:00
Thales Macedo Garitezi
113a4ad4b1
Merge pull request #12057 from thalesmg/ds-session-gc-m-20231128 
feat(ds): add session gc process
 2023-11-30 17:51:52 -03:00
Thales Macedo Garitezi
0388e1c1c4 fix(kafka_producer): add resource_opts to connector schema, and check for client connectivity 
Fixes https://emqx.atlassian.net/browse/EMQX-11494
 2023-11-30 17:51:32 -03:00
Ivan Dyachkov
9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Thales Macedo Garitezi
684d637fed test(bridge_api): workaround strange config syncing problem 
For some unknown reason, this test has difficulties in syncing the config correctly
between the nodes, while the equivalent in bridge_v2_api_SUITE doesn't.
 2023-11-30 19:51:35 +01:00
Ivan Dyachkov
ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
Zaiming (Stone) Shi
dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26 
1129 prepare for OTP 26
 2023-11-30 19:38:40 +01:00
Andrew Mayorov
ec471c0557
test(sessds): wait client disconnect propagates to broker 2023-11-30 20:14:57 +03:00
Serge Tupchii
f54bda7063 fix(emqx_mgmt_api_data_backup): add count and hasnext meta paging params 2023-11-30 19:10:12 +02:00
Zaiming (Stone) Shi
81e75cf068 test: fix bulk-kick test case flakyness 2023-11-30 17:30:25 +01:00
Zaiming (Stone) Shi
5427ebc5f1 fix: log error when failed to install plugin 
the error return is a map with details.
prior to this change only the reason is returned in the api response,
now the error map is logged at error level to help troubleshooting
 2023-11-30 16:49:30 +01:00
Ivan Dyachkov
6dd92c382f chore: 5.3.2 release 2023-11-30 16:18:44 +01:00
Thales Macedo Garitezi
cf6cb3e4ad chore: set low importance to config 2023-11-30 11:55:57 -03:00
Thales Macedo Garitezi
1ab009f081 refactor: rename supervisor 2023-11-30 11:55:48 -03:00
Zaiming (Stone) Shi
eed253af82 test: implement a new node restart helper func 2023-11-30 15:11:38 +01:00
Thales Macedo Garitezi
880f5e8f89 feat(ds): add session gc process 
Fixes https://emqx.atlassian.net/browse/EMQX-9744
 2023-11-30 09:38:59 -03:00
Zaiming (Stone) Shi
cf72c04fdd test: fix peer node stop and plugin SUITE typo 2023-11-30 09:53:36 +01:00
Zaiming (Stone) Shi
f2db4cc7fc chore: upgrade to gen_rpc 3.2.2 2023-11-30 09:42:44 +01:00
Zaiming (Stone) Shi
640b0df319 test: do not add -master erl flag for peer nodes 2023-11-30 08:49:57 +01:00
zhongwencool
773bd8ecb2
Merge pull request #12047 from thalesmg/test-more-flaky-tests-r53-20231128 
test(flaky): more adjustments
 2023-11-30 09:33:55 +08:00
Zaiming (Stone) Shi
4ecfe2be30 test: use peer module for slave and ct_slave 2023-11-29 21:45:53 +01:00
Thales Macedo Garitezi
62b763a8f8 test(gcp_pubsub_consumer): even more adjustments 2023-11-29 15:12:47 -03:00
Ivan Dyachkov
7f5433f6dd chore: 5.3.2-rc.1 2023-11-29 18:20:15 +01:00
Thales Macedo Garitezi
fdafe2493e
Merge pull request #12041 from thalesmg/ds-estimate-last-alive-m-20231128 
fix(ds_session): take conservative estimate of `last_alive_at` when bumping
 2023-11-29 13:43:59 -03:00
Zaiming (Stone) Shi
6f35f25163 chore: upgrade esockd to 5.9.8 for OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
1a563b4f65 chore: fix 0.0 match for OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
14644988e0 chore: change triple-quotes to single-quotes 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
e6eb97e104 chore: upgrade dependency brod_gssapi to work with OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
41194cacc8
Merge pull request #12030 from zhongwencool/redis-connector-v2-540 
Redis connector v2 540
 2023-11-29 16:13:09 +01:00
Kjell Winblad
a750fbbc52
Merge pull request #12051 from HJianBo/fix-http-bridge-bugs 
Fix http bridge bugs
 2023-11-29 15:48:51 +01:00
Zaiming (Stone) Shi
89cdfbca63 fix(emqx_vm): trim new-line in otp version string 2023-11-29 15:32:25 +01:00
Andrew Mayorov
ccef91437d
Merge pull request #12029 from keynslug/ft/EMQX-11049/qos2 
feat(sessds): provide QoS2 message replay support
 2023-11-29 14:10:16 +03:00
Serge Tupchii
009a15c7d0 fix(emqx_mgmt_api_data_backup): validate empty file in upload API 
fixes EMQX-11488
 2023-11-29 12:12:42 +02:00
Andrew Mayorov
d8691f1d64
refactor(sessds): rename marker → committed offset 
For better clarity.
 2023-11-29 13:04:42 +03:00
zhongwencool
0aec2f7605 feat: redis bridge v2 2023-11-29 16:06:46 +08:00
zhongwencool
8ec3857f4d
Merge pull request #12046 from savonarola/1128-fix-redis-54 
[5.4] Fix Redis connector
 2023-11-29 16:06:23 +08:00
William Yang
e90e843ff3
Merge pull request #12040 from qzhuyan/dev/william/quic-0.0.300 
feat(quic): bump to quicer-0.0.303
 2023-11-29 08:48:47 +01:00
JianBo He
d2e5f302a8 chore(http): add description for bridges v1 2023-11-29 13:47:59 +08:00
JianBo He
72bc046063 chore: avoid unnecessary default values being filled when querying via the v1 api. 
see: https://emqx.atlassian.net/browse/EMQX-11482
 2023-11-29 12:01:59 +08:00
JianBo He
d7bf8e97d2 chore: add tests case 2023-11-29 11:18:51 +08:00
Serge Tupchii
b0d4ba183e test: use emqx_cth_suite in emqx_coap_api_SUITE 
`emqx_coap_api_SUITE` was failing with `emqx_authn_schema_not_injected`
(missing authn schema mods in persistent_term) if it was run standalone.
 2023-11-28 20:02:21 +02:00
Serge Tupchii
3530ed66d5 fix(emqx_gateway_coap): take subscribe options from uri_query 
Closes #12031
 2023-11-28 20:02:21 +02:00
Thales Macedo Garitezi
095e7c4ecb test(flaky): more adjustments 2023-11-28 13:41:31 -03:00
Thales Macedo Garitezi
f6f61f4353
Merge pull request #12036 from thalesmg/test-more-flaky-tests-r54-20231127 
test: attempting to stabilize more flaky tests
 2023-11-28 13:40:49 -03:00
Ilya Averyanov
e5508105c7 fix(redis): start and load eredis app 2023-11-28 18:51:53 +03:00
Ilya Averyanov
55218e2df2 fix(redis): start and load eredis app 2023-11-28 18:51:19 +03:00
Zaiming (Stone) Shi
d93fdd8e9b
Merge pull request #12039 from zmstone/1128-sync-release-54 
Sync release-54 to master
 2023-11-28 15:52:49 +01:00
William Yang
64f6220cf5 feat(quic): bump to quicer-0.0.303 and emqtt 1.9.7 2023-11-28 14:43:47 +01:00
ieQu1
8a065ba6d0
Merge pull request #12027 from ieQu1/dev/ds-meta-persist-shard 
fix(ds): Persist shard table
 2023-11-28 14:33:57 +01:00
Thales Macedo Garitezi
f5c4fb5860 fix(ds_session): take conservative estimate of last_alive_at when bumping 
Addresses https://github.com/emqx/emqx/pull/12024#discussion_r1407432154
 2023-11-28 09:58:03 -03:00
Thales Macedo Garitezi
bd40b5c553 test(flaky): more adjustments 2023-11-28 09:48:40 -03:00
Ivan Dyachkov
2e11ab6a16 chore: 5.3.2-alpha.2 2023-11-28 13:46:28 +01:00
ieQu1
05e47254e2 fix(ds): Fixes related to the shards table 2023-11-28 12:59:09 +01:00
Zaiming (Stone) Shi
6c85e62d26 fix(schema): add namespaces 2023-11-28 12:07:42 +01:00
Zaiming (Stone) Shi
1b1cea24f3 Merge remote-tracking branch 'origin/release-54' into 1128-sync-release-54 2023-11-28 11:43:30 +01:00
Andrew Mayorov
34c9c022d0
chore(replayer): add comment describing what "until" means 2023-11-28 11:47:54 +03:00
JianBo He
bd6e9503e6 fix(http): compose the url and path in correctly format 2023-11-28 16:15:46 +08:00
zhongwencool
6e6e68c902
Merge pull request #12037 from zhongwencool/license-watermark-check 
fix: validate error when set license's watermark to 100%
 2023-11-28 16:04:39 +08:00
zhongwencool
29bcdb9a4a fix: validate error when set license's watermark to 100% 2023-11-28 13:16:16 +08:00
JianBo He
891ecc179d chore: fix flaky tests 2023-11-28 10:34:44 +08:00
JianBo He
c8b5c51bbc chore: fix failed test cases 2023-11-28 09:53:46 +08:00
JianBo He
cdb90ebe6b feat: rename webhook bridge to http bridge 2023-11-28 09:53:46 +08:00
JianBo He
dc99651690 test(bridge): ensure almost test cases passed 2023-11-28 09:53:46 +08:00
JianBo He
8954450c0b chore: fix compile warnings 2023-11-28 09:53:46 +08:00
JianBo He
96af7a74e8 feat: impl the http bridge v2 2023-11-28 09:53:46 +08:00
JianBo He
f9a1e747fd chore(http): break the bridge confs to connector and action parts 2023-11-28 09:53:46 +08:00
SergeTupchiy
3b1ae0f3df
Merge pull request #12017 from SergeTupchiy/EMQX-11319-data-backup-http-api 
EMQX-11319 data backup http api
 2023-11-27 19:23:54 +02:00
Thales Macedo Garitezi
e3fbf6c958 test: attempting to stabilize more flaky tests 2023-11-27 14:12:48 -03:00
Thales Macedo Garitezi
26e59f9508
Merge pull request #12024 from thalesmg/ds-session-expiry-m-20231124 
feat(ds): session expiry
 2023-11-27 14:11:38 -03:00
Stefan Strigler
ac370fed61
Merge pull request #12035 from sstrigler/EMQX-11382-a-new-field-should-be-added-to-the-interface-of-the-connector-to-return-the-actions-associated-with-it 
feat(emqx_connector): add field 'actions' in API response
 2023-11-27 17:39:44 +01:00
Zaiming (Stone) Shi
d933d0b9e0
Merge pull request #12016 from zmstone/1122-load-license-file 
1122 load license file
 2023-11-27 16:43:51 +01:00
Stefan Strigler
70978df5bb fix: use parse_id 2023-11-27 16:32:15 +01:00
Serge Tupchii
89fdbf3e73 fix(emqx_mgmt_cli): remove duplicated word in the log message 2023-11-27 17:30:58 +02:00
Serge Tupchii
64ee29af81 feat(emqx_management): implement data backup API 2023-11-27 17:30:58 +02:00
Andrew Mayorov
86685bdce2
feat(sessds): use integer tags for pubrange types 2023-11-27 18:27:15 +03:00
Andrew Mayorov
bb05281adb
refactor(sessds): add dedicated #ds_pubrange.tracks field 
This slightly simplifies the replayer code.
 2023-11-27 18:24:57 +03:00
Thales Macedo Garitezi
d88deb9ceb feat(ds): add session timer to bump last alive at timestamp 2023-11-27 12:19:52 -03:00
Stefan Strigler
048f4724a9 feat(emqx_connector): add field 'actions' in API response 
Also unify schemas, use emqx_connector_schema for the generic parts.
 2023-11-27 16:09:17 +01:00
Thales Macedo Garitezi
09c4e40511 refactor(ds): rename disconnected_at to last_alive_at, add more assertions 2023-11-27 11:48:44 -03:00
Thales Macedo Garitezi
f2dbddc315 test: attempting to stabilize more flaky tests 2023-11-27 11:36:32 -03:00
Thales Macedo Garitezi
6acdc6e432 feat: add /actions/:id/metrics/, /actions/:id/metrics/reset APIs 
Fixes https://emqx.atlassian.net/browse/EMQX-11381
 2023-11-27 11:33:11 -03:00
Andrew Mayorov
923898eadf
chore(chan): leave a TODO note for PUBREC handler 2023-11-27 09:55:46 +03:00
Andrew Mayorov
41973ee1fa
fix(sessds): please dialyzer with well-typed dummy msgs 2023-11-27 09:55:46 +03:00
Andrew Mayorov
46475fac66
feat(sessds): provide QoS2 message replay support 2023-11-27 09:55:45 +03:00
Andrew Mayorov
ce59cb71bb
chore: bump emqtt to 1.9.6 2023-11-27 09:55:45 +03:00
zhongwencool
b5a00ec6b2
Merge pull request #12023 from emqx/master 
chore: sync master to release-54
 2023-11-27 09:31:47 +08:00
ieQu1
bd7a84fe3e revert(ds): Don't duplicate the clean start in session_ds 2023-11-26 19:18:59 +01:00
Kjell Winblad
c85004b7ef chore: remove obsolete TODO 2023-11-24 21:22:27 +01:00
Kjell Winblad
c6c1d886f0 fix: make pgsql action schema properly 2023-11-24 21:19:26 +01:00
Thales Macedo Garitezi
839f9dbedb feat(ds): session expiry 
Fixes https://emqx.atlassian.net/browse/EMQX-11048
 2023-11-24 16:31:54 -03:00
Kjell Winblad
66945dcc5c fix: address more comments from @thalesmg 2023-11-24 20:31:29 +01:00
Kjell Winblad
30e248061f
fix: problems found by @thalesmg in code review 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-11-24 20:07:37 +01:00
Kjell Winblad
f070d80b1a fix: swagger examples for PostgreSQL, Matrix and Timescale 2023-11-24 18:50:52 +01:00
Kjell Winblad
d03674a505 fix: duplicate key in example 2023-11-24 17:59:56 +01:00
Kjell Winblad
fc7bedb81a fix: remove duplicated entry 2023-11-24 17:37:58 +01:00
Kjell Winblad
e920160805 fix: add enable and description fields to PostgreSQL connector 2023-11-24 17:33:39 +01:00
Kjell Winblad
3dca83c854 fix: all missing descriptions 2023-11-24 17:33:39 +01:00
Kjell Winblad
c5e281b84b fix: emqx_auth_postgresql test suites 2023-11-24 17:33:39 +01:00
Kjell Winblad
6ef9c6fe4a fix: ops 2023-11-24 17:33:39 +01:00
Kjell Winblad
f7296d549f fix: elvis problem 2023-11-24 17:33:39 +01:00
Kjell Winblad
64c015cf6f fix: unify the schema modules in emqx_bridge_pgsql 2023-11-24 17:33:39 +01:00
Kjell Winblad
b8f510d956 fix: add env variables about action info 2023-11-24 17:33:39 +01:00
Kjell Winblad
2e3028a8f8 fix(emqx_postgresql): fix lifecycle test 2023-11-24 17:33:39 +01:00
Kjell Winblad
dbe73c70b1 fix: dialyzer problem 2023-11-24 17:33:38 +01:00
Kjell Winblad
d5b62eead0 feat: split pgsql, matrix and timescale into connector action 
This commit splits the bridges pgsql, matrix and timescale into
connector and action.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11155
 2023-11-24 17:33:35 +01:00
Stefan Strigler
f8f8cf9f30
Merge pull request #11963 from sstrigler/EMQX-11156-bridge-v-2-mongo-db-support 
EMQX 11156 bridge v2 mongo db support
 2023-11-24 16:49:57 +01:00
zhongwencool
8f548f4cbb
Merge pull request #12020 from thalesmg/sync-r53-m-20231124 
chore: sync `release-53` to `master`
 2023-11-24 22:54:53 +08:00
Stefan Strigler
c1ef773e77 fix: check for sane state after regular shutdown 2023-11-24 15:09:35 +01:00
Stefan Strigler
4e077c951b feat(emqx_bridge_mongodb): port mongodb to shared connector and actions 2023-11-24 15:09:35 +01:00
Thales Macedo Garitezi
261fe8a831 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 10:10:09 -03:00
ieQu1
2d4b9a7b9d
Merge pull request #12019 from ieQu1/ds-clean-start 
Discard sessions when client connects with clean start = true
 2023-11-24 13:35:39 +01:00
Thales Macedo Garitezi
e95ec5b150 test: fix another flaky test 2023-11-24 09:24:21 -03:00
Thales Macedo Garitezi
f8fd95c683 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 09:22:24 -03:00
Thales Macedo Garitezi
6a3c438194
Merge pull request #12009 from thalesmg/fix-kafka-get-bridges-r53-20231123 
fix(bridge_api): don't mangle configs, use correct type as argument
 2023-11-24 08:56:24 -03:00
Stefan Strigler
9ebbc9bbea refactor(emqx_bridge): use more simplistic function to validate connector 2023-11-24 10:07:11 +01:00
Stefan Strigler
d2901afd1b fix(emqx_bridge_kafka): match example in api schema 2023-11-24 10:07:11 +01:00
ieQu1
e616e0746a feat(ds): Implement a function for dumping persistent session state 2023-11-24 04:39:46 +01:00
ieQu1
8dfcb69e52 feat(ds): Discard session when client connects with CleanStart=1 2023-11-24 04:17:28 +01:00
ieQu1
e8d18b0e09
Merge pull request #12014 from ieQu1/ds-qos0 
Support QoS0 messages in persistent_session_ds
 2023-11-23 23:43:18 +01:00
ieQu1
449bafc27e fix(ds): LTS trie handles empty topic levels 2023-11-23 22:16:38 +01:00
ieQu1
a158f25a40 fix(ds): Fix return type of emqx_persistent_session_ds:publish 2023-11-23 22:16:38 +01:00
ieQu1
c5bb86db67 feat(ds): Support QoS 0 2023-11-23 22:16:38 +01:00
ieQu1
1da9ec1d7d
Merge pull request #12015 from ieQu1/ds-lts-dont-store-message 
feat(ds): Don't store #message record in the DB
 2023-11-23 18:56:45 +01:00
Stefan Strigler
1c11a02a76
Merge pull request #12010 from sstrigler/EMQX-11415-e-5-3-2-api-should-return-404-when-the-built-in-database-authentication-doesnt-exist 
fix(emqx_auth): check authenticator exists in /authenticator/:id/users
 2023-11-23 18:08:48 +01:00
ieQu1
c69b82455e feat(ds): Don't store #message record in the DB 2023-11-23 17:56:35 +01:00
Zaiming (Stone) Shi
14077ec43b feat(license): allow setting 'default' license key 2023-11-23 17:52:17 +01:00
Zaiming (Stone) Shi
caaf8113fc feat(license): support loading license from file 2023-11-23 17:52:17 +01:00
Thales Macedo Garitezi
4597e1c5b9
Merge pull request #12011 from thalesmg/fix-kafka-get-bridges-m-20231123 
fix(bridge_api): don't mangle configs, use correct type as argument
 2023-11-23 13:15:39 -03:00
ieQu1
1ced8786fd feat(ds): Make session poll interval configurable 2023-11-23 17:10:03 +01:00
ieQu1
82e74d0201 feat(ds): Add a flag that forces all sessions to become durable 2023-11-23 17:10:03 +01:00
Stefan Strigler
8ba116d378 fix(emqx_auth): check authenticator exists in /authenticator/:id/users 2023-11-23 16:15:03 +01:00
Ilya Averyanov
ce77a12cd3
Merge pull request #12012 from savonarola/1123-fix-callback-types 
fix(ds): impose more flexible types for ds implementations
 2023-11-23 17:03:56 +02:00
Thales Macedo Garitezi
fdfa3213cc fix(bridge_api): don't mangle configs, use correct type as argument 
Fixes https://emqx.atlassian.net/browse/EMQX-11412

- The wrong type was being used in a list lookup function, resulting in the automatic
transformation being called erroneously and mangling the config.
- There was a left-over workaround still around which could still mangle the config.
 2023-11-23 11:41:26 -03:00
Zaiming (Stone) Shi
7375bc5f9b fix(license): allow CRLF in license keys 2023-11-23 15:39:18 +01:00
Zaiming (Stone) Shi
48b69bd60c chore: delete a TODO from perfectionism 2023-11-23 15:39:18 +01:00
Ilya Averyanov
29d42506d5 fix(ds): impose more flexible types for ds implementations 2023-11-23 17:16:11 +03:00
Thales Macedo Garitezi
8e8d6d2192
Merge pull request #12006 from thalesmg/test-debug-flaky-test-r53-20231122 
test(gcp_pubsub_consumer): fix flaky test
 2023-11-23 10:57:22 -03:00
Thales Macedo Garitezi
c89ec0b1f7 fix(bridge_api): don't mangle configs, use correct type as argument 
Fixes https://emqx.atlassian.net/browse/EMQX-11412

- The wrong type was being used in a list lookup function, resulting in the automatic
transformation being called erroneously and mangling the config.
- There was a left-over workaround still around which could still mangle the config.
 2023-11-23 10:25:36 -03:00
Thales Macedo Garitezi
f3693e5dbc fix(gcp_pubsub_producer): add missing references to api specs 2023-11-23 09:02:52 -03:00
Zaiming (Stone) Shi
209077593d
Merge pull request #11970 from zmstone/1108-add-v2-schema-dump 
1108 add v2 schema dump
 2023-11-22 23:42:54 +01:00
Thales Macedo Garitezi
db83457d13 test: fix flaky test 
The cause was that the call `sys:terminate/2` was timing out...

`exit/2` doens't always work:

```
 2023-11-22 19:14:40.974
killed async workers

Error: -22T19:14:40.974563+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15908.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 987, stack_size: 28, reductions: 1822; neighbours:
Error: -22T19:14:40.998051+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15908.7>.
2023-11-22T19:15:41.088752+00:00 [critical] Run stage failed: error:{badmatch,{timeout,#{expected_remaining => 1,mailbox => {messages,[]},msgs_so_far => []}}}, Stacktrace: [{emqx_bridge_gcp_pubsub_consumer_SUITE,'-t_async_worker_death_mid_pull/1-fun-17-',3,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1576}]},{emqx_bridge_gcp_pubsub_consumer_SUITE,t_async_worker_death_mid_pull,1,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1505}]}], Trace dump: "/emqx/_build/test/logs/ct_run.test@127.0.0.1.2023-11-22_19.14.27/snabbkaffe/1700680540975786370.log", mfa: undefined
Error: -22T19:15:46.095702+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15934.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 610, stack_size: 28, reductions: 1471; neighbours:
Error: -22T19:15:46.095192+00:00 [error] Supervisor: {local,ehttpc_sup}. Context: shutdown_error. Reason: killed. Offender: id={ehttpc_pool_sup,<<98,114,105,100,103,101,58,103,99,112,95,112,117,98,115,117,98,95,99,111,110,115,117,109,101,114,58,116,95,97,115,121,110,99,95,119,111,114,107,101,114,95,100,101,97,116,104,95,109,105,100,95,112,117,108,108,45,53,55,54,52,54,48,55,53,50,51,48,51,52,50,50,55,53,49>>},pid=<0.15903.7>.
Error: -22T19:15:46.095470+00:00 [error] Supervisor: {<0.15906.7>,ehttpc_worker_sup}. Context: shutdown_error. Reason: killed. Offender: id={worker,1},pid=<0.15924.7>.
Error: -22T19:15:46.096762+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15934.7>.
Warning: 2T19:15:46.098278+00:00 [warning] msg: remove_local_resource_failed, mfa: emqx_resource:remove_local/1(362), error: {error,timeout}, resource_id: <<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>
Error: -22T19:15:46.149090+00:00 [error] Generic server <0.15904.7> terminating. Reason: killed. Last message: {'EXIT',<0.15903.7>,killed}. State: {state,<<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>,1,random}.
Error: -22T19:15:46.149525+00:00 [error] crasher: initial call: ehttpc_pool:init/1, pid: <0.15904.7>, registered_name: [], exit: {killed,[{gen_server,decode_msg,9,[{file,"gen_server.erl"},{line,909}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [<0.15903.7>,ehttpc_sup,<0.15731.7>], message_queue_len: 0, messages: [], links: [], dictionary: [], trap_exit: true, status: running, heap_size: 376, stack_size: 28, reductions: 3428; neighbours:
```
 2023-11-22 17:41:32 -03:00
Zaiming (Stone) Shi
d9f964a44f test: fix test cases after schema type namespace change 2023-11-22 16:58:05 +01:00
Thales Macedo Garitezi
fc849f0c05 ci(test): add info to help diagnose flaky test 2023-11-22 12:36:10 -03:00
Thales Macedo Garitezi
6c9417efe0 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122 2023-11-22 12:02:34 -03:00
Zaiming (Stone) Shi
e065217e9a
Merge pull request #11997 from zmstone/1121-do-not-allow-special-char-as-prefix-for-resource-name 
1121 do not allow special char as prefix for resource name
 2023-11-22 15:46:11 +01:00
Thales Macedo Garitezi
274a254624
Merge pull request #12002 from thalesmg/multiple-action-infos-per-app-m-20231122 
feat(actions): allow multiple action info modules per application
 2023-11-22 10:42:56 -03:00
Thales Macedo Garitezi
bb549cdf8b feat(actions): allow multiple action info modules per application 2023-11-22 09:33:01 -03:00
Thales Macedo Garitezi
38d3a1d7d0 feat(actions): allow multiple action info modules per application 2023-11-22 09:30:37 -03:00
Zaiming (Stone) Shi
db33bc616a feat(schema): Add v2 scheam JSON dump 2023-11-22 13:12:35 +01:00
Zaiming (Stone) Shi
1b2c052646 docs: add type namespaces 2023-11-22 13:12:35 +01:00
Serge Tupchii
d9f95cdc56 feat: integrate OpenTelemetry log handler 2023-11-22 13:59:30 +02:00
Ilya Averyanov
a1b9a14fa1 feat(ds): allow fdb implementation for durable storage 2023-11-22 14:49:38 +03:00
Zaiming (Stone) Shi
3261a12140 fix(emqx_resource): do not allow leading _ or - as resource name 2023-11-22 10:58:54 +01:00
ieQu1
ee191803ea
Merge pull request #11990 from ieQu1/dev/ds-shards 
Implement a prototype of durable message sharding
 2023-11-22 00:57:54 +01:00