10069 Commits

Author	SHA1	Message	Date
Serge Tupchii	009a15c7d0	fix(emqx_mgmt_api_data_backup): validate empty file in upload API ... fixes EMQX-11488	2023-11-29 12:12:42 +02:00
Andrew Mayorov	d8691f1d64	refactor(sessds): rename marker → committed offset ... For better clarity.	2023-11-29 13:04:42 +03:00
zhongwencool	0aec2f7605	feat: redis bridge v2	2023-11-29 16:06:46 +08:00
zhongwencool	8ec3857f4d	Merge pull request #12046 from savonarola/1128-fix-redis-54 ... [5.4] Fix Redis connector	2023-11-29 16:06:23 +08:00
William Yang	e90e843ff3	Merge pull request #12040 from qzhuyan/dev/william/quic-0.0.300 ... feat(quic): bump to quicer-0.0.303	2023-11-29 08:48:47 +01:00
JianBo He	d2e5f302a8	chore(http): add description for bridges v1	2023-11-29 13:47:59 +08:00
JianBo He	72bc046063	chore: avoid unnecessary default values being filled when querying via the v1 api. ... see: https://emqx.atlassian.net/browse/EMQX-11482	2023-11-29 12:01:59 +08:00
JianBo He	d7bf8e97d2	chore: add tests case	2023-11-29 11:18:51 +08:00
Serge Tupchii	b0d4ba183e	test: use emqx_cth_suite in emqx_coap_api_SUITE ... `emqx_coap_api_SUITE` was failing with `emqx_authn_schema_not_injected` (missing authn schema mods in persistent_term) if it was run standalone.	2023-11-28 20:02:21 +02:00
Serge Tupchii	3530ed66d5	fix(emqx_gateway_coap): take subscribe options from uri_query ... Closes #12031	2023-11-28 20:02:21 +02:00
Thales Macedo Garitezi	095e7c4ecb	test(flaky): more adjustments	2023-11-28 13:41:31 -03:00
Thales Macedo Garitezi	f6f61f4353	Merge pull request #12036 from thalesmg/test-more-flaky-tests-r54-20231127 ... test: attempting to stabilize more flaky tests	2023-11-28 13:40:49 -03:00
Ilya Averyanov	e5508105c7	fix(redis): start and load eredis app	2023-11-28 18:51:53 +03:00
Ilya Averyanov	55218e2df2	fix(redis): start and load eredis app	2023-11-28 18:51:19 +03:00
Zaiming (Stone) Shi	d93fdd8e9b	Merge pull request #12039 from zmstone/1128-sync-release-54 ... Sync release-54 to master	2023-11-28 15:52:49 +01:00
William Yang	64f6220cf5	feat(quic): bump to quicer-0.0.303 and emqtt 1.9.7	2023-11-28 14:43:47 +01:00
ieQu1	8a065ba6d0	Merge pull request #12027 from ieQu1/dev/ds-meta-persist-shard ... fix(ds): Persist shard table	2023-11-28 14:33:57 +01:00
Thales Macedo Garitezi	f5c4fb5860	fix(ds_session): take conservative estimate of last_alive_at when bumping ... Addresses https://github.com/emqx/emqx/pull/12024#discussion_r1407432154	2023-11-28 09:58:03 -03:00
Thales Macedo Garitezi	bd40b5c553	test(flaky): more adjustments	2023-11-28 09:48:40 -03:00
Ivan Dyachkov	2e11ab6a16	chore: 5.3.2-alpha.2	2023-11-28 13:46:28 +01:00
ieQu1	05e47254e2	fix(ds): Fixes related to the shards table	2023-11-28 12:59:09 +01:00
Zaiming (Stone) Shi	6c85e62d26	fix(schema): add namespaces	2023-11-28 12:07:42 +01:00
Zaiming (Stone) Shi	1b1cea24f3	Merge remote-tracking branch 'origin/release-54' into 1128-sync-release-54	2023-11-28 11:43:30 +01:00
Andrew Mayorov	34c9c022d0	chore(replayer): add comment describing what "until" means	2023-11-28 11:47:54 +03:00
JianBo He	bd6e9503e6	fix(http): compose the url and path in correctly format	2023-11-28 16:15:46 +08:00
zhongwencool	6e6e68c902	Merge pull request #12037 from zhongwencool/license-watermark-check ... fix: validate error when set license's watermark to 100%	2023-11-28 16:04:39 +08:00
zhongwencool	29bcdb9a4a	fix: validate error when set license's watermark to 100%	2023-11-28 13:16:16 +08:00
JianBo He	891ecc179d	chore: fix flaky tests	2023-11-28 10:34:44 +08:00
JianBo He	c8b5c51bbc	chore: fix failed test cases	2023-11-28 09:53:46 +08:00
JianBo He	cdb90ebe6b	feat: rename webhook bridge to http bridge	2023-11-28 09:53:46 +08:00
JianBo He	dc99651690	test(bridge): ensure almost test cases passed	2023-11-28 09:53:46 +08:00
JianBo He	8954450c0b	chore: fix compile warnings	2023-11-28 09:53:46 +08:00
JianBo He	96af7a74e8	feat: impl the http bridge v2	2023-11-28 09:53:46 +08:00
JianBo He	f9a1e747fd	chore(http): break the bridge confs to connector and action parts	2023-11-28 09:53:46 +08:00
SergeTupchiy	3b1ae0f3df	Merge pull request #12017 from SergeTupchiy/EMQX-11319-data-backup-http-api ... EMQX-11319 data backup http api	2023-11-27 19:23:54 +02:00
Thales Macedo Garitezi	e3fbf6c958	test: attempting to stabilize more flaky tests	2023-11-27 14:12:48 -03:00
Thales Macedo Garitezi	26e59f9508	Merge pull request #12024 from thalesmg/ds-session-expiry-m-20231124 ... feat(ds): session expiry	2023-11-27 14:11:38 -03:00
Stefan Strigler	ac370fed61	Merge pull request #12035 from sstrigler/EMQX-11382-a-new-field-should-be-added-to-the-interface-of-the-connector-to-return-the-actions-associated-with-it ... feat(emqx_connector): add field 'actions' in API response	2023-11-27 17:39:44 +01:00
Zaiming (Stone) Shi	d933d0b9e0	Merge pull request #12016 from zmstone/1122-load-license-file ... 1122 load license file	2023-11-27 16:43:51 +01:00
Stefan Strigler	70978df5bb	fix: use parse_id	2023-11-27 16:32:15 +01:00
Serge Tupchii	89fdbf3e73	fix(emqx_mgmt_cli): remove duplicated word in the log message	2023-11-27 17:30:58 +02:00
Serge Tupchii	64ee29af81	feat(emqx_management): implement data backup API	2023-11-27 17:30:58 +02:00
Andrew Mayorov	86685bdce2	feat(sessds): use integer tags for pubrange types	2023-11-27 18:27:15 +03:00
Andrew Mayorov	bb05281adb	refactor(sessds): add dedicated #ds_pubrange.tracks field ... This slightly simplifies the replayer code.	2023-11-27 18:24:57 +03:00
Thales Macedo Garitezi	d88deb9ceb	feat(ds): add session timer to bump last alive at timestamp	2023-11-27 12:19:52 -03:00
Stefan Strigler	048f4724a9	feat(emqx_connector): add field 'actions' in API response ... Also unify schemas, use emqx_connector_schema for the generic parts.	2023-11-27 16:09:17 +01:00
Thales Macedo Garitezi	09c4e40511	refactor(ds): rename disconnected_at to last_alive_at, add more assertions	2023-11-27 11:48:44 -03:00
Thales Macedo Garitezi	f2dbddc315	test: attempting to stabilize more flaky tests	2023-11-27 11:36:32 -03:00
Thales Macedo Garitezi	6acdc6e432	feat: add /actions/:id/metrics/, /actions/:id/metrics/reset APIs ... Fixes https://emqx.atlassian.net/browse/EMQX-11381	2023-11-27 11:33:11 -03:00
Andrew Mayorov	923898eadf	chore(chan): leave a TODO note for PUBREC handler	2023-11-27 09:55:46 +03:00
Andrew Mayorov	41973ee1fa	fix(sessds): please dialyzer with well-typed dummy msgs	2023-11-27 09:55:46 +03:00
Andrew Mayorov	46475fac66	feat(sessds): provide QoS2 message replay support	2023-11-27 09:55:45 +03:00
Andrew Mayorov	ce59cb71bb	chore: bump emqtt to 1.9.6	2023-11-27 09:55:45 +03:00
zhongwencool	b5a00ec6b2	Merge pull request #12023 from emqx/master ... chore: sync master to release-54	2023-11-27 09:31:47 +08:00
ieQu1	bd7a84fe3e	revert(ds): Don't duplicate the clean start in session_ds	2023-11-26 19:18:59 +01:00
Kjell Winblad	c85004b7ef	chore: remove obsolete TODO	2023-11-24 21:22:27 +01:00
Kjell Winblad	c6c1d886f0	fix: make pgsql action schema properly	2023-11-24 21:19:26 +01:00
Thales Macedo Garitezi	839f9dbedb	feat(ds): session expiry ... Fixes https://emqx.atlassian.net/browse/EMQX-11048	2023-11-24 16:31:54 -03:00
Kjell Winblad	66945dcc5c	fix: address more comments from @thalesmg	2023-11-24 20:31:29 +01:00
Kjell Winblad	30e248061f	fix: problems found by @thalesmg in code review ... Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>	2023-11-24 20:07:37 +01:00
Kjell Winblad	f070d80b1a	fix: swagger examples for PostgreSQL, Matrix and Timescale	2023-11-24 18:50:52 +01:00
Kjell Winblad	d03674a505	fix: duplicate key in example	2023-11-24 17:59:56 +01:00
Kjell Winblad	fc7bedb81a	fix: remove duplicated entry	2023-11-24 17:37:58 +01:00
Kjell Winblad	e920160805	fix: add enable and description fields to PostgreSQL connector	2023-11-24 17:33:39 +01:00
Kjell Winblad	3dca83c854	fix: all missing descriptions	2023-11-24 17:33:39 +01:00
Kjell Winblad	c5e281b84b	fix: emqx_auth_postgresql test suites	2023-11-24 17:33:39 +01:00
Kjell Winblad	6ef9c6fe4a	fix: ops	2023-11-24 17:33:39 +01:00
Kjell Winblad	f7296d549f	fix: elvis problem	2023-11-24 17:33:39 +01:00
Kjell Winblad	64c015cf6f	fix: unify the schema modules in emqx_bridge_pgsql	2023-11-24 17:33:39 +01:00
Kjell Winblad	b8f510d956	fix: add env variables about action info	2023-11-24 17:33:39 +01:00
Kjell Winblad	2e3028a8f8	fix(emqx_postgresql): fix lifecycle test	2023-11-24 17:33:39 +01:00
Kjell Winblad	dbe73c70b1	fix: dialyzer problem	2023-11-24 17:33:38 +01:00
Kjell Winblad	d5b62eead0	feat: split pgsql, matrix and timescale into connector action ... This commit splits the bridges pgsql, matrix and timescale into connector and action. Fixes: https://emqx.atlassian.net/browse/EMQX-11155	2023-11-24 17:33:35 +01:00
Stefan Strigler	f8f8cf9f30	Merge pull request #11963 from sstrigler/EMQX-11156-bridge-v-2-mongo-db-support ... EMQX 11156 bridge v2 mongo db support	2023-11-24 16:49:57 +01:00
zhongwencool	8f548f4cbb	Merge pull request #12020 from thalesmg/sync-r53-m-20231124 ... chore: sync `release-53` to `master`	2023-11-24 22:54:53 +08:00
Stefan Strigler	c1ef773e77	fix: check for sane state after regular shutdown	2023-11-24 15:09:35 +01:00
Stefan Strigler	4e077c951b	feat(emqx_bridge_mongodb): port mongodb to shared connector and actions	2023-11-24 15:09:35 +01:00
Thales Macedo Garitezi	261fe8a831	Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124	2023-11-24 10:10:09 -03:00
ieQu1	2d4b9a7b9d	Merge pull request #12019 from ieQu1/ds-clean-start ... Discard sessions when client connects with clean start = true	2023-11-24 13:35:39 +01:00
Thales Macedo Garitezi	e95ec5b150	test: fix another flaky test	2023-11-24 09:24:21 -03:00
Thales Macedo Garitezi	f8fd95c683	Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124	2023-11-24 09:22:24 -03:00
Thales Macedo Garitezi	6a3c438194	Merge pull request #12009 from thalesmg/fix-kafka-get-bridges-r53-20231123 ... fix(bridge_api): don't mangle configs, use correct type as argument	2023-11-24 08:56:24 -03:00
Stefan Strigler	9ebbc9bbea	refactor(emqx_bridge): use more simplistic function to validate connector	2023-11-24 10:07:11 +01:00
Stefan Strigler	d2901afd1b	fix(emqx_bridge_kafka): match example in api schema	2023-11-24 10:07:11 +01:00
ieQu1	e616e0746a	feat(ds): Implement a function for dumping persistent session state	2023-11-24 04:39:46 +01:00
ieQu1	8dfcb69e52	feat(ds): Discard session when client connects with CleanStart=1	2023-11-24 04:17:28 +01:00
ieQu1	e8d18b0e09	Merge pull request #12014 from ieQu1/ds-qos0 ... Support QoS0 messages in persistent_session_ds	2023-11-23 23:43:18 +01:00
ieQu1	449bafc27e	fix(ds): LTS trie handles empty topic levels	2023-11-23 22:16:38 +01:00
ieQu1	a158f25a40	fix(ds): Fix return type of emqx_persistent_session_ds:publish	2023-11-23 22:16:38 +01:00
ieQu1	c5bb86db67	feat(ds): Support QoS 0	2023-11-23 22:16:38 +01:00
ieQu1	1da9ec1d7d	Merge pull request #12015 from ieQu1/ds-lts-dont-store-message ... feat(ds): Don't store #message record in the DB	2023-11-23 18:56:45 +01:00
Stefan Strigler	1c11a02a76	Merge pull request #12010 from sstrigler/EMQX-11415-e-5-3-2-api-should-return-404-when-the-built-in-database-authentication-doesnt-exist ... fix(emqx_auth): check authenticator exists in /authenticator/:id/users	2023-11-23 18:08:48 +01:00
ieQu1	c69b82455e	feat(ds): Don't store #message record in the DB	2023-11-23 17:56:35 +01:00
Zaiming (Stone) Shi	14077ec43b	feat(license): allow setting 'default' license key	2023-11-23 17:52:17 +01:00
Zaiming (Stone) Shi	caaf8113fc	feat(license): support loading license from file	2023-11-23 17:52:17 +01:00
Thales Macedo Garitezi	4597e1c5b9	Merge pull request #12011 from thalesmg/fix-kafka-get-bridges-m-20231123 ... fix(bridge_api): don't mangle configs, use correct type as argument	2023-11-23 13:15:39 -03:00
ieQu1	1ced8786fd	feat(ds): Make session poll interval configurable	2023-11-23 17:10:03 +01:00
ieQu1	82e74d0201	feat(ds): Add a flag that forces all sessions to become durable	2023-11-23 17:10:03 +01:00
Stefan Strigler	8ba116d378	fix(emqx_auth): check authenticator exists in /authenticator/:id/users	2023-11-23 16:15:03 +01:00
Ilya Averyanov	ce77a12cd3	Merge pull request #12012 from savonarola/1123-fix-callback-types ... fix(ds): impose more flexible types for ds implementations	2023-11-23 17:03:56 +02:00
Thales Macedo Garitezi	fdfa3213cc	fix(bridge_api): don't mangle configs, use correct type as argument ... Fixes https://emqx.atlassian.net/browse/EMQX-11412 - The wrong type was being used in a list lookup function, resulting in the automatic transformation being called erroneously and mangling the config. - There was a left-over workaround still around which could still mangle the config.	2023-11-23 11:41:26 -03:00
Zaiming (Stone) Shi	7375bc5f9b	fix(license): allow CRLF in license keys	2023-11-23 15:39:18 +01:00
Zaiming (Stone) Shi	48b69bd60c	chore: delete a TODO from perfectionism	2023-11-23 15:39:18 +01:00
Ilya Averyanov	29d42506d5	fix(ds): impose more flexible types for ds implementations	2023-11-23 17:16:11 +03:00
Thales Macedo Garitezi	8e8d6d2192	Merge pull request #12006 from thalesmg/test-debug-flaky-test-r53-20231122 ... test(gcp_pubsub_consumer): fix flaky test	2023-11-23 10:57:22 -03:00
Thales Macedo Garitezi	c89ec0b1f7	fix(bridge_api): don't mangle configs, use correct type as argument ... Fixes https://emqx.atlassian.net/browse/EMQX-11412 - The wrong type was being used in a list lookup function, resulting in the automatic transformation being called erroneously and mangling the config. - There was a left-over workaround still around which could still mangle the config.	2023-11-23 10:25:36 -03:00
Thales Macedo Garitezi	f3693e5dbc	fix(gcp_pubsub_producer): add missing references to api specs	2023-11-23 09:02:52 -03:00
Zaiming (Stone) Shi	209077593d	Merge pull request #11970 from zmstone/1108-add-v2-schema-dump ... 1108 add v2 schema dump	2023-11-22 23:42:54 +01:00
Thales Macedo Garitezi	db83457d13	test: fix flaky test ... The cause was that the call `sys:terminate/2` was timing out... `exit/2` doens't always work: ``` 2023-11-22 19:14:40.974 killed async workers Error: -22T19:14:40.974563+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15908.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 987, stack_size: 28, reductions: 1822; neighbours: Error: -22T19:14:40.998051+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15908.7>. 2023-11-22T19:15:41.088752+00:00 [critical] Run stage failed: error:{badmatch,{timeout,#{expected_remaining => 1,mailbox => {messages,[]},msgs_so_far => []}}}, Stacktrace: [{emqx_bridge_gcp_pubsub_consumer_SUITE,'-t_async_worker_death_mid_pull/1-fun-17-',3,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1576}]},{emqx_bridge_gcp_pubsub_consumer_SUITE,t_async_worker_death_mid_pull,1,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1505}]}], Trace dump: "/emqx/_build/test/logs/ct_run.test@127.0.0.1.2023-11-22_19.14.27/snabbkaffe/1700680540975786370.log", mfa: undefined Error: -22T19:15:46.095702+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15934.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 610, stack_size: 28, reductions: 1471; neighbours: Error: -22T19:15:46.095192+00:00 [error] Supervisor: {local,ehttpc_sup}. Context: shutdown_error. Reason: killed. Offender: id={ehttpc_pool_sup,<<98,114,105,100,103,101,58,103,99,112,95,112,117,98,115,117,98,95,99,111,110,115,117,109,101,114,58,116,95,97,115,121,110,99,95,119,111,114,107,101,114,95,100,101,97,116,104,95,109,105,100,95,112,117,108,108,45,53,55,54,52,54,48,55,53,50,51,48,51,52,50,50,55,53,49>>},pid=<0.15903.7>. Error: -22T19:15:46.095470+00:00 [error] Supervisor: {<0.15906.7>,ehttpc_worker_sup}. Context: shutdown_error. Reason: killed. Offender: id={worker,1},pid=<0.15924.7>. Error: -22T19:15:46.096762+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15934.7>. Warning: 2T19:15:46.098278+00:00 [warning] msg: remove_local_resource_failed, mfa: emqx_resource:remove_local/1(362), error: {error,timeout}, resource_id: <<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">> Error: -22T19:15:46.149090+00:00 [error] Generic server <0.15904.7> terminating. Reason: killed. Last message: {'EXIT',<0.15903.7>,killed}. State: {state,<<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>,1,random}. Error: -22T19:15:46.149525+00:00 [error] crasher: initial call: ehttpc_pool:init/1, pid: <0.15904.7>, registered_name: [], exit: {killed,[{gen_server,decode_msg,9,[{file,"gen_server.erl"},{line,909}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [<0.15903.7>,ehttpc_sup,<0.15731.7>], message_queue_len: 0, messages: [], links: [], dictionary: [], trap_exit: true, status: running, heap_size: 376, stack_size: 28, reductions: 3428; neighbours: ```	2023-11-22 17:41:32 -03:00
Zaiming (Stone) Shi	d9f964a44f	test: fix test cases after schema type namespace change	2023-11-22 16:58:05 +01:00
Thales Macedo Garitezi	fc849f0c05	ci(test): add info to help diagnose flaky test	2023-11-22 12:36:10 -03:00
Thales Macedo Garitezi	6c9417efe0	Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122	2023-11-22 12:02:34 -03:00
Zaiming (Stone) Shi	e065217e9a	Merge pull request #11997 from zmstone/1121-do-not-allow-special-char-as-prefix-for-resource-name ... 1121 do not allow special char as prefix for resource name	2023-11-22 15:46:11 +01:00
Thales Macedo Garitezi	274a254624	Merge pull request #12002 from thalesmg/multiple-action-infos-per-app-m-20231122 ... feat(actions): allow multiple action info modules per application	2023-11-22 10:42:56 -03:00
Thales Macedo Garitezi	bb549cdf8b	feat(actions): allow multiple action info modules per application	2023-11-22 09:33:01 -03:00
Thales Macedo Garitezi	38d3a1d7d0	feat(actions): allow multiple action info modules per application	2023-11-22 09:30:37 -03:00
Zaiming (Stone) Shi	db33bc616a	feat(schema): Add v2 scheam JSON dump	2023-11-22 13:12:35 +01:00
Zaiming (Stone) Shi	1b2c052646	docs: add type namespaces	2023-11-22 13:12:35 +01:00
Serge Tupchii	d9f95cdc56	feat: integrate OpenTelemetry log handler	2023-11-22 13:59:30 +02:00
Ilya Averyanov	a1b9a14fa1	feat(ds): allow fdb implementation for durable storage	2023-11-22 14:49:38 +03:00
Zaiming (Stone) Shi	3261a12140	fix(emqx_resource): do not allow leading _ or - as resource name	2023-11-22 10:58:54 +01:00
ieQu1	ee191803ea	Merge pull request #11990 from ieQu1/dev/ds-shards ... Implement a prototype of durable message sharding	2023-11-22 00:57:54 +01:00
Zaiming (Stone) Shi	869e73d637	Merge pull request #11966 from thalesmg/fix-kafka-parameters-r53-20231117 ... fix(kafka): don't return `parameters` from `/bridges` API	2023-11-22 00:36:39 +01:00
Zaiming (Stone) Shi	4fb2d3a503	Merge pull request #11996 from zmstone/1121-prepare-for-e5.3.2-alpha.1 ... chore: bump release version to e5.3.2-alpha.1	2023-11-21 22:01:05 +01:00
Thales Macedo Garitezi	39791511fc	chore: remove obsolete workaround	2023-11-21 17:38:13 -03:00
Thales Macedo Garitezi	01895fd660	Merge pull request #11995 from thalesmg/fix-action-api-bad-name-r53-20231121 ... fix(actions_api): don't crash on validation errors	2023-11-21 17:27:38 -03:00
Zaiming (Stone) Shi	5ebd954b16	chore: bump release version to e5.3.2-alpha.1	2023-11-21 21:19:05 +01:00
Zaiming (Stone) Shi	8fbdcab118	Merge pull request #11987 from zmstone/1120-do-not-crash-on-einval-after-check-cache ... fix(emqx_connection): handle socket activation error return	2023-11-21 20:31:56 +01:00
Thales Macedo Garitezi	3a8c332805	fix(actions_api): don't crash on validation errors ... Fixes https://emqx.atlassian.net/browse/EMQX-11394	2023-11-21 16:27:54 -03:00
ieQu1	3165b4f645	fix(ds): Abort application startup when rocksdb is not avialable	2023-11-21 20:27:07 +01:00
Thales Macedo Garitezi	11ec1a30a0	test(flaky): fix flaky pulsar test	2023-11-21 16:00:19 -03:00
Thales Macedo Garitezi	b3dffa4390	fix(kafka): don't return parameters from /bridges API ... Fixes https://emqx.atlassian.net/browse/EMQX-11412	2023-11-21 16:00:15 -03:00
ieQu1	3d823beb11	fix(ds): Apply review remarks	2023-11-21 19:37:01 +01:00
ieQu1	4d47490734	chore(ds): Rebase configuration	2023-11-21 19:37:01 +01:00
ieQu1	f5c71e8068	refactor(ds): Add a wrapper to the store batch API	2023-11-21 19:37:01 +01:00
ieQu1	2a1f7d946a	feat(ds): Shard messages by publisher client ID	2023-11-21 19:37:01 +01:00
ieQu1	62542e5844	feat(ds): Metadata storage for the replication layer	2023-11-21 19:37:01 +01:00
Kjell Winblad	b0d670aaa9	Merge pull request #11992 from kjellwinblad/kjell/backport/action_upgrade_downgrade_hooks ... fix(action): upgrade and downgrade strategy	2023-11-21 19:14:37 +01:00
Thales Macedo Garitezi	9e1796ec4f	feat(gcp_pubsub_producer): migrate GCP PubSub producer to actions ... Fixes https://emqx.atlassian.net/browse/EMQX-11157	2023-11-21 14:22:42 -03:00
Zaiming (Stone) Shi	fa91bacdfb	Merge pull request #11975 from zmstone/1119-fix-socket-close-race-condition ... fix(emqx_channel): do not log stale sock_close event as error	2023-11-21 16:47:03 +01:00
Kjell Winblad	fa7151f255	fix: port emqx_utils_maps:rename function from master ... The emqx_utils_maps:rename function is needed by action upgrade/downgrade hoos.	2023-11-21 15:18:22 +01:00
Thales Macedo Garitezi	6c9a8461f7	Merge pull request #11989 from thalesmg/confluent-action-m-20231120 ... feat: add confluent connector/action	2023-11-21 11:01:40 -03:00
Kjell Winblad	cd72dc11dd	fix: missing emqx_action_info module mapping	2023-11-21 14:22:29 +01:00
Thales Macedo Garitezi	068d151b14	Merge pull request #11983 from thalesmg/sync-r53-to-m-20231120 ... sync `release-53` to `master`	2023-11-21 09:02:51 -03:00
Thales Macedo Garitezi	759468efcf	Merge pull request #11986 from thalesmg/split-resource-opts-actions-r53-20231120 ... refactor: split `resource_opts` fields between connector and actions	2023-11-21 09:01:29 -03:00
Thales Macedo Garitezi	75ee3ced68	feat: add confluent connector/action ... Fixes https://emqx.atlassian.net/browse/EMQX-11044 See also: https://emqx.atlassian.net/wiki/spaces/EMQX/pages/712179934/Confluent+Integration	2023-11-21 08:59:42 -03:00
Kjell Winblad	6030bf6fa5	fix(action): upgrade and downgrade strategy ... This commit adds upgrade and downgrade hooks that are called when upgrading from a bridge V1 to connector and action or the other way around. The automatic translation is used if the callback is not defined. NOTE: Backported from master	2023-11-21 11:18:21 +01:00
Ilya Averyanov	c6fd1e4c75	Merge pull request #11971 from savonarola/1116-no-auth-rebalance-status ... feat(rebalance): improve rebalance usability	2023-11-21 12:10:18 +02:00
Andrew Mayorov	cd48c283db	Merge pull request #11988 from keynslug/ft/EMQX-11222/basic-config ... feat(schema): introduce separate root for new session persistence	2023-11-21 15:02:21 +07:00
JianBo He	9716b051d9	fix(ocpp): fix bad webscoket schema for HTTP API	2023-11-21 11:24:23 +08:00
Thales Macedo Garitezi	6f8630304d	Merge pull request #11980 from kjellwinblad/kjell/fix_fixup_callbacks/EMQX-11428 ... fix: bridge to action upgrade fixup hook should run after upgrade	2023-11-20 16:19:43 -03:00
Kjell Winblad	7fb5ade832	fix(kafka_producer): hocon renames kafka field	2023-11-20 19:20:13 +01:00
Kjell Winblad	d214ae8772	fix: problems found by @thalesmg in code review ... Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>	2023-11-20 18:59:57 +01:00
Andrew Mayorov	110a5a4896	feat(schema): introduce separate root for new session persistence ... With some knobs to choose a storage backend. Support only builtin RocksDB-based backend with minimal configuration for now.	2023-11-20 23:49:14 +07:00
Zaiming (Stone) Shi	b02711af79	refactor(emqx_ws_connection): rename cache to buffer for limiter	2023-11-20 17:47:20 +01:00
Andrew Mayorov	8e107ffe45	Merge pull request #11958 from keynslug/ft/EMQX-9745/preserve-acks ... feat(sessds): preserve acks / replays in session state	2023-11-20 23:40:22 +07:00
Kjell Winblad	edbfe090c9	feat: add custom callback for creating connector from bridge V1 config	2023-11-20 17:23:51 +01:00
Zaiming (Stone) Shi	ec19247271	refactor: rename limiter buffer related messages and var names	2023-11-20 16:55:26 +01:00
Andrew Mayorov	9684e79ee0	fix(sessds): ensure dup flag is on for replayed messages	2023-11-20 22:54:40 +07:00
Zaiming (Stone) Shi	8ec3b1db5d	fix(emqx_connection): handle socket activation error return	2023-11-20 16:51:39 +01:00
Kjell Winblad	3aa8044475	fix(action): upgrade and downgrade strategy ... This commit fixes the upgrade and downgrade strategy when upgrading from a bridge V1 to connector and action or the other way around so that the custom callbacks get the complete unchanged input instead of the result of the automatic translation. The automatic translation is used if the callback is not defined.	2023-11-20 16:40:27 +01:00
Thales Macedo Garitezi	f40f6bc5dd	refactor: split resource_opts fields between connector and actions	2023-11-20 12:37:36 -03:00
Stefan Strigler	1f1d9e58c6	fix(emqx_connector): don't crash in API on delete with active channels	2023-11-20 16:23:46 +01:00
Thales Macedo Garitezi	53e796bbd0	Merge remote-tracking branch 'origin/release-53' into sync-r53-to-m-20231120	2023-11-20 11:49:52 -03:00
Andrew Mayorov	ef7cfd0202	feat(sessds): add field to ds_pubrange for forward-compat	2023-11-20 19:56:55 +07:00
Thales Macedo Garitezi	b42d4e5ab0	Merge pull request #11950 from thalesmg/fix-bridge-v1-examples-r53-20231114 ... fix(bridges_v1): avoid merging action examples for non-v1 bridges	2023-11-20 09:12:23 -03:00
Thales Macedo Garitezi	926078b82c	Merge pull request #11960 from thalesmg/fix-kafka-producer-channel-status-r53-20231116 ... fix(kafka_producer): make status `connecting` while the client fails to connect	2023-11-20 09:12:05 -03:00
Kjell Winblad	79a764f117	fix: bridge to action upgrade fix up hook should run after upgrade ... This commit changes how the `emqx_action_info` callback `bridge_v1_to_action_fixup/1` works. It is now called after the automatic upgrade instead of before. Since the full Bridge V1 config might be needed to do the fixup, it is provided in a special field `<<"__bridge_v1_conf__">>`. The `<<"__bridge_v1_conf__">>` field is removed after the callback is called and can thus be ignored if it is not needed.	2023-11-20 12:06:21 +01:00
Stefan Strigler	84ff7b0b38	feat(emqx_bridge): action_info with dynamic lookup ... This allows a n:1 relation between v1 bridge_types to action/connector types as it's the case with mongodb for instance, where we had `mongodb_single` `mongodb_sharded` etc and the new implementation will just have `mongodb`.	2023-11-20 10:22:37 +01:00
Andrew Mayorov	d7974d835f	fix(sharesub): anticipate messages w/o redispatch header ... For instance, `emqx_session_mem` will push almost all of the messages that are still in the state to `emqx_shared_sub:redispatch/1`. After this commit, the session will no longer crash during channel terminate.	2023-11-20 15:53:24 +07:00
Andrew Mayorov	7081f1951f	refactor(sessds): use ds_pubrange record as is ... Instead of converting it into almost similar runtime representation.	2023-11-20 15:40:43 +07:00
Andrew Mayorov	a5ff4144fe	test(sessds): add complex testcase for session replay	2023-11-20 15:40:43 +07:00
Andrew Mayorov	1246d714c5	feat(sessds): preserve acks / ranges in mnesia for replays	2023-11-20 13:33:48 +07:00
Andrew Mayorov	5b40304d1f	chore(sessds): simplify subscriptions handling ... There's currently no point in storing parsed topic filters in the subscriptions table.	2023-11-20 13:33:48 +07:00
Andrew Mayorov	648b6ac63e	chore(sessds): rename iterators -> subscriptions ... Also try to make clearer the difference between 2 flavors of topic filter representation in use.	2023-11-20 13:33:48 +07:00
Zaiming (Stone) Shi	e73bf716ae	fix(emqx_channel): do not log stale sock_close event as error ... In some cases, EMQX may decide to close socket and mark connection at 'disconnected' state, for example, when DISCONNECTE packet is received, or, when failed to write data to socket. However, by the time EMQX decided to close the socket, the socket might have already been closed by peer, and the `tcp_closed` envet is already delivered to the process mailbox -- causing EMQX to handle sock_close event at 'disconnected' state.	2023-11-19 22:27:25 +01:00
Ilya Averyanov	e93e9ed108	feat(rebalance): improve rebalance usability ... * make availability API endpoint public * allow connections during wait_health_check interval * make availability status calculation more consistent and lightweight * refactor test to get rid of some mocks and to use cth	2023-11-18 21:29:16 +03:00
Thales Macedo Garitezi	eb3f54184e	refactor: address review comments and avoid transformations without schema knowledge	2023-11-17 16:16:34 -03:00
Kjell Winblad	86c126ffcd	feat: callbacks for fixup after automatic Bridge V1 upgrade/downgrade ... This commit adds callbacks to the emqx_action_info module for doing fixes (such as changing a field name) after the automatic split of a Bridge V1 config or the merge of connector and action configs for the compatibility layer.	2023-11-17 16:10:08 -03:00
Kjell Winblad	9feba802e9	chore: add convenience function for creating action schemas	2023-11-17 16:10:08 -03:00
Ilya Averyanov	58437cd35a	fix(mongodb): fix deadlock while stopping mongodb resource	2023-11-17 12:34:36 +03:00
zhongwencool	b2e168d508	feat: add prometheus.push_gateway.enable config	2023-11-17 09:48:39 +08:00
zhongwencool	c4eb9f86e7	chore: add refactor prometheus refactor changelog	2023-11-17 08:23:51 +08:00
zhongwencool	b343653769	chore: support list(tuple()) for schema_with_examples/3	2023-11-17 08:23:51 +08:00
zhongwencool	0c2610fa61	feat: support prometheus.enable_basic_auth	2023-11-17 08:23:51 +08:00
zhongwencool	c376a5db29	refactor: prometheus config	2023-11-17 08:23:51 +08:00
Thales Macedo Garitezi	b92821188b	fix(kafka_producer): make status connecting while the client fails to connect ... Fixes https://emqx.atlassian.net/browse/EMQX-11408 To make it consistent with the previous bridge behavior. Also, introduces macros for resource status to avoid problems with typos.	2023-11-16 14:50:23 -03:00
Ilya Averyanov	4c54efd8b1	fix(auth): fix deadlock while stopping mongodb resource	2023-11-16 19:29:08 +03:00
Thales Macedo Garitezi	36b5d58957	test: reorganize test suite a bit	2023-11-16 13:16:02 -03:00
Thales Macedo Garitezi	90571b7d8e	test: fix noise about undefined unofficial callbacks	2023-11-16 13:16:02 -03:00
Andrew Mayorov	656d575e2d	ci(router): fix flaky testcase	2023-11-16 17:53:20 +07:00
Andrew Mayorov	d019be5806	Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default ... feat(router): switch to v2 routing store by default	2023-11-16 15:56:26 +07:00
Andrew Mayorov	893e90b372	fix(kafka): use safe publish in consumer ... Routing with v2 schema is actually more strict with respect to input to `emqx_router` module routines. This causes Kafka consumer bridge to crash when it tries to publish a message to a topic that looks like a topic filter.	2023-11-15 22:37:00 +07:00
Andrew Mayorov	98f947f4f3	ci(router): fix flaky testcase	2023-11-15 22:28:52 +07:00
Andrew Mayorov	8919b08207	fix(utils): rename emqx_utils_stream:take/2 to consume/2 ... Which is more neutral and harder to confuse with a destructive `take` in collections.	2023-11-15 17:20:40 +07:00
Ivan Dyachkov	65ba381fd8	Merge remote-tracking branch 'upstream/master' into 1115-sync-master-to-r53	2023-11-15 09:25:03 +01:00
Thales Macedo Garitezi	f5545bbdf3	fix(schema_types): add missing secret() type	2023-11-14 16:08:13 -03:00
Thales Macedo Garitezi	5275a01727	fix(syskeeper): refactor action to new action info scheme	2023-11-14 19:49:28 +01:00
Thales Macedo Garitezi	d2cf29fae4	fix(schema): use hoconsc:union/1	2023-11-14 19:49:08 +01:00
Thales Macedo Garitezi	423fb32171	test: fix noise about undefined unofficial callbacks	2023-11-14 19:48:39 +01:00