yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,869 Commits 393 Branches 684 Tags 74 MiB
Commit Graph

9956 Commits

Author SHA1 Message Date
Kjell Winblad e920160805 fix: add enable and description fields to PostgreSQL connector 2023-11-24 17:33:39 +01:00
Kjell Winblad 3dca83c854 fix: all missing descriptions 2023-11-24 17:33:39 +01:00
Kjell Winblad c5e281b84b fix: emqx_auth_postgresql test suites 2023-11-24 17:33:39 +01:00
Kjell Winblad 6ef9c6fe4a fix: ops 2023-11-24 17:33:39 +01:00
Kjell Winblad f7296d549f fix: elvis problem 2023-11-24 17:33:39 +01:00
Kjell Winblad 64c015cf6f fix: unify the schema modules in emqx_bridge_pgsql 2023-11-24 17:33:39 +01:00
Kjell Winblad b8f510d956 fix: add env variables about action info 2023-11-24 17:33:39 +01:00
Kjell Winblad 2e3028a8f8 fix(emqx_postgresql): fix lifecycle test 2023-11-24 17:33:39 +01:00
Kjell Winblad dbe73c70b1 fix: dialyzer problem 2023-11-24 17:33:38 +01:00
Kjell Winblad d5b62eead0 feat: split pgsql, matrix and timescale into connector action 
This commit splits the bridges pgsql, matrix and timescale into
connector and action.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11155
 2023-11-24 17:33:35 +01:00
Stefan Strigler f8f8cf9f30
Merge pull request #11963 from sstrigler/EMQX-11156-bridge-v-2-mongo-db-support 
EMQX 11156 bridge v2 mongo db support
 2023-11-24 16:49:57 +01:00
zhongwencool 8f548f4cbb
Merge pull request #12020 from thalesmg/sync-r53-m-20231124 
chore: sync `release-53` to `master`
 2023-11-24 22:54:53 +08:00
Stefan Strigler c1ef773e77 fix: check for sane state after regular shutdown 2023-11-24 15:09:35 +01:00
Stefan Strigler 4e077c951b feat(emqx_bridge_mongodb): port mongodb to shared connector and actions 2023-11-24 15:09:35 +01:00
Thales Macedo Garitezi 261fe8a831 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 10:10:09 -03:00
ieQu1 2d4b9a7b9d
Merge pull request #12019 from ieQu1/ds-clean-start 
Discard sessions when client connects with clean start = true
 2023-11-24 13:35:39 +01:00
Thales Macedo Garitezi e95ec5b150 test: fix another flaky test 2023-11-24 09:24:21 -03:00
Thales Macedo Garitezi f8fd95c683 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 09:22:24 -03:00
Thales Macedo Garitezi 6a3c438194
Merge pull request #12009 from thalesmg/fix-kafka-get-bridges-r53-20231123 
fix(bridge_api): don't mangle configs, use correct type as argument
 2023-11-24 08:56:24 -03:00
Stefan Strigler 9ebbc9bbea refactor(emqx_bridge): use more simplistic function to validate connector 2023-11-24 10:07:11 +01:00
Stefan Strigler d2901afd1b fix(emqx_bridge_kafka): match example in api schema 2023-11-24 10:07:11 +01:00
ieQu1 e616e0746a feat(ds): Implement a function for dumping persistent session state 2023-11-24 04:39:46 +01:00
ieQu1 8dfcb69e52 feat(ds): Discard session when client connects with CleanStart=1 2023-11-24 04:17:28 +01:00
ieQu1 e8d18b0e09
Merge pull request #12014 from ieQu1/ds-qos0 
Support QoS0 messages in persistent_session_ds
 2023-11-23 23:43:18 +01:00
ieQu1 449bafc27e fix(ds): LTS trie handles empty topic levels 2023-11-23 22:16:38 +01:00
ieQu1 a158f25a40 fix(ds): Fix return type of emqx_persistent_session_ds:publish 2023-11-23 22:16:38 +01:00
ieQu1 c5bb86db67 feat(ds): Support QoS 0 2023-11-23 22:16:38 +01:00
ieQu1 1da9ec1d7d
Merge pull request #12015 from ieQu1/ds-lts-dont-store-message 
feat(ds): Don't store #message record in the DB
 2023-11-23 18:56:45 +01:00
Stefan Strigler 1c11a02a76
Merge pull request #12010 from sstrigler/EMQX-11415-e-5-3-2-api-should-return-404-when-the-built-in-database-authentication-doesnt-exist 
fix(emqx_auth): check authenticator exists in /authenticator/:id/users
 2023-11-23 18:08:48 +01:00
ieQu1 c69b82455e feat(ds): Don't store #message record in the DB 2023-11-23 17:56:35 +01:00
Zaiming (Stone) Shi 14077ec43b feat(license): allow setting 'default' license key 2023-11-23 17:52:17 +01:00
Zaiming (Stone) Shi caaf8113fc feat(license): support loading license from file 2023-11-23 17:52:17 +01:00
Thales Macedo Garitezi 4597e1c5b9
Merge pull request #12011 from thalesmg/fix-kafka-get-bridges-m-20231123 
fix(bridge_api): don't mangle configs, use correct type as argument
 2023-11-23 13:15:39 -03:00
ieQu1 1ced8786fd feat(ds): Make session poll interval configurable 2023-11-23 17:10:03 +01:00
ieQu1 82e74d0201 feat(ds): Add a flag that forces all sessions to become durable 2023-11-23 17:10:03 +01:00
Stefan Strigler 8ba116d378 fix(emqx_auth): check authenticator exists in /authenticator/:id/users 2023-11-23 16:15:03 +01:00
Ilya Averyanov ce77a12cd3
Merge pull request #12012 from savonarola/1123-fix-callback-types 
fix(ds): impose more flexible types for ds implementations
 2023-11-23 17:03:56 +02:00
Thales Macedo Garitezi fdfa3213cc fix(bridge_api): don't mangle configs, use correct type as argument 
Fixes https://emqx.atlassian.net/browse/EMQX-11412

- The wrong type was being used in a list lookup function, resulting in the automatic
transformation being called erroneously and mangling the config.
- There was a left-over workaround still around which could still mangle the config.
 2023-11-23 11:41:26 -03:00
Zaiming (Stone) Shi 7375bc5f9b fix(license): allow CRLF in license keys 2023-11-23 15:39:18 +01:00
Zaiming (Stone) Shi 48b69bd60c chore: delete a TODO from perfectionism 2023-11-23 15:39:18 +01:00
Ilya Averyanov 29d42506d5 fix(ds): impose more flexible types for ds implementations 2023-11-23 17:16:11 +03:00
Thales Macedo Garitezi 8e8d6d2192
Merge pull request #12006 from thalesmg/test-debug-flaky-test-r53-20231122 
test(gcp_pubsub_consumer): fix flaky test
 2023-11-23 10:57:22 -03:00
Thales Macedo Garitezi c89ec0b1f7 fix(bridge_api): don't mangle configs, use correct type as argument 
Fixes https://emqx.atlassian.net/browse/EMQX-11412

- The wrong type was being used in a list lookup function, resulting in the automatic
transformation being called erroneously and mangling the config.
- There was a left-over workaround still around which could still mangle the config.
 2023-11-23 10:25:36 -03:00
Thales Macedo Garitezi f3693e5dbc fix(gcp_pubsub_producer): add missing references to api specs 2023-11-23 09:02:52 -03:00
Zaiming (Stone) Shi 209077593d
Merge pull request #11970 from zmstone/1108-add-v2-schema-dump 
1108 add v2 schema dump
 2023-11-22 23:42:54 +01:00
Thales Macedo Garitezi db83457d13 test: fix flaky test 
The cause was that the call `sys:terminate/2` was timing out...

`exit/2` doens't always work:

```
 2023-11-22 19:14:40.974
killed async workers

Error: -22T19:14:40.974563+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15908.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 987, stack_size: 28, reductions: 1822; neighbours:
Error: -22T19:14:40.998051+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15908.7>.
2023-11-22T19:15:41.088752+00:00 [critical] Run stage failed: error:{badmatch,{timeout,#{expected_remaining => 1,mailbox => {messages,[]},msgs_so_far => []}}}, Stacktrace: [{emqx_bridge_gcp_pubsub_consumer_SUITE,'-t_async_worker_death_mid_pull/1-fun-17-',3,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1576}]},{emqx_bridge_gcp_pubsub_consumer_SUITE,t_async_worker_death_mid_pull,1,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1505}]}], Trace dump: "/emqx/_build/test/logs/ct_run.test@127.0.0.1.2023-11-22_19.14.27/snabbkaffe/1700680540975786370.log", mfa: undefined
Error: -22T19:15:46.095702+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15934.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 610, stack_size: 28, reductions: 1471; neighbours:
Error: -22T19:15:46.095192+00:00 [error] Supervisor: {local,ehttpc_sup}. Context: shutdown_error. Reason: killed. Offender: id={ehttpc_pool_sup,<<98,114,105,100,103,101,58,103,99,112,95,112,117,98,115,117,98,95,99,111,110,115,117,109,101,114,58,116,95,97,115,121,110,99,95,119,111,114,107,101,114,95,100,101,97,116,104,95,109,105,100,95,112,117,108,108,45,53,55,54,52,54,48,55,53,50,51,48,51,52,50,50,55,53,49>>},pid=<0.15903.7>.
Error: -22T19:15:46.095470+00:00 [error] Supervisor: {<0.15906.7>,ehttpc_worker_sup}. Context: shutdown_error. Reason: killed. Offender: id={worker,1},pid=<0.15924.7>.
Error: -22T19:15:46.096762+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15934.7>.
Warning: 2T19:15:46.098278+00:00 [warning] msg: remove_local_resource_failed, mfa: emqx_resource:remove_local/1(362), error: {error,timeout}, resource_id: <<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>
Error: -22T19:15:46.149090+00:00 [error] Generic server <0.15904.7> terminating. Reason: killed. Last message: {'EXIT',<0.15903.7>,killed}. State: {state,<<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>,1,random}.
Error: -22T19:15:46.149525+00:00 [error] crasher: initial call: ehttpc_pool:init/1, pid: <0.15904.7>, registered_name: [], exit: {killed,[{gen_server,decode_msg,9,[{file,"gen_server.erl"},{line,909}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [<0.15903.7>,ehttpc_sup,<0.15731.7>], message_queue_len: 0, messages: [], links: [], dictionary: [], trap_exit: true, status: running, heap_size: 376, stack_size: 28, reductions: 3428; neighbours:
```
 2023-11-22 17:41:32 -03:00
Zaiming (Stone) Shi d9f964a44f test: fix test cases after schema type namespace change 2023-11-22 16:58:05 +01:00
Thales Macedo Garitezi fc849f0c05 ci(test): add info to help diagnose flaky test 2023-11-22 12:36:10 -03:00
Thales Macedo Garitezi 6c9417efe0 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122 2023-11-22 12:02:34 -03:00
Zaiming (Stone) Shi e065217e9a
Merge pull request #11997 from zmstone/1121-do-not-allow-special-char-as-prefix-for-resource-name 
1121 do not allow special char as prefix for resource name
 2023-11-22 15:46:11 +01:00
Thales Macedo Garitezi 274a254624
Merge pull request #12002 from thalesmg/multiple-action-infos-per-app-m-20231122 
feat(actions): allow multiple action info modules per application
 2023-11-22 10:42:56 -03:00
Thales Macedo Garitezi bb549cdf8b feat(actions): allow multiple action info modules per application 2023-11-22 09:33:01 -03:00
Thales Macedo Garitezi 38d3a1d7d0 feat(actions): allow multiple action info modules per application 2023-11-22 09:30:37 -03:00
Zaiming (Stone) Shi db33bc616a feat(schema): Add v2 scheam JSON dump 2023-11-22 13:12:35 +01:00
Zaiming (Stone) Shi 1b2c052646 docs: add type namespaces 2023-11-22 13:12:35 +01:00
Serge Tupchii d9f95cdc56 feat: integrate OpenTelemetry log handler 2023-11-22 13:59:30 +02:00
Ilya Averyanov a1b9a14fa1 feat(ds): allow fdb implementation for durable storage 2023-11-22 14:49:38 +03:00
Zaiming (Stone) Shi 3261a12140 fix(emqx_resource): do not allow leading _ or - as resource name 2023-11-22 10:58:54 +01:00
ieQu1 ee191803ea
Merge pull request #11990 from ieQu1/dev/ds-shards 
Implement a prototype of durable message sharding
 2023-11-22 00:57:54 +01:00
Zaiming (Stone) Shi 869e73d637
Merge pull request #11966 from thalesmg/fix-kafka-parameters-r53-20231117 
fix(kafka): don't return `parameters` from `/bridges` API
 2023-11-22 00:36:39 +01:00
Zaiming (Stone) Shi 4fb2d3a503
Merge pull request #11996 from zmstone/1121-prepare-for-e5.3.2-alpha.1 
chore: bump release version to e5.3.2-alpha.1
 2023-11-21 22:01:05 +01:00
Thales Macedo Garitezi 39791511fc chore: remove obsolete workaround 2023-11-21 17:38:13 -03:00
Thales Macedo Garitezi 01895fd660
Merge pull request #11995 from thalesmg/fix-action-api-bad-name-r53-20231121 
fix(actions_api): don't crash on validation errors
 2023-11-21 17:27:38 -03:00
Zaiming (Stone) Shi 5ebd954b16 chore: bump release version to e5.3.2-alpha.1 2023-11-21 21:19:05 +01:00
Zaiming (Stone) Shi 8fbdcab118
Merge pull request #11987 from zmstone/1120-do-not-crash-on-einval-after-check-cache 
fix(emqx_connection): handle socket activation error return
 2023-11-21 20:31:56 +01:00
Thales Macedo Garitezi 3a8c332805 fix(actions_api): don't crash on validation errors 
Fixes https://emqx.atlassian.net/browse/EMQX-11394
 2023-11-21 16:27:54 -03:00
ieQu1 3165b4f645 fix(ds): Abort application startup when rocksdb is not avialable 2023-11-21 20:27:07 +01:00
Thales Macedo Garitezi 11ec1a30a0 test(flaky): fix flaky pulsar test 2023-11-21 16:00:19 -03:00
Thales Macedo Garitezi b3dffa4390 fix(kafka): don't return `parameters` from `/bridges` API 
Fixes https://emqx.atlassian.net/browse/EMQX-11412
 2023-11-21 16:00:15 -03:00
ieQu1 3d823beb11 fix(ds): Apply review remarks 2023-11-21 19:37:01 +01:00
ieQu1 4d47490734 chore(ds): Rebase configuration 2023-11-21 19:37:01 +01:00
ieQu1 f5c71e8068 refactor(ds): Add a wrapper to the store batch API 2023-11-21 19:37:01 +01:00
ieQu1 2a1f7d946a feat(ds): Shard messages by publisher client ID 2023-11-21 19:37:01 +01:00
ieQu1 62542e5844 feat(ds): Metadata storage for the replication layer 2023-11-21 19:37:01 +01:00
Kjell Winblad b0d670aaa9
Merge pull request #11992 from kjellwinblad/kjell/backport/action_upgrade_downgrade_hooks 
fix(action): upgrade and downgrade strategy
 2023-11-21 19:14:37 +01:00
Thales Macedo Garitezi 9e1796ec4f feat(gcp_pubsub_producer): migrate GCP PubSub producer to actions 
Fixes https://emqx.atlassian.net/browse/EMQX-11157
 2023-11-21 14:22:42 -03:00
Zaiming (Stone) Shi fa91bacdfb
Merge pull request #11975 from zmstone/1119-fix-socket-close-race-condition 
fix(emqx_channel): do not log stale sock_close event as error
 2023-11-21 16:47:03 +01:00
Kjell Winblad fa7151f255 fix: port emqx_utils_maps:rename function from master 
The emqx_utils_maps:rename function is needed by action
upgrade/downgrade hoos.
 2023-11-21 15:18:22 +01:00
Thales Macedo Garitezi 6c9a8461f7
Merge pull request #11989 from thalesmg/confluent-action-m-20231120 
feat: add confluent connector/action
 2023-11-21 11:01:40 -03:00
Kjell Winblad cd72dc11dd fix: missing emqx_action_info module mapping 2023-11-21 14:22:29 +01:00
Thales Macedo Garitezi 068d151b14
Merge pull request #11983 from thalesmg/sync-r53-to-m-20231120 
sync `release-53` to `master`
 2023-11-21 09:02:51 -03:00
Thales Macedo Garitezi 759468efcf
Merge pull request #11986 from thalesmg/split-resource-opts-actions-r53-20231120 
refactor: split `resource_opts` fields between connector and actions
 2023-11-21 09:01:29 -03:00
Thales Macedo Garitezi 75ee3ced68 feat: add confluent connector/action 
Fixes https://emqx.atlassian.net/browse/EMQX-11044

See also: https://emqx.atlassian.net/wiki/spaces/EMQX/pages/712179934/Confluent+Integration
 2023-11-21 08:59:42 -03:00
Kjell Winblad 6030bf6fa5 fix(action): upgrade and downgrade strategy 
This commit adds upgrade and downgrade hooks that are called when
upgrading from a bridge V1 to connector and action or the other way
around. The automatic translation is used if the callback is not
defined.

NOTE: Backported from master
 2023-11-21 11:18:21 +01:00
Ilya Averyanov c6fd1e4c75
Merge pull request #11971 from savonarola/1116-no-auth-rebalance-status 
feat(rebalance): improve rebalance usability
 2023-11-21 12:10:18 +02:00
Andrew Mayorov cd48c283db
Merge pull request #11988 from keynslug/ft/EMQX-11222/basic-config 
feat(schema): introduce separate root for new session persistence
 2023-11-21 15:02:21 +07:00
JianBo He 9716b051d9 fix(ocpp): fix bad webscoket schema for HTTP API 2023-11-21 11:24:23 +08:00
Thales Macedo Garitezi 6f8630304d
Merge pull request #11980 from kjellwinblad/kjell/fix_fixup_callbacks/EMQX-11428 
fix: bridge to action upgrade fixup hook should run after upgrade
 2023-11-20 16:19:43 -03:00
Kjell Winblad 7fb5ade832 fix(kafka_producer): hocon renames kafka field 2023-11-20 19:20:13 +01:00
Kjell Winblad d214ae8772
fix: problems found by @thalesmg in code review 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-11-20 18:59:57 +01:00
Andrew Mayorov 110a5a4896
feat(schema): introduce separate root for new session persistence 
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
 2023-11-20 23:49:14 +07:00
Zaiming (Stone) Shi b02711af79 refactor(emqx_ws_connection): rename cache to buffer for limiter 2023-11-20 17:47:20 +01:00
Andrew Mayorov 8e107ffe45
Merge pull request #11958 from keynslug/ft/EMQX-9745/preserve-acks 
feat(sessds): preserve acks / replays in session state
 2023-11-20 23:40:22 +07:00
Kjell Winblad edbfe090c9 feat: add custom callback for creating connector from bridge V1 config 2023-11-20 17:23:51 +01:00
Zaiming (Stone) Shi ec19247271 refactor: rename limiter buffer related messages and var names 2023-11-20 16:55:26 +01:00
Andrew Mayorov 9684e79ee0
fix(sessds): ensure dup flag is on for replayed messages 2023-11-20 22:54:40 +07:00
Zaiming (Stone) Shi 8ec3b1db5d fix(emqx_connection): handle socket activation error return 2023-11-20 16:51:39 +01:00
Kjell Winblad 3aa8044475 fix(action): upgrade and downgrade strategy 
This commit fixes the upgrade and downgrade strategy when upgrading
from a bridge V1 to connector and action or the other way around so that
the custom callbacks get the complete unchanged input instead of the
result of the automatic translation. The automatic translation is used
if the callback is not defined.
 2023-11-20 16:40:27 +01:00
Thales Macedo Garitezi f40f6bc5dd refactor: split `resource_opts` fields between connector and actions 2023-11-20 12:37:36 -03:00
Stefan Strigler 1f1d9e58c6 fix(emqx_connector): don't crash in API on delete with active channels 2023-11-20 16:23:46 +01:00
Thales Macedo Garitezi 53e796bbd0 Merge remote-tracking branch 'origin/release-53' into sync-r53-to-m-20231120 2023-11-20 11:49:52 -03:00
Andrew Mayorov ef7cfd0202
feat(sessds): add field to `ds_pubrange` for forward-compat 2023-11-20 19:56:55 +07:00
Thales Macedo Garitezi b42d4e5ab0
Merge pull request #11950 from thalesmg/fix-bridge-v1-examples-r53-20231114 
fix(bridges_v1): avoid merging action examples for non-v1 bridges
 2023-11-20 09:12:23 -03:00
Thales Macedo Garitezi 926078b82c
Merge pull request #11960 from thalesmg/fix-kafka-producer-channel-status-r53-20231116 
fix(kafka_producer): make status `connecting` while the client fails to connect
 2023-11-20 09:12:05 -03:00
Kjell Winblad 79a764f117 fix: bridge to action upgrade fix up hook should run after upgrade 
This commit changes how the `emqx_action_info` callback
`bridge_v1_to_action_fixup/1` works. It is now called after the
automatic upgrade instead of before. Since the full Bridge V1 config
might be needed to do the fixup, it is provided in a special field
`<<"__bridge_v1_conf__">>`. The `<<"__bridge_v1_conf__">>` field is
removed after the callback is called and can thus be ignored if it is
not needed.
 2023-11-20 12:06:21 +01:00
Stefan Strigler 84ff7b0b38 feat(emqx_bridge): action_info with dynamic lookup 
This allows a n:1 relation between v1 bridge_types to action/connector types as
it's the case with mongodb for instance, where we had `mongodb_single`
`mongodb_sharded` etc and the new implementation will just have `mongodb`.
 2023-11-20 10:22:37 +01:00
Andrew Mayorov d7974d835f
fix(sharesub): anticipate messages w/o redispatch header 
For instance, `emqx_session_mem` will push almost all of the messages
that are still in the state to `emqx_shared_sub:redispatch/1`. After
this commit, the session will no longer crash during channel terminate.
 2023-11-20 15:53:24 +07:00
Andrew Mayorov 7081f1951f
refactor(sessds): use `ds_pubrange` record as is 
Instead of converting it into almost similar runtime representation.
 2023-11-20 15:40:43 +07:00
Andrew Mayorov a5ff4144fe
test(sessds): add complex testcase for session replay 2023-11-20 15:40:43 +07:00
Andrew Mayorov 1246d714c5
feat(sessds): preserve acks / ranges in mnesia for replays 2023-11-20 13:33:48 +07:00
Andrew Mayorov 5b40304d1f
chore(sessds): simplify subscriptions handling 
There's currently no point in storing parsed topic filters in the
subscriptions table.
 2023-11-20 13:33:48 +07:00
Andrew Mayorov 648b6ac63e
chore(sessds): rename iterators -> subscriptions 
Also try to make clearer the difference between 2 flavors of topic
filter representation in use.
 2023-11-20 13:33:48 +07:00
Zaiming (Stone) Shi e73bf716ae fix(emqx_channel): do not log stale sock_close event as error 
In some cases, EMQX may decide to close socket and mark connection
at 'disconnected' state, for example, when DISCONNECTE packet is
received, or, when failed to write data to socket.
However, by the time EMQX decided to close the socket, the socket
might have already been closed by peer, and the `tcp_closed` envet
is already delivered to the process mailbox -- causing EMQX to
handle sock_close event at 'disconnected' state.
 2023-11-19 22:27:25 +01:00
Ilya Averyanov e93e9ed108 feat(rebalance): improve rebalance usability 
* make availability API endpoint public
* allow connections during wait_health_check interval
* make availability status calculation more consistent and lightweight
* refactor test to get rid of some mocks and to use cth
 2023-11-18 21:29:16 +03:00
Thales Macedo Garitezi eb3f54184e refactor: address review comments and avoid transformations without schema knowledge 2023-11-17 16:16:34 -03:00
Kjell Winblad 86c126ffcd feat: callbacks for fixup after automatic Bridge V1 upgrade/downgrade 
This commit adds callbacks to the emqx_action_info module for
doing fixes (such as changing a field name) after the automatic
split of a Bridge V1 config or the merge of connector and action
configs for the compatibility layer.
 2023-11-17 16:10:08 -03:00
Kjell Winblad 9feba802e9 chore: add convenience function for creating action schemas 2023-11-17 16:10:08 -03:00
Ilya Averyanov 58437cd35a fix(mongodb): fix deadlock while stopping mongodb resource 2023-11-17 12:34:36 +03:00
zhongwencool b2e168d508 feat: add prometheus.push_gateway.enable config 2023-11-17 09:48:39 +08:00
zhongwencool c4eb9f86e7 chore: add refactor prometheus refactor changelog 2023-11-17 08:23:51 +08:00
zhongwencool b343653769 chore: support list(tuple()) for schema_with_examples/3 2023-11-17 08:23:51 +08:00
zhongwencool 0c2610fa61 feat: support prometheus.enable_basic_auth 2023-11-17 08:23:51 +08:00
zhongwencool c376a5db29 refactor: prometheus config 2023-11-17 08:23:51 +08:00
Thales Macedo Garitezi b92821188b fix(kafka_producer): make status `connecting` while the client fails to connect 
Fixes https://emqx.atlassian.net/browse/EMQX-11408

To make it consistent with the previous bridge behavior.

Also, introduces macros for resource status to avoid problems with typos.
 2023-11-16 14:50:23 -03:00
Ilya Averyanov 4c54efd8b1 fix(auth): fix deadlock while stopping mongodb resource 2023-11-16 19:29:08 +03:00
Thales Macedo Garitezi 36b5d58957 test: reorganize test suite a bit 2023-11-16 13:16:02 -03:00
Thales Macedo Garitezi 90571b7d8e test: fix noise about undefined unofficial callbacks 2023-11-16 13:16:02 -03:00
Andrew Mayorov 656d575e2d
ci(router): fix flaky testcase 2023-11-16 17:53:20 +07:00
Andrew Mayorov d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default 
feat(router): switch to v2 routing store by default
 2023-11-16 15:56:26 +07:00
Andrew Mayorov 893e90b372
fix(kafka): use safe publish in consumer 
Routing with v2 schema is actually more strict with respect to input to
`emqx_router` module routines. This causes Kafka consumer bridge to
crash when it tries to publish a message to a topic that looks like a
topic filter.
 2023-11-15 22:37:00 +07:00
Andrew Mayorov 98f947f4f3
ci(router): fix flaky testcase 2023-11-15 22:28:52 +07:00
Andrew Mayorov 8919b08207
fix(utils): rename `emqx_utils_stream:take/2` to `consume/2` 
Which is more neutral and harder to confuse with a destructive `take` in
collections.
 2023-11-15 17:20:40 +07:00
Ivan Dyachkov 65ba381fd8 Merge remote-tracking branch 'upstream/master' into 1115-sync-master-to-r53 2023-11-15 09:25:03 +01:00
Thales Macedo Garitezi f5545bbdf3 fix(schema_types): add missing `secret()` type 2023-11-14 16:08:13 -03:00
Thales Macedo Garitezi 5275a01727 fix(syskeeper): refactor action to new action info scheme 2023-11-14 19:49:28 +01:00
Thales Macedo Garitezi d2cf29fae4 fix(schema): use `hoconsc:union/1` 2023-11-14 19:49:08 +01:00
Thales Macedo Garitezi 423fb32171 test: fix noise about undefined unofficial callbacks 2023-11-14 19:48:39 +01:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Thales Macedo Garitezi 2f1d88d414 fix(bridges_v1): avoid merging action examples for non-v1 bridges 
Since some new bridges might not have a V1 equivalent (i.e. they are not registered in
`emqx_bridge_enterprise`), we should avoid displaying their examples in the V1 API spec.
 2023-11-14 13:56:50 -03:00
Zaiming (Stone) Shi 88637f81d1
Merge pull request #11936 from zmstone/1112-readable-types 
refactor(schema): keep type converters close
 2023-11-14 17:56:07 +01:00
ieQu1 124c0e2dba fix(ds): Respect receive_maximum from the connection info 2023-11-14 16:06:19 +01:00
Ivan Dyachkov bbf00c2f71
Merge pull request #11946 from id/1114-sync-r53 
sync release-53 to master
 2023-11-14 13:38:24 +01:00
Thales Macedo Garitezi 01a2a3b1c0
Merge pull request #11930 from thalesmg/ds-session-discard-m-20231110 
feat(ds): implement session discard
 2023-11-14 09:30:34 -03:00
Thales Macedo Garitezi d6e9bbb95c fix(connector): validate connector name before converting ssl certs 
Fixes https://emqx.atlassian.net/browse/EMQX-11336

See also: https://github.com/emqx/emqx/pull/11540
 2023-11-14 09:29:59 -03:00
Andrew Mayorov 6812ee9d0f
fix(mgmt): hide route selection behind router interface 
Also introduce a generic _stream_ concept, mostly to deal with
iterating over 2 ETS tables at once with `ets:match_object/3`.
 2023-11-14 18:36:25 +07:00
Ivan Dyachkov 28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Ivan Dyachkov 1c57993c91 Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53 2023-11-14 10:44:35 +01:00
Andrew Mayorov d1c3b1c659
fix(bridges): handle secrets more consistently 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-11-14 16:05:54 +07:00
Andrew Mayorov d4dd4508c9
feat(greptimedb): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov 5d620465e2
feat(influx): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00