yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,667 Commits 393 Branches 669 Tags 74 MiB
14077ec43b
Commit Graph

9796 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
f40f6bc5dd refactor: split resource_opts fields between connector and actions 2023-11-20 12:37:36 -03:00
Stefan Strigler
1f1d9e58c6 fix(emqx_connector): don't crash in API on delete with active channels 2023-11-20 16:23:46 +01:00
Thales Macedo Garitezi
53e796bbd0 Merge remote-tracking branch 'origin/release-53' into sync-r53-to-m-20231120 2023-11-20 11:49:52 -03:00
Andrew Mayorov
ef7cfd0202
feat(sessds): add field to ds_pubrange for forward-compat 2023-11-20 19:56:55 +07:00
Thales Macedo Garitezi
b42d4e5ab0
Merge pull request #11950 from thalesmg/fix-bridge-v1-examples-r53-20231114 
fix(bridges_v1): avoid merging action examples for non-v1 bridges
 2023-11-20 09:12:23 -03:00
Thales Macedo Garitezi
926078b82c
Merge pull request #11960 from thalesmg/fix-kafka-producer-channel-status-r53-20231116 
fix(kafka_producer): make status `connecting` while the client fails to connect
 2023-11-20 09:12:05 -03:00
Kjell Winblad
79a764f117 fix: bridge to action upgrade fix up hook should run after upgrade 
This commit changes how the `emqx_action_info` callback
`bridge_v1_to_action_fixup/1` works. It is now called after the
automatic upgrade instead of before. Since the full Bridge V1 config
might be needed to do the fixup, it is provided in a special field
`<<"__bridge_v1_conf__">>`. The `<<"__bridge_v1_conf__">>` field is
removed after the callback is called and can thus be ignored if it is
not needed.
 2023-11-20 12:06:21 +01:00
Stefan Strigler
84ff7b0b38 feat(emqx_bridge): action_info with dynamic lookup 
This allows a n:1 relation between v1 bridge_types to action/connector types as
it's the case with mongodb for instance, where we had `mongodb_single`
`mongodb_sharded` etc and the new implementation will just have `mongodb`.
 2023-11-20 10:22:37 +01:00
Andrew Mayorov
7081f1951f
refactor(sessds): use ds_pubrange record as is 
Instead of converting it into almost similar runtime representation.
 2023-11-20 15:40:43 +07:00
Andrew Mayorov
a5ff4144fe
test(sessds): add complex testcase for session replay 2023-11-20 15:40:43 +07:00
Andrew Mayorov
1246d714c5
feat(sessds): preserve acks / ranges in mnesia for replays 2023-11-20 13:33:48 +07:00
Andrew Mayorov
5b40304d1f
chore(sessds): simplify subscriptions handling 
There's currently no point in storing parsed topic filters in the
subscriptions table.
 2023-11-20 13:33:48 +07:00
Andrew Mayorov
648b6ac63e
chore(sessds): rename iterators -> subscriptions 
Also try to make clearer the difference between 2 flavors of topic
filter representation in use.
 2023-11-20 13:33:48 +07:00
Zaiming (Stone) Shi
e73bf716ae fix(emqx_channel): do not log stale sock_close event as error 
In some cases, EMQX may decide to close socket and mark connection
at 'disconnected' state, for example, when DISCONNECTE packet is
received, or, when failed to write data to socket.
However, by the time EMQX decided to close the socket, the socket
might have already been closed by peer, and the `tcp_closed` envet
is already delivered to the process mailbox -- causing EMQX to
handle sock_close event at 'disconnected' state.
 2023-11-19 22:27:25 +01:00
Ilya Averyanov
e93e9ed108 feat(rebalance): improve rebalance usability 
* make availability API endpoint public
* allow connections during wait_health_check interval
* make availability status calculation more consistent and lightweight
* refactor test to get rid of some mocks and to use cth
 2023-11-18 21:29:16 +03:00
Thales Macedo Garitezi
eb3f54184e refactor: address review comments and avoid transformations without schema knowledge 2023-11-17 16:16:34 -03:00
Kjell Winblad
86c126ffcd feat: callbacks for fixup after automatic Bridge V1 upgrade/downgrade 
This commit adds callbacks to the emqx_action_info module for
doing fixes (such as changing a field name) after the automatic
split of a Bridge V1 config or the merge of connector and action
configs for the compatibility layer.
 2023-11-17 16:10:08 -03:00
Kjell Winblad
9feba802e9 chore: add convenience function for creating action schemas 2023-11-17 16:10:08 -03:00
Ilya Averyanov
58437cd35a fix(mongodb): fix deadlock while stopping mongodb resource 2023-11-17 12:34:36 +03:00
Thales Macedo Garitezi
b92821188b fix(kafka_producer): make status connecting while the client fails to connect 
Fixes https://emqx.atlassian.net/browse/EMQX-11408

To make it consistent with the previous bridge behavior.

Also, introduces macros for resource status to avoid problems with typos.
 2023-11-16 14:50:23 -03:00
Ilya Averyanov
4c54efd8b1 fix(auth): fix deadlock while stopping mongodb resource 2023-11-16 19:29:08 +03:00
Thales Macedo Garitezi
36b5d58957 test: reorganize test suite a bit 2023-11-16 13:16:02 -03:00
Thales Macedo Garitezi
90571b7d8e test: fix noise about undefined unofficial callbacks 2023-11-16 13:16:02 -03:00
Andrew Mayorov
98f947f4f3
ci(router): fix flaky testcase 2023-11-15 22:28:52 +07:00
Ivan Dyachkov
65ba381fd8 Merge remote-tracking branch 'upstream/master' into 1115-sync-master-to-r53 2023-11-15 09:25:03 +01:00
Thales Macedo Garitezi
f5545bbdf3 fix(schema_types): add missing secret() type 2023-11-14 16:08:13 -03:00
Thales Macedo Garitezi
5275a01727 fix(syskeeper): refactor action to new action info scheme 2023-11-14 19:49:28 +01:00
Thales Macedo Garitezi
d2cf29fae4 fix(schema): use hoconsc:union/1 2023-11-14 19:49:08 +01:00
Thales Macedo Garitezi
423fb32171 test: fix noise about undefined unofficial callbacks 2023-11-14 19:48:39 +01:00
Ivan Dyachkov
7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Thales Macedo Garitezi
2f1d88d414 fix(bridges_v1): avoid merging action examples for non-v1 bridges 
Since some new bridges might not have a V1 equivalent (i.e. they are not registered in
`emqx_bridge_enterprise`), we should avoid displaying their examples in the V1 API spec.
 2023-11-14 13:56:50 -03:00
Zaiming (Stone) Shi
88637f81d1
Merge pull request #11936 from zmstone/1112-readable-types 
refactor(schema): keep type converters close
 2023-11-14 17:56:07 +01:00
ieQu1
124c0e2dba fix(ds): Respect receive_maximum from the connection info 2023-11-14 16:06:19 +01:00
Ivan Dyachkov
bbf00c2f71
Merge pull request #11946 from id/1114-sync-r53 
sync release-53 to master
 2023-11-14 13:38:24 +01:00
Thales Macedo Garitezi
01a2a3b1c0
Merge pull request #11930 from thalesmg/ds-session-discard-m-20231110 
feat(ds): implement session discard
 2023-11-14 09:30:34 -03:00
Thales Macedo Garitezi
d6e9bbb95c fix(connector): validate connector name before converting ssl certs 
Fixes https://emqx.atlassian.net/browse/EMQX-11336

See also: https://github.com/emqx/emqx/pull/11540
 2023-11-14 09:29:59 -03:00
Ivan Dyachkov
28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Ivan Dyachkov
1c57993c91 Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53 2023-11-14 10:44:35 +01:00
Andrew Mayorov
d1c3b1c659
fix(bridges): handle secrets more consistently 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-11-14 16:05:54 +07:00
Andrew Mayorov
d4dd4508c9
feat(greptimedb): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov
5d620465e2
feat(influx): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov
7d6c9d2daa
feat(iotdb): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov
7817502b8b
feat(rocketmq): accept wrapped secrets as passwords 2023-11-14 16:05:53 +07:00
Andrew Mayorov
2449d54b1f
feat(pulsar): accept wrapped secrets as passwords 2023-11-14 16:05:52 +07:00
Andrew Mayorov
e2b7b33d14
feat(kinesis): accept wrapped secrets as passwords 2023-11-14 16:05:52 +07:00
Andrew Mayorov
93eaf0caee
feat(dynamo): accept wrapped secrets as passwords 2023-11-14 16:05:52 +07:00
Andrew Mayorov
7250059c60
feat(sqlserver): accept wrapped secrets as passwords 2023-11-14 16:05:52 +07:00
Andrew Mayorov
06861e377f
feat(ldap): accept wrapped secrets as passwords 2023-11-14 16:05:52 +07:00
Andrew Mayorov
fda395014c
test(kafka): verify file-based secrets work 2023-11-14 16:05:25 +07:00
Andrew Mayorov
aa458b65d6
test(kafka): simplify consumer testsuite matrix setup 2023-11-14 16:05:25 +07:00
Andrew Mayorov
18cd98def6
chore(test): fix formatting quirks 2023-11-14 16:05:25 +07:00
Andrew Mayorov
c300eb41a7
feat(kafka): accept wrapped secrets as passwords 2023-11-14 16:05:25 +07:00
Andrew Mayorov
9c5856029f
feat(cassa): accept wrapped secrets as passwords 2023-11-14 16:05:24 +07:00
Andrew Mayorov
8b4ac8eb4f
feat(oracle): accept wrapped secrets as passwords 2023-11-14 16:05:24 +07:00
Andrew Mayorov
34aeeab041
feat(rabbitmq): accept wrapped secrets as passwords 2023-11-14 16:05:24 +07:00
Andrew Mayorov
a69a78d024
feat(clickhouse): accept wrapped secrets as passwords 2023-11-14 16:05:24 +07:00
Andrew Mayorov
fc340a276e
feat(mongo): accept wrapped secrets as passwords 
Also test authorization with mongo in bridge / auth test suites.
 2023-11-14 16:05:24 +07:00
Andrew Mayorov
f827df2821
test(mysql): slightly simplify bridge testsuite 2023-11-14 16:05:23 +07:00
Andrew Mayorov
36e57a479d
feat(mysql): accept wrapped secrets as passwords 2023-11-14 16:05:23 +07:00
Andrew Mayorov
ad72ab7a8c
feat(redis): accept wrapped secrets as passwords 2023-11-14 16:05:23 +07:00
Andrew Mayorov
4385b2f020
feat(pgsql): accept wrapped secrets as passwords 
That are coming from `emqx_schema_secret`. Also adapt pgsql-related
connectors.
 2023-11-14 16:05:23 +07:00
Andrew Mayorov
2b36b48f98
feat(connector): use emqx_schema_secret as password field schema 2023-11-14 16:05:23 +07:00
Ivan Dyachkov
a49aea3b56 chore: bump app versions 2023-11-14 09:27:04 +01:00
Kjell Winblad
d682e6e23c test: fix test mock by calling exported function 2023-11-14 09:20:46 +01:00
Kjell Winblad
3bea3496af test: fix test case mock that broke due to removal 2023-11-14 09:20:46 +01:00
Kjell Winblad
093c8b0c6e docs: add missing copyright headers 2023-11-14 09:20:46 +01:00
Kjell Winblad
ca3e5eab11 refactor: remove emqx_bridge_v2_enterprise.erl 2023-11-14 09:20:46 +01:00
Kjell Winblad
e93b71d8d5 fix: problems found by @thalesmg in code review 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-11-14 09:20:46 +01:00
Kjell Winblad
49fdfef8c3 fix: azure event hub names are defined in one place 2023-11-14 09:20:46 +01:00
Kjell Winblad
5e8e407017 refactor: action schema retrival after PR feedback 2023-11-14 09:20:46 +01:00
Kjell Winblad
ab078647a5 chore: fix formatting problem 2023-11-14 09:20:46 +01:00
Kjell Winblad
bdab421885 fix(bridge_v2): name and copyright header 2023-11-14 09:20:46 +01:00
Kjell Winblad
ca2cdbc08d fix: bapi static check error 2023-11-14 09:20:46 +01:00
Kjell Winblad
7839f0cbc8 chore(bridge_v2): make dialyzer and bpapi happy 2023-11-14 09:20:46 +01:00
Kjell Winblad
d26a1b9afb fix(bridge_v1): no hard coded downgrade and upgrade type functions 2023-11-14 09:20:46 +01:00
Kjell Winblad
a6aa81b548 fix(emqx_bridge_v2): dialyzer found bad format parameter error 2023-11-14 09:20:46 +01:00
Kjell Winblad
3c778121a5 fix: bridge V1 type lookup issue 2023-11-14 09:20:46 +01:00
Kjell Winblad
77aaff137a test(emqx_bridge_v2_SUITE): fix incorrect assumption about return type 
This commit fixes an incorrect assumption about the return type of
emqx_bridge_v2:lookup/2 in emqx_bridge_v2_SUITE:t_load_config_success/1.
 2023-11-14 09:20:46 +01:00
Kjell Winblad
14e305e2a6 fix(emqx_bridge_v2): xref warning 2023-11-14 09:20:46 +01:00
Kjell Winblad
9eaee8f333 refactor(emqx_bridge_v2): make independent of Kafka 
This removes the Kafka specific knowledge from emqx_bridge_v2 and
makes it possible to add new Bridge V2 bridges without modifying
the emqx_bridge application.
 2023-11-14 09:20:46 +01:00
Kjell Winblad
cd5b1f9b96 docs(bridge_V2): type specs for operations 2023-11-14 09:20:46 +01:00
Kjell Winblad
99031f0dae refactor(bridge_v2): prefix compatibility functions with bridge_v1 2023-11-14 09:20:46 +01:00
Kjell Winblad
729e7df0d5 docs(bridge_v2): add specs for CRUD functions 2023-11-14 09:20:46 +01:00
Ivan Dyachkov
a9cf8fe78c chore: 5.3.1 2023-11-14 08:44:41 +01:00
Zaiming (Stone) Shi
ab1f3b6b2f fix(emqx_conf_schema_types): address review comments 2023-11-13 21:55:03 +01:00
Stefan Strigler
fc2891d6de
Merge pull request #11937 from sstrigler/EMQX-11318-crash-in-get-bridges-v-2-if-a-broken-bridge-is-configured 
Emqx 11318 crash in get bridges v 2 if a broken bridge is configured
 2023-11-13 17:28:56 +01:00
Zaiming (Stone) Shi
4c5d64abc2 refactor(schema): keep type converters close 2023-11-13 15:26:30 +01:00
Stefan Strigler
17b18849de test(emqx_bridge): test broken config 2023-11-13 14:52:06 +01:00
Zaiming (Stone) Shi
9b9908780c
Merge pull request #11926 from kjellwinblad/kjell/fix_bridge_v2_to_action_cluster_hocon/EMQX-11353 
fix: reflect bridge V1 upgreade in cluster.hocon
 2023-11-13 14:18:57 +01:00
Stefan Strigler
0b3645057d fix(emqx_bridge): don't try to handle what we don't understand 2023-11-13 14:08:54 +01:00
Stefan Strigler
0d33284672 fix(emqx_bridge): don't crash uninstallling broken action 2023-11-13 14:08:54 +01:00
Stefan Strigler
4fe1d332b3 fix(emqx_bridge): don't crash if there's no status field 2023-11-13 14:08:54 +01:00
Thales Macedo Garitezi
45dad2ed3a feat(ds): implement session discard 
Fixes https://emqx.atlassian.net/browse/EMQX-9739

Fixes some issues to ensure the session is discarded when the client connects with
`clean_start = true`, and added some cleanup to subscriptions/routes/iterators/streams.

> There is an API that session garbage collector can use to perform cleaning

We already have `emqx_session:destroy/1`, which could serve as an API for a periodic
session GC to use.
 2023-11-13 09:06:22 -03:00
Kjell Winblad
26fc9ed89d
fix: log warning message after suggestion from @zmstone 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-11-13 12:53:09 +01:00
Kjell Winblad
5838568d5d fix: hard coded call to schema module 
The user can configure the configuration loader and schema module so we
should not hard code a call to the default schema module.
 2023-11-13 11:41:40 +01:00
Zaiming (Stone) Shi
518b02fc70
Merge pull request #11931 from zmstone/1110-delete-stale-type-converters 
1110 delete stale type converters
 2023-11-13 10:27:51 +01:00
Zaiming (Stone) Shi
65efa2672e test(schema): delete stale reference to emqx_schema:file() type 2023-11-13 09:28:26 +01:00
Zaiming (Stone) Shi
2fda91ec59 refactor: delete default type converter for dashboard schema json 
prior to this commit, dashboard scheam json type conversion
has a 'fallback' type for all unknown types, which is 'string'

this commit removes this fallback and add all possible types
to the function clauses.

also, some of the old types which are no longer in use
are deleted from both swagger spec converter and dashboard
sechema converter
 2023-11-12 20:08:53 +01:00
JimMoen
952ab08991
fix: make dialyzer happy 2023-11-12 21:44:24 +08:00
JimMoen
20bf9ed26d
fix(gw_gbt32960): mountpoint should not with / suffix 2023-11-12 21:44:23 +08:00