refactor(sqlserver): move sqlserver bridge into its own app
This commit is contained in:
JimMoen
parent
fc62cfdfd8
commit
d5dce771bd
2
APL.txt
2
APL.txt
|
|
@ -186,7 +186,7 @@
|
|||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright {yyyy} {name of copyright owner}
|
||||
Copyright (c) 2016-2023 EMQ Technologies Co., Ltd.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
|
|
|
|||
|
|
@ -12,14 +12,11 @@ not used by authentication and authorization applications.
|
|||
|
||||
# Documentation links
|
||||
|
||||
For more information on Apache Kafka, please see its [official
|
||||
site](https://kafka.apache.org/).
|
||||
For more information about Apache Kafka, please see its [official site](https://kafka.apache.org/).
|
||||
|
||||
# Configurations
|
||||
|
||||
Please see [our official
|
||||
documentation](https://www.emqx.io/docs/en/v5.0/data-integration/data-bridge-kafka.html)
|
||||
for more detailed info.
|
||||
Please see [our official documentation](https://www.emqx.io/docs/en/v5.0/data-integration/data-bridge-kafka.html) for more detailed info.
|
||||
|
||||
# Contributing
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,94 @@
|
|||
Business Source License 1.1
|
||||
|
||||
Licensor: Hangzhou EMQ Technologies Co., Ltd.
|
||||
Licensed Work: EMQX Enterprise Edition
|
||||
The Licensed Work is (c) 2023
|
||||
Hangzhou EMQ Technologies Co., Ltd.
|
||||
Additional Use Grant: Students and educators are granted right to copy,
|
||||
modify, and create derivative work for research
|
||||
or education.
|
||||
Change Date: 2027-02-01
|
||||
Change License: Apache License, Version 2.0
|
||||
|
||||
For information about alternative licensing arrangements for the Software,
|
||||
please contact Licensor: https://www.emqx.com/en/contact
|
||||
|
||||
Notice
|
||||
|
||||
The Business Source License (this document, or the “License”) is not an Open
|
||||
Source license. However, the Licensed Work will eventually be made available
|
||||
under an Open Source License, as stated in this License.
|
||||
|
||||
License text copyright (c) 2017 MariaDB Corporation Ab, All Rights Reserved.
|
||||
“Business Source License” is a trademark of MariaDB Corporation Ab.
|
||||
|
||||
-----------------------------------------------------------------------------
|
||||
|
||||
Business Source License 1.1
|
||||
|
||||
Terms
|
||||
|
||||
The Licensor hereby grants you the right to copy, modify, create derivative
|
||||
works, redistribute, and make non-production use of the Licensed Work. The
|
||||
Licensor may make an Additional Use Grant, above, permitting limited
|
||||
production use.
|
||||
|
||||
Effective on the Change Date, or the fourth anniversary of the first publicly
|
||||
available distribution of a specific version of the Licensed Work under this
|
||||
License, whichever comes first, the Licensor hereby grants you rights under
|
||||
the terms of the Change License, and the rights granted in the paragraph
|
||||
above terminate.
|
||||
|
||||
If your use of the Licensed Work does not comply with the requirements
|
||||
currently in effect as described in this License, you must purchase a
|
||||
commercial license from the Licensor, its affiliated entities, or authorized
|
||||
resellers, or you must refrain from using the Licensed Work.
|
||||
|
||||
All copies of the original and modified Licensed Work, and derivative works
|
||||
of the Licensed Work, are subject to this License. This License applies
|
||||
separately for each version of the Licensed Work and the Change Date may vary
|
||||
for each version of the Licensed Work released by Licensor.
|
||||
|
||||
You must conspicuously display this License on each original or modified copy
|
||||
of the Licensed Work. If you receive the Licensed Work in original or
|
||||
modified form from a third party, the terms and conditions set forth in this
|
||||
License apply to your use of that work.
|
||||
|
||||
Any use of the Licensed Work in violation of this License will automatically
|
||||
terminate your rights under this License for the current and all other
|
||||
versions of the Licensed Work.
|
||||
|
||||
This License does not grant you any right in any trademark or logo of
|
||||
Licensor or its affiliates (provided that you may use a trademark or logo of
|
||||
Licensor as expressly required by this License).
|
||||
|
||||
TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON
|
||||
AN “AS IS” BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS,
|
||||
EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND
|
||||
TITLE.
|
||||
|
||||
MariaDB hereby grants you permission to use this License’s text to license
|
||||
your works, and to refer to it using the trademark “Business Source License”,
|
||||
as long as you comply with the Covenants of Licensor below.
|
||||
|
||||
Covenants of Licensor
|
||||
|
||||
In consideration of the right to use this License’s text and the “Business
|
||||
Source License” name and trademark, Licensor covenants to MariaDB, and to all
|
||||
other recipients of the licensed work to be provided by Licensor:
|
||||
|
||||
1. To specify as the Change License the GPL Version 2.0 or any later version,
|
||||
or a license that is compatible with GPL Version 2.0 or a later version,
|
||||
where “compatible” means that software provided under the Change License can
|
||||
be included in a program with software provided under GPL Version 2.0 or a
|
||||
later version. Licensor may specify additional Change Licenses without
|
||||
limitation.
|
||||
|
||||
2. To either: (a) specify an additional grant of rights to use that does not
|
||||
impose any additional restriction on the right granted in this License, as
|
||||
the Additional Use Grant; or (b) insert the text “None”.
|
||||
|
||||
3. To specify a Change Date.
|
||||
|
||||
4. Not to modify this License in any other way.
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
# EMQX SQL Server Bridge
|
||||
|
||||
[Microsoft SQL Server](https://www.microsoft.com/en-us/sql-server) is a relational database management system (RDBMS) that is developed and owned by Microsoft.
|
||||
Microsoft SQL Server offers a wide range of features, including support for high availability and disaster recovery,
|
||||
integration with other Microsoft products and services, and advanced security and encryption options.
|
||||
It also provides tools for data warehousing, business intelligence, and analytics, making it a versatile and powerful database platform.
|
||||
|
||||
The application is used to connect EMQX and Microsoft SQL Server.
|
||||
User can create a rule and easily ingest IoT data into Microsoft SQL Server by leveraging
|
||||
[EMQX Rules](https://docs.emqx.com/en/enterprise/v5.0/data-integration/rules.html).
|
||||
|
||||
<!---
|
||||
|
||||
# Documentation
|
||||
|
||||
- Refer to [Ingest data into Cassandra](https://docs.emqx.com/en/enterprise/v5.0/data-integration/data-bridge-cassa.html)
|
||||
for how to use EMQX dashboard to ingest IoT data into Cassandra.
|
||||
- Refer to [EMQX Rules](https://docs.emqx.com/en/enterprise/v5.0/data-integration/rules.html)
|
||||
for the EMQX rules engine introduction.
|
||||
|
||||
--->
|
||||
|
||||
# Documentation links
|
||||
|
||||
For more information about Microsoft SQL Server, please see the [official site](https://learn.microsoft.com/en-us/sql/sql-server/?view=sql-server-ver16)
|
||||
|
||||
# Configurations
|
||||
|
||||
|
||||
Please see [our official documentation](https://www.emqx.io/docs/en/v5.0/data-integration/data-bridge-sqlserver.html) for more detailed information.
|
||||
|
||||
# HTTP APIs
|
||||
|
||||
- Several APIs are provided for bridge management, which includes create bridge,
|
||||
update bridge, get bridge, stop or restart bridge and list bridges etc.
|
||||
|
||||
Refer to [API Docs - Bridges](https://docs.emqx.com/en/enterprise/v5.0/admin/api-docs.html#tag/Bridges) for more detailed information.
|
||||
|
||||
|
||||
# Contributing
|
||||
|
||||
Please see our [contributing.md](../../CONTRIBUTING.md).
|
||||
|
||||
|
||||
# License
|
||||
|
||||
EMQ Business Source License 1.1, refer to [LICENSE](BSL.txt).
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
toxiproxy
|
||||
sqlserver
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
%%--------------------------------------------------------------------
|
||||
%% Copyright (c) 2023 EMQ Technologies Co., Ltd. All Rights Reserved.
|
||||
%%--------------------------------------------------------------------
|
||||
|
||||
-define(SQLSERVER_DEFAULT_PORT, 1433).
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
%% -*- mode: erlang; -*-
|
||||
{erl_opts, [debug_info]}.
|
||||
{deps, [ {emqx_connector, {path, "../../apps/emqx_connector"}}
|
||||
, {emqx_resource, {path, "../../apps/emqx_resource"}}
|
||||
, {emqx_bridge, {path, "../../apps/emqx_bridge"}}
|
||||
]}.
|
||||
|
||||
{shell, [
|
||||
{apps, [emqx_bridge_sqlserver]}
|
||||
]}.
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
{application, emqx_bridge_sqlserver, [
|
||||
{description, "EMQX Enterprise SQL Server Bridge"},
|
||||
{vsn, "0.1.0"},
|
||||
{registered, []},
|
||||
{applications, [kernel, stdlib, odbc]},
|
||||
{env, []},
|
||||
{modules, []},
|
||||
{links, []}
|
||||
]}.
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
%%--------------------------------------------------------------------
|
||||
%% Copyright (c) 2022-2023 EMQ Technologies Co., Ltd. All Rights Reserved.
|
||||
%%--------------------------------------------------------------------
|
||||
-module(emqx_ee_bridge_sqlserver).
|
||||
-module(emqx_bridge_sqlserver).
|
||||
|
||||
-include_lib("typerefl/include/types.hrl").
|
||||
-include_lib("hocon/include/hoconsc.hrl").
|
||||
|
|
@ -96,7 +96,7 @@ fields("config") ->
|
|||
}
|
||||
)}
|
||||
] ++
|
||||
(emqx_ee_connector_sqlserver:fields(config) --
|
||||
(emqx_bridge_sqlserver_connector:fields(config) --
|
||||
emqx_connector_schema_lib:prepare_statement_fields());
|
||||
fields("creation_opts") ->
|
||||
emqx_resource_schema:fields("creation_opts");
|
||||
|
|
@ -2,14 +2,15 @@
|
|||
%% Copyright (c) 2023 EMQ Technologies Co., Ltd. All Rights Reserved.
|
||||
%%--------------------------------------------------------------------
|
||||
|
||||
-module(emqx_ee_connector_sqlserver).
|
||||
-module(emqx_bridge_sqlserver_connector).
|
||||
|
||||
-behaviour(emqx_resource).
|
||||
|
||||
-include("emqx_bridge_sqlserver.hrl").
|
||||
|
||||
-include_lib("kernel/include/file.hrl").
|
||||
-include_lib("emqx/include/logger.hrl").
|
||||
-include_lib("emqx_resource/include/emqx_resource.hrl").
|
||||
-include_lib("emqx_ee_connector/include/emqx_ee_connector.hrl").
|
||||
|
||||
-include_lib("typerefl/include/types.hrl").
|
||||
-include_lib("hocon/include/hoconsc.hrl").
|
||||
|
|
@ -51,7 +52,7 @@
|
|||
-define(SYNC_QUERY_MODE, handover).
|
||||
|
||||
-define(SQLSERVER_HOST_OPTIONS, #{
|
||||
default_port => 1433
|
||||
default_port => ?SQLSERVER_DEFAULT_PORT
|
||||
}).
|
||||
|
||||
-define(REQUEST_TIMEOUT(RESOURCE_OPTS),
|
||||
|
|
@ -2,11 +2,12 @@
|
|||
% Copyright (c) 2022-2023 EMQ Technologies Co., Ltd. All Rights Reserved.
|
||||
%%--------------------------------------------------------------------
|
||||
|
||||
-module(emqx_ee_bridge_sqlserver_SUITE).
|
||||
-module(emqx_bridge_sqlserver_SUITE).
|
||||
|
||||
-compile(nowarn_export_all).
|
||||
-compile(export_all).
|
||||
|
||||
-include("emqx_bridge_sqlserver/include/emqx_bridge_sqlserver.hrl").
|
||||
-include_lib("eunit/include/eunit.hrl").
|
||||
-include_lib("common_test/include/ct.hrl").
|
||||
-include_lib("snabbkaffe/include/snabbkaffe.hrl").
|
||||
|
|
@ -59,24 +60,30 @@
|
|||
%% How to run it locally (all commands are run in $PROJ_ROOT dir):
|
||||
%% A: run ct on host
|
||||
%% 1. Start all deps services
|
||||
%% ```bash
|
||||
%% sudo docker compose -f .ci/docker-compose-file/docker-compose.yaml \
|
||||
%% -f .ci/docker-compose-file/docker-compose-sqlserver.yaml \
|
||||
%% -f .ci/docker-compose-file/docker-compose-toxiproxy.yaml \
|
||||
%% up --build
|
||||
%% ```
|
||||
%%
|
||||
%% 2. Run use cases with special environment variables
|
||||
%% 11433 is toxiproxy exported port.
|
||||
%% Local:
|
||||
%% ```
|
||||
%% ```bash
|
||||
%% SQLSERVER_HOST=toxiproxy SQLSERVER_PORT=11433 \
|
||||
%% PROXY_HOST=toxiproxy PROXY_PORT=1433 \
|
||||
%% ./rebar3 as test ct -c -v --readable true --name ct@127.0.0.1 --suite lib-ee/emqx_ee_bridge/test/emqx_ee_bridge_sqlserver_SUITE.erl
|
||||
%% ./rebar3 as test ct -c -v --readable true --name ct@127.0.0.1 \
|
||||
%% --suite apps/emqx_bridge_sqlserver/test/emqx_bridge_sqlserver_SUITE.erl
|
||||
%% ```
|
||||
%%
|
||||
%% B: run ct in docker container
|
||||
%% run script:
|
||||
%% ./scripts/ct/run.sh --ci --app lib-ee/emqx_ee_bridge/ \
|
||||
%% -- --name 'test@127.0.0.1' -c -v --readable true --suite lib-ee/emqx_ee_bridge/test/emqx_ee_bridge_sqlserver_SUITE.erl
|
||||
%% ```bash
|
||||
%% ./scripts/ct/run.sh --ci --app apps/emqx_bridge_sqlserver/ -- \
|
||||
%% --name 'test@127.0.0.1' -c -v --readable true \
|
||||
%% --suite apps/emqx_bridge_sqlserver/test/emqx_bridge_sqlserver_SUITE.erl
|
||||
%% ````
|
||||
|
||||
%%------------------------------------------------------------------------------
|
||||
%% CT boilerplate
|
||||
|
|
@ -391,7 +398,7 @@ t_bad_parameter(Config) ->
|
|||
|
||||
common_init(ConfigT) ->
|
||||
Host = os:getenv("SQLSERVER_HOST", "toxiproxy"),
|
||||
Port = list_to_integer(os:getenv("SQLSERVER_PORT", "1433")),
|
||||
Port = list_to_integer(os:getenv("SQLSERVER_PORT", str(?SQLSERVER_DEFAULT_PORT))),
|
||||
|
||||
Config0 = [
|
||||
{sqlserver_host, Host},
|
||||
|
|
@ -631,7 +638,7 @@ conn_str([], Acc) ->
|
|||
conn_str([{driver, Driver} | Opts], Acc) ->
|
||||
conn_str(Opts, ["Driver=" ++ str(Driver) | Acc]);
|
||||
conn_str([{host, Host} | Opts], Acc) ->
|
||||
Port = proplists:get_value(port, Opts, "1433"),
|
||||
Port = proplists:get_value(port, Opts, str(?SQLSERVER_DEFAULT_PORT)),
|
||||
NOpts = proplists:delete(port, Opts),
|
||||
conn_str(NOpts, ["Server=" ++ str(Host) ++ "," ++ str(Port) | Acc]);
|
||||
conn_str([{port, Port} | Opts], Acc) ->
|
||||
|
|
@ -10,4 +10,3 @@ tdengine
|
|||
clickhouse
|
||||
dynamo
|
||||
rocketmq
|
||||
sqlserver
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
%% -*- mode: erlang; -*-
|
||||
{erl_opts, [debug_info]}.
|
||||
{deps, [ {emqx_connector, {path, "../../apps/emqx_connector"}}
|
||||
, {emqx_resource, {path, "../../apps/emqx_resource"}}
|
||||
|
|
|
|||
|
|
@ -11,7 +11,8 @@
|
|||
emqx_bridge_gcp_pubsub,
|
||||
emqx_bridge_cassandra,
|
||||
emqx_bridge_opents,
|
||||
emqx_bridge_pulsar
|
||||
emqx_bridge_pulsar,
|
||||
emqx_bridge_sqlserver
|
||||
]},
|
||||
{env, []},
|
||||
{modules, []},
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ api_schemas(Method) ->
|
|||
ref(emqx_ee_bridge_clickhouse, Method),
|
||||
ref(emqx_ee_bridge_dynamo, Method),
|
||||
ref(emqx_ee_bridge_rocketmq, Method),
|
||||
ref(emqx_ee_bridge_sqlserver, Method),
|
||||
ref(emqx_bridge_sqlserver, Method),
|
||||
ref(emqx_bridge_opents, Method),
|
||||
ref(emqx_bridge_pulsar, Method ++ "_producer"),
|
||||
ref(emqx_bridge_oracle, Method)
|
||||
|
|
@ -58,7 +58,7 @@ schema_modules() ->
|
|||
emqx_ee_bridge_clickhouse,
|
||||
emqx_ee_bridge_dynamo,
|
||||
emqx_ee_bridge_rocketmq,
|
||||
emqx_ee_bridge_sqlserver,
|
||||
emqx_bridge_sqlserver,
|
||||
emqx_bridge_opents,
|
||||
emqx_bridge_pulsar,
|
||||
emqx_bridge_oracle
|
||||
|
|
@ -100,7 +100,7 @@ resource_type(tdengine) -> emqx_ee_connector_tdengine;
|
|||
resource_type(clickhouse) -> emqx_ee_connector_clickhouse;
|
||||
resource_type(dynamo) -> emqx_ee_connector_dynamo;
|
||||
resource_type(rocketmq) -> emqx_ee_connector_rocketmq;
|
||||
resource_type(sqlserver) -> emqx_ee_connector_sqlserver;
|
||||
resource_type(sqlserver) -> emqx_bridge_sqlserver_connector;
|
||||
resource_type(opents) -> emqx_bridge_opents_connector;
|
||||
resource_type(pulsar_producer) -> emqx_bridge_pulsar_impl_producer;
|
||||
resource_type(oracle) -> emqx_oracle.
|
||||
|
|
@ -295,7 +295,7 @@ sqlserver_structs() ->
|
|||
[
|
||||
{sqlserver,
|
||||
mk(
|
||||
hoconsc:map(name, ref(emqx_ee_bridge_sqlserver, "config")),
|
||||
hoconsc:map(name, ref(emqx_bridge_sqlserver, "config")),
|
||||
#{
|
||||
desc => <<"Microsoft SQL Server Bridge Config">>,
|
||||
required => false
|
||||
|
|
|
|||
|
|
@ -1,4 +1,3 @@
|
|||
toxiproxy
|
||||
influxdb
|
||||
clickhouse
|
||||
sqlserver
|
||||
|
|
|
|||
|
|
@ -11,8 +11,7 @@
|
|||
tdengine,
|
||||
clickhouse,
|
||||
erlcloud,
|
||||
rocketmq,
|
||||
odbc
|
||||
rocketmq
|
||||
]},
|
||||
{env, []},
|
||||
{modules, []},
|
||||
|
|
|
|||
2
mix.exs
2
mix.exs
|
|
@ -170,6 +170,7 @@ defmodule EMQXUmbrella.MixProject do
|
|||
:emqx_bridge_rocketmq,
|
||||
:emqx_bridge_tdengine,
|
||||
:emqx_bridge_timescale,
|
||||
:emqx_bridge_sqlserver,
|
||||
:emqx_bridge_pulsar,
|
||||
:emqx_oracle,
|
||||
:emqx_bridge_oracle
|
||||
|
|
@ -379,6 +380,7 @@ defmodule EMQXUmbrella.MixProject do
|
|||
emqx_bridge_rocketmq: :permanent,
|
||||
emqx_bridge_tdengine: :permanent,
|
||||
emqx_bridge_timescale: :permanent,
|
||||
emqx_bridge_sqlserver: :permanent,
|
||||
emqx_oracle: :permanent,
|
||||
emqx_bridge_oracle: :permanent,
|
||||
emqx_ee_schema_registry: :permanent
|
||||
|
|
|
|||
|
|
@ -95,6 +95,7 @@ is_community_umbrella_app("apps/emqx_bridge_rocketmq") -> false;
|
|||
is_community_umbrella_app("apps/emqx_bridge_tdengine") -> false;
|
||||
is_community_umbrella_app("apps/emqx_bridge_timescale") -> false;
|
||||
is_community_umbrella_app("apps/emqx_bridge_oracle") -> false;
|
||||
is_community_umbrella_app("apps/emqx_bridge_sqlserver") -> false;
|
||||
is_community_umbrella_app("apps/emqx_oracle") -> false;
|
||||
is_community_umbrella_app(_) -> true.
|
||||
|
||||
|
|
@ -470,6 +471,7 @@ relx_apps_per_edition(ee) ->
|
|||
emqx_bridge_rocketmq,
|
||||
emqx_bridge_tdengine,
|
||||
emqx_bridge_timescale,
|
||||
emqx_bridge_sqlserver,
|
||||
emqx_oracle,
|
||||
emqx_bridge_oracle,
|
||||
emqx_ee_schema_registry
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
emqx_ee_bridge_sqlserver {
|
||||
emqx_bridge_sqlserver {
|
||||
|
||||
config_enable.desc:
|
||||
"""Enable or disable this bridge"""
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
emqx_ee_connector_sqlserver {
|
||||
emqx_bridge_sqlserver_connector {
|
||||
|
||||
server.desc:
|
||||
"""The IPv4 or IPv6 address or the hostname to connect to.<br/>
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
emqx_ee_bridge_sqlserver {
|
||||
emqx_bridge_sqlserver {
|
||||
|
||||
config_enable.desc:
|
||||
"""启用/禁用桥接"""
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
emqx_ee_connector_sqlserver {
|
||||
emqx_bridge_sqlserver_connector {
|
||||
|
||||
server.desc:
|
||||
"""将要连接的 IPv4 或 IPv6 地址，或者主机名。<br/>
|
||||
|
|
@ -193,7 +193,7 @@ for dep in ${CT_DEPS}; do
|
|||
;;
|
||||
pulsar)
|
||||
FILES+=( '.ci/docker-compose-file/docker-compose-pulsar.yaml' )
|
||||
;;
|
||||
;;
|
||||
oracle)
|
||||
FILES+=( '.ci/docker-compose-file/docker-compose-oracle.yaml' )
|
||||
;;
|
||||
|
|
|
|||
Loading…
Reference in New Issue