8914e006c2
refactor(dynamo): move dynamo bridge into its own app
2023-05-10 18:49:21 +08:00
066ed5c6ec
refactor(pgsql): move pgsql && matrix && timescale bridges into their own app
2023-05-10 16:51:33 +08:00
0f979b3d24
refactor(tdengine): move tdengine bridge into its own app
2023-05-10 14:12:57 +08:00
7fa166f034
Merge branch 'release-50' into file-transfer
...
* release-50: (73 commits)
feat: add RabbitMQ bridge
docs: improve rule engine labels and descriptions
chore: bump version && update changes
refactor(rocketmq): move rocketmq bridge into its own app
test: dashboard_listener_test crash
chore: bump chart versions
chore: bump ee version to e5.0.4-alpha.1
test: fix inter-suite flakiness
build: compatibility to make 4.4+
feat: add IotDB bridge
ci: ensure git safe dir in build_packages
ci: ensure git safe dir
test: check_oom's max_mailbox_size
feat: rename max_message_queue_len to max_mailbox_size
fix(buffer_worker): fix inflight count when updating inflight item
chore: prepare for v5.0.25-rc.1 release
docs: add change log entry
fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
chore: `MQTT X` -> `MQTTX`
chore: make sure brod_gssapi app is included in relese package
...
2023-05-09 23:30:22 +05:00
70cf1533db
feat: add RabbitMQ bridge
2023-05-09 14:32:26 +02:00
a119100a0e
refactor(rocketmq): move rocketmq bridge into its own app
2023-05-09 15:57:56 +08:00
270059f0c2
Merge pull request #10642 from thalesmg/fix-flaky-test-rule-engine-stop-v50
...
test: fix inter-suite flakiness
2023-05-08 15:37:26 -03:00
1ade7ce9a3
Merge pull request #10560 from sstrigler/EMQX-8390-5-0-x-io-tdb-bridge-port-conf-and-api-to-5-0
...
IoTDB Bridge
2023-05-08 20:27:01 +02:00
2667f06820
test: fix inter-suite flakiness
2023-05-08 13:33:46 -03:00
d231e708cb
feat: add IotDB bridge
2023-05-08 16:19:01 +02:00
4f396a36a9
Merge remote-tracking branch 'upstream/master' into release-50
2023-05-08 14:58:03 +08:00
d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app
2023-05-06 10:22:05 +08:00
dd3471bc22
Merge branch 'master' into file-transfer
...
* master: (194 commits)
fix(limiter): update change && fix deprecated version
chore: update changes
perf(limiter): simplify the memory represent of limiter configuration
ci(perf test): update tf variable name and set job timeout
ci: fix artifact name in scheduled packages workflow
fix: build_packages_cron.yaml workflow
ci: move scheduled builds to a separate workflow
build: check mnesia compatibility when generating mria config
docs: fix a typo in api doc description
feat(./dev): use command style and added 'ctl' command
test: fix delayed-pubish test case flakyness
refactor: remove raw_with_default config load option
chore: add changelog for trace timestrap
feat: increase the time precision of trace logs to microseconds
chore: make sure topic_metrics/rewrite's default is []
docs: Update changes/ce/perf-10417.en.md
chore: bump `snabbkaffe` to 1.0.8
ci: run static checks in separate jobs
chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
chore: remove unused mqtt cap 'subscription_identifiers'
...
2023-05-05 16:50:18 +03:00
518b6e1211
fix(tdengine): redact the password for the TDEngine example API
2023-05-05 13:51:38 +08:00
a4bb865e6a
feat: add validations for emqx_ee_conf_schema
2023-05-04 16:53:03 +08:00
277deee616
fix: add new sensitive keywords to redact checklist
2023-05-04 16:32:32 +08:00
8aa7c014e7
perf(buffer_worker): avoid calling `ets:info/2`
...
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637
During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.
Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full. Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60 ).
By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
a4bc15cfa1
Merge remote-tracking branch 'origin/master' into 0502-merge-release-50-back-to-master
2023-05-02 18:59:31 +02:00
d5f5f35787
Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master
2023-05-02 18:08:04 +02:00
90cf1ade74
chore: bump application versions
...
* emqx_connector 0.1.22
* emqx_bridge_gcp_pubsub 0.1.1
* emqx_bridge_kafka 0.1.2
* emqx_bridge_pulsar 0.1.1
* emqx_ee_connector 0.1.12
2023-05-02 17:29:22 +03:00
4575167607
feat(resource): drop `manager_id()` type
2023-05-02 17:29:20 +03:00
aaef95b1da
feat(resman): stop adding uniqueness to manager ids
...
Before this change, a separate `manager_id` / `instance_id` was used
as resource manager id, which made connector interface somewhat
inconsistent: part of function calls to connector implementation
used instance id as first argument while the rest used resource id
itself.
2023-05-02 17:28:26 +03:00
57cc854a4a
test(bridge): fix bridge map type filed converters
...
now the converters on map type fields only work at the wrapping map
level but not the values
2023-04-30 10:45:11 +02:00
b58d3e8f94
Merge pull request #10529 from zmstone/0426-ensure-buffer-worker-monitors-cassandra-conn-pid
...
0426 ensure buffer worker monitors cassandra conn pid
2023-04-27 22:54:30 +02:00
dd90b2f498
feat(oracle): Oracle Database integration
2023-04-27 13:07:20 -03:00
567413389c
Merge pull request #10519 from thalesmg/fix-flaky-res-test-v50
...
test(resource): fix flaky test
2023-04-27 09:33:40 -03:00
1f0148a819
Merge pull request #10524 from lafirest/fix/rocketmq_sensitive
...
fix(rocketmq): keep sensitive data safe in rocketmq logs and state
2023-04-27 10:16:05 +08:00
a8b000f062
refactor(sqlserver): support only sync mode at connector level
2023-04-26 20:44:56 +02:00
b76025f541
chore: merge upstream/release-50
2023-04-26 18:01:33 +02:00
c83d630c97
fix(cassandra): ensure async calls return connection pid
...
so the buffer worker can monitor it and perform retries
if the connection restarted
2023-04-26 14:33:37 +02:00
9eccfa5cdf
fix(dynamo): fix test case errors
2023-04-26 17:03:01 +08:00
6706fd90e1
fix(rocketmq): keep sensitive data safe in rocketmq logs and state
2023-04-26 16:10:35 +08:00
e467e082f0
fix(dynamo): remove all async callbacks of the Dynamo connector
2023-04-26 14:47:22 +08:00
bc1bdae55d
chore: reslove confilt for sync release-50 to master
2023-04-26 11:50:44 +08:00
9d893b49eb
Merge branch 'master' into sync-release-50-to-master
2023-04-26 10:54:46 +08:00
79cf5cad19
Merge pull request #10378 from thalesmg/pulsar-producer-e50
...
feat: implement Pulsar Producer bridge (e5.0)
2023-04-25 18:01:43 -03:00
687509886e
test: rm unused var warning
2023-04-25 17:37:20 -03:00
dc452deaf0
Merge pull request #10506 from lafirest/fix/dynamo_default_port
...
fix(dynamo): use correct default port for different schemas
2023-04-25 18:26:44 +08:00
3bb50a5751
fix(rocketmq): fix that the update of ACL info not working
2023-04-25 16:55:03 +08:00
33c27ac2ac
fix(dynamo): use correct default port for different schemas
2023-04-25 10:50:44 +08:00
99f3965f4e
feat(schema_registry): use rocksdb as table type for protobuf cache
2023-04-24 14:24:42 -03:00
377b143325
refactor: split `parse_server` into smaller functions, improve return type to use map
2023-04-24 14:17:29 -03:00
ad4be08bb2
feat: implement Pulsar Producer bridge (e5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-8398
2023-04-24 10:28:26 -03:00
7ce04358c4
fix(Dynamo): fix DynamoDB bridge status check error
2023-04-24 17:28:33 +08:00
2b0106e047
Merge pull request #10485 from JimMoen/fix-mssql-bridge-default
...
fix: use default health check timeout for sqlserver
2023-04-23 18:20:10 +08:00
d505b65ba8
fix: use default health check timeout for sqlserver
2023-04-23 15:47:19 +08:00
4e7472090b
fix: refine default sql and driver name for mssql bridge
2023-04-23 15:47:18 +08:00
5ad5d7ee8d
fix(opents): adjust code structure
2023-04-23 14:28:14 +08:00
6ef032026f
Merge pull request #10471 from lafirest/fix/dynamo_bridge
...
fix(dynamo): separate the implementation of connector and client of Dynamo bridge
2023-04-23 14:25:23 +08:00
932a327952
chore: make spellcheck and xref happy
2023-04-23 11:22:56 +08:00
0b46acda87
test(opents): add test cases for OpenTSDB
2023-04-23 11:03:20 +08:00
d826b0921d
fix(dynamo): separate the implementation of connector and client of Dynamo bridge
2023-04-23 10:57:36 +08:00
5074825075
feat(opents): OpenTSDB integration
2023-04-23 09:56:24 +08:00
ebb75b275e
feat(ft): update app versions
2023-04-21 17:52:13 +03:00
0211bcf030
Merge branch 'master' into file-transfer
...
* master: (279 commits)
chore: shorten ct/run.sh script
chore: rename cassandra_impl to cassandra_connector
chore: fix mix.exs checking
refactor(cassandra): move cassandra bridge into its own app
chore: apply review suggestions
chore: update changes/ce/fix-10449.en.md
test: add a test for authn {}
chore: add changlog for authn_http validation
fix: always check authn_http's header and ssl_option
chore: apply suggestions from code review
fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
ci: simplify find-apps.sh for ee apps
perf(emqx_resource): don't reactivate alarms on reoccurring errors
ci: check if Elixir files are formatted in pre-commit hook
fix(dynamo): fix field name errors
chore: remove *_collector for prometheus api's example
chore: make plugins config to low level
chore: re-split dynamo i18n file
...
2023-04-21 17:37:17 +03:00
b270623c46
chore: rename cassandra_impl to cassandra_connector
2023-04-21 16:49:25 +08:00
7704995279
fix(rocketmq): expose the driver parameter `sync_timeout` into the RocketMQ bridge configuration
2023-04-21 15:10:25 +08:00
5cc28a7b45
chore: fix mix.exs checking
2023-04-21 14:46:09 +08:00
bdce32e713
refactor(cassandra): move cassandra bridge into its own app
2023-04-21 14:45:38 +08:00
6532cfb337
Merge pull request #10452 from thalesmg/refactor-gcp-pubsub-bridge-v50
...
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
2023-04-21 14:42:00 +08:00
48402d0476
Merge pull request #10461 from lafirest/fix/dynamo_db_svrs
...
fix(rocketmq): allow setting multiple addresses in RocketMQ bridge
2023-04-21 14:37:37 +08:00
f602900a53
fix(rocketmq): fix that the status check of RocketMQ bridge may not accurate
2023-04-21 13:35:29 +08:00
d865998a63
fix(rocketmq): fix test cases
2023-04-21 11:02:14 +08:00
21473e7ca5
fix(dynamo): fix field name errors
2023-04-20 18:16:04 +08:00
6e12abff39
fix(rocketmq): allow setting multiple addresses in RocketMQ bridge
2023-04-20 17:58:12 +08:00
415830a0a2
Merge remote-tracking branch 'origin/release-50' into 0420-sync-release-50-to-masteer
2023-04-20 11:03:39 +02:00
7d3367467a
Merge pull request #10408 from kjellwinblad/kjell/rule_engine/add_missing_mongo_date_functions/EMQX-9244
...
feat: add mongo_date functions to the rule engine
2023-04-20 09:46:45 +02:00
a9bd91fcff
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
...
Fixes https://emqx.atlassian.net/browse/EMQX-9536
Note: since GCP PubSub is not shared by any authn/authz backend,
there's no need to separate its connector into another app.
2023-04-19 13:24:32 -03:00
e89f4d4565
fix(dynamo): fix terminology erros
...
- Changed `username` to `aws_access_key_id`
- Changed `password` to `aws_secret_access_key`
2023-04-19 16:26:42 +08:00
bc353b0a06
fix(dynamo): change `database` to `table` in the schema of the DynamoDB bridge
...
there is no term like `database` in DynamoDB, the correct concept should be `table`
2023-04-19 15:56:29 +08:00
9efbb71ae8
Merge pull request #10441 from zmstone/0418-fix-flakyness
...
test: pick random port number for gcp pubsub mock server
2023-04-18 19:11:30 +02:00
dc78ecb41c
chore: merge upstream/master
2023-04-18 17:33:32 +02:00
ad4b7d59eb
Merge pull request #10440 from HJianBo/fix-cassa-flaky-test
...
chore: fix flaky test in emqx_ee_bridge_cassa_SUITE
2023-04-18 22:44:54 +08:00
75813683be
Merge pull request #10371 from ft/EMQX-9257/resource-pool-switch
...
feat(respool): switch to `emqx_resource_pool`
2023-04-18 16:58:17 +03:00
87c5170696
chore: fix flaky test
2023-04-18 21:33:41 +08:00
97b8f00508
test: pick random port number for gcp pubsub mock server
2023-04-18 15:07:57 +02:00
a08c000222
test(mongo): do not rely on internals to clean database
2023-04-18 12:51:17 +03:00
21e19a33ce
feat(respool): switch to `emqx_resource_pool`
...
Which was previously known as `emqx_plugin_libs_pool`. This is part
of the effort to get rid of `emqx_plugin_libs` application.
2023-04-18 12:51:14 +03:00
8a3fccb330
style: fix variable name style
2023-04-18 10:32:37 +02:00
29584ca721
feat: add mongo_date functions to the rule engine
...
This commit adds mong_date built-in functions to the rule engine
SQL-like language. Corresponding functions already existed in EMQX 4.4
and this commit makes sure that EMQX 5.X also has these functions.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9244
2023-04-18 10:01:35 +02:00
cd1197925f
test: fix unit tests after type names changed
2023-04-18 09:48:28 +02:00
9d15247dd5
feat(schema_registry): add support for protobuf schemas
...
Fixes https://emqx.atlassian.net/browse/EMQX-9470
2023-04-17 13:04:54 -03:00
9fc8a498f8
chore: bump apps versions
2023-04-17 09:09:08 +02:00
7df0493312
Merge pull request #10390 from sstrigler/EMQX-9549-new-emqx-utils-app-to-collect-utility-modules
...
New emqx_utils app to collect utility modules
2023-04-14 20:33:11 +02:00
f0c13e0134
fix: stale ref to emqx_map_lib
2023-04-14 18:43:53 +02:00
e073bc90bc
refactor(buffer_worker): rename `s/queue/buffer/g`
2023-04-14 11:37:19 -03:00
edd1bc579f
fix: stale ref to emqx_json after rebase
2023-04-14 16:32:42 +02:00
4f2f0ebb6a
fix: bad ref to emqx_misc after rebase
2023-04-14 14:58:19 +02:00
062ce5f819
refactor: rename emqx_map_lib to emqx_utils_maps
2023-04-14 13:41:34 +02:00
6e8665365b
refactor: rename emqx_tables to emqx_utils_ets
2023-04-14 13:41:33 +02:00
d0df086c80
refactor: rename emqx_api_lib to emqx_utils_api
2023-04-14 13:41:33 +02:00
9c11bfce80
refactor: rename emqx_misc to emqx_utils
2023-04-14 13:41:27 +02:00
f8e9e54393
refactor: move emqx_json to emqx_utils_json
2023-04-14 13:31:27 +02:00
9ce0dbae7b
test: fix batch_size parameter
2023-04-14 13:11:45 +08:00
2484a79c7a
test: create bridge with invalid password
2023-04-14 12:52:08 +08:00
8cec62bd87
fix: async reply arg list
2023-04-14 12:43:36 +08:00
65c4234829
refactor: rename instance_id, which is also PoolName
2023-04-14 10:02:48 +08:00
2d1b94f7f7
fix: file mode 755 magic number
2023-04-14 10:02:48 +08:00
675f9894ab
fix: fix single quotes not applied for single query
...
- alwasy use `emqx_plugin_libs_rule:proc_batch_sql/3` to add single quotes for string
2023-04-14 10:02:47 +08:00
f8c0aa49a2
style: make elvis and shellcheck happy
2023-04-14 10:02:47 +08:00
JimMoen