966276127e
test: trying to make tests more stable
2023-03-16 13:43:01 -03:00
0587e0d3f3
refactor(kafka_consumer): change the generated consumer group id
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
41b8d47696
test(kafka_consumer): add more clusterized tests
2023-03-16 13:43:01 -03:00
19f9c35662
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
fc5dfa108a
fix(kafka_consumer): rename `force_utf8` to `none`
...
We actually enforce the key/value to be a valid UTF-8 string when
using `emqx_json:encode`, if we do encode using that, which is
template-dependent.
2023-03-16 13:43:01 -03:00
37a35b22df
docs(kafka): improve kafka consumer/producer API examples
2023-03-16 13:43:01 -03:00
53979b6261
feat(kafka_consumer): support multiple topic mappings, payload
...
templates and key/value encodings
Added after feedback from the product team.
2023-03-16 13:43:01 -03:00
f31f15e44e
chore(kafka_producer): make schema changes more backwards compatible
...
This will still require fixes to the frontend.
2023-03-16 13:43:01 -03:00
c182a4053e
fix(kafka_consumer): avoid leaking atoms in bridge probe API
2023-03-16 13:43:01 -03:00
27dfd98f46
fix(kafka): fix api methods schemas and bridge type enum
2023-03-16 13:43:01 -03:00
fecc08bb8f
refactor(kafka_consumer): apply review suggestions
2023-03-16 13:43:01 -03:00
2a3aef92bb
refactor(kafka_consumer): redact error info
2023-03-16 13:43:01 -03:00
9037314aea
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
1b78f22d92
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
e1fdd041b3
feat(kafka_consumer): add `offset_commit_interval_seconds` kafka parameter
2023-03-16 13:43:01 -03:00
1d5fe14a30
test: remove sleeps
2023-03-16 13:43:01 -03:00
65c15b3fae
refactor(kafka_consumer): move subscriber startup logic to separate fn
2023-03-16 13:43:01 -03:00
969fa03ecc
feat: implement kafka consumer
2023-03-16 13:43:01 -03:00
5ab5236ad3
test: fix flaky test
2023-03-16 13:42:38 -03:00
a530ccbe3d
Merge pull request #10095 from fix/EEC-782/mysql-prepstmt-exhaustion
...
fix(mysql): be explicit that batch queries are parameterless
2023-03-14 10:21:45 +03:00
e3595f2e79
chore(mria): Bump version to 0.4.0
2023-03-11 00:37:25 +01:00
f7c0d29478
test(tde): add testcase for a nasty string in SQL query
...
Similar to what we have in mysql and pgqsl testsuites.
2023-03-10 18:43:30 +03:00
0a7f6c7d03
fix(mysql): ensure proper escaping in batch inserts
...
Also hexencode non-utf8 binaries. This is essentially an heuristic.
We don't know column types in runtime, and there's no simple way
to find them out. Since we're already doing full binary scan during
escaping it should be cheap to bail out on non-utf8 strings and
hexencode them instead.
Also introduce separate function to highlight that this escaping
is MySQL-specific.
2023-03-10 18:42:04 +03:00
fc37d9b3cd
fix(mysql): be explicit that batch queries are parameterless
...
So that mysql client won't attempt to prepare them automatically, thus
trashing the server's prepared statements table and making interaction
overall heavier.
2023-03-10 18:42:04 +03:00
a86d06f043
chore: bump app versions following last merge-back
2023-03-10 16:44:15 +03:00
fe27604010
Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master
2023-03-08 16:46:45 +01:00
97e71c54d4
fix: use default template if timestamp is empty (undefined) in InfluxDB bridge
...
Closes EMQX-8926
2023-03-08 11:58:23 +01:00
984dd3446d
test(bridges): add test suite for DynamoDB
2023-03-08 11:13:51 +08:00
bfc2920a55
feat(bridges): introduce DynamoDB definition files from v4
2023-03-08 11:13:51 +08:00
b8b1a78025
feat(bridges): integrate the DynamoDB into data bridges
2023-03-08 11:13:38 +08:00
18ab7ed197
chore: bump app vsns
2023-03-06 15:31:28 -03:00
e9d3fc511f
chore(buffer_worker): change default `batch_time` to 0 and improve docs
2023-03-06 15:31:28 -03:00
9825998207
chore(gcp_pubsub): just deprecate `request_timeout` instead of removing
2023-03-06 10:12:38 -03:00
9b087a21f5
fix(gcp_pubsub): remove conflicting `request_timeout` option
...
Since the buffer worker schema already contains that configuration,
having it two places can lead to quite confusing behavior.
2023-03-06 10:12:38 -03:00
c01f62a1c1
chore: bump apps versions
2023-03-03 13:58:32 +01:00
c24a84e370
docs: fix spelling
2023-03-03 12:06:18 +01:00
67acdf0888
feat: add clickhouse database bridge
...
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
2023-03-02 12:22:11 +01:00
083330ad80
Merge remote-tracking branch 'origin/master' into 0301-merge-release-50-to-master
2023-03-01 08:53:03 +01:00
41cb7263ca
docs(influxdb): add a node to the write_syntax config
2023-02-28 21:53:32 +01:00
24f476e35f
test: add README to influxdb test script
2023-02-28 19:38:43 +01:00
6ce5029d79
Merge pull request #9881 from olcai/log-influxdb-is-alive-reason
...
fix(emqx_ee_connector): log reason for failure when starting influxdb connector
2023-02-28 09:49:08 +00:00
79bf77c2f1
Merge pull request #10020 from zmstone/0222-fix-bridge-async-mode-counters
...
fix(bridge): fix dropped counter and inflight gauge
2023-02-24 19:51:24 +01:00
9cbe64a132
fix(test): make strings json-friendly in kafka testsuite
2023-02-24 15:05:20 +03:00
44c0aae16b
fix(emqx_ee_connector): improve log message on influxdb start failures
2023-02-24 11:31:57 +01:00
da42c91fb2
test(emqx_ee_bridge): check influxdb:is_alive/2 return
2023-02-24 09:03:34 +01:00
8314c07147
chore: update influxdb-client-erl dep
2023-02-24 09:03:34 +01:00
603532dacd
fix(emqx_ee_connector): log reason for failure when starting influxdb connector
2023-02-24 09:03:34 +01:00
ada68e12e6
fix(kafka): change default value for memory_overload_protection to false
2023-02-22 13:46:51 +01:00
9316690c29
fix(schema): binary string for default values
...
A lot of the string value fields had default value defined in
schema as list-string rather than binary-string.
This caused the generated schema dump (in JSON format)
to have raw_default field as an integer array.
2023-02-21 09:09:51 +01:00
53ecfb98d0
chore: fix stale code comments
2023-02-21 09:09:51 +01:00
Thales Macedo Garitezi