Commit Graph

972 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
3818b75188
Merge pull request #12959 from thalesmg/kprodu-connector-hc-m-20240429
feat(kafka producer): add health check topic option
2024-05-03 12:48:48 -03:00
zmstone
3c5c76fcfc refactor: simplify https listener config for dashboard 2024-05-03 16:45:40 +02:00
Thales Macedo Garitezi
6f3da6b131 feat(kafka producer): add health check topic option
Fixes https://emqx.atlassian.net/browse/EMQX-12241

This allows more accurate health checking for Kafka Producers.  Without a topic, it's not
possible to actually probe the connection to partition leaders, so the connector might not
be reported as `disconnected` without testing a concrete topic.
2024-05-02 17:07:44 -03:00
Thales Macedo Garitezi
c71f73924b
Merge pull request #12961 from thalesmg/kconsu-custom-group-id-m-20240430
feat(kafka consumer): allow custom group id
2024-05-02 14:37:42 -03:00
Thales Macedo Garitezi
905d04f1c3 docs: improve descriptions 2024-05-02 11:19:14 -03:00
William Yang
8f780ae8bc
Merge pull request #12955 from qzhuyan/port/william/5/partial-chain
port: TLS partial chain
2024-05-02 15:34:25 +02:00
Thales Macedo Garitezi
14ef0b1e51 feat(kafka consumer): allow custom group id
Fixes https://emqx.atlassian.net/browse/EMQX-12273
Fixes EMQX-12273

When consuming messages in Kafka in Alibaba Cloud, the group needs to be configured in
advance, and then the consumer can use the group to consume messages. Automatic group
creation is generally not allowed online.
2024-05-01 13:53:29 -03:00
Thales Macedo Garitezi
42cb17360e Merge branch 'release-57' into sync-r57-m-20240430 2024-04-30 14:42:22 -03:00
Ilya Averyanov
80d724c504 feat(authn): add connection expire based on authn data 2024-04-30 17:04:55 +03:00
Andrew Mayorov
339036045d
feat(s3-aggreg): support custom and default S3 object HTTP headers
I.e. configured container decides default `Content-Type` header.
2024-04-30 10:48:15 +02:00
Andrew Mayorov
5b15b2d641
docs(s3-bridge): document aggregated upload action schema 2024-04-30 10:48:15 +02:00
William Yang
8eb463c58d feat(tls): update schema for TLS keyusage 2024-04-30 06:25:03 +02:00
William Yang
8bc3a86f63 feat(config): partial_chain 2024-04-30 06:25:03 +02:00
zmstone
7a05a4754f docs: expose zone config in schema doc 2024-04-29 13:57:18 +02:00
zmstone
b96c6c243a docs: improve mqtt_path doc 2024-04-27 15:19:59 +02:00
Ilia Averianov
a0e0a27f87
Merge pull request #12418 from savonarola/0129-fix-auth-claim-schema
feat(jwt_auth): improve verify_claims handling and docs
2024-04-25 19:05:13 +03:00
Ilya Averyanov
407b0cd0ca feat(jwt_auth): improve verify_claims handling and docs 2024-04-25 17:49:29 +03:00
zmstone
d2e0094dfd docs: improve doc for pulsar auth config fields 2024-04-24 11:18:48 +02:00
lafirest
0d1c13661f
Merge pull request #12895 from lafirest/fix/dyndb
fix(dynamo): Added missing keys for DynamoDB
2024-04-22 23:03:00 +08:00
firest
d5cdc07eab feat(rocketmq): add support for namespace and key dispatch strategy 2024-04-22 13:14:11 +08:00
firest
46f339dfab chore: updte change log 2024-04-22 13:12:01 +08:00
firest
a1e85e3c59 fix(dynamo): Added missing keys for DynamoDB 2024-04-22 13:09:34 +08:00
Kjell Winblad
6e99f01ecd
Merge pull request #12863 from kjellwinblad/kjell/new_trace_log_formatter/EMQX-12025
feat(emqx_trace): add JSON trace log entry formatter
2024-04-22 06:25:18 +02:00
Zaiming (Stone) Shi
307cd79be2
Merge pull request #12897 from zmstone/0418-support-dynamic-license
feat(license): add business-critical customer type
2024-04-20 09:03:58 +02:00
zmstone
ec83fbe3dc feat(license): add business critical customer type 2024-04-19 14:19:39 +02:00
Zaiming (Stone) Shi
a8b1224225
Merge pull request #12893 from zmstone/0416-add-is-template-flag-to-dashboard-schema
0416 add `is_template` flag to dashboard schema
2024-04-19 10:41:43 +02:00
Kjell Winblad
10957e7d79 fix: change name of the default trace log format from plain to text 2024-04-18 10:42:51 +02:00
Kjell Winblad
a6558740e8 feat(emqx_trace): add JSON trace log entry formatter
This commit makes it possible to select the JSON trace log entry
formatter when crating a trace pattern. This will make it easier for the
dashboard and automatic tests to parse the log entries.

Fixes: EMQX-12025 (partly)
2024-04-18 10:20:11 +02:00
Kjell Winblad
2a2fadfbff
Merge pull request #12827 from kjellwinblad/kjell/emqx_rule_tracing/EMQX-11966
Add rule ID tracing support
2024-04-18 09:06:31 +02:00
zmstone
51c8173174 feat(bridge): add is_template flag to bridge config fields 2024-04-17 13:10:55 +02:00
zmstone
ab0f1888bf docs: refine client_attrs doc 2024-04-16 08:31:07 +02:00
zmstone
b76b6fbe63 feat(variform): initialize client_attrs with variform
Moved regular expression extraction as a variform function.
2024-04-14 10:13:24 +02:00
zmstone
da5b01aa46 refactor(client_attr): allow more than one initial extraction 2024-04-13 01:00:25 +02:00
Kjell Winblad
9998940aa2 fix(trace): several improvements thanks to comments from @zmstone 2024-04-11 16:48:43 +02:00
Kjell Winblad
31142df5cf fix: default value of stop_action_after_template_rendering to true
This commit changes the default value for the
stop_action_after_template_rendering option of the apply rule HTTP API
endpoint so that it is true instead of false.
2024-04-09 16:20:37 +02:00
Kjell Winblad
9628a00a82 docs(emqx_rule_api apply rule): fix doc strings 2024-04-08 15:34:29 +02:00
Kjell Winblad
59a442cdb5 feat(rule trace): add support for ruleid as a trace type 2024-04-04 14:55:32 +02:00
Ivan Dyachkov
f4446ec680 Merge remote-tracking branch 'upstream/release-56' into 0328-sync-release-56 2024-03-28 09:59:54 +01:00
Zaiming (Stone) Shi
beb9152d50
Merge pull request #12750 from zmstone/0316-introduce-client-attrs
0316 introduce client_attrs
2024-03-27 16:19:47 +01:00
zmstone
450ef2d591 docs: refine docs 2024-03-27 12:42:01 +01:00
zmstone
874b1ff3f4 docs: fix epoch timestamp precision 2024-03-26 16:07:55 +01:00
zmstone
c42e980442 feat(log): configurable time format
now logs can be configured to use 'epoch' or 'rfc3339' time format
2024-03-26 13:09:32 +01:00
Thales Macedo Garitezi
59ff43ec87 feat(message validation): implement metrics
Fixes https://emqx.atlassian.net/browse/EMQX-12071
2024-03-25 10:55:01 -03:00
zmstone
df913f2450 docs: refine changelog 2024-03-23 14:11:30 +01:00
zmstone
3136ec5958 feat: allow mountpoint to use client_attrs 2024-03-23 10:16:05 +01:00
zmstone
5e9814d171 fix: add debug level logging for invalid client attributes 2024-03-23 10:16:05 +01:00
zmstone
0cf61932b6 feat: allow using client_attrs in authentication templates 2024-03-23 10:16:05 +01:00
zmstone
8254b801ae feat: support initialize client attribute from user property 2024-03-23 10:16:02 +01:00
zmstone
2fd0a2cd4d feat: support extracting initial client attrs from clientinfo 2024-03-23 10:16:02 +01:00
Thales Macedo Garitezi
e17f663fa5
Merge pull request #12749 from thalesmg/mv-followup1-m-20240320
Follow up features and fixes for message validation - part 1
2024-03-21 16:03:49 -03:00
Thales Macedo Garitezi
b8cd1c9020 feat(message validation api): add enable/disable HTTP API 2024-03-20 14:59:32 -03:00
Thales Macedo Garitezi
4944cc080e feat(message_validation): add ignore failure action 2024-03-20 14:31:43 -03:00
Zaiming (Stone) Shi
bede5a5b85
Merge pull request #12746 from zmstone/0320-add-throttle-log-for-auth-failure
feat: add authentication_failure throttled log
2024-03-20 17:06:36 +01:00
zmstone
30d4eb9250 docs: fix i18n message lable ID 2024-03-20 14:57:51 +01:00
Serge Tupchii
cb5fdb3c79 fix: rework In-flight / Mqueue API
- use timestamp 'position' to find the next chunk of data
- add 'start' position to response meta
- sort In-flight messages by insertion time
- sort Mqueue messages by priority
2024-03-20 15:25:33 +02:00
Andrew Mayorov
a8baff61ec
docs(dsrepl): describe briefly what n_sites is for 2024-03-20 13:20:25 +01:00
Ivan Dyachkov
923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
sync release-56
2024-03-19 19:09:41 +01:00
Zaiming (Stone) Shi
73121f3ccc
Merge pull request #12730 from zmstone/0318-docs-fix-escape-in-i18n
docs: there is no need to escape quotes in triple-quotes
2024-03-19 18:30:14 +01:00
Thales Macedo Garitezi
2eac54bf54 chore: rm move API
Only `reorder` API will be supported, after discussions with product and frontend teams.
2024-03-19 09:55:55 -03:00
Thales Macedo Garitezi
e8a8416e44 feat(message_validation): impose restrictions on validation name 2024-03-18 15:57:39 -03:00
Thales Macedo Garitezi
74c03377f2 feat(message_validation_api): implement reorder API 2024-03-18 14:30:52 -03:00
Thales Macedo Garitezi
f84a996671 feat: implement message validation
Fixes https://emqx.atlassian.net/browse/EMQX-11980
2024-03-18 13:11:39 -03:00
zmstone
313c16f944 docs: there is no need to escape quotes in triple-quotes 2024-03-18 16:54:04 +01:00
JianBo He
2a611e4508 chore: fix the error api desc 2024-03-18 18:27:28 +08:00
SergeTupchiy
1f38813cb9
Merge pull request #12561 from SergeTupchiy/EMQX-11861-client-mqueue-inflight-API
feat: add client mqueue/inflight messages API
2024-03-07 13:20:31 +02:00
Serge Tupchii
a2e761681e feat: add client mqueue/inflight messages API 2024-03-07 12:06:49 +02:00
Thales Macedo Garitezi
c5489fee90 Merge remote-tracking branch 'origin/release-56' into sync-r56-m-20240304 2024-03-05 09:08:59 -03:00
ieQu1
2e792da934
Merge pull request #12562 from ieQu1/dev/new-ds-schema
Refactor schema for the durable storage
2024-03-04 10:33:27 +01:00
JimMoen
1d58092176
fix: bridge v2 on_query matched ChannelId 2024-03-04 17:13:23 +08:00
JimMoen
9143d5994d
feat: refactor MS SQL Server bridge to connector and action 2024-03-04 17:13:21 +08:00
Zaiming (Stone) Shi
e9a5670cce
Merge pull request #12581 from zmstone/0223-add-json-schema
feat: schema registry supports json schema
2024-03-02 10:09:31 +01:00
zhongwencool
d56fb22208
Merge pull request #12595 from thalesmg/kafka-consumer-source-m-20240223
feat: migrate kafka consumer bridge to source + connector
2024-03-01 10:11:16 +08:00
Zaiming (Stone) Shi
b7e5ea2941 feat(schema_registry): add JSON schema 2024-02-29 21:03:52 +01:00
zhongwencool
bc9de20024
Merge pull request #12540 from zhongwencool/bridge-pulsar-v2
feat: pulsar bridge v2
2024-02-29 21:25:11 +08:00
Ilya Averyanov
d4519bda82
Merge pull request #12589 from savonarola/0222-retainer-interface
Make retainer pluggable
2024-02-29 10:31:15 +02:00
zhongwencool
e9e1daf962 chore: update some pulsar's desc 2024-02-29 16:23:37 +08:00
zhongwencool
7f1b4cef27 feat: pulsar bridge v2 2024-02-29 16:23:37 +08:00
Zaiming (Stone) Shi
704eceb7c7
Merge pull request #12599 from zmstone/0227-refactor-log-config-template
docs: add log docs
2024-02-28 17:58:14 +01:00
Ilya Averyanov
dfdf7455d3 feat(retainer): make additional implementations pluggable
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-28 18:43:17 +03:00
Thales Macedo Garitezi
6b9844ae82 feat: migrate kafka consumer bridge to source + connector
Fixes https://emqx.atlassian.net/browse/EMQX-11848
2024-02-27 17:52:33 -03:00
Zaiming (Stone) Shi
ef99037335 docs: add log docs 2024-02-27 19:24:56 +01:00
Zaiming (Stone) Shi
f87ee2d28b docs: refine throttled log time_window description 2024-02-26 10:28:58 +01:00
ieQu1
e126393cbf
fix(ds): Fix schema checker warnings for DS schema 2024-02-23 15:39:52 +01:00
ieQu1
dd2e35345f
docs(ds): Apply remarks 2024-02-23 15:08:13 +01:00
ieQu1
91ddbbcc3f
fix(sessds): Replace min- and max- batch size with batch_size 2024-02-23 15:08:13 +01:00
ieQu1
94b0ab983d
docs(ds): Add descriptions for the builtin's egress config 2024-02-23 15:08:13 +01:00
ieQu1
786e30056b
docs(ds): Add labels to the i18n for the storage schema 2024-02-23 15:08:13 +01:00
ieQu1
24337ecec7
feat(sessds): Move config schema to a separate root 2024-02-23 15:08:13 +01:00
ieQu1
8907e5afb3
chore(sessds): Remove deprecated schema 2024-02-23 15:08:11 +01:00
Kjell Winblad
7bb9d5d8f6
Merge pull request #12543 from kjellwinblad/kjell/refactor/dynamodb/EMQX-11456
feat: refactor DynamoDB bridge to connector and action
2024-02-22 09:47:57 +01:00
Serge Tupchii
ceb208f1eb fix: disable log throttling if primary log level is debug 2024-02-21 21:19:15 +02:00
Kjell Winblad
6561d989d6 feat: refactor DynamoDB bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11456
2024-02-21 14:58:48 +01:00
Thales Macedo Garitezi
4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
Zaiming (Stone) Shi
da340c92a1 Merge remote-tracking branch 'origin/master' into 0221-sync-release-55 2024-02-21 09:11:12 +01:00
Thales Macedo Garitezi
411e5800d1
docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-02-20 13:37:54 -03:00
SergeTupchiy
f72558737c
Merge pull request #12520 from SergeTupchiy/EMQX-11530-log-throttling
feat: implement log throttling
2024-02-20 15:52:46 +02:00
Kjell Winblad
86c85f3b45
Merge pull request #12512 from kjellwinblad/kjell/refactor/hstreamdb/EMQX-11458
feat: refactor HStreamDB bridge to connector and action
2024-02-20 13:30:29 +01:00
Serge Tupchii
9bd0d1ba1b feat: enable log throttling for potentially flooding log events 2024-02-20 14:20:21 +02:00
Serge Tupchii
98ba300f7c feat: implement log throttling 2024-02-20 13:23:17 +02:00
Thales Macedo Garitezi
9a32895a1a feat: convert gcp_pubsub_consumer to connector/source
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
Zaiming (Stone) Shi
68baa4dcaf docs: fix i18n 2024-02-19 13:57:44 +01:00
Kjell Winblad
def95aa22b docs(HStreamDB bridge): make pool size descriptions better 2024-02-19 11:33:37 +01:00
zhongwencool
2ac4bde102
Merge pull request #12487 from zhongwencool/rabbitmq-source-test
feat: don't publish mqtt message in rabbitmq's source
2024-02-19 17:36:12 +08:00
Shawn
ec888bc87f ci: fix test cases for clickhouse 2024-02-18 10:02:02 +08:00
Shawn
8ae0e78786 refactor: split clickhouse bridges to actions and connectors 2024-02-18 10:01:53 +08:00
Kjell Winblad
0cb28f5f40
docs: better descriptions and labels for configuration parameters
Thanks @zmstone for the suggestions

Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-02-16 11:50:51 +01:00
Zaiming (Stone) Shi
a389d78b42 chore: delete stale code 2024-02-16 10:32:13 +01:00
Andrew Mayorov
4484578c1f
Merge pull request #12507 from keynslug/fix/gw-app-no-scan
fix(gateway): avoid scanning modules of known gateway apps
2024-02-15 17:11:49 +01:00
Kjell Winblad
5e442fcbca fix: labels and descriptions 2024-02-15 11:08:08 +01:00
Serge Tupchii
7272ef25d4 feat(emqx_auth): implement API to re-order all authenticators/authz sources
Fixes: EMQX-11770
2024-02-14 14:35:46 +02:00
Andrew Mayorov
8f7b0ac498
docs(jt808): fix doc references 2024-02-13 16:27:41 +01:00
Kjell Winblad
366827390e
Merge pull request #12488 from kjellwinblad/kjell/refactor/rocketmq_bridge/EMQX-11467
feat: refactor RocketMQ bridge to connector and action
2024-02-13 16:19:21 +01:00
Ilya Averyanov
edd28be090
Merge pull request #12499 from savonarola/0209-emqx-banned-extended-rules
feat(banned): allow ban by clientid/username regexps, peerhost cidrs
2024-02-12 17:12:47 +02:00
Andrew Mayorov
5b7bb89f37
docs(s3): add missing labels and fix wording
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-02-12 10:39:42 +01:00
Ilya Averyanov
90fd2b26d3 feat(banned): allow ban by clientid/username regexps, peerhost cidrs 2024-02-10 17:59:22 +03:00
Andrew Mayorov
802c760406
feat(s3): introduce S3 connector and action
This is a trivial connector based on `emqx_s3` and simple action
that maps each incoming event into an S3 object upload. Due to
current `emqx_s3` limitation this bridge is compatible with
backends providing S3 API with path-style bucket access.
2024-02-09 14:16:32 +01:00
Kjell Winblad
e284a83f73 feat: refactor RocketMQ bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11467
2024-02-08 16:21:58 +01:00
zhongwencool
90ba2977fe feat: don't publish mqtt message in rabbitmq's source 2024-02-08 12:38:18 +08:00
zhongwencool
eace8f2fcb
Merge pull request #12423 from zhongwencool/rabbitmq-e560
feat: rabbitmq bridge v2 integration
2024-02-06 09:55:05 +08:00
zhongwencool
688701eedb chore: add desc for rabbitmq source 2024-02-06 07:57:13 +08:00
Zaiming (Stone) Shi
ce50055138 docs: fix unicode colon symbol 2024-02-05 16:43:03 +01:00
Zaiming (Stone) Shi
b1a05c7b59
Merge pull request #12326 from zmstone/0111-unregister-session-with-timestamp
0111 unregister session with timestamp
2024-02-02 17:00:23 +01:00
Kjell Winblad
fa0baf2f38
Merge pull request #12439 from kjellwinblad/kjell/refactor/oracle_bridge/EMQX-11464
feat: refactor Oracle bridge to connector and action
2024-02-02 10:53:15 +01:00
Zaiming (Stone) Shi
f0569d8ae8 refactor: use mria:async_dirty to group dirty ops 2024-02-02 09:26:43 +01:00
lafirest
24d8a9ab00
Merge pull request #12449 from lafirest/feat/td
feat(tdengine): improve the TDengine bridge to v2 style
2024-02-02 16:17:01 +08:00
firest
5291adf2c0 fix(opents): change the schema of tags to object style 2024-02-02 13:07:49 +08:00
firest
2241461acb chore: update change & bump version 2024-02-02 07:48:44 +08:00
firest
dfad020c49 feat(tdengine): improve the TDengine bridge to v2 style 2024-02-01 23:14:24 +08:00
Kjell Winblad
a4272c71dc feat: refactor Oracle bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11464
2024-02-01 16:03:05 +01:00
Zaiming (Stone) Shi
3e518c1876
Merge pull request #12427 from zmstone/0130-limit-kafka-partitions
0130 limit kafka partitions
2024-02-01 14:39:52 +01:00
Zaiming (Stone) Shi
8f9e6769cf docs: add missing desc 2024-02-01 11:08:54 +01:00
Zaiming (Stone) Shi
3b42a7425b feat(kafka_producer): add partitions_limit 2024-01-31 19:26:01 +01:00
lafirest
a60aeed539
Merge pull request #12398 from lafirest/feat/api-docs
feat(dashboard): expose the `swagger_support` option
2024-01-31 23:31:56 +08:00
Zaiming (Stone) Shi
87a2368e37 feat(metrics): add cluster_session guage 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi
509ab6f35a feat(api): add /sessions_count api to count sessions 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi
e9318752e6 feat: store session unregistration timestamp in emqx_cm_registry table 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi
4d0febd380 docs: fix schema name-version description 2024-01-31 09:50:12 +01:00
lafirest
c7f1e44513
Merge pull request #12424 from lafirest/fix/opentsdb
feat(opentsdb): supports more flexible tags schema
2024-01-30 22:05:30 +08:00
firest
81f96f1a68 feat(opentsdb): supports more flexible tags schema 2024-01-30 18:43:39 +08:00
Shawn
12da3c0986 feat: configurable server side message_expiry_interval 2024-01-29 17:29:52 +08:00
William Yang
933c00c7ad
Merge pull request #12392 from qzhuyan/perf/william/ws-no-utf8-validate
perf: new ws listener option to disable UTF-8 validation
2024-01-26 16:01:04 +01:00
Serge Tupchii
2d693402c5 refactor: split greptimedb bridge to actions and connectors 2024-01-26 11:01:53 +02:00
William Yang
726302ef6a perf: new ws listener option to disable UTF-8 validation 2024-01-26 09:55:17 +01:00
firest
c37097a150 feat(dashboard): expose the swagger_support option 2024-01-25 22:58:16 +08:00
Kjell Winblad
5bf37cbe0a
Merge pull request #12376 from kjellwinblad/kjell/kinesis
feat: refactor kinesis bridge to connector and action
2024-01-25 14:58:40 +01:00
Shawn
6a21766ce3 refactor: split cassandra bridges to actions and connectors 2024-01-25 16:01:44 +08:00
zhongwencool
186e1591df chore: document api-key path api only support bearerAuth 2024-01-25 16:01:43 +08:00
firest
b44420c14f fix(opentsdb): Enhanced the type support for template data 2024-01-25 16:01:43 +08:00
firest
83a8822798 chore(opents): bump version && update changes 2024-01-25 16:01:43 +08:00
firest
dad8a32e0b feat(opents): improve the OpentsDB bridge to v2 style 2024-01-25 16:01:43 +08:00