Commit Graph

843 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 3e518c1876
Merge pull request #12427 from zmstone/0130-limit-kafka-partitions
0130 limit kafka partitions
2024-02-01 14:39:52 +01:00
Zaiming (Stone) Shi 8f9e6769cf docs: add missing desc 2024-02-01 11:08:54 +01:00
Zaiming (Stone) Shi 3b42a7425b feat(kafka_producer): add partitions_limit 2024-01-31 19:26:01 +01:00
lafirest a60aeed539
Merge pull request #12398 from lafirest/feat/api-docs
feat(dashboard): expose the `swagger_support` option
2024-01-31 23:31:56 +08:00
Zaiming (Stone) Shi 87a2368e37 feat(metrics): add cluster_session guage 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 509ab6f35a feat(api): add /sessions_count api to count sessions 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi e9318752e6 feat: store session unregistration timestamp in emqx_cm_registry table 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 4d0febd380 docs: fix schema name-version description 2024-01-31 09:50:12 +01:00
lafirest c7f1e44513
Merge pull request #12424 from lafirest/fix/opentsdb
feat(opentsdb): supports more flexible tags schema
2024-01-30 22:05:30 +08:00
firest 81f96f1a68 feat(opentsdb): supports more flexible tags schema 2024-01-30 18:43:39 +08:00
Shawn 12da3c0986 feat: configurable server side message_expiry_interval 2024-01-29 17:29:52 +08:00
William Yang 933c00c7ad
Merge pull request #12392 from qzhuyan/perf/william/ws-no-utf8-validate
perf: new ws listener option to disable UTF-8 validation
2024-01-26 16:01:04 +01:00
Serge Tupchii 2d693402c5 refactor: split greptimedb bridge to actions and connectors 2024-01-26 11:01:53 +02:00
William Yang 726302ef6a perf: new ws listener option to disable UTF-8 validation 2024-01-26 09:55:17 +01:00
firest c37097a150 feat(dashboard): expose the `swagger_support` option 2024-01-25 22:58:16 +08:00
Kjell Winblad 5bf37cbe0a
Merge pull request #12376 from kjellwinblad/kjell/kinesis
feat: refactor kinesis bridge to connector and action
2024-01-25 14:58:40 +01:00
Shawn 6a21766ce3 refactor: split cassandra bridges to actions and connectors 2024-01-25 16:01:44 +08:00
zhongwencool 186e1591df chore: document api-key path api only support bearerAuth 2024-01-25 16:01:43 +08:00
firest b44420c14f fix(opentsdb): Enhanced the type support for template data 2024-01-25 16:01:43 +08:00
firest 83a8822798 chore(opents): bump version && update changes 2024-01-25 16:01:43 +08:00
firest dad8a32e0b feat(opents): improve the OpentsDB bridge to v2 style 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi d323fc7c27 feat(ps): add message gc 2024-01-25 16:01:43 +08:00
Kjell Winblad fa842736d2 feat: refactor kinesis bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11461
2024-01-24 18:31:03 +01:00
Xinyu Liu aafb683ec7
Merge pull request #12330 from emqx/bridge-v2-cassandra
refactor: split cassandra bridges to actions and connectors
2024-01-23 14:45:32 +08:00
zhongwencool 94692162b5
Merge pull request #12370 from zhongwencool/api-key-auth
chore: document api-key path api only support bearerAuth
2024-01-23 14:22:35 +08:00
lafirest 9d2a207184
Merge pull request #12353 from lafirest/feat/optsdb
feat(opents): improve the OpentsDB bridge to v2 style
2024-01-23 13:49:33 +08:00
zhongwencool ada2785b5d chore: es's base_url to server 2024-01-23 12:20:34 +08:00
zhongwencool 0dbaaa5d94 chore: document api-key path api only support bearerAuth 2024-01-23 10:31:43 +08:00
firest 28867d07e6 fix(opentsdb): Enhanced the type support for template data 2024-01-23 10:00:56 +08:00
Thales Macedo Garitezi d122340c13
Merge pull request #12338 from thalesmg/ds-message-gc-20240115
feat(ds): add message GC
2024-01-22 16:57:26 -03:00
Thales Macedo Garitezi 7035b4c8b3 feat(ps): add message gc 2024-01-22 14:53:17 -03:00
firest 3f7b913e88 chore(opents): bump version && update changes 2024-01-22 20:45:18 +08:00
firest e337e1dc40 feat(opents): improve the OpentsDB bridge to v2 style 2024-01-22 20:45:10 +08:00
Shawn b32c0fb0d8 refactor: split cassandra bridges to actions and connectors 2024-01-22 10:16:46 +08:00
JimMoen 5158395bcf
feat(prometheus): data integration prom data 2024-01-21 00:14:34 +08:00
JimMoen e0feb580b6
feat(prometheus): auth metrics with text/plain 2024-01-21 00:14:32 +08:00
Thales Macedo Garitezi e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489
feat: refactor MQTT bridge to source, action, and connector
2024-01-18 09:28:14 -03:00
zhongwencool 59797cfea7 feat: es's update support doc_as_upsert 2024-01-18 16:44:56 +08:00
zhongwencool 91368a57ff test: add es docker CI test 2024-01-18 16:44:56 +08:00
Thales Macedo Garitezi 1ad3100cad chore: add i18n 2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi ab66986f16 feat: add 'tags' field for action and connector 2024-01-16 13:57:48 +01:00
JianBo He b6d0365027
Merge pull request #12267 from HJianBo/new-timeout-param-for-invite
feat(cluster): supports inviting nodes to join the cluster in an asynchronous manner
2024-01-16 10:11:20 +08:00
Zaiming (Stone) Shi 58a54adbb4 Merge remote-tracking branch 'origin/master' into release-55 2024-01-15 17:58:48 +01:00
JianBo He 9441cf080b
chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-15 18:48:17 +08:00
zhongwencool ace443fc18 refactor: refactor es's action 2024-01-15 11:31:13 +08:00
JianBo He 4c40e754f4 chore: forward the async invite to leader node 2024-01-15 11:24:05 +08:00
zhongwencool e49d3ca50c feat: support elasticsearch bridge 2024-01-15 09:33:44 +08:00
JianBo He 93ef6766ef chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-13 15:29:58 +08:00
JianBo He 6ff4c560e4 feat: support invite node in async mananer 2024-01-13 15:29:58 +08:00
Andrew Mayorov 2dffd44985
feat(route-sync): allow to enable syncer pool only on cores/replicants
So we would able to roll it out and test more gradually.
2024-01-12 11:57:25 +01:00
Andrew Mayorov 0b3f5f7c37
feat(syncer): allow to turn syncer pool on/off through config 2024-01-12 11:57:24 +01:00
Zaiming (Stone) Shi 7c8a36fc06
Merge pull request #12289 from zmstone/0108-support-acl-cache-masks
0108 support acl cache excludes
2024-01-10 13:40:29 +00:00
Zaiming (Stone) Shi 9e8a67fd68 feat: support authz cache exclusion config
now one can configure a list of topic-filters to avoid
caching ACL check results

for example

authorization.cache.excludes = ["nocache/#"]

this means ACL check results for topics having 'nocache/' prefix
will not be cached
2024-01-10 13:52:00 +01:00
JianBo He 7b211b5a23
Merge pull request #12276 from lafirest/fix/iotdb_schema
fix(iotdb): improve the schema for IotDB
2024-01-10 10:21:01 +08:00
JianBo He adcda51eff
Merge pull request #12277 from HJianBo/feat-delete-delayed-messages-via-topic
feat: support to delete delayed messages via topic name
2024-01-10 09:05:19 +08:00
firest 2bf7ac00a0 fix(iotdb): improve the schema for IotDB 2024-01-09 16:29:10 +08:00
lafirest cc00dd80ee
Merge pull request #12272 from lafirest/feat/retain
feat(retain): add two new endpoints for retain API
2024-01-09 15:59:42 +08:00
JianBo He 9ce96bafa3 feat: support to delete delayed messages via topic name 2024-01-09 15:06:06 +08:00
JianBo He 9906fe0e35
chore: update rel/i18n/emqx_bridge_syskeeper.hocon
Co-authored-by: William Yang <mscame@gmail.com>
2024-01-09 10:41:54 +08:00
firest 23e9ed763e chore: bump version && update changes 2024-01-09 10:20:54 +08:00
firest c180325e0a feat(retain): add two new endpoints for retain API 2024-01-08 19:11:43 +08:00
firest 2fb1569363 fix(sysk): improve the seakeeper schema 2024-01-08 15:25:56 +08:00
lafirest 7f57ec47d5
Merge pull request #12261 from lafirest/fix/iotdb
feat(iotdb): improve the IoTDB bridge to v2 style
2024-01-08 09:46:07 +08:00
firest 6f4b22e376 test(iotdb): add template test case for IoTDB 2024-01-06 21:32:37 +08:00
firest d16458ccd0 feat(iotdb): improve the IoTDB bridge to v2 style 2024-01-05 16:54:31 +08:00
Shawn 2f08471303 fix: cannot found desc for influxdb_parameters 2024-01-05 11:59:13 +08:00
Shawn 9714092525 fix: fix: update more influxdb testcases 2024-01-05 11:20:27 +08:00
Shawn 8ebd233a46 chore: update influxdb-erl to 1.1.12 2024-01-04 10:20:52 +08:00
Shawn 7c7a0b2172 refactor: split influxdb bridges to actions and connectors 2024-01-02 18:01:22 +08:00
Thales Macedo Garitezi 035f5f977e Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231221 2023-12-21 17:53:05 -03:00
Andrew Mayorov da49909ac4
chore(typedoc): refine some descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-12-21 17:15:54 +01:00
Andrew Mayorov 85d7518a7d
feat(schema): provide type-level documentation snippets
For stuff like `duration()`, `bytesize()` and `secret()` for now.
2023-12-21 17:15:54 +01:00
Thales Macedo Garitezi 18b69809da Merge branch 'release-54' into sync-r54-m-20231221 2023-12-21 10:05:13 -03:00
zhongwencool 1c17121d44 chore: move redis_type into paramaters in actions 2023-12-19 17:09:08 +08:00
Zaiming (Stone) Shi 2be898ca4d refactor(auth/jwt): support raw rules from jwt acl claim 2023-12-19 08:10:38 +01:00
Stefan Strigler 5d7ae4b980
Merge pull request #12132 from sstrigler/EMQX-11154-bridge-v-2-my-sql-support
feat(emqx_bridge_mysql): port to shared connectors
2023-12-11 13:56:34 +01:00
Zaiming (Stone) Shi 5ad46ce077
Merge pull request #12129 from zmstone/1208-default-license-conn-25
refactor: update default enterprise license
2023-12-08 21:01:04 +01:00
Stefan Strigler 71607aa2ad feat(emqx_bridge_mysql): port to shared connectors 2023-12-08 16:15:18 +01:00
Zaiming (Stone) Shi 41679808fe chore: add version number in license apply link 2023-12-08 08:35:58 +01:00
Zaiming (Stone) Shi e5cf9e40ac refactor: update default enterprise license 2023-12-08 08:25:20 +01:00
SergeTupchiy 28ff53e99c
Merge pull request #11984 from SergeTupchiy/EMQX-10535-openetelmetry-tracing-new
Integrate OpenTelmetry tracing
2023-12-06 11:45:41 +02:00
Zaiming (Stone) Shi d0732aa2a6 refactor: delete UDP multicast clustering strategy 2023-12-06 08:39:08 +01:00
Serge Tupchii c8e69357cc feat(emqx_opentelemetry): use one global exporter config for all otel signals 2023-12-05 18:18:31 +02:00
Serge Tupchii 7fdc650448 feat: integrate OpenTelemetry traces 2023-12-05 18:11:00 +02:00
Zaiming (Stone) Shi 55621fbb86
Merge pull request #12074 from zmstone/1201-user-peer-module-for-all
test: replace 'slave' and 'ct_slave' with 'peer'
2023-12-01 21:35:40 +01:00
Zaiming (Stone) Shi 31749e6e73
Merge pull request #12080 from zmstone/1201-merge-release-53-to-master
1201 merge release 53 to master
2023-12-01 20:21:52 +01:00
ieQu1 476c300ecf
Merge pull request #12070 from ieQu1/dev/ds-min-max-batch-size
Various minor fixes
2023-12-01 17:57:15 +01:00
Zaiming (Stone) Shi 4fed23cc06 Merge remote-tracking branch 'origin/master' into release-53 2023-12-01 16:55:22 +01:00
ieQu1 2f282dd3f1
fix(sessds): Apply suggestions from code review to the docs
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-12-01 16:53:47 +01:00
Zaiming (Stone) Shi 4d5125d826 docs: refine some schema docs 2023-12-01 16:25:59 +01:00
Zaiming (Stone) Shi 4c79ab359f docs: remove tailing space 2023-12-01 16:15:56 +01:00
Zaiming (Stone) Shi 98c228e920 docs: It's Azure "Event Hubs" not "Event Hub" 2023-12-01 16:13:39 +01:00
Zaiming (Stone) Shi 5422de2510 docs: delete hidden config doc 2023-12-01 16:13:23 +01:00
Zaiming (Stone) Shi 8b41d30859 docs: refine doc for listener 'enable_authn' config 2023-12-01 13:08:19 +01:00
Zaiming (Stone) Shi 7f4d91d490 Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54 2023-12-01 08:32:22 +01:00
ieQu1 0e625d814a feat(sessds): Make batch size configurable
This change affects flow control. It allows to configure maximum size
of a batch, as well as fetch threshold.
2023-12-01 08:27:05 +01:00
Thales Macedo Garitezi 113a4ad4b1
Merge pull request #12057 from thalesmg/ds-session-gc-m-20231128
feat(ds): add session gc process
2023-11-30 17:51:52 -03:00
Ivan Dyachkov ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
Thales Macedo Garitezi 880f5e8f89 feat(ds): add session gc process
Fixes https://emqx.atlassian.net/browse/EMQX-9744
2023-11-30 09:38:59 -03:00
zhongwencool 0aec2f7605 feat: redis bridge v2 2023-11-29 16:06:46 +08:00
Zaiming (Stone) Shi 1b1cea24f3 Merge remote-tracking branch 'origin/release-54' into 1128-sync-release-54 2023-11-28 11:43:30 +01:00
JianBo He c8b5c51bbc chore: fix failed test cases 2023-11-28 09:53:46 +08:00
JianBo He f9a1e747fd chore(http): break the bridge confs to connector and action parts 2023-11-28 09:53:46 +08:00
Zaiming (Stone) Shi d933d0b9e0
Merge pull request #12016 from zmstone/1122-load-license-file
1122 load license file
2023-11-27 16:43:51 +01:00
Stefan Strigler 048f4724a9 feat(emqx_connector): add field 'actions' in API response
Also unify schemas, use emqx_connector_schema for the generic parts.
2023-11-27 16:09:17 +01:00
zhongwencool b5a00ec6b2
Merge pull request #12023 from emqx/master
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
Kjell Winblad 66945dcc5c fix: address more comments from @thalesmg 2023-11-24 20:31:29 +01:00
Kjell Winblad 30e248061f
fix: problems found by @thalesmg in code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-24 20:07:37 +01:00
Kjell Winblad e920160805 fix: add enable and description fields to PostgreSQL connector 2023-11-24 17:33:39 +01:00
Kjell Winblad 3dca83c854 fix: all missing descriptions 2023-11-24 17:33:39 +01:00
Stefan Strigler 4e077c951b feat(emqx_bridge_mongodb): port mongodb to shared connector and actions 2023-11-24 15:09:35 +01:00
Zaiming (Stone) Shi 14077ec43b feat(license): allow setting 'default' license key 2023-11-23 17:52:17 +01:00
Zaiming (Stone) Shi caaf8113fc feat(license): support loading license from file 2023-11-23 17:52:17 +01:00
Serge Tupchii d9f95cdc56 feat: integrate OpenTelemetry log handler 2023-11-22 13:59:30 +02:00
ieQu1 ee191803ea
Merge pull request #11990 from ieQu1/dev/ds-shards
Implement a prototype of durable message sharding
2023-11-22 00:57:54 +01:00
ieQu1 4d47490734 chore(ds): Rebase configuration 2023-11-21 19:37:01 +01:00
Thales Macedo Garitezi 9e1796ec4f feat(gcp_pubsub_producer): migrate GCP PubSub producer to actions
Fixes https://emqx.atlassian.net/browse/EMQX-11157
2023-11-21 14:22:42 -03:00
Thales Macedo Garitezi 6c9a8461f7
Merge pull request #11989 from thalesmg/confluent-action-m-20231120
feat: add confluent connector/action
2023-11-21 11:01:40 -03:00
Thales Macedo Garitezi 75ee3ced68 feat: add confluent connector/action
Fixes https://emqx.atlassian.net/browse/EMQX-11044

See also: https://emqx.atlassian.net/wiki/spaces/EMQX/pages/712179934/Confluent+Integration
2023-11-21 08:59:42 -03:00
Andrew Mayorov 110a5a4896
feat(schema): introduce separate root for new session persistence
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
2023-11-20 23:49:14 +07:00
zhongwencool b2e168d508 feat: add prometheus.push_gateway.enable config 2023-11-17 09:48:39 +08:00
zhongwencool c4eb9f86e7 chore: add refactor prometheus refactor changelog 2023-11-17 08:23:51 +08:00
zhongwencool f04cafe57e chore: update prometheus config desc 2023-11-17 08:23:51 +08:00
zhongwencool c376a5db29 refactor: prometheus config 2023-11-17 08:23:51 +08:00
Andrew Mayorov d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Andrew Mayorov 03843c6071
feat(router): switch to v2 routing store by default 2023-11-13 17:43:30 +07:00
JimMoen 4d493292fb
feat(gw_jt808): port jt808 gateway from 4.4 2023-11-12 21:44:18 +08:00
JimMoen 25e277bdc6
chore: fix schema desc and README.md typo 2023-11-12 21:43:10 +08:00
Zaiming (Stone) Shi ff5916e33f docs: fix bad html tag 2023-11-12 10:31:37 +01:00
Thales Macedo Garitezi b4e45f0189
Merge pull request #11923 from thalesmg/sync-m-to-r54-20231109
chore: sync `master` to `release-54`
2023-11-10 13:04:55 -03:00
firest 447c933ff8 fix(syskeeper): homogenize the Syskeeper bridge and update license data 2023-11-10 10:42:53 +08:00
firest 548e39e799 refactor(sysk): refactor the syskeeper to v2 style 2023-11-10 09:52:38 +08:00
firest ce83079c6b feat(sysk): integrated Nari Syskeeper 2000 as a new bridge backend 2023-11-10 09:51:43 +08:00
JianBo He 6500d21d98
Merge pull request #11885 from HJianBo/ocpp-gw
feat: Port OCPP gateway from v4
2023-11-10 09:32:40 +08:00
Thales Macedo Garitezi 7a237f6a3a Merge branch 'master' into sync-m-to-r54-20231109 2023-11-09 18:49:52 -03:00
Thales Macedo Garitezi 371a49304d Merge branch 'release-53' into sync-r53-to-m-20231109 2023-11-09 17:19:23 -03:00
Thales Macedo Garitezi eabd09051a feat(actions_api): add `/action_types` API 2023-11-09 13:54:57 -03:00
firest 2b97800c8b fix(limiter): simplify the descriptions for fields of the limiter 2023-11-09 18:00:09 +08:00
Ivan Dyachkov 0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
JianBo He 4b18631d49 chore: fix gateway failed test cases 2023-11-09 13:44:20 +08:00
JianBo He 2d958beae9 chore: try to fix spellcheck 2023-11-09 09:51:08 +08:00
Ivan Dyachkov 0c91bec98d chore: merge 'upstream/release-53' 2023-11-08 09:24:38 +01:00
Stefan Strigler fa740a1927 docs: fix inconsistencies for bridge and connector operations
It's only `start` allowed for now. Also fixed some wordings.
2023-11-07 13:58:23 +01:00
Zaiming (Stone) Shi f19904d43c
Merge pull request #11877 from zmstone/1102-rename-kafka-to-config
Rename connector channel related configs in bridge_v2 to 'parameters'
2023-11-07 13:41:55 +01:00
JianBo He 7bd5579980 chore: fix the documentation generation and example conf checking 2023-11-07 11:09:39 +08:00
Zaiming (Stone) Shi 9a26c03a5c refactor: unify top level field names for bridge v2 2023-11-06 21:45:08 +01:00
Thales Macedo Garitezi 2b8cf50a1d chore: rename `bridges_v2` -> `actions` in the public facing APIs
Fixes https://emqx.atlassian.net/browse/EMQX-11330

After feedback from Product team, we should rename `bridges_v2` to `actions` everywhere.
We'll start with the public facing APIs.

- HTTP API
- Hocon schema root key
2023-11-06 15:37:07 -03:00
JianBo He 8871738756 chore: ensure emqx_gateway_ocpp tests pass 2023-11-06 12:26:58 +08:00
JianBo He 7cab269e0b feat: port the ocpp gateway from version 4 2023-11-03 16:51:21 +08:00