Commit Graph

808 Commits

Author SHA1 Message Date
JimMoen e0feb580b6
feat(prometheus): auth metrics with text/plain 2024-01-21 00:14:32 +08:00
Thales Macedo Garitezi e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489
feat: refactor MQTT bridge to source, action, and connector
2024-01-18 09:28:14 -03:00
zhongwencool 59797cfea7 feat: es's update support doc_as_upsert 2024-01-18 16:44:56 +08:00
zhongwencool 91368a57ff test: add es docker CI test 2024-01-18 16:44:56 +08:00
Thales Macedo Garitezi 1ad3100cad chore: add i18n 2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi ab66986f16 feat: add 'tags' field for action and connector 2024-01-16 13:57:48 +01:00
JianBo He b6d0365027
Merge pull request #12267 from HJianBo/new-timeout-param-for-invite
feat(cluster): supports inviting nodes to join the cluster in an asynchronous manner
2024-01-16 10:11:20 +08:00
Zaiming (Stone) Shi 58a54adbb4 Merge remote-tracking branch 'origin/master' into release-55 2024-01-15 17:58:48 +01:00
JianBo He 9441cf080b
chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-15 18:48:17 +08:00
zhongwencool ace443fc18 refactor: refactor es's action 2024-01-15 11:31:13 +08:00
JianBo He 4c40e754f4 chore: forward the async invite to leader node 2024-01-15 11:24:05 +08:00
zhongwencool e49d3ca50c feat: support elasticsearch bridge 2024-01-15 09:33:44 +08:00
JianBo He 93ef6766ef chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-13 15:29:58 +08:00
JianBo He 6ff4c560e4 feat: support invite node in async mananer 2024-01-13 15:29:58 +08:00
Andrew Mayorov 2dffd44985
feat(route-sync): allow to enable syncer pool only on cores/replicants
So we would able to roll it out and test more gradually.
2024-01-12 11:57:25 +01:00
Andrew Mayorov 0b3f5f7c37
feat(syncer): allow to turn syncer pool on/off through config 2024-01-12 11:57:24 +01:00
Zaiming (Stone) Shi 7c8a36fc06
Merge pull request #12289 from zmstone/0108-support-acl-cache-masks
0108 support acl cache excludes
2024-01-10 13:40:29 +00:00
Zaiming (Stone) Shi 9e8a67fd68 feat: support authz cache exclusion config
now one can configure a list of topic-filters to avoid
caching ACL check results

for example

authorization.cache.excludes = ["nocache/#"]

this means ACL check results for topics having 'nocache/' prefix
will not be cached
2024-01-10 13:52:00 +01:00
JianBo He 7b211b5a23
Merge pull request #12276 from lafirest/fix/iotdb_schema
fix(iotdb): improve the schema for IotDB
2024-01-10 10:21:01 +08:00
JianBo He adcda51eff
Merge pull request #12277 from HJianBo/feat-delete-delayed-messages-via-topic
feat: support to delete delayed messages via topic name
2024-01-10 09:05:19 +08:00
firest 2bf7ac00a0 fix(iotdb): improve the schema for IotDB 2024-01-09 16:29:10 +08:00
lafirest cc00dd80ee
Merge pull request #12272 from lafirest/feat/retain
feat(retain): add two new endpoints for retain API
2024-01-09 15:59:42 +08:00
JianBo He 9ce96bafa3 feat: support to delete delayed messages via topic name 2024-01-09 15:06:06 +08:00
JianBo He 9906fe0e35
chore: update rel/i18n/emqx_bridge_syskeeper.hocon
Co-authored-by: William Yang <mscame@gmail.com>
2024-01-09 10:41:54 +08:00
firest 23e9ed763e chore: bump version && update changes 2024-01-09 10:20:54 +08:00
firest c180325e0a feat(retain): add two new endpoints for retain API 2024-01-08 19:11:43 +08:00
firest 2fb1569363 fix(sysk): improve the seakeeper schema 2024-01-08 15:25:56 +08:00
lafirest 7f57ec47d5
Merge pull request #12261 from lafirest/fix/iotdb
feat(iotdb): improve the IoTDB bridge to v2 style
2024-01-08 09:46:07 +08:00
firest 6f4b22e376 test(iotdb): add template test case for IoTDB 2024-01-06 21:32:37 +08:00
firest d16458ccd0 feat(iotdb): improve the IoTDB bridge to v2 style 2024-01-05 16:54:31 +08:00
Shawn 2f08471303 fix: cannot found desc for influxdb_parameters 2024-01-05 11:59:13 +08:00
Shawn 9714092525 fix: fix: update more influxdb testcases 2024-01-05 11:20:27 +08:00
Shawn 8ebd233a46 chore: update influxdb-erl to 1.1.12 2024-01-04 10:20:52 +08:00
Shawn 7c7a0b2172 refactor: split influxdb bridges to actions and connectors 2024-01-02 18:01:22 +08:00
Thales Macedo Garitezi 035f5f977e Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231221 2023-12-21 17:53:05 -03:00
Andrew Mayorov da49909ac4
chore(typedoc): refine some descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-12-21 17:15:54 +01:00
Andrew Mayorov 85d7518a7d
feat(schema): provide type-level documentation snippets
For stuff like `duration()`, `bytesize()` and `secret()` for now.
2023-12-21 17:15:54 +01:00
Thales Macedo Garitezi 18b69809da Merge branch 'release-54' into sync-r54-m-20231221 2023-12-21 10:05:13 -03:00
zhongwencool 1c17121d44 chore: move redis_type into paramaters in actions 2023-12-19 17:09:08 +08:00
Zaiming (Stone) Shi 2be898ca4d refactor(auth/jwt): support raw rules from jwt acl claim 2023-12-19 08:10:38 +01:00
Stefan Strigler 5d7ae4b980
Merge pull request #12132 from sstrigler/EMQX-11154-bridge-v-2-my-sql-support
feat(emqx_bridge_mysql): port to shared connectors
2023-12-11 13:56:34 +01:00
Zaiming (Stone) Shi 5ad46ce077
Merge pull request #12129 from zmstone/1208-default-license-conn-25
refactor: update default enterprise license
2023-12-08 21:01:04 +01:00
Stefan Strigler 71607aa2ad feat(emqx_bridge_mysql): port to shared connectors 2023-12-08 16:15:18 +01:00
Zaiming (Stone) Shi 41679808fe chore: add version number in license apply link 2023-12-08 08:35:58 +01:00
Zaiming (Stone) Shi e5cf9e40ac refactor: update default enterprise license 2023-12-08 08:25:20 +01:00
SergeTupchiy 28ff53e99c
Merge pull request #11984 from SergeTupchiy/EMQX-10535-openetelmetry-tracing-new
Integrate OpenTelmetry tracing
2023-12-06 11:45:41 +02:00
Zaiming (Stone) Shi d0732aa2a6 refactor: delete UDP multicast clustering strategy 2023-12-06 08:39:08 +01:00
Serge Tupchii c8e69357cc feat(emqx_opentelemetry): use one global exporter config for all otel signals 2023-12-05 18:18:31 +02:00
Serge Tupchii 7fdc650448 feat: integrate OpenTelemetry traces 2023-12-05 18:11:00 +02:00
Zaiming (Stone) Shi 55621fbb86
Merge pull request #12074 from zmstone/1201-user-peer-module-for-all
test: replace 'slave' and 'ct_slave' with 'peer'
2023-12-01 21:35:40 +01:00
Zaiming (Stone) Shi 31749e6e73
Merge pull request #12080 from zmstone/1201-merge-release-53-to-master
1201 merge release 53 to master
2023-12-01 20:21:52 +01:00
ieQu1 476c300ecf
Merge pull request #12070 from ieQu1/dev/ds-min-max-batch-size
Various minor fixes
2023-12-01 17:57:15 +01:00
Zaiming (Stone) Shi 4fed23cc06 Merge remote-tracking branch 'origin/master' into release-53 2023-12-01 16:55:22 +01:00
ieQu1 2f282dd3f1
fix(sessds): Apply suggestions from code review to the docs
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-12-01 16:53:47 +01:00
Zaiming (Stone) Shi 4d5125d826 docs: refine some schema docs 2023-12-01 16:25:59 +01:00
Zaiming (Stone) Shi 4c79ab359f docs: remove tailing space 2023-12-01 16:15:56 +01:00
Zaiming (Stone) Shi 98c228e920 docs: It's Azure "Event Hubs" not "Event Hub" 2023-12-01 16:13:39 +01:00
Zaiming (Stone) Shi 5422de2510 docs: delete hidden config doc 2023-12-01 16:13:23 +01:00
Zaiming (Stone) Shi 8b41d30859 docs: refine doc for listener 'enable_authn' config 2023-12-01 13:08:19 +01:00
Zaiming (Stone) Shi 7f4d91d490 Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54 2023-12-01 08:32:22 +01:00
ieQu1 0e625d814a feat(sessds): Make batch size configurable
This change affects flow control. It allows to configure maximum size
of a batch, as well as fetch threshold.
2023-12-01 08:27:05 +01:00
Thales Macedo Garitezi 113a4ad4b1
Merge pull request #12057 from thalesmg/ds-session-gc-m-20231128
feat(ds): add session gc process
2023-11-30 17:51:52 -03:00
Ivan Dyachkov ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
Thales Macedo Garitezi 880f5e8f89 feat(ds): add session gc process
Fixes https://emqx.atlassian.net/browse/EMQX-9744
2023-11-30 09:38:59 -03:00
zhongwencool 0aec2f7605 feat: redis bridge v2 2023-11-29 16:06:46 +08:00
Zaiming (Stone) Shi 1b1cea24f3 Merge remote-tracking branch 'origin/release-54' into 1128-sync-release-54 2023-11-28 11:43:30 +01:00
JianBo He c8b5c51bbc chore: fix failed test cases 2023-11-28 09:53:46 +08:00
JianBo He f9a1e747fd chore(http): break the bridge confs to connector and action parts 2023-11-28 09:53:46 +08:00
Zaiming (Stone) Shi d933d0b9e0
Merge pull request #12016 from zmstone/1122-load-license-file
1122 load license file
2023-11-27 16:43:51 +01:00
Stefan Strigler 048f4724a9 feat(emqx_connector): add field 'actions' in API response
Also unify schemas, use emqx_connector_schema for the generic parts.
2023-11-27 16:09:17 +01:00
zhongwencool b5a00ec6b2
Merge pull request #12023 from emqx/master
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
Kjell Winblad 66945dcc5c fix: address more comments from @thalesmg 2023-11-24 20:31:29 +01:00
Kjell Winblad 30e248061f
fix: problems found by @thalesmg in code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-24 20:07:37 +01:00
Kjell Winblad e920160805 fix: add enable and description fields to PostgreSQL connector 2023-11-24 17:33:39 +01:00
Kjell Winblad 3dca83c854 fix: all missing descriptions 2023-11-24 17:33:39 +01:00
Stefan Strigler 4e077c951b feat(emqx_bridge_mongodb): port mongodb to shared connector and actions 2023-11-24 15:09:35 +01:00
Zaiming (Stone) Shi 14077ec43b feat(license): allow setting 'default' license key 2023-11-23 17:52:17 +01:00
Zaiming (Stone) Shi caaf8113fc feat(license): support loading license from file 2023-11-23 17:52:17 +01:00
Serge Tupchii d9f95cdc56 feat: integrate OpenTelemetry log handler 2023-11-22 13:59:30 +02:00
ieQu1 ee191803ea
Merge pull request #11990 from ieQu1/dev/ds-shards
Implement a prototype of durable message sharding
2023-11-22 00:57:54 +01:00
ieQu1 4d47490734 chore(ds): Rebase configuration 2023-11-21 19:37:01 +01:00
Thales Macedo Garitezi 9e1796ec4f feat(gcp_pubsub_producer): migrate GCP PubSub producer to actions
Fixes https://emqx.atlassian.net/browse/EMQX-11157
2023-11-21 14:22:42 -03:00
Thales Macedo Garitezi 6c9a8461f7
Merge pull request #11989 from thalesmg/confluent-action-m-20231120
feat: add confluent connector/action
2023-11-21 11:01:40 -03:00
Thales Macedo Garitezi 75ee3ced68 feat: add confluent connector/action
Fixes https://emqx.atlassian.net/browse/EMQX-11044

See also: https://emqx.atlassian.net/wiki/spaces/EMQX/pages/712179934/Confluent+Integration
2023-11-21 08:59:42 -03:00
Andrew Mayorov 110a5a4896
feat(schema): introduce separate root for new session persistence
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
2023-11-20 23:49:14 +07:00
zhongwencool b2e168d508 feat: add prometheus.push_gateway.enable config 2023-11-17 09:48:39 +08:00
zhongwencool c4eb9f86e7 chore: add refactor prometheus refactor changelog 2023-11-17 08:23:51 +08:00
zhongwencool f04cafe57e chore: update prometheus config desc 2023-11-17 08:23:51 +08:00
zhongwencool c376a5db29 refactor: prometheus config 2023-11-17 08:23:51 +08:00
Andrew Mayorov d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
Ivan Dyachkov 7c0e345d3a Merge remote-tracking branch 'upstream/release-54' 2023-11-14 19:38:21 +01:00
Andrew Mayorov 03843c6071
feat(router): switch to v2 routing store by default 2023-11-13 17:43:30 +07:00
JimMoen 4d493292fb
feat(gw_jt808): port jt808 gateway from 4.4 2023-11-12 21:44:18 +08:00
JimMoen 25e277bdc6
chore: fix schema desc and README.md typo 2023-11-12 21:43:10 +08:00
Zaiming (Stone) Shi ff5916e33f docs: fix bad html tag 2023-11-12 10:31:37 +01:00
Thales Macedo Garitezi b4e45f0189
Merge pull request #11923 from thalesmg/sync-m-to-r54-20231109
chore: sync `master` to `release-54`
2023-11-10 13:04:55 -03:00
firest 447c933ff8 fix(syskeeper): homogenize the Syskeeper bridge and update license data 2023-11-10 10:42:53 +08:00
firest 548e39e799 refactor(sysk): refactor the syskeeper to v2 style 2023-11-10 09:52:38 +08:00
firest ce83079c6b feat(sysk): integrated Nari Syskeeper 2000 as a new bridge backend 2023-11-10 09:51:43 +08:00
JianBo He 6500d21d98
Merge pull request #11885 from HJianBo/ocpp-gw
feat: Port OCPP gateway from v4
2023-11-10 09:32:40 +08:00
Thales Macedo Garitezi 7a237f6a3a Merge branch 'master' into sync-m-to-r54-20231109 2023-11-09 18:49:52 -03:00
Thales Macedo Garitezi 371a49304d Merge branch 'release-53' into sync-r53-to-m-20231109 2023-11-09 17:19:23 -03:00
Thales Macedo Garitezi eabd09051a feat(actions_api): add `/action_types` API 2023-11-09 13:54:57 -03:00
firest 2b97800c8b fix(limiter): simplify the descriptions for fields of the limiter 2023-11-09 18:00:09 +08:00
Ivan Dyachkov 0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
JianBo He 4b18631d49 chore: fix gateway failed test cases 2023-11-09 13:44:20 +08:00
JianBo He 2d958beae9 chore: try to fix spellcheck 2023-11-09 09:51:08 +08:00
Ivan Dyachkov 0c91bec98d chore: merge 'upstream/release-53' 2023-11-08 09:24:38 +01:00
Stefan Strigler fa740a1927 docs: fix inconsistencies for bridge and connector operations
It's only `start` allowed for now. Also fixed some wordings.
2023-11-07 13:58:23 +01:00
Zaiming (Stone) Shi f19904d43c
Merge pull request #11877 from zmstone/1102-rename-kafka-to-config
Rename connector channel related configs in bridge_v2 to 'parameters'
2023-11-07 13:41:55 +01:00
JianBo He 7bd5579980 chore: fix the documentation generation and example conf checking 2023-11-07 11:09:39 +08:00
Zaiming (Stone) Shi 9a26c03a5c refactor: unify top level field names for bridge v2 2023-11-06 21:45:08 +01:00
Thales Macedo Garitezi 2b8cf50a1d chore: rename `bridges_v2` -> `actions` in the public facing APIs
Fixes https://emqx.atlassian.net/browse/EMQX-11330

After feedback from Product team, we should rename `bridges_v2` to `actions` everywhere.
We'll start with the public facing APIs.

- HTTP API
- Hocon schema root key
2023-11-06 15:37:07 -03:00
JianBo He 8871738756 chore: ensure emqx_gateway_ocpp tests pass 2023-11-06 12:26:58 +08:00
JianBo He 7cab269e0b feat: port the ocpp gateway from version 4 2023-11-03 16:51:21 +08:00
Zaiming (Stone) Shi afe6f79a66 feat(bridge): add description field to bridge and connector 2023-11-03 06:53:42 +01:00
Thales Macedo Garitezi 4bea65bf97 fix(bridge_v2_api): don't disable schema check 2023-11-01 12:44:40 -03:00
lafirest 17544dc410
Merge pull request #11852 from lafirest/feat/gbt_gw
feat(gbt32960): Port the GBT32960 gateway from v4
2023-11-01 21:03:18 +08:00
firest 30a72f557f fix(gateway): improve gateway schema modules
1. enhances the gateway name as an enum
2. make the schema more flexible and extensible without some hardcode
2023-11-01 18:55:36 +08:00
firest 3ec811e828 feat(gbt32960): Port the GBT32960 gateway from v4 2023-10-31 18:25:43 +08:00
Ivan Dyachkov b1ab213081 chore: merge 'upstream/release-53' into 1031-sync-r53 2023-10-31 11:06:25 +01:00
Kjell Winblad 9dc3a169b3 feat: split bridges into a connector part and a bridge part
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
Co-authored-by: Stefan Strigler <stefan.strigler@emqx.io>
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>

Several bridges should be able to share a connector pool defined by a
single connector. The connectors should be possible to enable and
disable similar to how one can disable and enable bridges. There should
also be an API for checking the status of a connector and for
add/edit/delete connectors similar to the current bridge API.

Issues:
https://emqx.atlassian.net/browse/EMQX-10805
2023-10-30 14:48:47 +01:00
zhongwencool e896957803
Merge pull request #11824 from zhongwencool/audit-log-reviewed
refactor: audit log review refactor
2023-10-30 17:51:15 +08:00
zhongwencool 0634ff61c0 fix: dialyzer warning 2023-10-30 09:57:30 +08:00
lafirest e63602c4b0
Merge pull request #11811 from lafirest/fix/api_rbac
fix(rbac): for compatibility with old data schema, extend the existing field as an extra
2023-10-27 17:56:54 +08:00
firest 4ba34f8f3e chore(rbac): fix CI errors & update change 2023-10-27 17:17:04 +08:00
zhongwencool 22223dc536 fix: return 400 when audit log is disabled 2023-10-27 16:11:30 +08:00
zhongwencool 995948f0e8 refactor: remove seq from audit record 2023-10-27 14:07:19 +08:00
firest cbfd02d1b0 refactor(ldap): merge the `ldap-bind` backend as a type for the `ldap` backend 2023-10-26 11:04:44 +08:00
zhongwencool ef692596f7 fix: don't crash when 401 and 403 unauthorize 2023-10-20 15:22:30 +08:00
zhongwencool 1d7aa9495a chore: make spellcheck happy 2023-10-20 14:47:24 +08:00
zhongwencool 141061c1e2 feat: add max_filter_size and ignore_high_frequency_request config 2023-10-20 14:47:24 +08:00
zhongwencool 926c804314 feat: add /audit http api to filter audit log 2023-10-20 14:47:24 +08:00
lafirest 74442b0d31
Merge pull request #11766 from lafirest/feat/rbac
feat(api_key): add RBAC feature for the API key
2023-10-20 09:24:41 +08:00
firest e095de7367 feat(api_key): add RBAC feature for the API key 2023-10-16 14:10:01 +08:00
Ilya Averyanov 4f14e8df22 chore(source dir): split out postgresql connector 2023-10-13 16:58:57 +03:00
Zaiming (Stone) Shi e7e696cd66 feat(rpc): add ipv6_only config 2023-10-10 10:05:14 +02:00
Zaiming (Stone) Shi 497e08448d feat(cluster): support ipv6 and tls on ipv6 for clustering
Made possible to configure inet6_tls for Erlang distribution
Also, added support to configure ipv6 listener for gen_rpc
2023-10-09 20:03:21 +02:00
Ilya Averyanov 1eb75b43c4 chore(auth): split emqx_authn and emqx_authz apps 2023-10-05 13:41:50 +03:00
firest 08ad09a68f fix(sso): refactor backen update logic
1. valid config always can update successfully
2. the `running` endpoint only return successfully created backend
3. enhancement of the `/sso` endpoint, and will check is the resource online
2023-09-27 20:53:10 +08:00
Zaiming (Stone) Shi 4e15edb5e4
Merge pull request #11661 from zmstone/0922-fix-json-log-formatter
0922 fix json log formatter
2023-09-26 14:13:41 +02:00
Zaiming (Stone) Shi 40608f23c8 docs: use code html tag for config value 'json' 2023-09-26 12:10:24 +02:00
Thales Macedo Garitezi 5d212e1086 fix(audit): only support audit log on enterprise edition
Fixes https://emqx.atlassian.net/browse/EMQX-11039
2023-09-25 15:59:35 -03:00
Zaiming (Stone) Shi 5e6996dc05 refactor: log formatter format mfa+line as m:f/a(line)
also improve json formatter when a field is iolist
2023-09-25 16:55:22 +02:00
Zaiming (Stone) Shi 1fed38c248 fix(logger): write 'json' format logs as JSON 2023-09-25 16:55:22 +02:00
firest 4f4868a46c fix(sso): Disable access to `logout` endpoint by the `API key` 2023-09-25 10:31:38 +08:00
JimMoen c9e0d4fc30
feat: saml integration for dashboard sso 2023-09-22 21:49:10 +08:00
firest 7286d15ca6 chore(sso): adjust the schema of the SSO LDAP backend 2023-09-22 13:48:20 +08:00
zhongwencool beea1be9f0
Merge pull request #11599 from zhongwencool/audit-log 2023-09-21 18:39:23 +08:00
zhongwencool 21bb209fb1 feat: added support for auditing API and CLI activity in logs 2023-09-21 16:35:46 +08:00