Commit Graph

1880 Commits

Author SHA1 Message Date
ieQu1
ba43da1b8e
fix(mria): Strictly check compatibility of the replicant with core 2024-03-26 16:32:01 +01:00
zmstone
874b1ff3f4 docs: fix epoch timestamp precision 2024-03-26 16:07:55 +01:00
zmstone
c42e980442 feat(log): configurable time format
now logs can be configured to use 'epoch' or 'rfc3339' time format
2024-03-26 13:09:32 +01:00
Ivan Dyachkov
2ae973821f chore: 5.6.0-rc.4 2024-03-25 20:00:10 +01:00
Zaiming (Stone) Shi
b06d3932af
Merge pull request #12768 from keynslug/fix/EMQX-12068/routing-schema-conflict
fix(routing): add heuristic for routing schema in empty cluster
2024-03-25 19:50:51 +01:00
ieQu1
8b963d5960
Merge pull request #12739 from ieQu1/dev/zone-durability
feat(sessds): Add zone overrides for session durability settings
2024-03-25 17:32:53 +01:00
Zaiming (Stone) Shi
c8cb03f0d9
Merge pull request #12773 from zmstone/0323-upgrade-http-clients
chore: upgrade http client libs gun-1.3.11 and ehttpc-0.4.13
2024-03-25 17:29:44 +01:00
zmstone
d32a1a892a docs: add change log 2024-03-25 16:56:48 +01:00
Ivan Dyachkov
fe3cc25855 chore: update instructions for routing schema conflict 2024-03-25 15:22:43 +01:00
Ivan Dyachkov
eb9e3aa9e6 chore: update changelog entry to include manual resolution steps 2024-03-25 12:00:06 +01:00
JimMoen
3e97e5528e
Merge pull request #12765 from JimMoen/EMQX-12043/subscribers-count
fix(stats): `'subscribers.count'` contains shared-subscriber
2024-03-25 17:19:13 +08:00
JimMoen
5f7f9e43f9
fix(stats): 'subscribers.count' contains shared-subscriber 2024-03-25 15:12:57 +08:00
zmstone
df913f2450 docs: refine changelog 2024-03-23 14:11:30 +01:00
zmstone
d7b54b3a53 docs: add changelog for pr #12773 2024-03-23 10:41:46 +01:00
zmstone
5e9814d171 fix: add debug level logging for invalid client attributes 2024-03-23 10:16:05 +01:00
zmstone
4f71c97854 docs: add changelog for PR #12750 2024-03-23 10:16:02 +01:00
Ivan Dyachkov
0210f83f06 chore: update changelog 2024-03-22 16:36:45 +01:00
Ivan Dyachkov
be1886fb91 chore: make dyalizer happy and update changelog 2024-03-22 15:56:44 +01:00
Andrew Mayorov
a0f500d1d0
chore: add changelog entry 2024-03-22 15:21:03 +01:00
Thales Macedo Garitezi
bc5775a988 fix(kafka_producer): handle ancient v1 config when migrating to actions
Fixes https://emqx.atlassian.net/browse/EMQX-12064
2024-03-22 09:45:27 -03:00
Serge Tupchii
d2a1a7f7cf chore: rename message_queue_too_long error reason to mailbox_overflow
`mailbox_overflow` is consistent with the corresponding config parameter:
 'force_shutdown.max_mailbox_size'
2024-03-22 12:20:20 +02:00
Ivan Dyachkov
93c80bc912 chore: 5.6.0-rc.3 2024-03-21 18:10:59 +01:00
Serge Tupchii
40eccb10d6 fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid 2024-03-21 17:36:39 +02:00
Ivan Dyachkov
49e0488498 chore: 5.6.0-rc.2 2024-03-20 17:11:37 +01:00
Zaiming (Stone) Shi
bede5a5b85
Merge pull request #12746 from zmstone/0320-add-throttle-log-for-auth-failure
feat: add authentication_failure throttled log
2024-03-20 17:06:36 +01:00
zmstone
90fce21adc docs: add change log for #12746
also updated #12520 to include newly added throttled log id
2024-03-20 14:43:28 +01:00
Serge Tupchii
cb5fdb3c79 fix: rework In-flight / Mqueue API
- use timestamp 'position' to find the next chunk of data
- add 'start' position to response meta
- sort In-flight messages by insertion time
- sort Mqueue messages by priority
2024-03-20 15:25:33 +02:00
ieQu1
0547b32727
feat(sessds): Add zone overrides for session durability settings 2024-03-19 22:51:06 +01:00
ieQu1
611e4f6710
Merge pull request #12740 from ieQu1/dev/kick-durable-sessions
fix(sessds): Channel must destroy sessions that are kicked
2024-03-19 21:30:40 +01:00
Ivan Dyachkov
923fd0a2a6
Merge pull request #12737 from id/0319-sync-release56
sync release-56
2024-03-19 19:09:41 +01:00
ieQu1
236685f714 fix(sessds): Channel must destroy sessions that are kicked 2024-03-19 17:49:04 +01:00
Ivan Dyachkov
f2dc940436 Merge remote-tracking branch 'upstream/release-56' into 0319-sync-release56 2024-03-19 15:20:08 +01:00
Thales Macedo Garitezi
a689ae72e3
Merge pull request #12711 from thalesmg/data-validation-m-20240311
feat: implement message validation
2024-03-19 11:12:28 -03:00
SergeTupchiy
92fd0e453a
Merge pull request #12719 from SergeTupchiy/EMQX-11900-multiple-clientid-username-clients-API
Support multiple clientid/username Qs params in clients API
2024-03-19 14:44:39 +02:00
zhongwencool
d7c01a4fe0 fix: cant replace source conf 2024-03-19 18:53:51 +08:00
Thales Macedo Garitezi
f84a996671 feat: implement message validation
Fixes https://emqx.atlassian.net/browse/EMQX-11980
2024-03-18 13:11:39 -03:00
Serge Tupchii
b3d44125f6 feat: support multiple clientid / username Qs params in "/clients" API
Additionally, add an option to specify which Client info fields to return in the response.
2024-03-18 18:00:26 +02:00
JimMoen
871c8e7790
docs: add changelog for PR 12714 2024-03-16 20:43:06 +08:00
ieQu1
a93f747afa fix(sessds): Return peername of the disconnected client in the REST 2024-03-15 13:48:53 +01:00
Ivan Dyachkov
ef1ce35e00 chore: add changelog 2024-03-14 20:41:01 +01:00
Thales Macedo Garitezi
e340ab7f1b
Merge pull request #12696 from thalesmg/fix-default-enable-r56-20240313
fix(bridges): fix default value for `enable` when attempting operations
2024-03-14 09:50:40 -03:00
Zaiming (Stone) Shi
cb77dea1e9
Merge pull request #12699 from zmstone/0313-fix-session-count-on-replicant-node
fix: do not crash on replicant node
2024-03-14 09:42:27 +01:00
Zaiming (Stone) Shi
2347b9fdd4
Merge pull request #12674 from zmstone/0308-minor-refactors
0308 minor refactors
2024-03-14 09:39:36 +01:00
Serge Tupchii
685f29d3f2 chore: upgrade hocon to 0.42.1
hocon 0.42.1 allows to use "b" or "B" (byte) unit in bytesize fields.
2024-03-13 19:30:28 +02:00
zmstone
ccd973d13e fix: do not crash on replicant node 2024-03-13 16:42:39 +01:00
Kjell Winblad
9d21372ff0
Merge pull request #12678 from kjellwinblad/kjell/dynamo_bridge/fix/error_log/EMQX-11934
fix: return error reason from dynamo connector status check
2024-03-13 15:27:08 +01:00
Thales Macedo Garitezi
b587ebac0c fix(bridges): fix default value for enable when attempting operations
Fixes https://emqx.atlassian.net/browse/EMQX-11999

Although the frontend doesn't send `enable` when creating a bridge/action/connector, the
default value in the schema is `true`, so when attempting to fetch it from the raw config
we should adhere to that default.
2024-03-13 11:25:22 -03:00
Kjell Winblad
9d999920be
Merge pull request #12681 from kjellwinblad/kjell/rocketmq_conn/remove_secret_from_logs/EMQX-11987
fix: redact secrets from RocketMQ debug log
2024-03-13 11:39:47 +01:00
JimMoen
d723a5edf7
Merge pull request #12670 from JimMoen/feat-shared-sub-count-monitor-current
feat(api): field `shared_subscriptions` in endpoint `/monitor_current`
2024-03-13 17:09:30 +08:00
zmstone
9357e6610a docs: add changelog for PR 12672 2024-03-13 09:56:23 +01:00
Kjell Winblad
f502ba87df docs: add change log entry 2024-03-12 10:32:25 +01:00
Kjell Winblad
29e0ddefbc docs: add change log entry 2024-03-12 10:22:19 +01:00
JimMoen
c0552d8877
docs: add changelog for PR 12663 2024-03-12 14:14:47 +08:00
JimMoen
e5b818e1a2
docs: add changelog for PR 12670 2024-03-11 17:44:37 +08:00
Kjell Winblad
69ddd51af1 docs: add change log entry 2024-03-08 12:35:39 +01:00
Kjell Winblad
060e02c4c4
Merge pull request #12653 from kjellwinblad/kjell/rule_engine/fix/subbits/support_non_byte_sizes/EMQX-11943
fix(rule_engine): support non-byte sized input to bin2hexstr
2024-03-08 09:41:03 +01:00
Kjell Winblad
6054499607
Merge pull request #12657 from kjellwinblad/kjell/rule_engine/fix/fun_calls_as_array_items/EMQX-11953
fix(rule SQL): allow expressions as array items
2024-03-08 09:39:46 +01:00
Shawn
163d095dca fix: port the changes for date_to_unix_ts SQL fun from 4.4 2024-03-08 10:48:08 +08:00
SergeTupchiy
1f38813cb9
Merge pull request #12561 from SergeTupchiy/EMQX-11861-client-mqueue-inflight-API
feat: add client mqueue/inflight messages API
2024-03-07 13:20:31 +02:00
Serge Tupchii
a2e761681e feat: add client mqueue/inflight messages API 2024-03-07 12:06:49 +02:00
zmstone
6a2731f2da Merge remote-tracking branch 'origin/master' into release-56 2024-03-07 09:36:37 +01:00
Kjell Winblad
59f9f6520d
docs: fix typos in changelog entry
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-03-07 09:28:57 +01:00
zmstone
9cbeb3720a Merge remote-tracking branch 'origin/release-55' into sync-5.5.1 2024-03-06 17:55:14 +01:00
Thales Macedo Garitezi
dc16e59f2c fix(gcp_pubsub_producer): check for topic existence when creating action
Fixes https://emqx.atlassian.net/browse/EMQX-11949
2024-03-06 13:43:50 -03:00
zmstone
e99546e009 Merge remote-tracking branch 'origin/release-56' into sync-5.5.1 2024-03-06 17:27:54 +01:00
Kjell Winblad
2fcd92dcd9 docs: add changelog entry 2024-03-06 15:52:42 +01:00
Kjell Winblad
cd3ecc4ad5 docs: add change log entry 2024-03-06 15:26:52 +01:00
Kjell Winblad
78d5f76f70
Merge pull request #12652 from kjellwinblad/kjell/rule_engine/fix/subbits/EMQX-11942
fix: add subbits/4 and subits/5 rule_engine functions
2024-03-06 15:20:30 +01:00
Zaiming (Stone) Shi
fc8b5d4522
Merge pull request #12646 from zmstone/0304-rule-engin-func-fix-zone-shift-in-date-string-parse
fix(rule_func): time zone shift at wrong precision
2024-03-06 13:22:30 +01:00
JianBo He
1737df05c7
Merge pull request #12641 from zmstone/0304-improve-text-log-formatter
0304 improve text log formatter
2024-03-06 09:12:18 +08:00
Kjell Winblad
f0aecaf16f docs: add changelog text for new rule engine subbits functions 2024-03-05 17:15:04 +01:00
Thales Macedo Garitezi
c5489fee90 Merge remote-tracking branch 'origin/release-56' into sync-r56-m-20240304 2024-03-05 09:08:59 -03:00
Ivan Dyachkov
acb6b5a0d2 chore: 5.5.1-rc.4 2024-03-05 12:04:34 +01:00
zmstone
f275e99aba docs: add changelog for PR 12641 2024-03-05 09:17:00 +01:00
Ivan Dyachkov
2251b85d65 chore: 5.5.1-rc.3 2024-03-05 08:41:44 +01:00
zmstone
9929025820 fix(rule_func): time zone shift at wrong precision 2024-03-04 21:25:49 +01:00
zmstone
3e3194fd21 chore: upgrade to hocon 0.42.0 2024-03-04 19:12:42 +01:00
Thales Macedo Garitezi
577ef41e45 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 12:48:06 -03:00
Ivan Dyachkov
d52008fb55 chore: update changelog 2024-03-04 16:44:12 +01:00
Ivan Dyachkov
7780b4f1d7 chore: prepare 5.5.1 release 2024-03-04 15:15:10 +01:00
firest
6d9b2a95c3 Merge remote-tracking branch 'origin/release-55' 2024-03-04 20:47:20 +08:00
Thales Macedo Garitezi
0dd9990f24 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 09:44:30 -03:00
ieQu1
2e792da934
Merge pull request #12562 from ieQu1/dev/new-ds-schema
Refactor schema for the durable storage
2024-03-04 10:33:27 +01:00
JimMoen
441b6f7a0d
docs: MS SQL Server bridge v2 change log 2024-03-04 17:14:05 +08:00
JimMoen
0a33f9d027
fix(calendar): leap year time to unix timestamp 2024-03-04 15:37:49 +08:00
Zaiming (Stone) Shi
e9a5670cce
Merge pull request #12581 from zmstone/0223-add-json-schema
feat: schema registry supports json schema
2024-03-02 10:09:31 +01:00
firest
1c1c4e497d chore: bump version && update change 2024-03-01 12:42:19 +08:00
zhongwencool
d56fb22208
Merge pull request #12595 from thalesmg/kafka-consumer-source-m-20240223
feat: migrate kafka consumer bridge to source + connector
2024-03-01 10:11:16 +08:00
Zaiming (Stone) Shi
5df1784b49 docs: add changelog for PR 12581 (JSON schema) 2024-02-29 21:07:56 +01:00
William Yang
b82973b36b
Merge pull request #11868 from qzhuyan/fix/william/takeover-pub-willmsg
fix: willmsg not published in takeover
2024-02-29 13:14:18 +01:00
lafirest
d2dda2535b
Merge pull request #12602 from lafirest/fix/iotdb_ping
fix(iotdb): use the `ping` API to check the status of the IOTDB connections
2024-02-29 12:26:10 +08:00
firest
0bebd66f05 fix(iotdb): make dialyzer happy 2024-02-29 10:51:12 +08:00
lafirest
d3fb72733e
Merge pull request #12610 from lafirest/fix/ldap_bind_health
fix(ldap): fixed that the connection to the LDAP connector could be disconnected after a period of time
2024-02-28 23:03:10 +08:00
lafirest
eaef753c36
Merge pull request #12608 from lafirest/fix/iotdb
fix(iotdb): fix function clause error when there is no `payload` field
2024-02-28 22:00:54 +08:00
firest
239fd81661 chore: update change 2024-02-28 21:58:03 +08:00
firest
ce2552712f chore: bump version && update change 2024-02-28 18:32:08 +08:00
JimMoen
0e12503a8d
chore: add fix change log 2024-02-28 15:25:54 +08:00
firest
c3a2cf6220 chore: update change && bump version 2024-02-28 13:18:48 +08:00
Thales Macedo Garitezi
6b9844ae82 feat: migrate kafka consumer bridge to source + connector
Fixes https://emqx.atlassian.net/browse/EMQX-11848
2024-02-27 17:52:33 -03:00
Thales Macedo Garitezi
787dcd0e86
Merge pull request #12577 from thalesmg/gcp-service-json-binary-m-20240223
feat(gcp_pubsub_bridges): make service account json a binary
2024-02-27 15:42:42 -03:00
lafirest
02de92a53d
Merge pull request #12601 from lafirest/fix/eldap_log
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 21:50:05 +08:00
firest
37026fa94f chore: update change 2024-02-27 21:03:53 +08:00
JimMoen
c1928f874e
chore: add change log 2024-02-27 16:01:30 +08:00
Zaiming (Stone) Shi
bb6a95bae8
Merge pull request #12590 from zmstone/0226-remove-mfa-from-logs
refactor: delete mfa from log metadata
2024-02-26 21:37:57 +01:00
Zaiming (Stone) Shi
e96886b612 docs: add changelog for PR 12590 2024-02-26 18:45:06 +01:00
Serge Tupchii
5d4c85cd84 chore: list throttled log events in the change-log 2024-02-26 15:09:38 +02:00
Thales Macedo Garitezi
f2c372d9ff feat(gcp_pubsub_bridges): make service account json a binary
Fixes https://emqx.atlassian.net/browse/EMQX-11384

Today, service_account_json config field is an embedded object (map()).

This requires user to embed a JSON object into the config file instead of embedding it as
a string.

We should support binary() type as input, but keep supporting map() for backward
compatibility.
2024-02-26 09:21:07 -03:00
ieQu1
e126393cbf
fix(ds): Fix schema checker warnings for DS schema 2024-02-23 15:39:52 +01:00
Zaiming (Stone) Shi
b3c7901cf2 docs: add changelog for 12576 2024-02-23 15:11:08 +01:00
lafirest
9f52b4fba8
Merge pull request #12566 from lafirest/fix/same_api
fix(api_key): enhanced bootstrap files for REST API keys
2024-02-23 10:57:37 +08:00
William Yang
07eec31a8a fix: willmsg not published in takeover 2024-02-22 16:35:59 +01:00
firest
26977fef9c chore: update changes 2024-02-22 18:54:50 +08:00
Kjell Winblad
72d340bd4b docs: add change log entry for DynamoDB bridge refactoring 2024-02-21 14:59:03 +01:00
Thales Macedo Garitezi
529211b9ac
Merge pull request #12500 from thalesmg/ds-list-client-api-m-20240209
feat(ds): list disconnected persistent sessions in clients API
2024-02-21 09:02:27 -03:00
Thales Macedo Garitezi
74dbbf6d9d Merge branch 'release-55' into sync-r55-m-20240220 2024-02-20 16:33:34 -03:00
Thales Macedo Garitezi
3a4c7f60e2 feat(ds): list disconnected persistent sessions in clients API
Fixes https://emqx.atlassian.net/browse/EMQX-11540

Note that not all information provided by disconnected in-memory sessions is available to
disconnected persistent sessions, nor does all of them make sense.
2024-02-20 14:52:55 -03:00
Zaiming (Stone) Shi
1de9e9dd01
Merge pull request #12541 from zmstone/0220-fail-fqdn-node-name-if-dns-discovery
Fail fast if node host is not IP address but cluster strategy is `dns` with `a` or `aaaa` record type
2024-02-20 17:48:02 +01:00
Thales Macedo Garitezi
b6689d178f fix(http_bridge): redact headers
Fixes https://emqx.atlassian.net/browse/EMQX-11864

```
2024-02-20T15:51:34.744509+00:00 [debug] msg: http_connector_received, mfa: emqx_bridge_http_connector:on_query_async/4(471), tag: QUERY_ASYNC, request: {"/bah",[{<<"Authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>},{<<"authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>}],<<"******">>}, state: #{port => 18083,request => #{path => [<<>>],body => undefined,headers => [{[<<"content-type">>],[<<"application/json">>]},{[<<"Authorization">>],#Fun<emqx_secret.0.85191762>}],method => [<<"undefined">>],max_retries => 2,request_timeout => 30000},host => "localhost",connect_timeout => 15000,pool_type => random,pool_name => <<"connector:http:p1">>,base_path => "/",installed_actions => #{<<"action:http:p1:connector:http:p1">> => #{path => [<<"/bah">>],body => undefined,headers => [{[<<"authorization">>],#Fun<emqx_secret.0.85191762>},{[<<"content-type">>],[<<"application/json">>]}],method => [<<"post">>],max_retries => 2,request_timeout => 30000,render_template_func => fun emqx_bridge_http_connector:render_template/2}}}, connector: <<"connector:http:p1">>, note: the request body is redacted due to security reasons
```
2024-02-20 12:57:13 -03:00
Zaiming (Stone) Shi
dc015e7a6f fix(schema): validate cluster strategy and node name
If cluster strategy is configured as `dns`, the node name
must be IP address
2024-02-20 16:09:54 +01:00
SergeTupchiy
f72558737c
Merge pull request #12520 from SergeTupchiy/EMQX-11530-log-throttling
feat: implement log throttling
2024-02-20 15:52:46 +02:00
Kjell Winblad
86c85f3b45
Merge pull request #12512 from kjellwinblad/kjell/refactor/hstreamdb/EMQX-11458
feat: refactor HStreamDB bridge to connector and action
2024-02-20 13:30:29 +01:00
Serge Tupchii
98ba300f7c feat: implement log throttling 2024-02-20 13:23:17 +02:00
Thales Macedo Garitezi
dbb6725a47 fix(config): apply config upgrade to deprecated configs
Fixes https://emqx.atlassian.net/browse/EMQX-11845

Fixes https://github.com/emqx/emqx/issues/12452
2024-02-19 09:02:43 -03:00
Zaiming (Stone) Shi
866fea7648
Merge pull request #12530 from zmstone/0217-improve-frame-too-large-logging-context
0217 improve frame too large logging context
2024-02-19 11:33:31 +01:00
Zaiming (Stone) Shi
29c5c37f82 docs: add changelog for PR 12530 2024-02-18 19:30:35 +01:00
Shawn
ec888bc87f ci: fix test cases for clickhouse 2024-02-18 10:02:02 +08:00
Shawn
8ae0e78786 refactor: split clickhouse bridges to actions and connectors 2024-02-18 10:01:53 +08:00
Zaiming (Stone) Shi
2b46cbab7a
Merge pull request #12517 from zmstone/0213-hocon-multiline-string-with-indentation
0213 hocon multiline string with indentation
2024-02-16 15:12:46 +01:00
Zaiming (Stone) Shi
242df0ffe9 chore: upgrade kafka_protocol from 4.1.3 to 4.1.5 2024-02-16 10:33:53 +01:00
Zaiming (Stone) Shi
b24321a8fc feat: upgrade to hocon-0.41.0
hocon 0.41.0 added support for multiline string indentation.
now there is no need to escape (unless there is backslash)
quotes and line feeds etc.
2024-02-16 10:32:13 +01:00
Kjell Winblad
0bf7a121e7 docs: add change log entry for HStreamDB bridge refactoring 2024-02-15 13:27:26 +01:00
Serge Tupchii
0d0e26d6af fix: set info level for potentially flooding log events 2024-02-15 11:02:03 +02:00
Serge Tupchii
7272ef25d4 feat(emqx_auth): implement API to re-order all authenticators/authz sources
Fixes: EMQX-11770
2024-02-14 14:35:46 +02:00
Kjell Winblad
366827390e
Merge pull request #12488 from kjellwinblad/kjell/refactor/rocketmq_bridge/EMQX-11467
feat: refactor RocketMQ bridge to connector and action
2024-02-13 16:19:21 +01:00
Ilya Averyanov
edd28be090
Merge pull request #12499 from savonarola/0209-emqx-banned-extended-rules
feat(banned): allow ban by clientid/username regexps, peerhost cidrs
2024-02-12 17:12:47 +02:00
Ilya Averyanov
8a4c66dc75 feat(banned): add warning about performance impact of matching rules 2024-02-12 12:56:03 +03:00
Zaiming (Stone) Shi
d4fb3e83f6 chore: upgrade to wolff 1.10.2
Wolff 1.10.2 added an enhancement which makes Kafka resource health
checks more lightweight and lowers the chance of false resource alerts.
2024-02-12 09:05:44 +01:00
Ilya Averyanov
90fd2b26d3 feat(banned): allow ban by clientid/username regexps, peerhost cidrs 2024-02-10 17:59:22 +03:00
Serge Tupchii
c3b7d76102 perf: un-deprecate mongodb topology pool_size and set it to 10 by default
Benchmarks proved that it effects performance:
10 ecpool workers / 10 topology pool workers outperform
100 ecpool / 1 topology worker.

Fixes: EMQX-11825
2024-02-09 16:28:53 +02:00
SergeTupchiy
755b59b7fe
Merge pull request #12494 from SergeTupchiy/EMQX-11825-mongodb-connector-perf
Improve mongodb connector performance
2024-02-09 13:12:12 +02:00
Zaiming (Stone) Shi
cd3cc41f90 docs: add change log for PR #12492 2024-02-09 10:49:51 +01:00
Serge Tupchii
1d9ce709f4 perf: un-deprecate mongodb topology pool_size and set it to 10 by default
Benchmarks proved that it effects performance:
10 ecpool workers / 10 topology pool workers outperform
100 ecpool / 1 topology worker.

Fixes: EMQX-11825
2024-02-08 20:39:19 +02:00
Kjell Winblad
511d1f732a docs: add change log entry for RocketMQ bridge refactoring 2024-02-08 17:50:10 +01:00
zhongwencool
eace8f2fcb
Merge pull request #12423 from zhongwencool/rabbitmq-e560
feat: rabbitmq bridge v2 integration
2024-02-06 09:55:05 +08:00
zhongwencool
d65e84e46b chore: add rabbitmq v2 changelog 2024-02-06 07:57:13 +08:00
Zaiming (Stone) Shi
d7b0456bb0 feat: rename emqx ctl cluster_sync 'tnxid' to 'inspect' 2024-02-05 16:50:55 +01:00
Thales Macedo Garitezi
e5b14a38ed
Merge pull request #12472 from thalesmg/fix-bridgev2-api-multicall-m-20240202
fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
2024-02-05 10:28:28 -03:00
Thales Macedo Garitezi
e1ff276d76
Merge pull request #12471 from thalesmg/fix-config-upgrade-m-20240202
fix(config): apply config upgrade to deprecated configs
2024-02-05 09:05:12 -03:00
Thales Macedo Garitezi
76d242df9b fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
Fixes https://emqx.atlassian.net/browse/EMQX-11834
2024-02-02 17:36:08 -03:00
Thales Macedo Garitezi
76401a302a fix(config): apply config upgrade to deprecated configs
Fixes https://emqx.atlassian.net/browse/EMQX-11845

Fixes https://github.com/emqx/emqx/issues/12452
2024-02-02 15:12:08 -03:00
Ivan Dyachkov
ed8660cb68
Merge pull request #12458 from id/0202-sync-release-55
sync release-55
2024-02-02 17:20:38 +01:00
SergeTupchiy
81433282be
Merge pull request #12467 from SergeTupchiy/autocluster-dns-aaaaa-record-type
feat(emqx_conf): support AAAA cluster dns record type
2024-02-02 18:13:06 +02:00
Zaiming (Stone) Shi
b1a05c7b59
Merge pull request #12326 from zmstone/0111-unregister-session-with-timestamp
0111 unregister session with timestamp
2024-02-02 17:00:23 +01:00
Serge Tupchii
5e0c3d4051 feat(emqx_conf): support AAAA cluster dns record type
Ekka autocluster DNS already supports AAAA records, so it's only needed to update enum in schema.
2024-02-02 12:13:29 +02:00
Kjell Winblad
fa0baf2f38
Merge pull request #12439 from kjellwinblad/kjell/refactor/oracle_bridge/EMQX-11464
feat: refactor Oracle bridge to connector and action
2024-02-02 10:53:15 +01:00
lafirest
24d8a9ab00
Merge pull request #12449 from lafirest/feat/td
feat(tdengine): improve the TDengine bridge to v2 style
2024-02-02 16:17:01 +08:00
Ivan Dyachkov
dd490de2e1 chore: emqx 5.5.0 2024-02-02 08:48:56 +01:00
firest
2241461acb chore: update change & bump version 2024-02-02 07:48:44 +08:00
Kjell Winblad
ee35ecb04d docs: add change log entry for Oracle bridge refactoring 2024-02-01 16:08:36 +01:00
Zaiming (Stone) Shi
3e518c1876
Merge pull request #12427 from zmstone/0130-limit-kafka-partitions
0130 limit kafka partitions
2024-02-01 14:39:52 +01:00
Ivan Dyachkov
3ad71f75e6 chore: emqx 5.5.0 2024-02-01 11:07:02 +01:00
Zaiming (Stone) Shi
3b42a7425b feat(kafka_producer): add partitions_limit 2024-01-31 19:26:01 +01:00
lafirest
a60aeed539
Merge pull request #12398 from lafirest/feat/api-docs
feat(dashboard): expose the `swagger_support` option
2024-01-31 23:31:56 +08:00
Zaiming (Stone) Shi
330baa6cc3 docs: add changelog for #12326 2024-01-31 09:50:12 +01:00
firest
13df7fa46e fix(dashboard): add test case & update change 2024-01-31 14:14:56 +08:00
Zaiming (Stone) Shi
55fdc83aa5 Merge remote-tracking branch 'origin/release-55' into 0130-syn-release-55-to-master 2024-01-30 12:06:43 +01:00
Xinyu Liu
db3f285054
Merge pull request #12417 from emqx/port-msg-expiry-interval-from44
feat: configurable server side message_expiry_interval
2024-01-30 17:10:32 +08:00
Shawn
30508d833c chore: add change logs for message-expiry-interval 2024-01-30 15:46:55 +08:00
JianBo He
4688b36cdf
Merge pull request #12396 from HJianBo/import_user_2
feat(import_users): support user's password in plain text
2024-01-30 09:00:00 +08:00
Thales Macedo Garitezi
47f61ba68a fix(cassandra_bridge): correctly insert null values into columns
Fixes https://emqx.atlassian.net/browse/EMQX-11822
2024-01-29 11:32:00 -03:00
JianBo He
ded1b4d8b3
chore: apply suggestions from code review
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2024-01-29 18:24:29 +08:00
JianBo He
bcbd48ae58 chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-29 10:49:07 +08:00
JianBo He
4c1f1d97cd chore: update changes 2024-01-29 10:49:07 +08:00
Zaiming (Stone) Shi
dbe2baa405
Merge pull request #12408 from zmstone/sync-release-55
Sync release 55
2024-01-27 19:20:58 +00:00
ieQu1
3c451c6ae6 test(sessds): Fix unstable tests 2024-01-26 17:49:33 +01:00
ieQu1
f5b9bd30aa fix(sessds): Apply review remarks 2024-01-26 17:49:33 +01:00
Thales Macedo Garitezi
6410f5a717 fix(resource_metrics): avoid detaching handler on crashes
Fixes https://emqx.atlassian.net/browse/EMQX-11821
2024-01-26 13:05:41 -03:00
Zaiming (Stone) Shi
2a40152721 Merge remote-tracking branch 'origin/release-55' into sync-release-55 2024-01-26 16:51:13 +01:00
William Yang
933c00c7ad
Merge pull request #12392 from qzhuyan/perf/william/ws-no-utf8-validate
perf: new ws listener option to disable UTF-8 validation
2024-01-26 16:01:04 +01:00
Serge Tupchii
2d693402c5 refactor: split greptimedb bridge to actions and connectors 2024-01-26 11:01:53 +02:00
William Yang
85e9731617 docs: changelog 2024-01-26 09:55:17 +01:00
JianBo He
e29a80d568
Merge pull request #12388 from qzhuyan/dev/william/per-listn-conn-cnt
feat: quic per listener conn count
2024-01-26 09:01:39 +08:00
Kjell Winblad
5bf37cbe0a
Merge pull request #12376 from kjellwinblad/kjell/kinesis
feat: refactor kinesis bridge to connector and action
2024-01-25 14:58:40 +01:00
zhongwencool
3b0736fc67
Merge pull request #12387 from thalesmg/sync-r55-m-20240124
sync release-55 to master
2024-01-25 17:35:41 +08:00
William Yang
0ae034a157 docs: changelog 2024-01-25 09:57:54 +01:00
zhongwencool
4f8accc31b
Merge branch 'master' into sync-r55-m-20240124 2024-01-25 16:09:05 +08:00
Shawn
f0cde3fc5b chore: update change logs for cassandra bridge_v2 2024-01-25 16:01:44 +08:00
firest
83a8822798 chore(opents): bump version && update changes 2024-01-25 16:01:43 +08:00
Thales Macedo Garitezi
d323fc7c27 feat(ps): add message gc 2024-01-25 16:01:43 +08:00
Ilya Averyanov
adf22f1f10 fix(mqtt_bridge): render valid messages from incomplete rule data 2024-01-25 16:01:42 +08:00
Serge Tupchii
f52cc93d9d perf(emqx_cm): use a dedicated pool for channel cleanup
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.

Fixes: EMQX-11743
2024-01-25 16:01:42 +08:00
zhongwencool
3bc6e546e0 fix: reboot emqx_dashboard after emqx_licencse 2024-01-25 12:29:04 +08:00
Shawn
33981661c1 chore: add changelogs for #12381 2024-01-25 09:21:04 +08:00
Thales Macedo Garitezi
a19cf253ae
Merge pull request #12359 from thalesmg/fix-connector-app-deps-r55-20240119
fix(machine_boot): inject connector/bridge apps as dependencies to `emqx_connector` app
2024-01-24 15:40:24 -03:00
Kjell Winblad
25a88ccd04 docs: add change log entry for Kinesis bridge refactoring 2024-01-24 18:36:56 +01:00
Thales Macedo Garitezi
b763e39b32
docs: improve descriptions
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2024-01-24 11:04:42 -03:00
Xinyu Liu
aafb683ec7
Merge pull request #12330 from emqx/bridge-v2-cassandra
refactor: split cassandra bridges to actions and connectors
2024-01-23 14:45:32 +08:00
lafirest
9d2a207184
Merge pull request #12353 from lafirest/feat/optsdb
feat(opents): improve the OpentsDB bridge to v2 style
2024-01-23 13:49:33 +08:00
Thales Macedo Garitezi
d122340c13
Merge pull request #12338 from thalesmg/ds-message-gc-20240115
feat(ds): add message GC
2024-01-22 16:57:26 -03:00
Thales Macedo Garitezi
7035b4c8b3 feat(ps): add message gc 2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi
3207f0ea80 fix(machine_boot): inject connector/bridge apps as dependencies to emqx_connector app
Fixes https://emqx.atlassian.net/browse/EMQX-11771

For the same reasons as we inject `emqx_bridge_*` applications as dependencies to
`emqx_bridge` when starting the node.  Already configured connectors are started when
`emqx_connector` application starts, and may lead to crashes and noise in the logs.  One
example is to configure a mongodb bridge and restart the node.
2024-01-22 11:13:34 -03:00
firest
3f7b913e88 chore(opents): bump version && update changes 2024-01-22 20:45:18 +08:00
JianBo He
2706e005ff
Merge pull request #12354 from SergeTupchiy/EMQX-11751-dashboard-import-timeout-rel55
Apply brdige post config changes in parallel
2024-01-22 17:24:55 +08:00
Shawn
837b19cb1e chore: update change logs for cassandra bridge_v2 2024-01-22 15:45:00 +08:00
JimMoen
c2f26e8e98
docs: bump change log 2024-01-22 10:47:37 +08:00
Serge Tupchii
a6568dec75 perf(emqx_bridge/connector): apply post config bridge/connector changes in parallel
This can greatly improve the performance when many bridges/connectors are being changed,
e.g. when a backup file is being imported.

Fixes: EMQX-11751
2024-01-19 21:44:31 +02:00
Zaiming (Stone) Shi
7d14a13b7b Merge remote-tracking branch 'origin/release-55' 2024-01-18 14:36:30 +01:00
zhongwencool
dae835635c fix: don't crash in http SUITE 2024-01-18 16:45:46 +08:00
Ilya Averyanov
74bf4042c5 fix(mqtt_bridge): render valid messages from incomplete rule data 2024-01-17 23:13:55 +03:00
Serge Tupchii
b472b56883 perf(emqx_cm): use a dedicated pool for channel cleanup
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.

Fixes: EMQX-11743
2024-01-16 19:08:02 +02:00
Zaiming (Stone) Shi
ab66986f16 feat: add 'tags' field for action and connector 2024-01-16 13:57:48 +01:00
JianBo He
b6d0365027
Merge pull request #12267 from HJianBo/new-timeout-param-for-invite
feat(cluster): supports inviting nodes to join the cluster in an asynchronous manner
2024-01-16 10:11:20 +08:00
Andrew Mayorov
f7bda457fa
chore: add changelog entry 2024-01-15 17:59:56 +01:00
Zaiming (Stone) Shi
58a54adbb4 Merge remote-tracking branch 'origin/master' into release-55 2024-01-15 17:58:48 +01:00
William Yang
c77837a9ea feat(quic): support reload with new binding port 2024-01-15 11:33:25 +01:00
Shawn
1668e9ac7d fix: cannot write literal numbers to influxdb 2024-01-15 09:29:09 +08:00
JianBo He
93ef6766ef chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-13 15:29:58 +08:00
JianBo He
81209e0ded chore: update changes 2024-01-13 15:29:58 +08:00
JianBo He
daad1521d4 chore: update changes 2024-01-13 15:29:58 +08:00
Thales Macedo Garitezi
a5e7db793b fix(mongodb_action): make batch_size hidden and fixed in the API
Fixes https://emqx.atlassian.net/browse/ED-1171

The old bridge schema already had this override, but it had not been ported to actions.
2024-01-12 10:04:32 -03:00
Thales Macedo Garitezi
d1ae6f5195
Merge pull request #12306 from thalesmg/fix-password-deobfuscate-m-20240111
fix(bridge_api): correctly deobfuscate secrets during dry run
2024-01-12 09:13:20 -03:00
Thales Macedo Garitezi
79a4a041e4 fix(bridge_api): correctly deobfuscate secrets during dry run
Fixes https://emqx.atlassian.net/browse/EMQX-11733
2024-01-11 15:42:14 -03:00
Ilya Averyanov
ef0850c71f
Merge pull request #12305 from savonarola/1101-fix-channel-info-update
fix(conn): avoid storing incomplete channel info
2024-01-11 18:48:53 +02:00
Ilya Averyanov
4c66a1135b
Merge pull request #12303 from savonarola/0111-fix-indexing
fix(retainer): fix topic search by index
2024-01-11 18:44:56 +02:00
Ilya Averyanov
7b0b2a0527 fix(conn): avoid storing incomplete channel info 2024-01-11 18:08:13 +03:00
Ilya Averyanov
09d524144b fix(retainer): fix topic search by index 2024-01-11 16:00:58 +03:00
JianBo He
b16920e796
Merge pull request #12285 from HJianBo/shorten-coap-params
feat(coap): support short param names
2024-01-11 09:09:09 +08:00
SergeTupchiy
8bea0711ac
Merge pull request #12196 from SergeTupchiy/EMQX-10891-route-cleanup-optimization
EMQX-10891 route cleanup optimization
2024-01-10 17:26:24 +02:00
Thales Macedo Garitezi
8532e0cf59
Merge pull request #12292 from thalesmg/fix-syskeeper-v2-only-m-20240110
fix(syskeeper_bridge): mark syskeeper forwarder as a v2-only action
2024-01-10 11:42:24 -03:00
Serge Tupchii
965ce5d446 perf: use mria:match_delete/2 to cleanup routes
This must be much more network efficient since both Mria and Mnesia
need only to replicate one op, e.g.: `{MatchPattern, clear_table}`
instead of replicating one per each key to be deleted.
2024-01-10 16:23:58 +02:00
Thales Macedo Garitezi
b395924569
Merge pull request #12283 from thalesmg/fix-gcp-pubsub-produ-resopts-m-20240109
fix(gcp_pubsub_producer): fix connector `resource_opts` schema
2024-01-10 11:08:39 -03:00
Zaiming (Stone) Shi
7c8a36fc06
Merge pull request #12289 from zmstone/0108-support-acl-cache-masks
0108 support acl cache excludes
2024-01-10 13:40:29 +00:00
Thales Macedo Garitezi
b69fc9af15 fix(gcp_pubsub_producer): fix connector resource_opts schema
Fixes https://emqx.atlassian.net/browse/EMQX-11703
2024-01-10 10:13:11 -03:00
Thales Macedo Garitezi
787f952ad8 fix(syskeeper_bridge): mark syskeeeper as a v2-only action
Fixes https://emqx.atlassian.net/browse/EMQX-11706
2024-01-10 10:09:51 -03:00
Zaiming (Stone) Shi
9e8a67fd68 feat: support authz cache exclusion config
now one can configure a list of topic-filters to avoid
caching ACL check results

for example

authorization.cache.excludes = ["nocache/#"]

this means ACL check results for topics having 'nocache/' prefix
will not be cached
2024-01-10 13:52:00 +01:00
Andrew Mayorov
e14e923f8a
chore: add changelog entry 2024-01-10 13:41:43 +01:00
Thales Macedo Garitezi
6d3c397be8
Merge pull request #12282 from thalesmg/fix-mysql-del-action-m-20240109
fix(mysql_bridge): allow deleting bridge when sql contains undefined fields
2024-01-10 09:03:19 -03:00
William Yang
8cd99cb0c8
Merge pull request #12274 from qzhuyan/dev/william/quic-listn-conf-reload
feat: quic listener conf reload
2024-01-10 11:52:52 +01:00
JianBo He
73c4838c77 chore: update changes 2024-01-10 14:58:59 +08:00
JianBo He
b420e116a8 chore: update changes 2024-01-10 09:52:27 +08:00
lafirest
3d7264c9ef
Merge pull request #12278 from lafirest/fix/limit
fix(api): adjust the maximum limit of page query to 10,000
2024-01-10 09:42:46 +08:00
JianBo He
adcda51eff
Merge pull request #12277 from HJianBo/feat-delete-delayed-messages-via-topic
feat: support to delete delayed messages via topic name
2024-01-10 09:05:19 +08:00
firest
4e046eaa3d chore: bump version && update changes 2024-01-10 06:58:01 +08:00
Thales Macedo Garitezi
abd4873091 fix(mysql_bridge): allow deleting bridge when sql contains undefined fields
Fixes https://emqx.atlassian.net/browse/EMQX-11655
2024-01-09 17:25:34 -03:00
Thales Macedo Garitezi
d0588b86b5 fix(mysql_bridge): check prepare statement error messages
Fixes https://emqx.atlassian.net/browse/EMQX-11648
2024-01-09 17:25:34 -03:00
William Yang
a415881997 chore: changelog 2024-01-09 20:31:27 +01:00
lafirest
cc00dd80ee
Merge pull request #12272 from lafirest/feat/retain
feat(retain): add two new endpoints for retain API
2024-01-09 15:59:42 +08:00
JianBo He
147b5f8953 chore: update changes 2024-01-09 15:17:35 +08:00
firest
23e9ed763e chore: bump version && update changes 2024-01-09 10:20:54 +08:00
Thales Macedo Garitezi
756980837c Merge branch 'release-54' into sync-r54-m-20240108 2024-01-08 17:39:42 -03:00