01207ef97b
fix(iotdb): fix function clause error when there is no `payload` field
2024-02-28 16:54:01 +08:00
660a1ce99d
chore: use gen_tcp:recv timeout to wait socket closed
2024-02-28 16:18:32 +08:00
6fd04e33f5
fix(prom): skip cert info for disabled ssl/wss/quic listeners
2024-02-28 15:39:33 +08:00
5f125047f5
refactor: remove unnecessary lists fold for `cerfile`
2024-02-28 14:53:20 +08:00
812e8ef314
fix: try-catch for unknow reason to inhibit api crash
2024-02-28 14:53:19 +08:00
4307aecefd
chore: wait_for cowboy listener release sock when stop_listener
2024-02-28 14:24:15 +08:00
c3a2cf6220
chore: update change && bump version
2024-02-28 13:18:48 +08:00
042a84c30f
fix(prom): api crash when tls cert file non existed
2024-02-28 11:35:55 +08:00
aa7def425d
fix: get grpc_timeout from hstreamdb connector config failed
2024-02-28 08:45:06 +08:00
28d664bae2
test(pulsar): simplify the test suite
2024-02-27 23:51:54 +01:00
367ffa8e80
feat(utils-fs): add function to compute relpaths naively
2024-02-27 23:41:50 +01:00
6b9844ae82
feat: migrate kafka consumer bridge to source + connector
...
Fixes https://emqx.atlassian.net/browse/EMQX-11848
2024-02-27 17:52:33 -03:00
787dcd0e86
Merge pull request #12577 from thalesmg/gcp-service-json-binary-m-20240223
...
feat(gcp_pubsub_bridges): make service account json a binary
2024-02-27 15:42:42 -03:00
ef99037335
docs: add log docs
2024-02-27 19:24:56 +01:00
6ea9d2a6d9
test(conf): simplify confsync test suite
2024-02-27 16:41:54 +01:00
ae79516fd2
test(cluster-rpc): ensure testsuite is side-effects-free
2024-02-27 16:41:54 +01:00
56eefe34d5
feat(cth-cluster): use workdir as cwd on each node
2024-02-27 16:41:53 +01:00
d60ff1e616
test(plugins): avoid using same workdir for different nodes
2024-02-27 16:41:53 +01:00
3a008c8b4c
refactor: bump ekka to 0.19.0 w/o mnesia boot phase
2024-02-27 16:41:50 +01:00
8a204f2ac1
Merge pull request #12596 from thalesmg/fix-header-logging-r55-20240226
...
fix: redact all headers from logs
2024-02-27 11:19:51 -03:00
77239da7ed
fix(iotdb): use the `ping` API to check the status of the IOTDB connections
2024-02-27 22:07:41 +08:00
02de92a53d
Merge pull request #12601 from lafirest/fix/eldap_log
...
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 21:50:05 +08:00
2ded751237
test: `nl` not allowed for shared-sub
2024-02-27 18:24:30 +08:00
54c542c795
chore: rename _probe_ to t_probe_
2024-02-27 17:57:15 +08:00
c67f2130f2
fix: check connector and bridge_v2 with the right schema
2024-02-27 17:57:10 +08:00
c6b1102b2b
fix(mgmt_client): `nl` not allowed for shared-sub
2024-02-27 17:51:39 +08:00
f3237a1bf7
chore: bump emqx_ldap version
2024-02-27 17:48:34 +08:00
5c69500c52
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 17:48:22 +08:00
9ae7ac04a2
Merge pull request #12597 from lafirest/fix/ldap_log
...
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 17:40:44 +08:00
1567c5beb0
Merge pull request #12580 from zmstone/0224-delete-stale-macro
...
chore: delete stale macro
2024-02-27 09:58:06 +01:00
9c9a2b8494
Merge pull request #12592 from zmstone/0226-duration_s-min-value
...
chore: for dashboad add a 'minimum' attribute for duration_s type
2024-02-27 09:57:33 +01:00
da13ec736c
chore: bump emqx_ldap version
2024-02-27 16:08:47 +08:00
c1da7449fe
test(mgmt_api): bulk/unbulk sub/unsub shared topic filter
2024-02-27 16:01:12 +08:00
a76415c4f6
fix(mgmt): sub/unsub a share subscription to the client via http api
...
- `/clients/:clientid/subscribe`
- `/clients/:clientid/subscribe/bulk`
- `/clients/:clientid/unsubscribe`
- `/clients/:clientid/unsubscribe/bulk`
2024-02-27 16:00:45 +08:00
6e73cfa2cf
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 15:47:46 +08:00
6c7b7741d5
feat(quic): mqtt 5.0 graceful shutdown in takeover
2024-02-26 23:05:49 +01:00
c8f9ffdc1f
chore: update some comments
2024-02-26 22:24:16 +01:00
88fc67b369
chore(willmsg): remove unreachable code
2024-02-26 22:24:16 +01:00
d5247cb567
refactor: update notes for willmsg
2024-02-26 22:24:16 +01:00
975c7429e5
chore: clean in tests
2024-02-26 22:24:13 +01:00
d8032f47ca
fix: redact all headers from logs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11904
Since headers are usually used for authentication and the headers used for that are very
flexible, we redact all headers from logs to avoid leaking anything.
2024-02-26 18:04:12 -03:00
bb6a95bae8
Merge pull request #12590 from zmstone/0226-remove-mfa-from-logs
...
refactor: delete mfa from log metadata
2024-02-26 21:37:57 +01:00
5326cc80ed
Merge pull request #12594 from savonarola/0226-add-ds-delete-interface
...
chore(ds): add delete callbacks
2024-02-26 20:07:21 +02:00
9ca5684b34
Merge pull request #12591 from thalesmg/dev-cls-m-20240226
...
test: make `clear_screen` safer
2024-02-26 13:40:16 -03:00
b010d34640
chore(ds): add delete callbacks
2024-02-26 17:35:13 +03:00
5d4c85cd84
chore: list throttled log events in the change-log
2024-02-26 15:09:38 +02:00
5b48b06d4a
fix: trace throttled log events
2024-02-26 15:09:38 +02:00
652e977b3c
chore: for dashboad add a 'minimum' attribute for duration_s type
2024-02-26 13:51:07 +01:00
c6195fcf50
test: make `clear_screen` safer
...
Useful when iterating on the tests in a loop, to get rid of all the garbaged printed
before the test itself beings.
Only actually does anything if the environment variable `CLEAR_SCREEN` is set to `true`
and only clears the screen the screen the first time it's encountered, so it's harmless
otherwise.
2024-02-26 09:31:06 -03:00
f2c372d9ff
feat(gcp_pubsub_bridges): make service account json a binary
...
Fixes https://emqx.atlassian.net/browse/EMQX-11384
Today, service_account_json config field is an embedded object (map()).
This requires user to embed a JSON object into the config file instead of embedding it as
a string.
We should support binary() type as input, but keep supporting map() for backward
compatibility.
2024-02-26 09:21:07 -03:00
00c2aeb1ab
refactor: delete mfa from log metadata
2024-02-26 12:15:00 +01:00
de5e4491f7
Merge pull request #12576 from zmstone/0223-hide-bridges-root-from-docs
...
docs: stop generating schema doc for 'bridges' root
2024-02-24 09:08:05 +01:00
94a01d23c1
chore: delete stale macro
...
EMQX_API_VERSION is no longer in use since 5.2.0
2024-02-24 09:06:25 +01:00
a3e81c5039
chore: prepare for release 5.5.1-rc.2
2024-02-24 08:59:51 +01:00
fe4c7cd2dc
fix(ds): Apply review remarks, and hide certain fields
2024-02-23 18:07:56 +01:00
e126393cbf
fix(ds): Fix schema checker warnings for DS schema
2024-02-23 15:39:52 +01:00
c18fc6a4bb
fix(sessds): Remove deprecated configuration parameters
2024-02-23 15:08:13 +01:00
91ddbbcc3f
fix(sessds): Replace min- and max- batch size with batch_size
2024-02-23 15:08:13 +01:00
94b0ab983d
docs(ds): Add descriptions for the builtin's egress config
2024-02-23 15:08:13 +01:00
17ab3c6362
chore(ds_schema): Use atoms for record and field names
2024-02-23 15:08:13 +01:00
24337ecec7
feat(sessds): Move config schema to a separate root
2024-02-23 15:08:13 +01:00
8907e5afb3
chore(sessds): Remove deprecated schema
2024-02-23 15:08:11 +01:00
2adfefe297
docs: stop generating schema doc for 'bridges' root
2024-02-23 14:51:57 +01:00
5af01c041b
Merge pull request #12559 from zmstone/0221-refactor-use-atom-fileds
...
refactor: use atoms for root config fields
2024-02-23 14:38:19 +01:00
b60b19a29a
fix(connectors): redact authorization headers when creating/updating connectors
...
Fixes https://emqx.atlassian.net/browse/EMQX-11895
2024-02-23 10:17:24 -03:00
15f919e60f
Merge pull request #12564 from thalesmg/bw-support-batch-list-resp-m-20240221
...
feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
2024-02-23 09:37:42 -03:00
cd091b4f75
Merge pull request #12572 from zhongwencool/api-config-error-log
...
fix(api): more detail error when don't allow update readonly key
2024-02-23 17:29:00 +08:00
f2619ce17a
Merge pull request #12571 from zhongwencool/delete-repeat-ci-test
...
test(bridge_v1): delete repeat probe_bridge_api code
2024-02-23 17:17:39 +08:00
fbf4afc3c5
fix(api): more detail error when don't allow update readonly key
2024-02-23 16:43:15 +08:00
46877e979b
chore: update copyright-year
2024-02-23 08:21:06 +01:00
d7faab4195
test(bridge_v1): delete repeat probe_bridge_api code
2024-02-23 14:43:42 +08:00
9f52b4fba8
Merge pull request #12566 from lafirest/fix/same_api
...
fix(api_key): enhanced bootstrap files for REST API keys
2024-02-23 10:57:37 +08:00
11d3df2775
Merge pull request #12569 from zhongwencool/fix-rabbitmq-action-crash-dropped-metrics
...
fix: rabbitmq action failed to increase dropped metrics
2024-02-23 09:53:55 +08:00
c08522bc1e
Merge pull request #12539 from zhongwencool/rocketmq-channel-status
...
fix: Rocketmq channel status
2024-02-23 09:50:10 +08:00
51a6ada015
fix: rabbitmq action failed to increase dropped metrics
2024-02-23 08:35:00 +08:00
eba74c1ca8
Merge pull request #12567 from thalesmg/fix-gcp-consumer-v1-api-m-20240222
...
fix(gcp_pubsub_consumer,mongodb_bridge): fix v1 config transformation for gcp pubsub consumer and mongodb bridges
2024-02-22 17:33:02 -03:00
d003f77021
feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
...
Fixes https://emqx.atlassian.net/browse/EMQX-11892
This allows callers of batching resources to receive results specific to their requests,
rather than a broad success or failure for the whole batch.
2024-02-22 16:26:02 -03:00
54499175d5
Merge pull request #12568 from keynslug/ft/EMQX-11888/s3-redact
...
fix(s3): try to obtain credentials with default lhttpc
2024-02-22 18:07:04 +01:00
2ff33f98ef
chore(willmsg): add come comments
2024-02-22 17:12:19 +01:00
e5a3574d89
fix: maybe send willmsg
2024-02-22 17:12:16 +01:00
a30e8818fd
fix(s3-bridge): check for unobtainable credentials in healthcheck
2024-02-22 16:56:14 +01:00
69c7f858af
fix(s3): try to obtain credentials with default lhttpc
...
Otherwise `emqx_s3_client` will try to do it later by contacting the S3
endpoint instead of the metadata server.
2024-02-22 16:56:13 +01:00
88b1d9ba88
refactor: use atoms for root config fields and types
2024-02-22 16:51:40 +01:00
6311b582ec
test(willmsg): session taken over before willmsg delay /session expire
2024-02-22 16:39:27 +01:00
5397402396
test(willmsg): test will delay and session expires
...
will delay > session expire
will delay < session expire
timer triggered events are handled in seq, exclude the case of
(will delay == session expire)
2024-02-22 16:39:27 +01:00
dd62280e04
fix: handle delayed willmsg, part 1
2024-02-22 16:39:27 +01:00
b76c701b1c
test(takeover): add back the delay when takeover
2024-02-22 16:39:27 +01:00
6243cf0b0c
fix(kick): defer willmsg publish when conn terminates
...
because kick means shutdown connection AND delete session
2024-02-22 16:39:27 +01:00
9da4896f57
fix(mqtt): ensure publish willmsg when session expires
2024-02-22 16:39:24 +01:00
07eec31a8a
fix: willmsg not published in takeover
2024-02-22 16:35:59 +01:00
47757a0983
fix(mongodb_bridge): remove `local_topic` from v1 config transformation
...
Fixes https://emqx.atlassian.net/browse/EMQX-11893
2024-02-22 12:23:26 -03:00
205b97f732
fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config
...
Fixes https://emqx.atlassian.net/browse/EMQX-11891
2024-02-22 10:27:28 -03:00
ae8f59979d
fix(utils): handle improper lists as well in `redact/1`
2024-02-22 13:35:11 +01:00
9893cc700b
Merge pull request #12565 from zhongwencool/fix-bpapi-bad-node
...
fix: bpapi announce's bad node name
2024-02-22 20:18:33 +08:00
f0aeaadc48
fix(api_key): enhanced bootstrap files for REST API keys
2024-02-22 18:40:24 +08:00
7bb9d5d8f6
Merge pull request #12543 from kjellwinblad/kjell/refactor/dynamodb/EMQX-11456
...
feat: refactor DynamoDB bridge to connector and action
2024-02-22 09:47:57 +01:00
ccf607ee2c
fix: bpapi announce's bad node name
2024-02-22 14:55:11 +08:00
985a3e3062
test: remove unused var warning
2024-02-21 18:14:46 -03:00
8e47503f7d
Merge pull request #12560 from SergeTupchiy/EMQX-11530-log-throttling-followup-fixes
...
fix: disable log throttling if primary log level is debug
2024-02-21 22:02:32 +02:00
f31e9e6a15
Merge pull request #12555 from thalesmg/ds-move-count-to-mgmt-m-20240221
...
refactor(mgmt): move persistent session counting function to mgmt module
2024-02-21 16:36:29 -03:00
ceb208f1eb
fix: disable log throttling if primary log level is debug
2024-02-21 21:19:15 +02:00
7feb444e31
Merge pull request #12557 from keynslug/fix/s3-bridge-startup
...
fix(s3-bridge): anticipate repeated connector start requests
2024-02-21 15:13:48 +01:00
9b6511b3e7
docs: fix spelling mistake in error message
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-21 14:57:55 +01:00
4678df2229
build: bump emqx_bridge_gcp_pubsub version
...
The CI check ./scripts/apps-version-check.sh requires that
emqx_bridge_gcp_pubsub's version number is bumped.
2024-02-21 14:59:03 +01:00
6561d989d6
feat: refactor DynamoDB bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11456
2024-02-21 14:58:48 +01:00
e98fb0db48
chore: bump `emqx_bridge_gcp_pubsub` version to 0.2.2
2024-02-21 14:28:58 +01:00
6b97983303
fix(s3-bridge): anticipate repeated connector start requests
...
Starting disconnected connector manually implies that there's no
attempt to stop it first.
2024-02-21 14:23:29 +01:00
191b435a48
chore: bump gcp bridge app version
2024-02-21 21:15:27 +08:00
ed23ad906a
fix: rabbitmq source metrics not working
2024-02-21 21:10:23 +08:00
415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
...
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
...
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
25cdc2a04b
chore: bump version to 5.5.1-rc.1
2024-02-21 09:57:05 -03:00
b14e51d7c4
docs: delete useless comments
2024-02-21 13:33:29 +01:00
03138f8345
refactor(mgmt): move persistent session counting function to mgmt module
...
Follow up to https://github.com/emqx/emqx/pull/12500#discussion_r1496634087
2024-02-21 09:12:56 -03:00
529211b9ac
Merge pull request #12500 from thalesmg/ds-list-client-api-m-20240209
...
feat(ds): list disconnected persistent sessions in clients API
2024-02-21 09:02:27 -03:00
92c00ee21a
Merge pull request #12550 from keynslug/fix/bridge-s3-resopts
...
fix(bridge-s3): expose connector-level `resource_opts` properly
2024-02-21 11:37:55 +01:00
333eb34300
test: call emqx_resource:remove_local
2024-02-21 10:53:22 +01:00
de93d1c0a3
chore: add bpapi.versions
2024-02-21 10:53:22 +01:00
9c1d5e49dd
chore: fix a typo
2024-02-21 10:53:22 +01:00
c8f9e12de1
refactor: deprecate emqx_resource_proto_v1 with v2
2024-02-21 10:53:22 +01:00
06e0a7f2ed
refactor: delete emqx_resource:remove api
...
this api is only used in tests,
changed to call the _local flavor instead
2024-02-21 10:53:22 +01:00
8541d2efc1
refactor: delete emqx_resource:recreate
...
this api is only used in tests, changed to call the _local flavor
instead
2024-02-21 10:53:22 +01:00
1878fded5a
refactor: delete emqx_resource:create/4 and /5
...
the functions are only used in tests,
now the tests are rewritten to call create_local/4 and /5 instead
2024-02-21 10:53:22 +01:00
0cbe626aed
Merge pull request #12549 from zmstone/0221-sync-release-55
...
Sync `release-55` with i18n fixes to master
2024-02-21 10:52:36 +01:00
d8b6ecd185
fix(bridge-s3): expose connector-level `resource_opts` properly
2024-02-21 10:49:04 +01:00
ee02079661
Merge pull request #12548 from zhongwencool/fix-rabbitmq-source-crash
...
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 16:34:43 +08:00
f7a7b7c8e0
Merge pull request #12545 from SergeTupchiy/fix-typo-tls_gc
...
chore(emqx_tls_certfile_gc): fix typo
2024-02-21 10:11:13 +02:00
da340c92a1
Merge remote-tracking branch 'origin/master' into 0221-sync-release-55
2024-02-21 09:11:12 +01:00
80be84c401
Merge pull request #12491 from zmstone/docs-sync-changes-for-i18n
...
docs: fix i18n
2024-02-21 09:10:25 +01:00
2a7a662c03
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 15:30:05 +08:00
65cfece0a3
chore: delete 5.5.bpapi2
...
> This particular warning won't be seen in the real life. Also, we need to start storing
bpapi files for each minor release, since the plan to use hot upgrade has been abandoned.
> ...The original procedure for BPAPI life cycle was designed under the assumption that
we'll support hot upgrade between minor releases, so we'll only ever need to keep one
BPAPI file per major release, and rewrite it whenever a new minor version is
released. We'll need to revisit it.
https://emqxio.slack.com/archives/C04H4RHKKEY/p1706862980546309
2024-02-20 17:35:28 -03:00
d469f4158e
chore: bump app vsns
2024-02-20 16:53:57 -03:00
74dbbf6d9d
Merge branch 'release-55' into sync-r55-m-20240220
2024-02-20 16:33:34 -03:00
dd9f96211b
chore(emqx_tls_certfile_gc): fix typo
2024-02-20 21:19:36 +02:00
3a4c7f60e2
feat(ds): list disconnected persistent sessions in clients API
...
Fixes https://emqx.atlassian.net/browse/EMQX-11540
Note that not all information provided by disconnected in-memory sessions is available to
disconnected persistent sessions, nor does all of them make sense.
2024-02-20 14:52:55 -03:00
1de9e9dd01
Merge pull request #12541 from zmstone/0220-fail-fqdn-node-name-if-dns-discovery
...
Fail fast if node host is not IP address but cluster strategy is `dns` with `a` or `aaaa` record type
2024-02-20 17:48:02 +01:00
b6689d178f
fix(http_bridge): redact headers
...
Fixes https://emqx.atlassian.net/browse/EMQX-11864
```
2024-02-20T15:51:34.744509+00:00 [debug] msg: http_connector_received, mfa: emqx_bridge_http_connector:on_query_async/4(471), tag: QUERY_ASYNC, request: {"/bah",[{<<"Authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>},{<<"authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>}],<<"******">>}, state: #{port => 18083,request => #{path => [<<>>],body => undefined,headers => [{[<<"content-type">>],[<<"application/json">>]},{[<<"Authorization">>],#Fun<emqx_secret.0.85191762>}],method => [<<"undefined">>],max_retries => 2,request_timeout => 30000},host => "localhost",connect_timeout => 15000,pool_type => random,pool_name => <<"connector:http:p1">>,base_path => "/",installed_actions => #{<<"action:http:p1:connector:http:p1">> => #{path => [<<"/bah">>],body => undefined,headers => [{[<<"authorization">>],#Fun<emqx_secret.0.85191762>},{[<<"content-type">>],[<<"application/json">>]}],method => [<<"post">>],max_retries => 2,request_timeout => 30000,render_template_func => fun emqx_bridge_http_connector:render_template/2}}}, connector: <<"connector:http:p1">>, note: the request body is redacted due to security reasons
```
2024-02-20 12:57:13 -03:00
dc015e7a6f
fix(schema): validate cluster strategy and node name
...
If cluster strategy is configured as `dns`, the node name
must be IP address
2024-02-20 16:09:54 +01:00
f72558737c
Merge pull request #12520 from SergeTupchiy/EMQX-11530-log-throttling
...
feat: implement log throttling
2024-02-20 15:52:46 +02:00
86c85f3b45
Merge pull request #12512 from kjellwinblad/kjell/refactor/hstreamdb/EMQX-11458
...
feat: refactor HStreamDB bridge to connector and action
2024-02-20 13:30:29 +01:00
9bd0d1ba1b
feat: enable log throttling for potentially flooding log events
2024-02-20 14:20:21 +02:00
98ba300f7c
feat: implement log throttling
2024-02-20 13:23:17 +02:00
9639a977a3
fix: hidden batch_time for mongodb
2024-02-20 09:42:49 +08:00
c7bfaf51fd
chore: typo error (templete -> template)
2024-02-20 09:41:09 +08:00
2561ff94ec
fix: check channel id exist when get_channel_status
2024-02-20 09:38:33 +08:00
146d89bd89
feat: convert `gcp_pubsub_consumer` to connector/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 17:54:26 -03:00
fd13a661cf
Merge pull request #12535 from zmstone/0219-refactor-delete-non-prod-code
...
refactor: delete non-prod code
2024-02-19 19:04:59 +01:00
11e4a295ed
fix: use connector type for constructing resource id
2024-02-19 14:42:35 -03:00
9a32895a1a
feat: convert `gcp_pubsub_consumer` to connector/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
ad53523e6b
refactor: delete non-prod code
...
So far the retainer backend type is always `built_in_database`.
The slightly over-engineered pre-implementation to support
another backend is likely not going to fly as the EMQX
resource frame work is mostly for auth and data integration.
i.e. not generic enough for retained messages.
2024-02-19 14:36:12 +01:00
firest