William Yang
975c7429e5
chore: clean in tests
2024-02-26 22:24:13 +01:00
Thales Macedo Garitezi
d8032f47ca
fix: redact all headers from logs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11904
Since headers are usually used for authentication and the headers used for that are very
flexible, we redact all headers from logs to avoid leaking anything.
2024-02-26 18:04:12 -03:00
Zaiming (Stone) Shi
bb6a95bae8
Merge pull request #12590 from zmstone/0226-remove-mfa-from-logs
...
refactor: delete mfa from log metadata
2024-02-26 21:37:57 +01:00
Ilya Averyanov
5326cc80ed
Merge pull request #12594 from savonarola/0226-add-ds-delete-interface
...
chore(ds): add delete callbacks
2024-02-26 20:07:21 +02:00
Thales Macedo Garitezi
9ca5684b34
Merge pull request #12591 from thalesmg/dev-cls-m-20240226
...
test: make `clear_screen` safer
2024-02-26 13:40:16 -03:00
Ilya Averyanov
b010d34640
chore(ds): add delete callbacks
2024-02-26 17:35:13 +03:00
Serge Tupchii
5d4c85cd84
chore: list throttled log events in the change-log
2024-02-26 15:09:38 +02:00
Serge Tupchii
5b48b06d4a
fix: trace throttled log events
2024-02-26 15:09:38 +02:00
Zaiming (Stone) Shi
652e977b3c
chore: for dashboad add a 'minimum' attribute for duration_s type
2024-02-26 13:51:07 +01:00
Thales Macedo Garitezi
c6195fcf50
test: make clear_screen safer
...
Useful when iterating on the tests in a loop, to get rid of all the garbaged printed
before the test itself beings.
Only actually does anything if the environment variable `CLEAR_SCREEN` is set to `true`
and only clears the screen the screen the first time it's encountered, so it's harmless
otherwise.
2024-02-26 09:31:06 -03:00
Thales Macedo Garitezi
f2c372d9ff
feat(gcp_pubsub_bridges): make service account json a binary
...
Fixes https://emqx.atlassian.net/browse/EMQX-11384
Today, service_account_json config field is an embedded object (map()).
This requires user to embed a JSON object into the config file instead of embedding it as
a string.
We should support binary() type as input, but keep supporting map() for backward
compatibility.
2024-02-26 09:21:07 -03:00
Zaiming (Stone) Shi
00c2aeb1ab
refactor: delete mfa from log metadata
2024-02-26 12:15:00 +01:00
Zaiming (Stone) Shi
de5e4491f7
Merge pull request #12576 from zmstone/0223-hide-bridges-root-from-docs
...
docs: stop generating schema doc for 'bridges' root
2024-02-24 09:08:05 +01:00
Zaiming (Stone) Shi
94a01d23c1
chore: delete stale macro
...
EMQX_API_VERSION is no longer in use since 5.2.0
2024-02-24 09:06:25 +01:00
Zaiming (Stone) Shi
a3e81c5039
chore: prepare for release 5.5.1-rc.2
2024-02-24 08:59:51 +01:00
ieQu1
fe4c7cd2dc
fix(ds): Apply review remarks, and hide certain fields
2024-02-23 18:07:56 +01:00
ieQu1
e126393cbf
fix(ds): Fix schema checker warnings for DS schema
2024-02-23 15:39:52 +01:00
ieQu1
c18fc6a4bb
fix(sessds): Remove deprecated configuration parameters
2024-02-23 15:08:13 +01:00
ieQu1
91ddbbcc3f
fix(sessds): Replace min- and max- batch size with batch_size
2024-02-23 15:08:13 +01:00
ieQu1
94b0ab983d
docs(ds): Add descriptions for the builtin's egress config
2024-02-23 15:08:13 +01:00
ieQu1
17ab3c6362
chore(ds_schema): Use atoms for record and field names
2024-02-23 15:08:13 +01:00
ieQu1
24337ecec7
feat(sessds): Move config schema to a separate root
2024-02-23 15:08:13 +01:00
ieQu1
8907e5afb3
chore(sessds): Remove deprecated schema
2024-02-23 15:08:11 +01:00
Zaiming (Stone) Shi
2adfefe297
docs: stop generating schema doc for 'bridges' root
2024-02-23 14:51:57 +01:00
Zaiming (Stone) Shi
5af01c041b
Merge pull request #12559 from zmstone/0221-refactor-use-atom-fileds
...
refactor: use atoms for root config fields
2024-02-23 14:38:19 +01:00
Thales Macedo Garitezi
b60b19a29a
fix(connectors): redact authorization headers when creating/updating connectors
...
Fixes https://emqx.atlassian.net/browse/EMQX-11895
2024-02-23 10:17:24 -03:00
Thales Macedo Garitezi
15f919e60f
Merge pull request #12564 from thalesmg/bw-support-batch-list-resp-m-20240221
...
feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
2024-02-23 09:37:42 -03:00
zhongwencool
cd091b4f75
Merge pull request #12572 from zhongwencool/api-config-error-log
...
fix(api): more detail error when don't allow update readonly key
2024-02-23 17:29:00 +08:00
zhongwencool
f2619ce17a
Merge pull request #12571 from zhongwencool/delete-repeat-ci-test
...
test(bridge_v1): delete repeat probe_bridge_api code
2024-02-23 17:17:39 +08:00
zhongwencool
fbf4afc3c5
fix(api): more detail error when don't allow update readonly key
2024-02-23 16:43:15 +08:00
Zaiming (Stone) Shi
46877e979b
chore: update copyright-year
2024-02-23 08:21:06 +01:00
zhongwencool
d7faab4195
test(bridge_v1): delete repeat probe_bridge_api code
2024-02-23 14:43:42 +08:00
lafirest
9f52b4fba8
Merge pull request #12566 from lafirest/fix/same_api
...
fix(api_key): enhanced bootstrap files for REST API keys
2024-02-23 10:57:37 +08:00
zhongwencool
11d3df2775
Merge pull request #12569 from zhongwencool/fix-rabbitmq-action-crash-dropped-metrics
...
fix: rabbitmq action failed to increase dropped metrics
2024-02-23 09:53:55 +08:00
zhongwencool
c08522bc1e
Merge pull request #12539 from zhongwencool/rocketmq-channel-status
...
fix: Rocketmq channel status
2024-02-23 09:50:10 +08:00
zhongwencool
51a6ada015
fix: rabbitmq action failed to increase dropped metrics
2024-02-23 08:35:00 +08:00
Thales Macedo Garitezi
eba74c1ca8
Merge pull request #12567 from thalesmg/fix-gcp-consumer-v1-api-m-20240222
...
fix(gcp_pubsub_consumer,mongodb_bridge): fix v1 config transformation for gcp pubsub consumer and mongodb bridges
2024-02-22 17:33:02 -03:00
Thales Macedo Garitezi
d003f77021
feat(resource): allow on_batch_query{,_async} to return a list of individual results
...
Fixes https://emqx.atlassian.net/browse/EMQX-11892
This allows callers of batching resources to receive results specific to their requests,
rather than a broad success or failure for the whole batch.
2024-02-22 16:26:02 -03:00
Andrew Mayorov
54499175d5
Merge pull request #12568 from keynslug/ft/EMQX-11888/s3-redact
...
fix(s3): try to obtain credentials with default lhttpc
2024-02-22 18:07:04 +01:00
William Yang
2ff33f98ef
chore(willmsg): add come comments
2024-02-22 17:12:19 +01:00
William Yang
e5a3574d89
fix: maybe send willmsg
2024-02-22 17:12:16 +01:00
Andrew Mayorov
a30e8818fd
fix(s3-bridge): check for unobtainable credentials in healthcheck
2024-02-22 16:56:14 +01:00
Andrew Mayorov
69c7f858af
fix(s3): try to obtain credentials with default lhttpc
...
Otherwise `emqx_s3_client` will try to do it later by contacting the S3
endpoint instead of the metadata server.
2024-02-22 16:56:13 +01:00
Zaiming (Stone) Shi
88b1d9ba88
refactor: use atoms for root config fields and types
2024-02-22 16:51:40 +01:00
William Yang
6311b582ec
test(willmsg): session taken over before willmsg delay /session expire
2024-02-22 16:39:27 +01:00
William Yang
5397402396
test(willmsg): test will delay and session expires
...
will delay > session expire
will delay < session expire
timer triggered events are handled in seq, exclude the case of
(will delay == session expire)
2024-02-22 16:39:27 +01:00
William Yang
dd62280e04
fix: handle delayed willmsg, part 1
2024-02-22 16:39:27 +01:00
William Yang
b76c701b1c
test(takeover): add back the delay when takeover
2024-02-22 16:39:27 +01:00
William Yang
6243cf0b0c
fix(kick): defer willmsg publish when conn terminates
...
because kick means shutdown connection AND delete session
2024-02-22 16:39:27 +01:00
William Yang
9da4896f57
fix(mqtt): ensure publish willmsg when session expires
2024-02-22 16:39:24 +01:00
William Yang
07eec31a8a
fix: willmsg not published in takeover
2024-02-22 16:35:59 +01:00
Thales Macedo Garitezi
47757a0983
fix(mongodb_bridge): remove local_topic from v1 config transformation
...
Fixes https://emqx.atlassian.net/browse/EMQX-11893
2024-02-22 12:23:26 -03:00
Thales Macedo Garitezi
205b97f732
fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config
...
Fixes https://emqx.atlassian.net/browse/EMQX-11891
2024-02-22 10:27:28 -03:00
Andrew Mayorov
ae8f59979d
fix(utils): handle improper lists as well in redact/1
2024-02-22 13:35:11 +01:00
zhongwencool
9893cc700b
Merge pull request #12565 from zhongwencool/fix-bpapi-bad-node
...
fix: bpapi announce's bad node name
2024-02-22 20:18:33 +08:00
firest
f0aeaadc48
fix(api_key): enhanced bootstrap files for REST API keys
2024-02-22 18:40:24 +08:00
Kjell Winblad
7bb9d5d8f6
Merge pull request #12543 from kjellwinblad/kjell/refactor/dynamodb/EMQX-11456
...
feat: refactor DynamoDB bridge to connector and action
2024-02-22 09:47:57 +01:00
zhongwencool
ccf607ee2c
fix: bpapi announce's bad node name
2024-02-22 14:55:11 +08:00
Thales Macedo Garitezi
985a3e3062
test: remove unused var warning
2024-02-21 18:14:46 -03:00
SergeTupchiy
8e47503f7d
Merge pull request #12560 from SergeTupchiy/EMQX-11530-log-throttling-followup-fixes
...
fix: disable log throttling if primary log level is debug
2024-02-21 22:02:32 +02:00
Thales Macedo Garitezi
f31e9e6a15
Merge pull request #12555 from thalesmg/ds-move-count-to-mgmt-m-20240221
...
refactor(mgmt): move persistent session counting function to mgmt module
2024-02-21 16:36:29 -03:00
Serge Tupchii
ceb208f1eb
fix: disable log throttling if primary log level is debug
2024-02-21 21:19:15 +02:00
Andrew Mayorov
7feb444e31
Merge pull request #12557 from keynslug/fix/s3-bridge-startup
...
fix(s3-bridge): anticipate repeated connector start requests
2024-02-21 15:13:48 +01:00
Kjell Winblad
9b6511b3e7
docs: fix spelling mistake in error message
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-21 14:57:55 +01:00
Kjell Winblad
4678df2229
build: bump emqx_bridge_gcp_pubsub version
...
The CI check ./scripts/apps-version-check.sh requires that
emqx_bridge_gcp_pubsub's version number is bumped.
2024-02-21 14:59:03 +01:00
Kjell Winblad
6561d989d6
feat: refactor DynamoDB bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11456
2024-02-21 14:58:48 +01:00
Andrew Mayorov
e98fb0db48
chore: bump emqx_bridge_gcp_pubsub version to 0.2.2
2024-02-21 14:28:58 +01:00
Andrew Mayorov
6b97983303
fix(s3-bridge): anticipate repeated connector start requests
...
Starting disconnected connector manually implies that there's no
attempt to stop it first.
2024-02-21 14:23:29 +01:00
zhongwencool
191b435a48
chore: bump gcp bridge app version
2024-02-21 21:15:27 +08:00
zhongwencool
ed23ad906a
fix: rabbitmq source metrics not working
2024-02-21 21:10:23 +08:00
Zaiming (Stone) Shi
415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
...
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
Thales Macedo Garitezi
4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
...
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
Thales Macedo Garitezi
25cdc2a04b
chore: bump version to 5.5.1-rc.1
2024-02-21 09:57:05 -03:00
Zaiming (Stone) Shi
b14e51d7c4
docs: delete useless comments
2024-02-21 13:33:29 +01:00
Thales Macedo Garitezi
03138f8345
refactor(mgmt): move persistent session counting function to mgmt module
...
Follow up to https://github.com/emqx/emqx/pull/12500#discussion_r1496634087
2024-02-21 09:12:56 -03:00
Thales Macedo Garitezi
529211b9ac
Merge pull request #12500 from thalesmg/ds-list-client-api-m-20240209
...
feat(ds): list disconnected persistent sessions in clients API
2024-02-21 09:02:27 -03:00
Andrew Mayorov
92c00ee21a
Merge pull request #12550 from keynslug/fix/bridge-s3-resopts
...
fix(bridge-s3): expose connector-level `resource_opts` properly
2024-02-21 11:37:55 +01:00
Zaiming (Stone) Shi
333eb34300
test: call emqx_resource:remove_local
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
de93d1c0a3
chore: add bpapi.versions
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
9c1d5e49dd
chore: fix a typo
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
c8f9e12de1
refactor: deprecate emqx_resource_proto_v1 with v2
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
06e0a7f2ed
refactor: delete emqx_resource:remove api
...
this api is only used in tests,
changed to call the _local flavor instead
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
8541d2efc1
refactor: delete emqx_resource:recreate
...
this api is only used in tests, changed to call the _local flavor
instead
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
1878fded5a
refactor: delete emqx_resource:create/4 and /5
...
the functions are only used in tests,
now the tests are rewritten to call create_local/4 and /5 instead
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
0cbe626aed
Merge pull request #12549 from zmstone/0221-sync-release-55
...
Sync `release-55` with i18n fixes to master
2024-02-21 10:52:36 +01:00
Andrew Mayorov
d8b6ecd185
fix(bridge-s3): expose connector-level resource_opts properly
2024-02-21 10:49:04 +01:00
zhongwencool
ee02079661
Merge pull request #12548 from zhongwencool/fix-rabbitmq-source-crash
...
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 16:34:43 +08:00
SergeTupchiy
f7a7b7c8e0
Merge pull request #12545 from SergeTupchiy/fix-typo-tls_gc
...
chore(emqx_tls_certfile_gc): fix typo
2024-02-21 10:11:13 +02:00
Zaiming (Stone) Shi
da340c92a1
Merge remote-tracking branch 'origin/master' into 0221-sync-release-55
2024-02-21 09:11:12 +01:00
Zaiming (Stone) Shi
80be84c401
Merge pull request #12491 from zmstone/docs-sync-changes-for-i18n
...
docs: fix i18n
2024-02-21 09:10:25 +01:00
zhongwencool
2a7a662c03
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 15:30:05 +08:00
Thales Macedo Garitezi
65cfece0a3
chore: delete 5.5.bpapi2
...
> This particular warning won't be seen in the real life. Also, we need to start storing
bpapi files for each minor release, since the plan to use hot upgrade has been abandoned.
> ...The original procedure for BPAPI life cycle was designed under the assumption that
we'll support hot upgrade between minor releases, so we'll only ever need to keep one
BPAPI file per major release, and rewrite it whenever a new minor version is
released. We'll need to revisit it.
https://emqxio.slack.com/archives/C04H4RHKKEY/p1706862980546309
2024-02-20 17:35:28 -03:00
Thales Macedo Garitezi
d469f4158e
chore: bump app vsns
2024-02-20 16:53:57 -03:00
Thales Macedo Garitezi
74dbbf6d9d
Merge branch 'release-55' into sync-r55-m-20240220
2024-02-20 16:33:34 -03:00
Serge Tupchii
dd9f96211b
chore(emqx_tls_certfile_gc): fix typo
2024-02-20 21:19:36 +02:00
Thales Macedo Garitezi
3a4c7f60e2
feat(ds): list disconnected persistent sessions in clients API
...
Fixes https://emqx.atlassian.net/browse/EMQX-11540
Note that not all information provided by disconnected in-memory sessions is available to
disconnected persistent sessions, nor does all of them make sense.
2024-02-20 14:52:55 -03:00
Zaiming (Stone) Shi
1de9e9dd01
Merge pull request #12541 from zmstone/0220-fail-fqdn-node-name-if-dns-discovery
...
Fail fast if node host is not IP address but cluster strategy is `dns` with `a` or `aaaa` record type
2024-02-20 17:48:02 +01:00
Thales Macedo Garitezi
b6689d178f
fix(http_bridge): redact headers
...
Fixes https://emqx.atlassian.net/browse/EMQX-11864
```
2024-02-20T15:51:34.744509+00:00 [debug] msg: http_connector_received, mfa: emqx_bridge_http_connector:on_query_async/4(471), tag: QUERY_ASYNC, request: {"/bah",[{<<"Authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>},{<<"authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>}],<<"******">>}, state: #{port => 18083,request => #{path => [<<>>],body => undefined,headers => [{[<<"content-type">>],[<<"application/json">>]},{[<<"Authorization">>],#Fun<emqx_secret.0.85191762>}],method => [<<"undefined">>],max_retries => 2,request_timeout => 30000},host => "localhost",connect_timeout => 15000,pool_type => random,pool_name => <<"connector:http:p1">>,base_path => "/",installed_actions => #{<<"action:http:p1:connector:http:p1">> => #{path => [<<"/bah">>],body => undefined,headers => [{[<<"authorization">>],#Fun<emqx_secret.0.85191762>},{[<<"content-type">>],[<<"application/json">>]}],method => [<<"post">>],max_retries => 2,request_timeout => 30000,render_template_func => fun emqx_bridge_http_connector:render_template/2}}}, connector: <<"connector:http:p1">>, note: the request body is redacted due to security reasons
```
2024-02-20 12:57:13 -03:00
Zaiming (Stone) Shi
dc015e7a6f
fix(schema): validate cluster strategy and node name
...
If cluster strategy is configured as `dns`, the node name
must be IP address
2024-02-20 16:09:54 +01:00
SergeTupchiy
f72558737c
Merge pull request #12520 from SergeTupchiy/EMQX-11530-log-throttling
...
feat: implement log throttling
2024-02-20 15:52:46 +02:00
Kjell Winblad
86c85f3b45
Merge pull request #12512 from kjellwinblad/kjell/refactor/hstreamdb/EMQX-11458
...
feat: refactor HStreamDB bridge to connector and action
2024-02-20 13:30:29 +01:00
Serge Tupchii
9bd0d1ba1b
feat: enable log throttling for potentially flooding log events
2024-02-20 14:20:21 +02:00
Serge Tupchii
98ba300f7c
feat: implement log throttling
2024-02-20 13:23:17 +02:00
zhongwencool
9639a977a3
fix: hidden batch_time for mongodb
2024-02-20 09:42:49 +08:00
zhongwencool
c7bfaf51fd
chore: typo error (templete -> template)
2024-02-20 09:41:09 +08:00
zhongwencool
2561ff94ec
fix: check channel id exist when get_channel_status
2024-02-20 09:38:33 +08:00
Thales Macedo Garitezi
146d89bd89
feat: convert gcp_pubsub_consumer to connector/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 17:54:26 -03:00
Zaiming (Stone) Shi
fd13a661cf
Merge pull request #12535 from zmstone/0219-refactor-delete-non-prod-code
...
refactor: delete non-prod code
2024-02-19 19:04:59 +01:00
Thales Macedo Garitezi
11e4a295ed
fix: use connector type for constructing resource id
2024-02-19 14:42:35 -03:00
Thales Macedo Garitezi
9a32895a1a
feat: convert gcp_pubsub_consumer to connector/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
Zaiming (Stone) Shi
ad53523e6b
refactor: delete non-prod code
...
So far the retainer backend type is always `built_in_database`.
The slightly over-engineered pre-implementation to support
another backend is likely not going to fly as the EMQX
resource frame work is mostly for auth and data integration.
i.e. not generic enough for retained messages.
2024-02-19 14:36:12 +01:00
Zaiming (Stone) Shi
68baa4dcaf
docs: fix i18n
2024-02-19 13:57:44 +01:00
Thales Macedo Garitezi
bee7fa630b
chore: bump app vsns
2024-02-19 09:05:13 -03:00
Thales Macedo Garitezi
dbb6725a47
fix(config): apply config upgrade to deprecated configs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11845
Fixes https://github.com/emqx/emqx/issues/12452
2024-02-19 09:02:43 -03:00
Kjell Winblad
def95aa22b
docs(HStreamDB bridge): make pool size descriptions better
2024-02-19 11:33:37 +01:00
Zaiming (Stone) Shi
866fea7648
Merge pull request #12530 from zmstone/0217-improve-frame-too-large-logging-context
...
0217 improve frame too large logging context
2024-02-19 11:33:31 +01:00
zhongwencool
2ac4bde102
Merge pull request #12487 from zhongwencool/rabbitmq-source-test
...
feat: don't publish mqtt message in rabbitmq's source
2024-02-19 17:36:12 +08:00
Zaiming (Stone) Shi
8b0e15e402
refactor: rename 'hint' to 'cause' for MQTT fram parse failure reason
...
'reason' is maybe the wrapping field's name, so it was not used.
'hint' however, per our logging convention, is usually a free text
description for human to read.
change to 'cause' here because the field is always an atom and
it's use as shutdown counter in esockd_connection_sup
2024-02-19 09:38:35 +01:00
Xinyu Liu
0fef19f86f
Merge pull request #12425 from emqx/clickhouse-bridge-v2
...
refactor: split clickhouse bridges to actions and connectors
2024-02-19 09:20:59 +08:00
Zaiming (Stone) Shi
668d3be390
test: for better test coverage
2024-02-18 19:23:35 +01:00
Zaiming (Stone) Shi
22dabcb3ea
feat(mqtt): add more logging context for frame_too_large error
2024-02-18 18:33:12 +01:00
Shawn
ce0c27e802
fix: correct the swagger example for clickhouse connectors
2024-02-18 14:49:58 +08:00
JimMoen
6aae3ba2ed
refactor(prom_test): assert fun prefix
2024-02-18 10:42:40 +08:00
Shawn
8ae0e78786
refactor: split clickhouse bridges to actions and connectors
2024-02-18 10:01:53 +08:00
Shawn
451c8ecaf5
ci: add env vars to run clickhouse tests locally
2024-02-18 10:00:47 +08:00
JimMoen
c61b558423
test(prom): start mock pushgateway
2024-02-18 03:39:18 +08:00
JimMoen
ba1d24d054
test(prom_api): '/prometheus/auth' and '/prometheus/data_integration'
2024-02-18 02:32:25 +08:00
JimMoen
7bfabd7865
refactor: meck license checker for prom cases
2024-02-18 02:32:21 +08:00
JimMoen
d45d925529
test(prom_api): /prometheus/stats in prom/json format
2024-02-18 02:14:36 +08:00
JimMoen
9e16f33c31
test: /prometheus/stats endpoint format in json
2024-02-18 02:14:34 +08:00
Zaiming (Stone) Shi
52cfeee2b1
Merge pull request #12526 from zmstone/0216-fix-flaky-tests
...
test(listeners): try to stop all listeners after each test
2024-02-16 15:33:04 +01:00
Zaiming (Stone) Shi
9ffda0187f
Merge pull request #12525 from zmstone/0216-docs-add-ref-to-bridge_mode
...
docs: add a reference to the origin of is_bridge and bridge_mode
2024-02-16 15:32:52 +01:00
ieQu1
b7f3f48d38
Merge pull request #12524 from emqx/fix/bridge-stop-snabbkaffe
...
test(bridge_mqtt): Stop snabbkaffe servers
2024-02-16 15:17:29 +01:00
Zaiming (Stone) Shi
2b46cbab7a
Merge pull request #12517 from zmstone/0213-hocon-multiline-string-with-indentation
...
0213 hocon multiline string with indentation
2024-02-16 15:12:46 +01:00
Zaiming (Stone) Shi
f78c30c9ff
test: reduce false warning messages in test logs
2024-02-16 14:38:51 +01:00
Zaiming (Stone) Shi
3fa262f9ca
test(listeners): try to stop all listeners after each test
...
- Reduce the `eaddrinuse` flakiness
- Use ?FUNCTION_NAME as listener name to avoid name clashing between
tests
- Call emqx:remove_config for adhoc listeners created for testing
tombestone is designed for default listeners
2024-02-16 14:37:06 +01:00
ieQu1
8cfb22f0b8
fix(ds): Retry getting the shard leader
2024-02-16 12:42:48 +01:00
Zaiming (Stone) Shi
17a0513962
docs: add a reference to the origin of is_bridge and bridge_mode
2024-02-16 12:24:57 +01:00
ieQu1
59e4db98f7
test(bridge_mqtt): Stop snabbkaffe servers
2024-02-16 12:14:19 +01:00
Zaiming (Stone) Shi
497f631b7c
Merge pull request #12522 from zmstone/0216-upgrade-kafka-protocol
...
chore: upgrade kafka_protocol from 4.1.3 to 4.1.5
2024-02-16 11:47:31 +01:00
Zaiming (Stone) Shi
f57f617ba3
refactor(schema): ensure roots/0 and namespace/0 for all schema modules
2024-02-16 11:35:32 +01:00
Andrew Mayorov
a945892c52
Merge pull request #12508 from keynslug/fix/explicit-tab-mgmt
...
feat: manage mria tables explicitly during apps startup
2024-02-16 11:11:59 +01:00
Zaiming (Stone) Shi
242df0ffe9
chore: upgrade kafka_protocol from 4.1.3 to 4.1.5
2024-02-16 10:33:53 +01:00
Zaiming (Stone) Shi
a389d78b42
chore: delete stale code
2024-02-16 10:32:13 +01:00
Zaiming (Stone) Shi
b24321a8fc
feat: upgrade to hocon-0.41.0
...
hocon 0.41.0 added support for multiline string indentation.
now there is no need to escape (unless there is backslash)
quotes and line feeds etc.
2024-02-16 10:32:13 +01:00
ieQu1
ab2a469aff
Merge pull request #12489 from ieQu1/dev/ds-retainer
...
Durable sessions support retained messages
2024-02-15 21:59:32 +01:00
Andrew Mayorov
4484578c1f
Merge pull request #12507 from keynslug/fix/gw-app-no-scan
...
fix(gateway): avoid scanning modules of known gateway apps
2024-02-15 17:11:49 +01:00
ieQu1
5f85105801
feat(sessds): Specialize the interval queue for positive numbers
2024-02-15 16:55:05 +01:00
ieQu1
3adbe65a58
refactor(sessds): Unify logic for QoS 1 and 2
2024-02-15 11:34:13 +01:00
ieQu1
c781240459
feat(sessds): Add support for the retainer
...
Note: this is currently not ideal. Retained messages won't be
redelivered.
2024-02-15 11:34:13 +01:00
Kjell Winblad
5e442fcbca
fix: labels and descriptions
2024-02-15 11:08:08 +01:00
Andrew Mayorov
6514659733
test(mgmt): update and simplify testsuites setup
2024-02-15 11:04:58 +01:00
ieQu1
94254ec05b
feat(sessds): Correct handling of gaps in the seqno series
2024-02-15 10:49:02 +01:00
Serge Tupchii
0d0e26d6af
fix: set info level for potentially flooding log events
2024-02-15 11:02:03 +02:00
Andrew Mayorov
cffb52ab28
test(plugins): update and simplify test suite
2024-02-14 19:00:04 +01:00
Kjell Winblad
4e8dfb48b7
fix: elvis problems
2024-02-14 17:54:14 +01:00
Kjell Winblad
a153d758c3
feat: refactor HStreamDB bridge to connector and action
...
This commit also upgrades the hstreamdb_erl driver library and change
the action/bridge to use the new hstreamdb_erl.
Much of the code for the new API is copied from:
be1a1604dd/lib-ee/emqx_rule_actions/src/emqx_backend_hstreamdb_actions.erlhttps://emqx.atlassian.net/browse/EMQX-11458
2024-02-14 17:34:35 +01:00
Serge Tupchii
7272ef25d4
feat(emqx_auth): implement API to re-order all authenticators/authz sources
...
Fixes: EMQX-11770
2024-02-14 14:35:46 +02:00
Andrew Mayorov
35bf805504
test(exhook): update and simplify testsuites setup
2024-02-13 21:02:39 +01:00
Andrew Mayorov
ece1c6c6dc
test(prometheus): update and simplify testsuites setup
2024-02-13 20:36:23 +01:00
Andrew Mayorov
a64850a84b
test(sso): update and simplify testsuites setup
2024-02-13 20:36:23 +01:00
Andrew Mayorov
b15e81baf2
test(slowsub): update and simplify testsuite setup
2024-02-13 20:36:23 +01:00
Andrew Mayorov
d0e507eba4
test(trie): fix testsuite setup
2024-02-13 20:36:22 +01:00
Andrew Mayorov
841fdea124
test(conf): update and simplify some testcases
2024-02-13 20:36:22 +01:00
Andrew Mayorov
0a9cbe3080
test(license): update and simplify tests
2024-02-13 20:36:22 +01:00
Andrew Mayorov
24dfa41722
feat(cth-suite): use cheaper heuristic for schema modules
2024-02-13 20:36:22 +01:00
Andrew Mayorov
8f2a4f7b19
fix(cth-suite): use cheaper check for loaded applications
2024-02-13 20:36:22 +01:00
Andrew Mayorov
82a4e6ef68
feat(psk): manage mria tables explicitly during startup
...
Also switch test suite to `emqx_cth_suite` tooling.
2024-02-13 20:36:22 +01:00
Andrew Mayorov
b07df487f0
feat(modules): manage mria tables explicitly during startup
2024-02-13 20:36:22 +01:00
Andrew Mayorov
b6d77c164e
feat(emqx-mgmt): manage mria tables explicitly during startup
2024-02-13 20:36:22 +01:00
Andrew Mayorov
21780e2126
feat(emqx-conf): manage mria tables explicitly during startup
2024-02-13 20:36:21 +01:00
Andrew Mayorov
8f7b0ac498
docs(jt808): fix doc references
2024-02-13 16:27:41 +01:00
Kjell Winblad
366827390e
Merge pull request #12488 from kjellwinblad/kjell/refactor/rocketmq_bridge/EMQX-11467
...
feat: refactor RocketMQ bridge to connector and action
2024-02-13 16:19:21 +01:00
Andrew Mayorov
0e1d27c836
fix(gateway): avoid scanning modules of known gateway apps
...
Since the set of known gateways is already predefined in `emqx_gateway_utils`
it makes no sense to scan over all modules just to find out the same set of
modules. Cutting out this scanning saves few seconds per each `emqx_conf`
application startup, which is especially noticeable when running tests.
2024-02-13 15:13:36 +01:00
Andrew Mayorov
6fbb6f6846
Merge pull request #12506 from keynslug/fix/EMQX-11830/recoverable
...
fix(s3-bridge): handle recoverable AWS errors
2024-02-13 12:44:23 +01:00
Andrew Mayorov
eff149e676
feat(emqx-auth-mnesia): manage mria tables explicitly during startup
2024-02-12 19:17:22 +01:00
Andrew Mayorov
21a5751575
feat(emqx): manage mria tables explicitly during startup
2024-02-12 19:17:09 +01:00
Andrew Mayorov
a4eac75b25
fix(s3-bridge): handle recoverable AWS errors
2024-02-12 18:20:03 +01:00
Ilya Averyanov
edd28be090
Merge pull request #12499 from savonarola/0209-emqx-banned-extended-rules
...
feat(banned): allow ban by clientid/username regexps, peerhost cidrs
2024-02-12 17:12:47 +02:00
Andrew Mayorov
a9fdf9f1f9
Merge pull request #12495 from keynslug/ft/EMQX-11830/s3-bridge-simple
...
feat(s3): introduce S3 connector and action
2024-02-12 16:05:22 +01:00
Andrew Mayorov
2b1231b980
fix(s3-bridge): ensure bridge and connector names are identical
2024-02-12 13:46:54 +01:00
Andrew Mayorov
0edc5b5992
docs(s3): fix README example
2024-02-12 10:44:18 +01:00
Kjell Winblad
976099f5fb
fix: cleanups due to problems found by @thalesmg
2024-02-12 10:16:17 +01:00
Zaiming (Stone) Shi
d4fb3e83f6
chore: upgrade to wolff 1.10.2
...
Wolff 1.10.2 added an enhancement which makes Kafka resource health
checks more lightweight and lowers the chance of false resource alerts.
2024-02-12 09:05:44 +01:00
Ilya Averyanov
90fd2b26d3
feat(banned): allow ban by clientid/username regexps, peerhost cidrs
2024-02-10 17:59:22 +03:00
Andrew Mayorov
0515c5528f
test: add facility for starting emqx_dashboard with emqx_cth_suite
2024-02-09 14:29:04 -03:00
Andrew Mayorov
c108262771
chore(s3-bridge): use emqx_connector_schema helper for brewity
2024-02-09 16:08:33 +01:00
Serge Tupchii
8ac8d14abd
chore(emqx_mongodb): bump app vsn to 0.1.6
2024-02-09 16:29:42 +02:00
Serge Tupchii
c3b7d76102
perf: un-deprecate mongodb topology pool_size and set it to 10 by default
...
Benchmarks proved that it effects performance:
10 ecpool workers / 10 topology pool workers outperform
100 ecpool / 1 topology worker.
Fixes: EMQX-11825
2024-02-09 16:28:53 +02:00
Serge Tupchii
e248cd6c66
perf: upgrade mongodb-erlang to v3.0.23
...
Fixes: EMQX-11825
2024-02-09 16:18:32 +02:00
Andrew Mayorov
f476f4343b
fix(release): add emqx_bridge_s3 to applications list
2024-02-09 14:16:32 +01:00
Andrew Mayorov
2a4e37869e
fix(dashboard): provide full context in startup errors
2024-02-09 14:16:32 +01:00
Andrew Mayorov
802c760406
feat(s3): introduce S3 connector and action
...
This is a trivial connector based on `emqx_s3` and simple action
that maps each incoming event into an S3 object upload. Due to
current `emqx_s3` limitation this bridge is compatible with
backends providing S3 API with path-style bucket access.
2024-02-09 14:16:32 +01:00
SergeTupchiy
755b59b7fe
Merge pull request #12494 from SergeTupchiy/EMQX-11825-mongodb-connector-perf
...
Improve mongodb connector performance
2024-02-09 13:12:12 +02:00
Zaiming (Stone) Shi
cbebc59115
Merge pull request #12492 from zmstone/0208-fix-return-receive-maximum-in-connack
...
fix(emqx_channel): return Receive-Maximum in CONNACK when no error
2024-02-09 12:08:46 +01:00
Andrew Mayorov
8f66bd9ddf
fix(s3-client): make log levels saner
2024-02-09 12:03:41 +01:00
Andrew Mayorov
4ff04ab1f3
feat(s3): separate concepts to make app reusable in bridges
2024-02-09 12:03:41 +01:00
Zaiming (Stone) Shi
a74c828e19
test: add test case to cover Receive-Maximum in CONNACK
2024-02-09 10:42:48 +01:00
Andrew Mayorov
a5266f68ec
feat(s3): switch schema to use secrets with loader support
...
This will make applications using `emqx_s3` follow the same
conventions as bridges and support loading secrets from files
at runtime.
2024-02-08 22:43:20 +01:00
Serge Tupchii
1d9ce709f4
perf: un-deprecate mongodb topology pool_size and set it to 10 by default
...
Benchmarks proved that it effects performance:
10 ecpool workers / 10 topology pool workers outperform
100 ecpool / 1 topology worker.
Fixes: EMQX-11825
2024-02-08 20:39:19 +02:00
Serge Tupchii
c9ee5addb4
perf: upgrade mongodb-erlang to v3.0.23
...
Fixes: EMQX-11825
2024-02-08 20:00:26 +02:00
Thales Macedo Garitezi
64367d834b
test(bridge_v2): actually check schema
2024-02-08 13:39:31 -03:00
Zaiming (Stone) Shi
cf22692c74
fix(emqx_channel): return Receive-Maximum in CONNACK when no error
2024-02-08 17:05:16 +01:00
Kjell Winblad
e284a83f73
feat: refactor RocketMQ bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11467
2024-02-08 16:21:58 +01:00
zhongwencool
a0f8e4f328
test: rafator rabbitmq test SUITE, delete redundance code
2024-02-08 12:38:18 +08:00
zhongwencool
90ba2977fe
feat: don't publish mqtt message in rabbitmq's source
2024-02-08 12:38:18 +08:00
ieQu1
811edb32a2
Merge pull request #12485 from ieQu1/dev/ds-fix-unsubscribe
...
fix(ds): Fix unsubscribe logic related to the replay
2024-02-07 21:38:25 +01:00
ieQu1
19c6d1127f
refactor(sessds): Extract subscription mgmt logic to separate module
2024-02-07 15:27:11 +01:00
ieQu1
3000a8f286
fix(sessds): Postpone deletion of the subscription until fully acked
2024-02-07 14:51:02 +01:00
lafirest
3f13aed3de
Merge pull request #12481 from lafirest/fix/action_info
...
fix(tdengine): remove redundant codes
2024-02-06 11:39:25 +08:00
zhongwencool
eace8f2fcb
Merge pull request #12423 from zhongwencool/rabbitmq-e560
...
feat: rabbitmq bridge v2 integration
2024-02-06 09:55:05 +08:00
ieQu1
30eb54e86b
fix(sessds): Delay unsubscribe until full ack of in-flight messages
2024-02-06 02:49:07 +01:00
ieQu1
a9c55f7568
feat(sessds): Consider #srs with only QoS0 messages fully acked
2024-02-06 02:49:07 +01:00
zhongwencool
d3db53626b
Merge pull request #12483 from zmstone/0205-docs-fix-typos
...
Improve emqx ctl conf cluster_sync CLI and help text
2024-02-06 09:11:35 +08:00
zhongwencool
3e599e5d8e
chore: add maybe feature enable on rabbitmq_bridge
2024-02-06 07:57:13 +08:00
zhongwencool
e3cb49db8e
chore: add copyright header
2024-02-06 07:57:13 +08:00
zhongwencool
688701eedb
chore: add desc for rabbitmq source
2024-02-06 07:57:13 +08:00
zhongwencool
b444c82a42
feat: rabbitmq bridge v2 integration
2024-02-06 07:57:12 +08:00
ieQu1
280fcd8c52
Merge pull request #12437 from ieQu1/dev/optimize_make_filter
...
Optimize emqx_ds_bitmask_keymapper:make_filter function.
2024-02-05 17:32:28 +01:00
ieQu1
4665837cf0
fix(ds): Apply review remarks
2024-02-05 16:52:06 +01:00
Zaiming (Stone) Shi
d7b0456bb0
feat: rename emqx ctl cluster_sync 'tnxid' to 'inspect'
2024-02-05 16:50:55 +01:00
Thales Macedo Garitezi
e5b14a38ed
Merge pull request #12472 from thalesmg/fix-bridgev2-api-multicall-m-20240202
...
fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
2024-02-05 10:28:28 -03:00
ieQu1
c7888ad1f1
Merge pull request #12475 from ieQu1/dev/lean-stream
...
Use a more compact data structure to represent streams
2024-02-05 13:55:24 +01:00
Thales Macedo Garitezi
e1ff276d76
Merge pull request #12471 from thalesmg/fix-config-upgrade-m-20240202
...
fix(config): apply config upgrade to deprecated configs
2024-02-05 09:05:12 -03:00
firest
b4a6be0e7c
fix(tdengine): remove redundant codes
2024-02-05 18:50:31 +08:00
lafirest
571786b6c5
Merge pull request #12480 from lafirest/fix/td_v1_schema
...
fix(tdengine): keep the bridge schema still compatible with v1
2024-02-05 18:17:55 +08:00
ieQu1
698ba3f271
fix(ds): Optimize emqx_ds_bitmask_keymapper:make_filter
...
This optimization makes idle polling faster
2024-02-05 10:54:19 +01:00
ieQu1
8edbec5929
refactor(ds): Clarify the language used in ds_bitmapper
2024-02-05 10:54:18 +01:00
Zaiming (Stone) Shi
75023f2ca3
Merge pull request #12442 from ieQu1/dev/ds-license-apache
...
Relicense apps/emqx_durable_storage under Apache 2.0
2024-02-05 10:16:52 +01:00
firest
4a2c5a3161
fix(tdengine): keep the bridge schema still compatible with v1
2024-02-05 13:22:17 +08:00
zhongwencool
f7b0d1bc9d
chore: add duplicated key assert in swagger.json
2024-02-04 11:53:37 +08:00
JimMoen
b1150a5f88
fix(schema): bridge oracle connector schema
2024-02-04 10:53:13 +08:00
ieQu1
2e2f3cb2aa
fix(sessds): Avoid stream hash collisions
2024-02-03 21:15:54 +01:00
ieQu1
2e56810ea2
refactor(ds): Use a simple improper list to represent the streams
2024-02-03 21:15:54 +01:00
Thales Macedo Garitezi
76d242df9b
fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
...
Fixes https://emqx.atlassian.net/browse/EMQX-11834
2024-02-02 17:36:08 -03:00
Thales Macedo Garitezi
76401a302a
fix(config): apply config upgrade to deprecated configs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11845
Fixes https://github.com/emqx/emqx/issues/12452
2024-02-02 15:12:08 -03:00
Ivan Dyachkov
ed8660cb68
Merge pull request #12458 from id/0202-sync-release-55
...
sync release-55
2024-02-02 17:20:38 +01:00
SergeTupchiy
81433282be
Merge pull request #12467 from SergeTupchiy/autocluster-dns-aaaaa-record-type
...
feat(emqx_conf): support AAAA cluster dns record type
2024-02-02 18:13:06 +02:00
Zaiming (Stone) Shi
b1a05c7b59
Merge pull request #12326 from zmstone/0111-unregister-session-with-timestamp
...
0111 unregister session with timestamp
2024-02-02 17:00:23 +01:00
ieQu1
e0b5d9fae7
Merge pull request #12464 from ieQu1/dev/feat-ds-cli
...
feat(ds): Add a CLI interface to inspect status of DS databases
2024-02-02 14:14:48 +01:00
Serge Tupchii
5e0c3d4051
feat(emqx_conf): support AAAA cluster dns record type
...
Ekka autocluster DNS already supports AAAA records, so it's only needed to update enum in schema.
2024-02-02 12:13:29 +02:00
Kjell Winblad
fa0baf2f38
Merge pull request #12439 from kjellwinblad/kjell/refactor/oracle_bridge/EMQX-11464
...
feat: refactor Oracle bridge to connector and action
2024-02-02 10:53:15 +01:00
Ivan Dyachkov
84b6d7d720
fix: remove 5.5.bpapi2 file
2024-02-02 10:38:50 +01:00
ieQu1
3d2ac97c61
feat(ds): Add a CLI interface to inspect status of DS databases
2024-02-02 10:11:01 +01:00
Zaiming (Stone) Shi
f0569d8ae8
refactor: use mria:async_dirty to group dirty ops
2024-02-02 09:26:43 +01:00
lafirest
24d8a9ab00
Merge pull request #12449 from lafirest/feat/td
...
feat(tdengine): improve the TDengine bridge to v2 style
2024-02-02 16:17:01 +08:00
Ivan Dyachkov
dd490de2e1
chore: emqx 5.5.0
2024-02-02 08:48:56 +01:00
Ivan Dyachkov
f1c7e716ce
chore: 5.5.0-rc.2
2024-02-02 08:48:56 +01:00
JianBo He
9aad7997ca
chore: compatible the contet-type sytanx
2024-02-02 08:48:56 +01:00
Ivan Dyachkov
dae3a94670
chore: 5.5.0-rc.1
2024-02-02 08:48:56 +01:00
