af3a604354
refactor: add emqx_connector_info behavior
...
This commit adds the behavior `emqx_connector_info`. The
`emqx_connector_info` behavior should be implement when creating a new
connector to provide information about the connector (such as connector
schema etc) to the `emqx_connetor` application.
The connector in the `emqx_bridge_dynamo` application has also been
refactored to use the new behavior (as a test to see that the behavior
is working as intended).
Fixes:
https://emqx.atlassian.net/browse/EMQX-11427
2024-03-18 13:50:23 +01:00
2a611e4508
chore: fix the error api desc
2024-03-18 18:27:28 +08:00
485d8dbbdb
feat: add the `get /source_types` endpoint to return all supported source
2024-03-18 18:11:06 +08:00
889eb48368
fix(shared-sub): update stats when unsub shared topic filter
2024-03-16 20:43:05 +08:00
59e280dc54
test: cluster consistented metrics and label for nodes running/stopped
...
- `emqx_cluster_sessions_count`
- `emqx_cluster_sessions_max`
- `emqx_subscriptions_shared_count`
- `emqx_subscriptions_shared_max`
- `emqx_cluster_nodes_running`
- `emqx_cluster_nodes_stopped`
2024-03-16 20:43:04 +08:00
55a8488a4f
test: fields in `/monitor_current`
...
- `retained_msg_count`
- `shared_subscriptions`
2024-03-16 20:43:03 +08:00
02b3292025
fix: cluster nodes running/stopped with node name as label
2024-03-16 20:39:34 +08:00
0e25eb287b
fix(prom): cluster session count/max is cluster consistented
2024-03-16 20:39:33 +08:00
62f8f889e1
refactor: func rename to fix typo
2024-03-16 20:39:32 +08:00
e56bee618b
fix: `shared_subscriptions` is cluster consistented
2024-03-16 20:39:31 +08:00
3f93780d8c
Merge pull request #12717 from id/0315-prep-5.6.0-rc.1
...
chore: 5.6.0-rc.1
2024-03-15 21:28:31 +01:00
e28992d586
chore: 5.6.0-rc.1
2024-03-15 17:08:00 +01:00
ea76dac760
chore: fix xref
2024-03-15 17:08:00 +01:00
1fecd01405
Merge pull request #12708 from kjellwinblad/kjell/dynamo_bridge/update_error/EMQX-11984
...
fix: DynamoDB connector status check takes too long
2024-03-15 16:20:19 +01:00
c22735b3f5
Merge pull request #12707 from ieQu1/dev/store-ds-session-ip
...
Store peer name in the durable session metadata.
2024-03-15 16:15:29 +01:00
a93f747afa
fix(sessds): Return peername of the disconnected client in the REST
2024-03-15 13:48:53 +01:00
b6cc9177a6
feat(sessds): Store peername in the persistent session state
2024-03-15 13:48:53 +01:00
e35e8847b9
Merge pull request #12685 from SergeTupchiy/refactor-emqx-mgmt-call-client
...
Refactor emqx mgmt call client
2024-03-15 10:01:49 +02:00
11ae04d810
test(ds): rm unused var warning
2024-03-14 17:16:24 -03:00
d8bdb3c9aa
fix(api/clients): drop expired sessions from durable storage
2024-03-14 20:57:53 +01:00
4e9f2c8f5d
fix: DynamoDB connector status check takes too long
...
The DynamoDB connector status checks takes very long when the server is
unavailable which makes the resource manager blocked for a long time.
This causes calls to update the Bridge config with the Bridge V1 API fail
due to a timeout when it calls the resource manager to remove the
channel.
A better fix would be to change the resource manager so that the status
check cannot block it for a long time. However, this is more complicated
so it needs to be done in a later commit. A new ticket has been created
for this task https://emqx.atlassian.net/browse/EMQX-12015 .
Fixes:
https://emqx.atlassian.net/browse/EMQX-11984
2024-03-14 17:21:55 +01:00
2ebc8dcc55
fix(ds): use `infinity` timeout when storing batches
2024-03-14 10:17:18 -03:00
e340ab7f1b
Merge pull request #12696 from thalesmg/fix-default-enable-r56-20240313
...
fix(bridges): fix default value for `enable` when attempting operations
2024-03-14 09:50:40 -03:00
69a9431471
Merge pull request #12665 from ieQu1/dev/sessds-hot-conf-disable
...
fix(sessds): Prevent hot update of session_persistence.enable config
2024-03-14 13:20:52 +01:00
d11949ac51
Merge pull request #12694 from thalesmg/fix-kconsu-tm-validation-r56-20240313
...
fix(kafka_consumer): validate topic mapping in v2 schema
2024-03-14 09:19:41 -03:00
93903eb04b
Merge pull request #12683 from kjellwinblad/kjell/kinesis2/fix/parameter_restrictions/EMQX-11983
...
fix(Amazon Kinesis Action): batch size restriction
2024-03-14 10:55:44 +01:00
cb77dea1e9
Merge pull request #12699 from zmstone/0313-fix-session-count-on-replicant-node
...
fix: do not crash on replicant node
2024-03-14 09:42:27 +01:00
afb7075f82
Merge pull request #12695 from lafirest/fix/opents
...
perf(opents): Improve the message processing efficiency of opentsdb
2024-03-14 09:32:42 +08:00
45fe0787ca
Merge pull request #12689 from lafirest/fix/ld_log
...
fix(ldap): lower the log level of LDAB to `debug`
2024-03-14 09:21:44 +08:00
4f97743652
Merge pull request #12687 from lafirest/fix/iotdb
...
fix(iotdb): Fix function_clause when the convert value is null
2024-03-14 08:52:18 +08:00
35fb6ee656
Merge pull request #12688 from lafirest/fix/tdengine
...
fix(tdengine): enhanced health check result, make it more sense
2024-03-14 08:50:04 +08:00
685f29d3f2
chore: upgrade hocon to 0.42.1
...
hocon 0.42.1 allows to use "b" or "B" (byte) unit in bytesize fields.
2024-03-13 19:30:28 +02:00
4ac6d0716a
Merge pull request #12698 from thalesmg/test-flaky-conf-r56-20240313
...
test(conf): fix flaky config sync testcases
2024-03-13 13:47:23 -03:00
ccd973d13e
fix: do not crash on replicant node
2024-03-13 16:42:39 +01:00
9d21372ff0
Merge pull request #12678 from kjellwinblad/kjell/dynamo_bridge/fix/error_log/EMQX-11934
...
fix: return error reason from dynamo connector status check
2024-03-13 15:27:08 +01:00
92088a5905
test(conf): fix flaky config sync testcases
...
Before the recent changes these testcases relied on the fact that
peer nodes that are part of the test cluster are started one after
the other. This is now not the case in general with `emqx_cth_cluster`,
so now we need to additionally find the "longest running node" in the
cluster before running test logic.
2024-03-13 11:26:49 -03:00
b587ebac0c
fix(bridges): fix default value for `enable` when attempting operations
...
Fixes https://emqx.atlassian.net/browse/EMQX-11999
Although the frontend doesn't send `enable` when creating a bridge/action/connector, the
default value in the schema is `true`, so when attempting to fetch it from the raw config
we should adhere to that default.
2024-03-13 11:25:22 -03:00
a58ee801b2
fix(tdengine): enhanced health check result, make it more sense
2024-03-13 21:30:30 +08:00
b156e55430
perf(opents): Improve the message processing efficiency of opentsdb
2024-03-13 21:06:52 +08:00
a852695950
fix(kafka_consumer): validate topic mapping in v2 schema
...
Fixes https://emqx.atlassian.net/browse/EMQX-12008
2024-03-13 09:57:46 -03:00
e514c4a2ac
fix(emqx_mgmt_api_clients): check that max_payload_bytes is higher than 0
2024-03-13 14:00:30 +02:00
e9febf231f
fix(ldap): lower the log level of LDAB to `debug`
2024-03-13 19:20:40 +08:00
7908e2d591
Merge pull request #12682 from kjellwinblad/kjell/hstream_bridge/fix/parameter_restrictions/EMQX-11939
...
fix(HStreamDB bridge/action): restrict configuration parameters
2024-03-13 11:40:53 +01:00
9d999920be
Merge pull request #12681 from kjellwinblad/kjell/rocketmq_conn/remove_secret_from_logs/EMQX-11987
...
fix: redact secrets from RocketMQ debug log
2024-03-13 11:39:47 +01:00
d723a5edf7
Merge pull request #12670 from JimMoen/feat-shared-sub-count-monitor-current
...
feat(api): field `shared_subscriptions` in endpoint `/monitor_current`
2024-03-13 17:09:30 +08:00
65be76aa06
refactor(emqx_mgmt): add call_client timeout and improve RPC error handling
2024-03-13 10:56:02 +02:00
4b3c6f8330
fix(iotdb): Fix function_clause when the convert value is null
2024-03-13 11:28:07 +08:00
8be02327b2
refactor(emqx_mgmt): avoid call_client RPC to all runing nodes if global cm_registry is enabled
2024-03-12 20:58:53 +02:00
4cd434dbe3
docs(kinesis action): add description for action resource_opts
2024-03-12 18:38:39 +01:00
1edf284fed
test: trigger hocon validators when checking schema
2024-03-12 14:32:16 -03:00
c90f4f5794
fix(DynamoDB connector): fix error in status check when no workers
2024-03-12 17:22:22 +01:00
7e73b79b75
fix(Amazon Kinesis Action): batch size restriction
...
Make sure that the Amazon Kinesis action has the same batch size
restriction as the Amazon Kinesis bridge.
Fixes:
https://emqx.atlassian.net/browse/EMQX-11983
2024-03-12 12:52:54 +01:00
da87abd9c8
fix(HStreamDB bridge/action): restrict configuration parameters
...
This commit restricts a few HStreamDB bridge/action parameters from
being any integer to being integers greater than 0. Lower values than 1
for these parameters resulted in runtime errors.
Fixes:
https://emqx.atlassian.net/browse/EMQX-11939
2024-03-12 11:01:15 +01:00
fc4603c078
fix: redact secrets from RocketMQ debug log
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11987
2024-03-12 10:32:25 +01:00
91514f3ace
refactor: clean up over complicated code
...
This commit cleans up overly complicated code and handles
the case when the worker pool is empty.
Thank you @thalesmg for suggesting this change.
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-03-12 10:22:29 +01:00
1d1cdc1009
fix: return error reason from dynamo connector status check
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11934
2024-03-12 10:22:08 +01:00
0edeff4786
test: fix flaky
2024-03-12 15:21:17 +08:00
bc9084db79
test: cpu utilization value
2024-03-12 14:14:46 +08:00
0297c7b83b
refactor: conditional enclosed in parentheses
2024-03-12 14:14:45 +08:00
207f38c42a
fix(vm): cpu usage/idle handled by single worker
2024-03-12 14:14:44 +08:00
f24a76e770
Merge pull request #12668 from emqx/leap-year-bug-sql-fun
...
fix: port the changes for date_to_unix_ts SQL fun from 4.4
2024-03-12 10:48:31 +08:00
552968e5fe
test(conf): fix flaky config sync testcases
...
Before the recent changes these testcases relied on the fact that
peer nodes that are part of the test cluster are started one after
the other. This is now not the case in general with `emqx_cth_cluster`,
so now we need to additionally find the "longest running node" in the
cluster before running test logic.
2024-03-11 13:03:13 +01:00
d725206bcb
Merge pull request #12624 from keynslug/fix/EMQX-11901/ds-error-class
...
feat(sessds): handle recoverable errors during replay
2024-03-11 12:41:45 +01:00
b673ad3e5c
Merge pull request #12673 from zmstone/0308-fix-base64-compatibility-on-otp25
...
fix(mgmt): avoid using base64:decode/2 and encode/2
2024-03-11 13:23:37 +02:00
a82d4e1f7a
fix: `retained_msg_count` and `shared_subscriptions` api schema
2024-03-11 17:44:11 +08:00
e7574a70fb
feat(api): field `shared_subscriptions` in endpoint `/monitor_current`
2024-03-11 17:44:10 +08:00
980a4c3a4d
fix(emqx_mgmt_api): use emqx_utils:bin_to_hexstr/2
2024-03-11 10:40:19 +02:00
f376aa8072
refactor(emqx_mgmt_api_clients): remove unnecessary function
2024-03-11 10:39:09 +02:00
68682c4231
Merge pull request #12671 from kjellwinblad/kjell/rule_engine/fix/unescape_rules_from_dashboard/EMQX-11847
...
feat(rule engine SQL): add an `unescape` function
2024-03-08 17:57:31 +01:00
2cc1c563df
fix(mgmt): avoid using base64:decode/2 and encode/2
...
they are not available in otp 25
2024-03-08 16:46:54 +01:00
d92fd3e42d
Merge pull request #12667 from thalesmg/ds-delete-m-20240307
...
feat(ds): implement `get_delete_streams`, `make_delete_iterator` and `delete_next` callbacks for builtin storage
2024-03-08 10:57:51 -03:00
79d7821222
Merge pull request #12655 from thalesmg/fix-kconsu-status-r56-20240306
...
fix(kafka_consumer): check client connectivity
2024-03-08 10:53:44 -03:00
6af01b916e
feat(ds): implement `get_delete_streams`, `make_delete_iterator` and `delete_next` callbacks for builtin storage
...
Part of https://emqx.atlassian.net/browse/EMQX-11841
2024-03-08 09:56:46 -03:00
5dfd864c1a
Merge pull request #12666 from thalesmg/test-flaky-machine-m-20240307
...
test(machine): attempt to stabilize flaky test
2024-03-08 08:57:35 -03:00
5a6f96212d
feat(rule engine SQL): add an `unescape` function
...
The added `unescape` function unescapes escape sequences, transforming
them back to their represented characters. The following escape
sequences are supported:
- Standard C escape sequences:
- `\n` for newline (LF)
- `\t` for horizontal tab (HT)
- `\r` for carriage return (CR)
- `\b` for backspace (BS)
- `\f` for formfeed (FF)
- `\v` for vertical tab (VT)
- `\'` for single quote (')
- `\"` for double quote (")
- `\\` for backslash (\)
- `\?` for question mark (?)
- `\a` for alert (bell, BEL)
- Hexadecimal escape codes:
- `\xH...` where `H...` is one or more hexadecimal digits (0-9, A-F,
a-f), allowing for the encoding of arbitrary utf32 characters.
If an escape sequence is not recognized, or if the hexadecimal escape
does not form a valid Unicode character, the function generates an
exception.
Fixes:
https://github.com/emqx/emqx/issues/12460
https://emqx.atlassian.net/browse/EMQX-11847
2024-03-08 12:05:06 +01:00
060e02c4c4
Merge pull request #12653 from kjellwinblad/kjell/rule_engine/fix/subbits/support_non_byte_sizes/EMQX-11943
...
fix(rule_engine): support non-byte sized input to bin2hexstr
2024-03-08 09:41:03 +01:00
6054499607
Merge pull request #12657 from kjellwinblad/kjell/rule_engine/fix/fun_calls_as_array_items/EMQX-11953
...
fix(rule SQL): allow expressions as array items
2024-03-08 09:39:46 +01:00
29111a2192
fix: dialyzer problems
2024-03-08 14:25:19 +08:00
163d095dca
fix: port the changes for date_to_unix_ts SQL fun from 4.4
2024-03-08 10:48:08 +08:00
963e0de0c3
fix(kafka_consumer): check client connectivity
...
Fixes https://emqx.atlassian.net/browse/EMQX-11945
2024-03-07 14:22:03 -03:00
f7e3afde16
test(ds): avoid introducing new macros
2024-03-07 16:49:20 +01:00
b52c9c0062
test(machine): attempt to stabilize flaky test
2024-03-07 10:23:02 -03:00
69427dc42d
test(ds): add tests for error mapping and replay recovery
2024-03-07 12:59:58 +01:00
e7e8771277
fix(sessds): set replay retry timer if initial `replay/3` fails
2024-03-07 12:59:58 +01:00
09905d78cd
chore(ds): make error handling slightly simpler
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-03-07 12:59:57 +01:00
b604c3dbd4
refactor(sessds): make replay error handling a bit more clear
...
Also leave a forgotten TODO.
2024-03-07 12:59:57 +01:00
3f3e33b2cb
fix(sessds): untangle pull and replay retry timers
...
And restore the convention that timer handler always manages only
its own timers.
2024-03-07 12:59:57 +01:00
b39c710ec2
fix(ds): tidy up few typespecs
2024-03-07 12:59:57 +01:00
1cf672e78d
feat(sessds): handle recoverable errors during replay
2024-03-07 12:59:57 +01:00
2146d9e1fe
feat(ds): introduce error classes in critical API functions
...
For now, only recoverable / unrecoverable errors are introduced.
2024-03-07 12:59:57 +01:00
1f38813cb9
Merge pull request #12561 from SergeTupchiy/EMQX-11861-client-mqueue-inflight-API
...
feat: add client mqueue/inflight messages API
2024-03-07 13:20:31 +02:00
c62776edaf
fix(sessds): Prevent hot update of session_persistence.enable config
2024-03-07 12:19:53 +01:00
a2e761681e
feat: add client mqueue/inflight messages API
2024-03-07 12:06:49 +02:00
6a2731f2da
Merge remote-tracking branch 'origin/master' into release-56
2024-03-07 09:36:37 +01:00
98034cb4dd
Merge pull request #12660 from zmstone/sync-5.5.1
...
Sync 5.5.1
2024-03-07 09:36:08 +01:00
c62dd56a4a
Merge pull request #12661 from thalesmg/ds-atomic-store-m-20240305
...
feat(ds): add atomic store API
2024-03-06 17:48:28 -03:00
f57f2fa1b7
chore: bump app version numbers
2024-03-06 19:37:06 +01:00
df3ebc5c56
Merge pull request #12659 from thalesmg/test-flaky-pulsar-r56-20240306
...
test(pulsar): fix flaky test
2024-03-06 15:29:01 -03:00
b421363661
Merge pull request #12656 from thalesmg/fix-gprodu-status-r56-20240306
...
fix(gcp_pubsub_producer): check for topic existence when creating action
2024-03-06 15:28:51 -03:00
5d87d400f4
feat(ds): add atomic store API
...
Part of https://emqx.atlassian.net/browse/EMQX-11841
2024-03-06 15:24:14 -03:00
dc16e59f2c
fix(gcp_pubsub_producer): check for topic existence when creating action
...
Fixes https://emqx.atlassian.net/browse/EMQX-11949
2024-03-06 13:43:50 -03:00
e99546e009
Merge remote-tracking branch 'origin/release-56' into sync-5.5.1
2024-03-06 17:27:54 +01:00
eacd803a37
test(pulsar): fix flaky test
2024-03-06 12:07:02 -03:00
fd7eacb953
test(elastic_search): fix flaky test
2024-03-06 11:38:22 -03:00
c206ee37e0
fix(rule SQL): allow expressions as array items
...
Fixes:
https://github.com/emqx/emqx/issues/12465
https://emqx.atlassian.net/browse/EMQX-11953
2024-03-06 15:38:19 +01:00
69114bc6c2
fix(rule_engine): support non-byte sized input to bin2hexstr
...
The rule engine subbits function can return a bitstring which size is
not divisible by 8. Therefore, it makes sense that the rule engine
function bin2hexstr can handle such bitstrings as well. This is fixed by
this commit.
Fixes:
https://github.com/emqx/emqx/issues/12586
https://emqx.atlassian.net/browse/EMQX-11943
2024-03-06 15:26:49 +01:00
78d5f76f70
Merge pull request #12652 from kjellwinblad/kjell/rule_engine/fix/subbits/EMQX-11942
...
fix: add subbits/4 and subits/5 rule_engine functions
2024-03-06 15:20:30 +01:00
fc8b5d4522
Merge pull request #12646 from zmstone/0304-rule-engin-func-fix-zone-shift-in-date-string-parse
...
fix(rule_func): time zone shift at wrong precision
2024-03-06 13:22:30 +01:00
f1e9da9048
Merge pull request #12604 from thalesmg/test-gcp-consu-tm-m-20240227
...
test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
2024-03-06 09:13:41 +08:00
1737df05c7
Merge pull request #12641 from zmstone/0304-improve-text-log-formatter
...
0304 improve text log formatter
2024-03-06 09:12:18 +08:00
8cf681ad3b
Merge pull request #12639 from kjellwinblad/kjell/fix_flaky_test_case/emqx_broker_SUITE.connected_client_count_group.quic.t_connected_client_count_transient_takeover
...
test(emqx_broker_SUITE): fix flaky test case
2024-03-05 18:02:07 +01:00
3285edc004
Merge pull request #12650 from savonarola/0305-refine-retainer-readme
...
chore(retainer): actualize README
2024-03-05 18:57:02 +02:00
3ce8dcaa3f
Merge pull request #12643 from thalesmg/test-flaky-elasticsearch-m-20240304
...
test(elastic_search): fix flaky test
2024-03-05 13:53:42 -03:00
24cb45a643
test(elastic_search): fix flaky test
2024-03-05 11:29:49 -03:00
69dcc69761
Merge pull request #12640 from thalesmg/sync-r56-m-20240304
...
sync `release-56` to `master`
2024-03-05 10:44:28 -03:00
365d054e01
fix: add subbits/4 and subits/5 rule_engine functions
...
The documentation for the family of subbits functions says that the
fifth and sixth parameters are optional (since they only make sense when
the forth parameter is 'integer'). However, before this commit
`subbits/4` and `subbits/5` did not exist.
Fixes:
https://emqx.atlassian.net/browse/EMQX-11942
https://github.com/emqx/emqx/issues/12587
2024-03-05 13:27:07 +01:00
676df7eb30
test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
...
Checks that, if a migrated bridge originally has a `topic_mapping` and is later updated
with V2 APIs (without topic mapping in the input), then the new V2 `topic` field prevails.
2024-03-05 09:12:24 -03:00
a840925a50
Merge pull request #12642 from thalesmg/ds-fix-drop-typespec-m-20240304
...
fix(ds): fix `drop_generation` typespec
2024-03-05 09:11:08 -03:00
c5489fee90
Merge remote-tracking branch 'origin/release-56' into sync-r56-m-20240304
2024-03-05 09:08:59 -03:00
ce50aed930
chore(retainer): actualize README
2024-03-05 13:08:36 +03:00
c8e42cf6b1
test(emqx_broker_SUITE): fix flaky test case
2024-03-05 10:14:07 +01:00
cb0066d639
chore: add tag for logs from MQTT connection modules
2024-03-05 08:39:53 +01:00
00b21cf9c5
Merge pull request #12634 from zmstone/0304-breaking-no-escape-for-triple-quote-string-characters
...
chore: upgrade to hocon 0.42.0
2024-03-05 08:32:50 +01:00
c2dcb507cf
Merge pull request #12637 from lafirest/merge-55
...
sync release-55 to master
2024-03-05 08:47:30 +08:00
e725064a2a
fix(emqx_conf_schema): use `timeout_duration_s()` type for `log.throttling.time_window` field
2024-03-04 22:41:28 +02:00
9929025820
fix(rule_func): time zone shift at wrong precision
2024-03-04 21:25:49 +01:00
61328eb91a
chore: prepare e5.6.0-alpha.2
2024-03-04 15:37:58 -03:00
3e3194fd21
chore: upgrade to hocon 0.42.0
2024-03-04 19:12:42 +01:00
06334798a5
fix(ds): fix `drop_generation` typespec
...
This typespec fix will be used downstream by other backends.
2024-03-04 14:15:59 -03:00
6c9eb16a95
refactor(logger): reorder log fields
...
tag > clientid > msg > peername > username > topic > [other fields]
2024-03-04 18:10:19 +01:00
577ef41e45
Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304
2024-03-04 12:48:06 -03:00
cdcab23c54
Merge pull request #12615 from id/0229-prepare-5.5.1-release
...
chore: prepare 5.5.1 release
2024-03-04 16:34:13 +01:00
79f0720d14
Merge pull request #12635 from lafirest/fix/r551
...
fix(http): fix that sensitive headers may be printed in log when querying
2024-03-04 22:32:14 +08:00
7780b4f1d7
chore: prepare 5.5.1 release
2024-03-04 15:15:10 +01:00
6d9b2a95c3
Merge remote-tracking branch 'origin/release-55'
2024-03-04 20:47:20 +08:00
0dd9990f24
Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304
2024-03-04 09:44:30 -03:00
8b66ea7852
Merge pull request #12625 from thalesmg/test-retainer-extensions-m-20240301
...
test(retainer): extend test suite for usage by other backends
2024-03-04 09:40:32 -03:00
c5eb09a72f
fix(http): fix that sensitive headers may be printed in log when querying
2024-03-04 20:36:01 +08:00
3e7311ee31
Merge pull request #12619 from JimMoen/EMQX-11468/refactor/sqlserver
...
feat: refactor MS SQL Server bridge to connector and action
2024-03-04 18:05:14 +08:00
2e792da934
Merge pull request #12562 from ieQu1/dev/new-ds-schema
...
Refactor schema for the durable storage
2024-03-04 10:33:27 +01:00
4623b73f47
fix: sqlserver connector auto reconnect
2024-03-04 17:13:25 +08:00
2e3003e0f1
test: sqlserver bridge v2
2024-03-04 17:13:24 +08:00
1d58092176
fix: bridge v2 `on_query` matched ChannelId
2024-03-04 17:13:23 +08:00
00d50479f5
fix: compatible with bridge v1
2024-03-04 17:13:22 +08:00
9143d5994d
feat: refactor MS SQL Server bridge to connector and action
2024-03-04 17:13:21 +08:00
0a33f9d027
fix(calendar): leap year time to unix timestamp
2024-03-04 15:37:49 +08:00
9f61239704
Merge remote-tracking branch 'upstream/release-56' into 0304-sync-release-56
2024-03-04 08:00:39 +01:00
e9a5670cce
Merge pull request #12581 from zmstone/0223-add-json-schema
...
feat: schema registry supports json schema
2024-03-02 10:09:31 +01:00
08ef2c7b8b
test(retainer): extend test suite for usage by other backends
...
Part of https://emqx.atlassian.net/browse/EMQX-11922
2024-03-01 17:27:15 -03:00
a20478fff0
chore: update scripts and workflows for 5.6.x
2024-03-01 09:01:34 +01:00
fa275bf0a2
refactor(schema_registry): use one func for both encode and decode check
2024-03-01 07:20:59 +01:00
1c1c4e497d
chore: bump version && update change
2024-03-01 12:42:19 +08:00
5a3a34cce7
fix(redact): enhanced the redact for sensitive headers
2024-03-01 12:30:26 +08:00
0670272188
fix: Revert "fix: redact all headers from logs"
...
This reverts commit d8032f47ca
2024-03-01 10:32:57 +08:00
d56fb22208
Merge pull request #12595 from thalesmg/kafka-consumer-source-m-20240223
...
feat: migrate kafka consumer bridge to source + connector
2024-03-01 10:11:16 +08:00
b7e5ea2941
feat(schema_registry): add JSON schema
2024-02-29 21:03:52 +01:00
5d94a910b9
test: reduce flakiness
2024-02-29 13:59:43 -03:00
21cf600242
chore: address previous review comments
2024-02-29 16:13:52 +01:00
66fbcdcefa
test: fix a compile warning
2024-02-29 16:13:52 +01:00
5afe000ada
Merge pull request #12612 from zmstone/0226-fix-schema-registry-replication
...
refactor: change schema registry SERD_TAB to ets
2024-02-29 14:58:41 +01:00
5bb769e659
Merge pull request #12616 from savonarola/0229-ds-interface
...
feat(ds): export types
2024-02-29 15:42:42 +02:00
16b3dc1166
refactor: use individual keys for subscriber id resources
2024-02-29 10:39:37 -03:00
bc9de20024
Merge pull request #12540 from zhongwencool/bridge-pulsar-v2
...
feat: pulsar bridge v2
2024-02-29 21:25:11 +08:00
0aa9a872a3
feat: check `brod_sup` for client id
2024-02-29 09:53:23 -03:00
b82973b36b
Merge pull request #11868 from qzhuyan/fix/william/takeover-pub-willmsg
...
fix: willmsg not published in takeover
2024-02-29 13:14:18 +01:00
b706caf294
feat(ds): export types
2024-02-29 14:27:18 +03:00
f87c17b540
Merge pull request #12611 from savonarola/0228-ds-interface
...
feat(ds): update delete/count interface
2024-02-29 10:31:51 +02:00
d4519bda82
Merge pull request #12589 from savonarola/0222-retainer-interface
...
Make retainer pluggable
2024-02-29 10:31:15 +02:00
3814203fa2
test(pulsar): add pulsar action v2 testcase
2024-02-29 16:23:37 +08:00
e9e1daf962
chore: update some pulsar's desc
2024-02-29 16:23:37 +08:00
650f9659a4
fix: create pulsar producer when on_add_channel
2024-02-29 16:23:37 +08:00
7f1b4cef27
feat: pulsar bridge v2
2024-02-29 16:23:37 +08:00
21e0ecfcce
Merge pull request #12583 from zhongwencool/wait-for-listener-stop
...
chore: wait_for cowboy listener release sock when stop_listener
2024-02-29 15:03:22 +08:00
d2dda2535b
Merge pull request #12602 from lafirest/fix/iotdb_ping
...
fix(iotdb): use the `ping` API to check the status of the IOTDB connections
2024-02-29 12:26:10 +08:00
0bebd66f05
fix(iotdb): make dialyzer happy
2024-02-29 10:51:12 +08:00
49b508544e
chore: port not released msg
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-02-29 10:49:29 +08:00
10c1245125
refactor: change schema registry SERD_TAB to ets
...
Prior to this change, it's a mria/mnesia table
which may cause nodes to overwrite each other.
2024-02-28 23:13:07 +01:00
d5ae0e5c53
feat(ds): update delete/count interface
2024-02-28 22:51:24 +03:00
704eceb7c7
Merge pull request #12599 from zmstone/0227-refactor-log-config-template
...
docs: add log docs
2024-02-28 17:58:14 +01:00
ef4ae92da4
Merge pull request #12523 from keynslug/ft/bump-ekka-0.19.0
...
refactor: bump ekka to 0.19.0 w/o mnesia boot phase
2024-02-28 17:46:29 +01:00
dfdf7455d3
feat(retainer): make additional implementations pluggable
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-28 18:43:17 +03:00
d94066a10a
chore(retainer): refactor types; remove hotwiring to builtin config
2024-02-28 18:43:16 +03:00
aa2ee9c409
chore(retainer): add backend creation to the behaviour
2024-02-28 18:43:16 +03:00
d3fb72733e
Merge pull request #12610 from lafirest/fix/ldap_bind_health
...
fix(ldap): fixed that the connection to the LDAP connector could be disconnected after a period of time
2024-02-28 23:03:10 +08:00
eaef753c36
Merge pull request #12608 from lafirest/fix/iotdb
...
fix(iotdb): fix function clause error when there is no `payload` field
2024-02-28 22:00:54 +08:00
20c4029b33
test(cluster-rpc): unload any mocks in `end_per_suite/1`
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-28 14:52:37 +01:00
b9f644c355
fix(ldap): fixed that the connection to the LDAP connector could be disconnected after a period of time
2024-02-28 21:46:03 +08:00
ce2552712f
chore: bump version && update change
2024-02-28 18:32:08 +08:00
c3b044a37b
fix(api-topics): respond with `count` when it's cheap to compute
2024-02-28 11:28:33 +01:00
bd578a799e
fix(api-topics): avoid doing full scans over router tables
2024-02-28 10:52:44 +01:00
9bb0868628
test(api-topics): add testcase verifying paged queries work
2024-02-28 10:52:41 +01:00
01207ef97b
fix(iotdb): fix function clause error when there is no `payload` field
2024-02-28 16:54:01 +08:00
660a1ce99d
chore: use gen_tcp:recv timeout to wait socket closed
2024-02-28 16:18:32 +08:00
6fd04e33f5
fix(prom): skip cert info for disabled ssl/wss/quic listeners
2024-02-28 15:39:33 +08:00
5f125047f5
refactor: remove unnecessary lists fold for `cerfile`
2024-02-28 14:53:20 +08:00
812e8ef314
fix: try-catch for unknow reason to inhibit api crash
2024-02-28 14:53:19 +08:00
4307aecefd
chore: wait_for cowboy listener release sock when stop_listener
2024-02-28 14:24:15 +08:00
c3a2cf6220
chore: update change && bump version
2024-02-28 13:18:48 +08:00
042a84c30f
fix(prom): api crash when tls cert file non existed
2024-02-28 11:35:55 +08:00
aa7def425d
fix: get grpc_timeout from hstreamdb connector config failed
2024-02-28 08:45:06 +08:00
28d664bae2
test(pulsar): simplify the test suite
2024-02-27 23:51:54 +01:00
367ffa8e80
feat(utils-fs): add function to compute relpaths naively
2024-02-27 23:41:50 +01:00
6b9844ae82
feat: migrate kafka consumer bridge to source + connector
...
Fixes https://emqx.atlassian.net/browse/EMQX-11848
2024-02-27 17:52:33 -03:00
787dcd0e86
Merge pull request #12577 from thalesmg/gcp-service-json-binary-m-20240223
...
feat(gcp_pubsub_bridges): make service account json a binary
2024-02-27 15:42:42 -03:00
ef99037335
docs: add log docs
2024-02-27 19:24:56 +01:00
6ea9d2a6d9
test(conf): simplify confsync test suite
2024-02-27 16:41:54 +01:00
ae79516fd2
test(cluster-rpc): ensure testsuite is side-effects-free
2024-02-27 16:41:54 +01:00
56eefe34d5
feat(cth-cluster): use workdir as cwd on each node
2024-02-27 16:41:53 +01:00
d60ff1e616
test(plugins): avoid using same workdir for different nodes
2024-02-27 16:41:53 +01:00
3a008c8b4c
refactor: bump ekka to 0.19.0 w/o mnesia boot phase
2024-02-27 16:41:50 +01:00
8a204f2ac1
Merge pull request #12596 from thalesmg/fix-header-logging-r55-20240226
...
fix: redact all headers from logs
2024-02-27 11:19:51 -03:00
77239da7ed
fix(iotdb): use the `ping` API to check the status of the IOTDB connections
2024-02-27 22:07:41 +08:00
02de92a53d
Merge pull request #12601 from lafirest/fix/eldap_log
...
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 21:50:05 +08:00
2ded751237
test: `nl` not allowed for shared-sub
2024-02-27 18:24:30 +08:00
54c542c795
chore: rename _probe_ to t_probe_
2024-02-27 17:57:15 +08:00
c67f2130f2
fix: check connector and bridge_v2 with the right schema
2024-02-27 17:57:10 +08:00
c6b1102b2b
fix(mgmt_client): `nl` not allowed for shared-sub
2024-02-27 17:51:39 +08:00
f3237a1bf7
chore: bump emqx_ldap version
2024-02-27 17:48:34 +08:00
5c69500c52
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 17:48:22 +08:00
9ae7ac04a2
Merge pull request #12597 from lafirest/fix/ldap_log
...
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 17:40:44 +08:00
1567c5beb0
Merge pull request #12580 from zmstone/0224-delete-stale-macro
...
chore: delete stale macro
2024-02-27 09:58:06 +01:00
9c9a2b8494
Merge pull request #12592 from zmstone/0226-duration_s-min-value
...
chore: for dashboad add a 'minimum' attribute for duration_s type
2024-02-27 09:57:33 +01:00
da13ec736c
chore: bump emqx_ldap version
2024-02-27 16:08:47 +08:00
c1da7449fe
test(mgmt_api): bulk/unbulk sub/unsub shared topic filter
2024-02-27 16:01:12 +08:00
a76415c4f6
fix(mgmt): sub/unsub a share subscription to the client via http api
...
- `/clients/:clientid/subscribe`
- `/clients/:clientid/subscribe/bulk`
- `/clients/:clientid/unsubscribe`
- `/clients/:clientid/unsubscribe/bulk`
2024-02-27 16:00:45 +08:00
6e73cfa2cf
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 15:47:46 +08:00
6c7b7741d5
feat(quic): mqtt 5.0 graceful shutdown in takeover
2024-02-26 23:05:49 +01:00
c8f9ffdc1f
chore: update some comments
2024-02-26 22:24:16 +01:00
88fc67b369
chore(willmsg): remove unreachable code
2024-02-26 22:24:16 +01:00
d5247cb567
refactor: update notes for willmsg
2024-02-26 22:24:16 +01:00
975c7429e5
chore: clean in tests
2024-02-26 22:24:13 +01:00
d8032f47ca
fix: redact all headers from logs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11904
Since headers are usually used for authentication and the headers used for that are very
flexible, we redact all headers from logs to avoid leaking anything.
2024-02-26 18:04:12 -03:00
bb6a95bae8
Merge pull request #12590 from zmstone/0226-remove-mfa-from-logs
...
refactor: delete mfa from log metadata
2024-02-26 21:37:57 +01:00
5326cc80ed
Merge pull request #12594 from savonarola/0226-add-ds-delete-interface
...
chore(ds): add delete callbacks
2024-02-26 20:07:21 +02:00
9ca5684b34
Merge pull request #12591 from thalesmg/dev-cls-m-20240226
...
test: make `clear_screen` safer
2024-02-26 13:40:16 -03:00
b010d34640
chore(ds): add delete callbacks
2024-02-26 17:35:13 +03:00
5d4c85cd84
chore: list throttled log events in the change-log
2024-02-26 15:09:38 +02:00
5b48b06d4a
fix: trace throttled log events
2024-02-26 15:09:38 +02:00
652e977b3c
chore: for dashboad add a 'minimum' attribute for duration_s type
2024-02-26 13:51:07 +01:00
c6195fcf50
test: make `clear_screen` safer
...
Useful when iterating on the tests in a loop, to get rid of all the garbaged printed
before the test itself beings.
Only actually does anything if the environment variable `CLEAR_SCREEN` is set to `true`
and only clears the screen the screen the first time it's encountered, so it's harmless
otherwise.
2024-02-26 09:31:06 -03:00
f2c372d9ff
feat(gcp_pubsub_bridges): make service account json a binary
...
Fixes https://emqx.atlassian.net/browse/EMQX-11384
Today, service_account_json config field is an embedded object (map()).
This requires user to embed a JSON object into the config file instead of embedding it as
a string.
We should support binary() type as input, but keep supporting map() for backward
compatibility.
2024-02-26 09:21:07 -03:00
00c2aeb1ab
refactor: delete mfa from log metadata
2024-02-26 12:15:00 +01:00
de5e4491f7
Merge pull request #12576 from zmstone/0223-hide-bridges-root-from-docs
...
docs: stop generating schema doc for 'bridges' root
2024-02-24 09:08:05 +01:00
94a01d23c1
chore: delete stale macro
...
EMQX_API_VERSION is no longer in use since 5.2.0
2024-02-24 09:06:25 +01:00
a3e81c5039
chore: prepare for release 5.5.1-rc.2
2024-02-24 08:59:51 +01:00
fe4c7cd2dc
fix(ds): Apply review remarks, and hide certain fields
2024-02-23 18:07:56 +01:00
e126393cbf
fix(ds): Fix schema checker warnings for DS schema
2024-02-23 15:39:52 +01:00
c18fc6a4bb
fix(sessds): Remove deprecated configuration parameters
2024-02-23 15:08:13 +01:00
91ddbbcc3f
fix(sessds): Replace min- and max- batch size with batch_size
2024-02-23 15:08:13 +01:00
94b0ab983d
docs(ds): Add descriptions for the builtin's egress config
2024-02-23 15:08:13 +01:00
17ab3c6362
chore(ds_schema): Use atoms for record and field names
2024-02-23 15:08:13 +01:00
24337ecec7
feat(sessds): Move config schema to a separate root
2024-02-23 15:08:13 +01:00
8907e5afb3
chore(sessds): Remove deprecated schema
2024-02-23 15:08:11 +01:00
2adfefe297
docs: stop generating schema doc for 'bridges' root
2024-02-23 14:51:57 +01:00
5af01c041b
Merge pull request #12559 from zmstone/0221-refactor-use-atom-fileds
...
refactor: use atoms for root config fields
2024-02-23 14:38:19 +01:00
b60b19a29a
fix(connectors): redact authorization headers when creating/updating connectors
...
Fixes https://emqx.atlassian.net/browse/EMQX-11895
2024-02-23 10:17:24 -03:00
15f919e60f
Merge pull request #12564 from thalesmg/bw-support-batch-list-resp-m-20240221
...
feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
2024-02-23 09:37:42 -03:00
cd091b4f75
Merge pull request #12572 from zhongwencool/api-config-error-log
...
fix(api): more detail error when don't allow update readonly key
2024-02-23 17:29:00 +08:00
f2619ce17a
Merge pull request #12571 from zhongwencool/delete-repeat-ci-test
...
test(bridge_v1): delete repeat probe_bridge_api code
2024-02-23 17:17:39 +08:00
fbf4afc3c5
fix(api): more detail error when don't allow update readonly key
2024-02-23 16:43:15 +08:00
46877e979b
chore: update copyright-year
2024-02-23 08:21:06 +01:00
d7faab4195
test(bridge_v1): delete repeat probe_bridge_api code
2024-02-23 14:43:42 +08:00
9f52b4fba8
Merge pull request #12566 from lafirest/fix/same_api
...
fix(api_key): enhanced bootstrap files for REST API keys
2024-02-23 10:57:37 +08:00
11d3df2775
Merge pull request #12569 from zhongwencool/fix-rabbitmq-action-crash-dropped-metrics
...
fix: rabbitmq action failed to increase dropped metrics
2024-02-23 09:53:55 +08:00
c08522bc1e
Merge pull request #12539 from zhongwencool/rocketmq-channel-status
...
fix: Rocketmq channel status
2024-02-23 09:50:10 +08:00
51a6ada015
fix: rabbitmq action failed to increase dropped metrics
2024-02-23 08:35:00 +08:00
eba74c1ca8
Merge pull request #12567 from thalesmg/fix-gcp-consumer-v1-api-m-20240222
...
fix(gcp_pubsub_consumer,mongodb_bridge): fix v1 config transformation for gcp pubsub consumer and mongodb bridges
2024-02-22 17:33:02 -03:00
d003f77021
feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
...
Fixes https://emqx.atlassian.net/browse/EMQX-11892
This allows callers of batching resources to receive results specific to their requests,
rather than a broad success or failure for the whole batch.
2024-02-22 16:26:02 -03:00
54499175d5
Merge pull request #12568 from keynslug/ft/EMQX-11888/s3-redact
...
fix(s3): try to obtain credentials with default lhttpc
2024-02-22 18:07:04 +01:00
2ff33f98ef
chore(willmsg): add come comments
2024-02-22 17:12:19 +01:00
e5a3574d89
fix: maybe send willmsg
2024-02-22 17:12:16 +01:00
a30e8818fd
fix(s3-bridge): check for unobtainable credentials in healthcheck
2024-02-22 16:56:14 +01:00
69c7f858af
fix(s3): try to obtain credentials with default lhttpc
...
Otherwise `emqx_s3_client` will try to do it later by contacting the S3
endpoint instead of the metadata server.
2024-02-22 16:56:13 +01:00
88b1d9ba88
refactor: use atoms for root config fields and types
2024-02-22 16:51:40 +01:00
6311b582ec
test(willmsg): session taken over before willmsg delay /session expire
2024-02-22 16:39:27 +01:00
5397402396
test(willmsg): test will delay and session expires
...
will delay > session expire
will delay < session expire
timer triggered events are handled in seq, exclude the case of
(will delay == session expire)
2024-02-22 16:39:27 +01:00
dd62280e04
fix: handle delayed willmsg, part 1
2024-02-22 16:39:27 +01:00
b76c701b1c
test(takeover): add back the delay when takeover
2024-02-22 16:39:27 +01:00
6243cf0b0c
fix(kick): defer willmsg publish when conn terminates
...
because kick means shutdown connection AND delete session
2024-02-22 16:39:27 +01:00
9da4896f57
fix(mqtt): ensure publish willmsg when session expires
2024-02-22 16:39:24 +01:00
07eec31a8a
fix: willmsg not published in takeover
2024-02-22 16:35:59 +01:00
47757a0983
fix(mongodb_bridge): remove `local_topic` from v1 config transformation
...
Fixes https://emqx.atlassian.net/browse/EMQX-11893
2024-02-22 12:23:26 -03:00
205b97f732
fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config
...
Fixes https://emqx.atlassian.net/browse/EMQX-11891
2024-02-22 10:27:28 -03:00
ae8f59979d
fix(utils): handle improper lists as well in `redact/1`
2024-02-22 13:35:11 +01:00
9893cc700b
Merge pull request #12565 from zhongwencool/fix-bpapi-bad-node
...
fix: bpapi announce's bad node name
2024-02-22 20:18:33 +08:00
f0aeaadc48
fix(api_key): enhanced bootstrap files for REST API keys
2024-02-22 18:40:24 +08:00
7bb9d5d8f6
Merge pull request #12543 from kjellwinblad/kjell/refactor/dynamodb/EMQX-11456
...
feat: refactor DynamoDB bridge to connector and action
2024-02-22 09:47:57 +01:00
ccf607ee2c
fix: bpapi announce's bad node name
2024-02-22 14:55:11 +08:00
985a3e3062
test: remove unused var warning
2024-02-21 18:14:46 -03:00
8e47503f7d
Merge pull request #12560 from SergeTupchiy/EMQX-11530-log-throttling-followup-fixes
...
fix: disable log throttling if primary log level is debug
2024-02-21 22:02:32 +02:00
f31e9e6a15
Merge pull request #12555 from thalesmg/ds-move-count-to-mgmt-m-20240221
...
refactor(mgmt): move persistent session counting function to mgmt module
2024-02-21 16:36:29 -03:00
ceb208f1eb
fix: disable log throttling if primary log level is debug
2024-02-21 21:19:15 +02:00
7feb444e31
Merge pull request #12557 from keynslug/fix/s3-bridge-startup
...
fix(s3-bridge): anticipate repeated connector start requests
2024-02-21 15:13:48 +01:00
9b6511b3e7
docs: fix spelling mistake in error message
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-21 14:57:55 +01:00
4678df2229
build: bump emqx_bridge_gcp_pubsub version
...
The CI check ./scripts/apps-version-check.sh requires that
emqx_bridge_gcp_pubsub's version number is bumped.
2024-02-21 14:59:03 +01:00
6561d989d6
feat: refactor DynamoDB bridge to connector and action
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11456
2024-02-21 14:58:48 +01:00
e98fb0db48
chore: bump `emqx_bridge_gcp_pubsub` version to 0.2.2
2024-02-21 14:28:58 +01:00
6b97983303
fix(s3-bridge): anticipate repeated connector start requests
...
Starting disconnected connector manually implies that there's no
attempt to stop it first.
2024-02-21 14:23:29 +01:00
191b435a48
chore: bump gcp bridge app version
2024-02-21 21:15:27 +08:00
ed23ad906a
fix: rabbitmq source metrics not working
2024-02-21 21:10:23 +08:00
415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
...
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
...
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
25cdc2a04b
chore: bump version to 5.5.1-rc.1
2024-02-21 09:57:05 -03:00
b14e51d7c4
docs: delete useless comments
2024-02-21 13:33:29 +01:00
03138f8345
refactor(mgmt): move persistent session counting function to mgmt module
...
Follow up to https://github.com/emqx/emqx/pull/12500#discussion_r1496634087
2024-02-21 09:12:56 -03:00
529211b9ac
Merge pull request #12500 from thalesmg/ds-list-client-api-m-20240209
...
feat(ds): list disconnected persistent sessions in clients API
2024-02-21 09:02:27 -03:00
92c00ee21a
Merge pull request #12550 from keynslug/fix/bridge-s3-resopts
...
fix(bridge-s3): expose connector-level `resource_opts` properly
2024-02-21 11:37:55 +01:00
333eb34300
test: call emqx_resource:remove_local
2024-02-21 10:53:22 +01:00
de93d1c0a3
chore: add bpapi.versions
2024-02-21 10:53:22 +01:00
9c1d5e49dd
chore: fix a typo
2024-02-21 10:53:22 +01:00
c8f9e12de1
refactor: deprecate emqx_resource_proto_v1 with v2
2024-02-21 10:53:22 +01:00
06e0a7f2ed
refactor: delete emqx_resource:remove api
...
this api is only used in tests,
changed to call the _local flavor instead
2024-02-21 10:53:22 +01:00
8541d2efc1
refactor: delete emqx_resource:recreate
...
this api is only used in tests, changed to call the _local flavor
instead
2024-02-21 10:53:22 +01:00
1878fded5a
refactor: delete emqx_resource:create/4 and /5
...
the functions are only used in tests,
now the tests are rewritten to call create_local/4 and /5 instead
2024-02-21 10:53:22 +01:00
0cbe626aed
Merge pull request #12549 from zmstone/0221-sync-release-55
...
Sync `release-55` with i18n fixes to master
2024-02-21 10:52:36 +01:00
d8b6ecd185
fix(bridge-s3): expose connector-level `resource_opts` properly
2024-02-21 10:49:04 +01:00
ee02079661
Merge pull request #12548 from zhongwencool/fix-rabbitmq-source-crash
...
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 16:34:43 +08:00
f7a7b7c8e0
Merge pull request #12545 from SergeTupchiy/fix-typo-tls_gc
...
chore(emqx_tls_certfile_gc): fix typo
2024-02-21 10:11:13 +02:00
da340c92a1
Merge remote-tracking branch 'origin/master' into 0221-sync-release-55
2024-02-21 09:11:12 +01:00
80be84c401
Merge pull request #12491 from zmstone/docs-sync-changes-for-i18n
...
docs: fix i18n
2024-02-21 09:10:25 +01:00
2a7a662c03
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 15:30:05 +08:00
65cfece0a3
chore: delete 5.5.bpapi2
...
> This particular warning won't be seen in the real life. Also, we need to start storing
bpapi files for each minor release, since the plan to use hot upgrade has been abandoned.
> ...The original procedure for BPAPI life cycle was designed under the assumption that
we'll support hot upgrade between minor releases, so we'll only ever need to keep one
BPAPI file per major release, and rewrite it whenever a new minor version is
released. We'll need to revisit it.
https://emqxio.slack.com/archives/C04H4RHKKEY/p1706862980546309
2024-02-20 17:35:28 -03:00
d469f4158e
chore: bump app vsns
2024-02-20 16:53:57 -03:00
74dbbf6d9d
Merge branch 'release-55' into sync-r55-m-20240220
2024-02-20 16:33:34 -03:00
dd9f96211b
chore(emqx_tls_certfile_gc): fix typo
2024-02-20 21:19:36 +02:00
3a4c7f60e2
feat(ds): list disconnected persistent sessions in clients API
...
Fixes https://emqx.atlassian.net/browse/EMQX-11540
Note that not all information provided by disconnected in-memory sessions is available to
disconnected persistent sessions, nor does all of them make sense.
2024-02-20 14:52:55 -03:00
1de9e9dd01
Merge pull request #12541 from zmstone/0220-fail-fqdn-node-name-if-dns-discovery
...
Fail fast if node host is not IP address but cluster strategy is `dns` with `a` or `aaaa` record type
2024-02-20 17:48:02 +01:00
b6689d178f
fix(http_bridge): redact headers
...
Fixes https://emqx.atlassian.net/browse/EMQX-11864
```
2024-02-20T15:51:34.744509+00:00 [debug] msg: http_connector_received, mfa: emqx_bridge_http_connector:on_query_async/4(471), tag: QUERY_ASYNC, request: {"/bah",[{<<"Authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>},{<<"authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>}],<<"******">>}, state: #{port => 18083,request => #{path => [<<>>],body => undefined,headers => [{[<<"content-type">>],[<<"application/json">>]},{[<<"Authorization">>],#Fun<emqx_secret.0.85191762>}],method => [<<"undefined">>],max_retries => 2,request_timeout => 30000},host => "localhost",connect_timeout => 15000,pool_type => random,pool_name => <<"connector:http:p1">>,base_path => "/",installed_actions => #{<<"action:http:p1:connector:http:p1">> => #{path => [<<"/bah">>],body => undefined,headers => [{[<<"authorization">>],#Fun<emqx_secret.0.85191762>},{[<<"content-type">>],[<<"application/json">>]}],method => [<<"post">>],max_retries => 2,request_timeout => 30000,render_template_func => fun emqx_bridge_http_connector:render_template/2}}}, connector: <<"connector:http:p1">>, note: the request body is redacted due to security reasons
```
2024-02-20 12:57:13 -03:00
dc015e7a6f
fix(schema): validate cluster strategy and node name
...
If cluster strategy is configured as `dns`, the node name
must be IP address
2024-02-20 16:09:54 +01:00
f72558737c
Merge pull request #12520 from SergeTupchiy/EMQX-11530-log-throttling
...
feat: implement log throttling
2024-02-20 15:52:46 +02:00
86c85f3b45
Merge pull request #12512 from kjellwinblad/kjell/refactor/hstreamdb/EMQX-11458
...
feat: refactor HStreamDB bridge to connector and action
2024-02-20 13:30:29 +01:00
9bd0d1ba1b
feat: enable log throttling for potentially flooding log events
2024-02-20 14:20:21 +02:00
98ba300f7c
feat: implement log throttling
2024-02-20 13:23:17 +02:00
9639a977a3
fix: hidden batch_time for mongodb
2024-02-20 09:42:49 +08:00
c7bfaf51fd
chore: typo error (templete -> template)
2024-02-20 09:41:09 +08:00
2561ff94ec
fix: check channel id exist when get_channel_status
2024-02-20 09:38:33 +08:00
146d89bd89
feat: convert `gcp_pubsub_consumer` to connector/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 17:54:26 -03:00
fd13a661cf
Merge pull request #12535 from zmstone/0219-refactor-delete-non-prod-code
...
refactor: delete non-prod code
2024-02-19 19:04:59 +01:00
11e4a295ed
fix: use connector type for constructing resource id
2024-02-19 14:42:35 -03:00
9a32895a1a
feat: convert `gcp_pubsub_consumer` to connector/source
...
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
ad53523e6b
refactor: delete non-prod code
...
So far the retainer backend type is always `built_in_database`.
The slightly over-engineered pre-implementation to support
another backend is likely not going to fly as the EMQX
resource frame work is mostly for auth and data integration.
i.e. not generic enough for retained messages.
2024-02-19 14:36:12 +01:00
68baa4dcaf
docs: fix i18n
2024-02-19 13:57:44 +01:00
bee7fa630b
chore: bump app vsns
2024-02-19 09:05:13 -03:00
dbb6725a47
fix(config): apply config upgrade to deprecated configs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11845
Fixes https://github.com/emqx/emqx/issues/12452
2024-02-19 09:02:43 -03:00
def95aa22b
docs(HStreamDB bridge): make pool size descriptions better
2024-02-19 11:33:37 +01:00
866fea7648
Merge pull request #12530 from zmstone/0217-improve-frame-too-large-logging-context
...
0217 improve frame too large logging context
2024-02-19 11:33:31 +01:00
2ac4bde102
Merge pull request #12487 from zhongwencool/rabbitmq-source-test
...
feat: don't publish mqtt message in rabbitmq's source
2024-02-19 17:36:12 +08:00
8b0e15e402
refactor: rename 'hint' to 'cause' for MQTT fram parse failure reason
...
'reason' is maybe the wrapping field's name, so it was not used.
'hint' however, per our logging convention, is usually a free text
description for human to read.
change to 'cause' here because the field is always an atom and
it's use as shutdown counter in esockd_connection_sup
2024-02-19 09:38:35 +01:00
0fef19f86f
Merge pull request #12425 from emqx/clickhouse-bridge-v2
...
refactor: split clickhouse bridges to actions and connectors
2024-02-19 09:20:59 +08:00
668d3be390
test: for better test coverage
2024-02-18 19:23:35 +01:00
22dabcb3ea
feat(mqtt): add more logging context for frame_too_large error
2024-02-18 18:33:12 +01:00
ce0c27e802
fix: correct the swagger example for clickhouse connectors
2024-02-18 14:49:58 +08:00
6aae3ba2ed
refactor(prom_test): assert fun prefix
2024-02-18 10:42:40 +08:00
8ae0e78786
refactor: split clickhouse bridges to actions and connectors
2024-02-18 10:01:53 +08:00
451c8ecaf5
ci: add env vars to run clickhouse tests locally
2024-02-18 10:00:47 +08:00
c61b558423
test(prom): start mock pushgateway
2024-02-18 03:39:18 +08:00
ba1d24d054
test(prom_api): '/prometheus/auth' and '/prometheus/data_integration'
2024-02-18 02:32:25 +08:00
7bfabd7865
refactor: meck license checker for prom cases
2024-02-18 02:32:21 +08:00
d45d925529
test(prom_api): `/prometheus/stats` in prom/json format
2024-02-18 02:14:36 +08:00
9e16f33c31
test: `/prometheus/stats` endpoint format in json
2024-02-18 02:14:34 +08:00
52cfeee2b1
Merge pull request #12526 from zmstone/0216-fix-flaky-tests
...
test(listeners): try to stop all listeners after each test
2024-02-16 15:33:04 +01:00
9ffda0187f
Merge pull request #12525 from zmstone/0216-docs-add-ref-to-bridge_mode
...
docs: add a reference to the origin of is_bridge and bridge_mode
2024-02-16 15:32:52 +01:00
b7f3f48d38
Merge pull request #12524 from emqx/fix/bridge-stop-snabbkaffe
...
test(bridge_mqtt): Stop snabbkaffe servers
2024-02-16 15:17:29 +01:00
2b46cbab7a
Merge pull request #12517 from zmstone/0213-hocon-multiline-string-with-indentation
...
0213 hocon multiline string with indentation
2024-02-16 15:12:46 +01:00
f78c30c9ff
test: reduce false warning messages in test logs
2024-02-16 14:38:51 +01:00
3fa262f9ca
test(listeners): try to stop all listeners after each test
...
- Reduce the `eaddrinuse` flakiness
- Use ?FUNCTION_NAME as listener name to avoid name clashing between
tests
- Call emqx:remove_config for adhoc listeners created for testing
tombestone is designed for default listeners
2024-02-16 14:37:06 +01:00
8cfb22f0b8
fix(ds): Retry getting the shard leader
2024-02-16 12:42:48 +01:00
17a0513962
docs: add a reference to the origin of is_bridge and bridge_mode
2024-02-16 12:24:57 +01:00
59e4db98f7
test(bridge_mqtt): Stop snabbkaffe servers
2024-02-16 12:14:19 +01:00
497f631b7c
Merge pull request #12522 from zmstone/0216-upgrade-kafka-protocol
...
chore: upgrade kafka_protocol from 4.1.3 to 4.1.5
2024-02-16 11:47:31 +01:00
f57f617ba3
refactor(schema): ensure roots/0 and namespace/0 for all schema modules
2024-02-16 11:35:32 +01:00
a945892c52
Merge pull request #12508 from keynslug/fix/explicit-tab-mgmt
...
feat: manage mria tables explicitly during apps startup
2024-02-16 11:11:59 +01:00
242df0ffe9
chore: upgrade kafka_protocol from 4.1.3 to 4.1.5
2024-02-16 10:33:53 +01:00
a389d78b42
chore: delete stale code
2024-02-16 10:32:13 +01:00
b24321a8fc
feat: upgrade to hocon-0.41.0
...
hocon 0.41.0 added support for multiline string indentation.
now there is no need to escape (unless there is backslash)
quotes and line feeds etc.
2024-02-16 10:32:13 +01:00
ab2a469aff
Merge pull request #12489 from ieQu1/dev/ds-retainer
...
Durable sessions support retained messages
2024-02-15 21:59:32 +01:00
4484578c1f
Merge pull request #12507 from keynslug/fix/gw-app-no-scan
...
fix(gateway): avoid scanning modules of known gateway apps
2024-02-15 17:11:49 +01:00
5f85105801
feat(sessds): Specialize the interval queue for positive numbers
2024-02-15 16:55:05 +01:00
3adbe65a58
refactor(sessds): Unify logic for QoS 1 and 2
2024-02-15 11:34:13 +01:00
c781240459
feat(sessds): Add support for the retainer
...
Note: this is currently not ideal. Retained messages won't be
redelivered.
2024-02-15 11:34:13 +01:00
5e442fcbca
fix: labels and descriptions
2024-02-15 11:08:08 +01:00
6514659733
test(mgmt): update and simplify testsuites setup
2024-02-15 11:04:58 +01:00
94254ec05b
feat(sessds): Correct handling of gaps in the seqno series
2024-02-15 10:49:02 +01:00
0d0e26d6af
fix: set `info` level for potentially flooding log events
2024-02-15 11:02:03 +02:00
cffb52ab28
test(plugins): update and simplify test suite
2024-02-14 19:00:04 +01:00
4e8dfb48b7
fix: elvis problems
2024-02-14 17:54:14 +01:00
a153d758c3
feat: refactor HStreamDB bridge to connector and action
...
This commit also upgrades the hstreamdb_erl driver library and change
the action/bridge to use the new hstreamdb_erl.
Much of the code for the new API is copied from:
be1a1604dd/lib-ee/emqx_rule_actions/src/emqx_backend_hstreamdb_actions.erlhttps://emqx.atlassian.net/browse/EMQX-11458
2024-02-14 17:34:35 +01:00
7272ef25d4
feat(emqx_auth): implement API to re-order all authenticators/authz sources
...
Fixes: EMQX-11770
2024-02-14 14:35:46 +02:00
35bf805504
test(exhook): update and simplify testsuites setup
2024-02-13 21:02:39 +01:00
ece1c6c6dc
test(prometheus): update and simplify testsuites setup
2024-02-13 20:36:23 +01:00
a64850a84b
test(sso): update and simplify testsuites setup
2024-02-13 20:36:23 +01:00
b15e81baf2
test(slowsub): update and simplify testsuite setup
2024-02-13 20:36:23 +01:00
d0e507eba4
test(trie): fix testsuite setup
2024-02-13 20:36:22 +01:00
841fdea124
test(conf): update and simplify some testcases
2024-02-13 20:36:22 +01:00
0a9cbe3080
test(license): update and simplify tests
2024-02-13 20:36:22 +01:00
24dfa41722
feat(cth-suite): use cheaper heuristic for schema modules
2024-02-13 20:36:22 +01:00
8f2a4f7b19
fix(cth-suite): use cheaper check for loaded applications
2024-02-13 20:36:22 +01:00
82a4e6ef68
feat(psk): manage mria tables explicitly during startup
...
Also switch test suite to `emqx_cth_suite` tooling.
2024-02-13 20:36:22 +01:00
b07df487f0
feat(modules): manage mria tables explicitly during startup
2024-02-13 20:36:22 +01:00
b6d77c164e
feat(emqx-mgmt): manage mria tables explicitly during startup
2024-02-13 20:36:22 +01:00
Kjell Winblad