bfca3ebc71
feat(variform): support array syntax '[' and ']'
2024-03-28 19:34:57 +01:00
5f26e4ed5e
feat(variform): implement variform engine
2024-03-28 19:34:57 +01:00
2e528d1dd8
Merge pull request #12802 from SergeTupchiy/EMQX-11826-prevent-left-node-from-rejoining-5.6.1
...
prevent a left node from rejoining the same cluster
2024-03-28 19:49:18 +02:00
ad95473aae
refactor: move string functions to emqx_variform
2024-03-28 18:03:37 +01:00
3eda182e9a
fix: prevent a node from discovering and re-joining the same cluster after it has (manually) left it.
2024-03-28 18:09:27 +02:00
9bf65a415b
feat(variform): add a variable transformer
2024-03-28 16:11:26 +01:00
35c43eb8a0
feat(sessds): handle recoverable errors in stream scheduler
2024-03-28 15:17:01 +01:00
fa66a640c3
fix(dsrepl): handle RPC errors gracefully when storage is down
2024-03-28 15:17:01 +01:00
63c017a72f
Merge pull request #12804 from SergeTupchiy/EMQX-12058-improve-force-shutdown-error-reason-5.6.1
...
chore: rename `message_queue_too_long` error reason to `mailbox_overflow` (5.6.1 port)
2024-03-28 16:14:51 +02:00
8fb4ef9fe3
test: fix flaky test
2024-03-28 10:53:44 -03:00
93c87fcb25
Merge pull request #12803 from SergeTupchiy/EMQX-11808-remove-uploaded-invalid-backups-5.6.1
...
fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid (5.6.1 port)
2024-03-28 15:37:10 +02:00
04bf763890
fix(kafka-based bridges): avoid trying to get raw config for replayq dir
...
Fixes https://emqx.atlassian.net/browse/EMQX-12049
2024-03-28 09:13:34 -03:00
b1855f95c1
fix(bpapi): Add exceptions for experimental features
2024-03-28 12:07:45 +01:00
8c6d8bdd12
fix(bpapi): Add exceptions for experimental features
2024-03-28 12:03:41 +01:00
906a77d167
chore: rename `message_queue_too_long` error reason to `mailbox_overflow`
...
`mailbox_overflow` is consistent with the corresponding config parameter:
'force_shutdown.max_mailbox_size'
2024-03-28 12:56:00 +02:00
1d1f595e6f
fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid
2024-03-28 12:54:51 +02:00
02b72d79fc
fix(bpapi): Fix comparison of releases during compatibility check
2024-03-28 11:21:30 +01:00
db9efb9317
chore: bump apps versions
2024-03-28 10:19:09 +01:00
f4446ec680
Merge remote-tracking branch 'upstream/release-56' into 0328-sync-release-56
2024-03-28 09:59:54 +01:00
ea917d6b2b
Merge pull request #12795 from id/0327-cut-5.6.0
...
chore: 5.6.0
2024-03-28 09:30:08 +01:00
11bc1aeea1
fix: update hstreamdb driver
2024-03-28 10:30:48 +03:00
cf29f97440
revert: "test: comment out hstreamdb:stop_client"
...
This reverts commits:
3536b32ad2b126c27a1b0bd91e615069230931d7..
d63f10928d
2024-03-28 14:32:18 +08:00
34b5cd7bfd
chore: reuse hstream_erl `0.5.18+v0.18.1`
...
- change ezstd dep vsn for bulid
2024-03-28 14:31:10 +08:00
d63f10928d
test: comment out hstreamdb:stop_client
2024-03-28 13:25:01 +08:00
3536b32ad2
ci: print the failed stacktrace
2024-03-28 11:17:26 +08:00
d7917d05c6
chore: upgrade hstreamdb to 0.5.22+v0.18.1
2024-03-28 10:11:53 +08:00
a279a399db
ci: Override ezstd dependency
2024-03-27 20:54:46 +01:00
4b3c29ded8
fix(bpapi): Add 5.5 dump
2024-03-27 17:39:38 +01:00
beb9152d50
Merge pull request #12750 from zmstone/0316-introduce-client-attrs
...
0316 introduce client_attrs
2024-03-27 16:19:47 +01:00
5ed20e15cd
fix(bpapi): Add 5.5 bpapi file
2024-03-27 13:45:00 +01:00
df0c67fe50
Merge pull request #12787 from thalesmg/mv-api-examples-m-20240326
...
docs(message validation): add swagger API examples
2024-03-27 09:06:58 -03:00
33d9619271
chore: 5.6.0
2024-03-27 11:17:13 +01:00
a3b730ff10
fix(emqx_session_mem): keep msg inflight insert timestamp in Inflight after retry
2024-03-27 11:31:51 +02:00
4f9daa0d4a
chore: bump version to 5.6.0-rc.6
2024-03-27 14:04:27 +08:00
d82b6d216d
chore: remove the duplicated time field
...
Fixes https://emqx.atlassian.net/browse/EMQX-12104
2024-03-27 12:09:21 +08:00
f127bf53b3
chore: 5.6.0-rc.5
2024-03-26 17:52:57 +01:00
291cf689e2
Merge pull request #12786 from ieQu1/dev/check-core-emqx-vsn
...
fix(mria): Strictly check compatibility of the replicant with core
2024-03-26 17:49:37 +01:00
ba43da1b8e
fix(mria): Strictly check compatibility of the replicant with core
2024-03-26 16:32:01 +01:00
da4a04c411
Merge pull request #12777 from thalesmg/test-flaky-conf-copy-m-20240325
...
test: attempt to fix flaky test
2024-03-26 11:52:12 -03:00
78c5f8177c
test: fix test case
2024-03-26 14:51:21 +01:00
10a43db742
docs(message validation): add swagger API examples
...
Fixes https://emqx.atlassian.net/browse/EMQX-12102
2024-03-26 09:39:05 -03:00
067f018337
test: fix test cases
2024-03-26 13:36:12 +01:00
b1c9bb63d7
chore: delete dead code
...
hocon schema validator never validates 'undefined'
2024-03-26 13:20:48 +01:00
c42e980442
feat(log): configurable time format
...
now logs can be configured to use 'epoch' or 'rfc3339' time format
2024-03-26 13:09:32 +01:00
cc37030265
Merge pull request #12770 from thalesmg/mv-metrics-m-20240321
...
feat(message validation): implement metrics
2024-03-26 08:59:05 -03:00
56a40d61ae
fix: bpapi undef abort on old node
2024-03-26 13:06:56 +08:00
2ae973821f
chore: 5.6.0-rc.4
2024-03-25 20:00:10 +01:00
b06d3932af
Merge pull request #12768 from keynslug/fix/EMQX-12068/routing-schema-conflict
...
fix(routing): add heuristic for routing schema in empty cluster
2024-03-25 19:50:51 +01:00
6259f95aab
test: attempt to fix flaky test
...
```
expected: {ok,<<...>>}
got: {error,enoent}
line: 187
comment: #{node => 'emqx_conf_app_SUITE9@127.0.0.1'}
```
2024-03-25 14:36:22 -03:00
0f426e6e77
feat(ds): make durable sessions handle will messages
...
Fixes https://emqx.atlassian.net/browse/EMQX-10431
2024-03-25 14:18:25 -03:00
8c1a1d21a7
feat(session): pass will message down to session when creating/opening
2024-03-25 14:18:23 -03:00
64eeeb7494
chore: verify status code
2024-03-25 10:05:39 -07:00
8b963d5960
Merge pull request #12739 from ieQu1/dev/zone-durability
...
feat(sessds): Add zone overrides for session durability settings
2024-03-25 17:32:53 +01:00
b8b0b809b4
chore: add a warning log if conflict is detected
2024-03-25 17:07:56 +01:00
9411d6078e
docs: remove unnecessary restart instruction
2024-03-25 16:50:51 +01:00
d38545010a
fix: case_clause
2024-03-25 16:43:24 +01:00
c93145cb97
fix: function clause
2024-03-25 16:40:53 +01:00
3aff9eb2a4
fix(route schema): allow boot if all peer nodes agree on one version
2024-03-25 16:33:27 +01:00
fe3cc25855
chore: update instructions for routing schema conflict
2024-03-25 15:22:43 +01:00
59ff43ec87
feat(message validation): implement metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12071
2024-03-25 10:55:01 -03:00
94c6e5bb38
Merge pull request #12727 from kjellwinblad/kjell/emqx_connector_info/EMQX-11427
...
refactor: add emqx_connector_info behavior
2024-03-25 14:31:40 +01:00
849fe0c2c8
feat(routing): add schema conflict resolution procedure
...
In the log message printed when a schema conflict in cluster routing
is detected.
2024-03-25 13:43:46 +01:00
f393d75a1e
Merge pull request #12771 from thalesmg/test-another-flaky-monitor-m-20240322
...
test: attempt to fix flaky test
2024-03-25 09:19:20 -03:00
e39fa5eaa9
docs(es_connector_info): add copyright header
2024-03-25 10:44:33 +01:00
3e97e5528e
Merge pull request #12765 from JimMoen/EMQX-12043/subscribers-count
...
fix(stats): `'subscribers.count'` contains shared-subscriber
2024-03-25 17:19:13 +08:00
5f7f9e43f9
fix(stats): `'subscribers.count'` contains shared-subscriber
2024-03-25 15:12:57 +08:00
2926518b3d
Merge pull request #12764 from zhongwencool/fix-bpapi-undef
...
fix: bpapi undef on old node
2024-03-25 13:37:37 +08:00
df913f2450
docs: refine changelog
2024-03-23 14:11:30 +01:00
22838f027a
fix: mountpoint template render should not replace unknown as undefined
...
For backward compatibility, the unknown vars used in mountpoint
is kept unchanged.
e.g. '${unknown}/foo/bar' should be rendered as '${unknown}/foo/bar'
but not 'undefined/foo/bar'
2024-03-23 10:16:05 +01:00
3136ec5958
feat: allow mountpoint to use client_attrs
2024-03-23 10:16:05 +01:00
8e8fc6a3d1
test: fix config test
2024-03-23 10:16:05 +01:00
5e9814d171
fix: add debug level logging for invalid client attributes
2024-03-23 10:16:05 +01:00
0cf61932b6
feat: allow using client_attrs in authentication templates
2024-03-23 10:16:05 +01:00
8254b801ae
feat: support initialize client attribute from user property
2024-03-23 10:16:02 +01:00
2fd0a2cd4d
feat: support extracting initial client attrs from clientinfo
2024-03-23 10:16:02 +01:00
c75840306b
fix: restrict client_attr key and value string format
...
The keys and values are used to render templates for
authz rules, such as topic names, and SQL statements etc.
2024-03-23 10:16:02 +01:00
9ec99fef4a
feat: allow client_attr used in authz rules
2024-03-23 10:16:02 +01:00
e5816f5a13
refactor: rename attr to client_attr
...
client_attr is unique enough for all contexts
so the name can be unified from external responses
to internal template rendering, and rule-engine template rendering
2024-03-23 10:16:02 +01:00
cc4805b1ac
feat: extract attrs field from http and jwt auth response
2024-03-23 10:16:02 +01:00
f9d51484cc
feat: introuce attrs field in clientinfo
...
the 'attrs' field is to be used as extra attributes extracted
from various client properties
for example, HTTP auth result can return an extra client_attributes
field in the result which will get merged into the original
attrs field in clientinfo
2024-03-23 10:16:02 +01:00
ef0fee52f4
fix: query_by_clientid with v0.7 sql api
2024-03-22 17:09:23 -07:00
1fcd06f4bc
feat: update greptimedb client lib and ci version
2024-03-22 15:33:15 -07:00
303d0ed81b
test: attempt to fix flaky test
2024-03-22 16:29:00 -03:00
70378052dd
Merge pull request #12769 from SergeTupchiy/EMQX-12058-handle-client-shutdown-errors
...
fix:(emqx_mgmt): handle client shutdown errors in call_client
2024-03-22 17:43:57 +02:00
cbc84900b2
chore: update bpapi versions dump
2024-03-22 16:24:52 +01:00
9986de49d1
chore: make dyalizer happy
2024-03-22 16:16:49 +01:00
be1886fb91
chore: make dyalizer happy and update changelog
2024-03-22 15:56:44 +01:00
ed0b5ac939
fix(emqx_mgmt): handle client shutdown errors in call_client
...
The client process may be shutdown normally during/shortly before the call.
2024-03-22 16:47:02 +02:00
b81a11b790
chore(routing): mark function that is now an RPC target
2024-03-22 15:14:53 +01:00
ba276d807f
fix(routing): add heuristic for routing schema in empty cluster
...
When the cluster is empty but comprised of different versions of EMQX,
try to additionally ask the cluster nodes for the routing schema in use,
to be able to make more informed decisions.
2024-03-22 15:14:32 +01:00
bc5775a988
fix(kafka_producer): handle ancient v1 config when migrating to actions
...
Fixes https://emqx.atlassian.net/browse/EMQX-12064
2024-03-22 09:45:27 -03:00
d2a1a7f7cf
chore: rename `message_queue_too_long` error reason to `mailbox_overflow`
...
`mailbox_overflow` is consistent with the corresponding config parameter:
'force_shutdown.max_mailbox_size'
2024-03-22 12:20:20 +02:00
42faffc320
refactor: uniform shared_sub table macros
2024-03-22 18:03:13 +08:00
45eedbb9c6
fix: bpapi undef on old node
2024-03-22 17:02:09 +08:00
2b684c7bcc
fix(emqx_connector_info): refactoring based on thalesmg's comments
2024-03-22 09:29:26 +01:00
5d77e1b01e
fix: return not_implemented error for mqueue/inflight msgs queries
2024-03-22 10:02:31 +02:00
9b085b5797
fix(api_schema): removed metrics schema in api spec
...
- Followup [PR#6622](https://github.com/emqx/emqx/pull/6622 ).
2024-03-22 10:57:14 +08:00
23ad37f566
Merge pull request #12762 from thalesmg/ds-fix-sync-egress-reply-m-20240321
...
fix(ds): add caller to pending replies before flushing and reply failures to sync callers
2024-03-21 16:42:42 -03:00
cfd4a1297f
Merge pull request #12761 from thalesmg/test-fix-flaky-dash-mon-m-20240321
...
test: fix flaky test
2024-03-21 16:04:08 -03:00
e17f663fa5
Merge pull request #12749 from thalesmg/mv-followup1-m-20240320
...
Follow up features and fixes for message validation - part 1
2024-03-21 16:03:49 -03:00
796c04e7a8
test: fix flaky test
...
We should emit the trace event before replying to callers.
Example failure:
https://github.com/emqx/emqx/actions/runs/8378977952/job/22946318696#step:6:182
```
=CRITICAL REPORT==== 21-Mar-2024::17:45:37.676024 ===
"check stage" failed: error
{assertMatch,[{module,emqx_ds_storage_bitfield_lts_SUITE},
{line,270},
{expression,"? of_kind ( emqx_ds_replication_layer_egress_flush , Trace )"},
{pattern,"[ # { batch := [ _ , _ , _ ] } ]"},
{value,[]}]}
Stacktrace: [{emqx_ds_storage_bitfield_lts_SUITE,
'-t_atomic_store_batch/1-fun-1-',1,
[{file,
"/__w/emqx/emqx/apps/emqx_durable_storage/test/emqx_ds_storage_bitfield_lts_SUITE.erl"},
{line,270}]},
{emqx_ds_storage_bitfield_lts_SUITE,t_atomic_store_batch,1,
[{file,
"/__w/emqx/emqx/apps/emqx_durable_storage/test/emqx_ds_storage_bitfield_lts_SUITE.erl"},
{line,249}]}]
```
2024-03-21 15:47:29 -03:00
68af211130
fix(ds): reply sync callers after raft store failure
2024-03-21 15:40:21 -03:00
70737a437a
fix(ds): add caller to pending replies before flushing
2024-03-21 14:39:21 -03:00
93c80bc912
chore: 5.6.0-rc.3
2024-03-21 18:10:59 +01:00
fa7ec231e3
test: fix flaky test
2024-03-21 14:07:09 -03:00
e2a2295c99
Merge pull request #12759 from SergeTupchiy/EMQX-11808-remove-uploaded-invalid-backups
...
fix(emqx_mgmt_data_backup): remove invalid uploaded backup files
2024-03-21 18:58:48 +02:00
00aa7b5621
feat: create new `message.validation_failed` hookpoint and rule engine event
2024-03-21 13:46:27 -03:00
7d7c6685d4
Merge pull request #12753 from thalesmg/test-flaky-retry-m-20240321
...
test: attempt to stabilize flaky test
2024-03-21 13:43:51 -03:00
40eccb10d6
fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid
2024-03-21 17:36:39 +02:00
4bfc51c052
refactor(emqx_connector_info): improve documentation and type specs
2024-03-21 16:08:31 +01:00
7d6fde5960
fix(topics-api): respond with correct totals to paged queries
2024-03-21 16:02:16 +01:00
e837791f94
fix(kafka-based bridges): avoid trying to get raw config for replayq dir
...
Fixes https://emqx.atlassian.net/browse/EMQX-12049
2024-03-21 11:49:00 -03:00
64399b6861
test: attempt to stabilize flaky test
2024-03-21 11:04:14 -03:00
89befa580e
refactor(emqx_connector_info): remove code that is no longer needed
2024-03-21 11:58:54 +01:00
cada944350
feat(sessds): Expose relevant durable session info in the REST API
2024-03-21 10:37:04 +01:00
a3b75eb39d
refactor(remaining connectors): to use emqx_connector_info
...
This commit refactors the sqlserver, iotdb, elasticsearch, opents,
greptimedb, pulsar, tdengine, rabbitmq and s3 connectors to use the
`emqx_connector_info` behavior.
2024-03-21 09:47:57 +01:00
62030e8942
feat(message validation): forbid repeated schema checks
...
Fixes https://emqx.atlassian.net/browse/EMQX-12054
2024-03-20 15:26:16 -03:00
b8cd1c9020
feat(message validation api): add enable/disable HTTP API
2024-03-20 14:59:32 -03:00
4944cc080e
feat(message_validation): add `ignore` failure action
2024-03-20 14:31:43 -03:00
8753e3583f
feat(message_validation): add `none` log level
2024-03-20 14:31:26 -03:00
e767f01e0a
fix(message_validation): take `enable` into account
2024-03-20 13:57:12 -03:00
7aa287c6c1
fix: add message validation schema to `emqx_enterprise_schema`
2024-03-20 13:37:01 -03:00
49e0488498
chore: 5.6.0-rc.2
2024-03-20 17:11:37 +01:00
bede5a5b85
Merge pull request #12746 from zmstone/0320-add-throttle-log-for-auth-failure
...
feat: add authentication_failure throttled log
2024-03-20 17:06:36 +01:00
27c1f1a4e4
Merge pull request #12732 from SergeTupchiy/EMQX-11861-client-mqueue-inflight-API-updates
...
EMQX-11861 client mqueue inflight api updates
2024-03-20 16:27:44 +02:00
7257fe526b
fix(ci): add `ra` to emqx app dependencies as well
2024-03-20 14:46:53 +01:00
90fce21adc
docs: add change log for #12746
...
also updated #12520 to include newly added throttled log id
2024-03-20 14:43:28 +01:00
38e30622f3
chore(license): add LICENSE tag to logs
2024-03-20 14:42:01 +01:00
2c81b56377
feat: add authentication_failure throttled log
...
also included changes:
1. add 'username' to log metadata
2. add 'tag' to authz warning logs
2024-03-20 14:42:01 +01:00
e5f9e7631d
Merge pull request #12736 from zmstone/0319-json-schema-draft-06-as-default-version
...
chore: upgrade to jesse 1.8.0 for draft-06 as default version
2024-03-20 14:39:37 +01:00
27d109ce6d
refactor(syskeeper connector): to use emqx_connector_info
...
This commit refactors the syskeeper connector to use the
`emqx_connector_info` behavior.
2024-03-20 14:29:02 +01:00
cb5fdb3c79
fix: rework In-flight / Mqueue API
...
- use timestamp 'position' to find the next chunk of data
- add 'start' position to response meta
- sort In-flight messages by insertion time
- sort Mqueue messages by priority
2024-03-20 15:25:33 +02:00
188ab71673
refactor(rocketmq connector): to use emqx_connector_info
...
This commit refactors the rocketmq connector to use the
`emqx_connector_info` behavior.
2024-03-20 14:02:18 +01:00
aa368bb9d0
refactor(redis connector): to use emqx_connector_info
...
This commit refactors the redis connector to use the
`emqx_connector_info` behavior.
2024-03-20 13:49:28 +01:00
12717baa0b
refactor(mqtt connector): to use emqx_connector_info
...
This commit refactors the mqtt connector to use the
`emqx_connector_info` behavior.
2024-03-20 13:33:13 +01:00
e55e1dd1b2
chore: whitelist `ra` to make RPCs w/o BPAPIs
2024-03-20 13:20:25 +01:00
a8baff61ec
docs(dsrepl): describe briefly what `n_sites` is for
2024-03-20 13:20:25 +01:00
efac5c6197
test(ds): stabilize `t_message_gc` testcase
2024-03-20 13:20:25 +01:00
fe50a1711b
fix(ds-egress): drop pending batch on failures
...
Before this commit, messages in the current batch will be retried as
part of next batch. This could have led to message duplication which is
probably not what the user wants by default.
2024-03-20 13:20:25 +01:00
a1f5de3f5b
fix(dsrepl): turn memoize into a safer function
2024-03-20 13:20:24 +01:00
d39ca41070
chore(dsrepl): mark per-node `add_generation` RPC target obsolete
...
Also annotate internal exports with comments according with their
intended use.
2024-03-20 13:20:24 +01:00
35b18f9125
fix(dsrepl): properly handle error conditions in generation mgmt
...
Also update few outdated typespecs. Also make error reasons easier
to comprehend.
2024-03-20 13:20:24 +01:00
f2268aa69a
fix(dsrepl): use correct base dir for ra system stuff
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-03-20 13:20:24 +01:00
404e919494
refactor(dsrepl): make shard allocator more robust and consistent
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-03-20 13:20:24 +01:00
0e18bd6e80
fix(dsrepl): increase replication site id bitsize back
...
In order to minimize chances of site id collision to practically zero.
2024-03-20 13:20:24 +01:00
ac9700dd28
fix(dsrepl): split shard allocator into a separate module
2024-03-20 13:20:23 +01:00
1b647035d0
chore(dsrepl): make dialyzer a bit happier
2024-03-20 13:20:23 +01:00
611b3f0e07
feat(dsrepl): use more straightforward way to drop ra shards
2024-03-20 13:20:23 +01:00
74881e8706
feat(dsrepl): make storage layer unaware of granularity of time
...
Storage also becomes a bit more pure, depending on the upper layer to
provide the timestamps, which also makes it possible to handle more
operations idempotently.
2024-03-20 13:20:23 +01:00
783f769a98
fix(mgmt): Fix bulk kickout of durable sessions
2024-03-20 12:52:44 +01:00
zmstone