71f5eaf11e
fix(data_backup): import `cluster.links` conf
2024-06-19 20:47:06 +03:00
ef5cf4fac3
chore(clusterlink): fix comment
2024-06-19 20:46:00 +03:00
2b0146663a
feat(queue): add smoke test for ds shared subs
2024-06-19 19:32:23 +03:00
e3c4816035
feat(queue): move group subscription state machine to its own module
2024-06-19 19:22:10 +03:00
9559ba2f06
test: remove unnecessary lines
2024-06-19 13:22:02 -03:00
021b6b3902
Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master
2024-06-19 15:06:41 +02:00
02a6ee1ef4
Merge pull request #13279 from zmstone/0617-json-decode-payload-for-map_to_redis_hset_args
...
fix(redis): json decode if arg for map_to_redis_hset_args is string
2024-06-19 14:55:21 +02:00
213e4785e7
Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master
2024-06-19 14:54:48 +02:00
ff2e6e1434
test(greptime): revert driver patch to test ci again
2024-06-19 09:12:30 -03:00
95d387a790
Merge pull request #13294 from zmstone/0618-per-release-edition-code-injection
...
0618 per release edition code injection
2024-06-19 11:53:57 +02:00
f0e5721959
refactor: make dialyzer happy
2024-06-19 10:42:08 +02:00
0d098a01ef
Merge pull request #13293 from SergeTupchiy/EMQX-12345-reindex-retained-msgs-on-import
...
reindex retained msgs on import
2024-06-19 11:23:08 +03:00
ffec1c7fe0
fix(greptime): attempt patch for cleaning up channels when worker terminates
2024-06-18 22:16:03 -03:00
9dd419d822
test(greptime): attempt to fix flaky tests
...
```
=WARNING REPORT==== 18-Jun-2024::12:42:04.739109 ===
reason: no_endpoints
msg: failed_to_start_greptimedb_connector
client: #{protocol => http,
pool => <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>,
cli_opts =>
[{endpoints,[{http,"toxiproxy",4001}]},
{pool_size,2},
{pool_type,random},
{auto_reconnect,1},
{gprc_options,#{sync_start => true,
connect_timeout => 5000}},
{dbname,"public"},
{auth,{basic,#{password => <<"******">>,
username => "greptime_user"}}},
{https_enabled,false}]}
connector: <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>
```
2024-06-18 18:23:22 -03:00
54fc605cc5
refactor: move schema injection from hrl to erl
2024-06-18 22:22:53 +02:00
a53fa977df
refactor: delete an unused macro
2024-06-18 22:22:48 +02:00
e69ba33420
Merge pull request #13291 from thalesmg/fix-ds-mgmt-node-down-r57-20240618
...
fix(dsreplmeta): check site status when fetching shard info
2024-06-18 15:14:08 -03:00
bca743054b
feat(queue): implement backbones of queue agent, leader and leader registry
2024-06-18 21:03:51 +03:00
4c54ab6379
test: remove tests per review request
2024-06-18 13:08:19 -03:00
4a318f8f51
style: fix type problem found by dialyzer
2024-06-18 17:02:26 +02:00
baa7996289
fix(data_backup): allow exporting `ram_copies` Mnesia tables
...
Currently, ram tables can be used for message retainer.
2024-06-18 17:31:33 +03:00
60882a616e
fix(data_backup): re-index retained messages after importing
2024-06-18 17:31:19 +03:00
6bde6aa711
Merge pull request #13280 from thalesmg/fix-plugin-elixir-r57-20240617
...
fix(plugins): do not stop Elixir protected apps
2024-06-18 10:41:12 -03:00
8ce16fd7d9
fix(dsreplmeta): check site status when fetching shard info
...
Fixes https://emqx.atlassian.net/browse/EMQX-12356
2024-06-18 10:40:09 -03:00
a4f855108c
fix: do not crash when showing rule unsing the cmd interface
...
Before the change the command line command
$ bin/emqx ctl rules show rule_0hyd
would crash if the rule had bridge actions. This has now been fixed by
adding a handler for printing bridge actions.
Fixes:
https://emqx.atlassian.net/browse/EMQX-12548
2024-06-18 15:17:25 +02:00
3f7723b2dc
Merge pull request #13274 from kjellwinblad/kjell/add_ssl_to_rocksdb/EMQX-12289
...
feat: add SSL support to RocketMQ connector
2024-06-18 09:38:28 +02:00
20cffb54d4
Merge branch 'release-57' into sync-r57-m-20240617
2024-06-17 17:10:12 -03:00
6897f0141b
fix(plugins): do not stop Elixir protected apps
...
On one hand, Elixir plugins might include Elixir itself, when targetting a non-Elixir EMQX
release. If, on the other hand, the EMQX release already includes Elixir, we shouldn't
stop Elixir nor IEx, or the running IEx shell will break.
2024-06-17 16:42:23 -03:00
ad993437aa
Merge pull request #13278 from savonarola/0617-release-57-sync
...
Sync release-57
2024-06-17 21:25:00 +03:00
9479c8d33b
fix(redis): json decode if arg for map_to_redis_hset_args is string
2024-06-17 20:19:53 +02:00
eede9f349e
Merge pull request #13276 from keynslug/fix/ds/inherited-lts-lost
...
fix(dsstore): persist inherited LTS trie
2024-06-17 20:25:31 +03:00
ae22a64157
Merge pull request #13277 from zmstone/0617-handle-kafka-message_too_large-return
...
0617 handle kafka message too large return
2024-06-17 19:15:33 +02:00
0cbbae9655
fix: make fixes due to comments from @thalesmg
2024-06-17 19:06:12 +02:00
c39c544c96
feat: add SSL support to RocketMQ connector
2024-06-17 19:06:12 +02:00
b64f0c0ca7
Merge pull request #13231 from thalesmg/mt-test-endpoint-r57-20240611
...
feat(message transformation): implement dryrun endpoint
2024-06-17 13:11:39 -03:00
240bb09070
Merge pull request #13275 from thalesmg/test-flaky-sk-r57-20240617
...
test: attempt to fix flaky test
2024-06-17 13:01:08 -03:00
f8e6aab86f
Merge branch 'release-57' into 0617-release-57-sync
...
* release-57:
chore(auth,http): cache REs for parsing URIs
fix(auth,http): improve URI handling
chore: revert ULOG/ELOG
test: generate dispatch.eterm in dashboard test
docs: refine change log
feat: make the dashboard restart quicker
chore: fix typo
fix(http authz): handle unknown content types in responses
chore: change types of mysql and mongodb fields to `template()`
fix(client mgmt api): allow projecting `client_attrs` from client fields
fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-17 18:53:39 +03:00
82588fbc35
chore(dsstore): refine module and callback typespecs
2024-06-17 17:53:39 +02:00
05d97397d3
fix(dsstore): use correct comparison when walking over generations
2024-06-17 17:53:39 +02:00
22009bcc58
test(dsstore): verify that inherited TLS trie is persisted
2024-06-17 17:53:38 +02:00
68f6556856
fix(dsstore): persist inherited LTS trie
...
Before this commit, inherited trie was actually only kept in memory
cache.
Also simplify storage backend behaviour around inheriting previous
generation's legacy.
2024-06-17 17:53:38 +02:00
830266b4d5
Merge pull request #13126 from SergeTupchiy/EMQX-11967-cross-cluster-route-replication-syncer-actor
...
cluster linking
2024-06-17 18:52:46 +03:00
3ac4ddcbe3
fix(kafka): handle message_too_large
...
bump 'failed' counter
2024-06-17 17:27:25 +02:00
00dfdc22cf
chore: pin Kafka client wolff 0.10.5
2024-06-17 17:27:25 +02:00
5ab4b321a0
Merge pull request #13267 from zhongwencool/use-redbug-for-trace
...
chore(debug): use redbug for default trace tools
2024-06-17 22:10:18 +08:00
fb492e3dc5
Merge pull request #13264 from thalesmg/fix-postgres-disabled-prepared-r57-20240614
...
fix(postgres): authn/authz/batch requests when prepared statements are disabled
2024-06-17 11:05:05 -03:00
5665a4917c
Merge pull request #13266 from thalesmg/fix-s3-release-app-r57-20240614
...
fix: add `emqx_s3` to `reboot_lists.eterm`
2024-06-17 10:48:12 -03:00
ec4f462684
refactor: apply review requests
2024-06-17 10:39:32 -03:00
1464094967
test: attempt to fix flaky test
...
```
%%% emqx_persistent_session_ds_SUITE ==> t_session_discard_persistent_to_persistent: FAILED
%%% emqx_persistent_session_ds_SUITE ==> {noproc,{gen_server,call,[snabbkaffe_collector,flush_trace,infinity]}}
```
2024-06-17 10:34:57 -03:00
b71212b022
refactor: apply review comments
2024-06-17 09:40:33 -03:00
f5eb3e7471
Merge pull request #13250 from thalesmg/ds-singleton-m-20240613
...
feat: enforce singleton discovery strategy when using DS `builtin_local` backend
2024-06-17 09:34:05 -03:00
19c9f0d76f
chore(auth,http): cache REs for parsing URIs
2024-06-17 14:42:29 +03:00
f7ac829f28
fix(auth,http): improve URI handling
2024-06-17 14:42:29 +03:00
a905a6048c
chore(clusterlink): rename link `upstream` field to `name`
2024-06-17 12:24:51 +03:00
a95a08efd3
test(clusterlink): add more test cases
2024-06-17 12:24:51 +03:00
44c37571cc
fix(clusterlink): ignore not_registered error
2024-06-17 12:24:51 +03:00
00f912928f
fix: fix emqx_external_broker:forward/1 spec
2024-06-17 12:24:51 +03:00
ed16ff07df
refactor(broker): simplify external broker behaviour
2024-06-17 12:24:51 +03:00
d282c61120
feat(clusterlink): update only necessary resources when a link config is changed
2024-06-17 12:24:50 +03:00
ff16521d4f
fix(clusterlink): add schema descriptions, fix dialyzer warnings, add resource_opts
2024-06-17 12:23:52 +03:00
94e81ba812
feat(clusterlink): implement actor config handler
2024-06-17 12:23:52 +03:00
c871b37453
fix(clusterlink): add link topics schema validator
2024-06-17 12:23:52 +03:00
780a0bf807
fix(cluster-link): clear exit signal of failed-to-connect client
2024-06-17 12:23:52 +03:00
d0df4de2a3
test(cluster-link): add e2e replication actor GC testcase
2024-06-17 12:23:52 +03:00
c4840b30d2
fix(cluster-link): deduplicate routes down to dest cluster
2024-06-17 12:23:52 +03:00
ede35df24a
fix(cluster-link): cancel heartbeats on client down
2024-06-17 12:23:52 +03:00
e0604e3af6
fix(cluster-link): anticipate clients may occasionally retry
2024-06-17 12:23:52 +03:00
0219b8bd4d
feat(cluster-link): add simple replication actor GC process
2024-06-17 12:23:52 +03:00
7fccb5dbc9
test(topic): add more `intersection/2` testcases
2024-06-17 12:23:52 +03:00
7b8f466adf
feat(topic): avoid `lists:reverse` when intersecting
2024-06-17 12:23:52 +03:00
24be189728
fix(topic): respect special topic rules when intersecting
2024-06-17 12:23:51 +03:00
de1ac131f7
test(cluster-link): fix test suite setup
2024-06-17 12:23:51 +03:00
58eaf07627
fix(clusterlink): valide config to disallow duplicated cluster links
2024-06-17 12:23:51 +03:00
54d51d0982
test(cluster-link): draft basic integration test suite
2024-06-17 12:23:51 +03:00
e9c24090d4
fix(cluster-link): avoid starting ps syncer if persistence disabled
2024-06-17 12:23:51 +03:00
036c7e8492
test(cluster-link): adapt extrouter testsuite to new APIs
2024-06-17 12:23:51 +03:00
45eda4f3b9
fix(cluster-link): adapt heartbeat / reincarnation handling to new API
2024-06-17 12:23:51 +03:00
43d114546c
feat(cluster-link): preserve replication actor state in pdict
2024-06-17 12:23:51 +03:00
5771a41a32
fix(cluster-link): ensure replication actor bootstraps do heartbeats
2024-06-17 12:23:51 +03:00
d4b449c6e1
feat(cluster-link): implement replication actor heartbeats
2024-06-17 12:23:51 +03:00
faa4420e1f
fix(clusterlink): improve actor error handling
...
Add status and error reason to the actor state, report alarms.
2024-06-17 12:23:51 +03:00
21711c6e0d
fix(clusterlink): communicate bootstrap requirements via actor handshake
...
`session_present` flag is not reliable to decide whether bootstrap is needed if durable sessions is enabled.
In this case, the client session may survive cluster restart while all the external routes are lost, as they are not persistent.
2024-06-17 12:23:51 +03:00
d5e82cdfac
refactor(clusterlink): avoid unnecessary `maybe_` external_broker CB names
2024-06-17 12:23:51 +03:00
d578ac3f9e
fix(clusterlink): match correct timer ref in router actor
2024-06-17 12:23:51 +03:00
b1aeb35370
feat(clusterlink): implement actor init handshake
2024-06-17 12:23:51 +03:00
ac19cf89df
chore(clusterlink): remove code related to the rejected coordinator-based implementation
2024-06-17 12:23:51 +03:00
e26e7acaa1
refactor(clusterlink): use `emqx_bridge_mqtt_lib:clientid_base/1` to construct routesync client id
2024-06-17 12:23:51 +03:00
e7305c62ee
feat(clusterlink): replicate shared subscription and persistent session routes
2024-06-17 12:23:51 +03:00
f036b641eb
feat(clusterlink): integrate node local routes replication and message forwarding
2024-06-17 12:23:51 +03:00
7df91d852c
feat(clusterlink): integrate node local syncer/actor implementation
...
- support multiple cluster links in extrouter
- apply extrouter ops on 'message.publish' hook
- fix several minor bugs
2024-06-17 12:23:51 +03:00
cbd01ae818
feat(clusterlink): add node-local route sync actor implementation
2024-06-17 12:23:51 +03:00
7b95273218
feat(routesync): make syncer a bit more generic and reusable
2024-06-17 12:23:51 +03:00
5bd9ee5c7f
feat(utils): add `emqx_utils_ets:keyfoldl/3` function
...
Designed to be used with `bag` / `duplicate_bag` tables.
2024-06-17 12:23:51 +03:00
4097585f5d
fix(clusterlink): ensure extrouter works on replicants
...
This is sort of a quick fix to make things safe, but it will likely be
a subject to the same drawbacks as the regular router in high-latency
deployments: reduced throughput.
2024-06-17 12:23:51 +03:00
a53524c826
fix(cth-cluster): fix occasional case clauses during cluster bootup
2024-06-17 12:23:51 +03:00
2dd99c5a08
feat(clusterlink): add facility to reconstruct remote routing table
2024-06-17 12:23:51 +03:00
f08342c704
feat: cluster link prototype WIP
2024-06-17 12:23:48 +03:00
98a54994c0
Merge pull request #13196 from lafirest/fix/rules_len
...
fix(authz_mnesia): add a soft limit in the API for the length of ACL rules
2024-06-17 15:47:59 +08:00
17809a5803
chore: fix dialyzer warning
2024-06-17 14:07:30 +08:00
1716852057
fix(authz_mnesia): improve field names and changes
2024-06-17 10:09:25 +08:00
Serge Tupchii