Commit Graph

12681 Commits

Author SHA1 Message Date
Andrew Mayorov fb9afd8313
feat(bridge-s3): beautify posix write errors 2024-06-25 16:51:07 +02:00
Andrew Mayorov f3ffbd4710
feat(bridge-s3): provide more meaningful error details in status 2024-06-25 16:51:06 +02:00
Andrew Mayorov 98e4ea6fde
feat(bridge-s3): make validation errors more readable
And also turn them into schema-level validations, instead of
bridge-level error conditions.
2024-06-25 16:51:06 +02:00
Thales Macedo Garitezi 1a497bcaf2 fix(greptime): correctly define grpc options for `grpcbox_channel`
Will probably fix CI flakiness.
2024-06-25 09:47:31 -03:00
Thales Macedo Garitezi c9ec5ac87b test(gcp consumer): stabilize flaky test
https://github.com/emqx/emqx/actions/runs/9614788348/job/26526973635?pr=13317#step:5:1463

```
%%% emqx_bridge_gcp_pubsub_consumer_SUITE ==> t_connection_down_before_starting: FAILED
%%% emqx_bridge_gcp_pubsub_consumer_SUITE ==> {{panic,
     #{msg => "Unexpected result",
       result =>
           {run_stage_failed,error,
               {assertMatch,
                   [{module,emqx_bridge_gcp_pubsub_consumer_SUITE},
                    {line,1451},
                    {expression,"health_check ( Config )"},
                    {pattern,"{ ok , connecting }"},
                    {value,{ok,disconnected}}]},
               [{emqx_bridge_gcp_pubsub_consumer_SUITE,
                    '-t_connection_down_before_starting/1-fun-11-',4,
                    [{file,
                         "/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},
                     {line,1451}]},
                {emqx_bridge_gcp_pubsub_consumer_SUITE,
                    t_connection_down_before_starting,1,
                    [{file,
                         "/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},
                     {line,1427}]}]}}},
 [{emqx_bridge_gcp_pubsub_consumer_SUITE,t_connection_down_before_starting,1,
      [{file,
           "/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},
       {line,1462}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1793}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1302}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1234}]}]}
```
2024-06-25 09:36:55 -03:00
ieQu1 9f30da334f
Merge pull request #13248 from ieQu1/dev/EMQX-12491-local-backend
replace builtin DS backend with builtin_local and builtin_raft
2024-06-24 23:07:57 +02:00
ieQu1 d349f84f04
fix(ds): Apply remarks 2024-06-24 20:20:40 +02:00
Andrew Mayorov d12b985507
test(api-clients): refactor suite to isolate flakiness 2024-06-24 16:54:20 +02:00
zmstone 5d1608f34b chore: minimize oss/platform diff 2024-06-24 15:30:03 +02:00
Thales Macedo Garitezi debf1e6cd5
Merge pull request #13316 from thalesmg/fix-mt-read-user-prop-m-20240621
fix(message transformation): correctly read from user properties in context
2024-06-24 09:31:48 -03:00
firest abc255bb02 fix(oidc): make CI happy 2024-06-24 18:11:02 +08:00
firest 892420e2c6 feat(oidc): be more compatible with okta 2024-06-24 16:51:54 +08:00
firest ddb197951e feat(oidc): implement JWKS, private_key_jwt, DPoP 2024-06-24 16:51:43 +08:00
firest 9c0df3c0a8 feat(oidc): support the PKCE extension 2024-06-24 16:51:32 +08:00
firest 5e2693c9b4 feat(oidc): implement session management 2024-06-24 16:51:20 +08:00
firest 512b4b9cbb feat(sso): add OIDC support 2024-06-24 16:50:58 +08:00
JianBo He 23acbb664e
Merge pull request #13258 from HJianBo/fix-boot-order-for-gws
fix: gateway app must depends on all auth apps
2024-06-24 16:31:27 +08:00
zmstone f9e17d6c25
Merge pull request #13269 from JimMoen/improve-plugin-behavior
fix: mark fresh install to cp the default configuration file directly
2024-06-24 09:17:25 +02:00
Ivan Dyachkov 4a81c3ac18
Merge pull request #13298 from qzhuyan/fix/william/hot-conf-disable-partial-chain
fix(tls): disable partial_chain in hot config
2024-06-24 09:02:07 +02:00
ieQu1 c0472a06f1
test(ds): Set initial conditions in repl. suite explicitly 2024-06-22 02:55:05 +02:00
Thales Macedo Garitezi ec83d999bf Merge remote-tracking branch 'origin/release-57' into 20240621-sync-r57-m 2024-06-21 14:11:08 -03:00
ieQu1 9a58d71378
fix(ds): Move DS backend supervision trees to their own apps 2024-06-21 18:22:04 +02:00
ieQu1 3d69ec496a
fix(schema): Transform config of `singleton' discovery_strategy 2024-06-21 17:16:25 +02:00
ieQu1 d7d878fd43
test(ds): Fix emqx standalone test profile selection 2024-06-21 17:16:25 +02:00
ieQu1 8aa27488b6
test: Disable certain DS-related suites in CE 2024-06-21 17:16:24 +02:00
ieQu1 1d3b1868fb
test(ds): Use close_db API 2024-06-21 17:16:24 +02:00
ieQu1 b2f7815a7f
test(ds): Don't start raft explicitly 2024-06-21 17:16:24 +02:00
ieQu1 3851fc189f
fix(ds): Avoid reverse dependencies from storage to repl. layer 2024-06-21 17:16:24 +02:00
ieQu1 bc915216a0
feat(ds): Support metrics for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1 be6c5e172f
fix(ds): Disable DS management APIs for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1 99c9b56cf3
feat(ds_buffer): Add `ls' function to list all local buffers 2024-06-21 17:16:24 +02:00
ieQu1 5a8818edf3
feat(ds): Add schema for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1 4484f30021
feat(ds_schema): Add schema for builtin_local backend 2024-06-21 17:16:24 +02:00
ieQu1 8990b1312b
refactor(ds_schema): Rename backend builtin -> builtin_raft 2024-06-21 17:16:24 +02:00
ieQu1 ea48b1265d
refactor(ds_schema): Extract common builtin fields 2024-06-21 17:16:24 +02:00
ieQu1 abe41de19b
refactor(ds_schema): builtin_local_buffer -> builtin_buffer 2024-06-21 17:16:24 +02:00
ieQu1 ecb172b07e
refactor(ds): Rename egress metrics to 'buffer' 2024-06-21 17:16:24 +02:00
ieQu1 09c3ae795d
refactor(ds_raft): Replace egress server with common emqx_ds_buffer 2024-06-21 17:16:24 +02:00
ieQu1 a0fbd37e58
refactor(emqx): Use emqx_ds_backends application 2024-06-21 17:16:24 +02:00
ieQu1 ef09cfcd71
feat(ds): Add `emqx_ds_backends` application 2024-06-21 17:16:24 +02:00
ieQu1 279619fc80
feat(ds): Add `builtin_local' backend 2024-06-21 17:16:24 +02:00
ieQu1 a8ea0ae4e5
refactor(ds): Extract DS replication layer to a separate application 2024-06-21 17:16:24 +02:00
ieQu1 63f1856a2c
feat(ds): Dynamic backend registration 2024-06-21 17:16:24 +02:00
ieQu1 83dc8f4d77
fix(ds): Fix return values of emqx_ds_storage_layer functions 2024-06-21 17:16:23 +02:00
ieQu1 a18d1987a2
test(ds): Add a helper function for diffing messages 2024-06-21 17:16:23 +02:00
Ilia Averianov f1b8c356a6
Merge pull request #13299 from savonarola/0617-shared-sub-leader-poc
feat(queue): implement PoC version of session ↔️ shared group leader interaction
2024-06-21 16:45:17 +03:00
Thales Macedo Garitezi 28c7d94bd2 fix(message transformation): correctly read from user properties in context
Fixes https://emqx.atlassian.net/browse/EMQX-12582
2024-06-21 09:39:22 -03:00
Ilya Averyanov db28a042d5 feat(queue): handle renew_lease_timeout 2024-06-21 15:18:37 +03:00
zhongwencool 75a524c916 test: add more debug msg to flaky cluster_rpc SUITE 2024-06-21 18:04:24 +08:00
Ilya Averyanov 2096755ad6 feat(queue): reorganize group sm callbacks and methods 2024-06-21 13:01:38 +03:00
Ilya Averyanov b9c5911883 feat(queue): implement `find_leader_timeout` event and so the `connecting` group_sm state 2024-06-21 12:22:17 +03:00
SergeTupchiy fb266fbf8c
Merge pull request #13300 from SergeTupchiy/EMQX-12195-cluster-link-conf-backup
cluster link conf backup
2024-06-21 11:42:57 +03:00
zmstone 8a99995810 docs: fix regex_extract function doc 2024-06-21 09:25:09 +02:00
JianBo He d8e6c07ca0 chore: fix failed tests 2024-06-21 11:04:23 +08:00
JianBo He a8af90f912 chore: add changes 2024-06-21 09:39:14 +08:00
JianBo He 08f085d823 fix: gateway app must depends on all auth apps 2024-06-21 09:39:14 +08:00
Ilya Averyanov 979fb58e50 feat(queue): use tp to trace new lease events 2024-06-20 21:12:15 +03:00
Ilya Averyanov 1205e34650 feat(queue): remove too useless header file 2024-06-20 20:48:56 +03:00
JimMoen acc8bf3405
Merge pull request #13222 from JimMoen/fix-will-qos-retain-flag-check
fix: check willretain and willqos when WillFlag set to `true`
2024-06-21 01:28:23 +08:00
Ivan Dyachkov ba4fb0d3f9
Merge pull request #13290 from kjellwinblad/kjell/format_rule_cli/EMQX-12548
fix: do not crash when showing rule unsing the cmd interface
2024-06-20 14:55:33 +02:00
zmstone 2a0071aa01
Merge pull request #13295 from thalesmg/test-flaky-greptime-mkII-r57-20240619
test(greptime): attempt to fix flaky tests (attempt without driver patch)
2024-06-20 14:49:21 +02:00
Ilya Averyanov be175d205c feat(queue): add JIRA task links for TODOs 2024-06-20 15:38:16 +03:00
Ilya Averyanov a9c976b6c1 feat(queue): fix notes about the usage of agent<->leader protocol-related data structures 2024-06-20 15:24:25 +03:00
Ilya Averyanov d0cff63ed6 feat(queue): remove unused hrl-guard 2024-06-20 15:18:47 +03:00
Ilya Averyanov a92460d38f feat(queue): improve hints for macro clauses 2024-06-20 15:17:08 +03:00
Serge Tupchii ac8762be10 chore: upgrade ekka to 0.19.5 (mria 0.8.8)
mria 0.8.8 heals a network partition once majority of core nodes are alive.
Previously, the autoheal worked only when all core nodes were reachable.

Fixes: EMQX-10974
2024-06-20 12:53:58 +03:00
William Yang 464a0a82f0 fix(tls): move default_root_fun to ?CONST_MOD_V1 2024-06-20 11:38:52 +02:00
zmstone 7d32275ebe chore: minimize oss/platform diff 2024-06-20 10:36:01 +02:00
Kjell Winblad 12859b8959 fix: handle yet another action format 2024-06-20 10:03:49 +02:00
JimMoen 675abd7512
test: will retain and willqos in connect flags 2024-06-20 10:32:15 +08:00
JimMoen cf9d6943d5
fix: check willretain and willqos when WillFlag set to `true` 2024-06-20 10:32:15 +08:00
JianBo He c307c1dfc2 chore: cut v5.7.1-alpha.1 2024-06-20 09:40:42 +08:00
Ilya Averyanov 6da10036dc feat(queue): fix dialyzer issues 2024-06-19 22:25:53 +03:00
Ilya Averyanov a4bbab4aa2 feat(queue): document general entity layout 2024-06-19 22:24:20 +03:00
JimMoen ff742d926a
fix: `?SLOG` with msg field 2024-06-20 02:18:26 +08:00
Serge Tupchii 71f5eaf11e fix(data_backup): import `cluster.links` conf 2024-06-19 20:47:06 +03:00
Serge Tupchii ef5cf4fac3 chore(clusterlink): fix comment 2024-06-19 20:46:00 +03:00
Ilya Averyanov 2b0146663a feat(queue): add smoke test for ds shared subs 2024-06-19 19:32:23 +03:00
Ilya Averyanov e3c4816035 feat(queue): move group subscription state machine to its own module 2024-06-19 19:22:10 +03:00
Thales Macedo Garitezi 9559ba2f06 test: remove unnecessary lines 2024-06-19 13:22:02 -03:00
William Yang 8d04545f03 test(tls): root_fun is set even when partial_chain is false 2024-06-19 17:34:04 +02:00
William Yang 41239ae766 fix(tls): disable partial_chain in hot config 2024-06-19 17:03:05 +02:00
zmstone 021b6b3902 Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master 2024-06-19 15:06:41 +02:00
zmstone 02a6ee1ef4
Merge pull request #13279 from zmstone/0617-json-decode-payload-for-map_to_redis_hset_args
fix(redis): json decode if arg for map_to_redis_hset_args is string
2024-06-19 14:55:21 +02:00
zmstone 213e4785e7 Merge remote-tracking branch 'origin/release-57' into 0619-sync-release-57-to-master 2024-06-19 14:54:48 +02:00
JimMoen a6fa3e82d9
fix: prevent use pd 2024-06-19 20:15:07 +08:00
JimMoen b7b2a08399
fix: match plugin_not_found reason 2024-06-19 20:15:07 +08:00
JimMoen 28293284a7
fix: let crash when plugin not found in cluster 2024-06-19 20:15:07 +08:00
JimMoen 19e039e0d2
fix: mark fresh install to cp the default configuration file directly 2024-06-19 20:15:07 +08:00
Thales Macedo Garitezi ff2e6e1434 test(greptime): revert driver patch to test ci again 2024-06-19 09:12:30 -03:00
zmstone 95d387a790
Merge pull request #13294 from zmstone/0618-per-release-edition-code-injection
0618 per release edition code injection
2024-06-19 11:53:57 +02:00
zmstone f0e5721959 refactor: make dialyzer happy 2024-06-19 10:42:08 +02:00
SergeTupchiy 0d098a01ef
Merge pull request #13293 from SergeTupchiy/EMQX-12345-reindex-retained-msgs-on-import
reindex retained msgs on import
2024-06-19 11:23:08 +03:00
Thales Macedo Garitezi ffec1c7fe0 fix(greptime): attempt patch for cleaning up channels when worker terminates 2024-06-18 22:16:03 -03:00
Thales Macedo Garitezi 9dd419d822 test(greptime): attempt to fix flaky tests
```
=WARNING REPORT==== 18-Jun-2024::12:42:04.739109 ===
    reason: no_endpoints
    msg: failed_to_start_greptimedb_connector
    client: #{protocol => http,
              pool => <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>,
              cli_opts =>
                  [{endpoints,[{http,"toxiproxy",4001}]},
                   {pool_size,2},
                   {pool_type,random},
                   {auto_reconnect,1},
                   {gprc_options,#{sync_start => true,
                                   connect_timeout => 5000}},
                   {dbname,"public"},
                   {auth,{basic,#{password => <<"******">>,
                                  username => "greptime_user"}}},
                   {https_enabled,false}]}
    connector: <<"connector:greptimedb:emqx_bridge_greptimedb_SUITE">>
```
2024-06-18 18:23:22 -03:00
zmstone 54fc605cc5 refactor: move schema injection from hrl to erl 2024-06-18 22:22:53 +02:00
zmstone a53fa977df refactor: delete an unused macro 2024-06-18 22:22:48 +02:00
Thales Macedo Garitezi e69ba33420
Merge pull request #13291 from thalesmg/fix-ds-mgmt-node-down-r57-20240618
fix(dsreplmeta): check site status when fetching shard info
2024-06-18 15:14:08 -03:00
Ilya Averyanov bca743054b feat(queue): implement backbones of queue agent, leader and leader registry 2024-06-18 21:03:51 +03:00
Thales Macedo Garitezi 4c54ab6379 test: remove tests per review request 2024-06-18 13:08:19 -03:00
Kjell Winblad 4a318f8f51 style: fix type problem found by dialyzer 2024-06-18 17:02:26 +02:00
Serge Tupchii baa7996289 fix(data_backup): allow exporting `ram_copies` Mnesia tables
Currently, ram tables can be used for message retainer.
2024-06-18 17:31:33 +03:00
Serge Tupchii 60882a616e fix(data_backup): re-index retained messages after importing 2024-06-18 17:31:19 +03:00
Thales Macedo Garitezi 6bde6aa711
Merge pull request #13280 from thalesmg/fix-plugin-elixir-r57-20240617
fix(plugins): do not stop Elixir protected apps
2024-06-18 10:41:12 -03:00
Thales Macedo Garitezi 8ce16fd7d9 fix(dsreplmeta): check site status when fetching shard info
Fixes https://emqx.atlassian.net/browse/EMQX-12356
2024-06-18 10:40:09 -03:00
Kjell Winblad a4f855108c fix: do not crash when showing rule unsing the cmd interface
Before the change the command line command

    $ bin/emqx ctl rules show rule_0hyd

would crash if the rule had bridge actions. This has now been fixed by
adding a handler for printing  bridge actions.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12548
2024-06-18 15:17:25 +02:00
Kjell Winblad 3f7723b2dc
Merge pull request #13274 from kjellwinblad/kjell/add_ssl_to_rocksdb/EMQX-12289
feat: add SSL support to RocketMQ connector
2024-06-18 09:38:28 +02:00
Thales Macedo Garitezi 20cffb54d4 Merge branch 'release-57' into sync-r57-m-20240617 2024-06-17 17:10:12 -03:00
Thales Macedo Garitezi 6897f0141b fix(plugins): do not stop Elixir protected apps
On one hand, Elixir plugins might include Elixir itself, when targetting a non-Elixir EMQX
release.  If, on the other hand, the EMQX release already includes Elixir, we shouldn't
stop Elixir nor IEx, or the running IEx shell will break.
2024-06-17 16:42:23 -03:00
Ilia Averianov ad993437aa
Merge pull request #13278 from savonarola/0617-release-57-sync
Sync release-57
2024-06-17 21:25:00 +03:00
zmstone 9479c8d33b fix(redis): json decode if arg for map_to_redis_hset_args is string 2024-06-17 20:19:53 +02:00
Andrew Mayorov eede9f349e
Merge pull request #13276 from keynslug/fix/ds/inherited-lts-lost
fix(dsstore): persist inherited LTS trie
2024-06-17 20:25:31 +03:00
zmstone ae22a64157
Merge pull request #13277 from zmstone/0617-handle-kafka-message_too_large-return
0617 handle kafka message too large return
2024-06-17 19:15:33 +02:00
Kjell Winblad 0cbbae9655 fix: make fixes due to comments from @thalesmg 2024-06-17 19:06:12 +02:00
Kjell Winblad c39c544c96 feat: add SSL support to RocketMQ connector 2024-06-17 19:06:12 +02:00
Thales Macedo Garitezi b64f0c0ca7
Merge pull request #13231 from thalesmg/mt-test-endpoint-r57-20240611
feat(message transformation): implement dryrun endpoint
2024-06-17 13:11:39 -03:00
Thales Macedo Garitezi 240bb09070
Merge pull request #13275 from thalesmg/test-flaky-sk-r57-20240617
test: attempt to fix flaky test
2024-06-17 13:01:08 -03:00
Ilya Averyanov f8e6aab86f Merge branch 'release-57' into 0617-release-57-sync
* release-57:
  chore(auth,http): cache REs for parsing URIs
  fix(auth,http): improve URI handling
  chore: revert ULOG/ELOG
  test: generate dispatch.eterm in dashboard test
  docs: refine change log
  feat: make the dashboard restart quicker
  chore: fix typo
  fix(http authz): handle unknown content types in responses
  chore: change types of mysql and mongodb fields to `template()`
  fix(client mgmt api): allow projecting `client_attrs` from client fields
  fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-17 18:53:39 +03:00
Andrew Mayorov 82588fbc35
chore(dsstore): refine module and callback typespecs 2024-06-17 17:53:39 +02:00
Andrew Mayorov 05d97397d3
fix(dsstore): use correct comparison when walking over generations 2024-06-17 17:53:39 +02:00
Andrew Mayorov 22009bcc58
test(dsstore): verify that inherited TLS trie is persisted 2024-06-17 17:53:38 +02:00
Andrew Mayorov 68f6556856
fix(dsstore): persist inherited LTS trie
Before this commit, inherited trie was actually only kept in memory
cache.

Also simplify storage backend behaviour around inheriting previous
generation's legacy.
2024-06-17 17:53:38 +02:00
SergeTupchiy 830266b4d5
Merge pull request #13126 from SergeTupchiy/EMQX-11967-cross-cluster-route-replication-syncer-actor
cluster linking
2024-06-17 18:52:46 +03:00
zmstone 3ac4ddcbe3 fix(kafka): handle message_too_large
bump 'failed' counter
2024-06-17 17:27:25 +02:00
zmstone 00dfdc22cf chore: pin Kafka client wolff 0.10.5 2024-06-17 17:27:25 +02:00
zhongwencool 5ab4b321a0
Merge pull request #13267 from zhongwencool/use-redbug-for-trace
chore(debug): use redbug for default trace tools
2024-06-17 22:10:18 +08:00
Thales Macedo Garitezi fb492e3dc5
Merge pull request #13264 from thalesmg/fix-postgres-disabled-prepared-r57-20240614
fix(postgres): authn/authz/batch requests when prepared statements are disabled
2024-06-17 11:05:05 -03:00
Thales Macedo Garitezi 5665a4917c
Merge pull request #13266 from thalesmg/fix-s3-release-app-r57-20240614
fix: add `emqx_s3` to `reboot_lists.eterm`
2024-06-17 10:48:12 -03:00
Thales Macedo Garitezi ec4f462684 refactor: apply review requests 2024-06-17 10:39:32 -03:00
Thales Macedo Garitezi 1464094967 test: attempt to fix flaky test
```
%%% emqx_persistent_session_ds_SUITE ==> t_session_discard_persistent_to_persistent: FAILED
%%% emqx_persistent_session_ds_SUITE ==> {noproc,{gen_server,call,[snabbkaffe_collector,flush_trace,infinity]}}
```
2024-06-17 10:34:57 -03:00
Thales Macedo Garitezi b71212b022 refactor: apply review comments 2024-06-17 09:40:33 -03:00
Thales Macedo Garitezi f5eb3e7471
Merge pull request #13250 from thalesmg/ds-singleton-m-20240613
feat: enforce singleton discovery strategy when using DS `builtin_local` backend
2024-06-17 09:34:05 -03:00
Ilya Averyanov 19c9f0d76f chore(auth,http): cache REs for parsing URIs 2024-06-17 14:42:29 +03:00
Ilya Averyanov f7ac829f28 fix(auth,http): improve URI handling 2024-06-17 14:42:29 +03:00
Serge Tupchii a905a6048c chore(clusterlink): rename link `upstream` field to `name` 2024-06-17 12:24:51 +03:00
Serge Tupchii a95a08efd3 test(clusterlink): add more test cases 2024-06-17 12:24:51 +03:00
Serge Tupchii 44c37571cc fix(clusterlink): ignore not_registered error 2024-06-17 12:24:51 +03:00
Serge Tupchii 00f912928f fix: fix emqx_external_broker:forward/1 spec 2024-06-17 12:24:51 +03:00
Andrew Mayorov ed16ff07df refactor(broker): simplify external broker behaviour 2024-06-17 12:24:51 +03:00
Serge Tupchii d282c61120 feat(clusterlink): update only necessary resources when a link config is changed 2024-06-17 12:24:50 +03:00
Serge Tupchii ff16521d4f fix(clusterlink): add schema descriptions, fix dialyzer warnings, add resource_opts 2024-06-17 12:23:52 +03:00
Serge Tupchii 94e81ba812 feat(clusterlink): implement actor config handler 2024-06-17 12:23:52 +03:00
Serge Tupchii c871b37453 fix(clusterlink): add link topics schema validator 2024-06-17 12:23:52 +03:00
Andrew Mayorov 780a0bf807 fix(cluster-link): clear exit signal of failed-to-connect client 2024-06-17 12:23:52 +03:00
Andrew Mayorov d0df4de2a3 test(cluster-link): add e2e replication actor GC testcase 2024-06-17 12:23:52 +03:00
Andrew Mayorov c4840b30d2 fix(cluster-link): deduplicate routes down to dest cluster 2024-06-17 12:23:52 +03:00
Andrew Mayorov ede35df24a fix(cluster-link): cancel heartbeats on client down 2024-06-17 12:23:52 +03:00
Andrew Mayorov e0604e3af6 fix(cluster-link): anticipate clients may occasionally retry 2024-06-17 12:23:52 +03:00
Andrew Mayorov 0219b8bd4d feat(cluster-link): add simple replication actor GC process 2024-06-17 12:23:52 +03:00
Andrew Mayorov 7fccb5dbc9 test(topic): add more `intersection/2` testcases 2024-06-17 12:23:52 +03:00
Andrew Mayorov 7b8f466adf feat(topic): avoid `lists:reverse` when intersecting 2024-06-17 12:23:52 +03:00
Andrew Mayorov 24be189728 fix(topic): respect special topic rules when intersecting 2024-06-17 12:23:51 +03:00
Andrew Mayorov de1ac131f7 test(cluster-link): fix test suite setup 2024-06-17 12:23:51 +03:00
Serge Tupchii 58eaf07627 fix(clusterlink): valide config to disallow duplicated cluster links 2024-06-17 12:23:51 +03:00
Andrew Mayorov 54d51d0982 test(cluster-link): draft basic integration test suite 2024-06-17 12:23:51 +03:00
Andrew Mayorov e9c24090d4 fix(cluster-link): avoid starting ps syncer if persistence disabled 2024-06-17 12:23:51 +03:00
Andrew Mayorov 036c7e8492 test(cluster-link): adapt extrouter testsuite to new APIs 2024-06-17 12:23:51 +03:00
Andrew Mayorov 45eda4f3b9 fix(cluster-link): adapt heartbeat / reincarnation handling to new API 2024-06-17 12:23:51 +03:00
Andrew Mayorov 43d114546c feat(cluster-link): preserve replication actor state in pdict 2024-06-17 12:23:51 +03:00
Andrew Mayorov 5771a41a32 fix(cluster-link): ensure replication actor bootstraps do heartbeats 2024-06-17 12:23:51 +03:00
Andrew Mayorov d4b449c6e1 feat(cluster-link): implement replication actor heartbeats 2024-06-17 12:23:51 +03:00
Serge Tupchii faa4420e1f fix(clusterlink): improve actor error handling
Add status and error reason to the actor state, report alarms.
2024-06-17 12:23:51 +03:00
Serge Tupchii 21711c6e0d fix(clusterlink): communicate bootstrap requirements via actor handshake
`session_present` flag is not reliable to decide whether bootstrap is needed if durable sessions is enabled.
In this case, the client session may survive cluster restart while all the external routes are lost, as they are not persistent.
2024-06-17 12:23:51 +03:00
Serge Tupchii d5e82cdfac refactor(clusterlink): avoid unnecessary `maybe_` external_broker CB names 2024-06-17 12:23:51 +03:00
Serge Tupchii d578ac3f9e fix(clusterlink): match correct timer ref in router actor 2024-06-17 12:23:51 +03:00
Serge Tupchii b1aeb35370 feat(clusterlink): implement actor init handshake 2024-06-17 12:23:51 +03:00
Serge Tupchii ac19cf89df chore(clusterlink): remove code related to the rejected coordinator-based implementation 2024-06-17 12:23:51 +03:00
Serge Tupchii e26e7acaa1 refactor(clusterlink): use `emqx_bridge_mqtt_lib:clientid_base/1` to construct routesync client id 2024-06-17 12:23:51 +03:00
Serge Tupchii e7305c62ee feat(clusterlink): replicate shared subscription and persistent session routes 2024-06-17 12:23:51 +03:00
Serge Tupchii f036b641eb feat(clusterlink): integrate node local routes replication and message forwarding 2024-06-17 12:23:51 +03:00
Serge Tupchii 7df91d852c feat(clusterlink): integrate node local syncer/actor implementation
- support multiple cluster links in extrouter
 - apply extrouter ops on 'message.publish' hook
 - fix several minor bugs
2024-06-17 12:23:51 +03:00
Andrew Mayorov cbd01ae818 feat(clusterlink): add node-local route sync actor implementation 2024-06-17 12:23:51 +03:00
Andrew Mayorov 7b95273218 feat(routesync): make syncer a bit more generic and reusable 2024-06-17 12:23:51 +03:00
Andrew Mayorov 5bd9ee5c7f feat(utils): add `emqx_utils_ets:keyfoldl/3` function
Designed to be used with `bag` / `duplicate_bag` tables.
2024-06-17 12:23:51 +03:00
Andrew Mayorov 4097585f5d fix(clusterlink): ensure extrouter works on replicants
This is sort of a quick fix to make things safe, but it will likely be
a subject to the same drawbacks as the regular router in high-latency
deployments: reduced throughput.
2024-06-17 12:23:51 +03:00
Andrew Mayorov a53524c826 fix(cth-cluster): fix occasional case clauses during cluster bootup 2024-06-17 12:23:51 +03:00
Andrew Mayorov 2dd99c5a08 feat(clusterlink): add facility to reconstruct remote routing table 2024-06-17 12:23:51 +03:00
Serge Tupchii f08342c704 feat: cluster link prototype WIP 2024-06-17 12:23:48 +03:00
lafirest 98a54994c0
Merge pull request #13196 from lafirest/fix/rules_len
fix(authz_mnesia): add a soft limit in the API for the length of ACL rules
2024-06-17 15:47:59 +08:00
zhongwencool 17809a5803 chore: fix dialyzer warning 2024-06-17 14:07:30 +08:00
firest 1716852057 fix(authz_mnesia): improve field names and changes 2024-06-17 10:09:25 +08:00
zhongwencool a49e5877ec chore(debug): use redbug for default trace tools 2024-06-17 09:28:23 +08:00
zmstone 8daaf7b727
Merge pull request #13233 from zmstone/0611-expose-regex_extract-func
fix(emqx_rule_funcs): expose regex_extract function to rule engine
2024-06-16 16:16:48 +02:00
zmstone 2b7f3a597f
Merge pull request #13245 from qzhuyan/merge/william/sync-rel57-to-master
sync release57 to master
2024-06-15 09:13:55 +02:00
zhongwencool d433fc7c30
Merge pull request #13236 from zhongwencool/bug-schema-validation
fix: some minor bug fixes
2024-06-15 06:40:28 +08:00
Thales Macedo Garitezi 198ccd320b fix: add `emqx_s3` to `reboot_lists.eterm`
Otherwise, it's unconditionally started as a permanent application instead of being
controlled by `emqx_machine`.

```sh
ͳ rg start_boot _build/emqx-enterprise/rel/emqx/releases/5.7.0-g4e6be8a6/start.script | rg emqx_s3
     {apply,{application,start_boot,[emqx_s3,permanent]}},
```
2024-06-14 15:06:18 -03:00
Thales Macedo Garitezi 6e702cd16c fix(postgres): fix usage by authn/authz modules when prepared statements are disabled
Fixes https://emqx.atlassian.net/browse/EMQX-12550
2024-06-14 12:19:46 -03:00
zmstone 626aae6edf chore: fix bad conflict resolution 2024-06-14 16:57:53 +02:00
zhongwencool d95f17fe77 chore: revert ULOG/ELOG 2024-06-14 22:11:27 +08:00
Thales Macedo Garitezi fd5e844cc1 fix(postgres): correctly handle batch queries with disabled prepared statements
Fixes https://emqx.atlassian.net/browse/EMQX-12549
2024-06-14 11:10:21 -03:00
zhongwencool f713f13b2c test: generate dispatch.eterm in dashboard test 2024-06-14 21:52:58 +08:00
Thales Macedo Garitezi 9b3c806ba7 feat(message transformation): implement dryrun endpoint
Follow up to https://github.com/emqx/emqx/pull/13199
2024-06-14 10:19:36 -03:00
zhongwencool a6e3a09118 feat: make the dashboard restart quicker 2024-06-14 18:22:34 +08:00
JimMoen 0f0e7d18db
Merge pull request #13261 from JimMoen/fix-typo
chore: fix typo
2024-06-14 16:55:35 +08:00
JimMoen 6823c79ae0
chore: fix typo 2024-06-14 16:04:11 +08:00
William Yang b86d631744 test: fix tc t_handle_outing_non_utf8_topic 2024-06-14 08:17:19 +02:00
William Yang d39b8caff9 Merge branch 'release-57' into merge/william/sync-rel57-to-master 2024-06-14 08:17:09 +02:00
lafirest afd4b46f72
Merge pull request #13225 from lafirest/fix/mysql_redact
fix(auth): redact sensitive data for the authn/authz APIs
2024-06-14 09:19:21 +08:00
Thales Macedo Garitezi 01635722e9
docs: improve descriptions
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2024-06-13 18:34:44 -03:00
Thales Macedo Garitezi 52e6c88941 feat: enforce singleton discovery strategy when using DS `builtin_local` backend
Fixes https://emqx.atlassian.net/browse/EMQX-12493
2024-06-13 17:04:23 -03:00