This introduces a refactoring that will hopefully help avoid conflicts with ongoing work to migrate our build / CI tooling to Mix.
Fixes https://emqx.atlassian.net/browse/EMQX-12527