You're right, you shouldn't have to design your project around your tools. Rather you describe the individual components of your system, really just document them in a structured manner, and let your tool manage the resulting graph.
Here's a link the Github repo in case you're interested: https://github.com/garden-io/garden. Always happy to get some feedback!
That's why we spent so much time working out the plugin mechanism, and figuring out what's native to the core framework. Terraform, for example, nails the infrastructure stuff quite well and strikes a good balance there, so we used that as a reference. But we added specific primitives for developing the application layer, and made that pluggable through a similar approach. Providers figure out the _how_, but the relationships in the graph are predictable.
Case in point, we now have a handful of different ways to deploy code. There's a Helm module type, which is very flexible but carries all the complexity and manual effort you need to make Helm charts. On the other side there's OpenFaaS, which is very constrained in features, but super easy to get going. So you can pick and choose depending on the level of abstraction you need.
Plus you'll soon be able to make your own, so you can tailor the abstractions to your needs, but keep the consistency of a single core dev tool.