I think it's a great example to say it is idiosyncratic that I can create a PD by creating a PV, but I can't reciprocally enumerate PDs by listing PVs. Another: Kubernetes secrets are hidden inside K8S and they are not generally accessible through any vendor-provided secrets managers outside of K8S.
I love Ingress, because it allows me to create new hostname-driven application endpoints inside of my cluster without any permissions on the AWS side. The cluster has a load balancer with a star certificate attached to it, the DNS has a star record similarly, and I can create new Ingress records without reaching outside of the cluster.
My InfoSec department hates stuff like that! Now if I use Kubernetes that means they have to learn Kubernetes too, and maybe even build out additional tooling around it, because their AWS-based automated service inventories are not reliable anymore.
I don't really have those star records (I'd really like to have them, it would be really convenient for me as a developer, so I don't have to ask for permission and go through layers of approval when I need to do stupid things, like my job...)
The unfortunate reality that I've encountered is that those teams are more likely to see the encapsulation features like this Ingress configuration as a pure security risk with no benefit to them, plain and simple.
K8s needs a role with the correct IAM permissions to create the resources on the AWS side, it can't do it magically. Which means that someone granted these permissions before.
If anything, your InfoSec department should love that. IAM is pretty fine-grained, so they can control what gets created. Types of resources, and whatnot. Just don't grant star permissions to all of EC2.
This is approaching Terraform levels of complexity though. Unless you mean that cloud providers should do that when you create a volume, in which case I understand where you are coming from and well, they could do that. Although I'd much have them exposing a native K8s interface to take care of this PV and PVC for me and not showing me the underlying volumes at all.
Why not use Kubernetes to declaratively define nodes, disks, load balancers, firewall rules, DNS rules...? If you look at Terraform, for example, it's just an "object store" (the state file) that you modify in order to "converge" your infrastructure.
If you're already using Kubernetes via GKE or a similar IaaS, then of course Kubernetes has underlying primitives (GCE disks, GLBCs, etc.) that are exposed. But those primitives are exposed because GKE runs on top of GCP.
What I'm proposing is a "KaaS" where Kubernetes is the control plane, without the baggage of an additional API.
You don't specify a node but the CPU/RAM your application requires and let K8S scale up the cluster if necessary. Likewise, you specifiy the storage needed instead of managing disks, and set network policies to control load balancing and routing. If you want to live in Kubernetes only today, you can absolutely do that. Launch your cluster once and then use kubectl and yaml for the rest.
