It is the most important step that you can package containers without having to know the production secrets and to have a "standard" was to retrieve them.
No one, without production access, has a way to obtain them.
> Currently, anyone with root on any node can read any secret from the apiserver, by impersonating the kubelet
> If multiple replicas of etcd are run, then the secrets will be shared between them. By default, etcd does not secure peer-to-peer communication with SSL/TLS, though this can be configured.
Agree, it would be better if the nodes could not trivially request other secrets. But you can request new containers placed on any node and a root escape that works on one node almost certainly works on the rest.
We usually recommend subdividing the node acls by namespace when running disjoint node sets (where tenant A can't schedule onto tenant B's nodes). More fiddly than it has to be in Kubernetes today.
We are currently facing the same problem and came to the same conclusion, we are subdividing our node pools by namespace (and using namespacing for multi-tenancy) but it "feels" very convoluted and a smell that there's a feature missing in Kubernetes itself to handle this more graciously.
I think in this case you'd already be compromised if you have to worry about a user being able to essentially maliciously place a container. Really a job for RBAC and not the scheduler itself.
