The last bit doesn’t make a whole lot of sense because AWS is all hosted in racks. It’s just that people don’t typically set up virtualized networking that way.
We did and it was fantastic. All of our “environments” were overlay networks spanning our hypervisors and we provided “ops” services outside those networks just like AWS where they just got an interface in the environments.
I'm convinced that there is no other way to manage networks after this. The ops team has their own completely separate view of the infrastructure that can be managed, moved, and shifted around so long as you keep the fiction the same.