Listen, I agree that this happens. But in my experience it happens so rarely that it does not outweigh all the negatives of having to drag a bunch of people to the office against their will day in, day out.
As for slack and such being a poor replacement to face to face, I'd say that it's because not all people are cut out for remote work. For me personally, I don't like it when somebody chats me up about things that are better suited for a slack (or even email) conversation. Things like getting the spec details buttoned down (which is actually the bulk of shop conversations, unless you're at a very early stage in your project) - you'd want to have a paper trail/reference to go back to once you start implementing. And remembering details from hallway conversations is hard, so you have to distract people from their work again or get blocked.