It's about spontaneous and unplanned interactions that provide new paths for you. Yes, you can still have those remotely - but they are no where nearly as frequent, nor anywhere near as impactful. They simply can't be - your fighting millions of years of social evolution if you think otherwise. Heck I doubt most of us remember a fraction of spontaneous in person interactions with others in our day to day life - but let me tell you there are a handful in my past that made all the difference in where I am today. None would have happened if I had been working remotely.
Not one of them.
You simply can't quantize the value of those interactions and I fear for those who are seemingly so willing to casually tossing those away. Yes, commuting sucks. Yes, committing to a schedule sucks. Yes, listening to the same story from Bob for the millionth time while you are trying to focus on something else sucks. But there are other benefits that are irreplaceable too.
Probably the best illustration I can think of in pop culture is the Star Trek:TNG episode where Picard's artificial heart dies and he has that whole near death experience with Q. It's a great analogy for this whole discussion. I'm too am glad I had the bloody nose early on and the much more interesting life overall.