I agree, to an extent.
I find that if I need to be "attentive" then having something going on helps. For me it is podcasts while driving. Having nothing noise my mind tends to wander which isn't always ideal because my focus should still be on the road despite engaging my brain with passive knowledge.
But for the rest of your comments? I agree wholeheartedly. I used to have the headphones in walking dogs, listening to podcasts, but I've stopped doing that and find the time outside far more relaxing, because that is time I should let my mind wander and venture. At work? Music to fill the voids, but I mute it if I'm trying to roll over some sort of problem in my head. Other strategies such as leaving the desk or using physical mediums (writing / sketching problems in notebooks) help too, but that's not really where we're at in this discussion.
Ultimately I think people need to just be aware that how they troubleshoot and relax won't be the same as another's. But it takes discomfort and experimentation to find what works for you too. Try podcasts where you would use music. Try the opposite. Try nothing where you would try something. Find what helps, and always be willing to re-test with new/old atmospheres.