I don't see you disagreeing that the hypothetical voting shares via proxy is essentially more "passive" than the decision making they're doing now. I believe what you're saying is that they simply shouldn't vote at all.
I guess that's basically the most "passive" option, but it would have some weird results. I believe - and I'm well informed but not an expert when it comes to this - we'd find that at a lot of companies, if you take away the large institutional investors, the portion that remains includes a large number of activist shareholders of various stripes. Those activists would necessarily be strengthened in some ways by the majority of shareholders sitting out every vote.
There's an utterly pretentious Rush lyric that comes to mind, about choosing not to decide still being a choice, or something.
edit: I see that Bogle kind of hit the same point:
Limit the voting power of corporate shares held by index managers. But such a step would, in substance, transfer voting rights from corporate stock owners, who care about the long-term, to corporate stock renters, who do not... an absurd outcome.