I think this is almost true. It would be true if index funds held all the stock. But since they don't and managed funds still exist, the stock price will go down when managed funds decide to sell. When the stock price goes down, the shares become a lower fraction of the index, so the index funds will also sell some.
I think the main point is that index funds still rely on traditional market players to effectively allocate risk. And as index funds take up more of the market, they become less able to do that. Right?