We are talking about riches here, not transitioning from poverty to an upper-middle class income. What you say is absolutely true, but only up to a point, because money has diminishing marginal returns to happiness.
Why? Because of everything you state: it is unpleasant to have to think about how you'll feed and house yourself. But it does not require riches (in western living context) to be able to stop worrying about that.
Where money stops driving happiness varies by person, and by regional cost of living, but the research we have available suggests that it's not an extraordinarily high amount. Maybe $90k or so for most people.
But if you're making $150k, will having $10m in the bank make your life dramatically happier? The evidence suggests that no, it will not. That's when money fails to buy happiness.
