> But it's always strange to me when people say the same thing to mean the opposite, that prosperity is just luck, and not the result of anyone's work at all.

It's easier to achieve success, when you're born in rich family, rich country. It's easier, when you have good health, high intelligence and great people skills. In modern world these factors make difference between success and failure. That's why being "self-made" in most case is over hyped. We can compensate our flaws, but some start without them. In almost every case, you won't reach the top from bottom, because gap in wealth is too huge. "Self-made" is survivor bias.