The picture of silicon industry that you paint is very far from reality. Neither Taiwan nor South Korea are third world countries and baking top chips isn't the same as sewing cotton T-shirts or gathering strawberries under the scorching sun. Instead of "cheaper labor", you need top talent and very high quality equipment. That top talent collects very good salaries.
If chip production was just a matter of money, both China and the U.S. would rule the roost. The real bottleneck is talented and loyal engineers.