>Anecdotally I have found it faster to order pcbs and get them shipped from China than get them built locally.
I'm not convinced this is because of the workers still. What you have in Shenzhen is that you have everything in one place to build that PCB. Everything your factory doesn't have, you can get from a neighboring company. The supply chains in the US have been starved for years, so if you want to do the same from the US, you have to wait for parts to be shipped from China. Making quick iteration impossible.
Shipping only the finished PCB is of course faster than if the development of the PCB needs multiple shipments of parts.
I don't have a solution for this, but the US outsourced so much manufacturing, that it can't manufacture efficiently anymore.