I think it has been the gradual march of technological progress and globalization. For example remember the civil war was largely fought over protectionist trade policies for northern manufacturers. And that was in the age of sail when many goods would not even make if from North America to Europe. Similar protectionist policies would be unthinkable now. Also the rise of America from being an upstart industrial power like China and India are today to being a wealthy nation with (relatively) high standards of living for its low and middle class.
Unions are fascinating, honestly I am surprised that the era in which they were strong ever existed so early in history. They rely on having a monopoly on labor or at least a large percentage of it but with modern transportation the size of the labor pool is just too large and too poor for a relatively small, wealthy nation like the US to organize it.
If I had to point to seminal events causing the change, they would be construction of the Suez and Panama Canal, MacArthur's reconstruction of Japan, and Nixon opening trade with China.
This last one could be the reason we are all still alive today, but clearly the American middle and lower class have born the brunt of the costs relative to where we all were after WWII.
Btw thanks for the discussion. Has been a while since I read about panic of 1819! I also blame the aspects of Keynesian economics that do not constrain the growth of government debt for increasing the power of existing wealth relative to the working class, but I admit that is a more questionable opinion.