What percentage of the jobs in a modern office are truly necessary? If automation had the ability to kill jobs over the long term, we'd all have been idle since the industrial revolution. But instead we keep inventing new things that we need.

See above.

A mix of political scientists, politicians, investors, entrepreneurs, lawyers, judges, scientists, technologists, and economists have tried to mold society to their own theoretical vision for at least 150+ years. Society then reacts to that in both good and bad ways. This distorts the vision, as society changes it to its concerns. And the cycle repeats.

I think of Karl Polyani's The Great Transformation has a great way of looking at the attempts to force "market society" on England in the 1700 and 1800s, and the reaction that all societies exhibit in the face of unconstrained technological or economic change. Both the imposition of change and reaction to it can be violent, it's hard to predict. We've had such a relatively steady state since WW2 in the developed nations that we're not used to this cycle.