> What would you suggest software engineers do?
I would expect they would have the awareness and wherewithal to recognize the existential threat for what it is, and then take the appropriate actions to preserve their futures and thus allow them and their families to survive.
We are closing in on solutions to problem domains where discoveries can potentially have global impacts and consequences.
Yes, we all have to eat and some of us have to pay a mortgage or are up to our eyeballs in education debt. These are minuscule worries in comparison to the risk in the thought experiment I describe below.
Technology shocks cannot be predicted, just like the stock market can't be predicted. Past performance doesn't predict future results.
Say someone develops general AI, if costs of labor suddenly drop to 0 for some but not all and its kept secret, that would be exploited to consolidate, pressure, out-compete, leverage, buyout, centralize, and then hold hostage to attain ever more increasing amounts of control and power.
Less and less people would have jobs, economy goes over a cliff, and unrest occurs as food security and other safety nets become overwhelmed. It doesn't happen immediately unless it becomes broadly known.
As we have more people on this planet, the likelihood of having dramatic shifts in technology increases wrt time, the rate of change of progress will continue increasing unless we place limits and enforce behaviors that will preserve our society/future.
So far our leaders haven't been up to the task as we have seen with climate change, its been 30 years since we've known, and the Oil company misinformation campaigns were damn effective, while there was no effective action by our leaders who are paid well for doing almost nothing of consequence to reverse or slow climate changes progress.
The reason I address that is because its an existential threat. Most people don't know or were not educated as to why Africa and South America never rapidly developed like the northern hemispheres. The largest factor was the latent energy of the tropical climate made the environment so adverse to human settlement (increasing mortality), and animal husbandry (parasites, and other mortality related factors) making it almost impossible to have a break even ratio in birth/death rates in certain areas let alone grow.
As a final thought experiment here's some food for thought, say you come across a discovery with global consequence as part of your work. Like an episode in the show Human Target, where there is a key that will open any door you care to open, do you take it?
Have you considered that if you even accidentally hinted in any way that could be intuited that you knew about its existence, you would almost certainly have it stolen, be tortured, and likely killed just for the possibility, right alongside anyone close to you.
Most people live in a nice candy-coated reality and underestimate the potential ruthlessness of others. If you don't have the awareness, power, and resources to defend yourself at all times, what do you think will be the most likely outcome in that kind of scenario?
The moral of the story is, having anything including just an awareness of certain things, with global or geopolitical value immediately makes you a potential target (victim).
Most people in rapidly developing and new fields fail to consider the potential personal risks of doing work where that might be a potentiality because it is an out of context problem for them.