No one knows how far off true AGI is, just like no one in 1940 (or 1910) knew how far off fission weapons were.
EDIT: I quite liked this article from a few years back [0], and the fission weapon prediction example is stolen from there.
It was hard to predict when or if such a thing could be made, but everyone knew what was under discussion.
Compare this to AGI, some vaguely emergent property of a complex computer system that no one can define to anyone else's satisfaction. Attempts to be more precise what AGI is, how it would first manifest itself, and why on earth we should be afraid of it, rapidly devolve into nerd ghost stories.
1932 neutron discovered
1942 first atomic reactor
1945 fission bomb
1897 electron discovered
1940's vacuum tube computers
1970's integrated circuits
1980's first AI wave fails, AI winter begins
2012 AI spring begins
2019 AI can consistently recognize a jpeg of a cat, but still not walk like a cat
???? Human level AGI
1943 First mathematical neural network model
1958 Learning neural network classifies objects in spy plane photos
1965 Deep learning with multi-layer perceptrons
2010 ImageNet error rate 28%
2011 ImageNet error rate 25%
2012 ImageNet error rate 16%
2013 ImageNet error rate 11%
2017 ImageNet error rate 3%
2019 Pre-AGIExtrapolating as you seem to be here, when should I expect to see a total conversion reactor show up? I want 100% of the energy in that Uranium, dammit - not the piddly percentages you get from fission!
Seriously, I think you overestimate how predictable nuclear weapons were. Fission was discovered in 1938.
We haven't even had the AGI equivalent of the Rutherford model of the atom yet: what's the definition of consciousness? What is even the definition of intelligence?
All these things have surged incredibly in less than a decade.
It's always a long way off until it isn't.
Not at all, these are all one-trick poneys and bring you nowhere close to real AGI which is akin to human intelligence.
On the 2nd of December, 1942 he led an experiment at Chicago Pile 1 [1] that initiated the first self-sustaining nuclear reaction. And it was made with Uranium.
In fairness to Fermi, nuclear fission was discovered in 1938 [2] and published in early 1939.
0: https://books.google.com/books?id=aSgFMMNQ6G4C&pg=PA813&lpg=...
1: https://en.wikipedia.org/wiki/Chicago_Pile-1
2: https://en.wikipedia.org/wiki/Nuclear_fission#Discovery_of_n...
But the fact that Fermi was doing such a calculation in the first place proves that we knew in principle how a fission weapon could work, even if we didn't know "how far off [they] were". As soon as we figured out the moon was just a rock 240,000 miles away, we knew in principle we could go there, even if we didn't know how far off that would be.
By contrast, we don't know what consciousness or intelligence even is. A child could define what walking on the moon is, and Fermi was able to define a self-sustaining nuclear reaction as soon as he learned what nuclear reactions were. What even is the definition of consciousness?
You are moving goalposts. You mentioned in the first place "fission weapons" and now you take a quote about "nuclear fission reactor" which is a whole different thing.
Almost nobody really knows how developed is the state-of-the-art theory / applied technology in confidentials advances that the usual suspects may have already achieved. I.E. deepmind, openai, baidu, nsa, etc.
AGI could have already been achieved - even theoretically - somewhere, and like when Edison got to make work a light bulb, we're still using oil and not knowing anything about electricity, or light bulbs or energy distribution networks / infrastructure.
The actual current - new, mostly unimplemented yet - technology level.
Back then you wouldn't have believed if someone had said you "hey, city nights in ten years won't be dark anymore"
This is what happened when it became known nuclear weapons were a viable concept. The technology shifted power to such an extreme degree that it was impossible not to invest in it, and the delay from «likely impossible» to «done» happened too fast for most observers to notice.
Nuclear weapons required enriched uranium, and the gaseous diffusion process of the time was insanely power-hungry. Like non-negligable (>1% ?) percentage of the US's entire electrical generation power-hungry.