Take drones for example. The government got really good at those because they made them jet-powered (lol) and blew a bunch of money on server-grade FPGA’s in each one of them.

You can’t really just buy a lot of GPUs to make an LLM work, you need iterative development of architecture and training methods.

Like maybe the government invented self-attention before 2017, but if they didn’t, then the constraint is training time, and the government has the same number of seconds as the rest of us.

Even the Manhattan project had nuclear research going on in public universities at the time.

Nothing of the sort here for the attention mechanism which underpins LLMs we know today.

Fundamental research isn't something you just throw money at and acquire. All we had back then were cleverbot and other expert systems.

If today's hardware was available 20 yrs ago, this would've been possible just like the moon landing could've been faked if it took place 20+ yrs later. The technology wasn't available at the time (GPUs in this case, and generally no experience in doing such advanced trick techniques for movies back then)

These models are having such a strong effect now because we've finally got the hardware to run them

Because the hardware has not existed.

This said by accident I've seen hardware that was brought to a testing company by federal marshals that was massively parallel custom hardware that was likely for signal processing a lot of channels at once. So there is plenty of custom hardware out there, but these items have not been produced at the scale needed (from what anyone can tell) and, again from what we can tell, they don't have the general processing capability that GPU/TPU driven LLMs have.

They are not deep learning/neural nets.

Also fun fact as a pedant tax: Symantec is so named because they started out as transcription software, hit a wall, and pivoted to security SW.