That’s what I have been up to as well. I just think we’re so traumatized from the past 15 years of software engineering (kidding) that we’re stuck thinking “hey, let’s add memcached, that solved many problems like this before!”
Natural language is weird. It’s linear in the sense that you can’t take back what you said.
However, subjects and verbs and parentheticals and objects are like electron clouds. We only know statistically where it’s going to be. And damn, you’re reading this now, somehow, anyway.
Seems like the transformer model really is a new technology. But it’s just that we can actually run useful NLP neural networks now, which has opened the floodgate of innovation in various subfields.
(Personally bothers me because absolutely no one took it seriously when I was starting my career!
Lol. I had a meeting with my AI professor where they told me ANN’s were nonsense even if we had computers 1000x more powerful. Prune the search tree! That was not very long ago).
