If you asked GPT to write a program that multiplies 32 and 64 it could do it.
Since the thing is a computer, why can’t it answer queries by writing and executing a program? Does a transformer-based AI have to be 100% transformer?
Because the optimization logic of tweaking the gpt transforms via training data and nudging weights and biases to achieve token prediction is not a path to utilizing the full potential of the gpt transforms.
This is simply not how transformer models work. Perhaps overly simplified, it maps an input string to an output string by pushing it through a neural network. But enabling it to access databases, APIs and to write and execute programs on accellerators would indeed significantly increase its capabilities.
