Today's computers don't spend most of their time in I/O. Average CPU runs idle most of the time. I/O does not require CPU power. DMA exists since the 80s or whenever. Software and segmentation limits such as operating systems and traversal between execution rings are not the chemical hard barrier brains have.

You're correct that our architecture isn't adequate and biggest achievements lie there. I/O is not the problem, in fact, we have faster I/O. Because our I/O is dumb. We can place massive amounts of data in a linear memory buffer. But brains use massively associated memory structures. I/O of a network packet is easy. Associating that packet with preexisting context (such as TCP connection) is not that easy, requires structures, algorithms, memory locality, threading correctness, and procedural computing steps, because we abstract the context over a series of flat data.

If you're working on a subject hard, just a random flying info about something else that concerns you might trigger "I can't think about that right now" reaction in your brain, but the information has been digested. The packet has reached the adequate layer 7 ingress buffer just like that, but you don't want to context switch to the respective application intentionally.

There is also an elephant in the room and that is the native language, which shapes the way we think and process information. Imagine a CPU receiving an automatic microcode update the same moment when you as a programmer defined an abstract TCP stack in C or assembler, so it can optimize itself to the point of being able to switching to "thinking in TCP" mode.

Pretend you have any hardware you want, today. What would you do with it? What model would you train? How do you know available hardware is the bottleneck and not model architecture?

Neuromorphic Computing is already a thing, and Intel's already developing chips (Loihi 2). But it's not as powerful as GPU's yet, and it only runs spiking neural networks.

That would be FPGA's.