undefined | Better HN

0 pointsjhanschoo2y ago0 comments

A basic transformer architecture performs only a bounded amount of computation per generated token, so it can never emulate a machine computing sufficiently hard problems.

0 comments

3 comments · 1 top-level

EVa5I7bHFq9mnYK2y ago· 2 in thread

Yes, because it's feed forward. It must have loops to be a Turing machine.

phkahler2y ago

It does. The output is fed back in.

jhanschooOP2y ago

It indeed does, but it must generate a token per loop, and can thereby solve some linearly complex problems, but it cannot solve harder problems.

j / k navigate · click thread line to collapse