undefined | Better HN

0 points4bpp1mo ago0 comments

Yeah, that's what I meant with "initialised as identity and the training process did not get to change them much".

0 comments

There are explicit residual connections in a transformer block. Look up "residual connections" in Google images and you will see.

j / k navigate · click thread line to collapse

SCLeo1mo ago

There are explicit residual connections in a transformer block. Look up "residual connections" in Google images and you will see.

j / k navigate · click thread line to collapse