undefined | Better HN

0 pointssolenoid09372mo ago0 comments

They have not, every successful pre-train as of late has had performance increases greater than what the scaling laws predict.

0 comments

1 comments · 1 top-level

0x3f2mo ago

Those gains are arch based, data quality based, etc. Scaling laws only relate to data volume and compute, holding other factors constant.

j / k navigate · click thread line to collapse