Pretraining Under Infinite Compute (opens in new tab)

(arxiv.org)

3 pointsjedharris5mo ago1 comments

1 comments

"Our results show that simple algorithmic improvements can enable significantly more data-efficient pre-training in a compute-rich future."

j / k navigate · click thread line to collapse

(arxiv.org)

3 pointsjedharris5mo ago1 comments

"Our results show that simple algorithmic improvements can enable significantly more data-efficient pre-training in a compute-rich future."

j / k navigate · click thread line to collapse