Where does it say that? I tried to find more detail. Thanks.
https://github.com/DGoettlich/history-llms/blob/main/ranke-4...
"To keep training expenses down, we train one checkpoint on data up to 1900, then continuously pretrain further checkpoints on 20B tokens of data 1900-${cutoff}$. "
