Literally they learn by both, training process is filled with "exercises" for the weights to be shaped by, not just "seeing" tokens :) At least the typical LLM that outputs tokens have to actually "practice" doing so.