You can supply the model with a list of facts already, that’s not the problem. Within the context window the model is able to learn and generalize new information.

Fine tuning is very unintelligent in the sense that it doesn’t take the context of the training samples into account. It’s a dumb optimizer that’s trying to minimize next token loss. Gradient descent is not beholden to the behaviors you taught in the instruct fine tune step.