The goal would be to find a way to make ML extrapolate only pixels that really describe actual really present features and never imagining detail that wasn't there in the first place. Now I am no expert at the matter, but what I know of deep learning models they are really good at the latter as they basically make statistic guesses on what would be plausible.

Getting a plausible guess on what looks like a convincing answer works really well for answering a question. But the problem at hand is more like predicting the words someone said based on the first and last word in a sentence. Imagine a criminal case where the evidence is fragmented like that: I am pretty sure a LLM could give a convincing prediction here, but I am not sure how much you could rely on that prediction being reflective of what was actually said. I certainly wouldn't feel comfortable with a conviction the result of that prediction even if it was reflective of the ground truth in 90% of times.