To succeed this well in math, you can't just do better probabilistic generation, you need verifiable search.

You need to verify what you're doing, detect when you make a mistake, and backtrack to try a different approach.