I think the point is to train your ability to focus on things that you don't naturally focus on. If you like thinking about math and you can easily focus on it without effort, presumably it doesn't have the same training effect. I think the idea of mindfulness is to learn to build up "reflexes" in your mind that cause you to notice certain thoughts at the conscious level. Focusing on your breathing is something so boring that random thoughts will continually enter your mind. Sometimes it will take a significant period of time to consciously notice that you're no longer focusing on breathing. Over time you train yourself to recognize those thoughts as soon as the enter your mind. In other words you train yourself to lift unconscious thoughts to conscious meta thoughts, i.e. feeling angry v.s. thinking "hey, I'm feeling angry".

For example if somebody does something annoying, you might reflexively think or say something bad to that person. The idea of mindfulness is to train your mind to process thoughts on a conscious level before acting on them. You will have the conscious thought "oh, I feel annoyed by this" and then be able to make a conscious decision about your reaction, instead of a reflexive one. Or instead of randomly browsing the web, you consciously notice "hey, I'm browsing the web, lets get back to work" ;) I'm certainly not an expert on the subject, but this is my understanding of it. If you've never tried it and want to get a clearer understanding, try it now. Try focusing on your breath for 2 minutes (or heartbeat, or the number 5, or whatever, as long as it's the same simple boring thing over those 2 minutes).