Sure, if you stop eating you will lose some weight and die. But, you can die of starvation while obese. In animal studies, it is possible to reduce calories and the animals won't lose weight or to add a lot of extra calories and not become obese.
Calories have something to do with it because they are one part of the equation. Calories in does equal calories out. Calories In = Calories Stored + Calories Expended. But, the mistake is in thinking that the law of thermodynamics means that the type of calories in doesn't impact the ratio of calories stored/expended -- the variables are not independent. The content of the food you eat impacts: your hunger, your energy level, fat storage rates, and your metabolic rate, all of which mean that attempting to cut or add calories may not have the expected impact depending on what the food is.