1. By definition, they're the device with the most current running through it, and typically generating the most heat. Switched mode power supplies have efficiencies up to around 80%, linear mode power supplies closer to 50%. They need to be cooled, but typically aren't or have insufficient cooling. 2. They're typically chock-full of electrolytic capacitors. 3. Power supplies have been engaged in a race to the bottom for decades. You're almost always getting the cheapest part from the cheapest factory.

The power supply should always be the part you troubleshoot first, even if you see something else smoking. Chances are pretty good the reason the other thing is smoking is because the power supply is putting out too much voltage.