No, it's not the power, either, as low resistant paths can be extremely dangerous (such as wet skin). Really, it's a 2 dimensional space that makes for a less catchy aphorism. "It's not the voltage that kills you, it's the combination of voltage and conductivity".
Honestly I kinda hate that saying. It's like "it's not the fall that kills you, it's the sudden decelleration on impact." For most situations people encounter, it's high voltage sources with more than enough power which are dangerous.