Sure. Specific energy (or energy per unit mass) between different types of materials makes a huge difference. For example (Source here[0]):
Material Type of generation Specific energy (MJ/Kg)
Hydrogen Fusion 639,780,320
Coal Oxidation 24.0-35.0
Note the specific energy of a Kg of burned coal compared with a Kg of fused hydrogen. Fused hydrogen generates roughly 200,000 times the energy per unit mass than burning coal.You're right. Cost and availability play into this as well. There are estimated to be ~1.06 trillion tons of coal on earth[1], hydrogen is the most abundant element in the universe and even makes up a significant amount of the mass of coal.
Burning hydrogen/hydrocarbons is, compared to fusing hydrogen, an incredibly inefficient process.
I'd say that being able to generate 200,000 times the energy per unit mass is an important consideration.
As for availability, hydrogen is more abundant and cheaper to produce (unless you have petatons of plant matter, the right conditions and a few tens of millions of years at no cost to you) than any fossil fuels. Or just about anything else.
[0] https://en.wikipedia.org/wiki/Energy_density#List_of_materia...
[1] https://www.worldcoal.org/coal-facts/what-is-coal-where-is-i...
Edit: Clarified availability.