I think heat pumps make sense to use when available, but that's kind of separate from electric heat sources. If you actually have to source your heat from the power source itself, it's cheaper to get it cookies.
I have a heat pump on my house, but there's also a high-efficiency furnace and its COP is over 90% combusting gas.