Broadly: you seem to be implying that you think a ~0.5% fatality rate just "takes a long time to develop". Even if you posit 100% infections in the US (laughably wrong given the trajectory of outbreak, but whatever), you have to accept an IFR almost 0.2%.
Given the ~170M Americans already vaccinated, any side effect worth discussing would need to be at least as bad as killing 334000 of them. And needless no vaccine in history, maybe no widely prescribed drug of any kind, has turned up a long tail side effect analysis like that. Remember when everyone lost their shit about Vioxx? Best estimates are that 38k people died; an order of magnitude lower.