I'm sure people will be tempted to jump in here and state their case as to why certain long term effects are not possible, and why this is conspiracy theory misinformation. But that's not the point. The reality is that humans regularly fail to understand infrequent or long-term consequences of new innovations, despite the best of intentions and despite the use of the best science available at the time. This has happened repeatedly in the past - see numerous prescription drugs that were recalled, or the use of phthalates in shampoos, or asbestos, or whatever else.
In the case of J&J, consider that this is happening against a backdrop of a continuously changing story about the AstraZeneca vaccine, increasing reports of COVID vaccines affecting menstruation (https://vajenda.substack.com/p/the-covid-19-vaccine-and-mens...), and a rushed emergency use authorization process that bypasses typical methodology for approvals of vaccines. The comparison isn't between the IFR of COVID-19 and the fatality rate associated with this vaccine. It's between the IFR of COVID-19, which is known to be very low, and the unbounded potential of near-term or long-term fatality or other side effects from the vaccine.