I appreciate your overall point, but it's probably also worth considering the concentration or dilution of that waste. When you mention eating or drinking the atoms, or living next to a pile of them, the radiation dose will obviously depend both on the atoms' half-lives and on the overall number of atoms.

There's an extra Avogadro factor of around 23 orders of magnitude when going between atoms and grams, so those atoms really add up -- if they are highly pure and concentrated, at least.