You don't necessarily have to delete stuff from cold storage right away. You just need to have a process to remove deleted PII when you retrieve/rewrite your backups.

And remove it from the off site backup, like AWS, which helpfully makes copies of the backup data, and you have to remove it from all of them.

Presumably you are also required to delete it from any training data for an ML model, does that make any models trained on previous set of data illegal to use?

Not true.