So is 'the right to be forgotten' under GDPR. As such, Tesla, OpenAI, etc. cannot legally train their models on EU customer data because we can at any point request a hard delete, and they wouldn't be able to comply without removing it all from the dataset and retraining.
In reality though, I assume they still do it and merely pretend they comply with it.