You also won't be able to keep backups of this data longer than is necessary for operational restore purposes (more on that below).
The rule is that you shouldn’t keep personal data for longer than is necessary for the purpose for which it was collected.
There are five exceptions to this, one of which is:
2) for compliance with a legal obligation which requires processing by Union or Member State law to which the controller is subject or for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller.
This addresses the need to meet other regulatory requirements that you mentioned.
You'll need to keep a metadata record of what you have deleted.
In the event that you have to restore data from a backup for operational purposes, you need to cross reference it to the record of deletions that occurred since the backup was created to ensure that any such data is either not restored, or is immediately deleted again.
This is only a fraction of an organization's obligations under GDPR, being those most directly relevant to your question.
Disclosure: I work for a company that provide solutions in this space.