I did this with mail from the tax service addressed to the previous tenant. The result? A few days later I received the same mail in my inbox again, plus another one. Returned both of them, a few weeks later I received 4 mails.
When I started receiving more than 20 mails in one day, all from the tax service to that previous tenant, I bought a shredder and shredded the entire stack of mails. Some years later, I was still shredding mails. My magnanimity to correct for government failures only goes so far.