Then you reduce the credibility of spam.com and possibly any links to whoever setup that domain etc.
Not whatever IP was used.
I'm not being idealistic when I say that I should be able to send an email over internet and have it delivered without someone banning me for reasons beyond my control.