I've "unfriended" recruiters I've mistakenly added in the past.
However, if you're looking for talent, it is helpful to have such "connectors" in your network - without those recruiters, when I was doing a check on my network I didn't see some of the people I was trying to hire - this was only the case when I had switched jobs/domains.