Calling people who are against illegal immigration "racist" just makes it worse.
A majority of people are fine with legal migration, a supermajority of people think illegal immigrants should get deported. So no, the issue most see isn't that they don't like Spanish, the issue is that they are here illegally.