That's simply not correct. The literature abounds with all sorts of correlates with race, like propensity for sickle cell anemia, vitamin D deficiencies, susceptibility to alcohol, and morphological differences. These are just as empirically justified as any classification of species, and just as with species, not all of those properties need apply to every single member in that category.
So to the extent that we find "species" a meaningful category when applied correctly, then we should also find "race" a meaningful category when applied correctly. The key in both scenarios is to apply them correctly, and we should abandon them when we find more precise metrics.
That said, you are correct that there are also numerous cultural and social properties that are sometimes lumped in with race in a manner that we don't see with species, mainly because "species" hasn't been politicized. That doesn't imply that there's nothing "there" once you tune out that baggage.