> (side note, aren't middle-eastern Muslims generally considered Semites as in descendants of Shem?)
Arabs and Jews are both Semitic (in the sense of speaking Semitic languages and having Levantine origins), but the word “antisemitism” refers specifically to Jew hatred. This is mostly a historical quirk of 19th century Germans trying to come up with a more scientific sounding phrase than Judenhass.
(Also, to be pedantic: there are non-Semitic middle-eastern Muslims, as well as semitic middle-eastern non-Muslims/Jews. It turns out that “semitic” itself isn’t a super useful category, which is why “antisemitism” should really be read as a single lexeme rather than “hatred of all semites”.)