Arabs and Jews are both Semitic (in the sense of speaking Semitic languages and having Levantine origins), but the word “antisemitism” refers specifically to Jew hatred. This is mostly a historical quirk of 19th century Germans trying to come up with a more scientific sounding phrase than Judenhass.

(Also, to be pedantic: there are non-Semitic middle-eastern Muslims, as well as semitic middle-eastern non-Muslims/Jews. It turns out that “semitic” itself isn’t a super useful category, which is why “antisemitism” should really be read as a single lexeme rather than “hatred of all semites”.)

This has the same energy as “technically Elon Musk is African-American”.

Sometimes words have come to mean something specific that doesn’t precisely correspond to their literal components or etymology, and pretending not to understand this just impedes communication.