I initially had this same belief, but I realized I had such a limited understanding of race after reading the wikipedia article on it [0]. That we divide not based on actual genetics but visible markers makes it a social construct. And the way we've chosen to look at race politically has changed over time.

"Because the variation of physical traits is clinal and nonconcordant, anthropologists of the late 19th and early 20th centuries discovered that the more traits and the more human groups they measured, the fewer discrete differences they observed among races and the more categories they had to create to classify human beings. The number of races observed expanded to the 1930s and 1950s, and eventually anthropologists concluded that there were no discrete races.[93] Twentieth and 21st century biomedical researchers have discovered this same feature when evaluating human variation at the level of alleles and allele frequencies. Nature has not created four or five distinct, nonoverlapping genetic groups of people."

0: https://en.wikipedia.org/wiki/Race_(human_categorization)