In Switzerland live over 41% migrants and children of migrants (just 8.4%). So the native Swiss are not just “surrounded” but greatly diminished.
Eastern Europeans, Jews (of course), and Russians were also at times not “white.”
Maybe we should just keep expanding it. Declare everyone white. Black people are just white people with more melanin.
Then we can be racist about aliens from outer space.
This idea is mostly a modern fabrication. Various more granular ethnic biases were of course present throughout American history, but those were never conflated with racial categories: in times and places where the white vs. black racial division was relevant, the ethnic groups you're referencing were always considered "white".
And the types of discrimination that people in white ethnic groups sometimes experienced was of of a type and of a degree vastly different from that experienced by black people. They're really two very distinct phenomena, and weren't evenly distributed throughout the US -- black people in the South had the legacy of slavery, Jim Crow segregation, and other horrifying things to deal with, whereas white immigrant communities in the Northeast or Midwest never experienced anything remotely similar.
Its also why they get left behind by everyone who has.
Its an ever growing complex and unpredictable world. Sameness is not a strength in complexity theory.