The faster-than-light neutrino experiment similarly went "we've tried to account for everything we can think of and still can't figure it out" when they published. It turned out to be a measurement error.
The same may be true here, and I think it's the most likely explanation.
I'd be interested in whether the same model can be trained to predict patient wealth, hair color, style of clothing, religion, etc. from the same x-ray data sets.
I am dismayed by your example that runs counter to the modern science.
While "faster than light neutrino" was highly unexpected and rather suspect from the start, the "bone geometry differs slightly between ethnic groups" is well established among the anthropologists of humans. There are also parallels in wider biology of animals - mentioning that to underscore it's as scientifically expected, and not merely construed for humans alone.
The question here was how exactly is AI detecting it this well from chest X-rays; the question centered around AI and possibly if it would unexpectedly influence the medical processes - rather than around the bone geometry itself.
What if it was an 8x8 grayscale photo of their face? We wouldn't be particularly surprised that it can guess race. The fact that we struggle to detect patterns in the data doesn't mean they don't exist.
That's 1/4 the size of the texture of a Minecraft block. An AI could easily be trained to identify the block type of a Minecraft block based on an 8x8 subsample of one of its textures, and there's no reason something similar couldn't happen with biological race. Unless you a priori assume all racial differences are socially constructed and/or only skin deep.
Forgive me if I don't consider "you can tell someone's race from physical features" to be quite as extraordinary a result as "particles can travel faster than light".
