They’re not actually smart enough to keep track of what every other team is doing, at any point in time.
FOSS works because there’s usually only at most a few dozen balls in the air simultaneously, so someone who believes they can keep track of balls in the air gets by without looking ridiculous.
But that’s just not viable when there are hundreds or thousands of balls in the air simultaneously… their eye muscles literally couldn’t move and focus fast enough.