My niece was not fast enough to be invited to the trials this year (missed by .03 seconds in her favorite event), but her time would have put her into the second heat in Paris. She's the ~150th fastest person in the USA, but would have come in ~25th place in the Olympics. It's the same situation in China, Australia, Canada, UK, etc.
Most countries only have a small handful of elite swimmers. The power nations can each provide 20-50 swimmers fast enough to get to the semi-finals in every single event. They're analyzing and optimizing for everything. This is why most of the elite swimmers not from these countries go and train in the powerhouse countries. And why the powerhouse countries don't care that they do. I'd bet that 90% of all the medal winners this year do their training in 5 countries.