People in the Olympics are the best athletes in the world. And I bet world class athletes need a big pool to compete in.
That gives you a pretty decent sense of the scale of an Olympic sized swimming pool without ever having seen one.
For example, I made a comment about a similar comparison in a post here on HN a while ago because their example was simply ridiculous [0]. The author in that article referenced the volume of a large body of water as being "about 240 billion Olympic sized pools". I believe that they could've used almost anything else that most people would identify as being very large as a reference but instead they chose a large swimming pool. If an item is used as a reference the scale of the reference and the object should be chosen such that the reader could quickly understand that while one may be big, the other is bigger by some easily pictured amount.
