If you have lots of highly qualified niche traffic -- the kinds of people that advertisers actually want to reach -- along with a solid in-house ad sales team, you can make quite a bit of money off even a moderate traffic site.
If you have tons of traffic, but you know nothing about the audience, you will get very little money for your ads. Why do you think the washingtonpost.com has to run ads for home refinancing and mesothelioma ambulance chasers? Because there aren't many advertisers willing to pay a premium to target "any schmuck reading the newspaper."
Advertisers aren't stupid. CPM rates have basically been comditized unless there's something special about your site or your advertising model/targeting.