In practice, this is pretty much the case. State schools (e.g. the University of Virginia, University of California-Berkeley, Georgia Tech, etc.) are often quite good in terms of the education provided, and offer lower tuition rates to in-state residents. Forbes and U.S. News & World Report maintain rankings of the top 100 or so colleges in the United States; my school was ranked in the 60-70 range for computer science. Having spent a number of years at big (and small) companies in the Bay Area, I feel it more than adequately prepared me for the workplace.
Maybe the sense of “a few elite schools, and lots of bad ones” is reinforced by the popular media? Hollywood tends to focus on just the Ivies and MIT.