I think the vast majority has technically learned maths and physics outside the college classroom. The teaching quality I'm getting at my university isn't the greatest (bar a few exceptions), and I find myself regularly going to Khan Academy or 3blue1brown to understand a concept better. This is learning outside the classroom!
In a lot of ways, I find universities to be assessment centres and certificate issuers. Not in all ways of course, certain aspects such as labs, coordination of group projects, someone to ask questions to get back to you instantly, are all things which cannot be replaced in any circumstance. But when it comes to the dry theoretical stuff, I'm not sure why I would go to a lecture if I know I can get a better quality of teaching somewhere else online.
Of course this is only true for a few subjects (maths and entry level physics/engineering). I unfortunately have to rely on the lectures provided for the more specialist modules I'm being assessed on (such as heat transfer and chemical engineering principles). This is mainly so that I can comply with the teacher's specific assessment criteria and the terminology/definitions/textbook/ordering they use. I may be able to find heat transfer explanations somewhere else online, and they probably would explain things better, but they wouldn't help much when it came time to use the terminology that the lecturer expects you to use during an assessment, or if you're expected to use the equation sheet that the teacher made for you and expects you to use.
