He is successful at sports to such a degree that his upbringing is clearly not a complete barrier to success. Yet none of the students at his high school are proficient in english or math. This indicates that it's probably not the students' fault.
I'm guessing if you went and talked with the parents and peers at those schools that they would place high value on athletics and low value on academics.