It's not the highway that gets you (typically). It's all the stupid little roads that are built to substantially lesser standards. Steeper grades, tighter curve, hard 90 junctions, etc, etc.

You carry just a hair too much speed into a curve because you're anticipating not wanting to have to use any gas pedal on the rise just beyond and you wind up in the ditch. Or you go wide into a curb because you took a less optimal gap in traffic at a ~5 roll when taking a left turn because that way makes you less likely to get T-boned than coming to a stop and trying to find an even bigger gap in traffic. Or you slide backwards down some stupid driveway or bump something in a parking lot (ask any delivery, parking lots and driveways are the worst).