In my experience it has more to do with quality and affordability. If you like your Toyota Camry, you're probably going to get another Camry. Subarus are slightly different in that they have cornered a niche (snowy mountain driving) that many owners swear by.
Subaru has the snowy country people and gays down. Nissan used to have the same type of affinity with black customers - when treating alienated customers with respect is a novelty, people demonstrate loyalty.