9 times out of ten it's because sites use cloaking and serve up all of the contents to search bots, but then paywall out end users, so it's kind of a hoisted by their own petard kind of situation.
And, I mean, people can choose to not follow those links. To the rest of us they're often very welcome, and we aren't subscribing to every random site for the once in a millennia worthwhile article.