"I think it’s partially unresolved cognitive dissonance between rhetoric and reality..."I'm almost certain you're right, cognitive dissonance is a key disrupting factor in many of our lives. Not only is there so much more information for people to process these days but much of it conflicted in ways that doesn't allow one to come to definitive conclusions.
As you say, 'happiness is a measure of the delta between expectation and reality' and in this regard we've seen better outcomes in societies where expectations are low.
I remain pessimistic about us being able to resolve this in the modern world as everything around us conspires to make one's expectations as high as possible. Essentially, our whole economic system is geared to making us believe that the more stuff and junk we accumulate the happier we'll be.
I see no easy or practical way of breaking this circular problem. Year by year, advertising and commercialism gets more intense and the problem only gets worse - moreover, technologies such as the internet have only made matters worse.
Promoting these high expectations is now of itself a huge part of the economy that if undone would alone cause hardships. One only has to look at Google and Amazon to realize that.
It's really all very depressing.