With the caveman example, you were making it too easy on yourself, because it is not hard to make improvements from that situation and call it a golden age. Comparing how the top billion live now versus the bottom billion is a fairer comparison, and yeah so is looking at the last 100-200 years. Creature comforts are nice and make life luxurious, but there are people in the bottom billion that are happier than people in the top billion without as much.
You are right to point out that there are people who are going to be miserable despite modern advantages; this was in fact my point. The things that actually make people happy have to do with good health, social integration, exposure to nature, love and compassion, and having a sense of mastery over one's life outcomes.
The issue is that our civilization, through the technologies it's chosen to accentuate, makes it harder than it has to be achieve all of the things that actually give happiness. And using more technology to solve problems caused by technology, rather than moderating the enthusiasm of adding and layering technology because technology is just good, exacerbates these issues further. It's a problem because it's unacknowledged irony.
For instance, industrial agriculture keeps more people calorically satiated, but has lead to an epidemic of diabetes; hyperspecialization required to maintain complex technologies, means people have smaller families later, and move far apart from one another. Nature is a victim of technology, since industrialization and urbanization require a lot of space, and produce waste which disrupts ecosystems and eventually our own bodies.
(Mastery is more difficult and subjective, but it's affected partially by feeling like you own the outcomes of your own labour, and addictions to comfort end up displacing the pursuit of mastery, and there are ways in which e.g. market volatility and income inequality (things modernity has exacerbated) can put people in a position where they don't have control of their life at all.)
Some technological solutions to these technological problems work. Before COVID we could fly to meet our families, negating geographical distance. Some just add complexity for no good reason. Fitness apps and calorie counting are there to make up for the difficulties produced by mass industrial agriculture that is not in anyone's control, because addictions are profitable.