I wasn't at all trying to say that only a few places are optimizing for bad things, but as you see all over this thread, there's a number of companies that immediately come to people's minds as bad actors when it comes to A/B testing - Google, Meta, Microsoft. There's plenty of other companies that are more ethical about it, or use it as part of rolling out general changes and collecting feedback. I know half of the time I log into the AWS Console it has some sort of "Hey, we're testing out a new upcoming UX for this page. Click here if you want to go back to the old one", which seems like a decent way for them to get feedback on the new designs while not drastically disrupting things.

Talking to people is great, but getting a representative sample is hard, and often people are bad at both understanding what they want, expressing it, or even being accurate about how they use things. I know when I was working closer to the UX side of the business before, I was constantly surprised by both what users would say they want AND by how users actually used the products.

In my mind, A/B testing is good as a sort of "final pass" to serve as broad, semi-random validation that the change you're looking to make does actually do the thing that it's intended to do. It's not great for early on when you don't really know what to measure or look for, or if the change is remotely reasonable, but it can help check for if your focus group/user panel happened to be weirdly skewed in their usage/desires.