Some disjointed thoughts:
* If a consumer favors their Amex over their other cards, then they favor retailers who accept Amex. Thus the higher fee is justified with more repeat business.
* My mother's company has one line of credit: her Amex. If her Amex is refused, she has to pay with her personal credit or debit card, then go through the rigmarole of a personal refund. In corporate America, Amex is the standard due to its ease of internal accounting and external reputation. With the former, the better reporting of transaction information justifies its use in the accounting department. With the latter . . .
* Amex has established itself sufficiently as a luxury brand,
e.g. if you take your client out to lunch, it could be out-of-place to pay with anything but an Amex.
Furthermore, if your business does not accept Amex, it alienates luxury-seeking consumers
e.g. if a Michelin-starred restaurant refused an Amex card, it would experience a more precipitous drop in return rate than if a neighborhood diner refused Amex.
* "the business will only lose revenue from customers whose only form of payment is Amex" -> This is true on a per-transaction basis, but I defined "quality" to include repeat business and, per my first above point about business, restricting consumers could discourage repeat business.