I totally understand that as well. However, wouldn't the people with the most expendable money also most likely have the newest technology. Therefore the customers you are probably going to have the least issues with, would most likely have newer hardware anyway.
I understand you leave money on the table, but you would spend less on support and less money trying to get your website to be 200ms faster to load. This is speaking from an Indie Hacker perspective. This could be potentially worth millions of dollars for a bigger business.