The poor may not be buying Rolex watches, but you can be sure that there are a lot of kids out there lusting after $400 basketball shoes. And you might be surprised at what counts as a luxury sometimes - liquor, steak, and other goods that even poor people can sometimes afford.
If you start taxing luxury goods, all you do is put creature comforts further out of the reach of the majority. That is regressive.