Of course its controversial either way. Let's assume for our purposes that DEI is purely about being race, sex, gender, sexuality, age blind (which we can absolutely argue about separately). It would absolutely be controversial because it's replacing a system that previously wasn't blind to those characteristics and therefore has a large constituency of people it favoured. If I'm a rich white kid from a good family who went to a top school who gets into an Internship at Goldman because my Dad is golfing buddies with a Partner there then of course I would be opposed to DEI. And guess what? Rich white kids from good families and top schools have quite a lot of political capital.

And that's only those who directly materially lose out. Implicit in DEI is a suggestion that the American system is not a meritocracy, and if you accept that claim you are attacking the identity of a lot of powerful people who genuinely believe they got to where they are through unique skills and effort and not because they had any sort of advantage.