As a programmer the problem with feature toggles is this, lets say we have 1 feature toggle with on been 1 and off been 0.
For one feature that means we have
1,0 states (two states).
For two features we have
1,0/1,0 (four states).
By the time you get to 10 feature toggles you have
1111111111 (1024 possible states).
In case I wasn't clear hammering home this obvious (to us but sadly not managers usually) point, feature flags are binary and when you have 16 of them you have 65536 possible states.
Now as a programmer that frightens me because the possible paths through the system has become incredibly large for us to handle and it's a UX/UI disaster unless handled very carefully, you end up with features that interact with other features (set a do not back up flag on a file, then a different flag for always back up all files) in unpredictable ways for us and for users.
You see this complexity in things like hierarchical role based permission systems and the like.
Not sure what the solution is but I can understand why programmers and users push back on adding features (not least because as a programmer I know that doubling the complexity for 1-5% of users just seems like a poor trade off in general - there are of course specific cases where it makes sense like the 5% of users is roughly the percentage who are paying for your product etc.).