The issue there isn't the IRS though, it's the tax code. The IRS is absolutely a department that needs to be staffed to maximize their revenue.
If you have an easy to navigate tax-code that doesn't require a doctoral understanding of economics, then there's not a problem for the everyday person (TBH you can also say the same thing about the law in general).
>>“Anyone opposing more police officers and equipment is breaking the law” is similar to your argument.
Here's a better statement: I believe the people who are most actively working and spending money to depower the IRS are those who seek the most to gain by hiding their financial doings. They seek to deregulate industry as a whole in the name of corporate gains while simultaneously infiltrating various branches of the government to further this agenda. These people are doing so in an (successful) effort to push more wealth from the majority into the hands of a minority. I believe this is bad for society as a whole.