A simple thing you can do is publicly state, "we encourage women and other minorities to apply". And that's all. You encourage them. You don't have to give them any preferential treatment to pad your numbers (and I don't think anyone ever really does that, but a lot of people seem to be afraid that it happens). You just have to explicitly direct your invitation to them. That's enough to increase your hiring pool and give your company culture a nudge in the right direction. If you object to the alleged benefits of having a diverse staff, I can't imagine you would object to attracting more people to apply who otherwise wouldn't.
A further point I want to emphasise: what is merit? Who decides what is merit? If you think being nice to others has no merit (or is not "good performance"), then you may end up hiring toxic employees like Fowler's sexual harasser and then keep him around because of your merit metric. This will end up costing you good employees like Fowler.