I'm not willing to assume that a more diverse project and larger potential contributor pool will automatically produce better code. Not all contributors are of equal value. Some contributors are of net-negative value, particularly those like Kay who persist in such. It is highly desirable to limit the potential contributor pool to as few of those individuals as possible.
Size is not the sole meaningful measure of a contributor pool. Quality is equally - and often more - important.
When dealing with matters tangential to a core purpose, sometimes the ends do justify the means. When was the last time you thanked an open source developer for stopping development on features you needed (perhaps permanently so) in order to encourage the surrounding community to be nicer to one another?