I get the tradeoff, the problem is when you have 90 projects that customize the library for 90 different needs, and then you want to update those libraries. At
that point, you've basically made 90 Modernizr forks. Does it make more sense for you to maintain 90 forks of a library whose codebase you aren't developing and try to keep them current, or to bake-in the parts of Modernizr (with its edge-cases included) right into your codebase for each project, and then only have to maintain your own codebase, instead of your codebase + a library fork.
If you're using the majority of Modernizr tests it makes sense to use the full library, but for the most part me and the other devs I've talked to usually drop the whole library in for ease of maintenance, but mainly use it for one feature alone: touchscreen detection.
Lately I've tried putting some of my HTML on a diet by replacing my need for Modernizr for the purpose of touschreen detection with this snippet:
if(('ontouchstart' in window)||(navigator.msMaxTouchPoints>0)){
// code for touchscreens here…
}
This keeps the test and the result together in the code, and eliminates the need for a library, which can also be an additional single-point-of-failure outside of your control, especially if hosted by a CDN.