It does help with browser quirks, but if you aren't supporting <IE10, then you really don't need it. I've made it a point to pull out jQuery from every project I have. Nowadays, I can test in Chrome as I develop and then test any other browsers at the very end and I rarely have any browser specific bugs. Crazy how far web dev has come.
You do really need it, at least for me. I'd rather write $ sign calls instead of getElementByID or querySelector just to start with. The sizzle library in jQuery is so advanced, that you can't do complex queries on the dom yet with today's browser.
There are small libraries that help with making native DOM operations more jQuery-like. We opted just to write our own since we didn't need a good portion or even the smaller libraries.
