First, don't discount the longer compilation times. After learning Lisp, I became rather addicted to the rapid turn-around time of compiling code at the level of individual functions. I can tolerate C because, with a relatively clean build setup, it compiles quickly. g++ on my project is between 10 and 20 times slower than gcc. This means that, as I make minor changes, I'm more likely to have to wait a while for a build to finish. This means I'm more likely to slack off in between builds. This kills (my) productivity horribly.
Second, auxiliary tool support on Unix leaves a lot to be desired. I sprinkle my code with templates in places where they promote readability, and I use STL containers. gdb, at least the version I'm using, does not support displaying data in STL containers. This means that when I have to track down a problem with data in a map, gdb can't help me. I have run into nasty problems for which I had to write custom print-this-templated-structure-to-log code. In addition, g++ error messages which involve templates absolutely suck. Luckily, I can quickly figure out what I need to change just by looking at the affected line.
After thinking through these issues, I concluded that C++ wasted more of my time than it saved. YMMV, of course.
There are plenty of gdb macros that support stl containers even 18 months ago.
LLVM is also written in C++ for good reasons.
A lot of larger C++ projects use "pimpl" for this reason (http://en.wikipedia.org/wiki/Opaque_pointer). Also in professional projects with large code bases that I've worked on in the past we've always had a distributed compilation environment (distcc, icecream, etc.)
(Note that I've only played around with Factor for a few hours.)