For I/O and memory bound processes, it makes it worse, saturating further buses that are already saturated. For regular. For CPU bound, it may help or not, depending on many factors, like cache/memory contention, nature of the operations...

Compilation can get boosts because while some threads are waiting for I/O others are crunching source files. Also the variety of computation is high enough so multiple threads don't overlap too much on functional unit usage. If you try to build from a filesystem in memory, you'll find way a less impressive speedup (if any).