Traffic and the comparative size of automobiles, compared to buses, means buses get stuck in congestion and delayed more readily in shared roadways. Buses almost always have to compromise with the surrounding vehicles and yeild right-of-way to others, which pretty much means they go slower, and you suffer a longer ride.
Waiting for a bus usually means waiting outdoors. There are tons of reasons why bus stops are no fun at all.