Downstream destruction is of concern because that entire region depends on flood control. The habitability and agricultural production of the corridor between Oroville and Sacramento relies upon, and was created by, the flood control of Plumas, Yolo, Butte, etc counties. There are a number of canals, diversions, and reservoirs both above and below the Oroville to further control flow the dam as it reaches our rivers. However, most of the re-routing is also man-made and thus untested for such an event.
Dirt hillsides are not designed to have 100,000 cubic feet of water rolling down them. They tend to erode extremely quickly and can quickly turn into uncontrolable discharges if the lip erodes.
For the main service spillway, the main concern is that the flow of water will cause erosion damage upstream of the existing damaged site[1]. If the damage to the concrete continues up the spillway to the top, it could render the spillway inoperable.
For the emergency spillway, the main concern is that a continued flow of water would erode the soil off the hill to the point that the hill would no longer support the spillway (the concrete lip at the top of the hill). If this happened, the spillway would fail. The effect wouldn't be as severe as if the dam failed (because there would still be a large hill in between the water and where it wants to flow), but the erosion along the path of the water would get out of control pretty quickly.
