I'm not sure about "plummet" at least until you get materially more expensive than taxis. But you're right that decreases in demand mean not only fewer fares but also a less viable service at the margins.

When I've looked, it's appeared as if Uber is only marginally viable around where my house is. Decrease the number of riders and I could imagine the number of drivers dropping further to the point where it's not really a usable service.