When did they agree on $X? There's no contract between the drivers and Uber stating they they will work for a given amount.
The app sets rates based on supply and demand. Uber offers them $X. The drivers decide they don't want to work for less than $Y. If demand is sufficient, Uber eventually offers them $Y.
Surge pricing is basically a negotiation between company and driver. Just because the drivers have found a case in which they have extra leverage doesn't make it wrong or fraudulent.