Right, in general a lot of uses of mutexes are to temporarily violate invariants that are otherwise upheld while the mutex is released, something that can usually be reasoned out locally. Reasoning about cancellation at an await point is inherently non-local and so is much harder to do. (And Rust is all about scaling up local reasoning to global correctness, so async cancellation feels like a knife in the back to many practitioners.)
The generally recommended alternative is message passing/channels/"actor model" where there's a single owner of data which ensures cancellation doesn't occur -- or, at least that if cancellation happens the corresponding invalid state is torn down as well. But that has its own pitfalls, such as starvation.
This is all very unsatisfying, unfortunately.