This is mostly a concern with the Rust stdlib, though. And it's in principle fixable, by writing new varieties of those stdlib functions that take raw-pointer or &UnsafeCell<...> arguments, and delegating the "safe" varieties to those.
There's compiler-level traits like `Iterator` and `Future` which enforce references. If wanting to do intrusive pointers into them, one risks creating overlapping references: https://github.com/tokio-rs/tokio/issues/3399