Non-terminating functions are still pure functions in the mathematical sense. While it does somewhat complicate the use of programs as proofs, a pure function doesn't need to have a defined value for every possible input. A better example might have been unsafePerformIO which, like "unsafe" in Rust, is meant to be used to construct a pure interface to impure code but depends on the programmer to handle it properly. The difference is that Rust doesn't require "unsafe" around all side-effects, just those that may impact memory safety.
I would agree that Option and Result types represent "a step in the direction of purity", but even a little bit of impurity nullifies referential transparency and inhibits equational reasoning.
> ... it's the aspect of purity that's salient when we're talking about whether it makes sense to regard these particular constructs as monads.
A construct is a monad if it obeys the monad laws for all well-typed inputs. In languages like Rust or JS which lack any type-level enforcement of purity the constructs are only monads under the condition that the inputs happen to be pure, not in the general case. For example, for any functor (which includes all monads) we have the law "map f . map g == map (f . g)". However, if f and g may have side effects then substituting one side for the other will interleave the effects in a different order and potentially change the result, so the monad laws are not satisfied.
