It's not a pointer. `std::optional<T>` is a class that directly contains a `T` (within a union) and happens to offer `operator*` and `operator->` providing an API similar to a smart pointer. This does not make it a pointer (there's nothing else it could be pointing to).
Also, in an ideal world, stdlib implementations would support error-checking for `std::optional::operator*`. "Undefined behavior" just means the standard doesn't specify what should happen. Both "let's use it for unsafe optimizations" and "trigger assertion failures" are valid implementions of undefined behavior, and a good implemention should allow the user to make this choice.
For gcc, see -D_GLIBCXX_ASSERTIONS. Though I'll say that it's unfortunate that C++ implementations tend to default to "unsafe optimizations" and that the opt-in to safety is not standardized.