Using rust's definition of unsafe which is roughly "can cause undefined behaviour" then it seems to me isolating use of these features isn't possible. What is C without:
* Dereferencing pointers * Array access * Incrementing signed integers
You can do all of the above without invoking UB, but you can't separate the features in C that can cause UB from the ones that can't.