Yeah, and we're slowly slowly moving towards that model. One barrier is that some safe languages, like Rust, support far fewer target platforms than the Linux kernel does. There are C compilers for everything, but only a few Rust backends.
Something similar to Wuffs[0] (posted on HN very recently), which compiles down to C, might be a good compromise between portability and safe languages. (There may be some contorted way to have Rust emit C, too.)
[0]: https://github.com/google/wuffs