It is specifically running applications that always trips me up here. As a user/operator of the computer, I have been bitten by applications being too locked down for them to be useful in the past. I /think/ we have gotten better such that it is easy to have better OS behavior when it wants to restrict an application. But specifically sandboxing by default has been a source of terrible application behavior for me, in the past. Is a lot like using a shadow banned account where everything looks correct, but nothing is actually showing up. Very confusing.
Now, I think your point on restricting the libraries that are imported to a program makes a ton of sense. I'm not entirely clear where I would want the "breaker box" of what an application is allowed to do to be located, but it is crazy how much just importing some library will do in many programs.