The biggest thing I've hit is linking issues with Rust -sys crates (ie. C/C++ libraries wrapped). There's some very strange behavior if you include gcc and clang (for example) into the same environment.
My biggest issue with Nix tbh is it adds an annoying step to every random repo I want to clone and build.