For reference, I use Nix to manage three different machines, always via home-manager and nix-darwin (I left NixOS awhile ago and haven't looked back). I don't think it's "inevitable" that you'll hit the difficulty wall, but certainly likely.

As an example, I was recently playing with Pyinfra which is like a pure Python version of Ansible. It turns out that one of the dependencies uses an archaic version of setuptools and the package owner had inserted some _very_ hacky code that ended up breaking on two of my systems. Now I'm relatively experienced with Nix, so it took me a few hours to track down, but it would have been days if not impossible for a beginner.

Nowadays I package brew along with my machines and as soon as something smells funky in Nix I just manage it with brew. Much more peaceful.