It's not generally that bad unless you want the latest and greatest. For example, on Debian, you can easily install Perl modules from APT instead of CPAN (which I've found temperamental in practise). The only trouble is that the modules are often older than those you can get through the dedicated PM channel (since obviously the maintainers have to package up their release in line with Debian standards).
Package management is still a much better idea than manual dependency installation - even NuGet for .Net is an improvement as it supports automatic restoration of dependencies at build time. However, as we are immersed in so many different platforms and technologies, it is extremely difficult to have a one-size-fits-all PM. Linux PMs are probably the closest.