Running NSS and PAM modules in-process was IMO a historical mistake. Convenient in one aspect, rather inconvenient and impractical in many others.
If we were to redo these today, I think we'd do them out-of-process and talk to them over UNIX domain sockets or pipes.
The transitive dependencies of NSS and PAM modules can also wreak havoc with your application. There are many bugs which have arisen because of these limitations. Yes, looking at you, GnuTLS.