You don’t want to integrate it into the server, because that limits choice. There are a lot of features that you could put into a clipboard manager that not everyone would want, and you don’t want to force everyone to just have the least common denominator of those features.
For example, one commenter pointed out that he uses one that keeps a permanent database of everything he has ever copied. Maybe you want to sync your clipboard between computers, while I just want mine to keep the last 10 clipboards and throw away the rest. (Actually I use Emacs which has it’s own internal kill-ring, so I don’t use a clipboard manager.)