Like one thing I learned some time ago: swap-out in itself is not a bad thing. swap-out on it's own means the kernel is pushing memory pages it currently doesn't need to disk. It does this to prepare for a low-memory situation so if push comes to shove and it has to move pages to disk, some pages are already written to disk. And if the page is dirtied later on before needing to swap it back in, alright, we wasted some iops. Oh no. This occurs quite a bit for example for long-running processes with rarely used code paths, or with processes that do something once a day or so.

swap-in on the other hand is nasty for the latency of processes. Which, again, may or may not be something to care about. If a once-a-day monitoring script starts a few milliseconds slower because data has to be swapped in... so what?

It just becomes an issue if the system starts trashing and rapidly cycling pages in and out of swap. But in such a situation, the system would start randomly killing services without swap, which is also not entirely conductive to a properly working system. Especially because it'll start killing stuff using a lot of memory... which, on a server, tends to be the thing you want running.