Server mode is much less likely to incur GC. Were you causing enough memory usage to force your app to actually free memory?
It will intentionally use more memory for the sake of throughput, hence why this post has all .NET program flag for it, as it's a _speed_ benchmark primarily.
I cant say for certain, but I'm pretty sure I manually GC'ed as a test and it didn't seem to help. Its been a while and the C# ended up being a temporary approach until I saw how much less memory the golang version used.
They performed almost equivalently for RPS I believe.