If you have a lot of small requests, with only few requests active at the same time, but many requests per second overall, with each making a few allocations, you will have a small live heap size, while quickly reaching the threshold for another GC.
This way you get a lot of GC runs. Latency isn't affected too much because Go is quite good at keeping the stop-the-world's short. You might have interleaving application/stop-the-world in a 50/50 ratio of computation time (that's something you can diagnose very easily with go tool trace btw).
Having a higher GOGC threshold might help a lot there, since it will make stop-the-world's less frequent, while keeping their duration mostly unchanged (as that scales proportionally to live heap size).
That's obviously just a guess based on the limited info I have though.
