Your installed base is quite large indeed! Your testing load was a drop in the bucket of our daily usage, but once you released to VS users we noticed. Should be straightforward to design something that works for this access pattern and load, now we know what you need. Typeahead package name completion would be a neat feature.

Before you start sending couple thousand QPS to any server it's generally not a bad idea to test if that server can handle that much, sometimes it is even worthwhile notifying the other team about the intended change.

Overall when you "tested" something, but it still breaks in production and requires a rollback it's usually a sign that your testing strategy isn't could use some improvement - what is the point of testing if it doesn't prevent failures from happening