Imagine you have 100 documents in a database and you "query" the documents and return 20 candidate results.
Similarity gave you 20 results but Re-ranking sorted those results further providing relevance.
That 4K is per document.
Edit: With sorted relevance, you can drop the lower scoring documents according to the model's confidence that the information in the subset is adequate to answer the query.
