You spend most of the time waiting for the actual resolve to complete, not for the "transfer" time between you and VPS.
Source: I'm running Unbound on my notebook, I'm actually queried the stats for some heated discussion on reddit.
For example my current stats_noreset:
histogram.000000.000512.to.000000.001024=17
histogram.000000.001024.to.000000.002048=33
histogram.000000.002048.to.000000.004096=251
histogram.000000.004096.to.000000.008192=509
histogram.000000.008192.to.000000.016384=1161
histogram.000000.016384.to.000000.032768=1891
histogram.000000.032768.to.000000.065536=2611
histogram.000000.065536.to.000000.131072=3197
histogram.000000.131072.to.000000.262144=2502
histogram.000000.262144.to.000000.524288=1547
histogram.000000.524288.to.000001.000000=857
histogram.000001.000000.to.000002.000000=121
histogram.000002.000000.to.000004.000000=70
histogram.000004.000000.to.000008.000000=22
histogram.000008.000000.to.000016.000000=441
histogram.000016.000000.to.000032.000000=80
As you can see most of queries are completed in a way below 500ms. Adding another 20-40ms on top that doesn't change anything, because caching is a thing and with Unbound you can even ask to actually refresh the expiring records, so you would be served a fresh one from the cache every time, though I never bothered with it, it works fine even without it.