My guess to where we are talking past each other is that I am asserting you can go from the work to the time. So, you should ask for that. You cannot go from the time to the work. (Again, both are assertions.) You seem to be saying you should just ask for the time, and only dig on the work if you aren't satisfied with the time.
If you are able to turn every estimation session into a series of back and forths where it is "how long?" followed by "why?" if you aren't satisfied, then I feel we are essentially agreeing. Whether you are asking for it or not, you want them to estimate the work required and to summarize it into a time.
And to be perfectly clear, going two people removed from the work, this is required. Similarly, getting 3 people removed from the work, the relevant question will not be "how long" but "how many dollars?"
Similarly, it would be nice to think everyone will eventually need the skill of estimating the value of a feature or product. Because, at the end of the day, that is what is most important.
However, I'm assuming anyone asking someone specifically for an estimate is one of their managers. And they should have more familiarity with what they are asking their colleagues to estimate. And I'm also asserting each of these skills is not trivial. And that they build on each other.