I'm sorry, but I don't think that's what they were saying at all. To me, it sounded like they were saying that when a developer works alone at nights or on the weekend, they cannot ask other members of the team, or the person who originally filed the ticket, any questions. Thus, if the ticket is vague, or if they have a question about a part of the codebase they are less familiar with, they have to spend a lot of time trying to divine the answers themselves when they could just walk over or message their colleague for the answer on a normal work day. Spending three hours rederiving trivial knowledge that other members of your team already know (such as which package something is in, or the exact set of inputs that reliably reproduce a bug), IS spending time on the wrong things.
I agree that a manager shouldn't micromanage to the point that they believe developers cannot be productive without their manager physically present, but that is not what they were talking about. The comment isn't about the developer needing their manager's insight or guidance, but about the developer needing the expertise of the rest of the team, which they will not have access to at nights or on the weekend.