Yeah, I also mentioned how I wasn't sure how well founded that argument is.
On the other hand, especially for man pages that are not huge, you can get a good grasp and overview by just scrolling linearly through the man page. At least for me, who tends to think very visually, I then have the impression I got a better feeling for the structure and covered content of the document than if it was hierarchical. (Again, this breaks down once the document gets too large.)
Overall, that annoyed "oh gosh, I just wanted a quick scroll through the man page" feeling when encountering a man page that was just a stub for the info document probably comes from somewhere after all.