Yeah, sometimes I use that for a site that is otherwise well formatted but the font is just small. For sites I visit often I just save that as a site specific preference so I do agree with you in some cases.
However, with reader mode it goes straight to my preferred font size, has nice margins, and basically everything is formatted in a very readable way. It still reflows and changes width with the window, but the rest of the formatting is better as well. I really hate text that goes right to the edges of the windows with no margins. Just a personal preference, but for me the quickest path to dramatically improved readability on some sites is reader mode.