In a perfect world I would research each comment, do basic background checkes against stated facts, think through the implications of the comment, etc. and then form an unbiased opinion. I would do this with every comment thread. After I had done this I would be uniquely qualified to form an opinion on every comment, and I would be wise because I'd read and understood all the comments. I'd also have spent most of my day doing so.
The reason comment scores, and usernames as a proxy for scores, are interesting is that it's a good way of cutting through the noise. Most people just don't have the time to read through all comments on a thread and form an opinion on each one seperately.
That's what I'm really missing about the points - I don't care how many points a certain poster or a certain comment has, but I do care greatly to have an indicator of quality so that I can skim through a thread and pick out the best comments. Particularly on long comment threads where I don't know much about the subject, or on busy days.