I haven't gone through CS50 but I took the MIT equivalent online. From the perspective of someone who has done a fair bit of programming but is not a professional programmer (and doesn't do a lot these days), it was somewhat time consuming but not terribly hard for me.
However, my thought when taking it was that if I were a freshman taking this who had minimal experience with computers, I'd have been pretty lost. Not so much because I couldn't have handled individual aspects of the course but there would have just been too much new from the practicalities of a programming environment to a lot of the basic programming concepts to the more theoretical aspects.
