We have been doing this for a little while now at
http://www.15seconds.me/. It's a little bit simpler than what Partsie is doing in that we only allow an "up" or "down" vote.
The viewer ratings below the video here: http://www.15seconds.me/404211799 can be used to scrub through the video and get to the parts that the audience enjoyed the most. We only allow people to vote on the videos at the time they are broadcast though.
Great to see others are thinking along the same lines that we are. I believe that there is a big need to attach as much rich meta data as possible to videos.
