Understanding that is useful when comparing products using reviews.
For a time, Target used Amazon's website code. Here is what they found. People leave reviews of books that they like. People only bother reviewing microwave machines when they didn't work.
This meant that Amazon's code didn't work out as well for Target as it did for Amazon...
On the other hand, you can't trust expert reviews either. The experts spend too little time with each product to form an actual opinion. They often form their opinions based on some very shallow factors. A canonical example of this is mobile phone reviews by sites like Engadget or Gizmodo. They review so many phones that they can spend only a few days with each.
Edit: one of the worst things for this is reviews of hard drives. While there certainly are hard numbers to go off, there are also lots of opinions: "Well I've owned 6 Seagate drives over the past 4 years, and they all ended up dead. Seagate products are crap!" That does not mean that all Seagate drives are crap, maybe just laptop ones. Or maybe your controller is bad and you keep attaching good drives to it. Or maybe you keep sticking these drives into a case with poor ventilation. On top of that your sample size is too small.
Start with the premise that the person you most want to give you advice is someone who is passionate about a topic or at least has done a lot research on it. If you ask them for advice, first they'll give you an overview, which hopefully includes what attributes you should be thinking about. Then they'll recommend a small number of models to consider, based on what you seem to be interested in. If you're doing this with other people around, perhaps a few other people will pipe in with suggestions.
So - for example - if you're looking for 3-4 player board games more interesting than Sorry and Monopoly - so interesting that they have a decent chance of getting your friends and/or family members more into gaming - I prepared a guide on our site that does just that:
http://obviously.com/619/gateway-games
(Note: site styling is incomplete - we're not quite ready to launch)
The essential idea is that if you narrow a topic down enough (i.e. 7200 RPM HDDs that are very quiet and can easily survive a several foot drop onto a hard floor), there's probably somebody out there who's taken a great interest in that narrow niche and can write about it very well.
On the other hand, it's hard to write about the too-broad hard drive category as a whole, and it's also difficult to talk about one hard drive with little context.
Then there's also the issue of reviews with an "agenda" - self-explanatory, no need for me to say more.
One thing that seems to work is to see someone I know using a gadget, to actually see that gadget being used, and talk to that individual briefly about the gadget. The info I get this way seems quite a bit more trustworthy than online reviews. Of course, this could also be an artifact of the way we form opinions.
Another empirical observation: on Amazon, a significant amount of negative reviews is typically an indication of a real issue. But the absence of negative reviews doesn't say much. Positive reviews do not appear to be quite as relevant-information-rich, unless there's an extremely large amount of them.
A lot of people got angry at the time, and publicly declared they wouldn't review any more. But that's not really a big loss. Their reviews might be the most qualified out there, but as long as they were anonymous I would be unable to put them in context, and they would be of very limited value to me.
I don't trust good restaurant reviews, and only trust bad ones if they cite specific issues, like using wrong ingredients, etc. I trust my friends and family when they say a place is good or bad much more because I know their tastes and I can ask followup questions.
For electronics, I pretty much don't trust anyone with anything unless something is backed by the manufacturer. I only trust Apple products because of Apple Care. Everyone has had a bad experience with a laptop, a phone, a tablet, a mouse, a USB cable, etc. It's not worth listening to most times since it's usually a horror story from 10 years ago about a company that who's had their entire manufacturing line re-tooled since.
On Amazon for a generic product, the biggest indication to me is if there are lots of reviews. That means lots of people own the product. If most of the reviews are one star, it's a crappy product. If not, it could be good or bad and I personally find very little correlation between the way I perceive a product and its Amazon reviews.
This can provide its own bias. For instance, your average film critic has seen so many versions of stupid action movie that they find those movies pretty tiresome.
My favorite reviews are currently from America's Test Kitchen. Their equipment reviews are especially insightful and well written to provide a thorough expert opinion on products. Wonderfully, they've found a business model that allows for their independence.
"Star ratings are completely screwed"
"Long written reviews "...are also extremely subjective."
Totally agree. Here is my analysis and resolution: http://bit.ly/LRSdkJ or
If you want to fix this with me drop me a note. I have an idea, a design and a viable business model. :)
> For a time, Target used Amazon's website code.
It strikes me as awfully pessimistic to immediately assume someone's clicking their mouse thinking "Haha, Gmail is down! Take THAT Google! >upvote<" when the result is very real service disruption for a lot of people.
"This is your pilot speaking. The engines are currently not on fire and are still attached to the plane. That is all."
Also known as the 3/11 rule. On average people tell 3 others when they have had positive experience. And on average they tell 11 others when they have a negative experience. [proper citation appreciated]
Add the effect of internet, and multiply by Google effect. More recent posts are given higher relevance, and of course more frequent occurrences are given higher relevance.
News agencies and publishers have known this for ages - after all, there's no news like bad news.
Every productguy/developer should know this. :)