this was the unbelievable useful tool to get competitive data (traffic, queries) about other websites. it was not perfect, the data was sometimes months late. but it was good enough to compare websites with each other.
now it's gone. probably it didn't fit into google strategy. probably someone at google decided that "we are not in the business of providing competitive intelligence data of websites" (i guess that, as the google adplanner was stripped of lots of website data in the same timeframe )
the same thing with the product mentioned above. it will be killed. maybe not this year, maybe not next year, but sure as hell sometime.
after killing google reader and buzz I just don't trust them anymore and won't put anything of value into their ecosystem.
they should find a way to just support everything forever even if development is killed.
Also: they didn't kill google reader. And Google+ is not comparable to Buzz. Google is clearly not going to kill that off.
but for anything that is not directly relevant for search and advertising, all bets are off.
People do not have a vested interest in completing the questions honestly or on the outcome. Their motivation is to get to the premium content, and although there will be some that are honest, most people are trying to get "free" content and have the most motivation towards not giving value (money or answers that betray their privacy/thoughts) back.
My day job is primarily based on analysing survey results, and I can tell you that the only way to get decent usable results from "paying" people is to make sure that they have a vested interest in the results of the survey and that the payment is a (nice) side-effect of completing the survey and not the goal.
Edit: Of course, this type of survey can be useful if you need to "validate" something that otherwise you can't (i.e. to falsify results). Simply choose multiple choice questions and stack the answers in the right order (people usually click in certain pattern when doing it at "random"). Keep repeating or extending your survey until you have the right ratio of participants in favour of your preposition, then call it a day and publish!
I always thought that if you have a question (like "should the government fund more particle physics") and you ask people with a vested interest (particle physicists) you'll get an answer aligned with their interest ("absolutely, extremely important") in which case why do a survey at all when you know what answers they're going to give?
You need to pick your respondent cohort carefully if you are trying to ask a question of opinion (particularly more so than when asking a question of fact) to eliminate bias in your cohort. So in your example, rather than surveying particle physicists, you would look for another group of interested parties (e.g. a range of tax payers or non-specialist funding bodies) who have a vested interest (in this case in how their money is spent).
My work, for instance, revolves around surveying doctors on their experiences of medical training and career needs/aspirations. The vested interest of our participants is hopefully clear (influence on their training/career/work environment). We ask a wide range of factual and opinion based questions, eliminate biases (or segment responses on biases) due to various factors (e.g. location, age, gender etc.) and report our findings in the context of our cohort, i.e. "these are the experiences and opinions of doctors".
In short, you are right, you need to consider the makeup of your cohort and also how you report your findings in respect of that. Having a vested interest doesn't necessarily mean being biased, and sometimes that bias is what the survey aims to report on. In a survey like the google ones, that bias is money/free content, and thats all you will discover.
Asking particle physicists would they rather have X than Y when both are relevant to their interests would yield better results than asking a butcher particle physics-related questions and dangling a carrot in front of him for clicking through it.
It appears to only show the relative accuracy of certain types of questions in relation to other particular methods of internet surveys. Its not something that I would use to back significant spending decisions.
Ideally, you could randomize the order of the answers for each person, and later divide them up by order presented to detect any bias. This is something that online surveys make much easier than paper surveys.
I used it to ask how much people were willing to pay to collect video customer reviews and a lot of the responses were 11111111, 1000000 or 12345 dollars per month though.
1. As the researcher you can set a range of validity for open-ended numeric if you know it ahead of time. We recommend doing this.
2. The system automatically detects outliers in all cases and puts them in a separate bucket to reduce noise.
An example of open-ended numeric for pricing is here:
https://www.google.com/insights/consumersurveys/view?survey=...
You can see my survey at https://www.google.com/insights/consumersurveys/view?survey=...
Hmmmmm.
No, wait. Targeting is 50 cents a person, not 10. So it would be about $550. Still, though.
http://www.google.com/insights/consumersurveys/view?survey=b...
The patterns in subpopulations seem to match general expectations. Very cool.
Almost no responses in the $100k+ incomes. I wonder if that's a general problem with the method?
You can compare those to some of the other polls aggregated at Real Clear Politics here: http://www.realclearpolitics.com/epolls/latest_polls/
For those people interested in the accuracy & validity of the GCS platform, this is a great place to start.
As an aside, google launched this about 6 months ago and pretty much has everyone in that business scared. Ultimately it all boils down to having profiles on a lot of users and selling access to them. Google's user base (and what they know about them) dwarfs what everyone else has combined. If they wanted to, with a bit of effort they could own that market in 6-12 months.
I don't see how this can work. The only time I see it some-what-working is while gathering a massive (really massive) amount of submissions for a decision involving only two options. Maybe then... but even then it wouldn't be possible to determine the actual percentage of users suggesting one or the other option.
Also, this is a pretty new tool.
disclaimer: friend owns that site
