But in keeping with Jason Cohen's post, http://blog.asmartbear.com/customer-validation.html , "When ten people say they'll give you money if you build this thing, that's the only validation that counts."
So: would you ever pay $1-$2 a month for search, or is Google et al cutting it just fine for you?
(my first HN post, hoping I'm not breaking any rules I'm not aware of here)
Can you give examples of the scraper sites you are talking about? What terms are you searching for?
I wouldn't mind so much, but a) the site isn't as user friendly as stackoverflow and b) obviously isn't updated with answers as fast as stackoverflow is.
Obviously I should just be searching stackoverflow directly, but I'm afraid that habit of going to google first is a difficult one to break.
If you are using chrome, you might be interested in this extension: https://chrome.google.com/webstore/detail/gledhololmniapejef...
There might be such extensions for other browsers, I don't really know of any.
It's easy enough to ignore the big, repeat offender sites, but not so much for the one-off sites, and avoidable or not they're still there cluttering my SERPs.
Jason Cohen's advice is probably not aimed at people addressing such fundamental problems. Imagine Page and Brin going around asking people if they'd pay for better search results. People would laugh at them.
Even if you get a few million users paying a dollar a month before you stop sucking, your total revenue is barely equivalent to google's orange juice budget. They have thousands of engineers working on making search better plus a pretty good working search engine plus huge infrastructure. What do you have?
edit: Oh yes, and if you do manage to make search better in a way that google hasn't thought of, your best bet is to sell it to google.
Why aren't you using Duck Duck Go? [ http://duckduckgo.com/ ]
It's 10 times faster than Google to find stuff if you're a programmer, you can just press J, K like it was VIM and just focus your eyes on the green box to scan like 100 webpages instantly.
It's even faster if you don't use Chromium but a browser with a sane and configurable BACK button (ex. Opera ex. BACKSPACE) instead of moveyour-hand + ALT + ARROW nonsense.
Set homepage to DuckDuckGo and just press backspace with pinky to get back to entering a new search. IMO even if you want to end up at a google service like Google News it's faster, ex. "!news wikileaks"
As a long time user of google and someone who has given DDG a significant 'chance' I am still using google as my default search.
Part of the reason is information density; I make a search on google and just instantly get more on the screen, and probably because I am so used to google i absorb that information much more quickly. I look at DDG and my brain hurts just a tiny bit trying to parse it all.. not sure how to explain it exactly. But as someone who really is into the idea of a search engine made for power users / programmers.. ddg just wasn't doing it for me.
Re keyboard shortcuts: this is not a selling point for me, nor should it be for any power user. It is trivial to map backspace to the back button in any leading browser. I personally use 'b' for back, 'j' for down, 'k' for up.
Also, I don't have any illusions about killing Google at this point. You don't need to kill Google to make a niche search business that produces a profit, was more my thinking.
I don't think it's a technological hurdle at all, scraping websites is fairly standard nowadays. The challenge is a logical one. You just need to mentally think up of a better logical algorithm than Page Rank.
Page Rank is a very robust heuristic with many perhaps unwitting parallels to social psychology. How people gain status in groups and social norms are created is similar to how websites rank. Perhaps you could catch up on social psychology theory or other seemingly unrelated fields to see if any sparks fly about how you could apply it to the web.
Given that you are able to solve the above, how to store the terabyte of index generated daily?
Assuming you solve that, next come the user experience, how to make it easy for users to adopt your search. How can they filter the search results? How to data visualize so that users can better understand it, etc.
Now, does $1-$2 make sense?
There's no product as of now. I have no idea about storing the terabytes of data, or data visualization, or filtering. I'm not qualified to solve most of those problems. I'm just curious if anyone feels that this kind of search would be worth paying for.
(a) everything is over SSL, (b) there are no advertisements, (c) there is a strict pro-user privacy policy without keeping logs and user profiling.
I think the saying "If you are not paying for the product, you are the product" has a strong element of truth in it in most cases.
The question if you would pay for it is directly because of Jason Cohen's post. If it's not worth paying for at some level, it's probably not a problem that's really screaming for a solution.
I'd pay for Google, for example. The fact that I don't have to pay for it is nice, but I'd cough up every month for it if it was a premium service, because it does something I absolutely need.
1) Because there are so many free providers out there.
2) If the free providers don't find something, I don't know what I'm missing out on, so there is no incentive to pay. If I know that I'm missing out on something I probably have enough information to know how to find it without paying anyway.
What you are doing is asking people to pay for you to help them find something when as far as they can tell they have already found it. You can't demonstrate what value you provide to the buyer without giving all that value away.
All that said, I don't think that makes it a bad idea, just not one that you can charge the user for. If you build a system where user feedback effects search rankings, and you can convince enough people to use it and actually give you feedback on sites, and if you got enough users I think there would be other ways of generating revenue (probably selling to Google as someone else suggested would be your best bet, assuming of course they didn't just build it themselves)
[Edit: Google have tried something similar to this before - http://googlesystem.blogspot.com/2008/11/google-searchwiki-l... but it seems to have disappeared now so I'm guessing it was an experiment that didn't work]
I've been working with search technology for a while now, and while crappy search is easy, good search is hard. There will always be more people trying to game the system than you have people trying to stop them, so the battle to provide good results will never end.
In short, I cannot envision ever paying for general search. Unless it was in a very specialized vertical industry, like court cases, and my current search needs weren't being met (which they currently are, in all fields I use search for).
For generalized search though, I think it's gonna be hard to find many users who will pay. Looking at Google Zeitgeist as an indicator of what most people search for, it look to me like most searches are for fairly trivial information. Most searches don't require the best results, just a number of good results. Google and Bing both do a good job providing good results.. so what's your selling point going to be?
I would easily pay a couple hundred bucks a year for something significantly better than google in anyway.
