Ask HN: Web crawler/data product ideas?

Probably not what you are looking for, but I would love to see more real-time data curation - I would certainly be willing to hand over cash for some decently priced real time data feeds - stock tickers, sports events, raw traffic data, headlines, currency, flights - many of these are available in pull formats, but it would be great to get a push feed.

one big market for you is researchers. if only i had a penny for each time some economist came to me asking for a list of xxx in yyy, where x could be anything from flight prices, to businesses, to startups and yyy could be city / state/ coutnry /world. The internet is really good for current snapshots, but what reserachers most often need is repeated snapshots of the same thing. that helps us analyze how things change over itme. if you're a data company, please, please dont overwrite your tables to "update". save the old data. its really really useful. for research. and more importantly, for you.

Priconomic just refocus to this business model

Have you looked at http://enigma.io/? They won TC Disrupt last year