kez

I am currently working on what I hope will be a startup (lean, bootstrapped etc) and I am dealing with thousands of feeds.

Presently I am batching 5-10 feeds to download in batches of threads from Ruby using FeedZirra (https://github.com/pauldix/feedzirra) and then parse.

Has anyone been in a similar situation and done something particularly innovative they care to share? I plan on ranking feeds by frequency of updates after some analysis, but in the mean time I am resigned to pulling everything down in as quick a time as possible.

I would love to use Superfeedr for this, but cost is prohibitive for me and I do not want to stump up the cash to pay for the credits whilst in development (although I could move to this in the future).

Not so bothered about the technology/language - this is a hodgepodge of Ruby, Ramaze, MySQL, Solr and good old file system storage.

Advanced thanks and appreciation of any and all comments!

In the spirit of identifying niches that HNers could develop products and services around, I'd like to ask what your problems are.

I'm guessing the majority of readers here are not self-employed or actively working on startups, and most niche-identifying-problems stem from the work place. I'll start...

I want to be able to scribe meeting notes and have them recorded digitally for circulation to colleagues and CRM systems. This could take the form of a printed note-taking template, where by OCR could determine different sections (attendees, date, actions, notes) when scanning them in. It is not good etiquette (here) to scribe notes on a laptop during a meeting, and typing them up by hand is very time consuming.

What are your problems?

Back in the early 2000s it seemed like your choices for database-driven web sites were MySQL, PGSQL, BerkleyDB (for the Perlites) and maybe SQLite. Enterprises had their Oracle and SQL Server.

Now, when trying to get stuck into a bit of NoSQL/schema-free/document store databases for the web, I am overwhelmed by the number of options, and am struggling to understand the best one for the job.

Do people genuinely believe that the world needs this many NoSQL systems, or are we just in the infancy/resurgence of schema-free, and things are yet to settle down?