webvet

Before anybody gives me a rap on the knuckle, I'll clarify our position directly:

Like most (if not all), we're aware of the fact that there are many (good) site map generators already available out there.

The reason why we wish to 'roll our own' is two-fold:

1) As a training exercise for some of our interns and

2) We rather like building things from scratch and in the process, usually create something with at least some unique features

...and this brings me back to the subject of my thread:

What features would folks here like to have in a web-based site map generator app?

Thanks in advance for all suggestions.

Please check out:

http://httpd.co/damn_quick_mvps/

If not you, could someone you know be interested?

Having struggled a lot with our current VPS, finally decided to go the cloud way.

[Those interested to know more about what we've been facing can refer to my two recent threads:

1. https://news.ycombinator.com/item?id=6348903

2. https://news.ycombinator.com/item?id=6367198 ]

We're now trying to decide between linode and DigitalOcean.

One major difference between the two that we think we've found is that while DO offers SSD storage, linode (we think, again), has regular hard drives.

What do you all suggest? Linode or DO? Or something else?

The server hosting one of our live apps is under high loads and gets into a state where it stops responding to http requests several times a day. These outages last typically anywhere from a few minutes all the way up to half an hour or so.

The major culprit here is MySQL load and we've been working on optimizing this (those interested can refer to my previous thread on this: https://news.ycombinator.com/item?id=6348903).

For now we've found that restarting the httpd and mysqld services brings things back to normal almost immediately.

While we continue to work on a - more elegant - solution, we're thinking of writing a bash/shell script that runs on hourly cron, checks load average (via uptime or /proc/loadavg) and if found higher than a threshold, restarts the services.

Can anyone think of any downside to this (used at least as a temporary measure)?

We have been dealing with a huge MySQL database on one of our projects.

Total byte size: > 150GB, Record count: > 100M (and growing daily)

Primary issue is of performance (this is a live site), currently hosted on a (fairly powerful) VPS.

Not having had in-house prior experience with such data volumes on MySQL, we hacked our own table partitioning architecture of sorts and also optimized the queries, table structures etc. to the max of our abilities. All this has been working reasonably well for us so far.

But now, with rising traffic levels, things are beginning to slow down again.

At this point, we're considering the following 3 options:

1) Throw more hardware resources at it Pro: Instant gratification :) Cons: Cost + vicious cycle(?)

2) Try to optimize the database/config, table structures and queries further Pro: Always useful Cons: Time + Effort

and

3) Migrate to PostgreSQL (or something else?) Pros/Cons: Still studying

What do HNers with experience in such matters suggest? We're open to all suggestions. Thanks in advance.

We needed to capture screenshots of some 50k+ URLs for one of our proprietary projects. After looking high and low, we found many services - some offered totally free usage while others offered (free +) paid plans.

On further study, we found that most of the free services were either defunct/broken or very restrictive and only the paid/commercial ones were going to be any good.

While exploring third-party solutions, we were also working on a roll-our-own hack which, eventually, came about quite nice recently.

At this point, I'm wondering if it would be worth while to launch a service of our own.

All comments, questions, ideas, suggestions etc. welcomed. :)