Hi everybody, I'm doctor copyright-infringement!
Now, I can say, ``What if this box were really slow to respond to network requests 10% of the time?'' and just do it.
This is not ``Yay, I don't ever have to worry about any other computer because I can simulate anything.''
This is me sitting next to my QA guy (who really understands what I did better than I do so far) and having him say, ``Why don't you just make it fast for the first ten minutes, then slow down socket operations briefly, then switch a pattern of speedup and slowdown on disk reads?''
I want to do more fault injection stuff. I'm being asked to do stuff like manipulate the data (sometimes), or lie about the results of certain operations.
My laptop is also able to variate CPU clock speed from 2GHz to 200MHz. It makes me easy to execute CPU-bound benchmarks of my programs.
And regardless of speed, it's good to test your software on a variety of systems.
This doesn't take the place of real-world testing. It takes the place of setting up a network and putting a modem between my client and server then attaching a tape drive to my server to see what happens.
So far, it's been quite useful, but I've got a ways to go.
This seems like it'll get you 80% of what you want. But it'd be useful to have an actual disk to test on as well.
I think this is a great idea, but having worked on benchmarking database structures in the past, I'd be weary of using them for any type of real benchmark.
For one, trying to model a real disk would get very complicated very fast. Say, the access time of the disk will be a function of the position on disk, so it would be unrealistic to get random delays when scanning a large chunk of contiguous data or by just having a few delays in a random-access heavy load would also be extremely unfair.
In short, trying to model complicated disk latencies is pretty hard, and usually if you are programming with some model of disk in mind, building a disk latency simulator under that same model may end up giving you a false sense of security.
For what it's worth, I'd favor getting a cheap hard disk and trying the load there instead.
If your customers are average joes, then yes I would say you should have a test environment in place that closely represents those customers.
Even after all that the question I have is which set of customers brings in the most money for you? Lets say you have a mix of big business that run fast machines, and average joes that run slow machines. If 90% of your money comes from big business, why bother trying to make software for the slow machines? Just slap on a system requirement and it should cover them, so-to-speak.
A joe-blow at home on the other hand is either at least at the "corporate" level, or even higher for gaming or media-centering.
It's going a bit beyond just simple performance, too. For example, I can add logging of reads on client network file descriptors like this:
https://github.com/dustin/labrea/blob/master/examples/lognet...
There is a nice Firefox plugin (Firefox Throttle) that lets you throttle uploads and downloads with a single button click. If, like most web developers, you do development on localhost using the plugin lets you see how your user's experience uploading large files or downloading image rich pages with network latency.
Unfortunately its been pulled from the addons library search but you can download it directly (for now) using the ftp url in the comments here:
http://support.mozilla.com/en-US/questions/755876#answer-107...
I'm a firm believer that developers should have old hardware to test on. :)
Development is on fast intel single or dual core machines with slow disks and production is slow many core sparc machines with fast disks.