tdenkinger

Say you have a piece of digital content (a song or book), and you sell it on iTunes, Amazon and on your own website. You're interesting in finding out which of those sales outlets your customers prefer.

So, you devise a test: you'll create three splash pages, each has some marketing copy and a single link to one of those sales outlets. Randomly, one third of your visitors will see the iTunes link, one third the Amazon link and the remaining third will see a link to your own website's sale page. The visitor is identifiable, so the same link is presented on subsequent visits.

I don't think this actually tells you about what your customer's like. But I'm having a hard time saying why. If I'm presented with iTunes but I don't like iTunes and never click through, does that really say anything useful about whether I like the alternatives?

That we're even doing the test is a nice change of pace, but I hate the idea of doing a test and claiming the results mean something that they may not. I'm happy to hear that my misgivings are wrong.

I have always expected that people dedicated to software QA are able to write code. Not large, complicated systems. Just straight-forward procedural scripts in a well-known language.

Am I asking too much?

It seems to me that without that knowledge, you lose a lot of perspective on how software works. I think you also possibly miss opportunities to automate some of your most tedious tasks.