Flintstoning your way around hard technical challenges (opens in new tab)

I have seen this pattern multiple times: (1) tell client completely underdefined problem can be solved automatically, (2) tell developer to implement it, (3) developer spends months trying to automate it, (4) days before the deadline the code does something that would barely count as acceptable, (5) an intern is hired last-minute to solve the problem manually, (6) "we will do it by hand until we have fixed the code".

The client will never know (5). The intern will eventually do it manually full-time. The code is of course never fixed.

Months later, the developer finally has enough understanding of the problem to find out that it has been a research topic for decades, with no satisfying results so far.

We have a website where customers of our company can upload data sets for various domain-specific modelling software, and we can do various integrity and quality checks on the data for them.

If it gains traction we'll maybe automate parts of it some day, but for now it's lovingly referred to as AaaS, or Arnold-as-a-service.

A friend told me a story of a job he got with a consulting company working for a phone company. Their job was to move all of the data from the old system to the new system. They had a multiyear contract where they charged millions to move this data.

When he got there, two terminals were set up - one the old system and the other the new. His job was to read the data from one screen and type it by hand into the other. He was one of hundreds of low paid contractors hired for typing skills.

After a few days of that drudgery, he decidedly said screw this, ^C the app, and ended up at a shell. He spent the next two weeks figuring out how to copy the data file from the old system to the new, and then used a handful of shell scripts to successfully convert the majority of the data.

He showed his boss what he'd done, and was swiftly fired. If anybody found out, they didn't want to lose their lucrative multiyear contract. They'd rather type it all in by hand.

This is referred to as the "Wizard of Oz" method (or "experimenter-in-the-loop"). It's very common in HCI research, as well as in product prototyping in various industries.

https://en.wikipedia.org/wiki/Wizard_of_Oz_experiment

A variation of the technique is also common in CS user studies, where the novel tool under study works, but is too computationally intensive (i.e. slow) to actually use in the study. In this variation, the tool's results are precomputed, and the tool's interface is mocked up so that it just retrieves precomputed results (or it delegates to a human researcher playing the "wizard").

There was some research at IBM, which I have tried to find a reference for, that tested: "What if we wanted to build a better word processor, after having completely solved speech recognition?" They devised an experiment where the subject, given the task of say creating a document, would speak and "the system" would respond by entering text and the usual things that word processors do. "The system" was a CRT and a human confederate behind a curtain who would type stuff and otherwise respond to the subject's commands.

There's another aspect to this. Sometimes that human isn't 'in' the system but can be the user of it.

I recently made a script to document the code paths from any GraphQL or REST endpoint to code lines taking a database lock. It was a hack with false positives. I 'fixed' it by making it an interactive app instead of a script.

“Wizard of Oz prototyping” is a better name for this idea.

I am always worried about solutions like this -- what if one day, the human makes a mistake and deletes newest photos instead of the oldest ones? What if the VA I have chosen misinterprets my instructions and does nothing?

With programming, as long as I take reasonable precautions, I will know that either the task will get done, or I'll get notifications that something is wrong. With humans? Not so much.

(Let's just not forget "reasonable precautions" part -- there is a surprising amount of people who apply the sloppiest programming for the dangerous actions. People who think that "let's use date parser which auto-detects date format" and "any error code means we can delete document" are good ideas)

How hard can a script to delete documents based on timestamp possibly be?

I have always heard this as to "mechanical turk the problem"

There's an XKCD for this: https://xkcd.com/1319/

Also, anecdotally, I literally automate to prevent RSI[0]. There's only so much the human body can manually do with a computer

[0] https://en.wikipedia.org/wiki/Repetitive_strain_injury