I think, for my next job, one of those questions I will ask when the time comes to ask if I have anything I am curious about in the interview is the choice, when a choice must be made, between procedure and results ... which am I expected to prioritize?
Probably won't win me any callbacks.
Obviously there can be misguided procedures, but any decision made or operation done can be misguided.
What then?
Consider the current thread on the whole "toaster in the dishwasher" topic, during which someone related an incident wherein an entire server site was immersed in water but still functioning (https://news.ycombinator.com/item?id=41251234). The site manager followed procedure (wait a while, not cut the power, perform risk assessment) and it resulted in total loss, but the poster wanted to "cut the power, pump the water out of the bunker ASAP and immediately clean the whole lot with pure water." Here we have a tension between procedure and results. Procedure ended up causing total site loss, which was completely avoidable.
Similarly, a current thread on an ER doctor not following the usual procedures during a mass casualty event was lauded. A choice had to be made. Here, results won.
I just like to know this sort of thing about a work culture in advance. Letter of the law versus the spirit of the law, and so on.
Also people do charge all the fucking time, and don't put their phone into airplane mode. :|
Do not underestimate how even minimal training can be extremely effective at scale. I know from experience that I am a person who does not freeze, I focus. When shit happens, I act but I can act without thinking. Because of that mantra, I have a plan to follow and I will act correctly.
Also don't underestimate the effect of priming. It reminds you there is a plan. When pilots prepare for takeoff, they briefly review how to handle emergencies during takeoff. Not because they don't know, but to bring those procedures to attention and have them ready in their mind.
For the mantra to work, all it has to do is remind you those procedures you vaguely know exist.
Recently I pushed a change straight to production by myself with no approval and violated many rules but it saved us and carried us for weeks. Worse case if it broke, rollback would happen under in under 30 seconds.
I did it because not only did I triple check, I’ve kept mental track of the number of regressions and issues that have been logged against all of my work throughout my career. I’m good at determining risk and I know my bug rate is very low (I git blame every bug to find out who and why caused it. I don’t tell my coworkers but it does play a role in who gets what kind of tickets.) I did what I did because frankly I know it was going to work and no one was going to complain. And truthfully, I’ve done this at different companies several times. Of course, it’s still never a light decision and I rarely ever do it.
But if someone asks me if they could do the same thing, I would not be able to tell them. They would have to keep track of the same details and to be honest, if someone is asking if they can break rules, they probably shouldn’t.
This is the “tension.”
In fact, they were completely misguided about their own accuracy because they had systematically ignored or not understood the errors that they had made in the past.
The challenge as a tech lead or manager is telling the difference between you and them. Or even telling whether you are them. My own tendency would be to fire both of you.
I uphold a certain quality of work and I expect my peers to do the same. Everyone makes mistakes but even mistakes can be modeled.
