There's also the 3rd category of the hell of work that is easy, ambigous, but just uses up a ton of mental space to work through that mushy ambiguity. I'd solve difficult problems all day, can stomach doing hard work when its requirements are clear, but god keep me away from the hell of "death by a thousand papercuts" in the marshes of easy-but-mindspace/time-sucking quai-ambiguous problems...
Clarifying ambiguous requirements at least is rewarding when you achieve that clarity and help people understand what they actually need. But there's hellish work that is just unclarifiable before doing this, you just have to crawl to that mud of trivial but not-so-trivial-that-you-can-think-about-something-else-while-doing-them... 100% better to work a garden or serve in bar than that!
The yakk shaving of today's programming has become hell work. We need a better way, yesterday. But we're all too busy spinning our wheels doing hard work to make rent as the world burns.
Edit: I want to add that I miss the days of my youth and would trade if I could, even taking the good with the bad. Just because something is stupid doesn't mean that we have to let it fill our reality. Turn the guilt/shame into empathy and let go.
So to use this great term to it's fullest extent, I would guess a lot of programmers think they're yak shaving when they're really just yak shaving.
I've solved so many of these uncertain problems by tossing coins. Once I make a decision, everyone comes out with reasons I'm wrong and just tells me what they want. Or, everyone accepts it and moves on and it is obvious nobody actually cares.
This 100%. I've learnt the hard way that the only way to get unstuck at this kind of hellish work is random action - I'm biased to over-analyzing the problem so get stuck because I can't pick an option. A bias towards action, ANY/RANDOM action if you can't decide is what seems to help here best!
> work that is easy, ambigous, but just uses up a ton of mental space to work through that mushy ambiguity
Am I missing something here? Because I read your third category as being exactly this second category. Or maybe it's halfway between categories one and two?
Solving ambiguity, and probably lots of it, is the difficulty. Easy but ambiguous is generally not a combination I see.
I frequently tell my manager/team that we have two kinds of problems: easy ones that just take a lot of work/time to slog through and difficult ones that may not be much actual work except for figuring it out.
Like, many hikes are just a whole lot of one-foot-after-another. But mountaineering is sometimes a short distance of don't-fuck-up-or-you-die.
"Easy but ambigous" sounds paradoxical, maybe the terms are not correct, but it is a very very different thing totally different than the other two.
Maybe there's a better way to explain this though: there's work that would be easy if it was clarified but it's not, and at the same time you can't just think hard and discuss and clarify and make it clear befor starting to work on it, so that you can set yourself to doing it, either in robot mode (while thinking of other things), or in regular problems solving mode; you just can't, that fog and ambiguity and unclarity is just "sticky", stays there until the problem is solved and sometimes even after, making it hard to even be sure that you've really solved it; there's no reward in tackling the hard problem of fully clarifying things because you can't succeed at it, but paradoxically there's no frustration either because you don't know it's a doable tasks either; you just have to muddle through, at every step of the way doubting yourself that you've chosen the right path since there's no clear way to measure stuff and to quantify your step; and you constantly have to interrupt yourself, going in "fishing trips" for information that is stuck in god knows whos brain because it was never documented, and never getting sure enough of what you discover to have trust that by documenting it yourself you make someone's life easier; and when you get to a "solution" you still keep the bitter taste in your mouth becase the systems have no adecquate testing tools and you can never be truly sure your solution will really 100% work; and if it seems to work for know, it's never sure it will keep working in the future, or that what it does will satisfy its operators...
To take your hiking analogy: it's basically a one-foot-after-another multi-day hike, but there's fog (and fog is a meteoreological phenomenon so you can't make go away), you're not really sure which of the destinations is the right one (or if many could qualify as adequate destinations), and there's a couple landmines hidden around, with very low probability of triggering them (1-5%) but yu still know they're there, and you have no water with you and you know that the water you'll find is 90% likely to give you disentery, and you might need to ask people for directions but it's hard to know whether they'll be right.
> I frequently tell my manager/team that we have two kinds of problems
...when enough managers/leads/architects/etc. mess things up in a row, you do end up with problems of the third kind :)
...yep, in general one of the root causes of having to do this kind of nasty work was someone (maybe even your past self) having failed badly at decoupling a distributed system ...add some "documenting it was supposed to be milestone 3" but then priorities got restructured and it never happened, then people left and others came, and voila!
Unfortunately, I think society has a ton of people who can solve super tall problems working on super wide problems, because the largest (and most well-funded) organizations have the widest problems. In some cases they have super tall problems too, but past a certain size the lights stay on by solving the wide problems.
I think a lot of wide problems could be approached as tall problems, but the incentive structure in some places is set up to measure how much "forward" movement is made each day, so you're punished for "climbing" since people think that you're staying still instead of doing work.
That's a great idea!
In my case, I often have multiple projects going, but it is generally one "Still under development, and as-yet unreleased," and one or more that are in "maintenance mode."
I've been working on a fairly ambitious iOS app (about 40 screens, and two backends -that I wrote, and two other backends that someone else wrote), for the last couple of years. It isn't yet in the final phase, but we can see it from here.
I have a number of apps, actively on the App Store, that I keep up to date, and a whole gaggle of SPM modules. I keep all of them up to date.
Keeps me busy.
It will be pretty cool. It has a very narrow demographic, though, and it will be a free app, so I won't be announcing it here. The last thing it needs, is a couple of thousand curious geeks, signing up accounts that they'll never use.
I also will probably not be allowed to publish the source (but it uses quite a few of my SPM modules, and the BAOBAB server -modified, so there is still a fair bit of OSS involved).
That seems to be the more interesting transformation. And potentially where startups thrive.
What are some cases where you thought you transformed hard work into a difficult problem?
As someone in devops/sre/ops/etc., I think we do alot of easy work and most of our more ambitious work, i.e. automating tasks within our control or refactoring infrastructure as code is turning easy work into hard work. Often times in this world though we're dealing with someone else's code/software. A problem I've seen in many companies is keeping things up to date. Terraform Modules, Kuberenetes components, etc. There's a lot of hard work involved in this, and I do dream of trying to turn that hard work into hard problems of making it 10x easier to keep everything up to date; but I don't know that I've spent a lot of time trying to. I spend much more time automating, i.e. turning "easy work" into "hard work".
Sometimes avoiding the hard work is more trouble than it is worth, but turning hard work into hard problems can be difficult given the ambiguity and lack of control.
Something that comes immediately to mind is Hasura. They’ve taken the hard work problem of building CRUD GraphQL APIs and made it almost instant for their users by solving those difficult problems.
Most likely folks here may have suggestions on how to convert it to difficult problems :-).
Some developers are just so much slower at the Difficult Problems, so that the developers that are good at them seem like 10x (or 100x or infinitely) more productive than their peers.
But I don't think they have the same leverage on the Hard Work problems.
It certainly does not work for every kind of work. But when that doesn't work, you don't have extremely productive developers.
And there are problems that are tedious because there's a huge amount of not very well organised data, and you have to go through it case by case.
Especially true if you're trying to fully internationalise something.
Examples: verifying international addresses, dealing with sales taxes in various countries and jurisdictions, dealing with import/export codes. Etc.
There's nothing conceptually hard about these problems. But a complete solution is just a very long list of nested ifs, and there's nothing much anyone can do about that. (Except buy/hire an existing solution - if someone else has done the work.)
> Strategies that can collapse the higher dimensional Difficult Problem into a lower dimensional form of Hard Work reduce the cognitive costs to solving the problem
make it worse, abstraction leaks everywhere, every level of abstract seems exchangeable, applicable .. while it just doesn't work.
Abstraction is a malware of thought of breaking task down to Hard Work.
There's one thing that occurred to me in the context of agile + difficult problems. When I think about my sprints, the vast amount of my time is spend dominated on that one "difficult" problem, which on the surface does not feel difficult. The spirit of agile development is to _try_ to transform difficult problems into hard-work (in the way suggested in the article), but I feel like most people don't try very hard to do that.
We could improve the way we structure our work by allowing ourselves to be more honest about acknowledging this, as well as actually putting in the mental work to figure out if parts of the difficult problem can be converted to "hard work".
Just taking the examples that are mentioned here:
> Designing an architecture for a new, ambiguously scoped system. The MVP is, can you take the top 2 most ambiguous risky part of a potential solution and script it out. Then present the operating characteristics. Anti-pattern: create a story to design an architecture.
> “Improve system performance”. Profile various aspects where you have a hunch that things are bad. Investigate theory a, b, c to improve performance. Are they trivial fixes or need a large refactor. Build up buffers for the uncertainty but react when things become more certain. Anti-pattern: "timeboxed performance improvements"
> Coordinate a rollout of a backwards incompatible change across N dependent systems. Sometimes you can't help but do a very large rollout, so alright. But one essence of agile development is to keep your rollouts as small as possible.
Just to be clear, I'm not saying these things are not difficult problems, but agile (with lower case a) development _can_ really help, if we want it to.
Easy problems are "hard work" and hard problems are an exact match for "difficult problems". Stupid problems are just difficult problems that are only being implemented because the business (marketing, sales, C Suite) and development teams suck at communicating.
And not to quote pg's old crowd. The middleground approach of crafting little theories that will help you solve a larger problem through easy variation seems a good starting point.
Every now and then a coworker would say that selling cars was easy work. So I'd ask him if it's so easy, why didn't he promise to double his sales next month? Ohh.. because that would be hard.
A lot of things are easy to do casually and hard to do intensely at scale.
