undefined | Better HN

0 pointsKirinDave8y ago0 comments

> If managers fail because the role is too hard for them, we can't expect to get better people, we need to make the role easier

Or maybe we need to start recognizing that engineering management is a thing that has value, requires training, and is distinct from being an engineer with a strong personality.

It's telling to me that rather than elevate the role you're more inclined to destroy it.

> Their essay makes a positive contribution to the conversation

If this weren't a post-Zappos, post-Github, post-Uber world it might. As such it reads like a stale take from 2009, to me. But sure, we can just ignore the cycle of failure in the industry and keep writing the same stale think pieces. Hey check out my fresh new monad tutorial while you're at it, Monads are like burrito aquaducts where the black beans and delicious salsa flow from computation to computation across the continent.

> You're putting the cart before the horse there; in my experience a vertically-layered "microservice organization" like you describe is the product of overmanagement (Conway's law in action).

I've described, with the most pessimistic lens, 2 layers of management to the top of your company, with a focus on team collaboration. Having managers with strong engineering experience and technical sympathy decreases the depth of your organization, it doesn't increase it.

Please don't misrepresent my argument in your rush to dump on the idea of management.

> You don't need a deep technical roadmap, only a product roadmap, because you integrate with other teams at the product level, not the deep technical level. Even in the presence of relatively good management I've found that that's what works best. Think Amazon's famous "platform memo".

Funny then how Amazon has technical management. But as an example of planning, imagine a platform team runs a flight of APIs. There are multiple clients to these APIs all shipping products. Each has their own feature needs. If those teams collaborate on dependencies and optimize around what they need, everyone gets what they need faster. Not coordinating or planning provides a random outcome in a space dominated by sub-optimal outcomes.

> Engineering matters, but planning of engineering has been a counterproductive activity every time I've seen it attempted. Planning and management should be done at the feature level; when it comes to the technical "architecture", trust your technical professionals to figure it out themselves

This entire discussion is predicated on the notion that your managers are technical professionals facilitating this. It's weird how many people refuse to accept the idea that people can exist, while in the same forum glorifying hybrid roles like startup CTOs in the same forum who are obligated to do a lot of management functions AND be the basis of a strong technical department.

P.S., I apologise for the brevity and potential grammar errors. I wrote this on mobile, figuring a prompt response was more useful than an expansive response.

0 comments

3 comments · 1 top-level

lmm8y ago· 2 in thread

> Or maybe we need to start recognizing that engineering management is a thing that has value, requires training, and is distinct from being an engineer with a strong personality.

I'd say that's the real "just ignore the cycle of failure in the industry and keep writing the same stale think pieces" approach. We've had decades of talking about how vital good management is and how better training for management is a great idea, and to what avail?

> It's telling to me that rather than elevate the role you're more inclined to destroy it.

What it should tell you is that in my experience, steps in the direction of destroying the role have resulted in better outcomes, whereas steps in the direction of elevating the role haven't.

> I've described, with the most pessimistic lens, 2 layers of management to the top of your company, with a focus on team collaboration. Having managers with strong engineering experience and technical sympathy decreases the depth of your organization, it doesn't increase it.

I was thinking about technically-vertical rather than organizationally-vertical. Cases where the implementation of a single customer-facing feature - the response to a single customer request - is handled by components that are the responsibilities of several different teams with their own management. That's how you'd end up needing to coordinate roadmaps between different teams, and I see that as indicating poor structuring of responsibilities, usually a result of a team taking exclusive control of an area that should be shared, which is usually a management phenomenon. You want to be in a position where whichever team needs a piece of functionality for a given feature that's their responsibility can simply implement it, whichever layer of the technical stack it resides in - where the only responsibilities that could belong to other teams are product-feature level responsibilities, not technical-area responsibilities.

> But as an example of planning, imagine a platform team runs a flight of APIs. There are multiple clients to these APIs all shipping products. Each has their own feature needs. If those teams collaborate on dependencies and optimize around what they need, everyone gets what they need faster. Not coordinating or planning provides a random outcome in a space dominated by sub-optimal outcomes.

It's an appealing thought, but it just doesn't match my experience of how it works out in practice. The best outcome is often one that isn't imagined - is barely imaginable - until it emerges from the technical constraints (a la http://wiki.c2.com/?WhatIsAnAdvancer ). Letting API feature implementation happen just in time under the person who has a direct use case for that feature raises both the ceiling and the floor on the implementation quality: they're working under the best conditions for inspired insight to happen, and the fact that they're using the feature means their implementation will at least work and fulfil at least one use case, and then the second user of the feature can expand it to suit their use case while keeping the existing use case working (people say that you'll get stuck in a local optimum that fits the first use case but can't possibly be adapted to the second, but I just haven't seen that happen in practice; code that was written for a single use case inherently tends to be simple and malleable). The absolute worst case is that the API ends up with two parallel but different implementations of the same feature to fit two different use cases, which is non-ideal but not actually all that bad in the grand scheme of things.

Whereas planning and coordinating via management as I've seen it practiced just goes wrong too badly, too often. You agree on an approach that everyone who comes to actually use the API agrees is wrong, but no-one wants to revisit the decisionmaking process. Or one manager brings a particular API under their control to expand their own importance, and prioritizes their own needs - or worse, deprioritizes work on it because they're not actually using that API for anything important but won't allow others to improve it. Or it emerges that what was initially assumed to be a single common feature actually has use cases that are so different that they should be implemented separately, but any team that proposed to fork off their own variant would be assumed to be empire-building (or, equally and oppositely, would be given insufficient support in their work), so everyone keeps muddling through with an awkward compromise.

> This entire discussion is predicated on the notion that your managers are technical professionals facilitating this. It's weird how many people refuse to accept the idea that people can exist

I don't think good technical architects/technical planners exist - I've just never encountered anyone who was able to produce better results by making technical decisions ahead of time (across a wide variety of management styles and techniques) than leaving them to implementation. Even if they did exist, I haven't ever seen an organization that was able to systematically identify, recruit, or train that skillset, and I do think they've been trying quite hard in various ways.

> while in the same forum glorifying hybrid roles like startup CTOs in the same forum who are obligated to do a lot of management functions AND be the basis of a strong technical department

I don't think I've ever been one of those people, but for what it's worth: most startups fail, and that those that succeed often do so by doing things that don't scale; in any case a startup CTO is rarely doing all the activities you listed earlier. Often startups simply don't do conventional line management at all, which works until it suddenly doesn't. Often startups don't have multiple distinct technical teams, which makes the whole notion of coordinating roadmaps between them moot.

KirinDaveOP8y ago

TL;WR: We could go back and forth on this for a long time. I doubt we'll see eye to eye. But I won't back down on one thing: Ignoring the massive lawsuits and public scandal that "unmanaged" organizations have incurred and how easy it was for them to go wrong is a bad plan. Ignoring the systemic flop that holocracy has turned out to be is a bad plan. Pretending that medium and large software organizations can "run themselves" ignores what organizations with unprecedented success are actually doing, today.

But of course, what says "Sunday Night" like a good fisking?

> I'd say that's the real "just ignore the cycle of failure in the industry and keep writing the same stale think pieces" approach. We've had decades of talking about how vital good management is and how better training for management is a great idea, and to what avail?

Well it wasn't terribly actionable advice before, say, 2008, now was it? The number of people who could get a strong foundational education with industry focus in CS was quite low until the early 2000's, and most folks want at least 5 years experience.

So really, the only supply of technically proficient managers was folks who were unusually talented AND escaped the orbit of organizations that provided quality experience. So like, Apple? Sun maybe. NeXT? Early Intel was boss, and I knew some amazing folks from SGI.

I had a very negative experience of tech managers too, until I met Bruce Horn from Apple. My earlier experiences were a complete train wreck. I met him in 2006.

We've actually seen a really great wave of folks tackling the idea. Folks like Camille Fournier or Dave Smith producing really great material for consumption. I'm grateful to both.

> I was thinking about technically-vertical rather than organizationally-vertical. Cases where the implementation of a single customer-facing feature - the response to a single customer request - is handled by components that are the responsibilities of several different teams with their own management... That's how you'd end up needing to coordinate roadmaps between different teams, and I see that as indicating poor structuring of responsibilities, usually a result of a team taking exclusive control of an area that should be shared, which is usually a management phenomenon.

Pardon me, so you're suggesting you hire folks who can both implement the API and the distributed systems AND build the mobile client apps, and also the website?

That's how you get mediocre outputs.

> It's an appealing thought, but it just doesn't match my experience of how it works out in practice.

That much is clear.

> Or one manager brings a particular API under their control to expand their own importance, and prioritizes their own needs - or worse, deprioritizes work on it because they're not actually using that API for anything important but won't allow others to improve it. Or it emerges that what was initially assumed to be a single common feature actually has use cases that are so different that they should be implemented separately, but any team that proposed to fork off their own variant would be assumed to be empire-building (or, equally and oppositely, would be given insufficient support in their work), so everyone keeps muddling through with an awkward compromise.

I'm not sure if you fully understand the nuance of this conversation though. I'm not objecting to the notion that bad non-technical management exists. Or even that bad technical management has existed. I cannot deny you've experienced this nor am I interested in doing so.

I'm saying: we need to do better. It's important to do better. And it's worth noting that the really amazing companies we see out there that do huge, good work? Netflix, Google, Mozilla, even Amazon (who I don't think very highly of) etc? They don't do that holocracy thing or that flat organizational structure. Heck, they've even gone so far is to separate site reliability from product engineering entirely. And most folks who I respect as engineer say this separation is the right call.

Our industry has a history of improvement. Not just in terms of technical accomplishment, but in terms of organizational success. Once even small software projects were complete nightmares, now most things ship and we're used to seeing a slew of somewhat functional software products that are lacking not for features but for product-market fit! Now, a previously unimaginably complex baggage checking system that was a famous failure is actually a 1 semester college product for upper-division software engineering and DAMN if they don't do a credible job of it (even though Kids These Days(tm) still don't learn enough about how SLAs are met in industry).

Engineering management and planning has had a similar curve of improvement.

> (people say that you'll get stuck in a local optimum that fits the first use case but can't possibly be adapted to the second, but I just haven't seen that happen in practice; code that was written for a single use case inherently tends to be simple and malleable)

Aside: I've certainly had that happen! At Level we originally based the system around a primary spending account and secondary asset accounts because the staff and everyone who we interviewed had that configuration. We quickly learned that we had a biased sample and many people have multiple spending accounts. It took a very clever engineer to unkink our system to solve for that problem. Wish I had listened to our advisor, it would have saved us 3 months!

> I don't think good technical architects/technical planners exist - I've just never encountered anyone who was able to produce better results by making technical decisions ahead of time (across a wide variety of management styles and techniques) than leaving them to implementation.

I think you have a curiously warped view of architecture. That it spans quarters or even months into the future. It doesn't. Often times, it's as much about auditing the existing babel of engineer rodeo projects to find out what's closest to meeting the requirements and helping those engineers sketch out how to take it the remaining agonizing 20%.

> and that those that succeed often do so by doing things that don't scale; in any case a startup CTO is rarely doing all the activities you listed earlier.

I know several post-exit CTOs because we hang out and we all have similar stories about doing this. I also know a lot of would-have-beens who said they didn't care about things like hiring and "just wanted to code" or functioned more like VPs of engineering (which are a pure management function).

> Often startups don't have multiple distinct technical teams

Usually it's "teams" of 1. You have the mobile specialist, the API owner (often the CTO), the website specialist. This is a very simple version of the larger exercise, but it's often even more important to keep track of how things are progressing because you have so little tolerance of wasted time.

lmm8y ago

> Pardon me, so you're suggesting you hire folks who can both implement the API and the distributed systems AND build the mobile client apps, and also the website?

Not necessarily the same individual, but the same team. If a particular product needs an API and a distributed system and a mobile client app and a website, it should be the responsibility of a group of people that has that skillset between them who sit together, have daily standups together, and all of that. I would expect the distributed systems specialist to be willing and able to add a line to the website or vice versa; different team members have different specialities but everyone owns the product and anyone picking up a feature takes responsibility for delivery of that feature. If a feature requires two people's skillsets they can pair on it and collaborate directly; if it requires more than that, find a way to break it down into smaller features that can be implemented individually but will still constitute customer-facing product features. (This is something people often assume will be impossible, but it almost always turns out to be easy once you try, IME).

If there's a function that's sufficiently universal and valuable that you want to share it (e.g. server administration), you have to stop doing deep integration across that feature boundary and pare it down to a more product-like interface (e.g. your server administrators offer a way for your application deployment to specify which libraries it depends on, rather than managing directly which libraries are installed on which server) where a product-like roadmap is sufficient. If there's a one-off feature that requires deep integration with another team, that feature should be done by a pair with one individual from each team, who can coordinate directly with each other.

> I'm saying: we need to do better. It's important to do better. And it's worth noting that the really amazing companies we see out there that do huge, good work? Netflix, Google, Mozilla, even Amazon (who I don't think very highly of) etc? They don't do that holocracy thing or that flat organizational structure. Heck, they've even gone so far is to separate site reliability from product engineering entirely. And most folks who I respect as engineer say this separation is the right call.

I don't see the giants as the ones doing the amazing work - their big innovations happened when they were smaller - and within the class of giants I would loosely guess that the more effective ones follow a lower-management, more-segregated-teams style. (FWIW I think Facebook - conspicuously absent from your list - have been more effective among the giants, both in terms of making a better product and in terms of producing high-quality technical libraries). But we're getting beyond the level where I can really have an informed opinion; I haven't worked for enough large companies to have a statistically valid sample, and while in the absence of proper studies I'm inclined to trust to my own experience, I could've just been lucky at the less-heavily managed ones and unlucky at the more-heavily managed ones.

> I think you have a curiously warped view of architecture. That it spans quarters or even months into the future. It doesn't. Often times, it's as much about auditing the existing babel of engineer rodeo projects to find out what's closest to meeting the requirements and helping those engineers sketch out how to take it the remaining agonizing 20%.

My view is based on what I've seen people in "architect" positions do; I'd certainly say there's a lot of variation in what it means between different companies, but it's always been some form of making technical decisions earlier than during coding (even if just at the start of the week after scoping out a particular feature) and as such I've always found it counterproductive.

Pair-programming with a developer on the same team who knows the codebase well can be extremely positive if that's what you meant, and I suppose you could consider that person an "architect". But I find it's vital that that person not be any kind of formal manager: the easiest way for it to go wrong is when the junior partner gets intimidated and stops contributing.

1 more reply

j / k navigate · click thread line to collapse