I find this distinction harmful. Are we going back to the days where there were programmers who did the grunt work and architects who told them what to do?
I prefer how my company career ladder, where there are no architect titles. Each level is 'fractal', containing almost the same responsibilities but at different scales and scope.
So for example, as a Junior Engineer you will define the design (architecture) of a feature within a service, while as a Staff Engineer you will oversee the architecture of a system spanning several teams. This way architecture is everyone's responsibility, and people have a natural path to grow their skills.
Doesn't this distinction exist in every profession? Experienced people start doing less and less of the day to day tasks, and start doing more strategic work.
Only in software do we think this is a terrible thing, and that an acceptable use of someones 10 or 20 years experiences is for them to be down in the trenches centering divs with the new grads.
Hence terms like "staff engineer", which is a modern architect but with a name that pays lip-service to egalitarian fantasies.
Building some theoretical construct and having other people build it is by itself bad architecture, resulting in bad team culture and bad systems.
Agreed on fractal model with healthy room for mistakes that should be carved in for everyone - it's the only model that can enable healthy growth. Non-practicing architects are 0, and non-architectign engineerign is 0 too.
This said, as much as I hated the architect idea earlier in my career (because I was architecting regardless of my title) - the concern and ability to architect things is a very real one. It can only be learned by taking responsibility and having room for mistsakes.
If nothing else it's a shift in the responsibly of architecture. In the most rancid organizations there are silos where devs only code and architects only design. The joke of the Ivory Tower Architect is real. On the other hand as a dev II (between junior and senior) I was asked to design and implement a large scale green field distributed system that interacts with other teams. Our ladder and job titles not only allow this, but it's encouraged and sometimes expected.
This is how you end up with antipattern-laden monstrosities. And I don't mean that in the dogmatic sense, I mean juniors will find "clever" ways of getting the job done, often with far too much abstraction and reinventing core technologies like message-passing, task schedulers, databases, ORMs... It can be a good learning experience but they need a active mentor/architect overseeing every major decision.
If you are going to reinvent something you should at least be aware of it. Juniors just don't have this broad perspective yet, almost by definition.
OTOH, one of the my pet peeves is an "architect" who gives this advice I am giving but doesn't actually have the requisite technical background, i.e. saying "don't reinvent the wheel" while looking at a teacup.
ah haha I just stepped away from my work computer after creating a new branch and writing a "short term fix" in a particular module instead of fixing the root cause because that would involve talking to multiple teams. Instead, I can just do some simple string manipulation just for the one bug I have...
I am trying to talk myself into or out of creating this pull request...
It’s not like SDE 3 (senior) makes you immune to architecture over-complication.
In your example, a mid level dev would oversee the architecture of the service. This dev would work to the junior to ensure their approach fits the rest of the service, and review their work. In turn a senior eng would oversee the architecture of the system owned by the team (possibly involving several services), and keep tabs with each one.
Delegation doesn't mean lack of accountability.
Large organizations working on grand projects benefit from slicing up roles so that experienced specialists can anticipate and prepare solutions for the uncommon challenges that arise in those projects (and so that the bureaucracy of large orgs can be managed at all).
Meanwhile, smaller organizations, or groups working on more modestly scoped projects, benefit from generalists who can hold much of the project in their head and develop highly cohesive solutions that can be delivered efficiently.
A skyscraper requires seasoned architects or its going to crumble under its own weight, but even an extensive residential home renovation can be planned by a general contractor and done way cheaper than if many specialists were brought in.
It sounds like there are some organizations/projects you wouldn't want to be involved with, but that doesn't mean that those organizations/projects are doing something harmful or regressive. They just don't suit the way you want to approach work.
The largest tech companies out there (usually? anecdotal) do not have this distinction, it's the smaller and older companies that continue to enforce this arbitrary divide.
As long as Conway's law holds, system design is going to be at the mercy of the person who designs the team organization. It doesn't really matter what title you give things.
I can tell you from experience that powering through a conways law mismatch doesnt yield good results. "communications overhead" feels abstract, so let me make it more concrete. When the org structure has no mapping to the software structure, the following bad things happen:
- there is no clear rule for who's job it is to fix a bug. You have to always inspect the code and guess.
- same issue for code review, and your code quality standard ends up beholden to the team with the tightest deadlines
- nobody knows which manager to escalate issues to, and the only effective strategy is to haul a bunch of PMs and senior engineers into a conf call to analyze and decide
- the testing and release strategy becomes one size fits all, and cannot make any assumptions about independent subcomponents of software -- you have to test and release the entire product in one go
This is the form communication overhead takes.
So literally that's "hiding" an actual distinction that it's already there.
I agree, every dev should be able to design systems. With software development becoming a highly desirable career due to money, this is just not true and possible anymore.
The world is full of productive developers that are not skilled enough to design systems. Designing systems is also incredibly hard, I have rarely seen (in my very limited sample) more than a few devs able to do it in a company.
So we have a category of people who put a lot of effort in becoming good at something, but we don't recognize them because "everybody should know that too". I think that's not positive.
People who want to grow in the design skills are free to do it and resources should be available, but the reality is already that people at higher levels tell you what to do. Isn't that what you would expect, somebody with more experience driving you?
This is one of those arguments that looks good on paper, but causes untold pain in practice.
The job title may say Staff Engineer, JavaScript Ninja, Principal Engineer, Senior Architect or any combination of the above, but in the end someone has to do the job.
When your company didn’t use the specific title of architect the role of architect was still distributed between different engineering levels. For example you had the role of X Engineer + Architect… this can work well if the project is mostly of technical nature and doesn’t require a lot of technical requirements elicitation, negotiation, synchronization and other people and organization focused work. Because if it does contain such work, you’ll be spending so much time in meetings that you won’t have time to write code any more.
Other companies assign the architect role to a specific person, or have multiple kinds of architects. An important duty of architects is alignment within and outside the organization and mediation between business and engineering.
In large-scale projects such work is especially important and can’t be done by someone with the title of developer, because they would probably get bored out of their mind and also not actually develop anything any more.
Neither of these approaches is wrong. I’ve worked with dev teams that expect to receive a design/spec that they will implement and at the other end of the spectrum with teams that love to design their own architecture. I will provide that spec for the former and I will make sure that their architecture remains cohesive with the platform architecture for the latter.
And it did happen to me that teams would insist on designing their own architecture and then create an over-engineered heap of code. Too bad for them, because they have to fix the bugs, but still ok for the platform, since all the interfaces and platform requirements mesh as they should.
Whatever level the architect is operating at, it’s crucial that they keep their technical skills sharp through building small prototypes, reading and analyzing code, code-review, etc. I can say from my own experience that whenever I neglected that, I started to become detached from the reality of the project and my decisions were poorer as my problem-solving approach became too abstract.
The prevailing sentiment here seems to me to be that we should do away with the title architect and allow senior developers to perform that function. I believe I saw a reference to Hohpe's "The Architect Elevator" as an example of the old school of thought in the comments here. Check out this guest blog of his https://martinfowler.com/articles/architect-elevator.html where he makes the case that senior developers should set the architecture for services and systems but that there is also the need (in larger organizations) for architects to advise on software portfolios and technology radars.
Sure, but at some point "spanning several teams" means you will not be close to the code anymore. You'll spend most of your time in meetings and it increasingly becomes difficult to stay close to the code. Doing "grunt work" is increasingly a waste of your time and a distraction from doing things that have impact at your level. At that point you end up telling people what to build and having them build it.
You're fooling yourself if you believe that that's not the norm already.
In fact, it boggles the mind how anyone believes that inexperienced people who are taking their first steps writing code and don't know what a software architecture is have as much say in the daily workings of a project than the grey bearded guys with a couple of decades of experience under their belt.
There's just a lot of other activities that Systems Engineer does (and still consults them with the team, but is at the end responsible for doing them). (all the bullet points in "The realization")
No I don't think so. Where in the article did the author said he is telling other people what to do? What he said is
> proposed software architecture consulted it with key developers, and made sure everyone is onboard
Being an architect, or staff engineer in modern term, means that the person will need to use his experience and expertise to help the team to find appropriate solutions, present to the team, list out pros and cons, let them pick, and drive the change with other stakeholders.
We - unaffected bystanders - don't really need to have opinions.
It always seemed to me that people with more of a managerial background would be better managers - is software/system development the only field where masters of their craft ultimately become directors?
People with managerial backgrounds have....what? Nothing really. They have to guess at any plan presented by technical people, are always suspicious they are being screwed over on estimations and real problem areas, and are unable to correctly identify when people are doing good work -- thus also being unable to set a healthy engineering culture for success. That's why most managers are demoralizing for engineers. They just don't get it.
It's interesting to note that alot of the most successful startups in SV are not from MBA's but engineers with masters or PhDs....it's not a coincidence I think. They have the practical experience to lead a real world venture to success.
Managers are good at managing departments like insurance claim processing or bad debt collections, which any human can learn in a few weeks fundamentally.
People with managerial backgrounds can become quite adept at helping you:
- Identify blindspots in your biases and behavior that keep you from peak performance
- Avoid working on stuff that's not valuable to your team
- Settle disputes within a group
- Motivate you and keep you engaged/fulfilled with your work
- Get unstuck with personal problems
This is not an exhaustive list and you don't have to have a 'managerial background' to master stuff like this. I am an engineer who has had to learn management as a startup founder. I used to distrust the whole management thing but that kept me from growing as a teammate. Management is not only useful in 'non technical' jobs, it's useful in all human endeavors it's why we study it so much and why it has so much leverage.
Reminds me of “What would you say you do here?” https://youtu.be/m4OvQIGDg4I
After that, you have a set of "programming tools" (I don't want it to sound mechanistic, because it is the opposite of that), which are your team and reports, and they will be able to fill in the details (and the details here can be significant pieces of design and architecture by themselves). And your role is to choose the right tools, and allow them to work to their full potential. This means clearing obstacles, clear communication, technical help at times, mentorship, aligning expectations and giving them clear paths for growth.
All these things can be considered engineering at a larger scale. You want to get a really big system shipped and productive? This is the work, these are the skills you need.
Because those tinkerers get to a point where they want to be the ones making the decisions, controlling the culture, technology, and direction of the company. Without position you have no power and without power you can't affect change.
I don't know enough to say whether this is a common pattern or not, but if it is, it could perhaps be that tinkerers tend to gather a vast breadth of knowledge that can be very useful when making strategic decisions. They reach the point where they know enough to understand what questions to ask on many topics, even if they are an expert in only a few (or none) of them.
I don't know how else you pick up the skills to be an effective technical manager
It's a lot easier to spot developers or contractors bullshitting when you've been in their shoes.
The distinction dates back to the industrial revolution, where you had manufacturing line workers and line managers. A manager would usually be an owner's relative or someone they trusted – more loyal to the company than the unions.
This distinction perpetuated well into current age, just notice how much implicit bias there is about "programmers don't have people skills" to keep workers accepting a career ceiling. Most managers aren't skilled either, and not respected by the workers due to it, but companies won't keep them from managing because they need someone to be responsible for plans, estimates, OKRs, etc.
I guess in software it's "more" common due to survivorship bias – the business of software is so messed up and people have so little idea how to manage it, that companies without experienced leaders have a smaller chance. Strong companies and teams in the field have leaders with enough hands-on experience to have natural authority.
Probably ambition. At a certain level of experience, you realize you cannot create your technological dreamscape by yourself.
Becoming lord of a technology company and directing its resources is like programming the ultimate computer.
I didn't get the impression that he's your typical CTO and completely hands off with code and development. If anything, it sounded like he's still very much in the trenches but has learned how to delegate work well and pick which problems are worth the time investment.
When it came to his work on Oculus and the work he's about to do in the field of AGI it sounded like he'll definitely be making direct contributions. It's entirely possible that I misunderstood his stance throughout the interview though and he's a more hands off guy now.
I'd love to be able to find folks that can do what I do -- it's probably our #1 issue holding us back -- and direct them to meet some business and technical goals, but I've yet to work at an organization that supports that mode of management.
Theatre and Cinema are two other fields where folks have been known to work their way up to director/producer/megalomaniac...
I think Weinberg in "The Psychology of Computer Programming" talks about the performative nature of writing code.
I've spent most of my career fixing bugs or improving existing apps/systems in C++. Every time I interview somewhere they want me to design a (usually offline) program or service and I don't know how best to do that. At this rate I'll be stuck at Senior Engineer for the rest of my career. (I'm already 14 years in).
* RPC = function call
* API design = function signature
* data schema = simple structs
* message queue = list
* load balancer = simple facade object
* database = hash table / in memory SQLite
Once you have a rough sketch in place, you can run your prototype and test different behaviours.
It is a bit "tricky" to tackle asynchronous and concurrency, which you should do from the start. I prefer to do everything based on callbacks, and run my simulation from a static input list of events, ticking every component with a tick() method. This way I can generate different scenarios, save them to JSON, replay them.
Here's some examples:
- https://github.com/wesen/summer-pasture-netflix
cell.py is a kubernetes prototype (blackboard system), while app.py is the start of the netflix event notification system.
I am thinking of writing a course / book about the subject, and have a bunch of (wild) notes here: https://publish.obsidian.md/manuel/ZK/Structure+Notes/2+-+So...
Systems design is for me quintessentially embedded system architecture for a microcontroller, a SoC or combination. It could be a smartphone or console, or medical device, etc. This is a very broad domain… in my case I work exclusively on non safety-critical SW, so I’ve focused on learning Linux internals and system programming which I combine with an architecture framework. Typically I design middleware and daemons which work together to cover a specific area of a product. I work with different kinds of specialists (HW, kernel, performance, etc) and a lead SW architect for each specific platform. The system architect is a different person that oversees both SW and HW and requires skills in both.
For Linux I cand recommend the classics: Stevens, Kerrisk, Love. I found Chris Simmonds’ book to be a good general intro to embedded programming; it has much info which also helps with embedded system design. I’ve also picked up ideas from Making Embedded Systems by White or Embedded Systems Architecture by Noergaard.
Application design refers to individual applications running on an OS. These are the typical MVC or layered architecture styles. The official OS UI framework will likely define a recommended architecture while headless services are quite flexible and support several architectural styles.
Vol 1 of the Pattern Oriented Software architecture is a good intro. I have Software Architecture Patterns by Richards on my reading list, since it covers more recent styles like microservices.
You can spend years learning all about great design patterns that make sense for an IoT or embedded system but have completely backwards ideas about how to organize code for a game that needs to process 1000 unit movements per frame.
But once you figure out some narrower domain that you like, you should be able to find books about it , (if you're good at reading code) can start studying open source approaches to it, and can start prototyping small projects to gain some hands-on experience and muscle memory.
I don't know about the book, but I think MATLAB Tech Talk about systems engineering might be a good start for you: https://www.youtube.com/playlist?list=PLn8PRpmsu08owzDpgnQr7...
Part 3 is about designing a residential toaster :)
I disagree.
In my experience hiring the right people limits the need for organization and process. Obviously you can't eliminate those completely but if you hire the right people your org will generally succeed in spite of a lack of organization and process.
I find the larger problem is how most engineers define "smart people" - they typically only mean someone with a high technical aptitude when really things like communication skills, collaboration, motivation, etc. are better indicators success on a software team.
This is true. I like Marty Cagan's take on it in his article "Scaling with Process vs. People"[1]
But I agree if you have a lot of smart people you can go for a lean process, and let everyone mostly guide themselves. But you do have to provide that freedom.
For 'simple' systems, Systems Engineer is not needed. Typically Software Engineers share architectural responsibilities.
I recommend listening to this https://youtu.be/pSfZutP9H-U?t=386 from 6:30 to 10min. I find it very valuable.
> I find the larger problem is how most engineers define "smart people" - they typically only mean someone with a high technical aptitude when really things like communication skills, collaboration, motivation, etc. are better indicators success on a software team.
I agree 100%.
Most technical people view disagreement as a negative for the person doing the disagreeing, I've seen it play out too many times over my career.
Netflix has a presentation about this somewhere...
I highly respect the approach of architecture in the team/agile environment and seniority in engineering. But at a given scale, someone is there to orchestrate all of it and does the overall plan. That person is a solution architect.
Personally I noticed, that there's always something to analyze more in-depth, something to document better, yet another thing to teach others, however other engineers that I work with don't really care about creating coherent view of solutions they create and they don't care about providing clear description/instructions/guidelines for their solutions.
So do you depend on developers to prepare complete solutions and only verify, if they do everything that you would expect or do you actually have to guide them through details and support them in design, so you can together create something that works according to architecture?
I've been a solution architect embedded with a team of 4 devs, but was constantly dragged into meetings by more senior architects collaborating on the wider platform we were re-writing. That collaboration was important, but it consumed too much time. At the time I lost connectivity with the developers and they were to a degree twiddling their thumbs while a design was nailed out.
Since then I've come to see the benefit of design and architecture stages including the actual developers as much as possible/as much as they want to be included. By that I mean specifically things like getting developers to pair with you on proof of concept work, where you need some form of discovery that will lead to a design decision; pairing with developers on pieces of analysis and documentation around the current state of the system/application, or business/user workflows (in teams where there has been no business analyst).
With conceptual/high level design you might not need much developer input, but when you get into the detailed design/logical design dev input can be crucial (especially if you don't know a lot about the current state of a system). When I'm putting together a design I like to go as wide as possible first with the scope, and then fill out the scale of each area with details after further analysis. Getting your developers (if they're available), to help fill in some of those gaps can be beneficial.
Getting that early engagement with the actual developers is key, and in my opinion taking a more agile and scrum approach to the architecture is much better than doing all the big design up front with zero developer input and then throwing it over the fence. That said, your development team needs to be geared for this, and some might not be, if all devs are busy delivering feature A while the architect looks ahead at what's next. But if I can get time from at least 1 developer who has a good handle on the current code and tech stack, then that's definitely something I capitalize on.
One worry I’ve had with taking jobs with that title is that the salary range seems significantly lower on average. Even if the specific job paid well, I worry about pigeonholing myself into those roles.
