How to become good at system design?

Designing Data-Intensive Applications, Martin Kleppmann

https://www.youtube.com/c/SystemDesignInterview

https://www.educative.io/courses/grokking-the-system-design-...

In addition to the responses talking about getting better with software specifically, I think it's helpful to work get better about thinking in a systems way, generally. There are systems all around us, providing plenty of opportunity to practice systems thinking. For instance right now looking out my window at my backyard I see a drainage basin with frozen water there. I can start asking questions like "where did that water come from," "how long will it stay in that drainage basin", "where does it go when it leaves," etc. Or looking at the next house over and seeing the steam coming out a chimney -- where does that steam come from? What is the heat source? How did that heat get to that house? Etc.

System design is, primarily, a way of thinking -- identifying how given things are actually connected, thinking through how they are connected, what happens up and down those connections as things change. Software systems are just one particular example of this (and, as a bonus, when you start to include non-software parts, such as the humans actually using the software, your software system design will be stronger).

I'm sure others will cite this too, but I don't see it yet at the time I'm writing this, so for a specific book let me recommend Thinking in Systems by Donella Meadows[0]

[0]https://www.chelseagreen.com/product/thinking-in-systems/

Please don't bother with the whole charade of you have to do it to learn it. This kind of advice places more importance on people working at internet companies which have achieved scale. If you are at these companies then you have nothing to worry, experience will teach you. If you are not, follow @cppr's advice and read through DDIA. It will give you good enough introduction to data systems - not just databases. Then I would suggest to learn how message queues work since that part isn't covered in the book. Do a lot of mock designs and attack your designs with different load parameters - how much data do I store, how many machines (if more than one is necesary), can I do it simpler. Honestly if you can estimate storage and throughput well then you are fine. Pick up some distributed systems papers if you are very interested. Once you have these down you would be able to pick more topics yourself. Good luck. Keep learning.

Sorry, there is no way to be good at something without doing it.

After some initial reading try to get your hands dirty with any large scale application you can get access to.

The book System Design Interview is quite good and pragmatic. Personally, I found Designing Data-Intensive Applications less useful because it works on a much lower level. Like if you want to know how database internals work, then read DDIA. If you want to understand how to author a scalable API with layers of caching and distributed data storage, read SDI.

In addition to the great resources already mentioned here, we’ve been making hands-on videos and lessons at Exponent that show you system design in practice, with interview prep as the main focus but advice and discussions with people across the industry.

Check out some of the free videos on our site[0] and YouTube channel.[1]

[0] https://tryexponent.com/courses/system-design-interview

[1] https://youtube.com/c/exponentTV

I think that good analytical skills are prerequisite. You must know how each component/module will interact /interfere with other components. I don't know if there are any labs that might help you. Which existing application do you consider to be large scale application?

my advice is that you need to both consume and produce system design.

don’t rely on interviewing people to test yourself. you have to practice talking or writing and stop settling with the feeling that you just learned something and will have the judgement to know when to use xyz when problem abc comes in front of you.

i built enumerable.co so that i can evaluate myself, to practice producing the knowledge i’m consuming, and reach the edge of my understanding.

there’s no database and i built over a few days, so it’s just a single player way to figure out how much you really don’t know

Learn ho to modularize and how to keep cohesion high and coupling low. Avoid horizontal dependencies, don't mix abstraction layers, learn to isolate, apply tagless final.

Use formal API definitions.

VMWare.

GNS3.

Docker.

Remember that large scale, complicated systems that work invariably start out as small simple systems that work, so build it virtually first, where you can break and re-build to your hearts content.

people who works on CRUD applications will stay far.. far.. away from the system design and distributed systems.