Threadripper 3990X: The Quest To Compile 1B Lines Of C++ On 64 Cores (opens in new tab)

In fact, if compile times are being limited by storage there should be some quick wins in configuration terms - building intermediates to RAM, cache warming, etc - that can enable better performance than faster storage.

That sounds very low for modern SSDs, even consumer-grade. Have you tried different vendors?

Pair it with an Optane for L2 cache and it will speed up normal SSD use too ;)

The SMART stats of the drive says it's at 88% health out of 100%, AKA it'll be dead when it reaches 0%. This is the wear and tear on the drive after ~6 years of full time usage on my primary all around dev / video creating / gaming workstation. It's been powered on 112 times for a grand total of 53,152 running hours and I've written 31TB total to it. 53,152 hours is 2,214 days or a little over 6 years. I keep my workstation on all the time short of power outages that drain my UPS or if I leave my place for days.

Here's a screenshot of all of the SMART stats: https://twitter.com/nickjanetakis/status/1357127351772012544

I go out of my way to save large files (videos) and other media (games, etc.) on a HDD but generally most apps are installed on the SSD and I don't really think about the writes.

You can see this from the warranty for example, which for the Samsung 970 EVO[1] goes linearly from 150TBW for the 250GB model up to 1200TBW for the 2000GB model.

So if you take the 1000GB model with its 600TBW warranty, you can write 50BG of data per day for over 32 years before you're exhausted the drive write warranty.

[1]: https://www.samsung.com/semiconductor/minisite/ssd/product/c... (under "MORE SPECS")

Generally you're looking at hundreds of terabytes, if not more than a petabyte in total write capacity before the drive is unusable.

This is for older drives (~6 years old as I said), and I don't know enough about storage technology and where it's come since then to say, but I imagine things probably have not gotten worse.

[0]: https://techreport.com/review/27909/the-ssd-endurance-experi...

You won't be able to buy ECC ram marketed for speeds beyond JEDEC standards but that does not mean that you cannot clock them higher.

This is fairly common in larger projects, so you end up having to do some hackery to manually sequence make invocations if you want to use recursive make (which is pretty awful).

Honestly, for large projects, Make is an insane choice, notwithstanding the fact that people who are sufficiently savvy at Make can sometimes make it work. (If your tools are bad, you can make up for it with extra staff time and expertise.)

It would be an interesting academic exercise to create a setup on Linux that did the same thing but using the extant build tools, and then to do a deep dive into how much of the computer was actually compiling code and how much of it was running the machinery of the OS/IDE/Linkages. A LONG time ago we used to do that at Sun to profile the "efficiency" of a machine in order to maximize the work it did for the user per unit time, vs other things. That sort of activity however became less of a priority as the CPUs literally became 1000x faster.

Someone should really formalize a standard for declaring dependencies which build systems can share between each other.

You can also use command files[1] to pass options in instead of using the command line.

[1] https://docs.microsoft.com/en-us/cpp/build/reference/linking... to pass

If it's crapping out when you give it -j8, that seems to strongly suggest you're running into limited resources somewhere.

I'm no expert in the intricacies of parallel builds, but as far as I know you can still have dependencies between targets that will limit parallelism.