"Knowledge worker productivity is the biggest of the 21st century management challenges." - Peter Drucker (the founder of modern management)
You don't necessarily need to quantify that exactly. If you've worked on a team with people like that, then you know it. Their managers know it too.
To your main point, the size of the markets is drastically different. There is only so much room for headline rockstars and entertainers, but there's way more room for talented technologists. Having "rockstars" in the engineering world doesn't really reduce the size of the pie for others.
If you can't prove your superior productivity, you are not going to get paid for it. Which is going to incentivize you to optimize for your company's promotion rubric instead, if you don't get fed up first. This leads to the lamented "promotion-driven design".
Or you can ditch the middle man and take your talents straight to the market, through consulting or incorporation.
The person who understands and can solve your problems effectively is more valuable than a clueless person who cannot. It doesn't matter how many hours they spend in the office. What matters is technical knowledge on the sometimes confusing topics that arise at work.
We can take or reject projects, so if you are a bad predicter of use/time to completion, you are going to do worse.
Or I'm a manager. I help my engineers get work done. Let's consider the happy case and assume we can measure our team's revenue. How much of that revenue is attributable to the manager's superior skill, and how much to the rockstars under him/her? What would an experiment look like; the engineers manage themselves, or we compare with the previous manager?
No company that I know goes this far. In their defense, it is a challenge. An open problem, even.
You wouldn't run an A/B test to quantify your impact, but you could pretty easily do before/after metrics (or just collect them on the way during a graduated rollout) for any feature you're rolling out. It should be straightforward to quantify productivity benefit for a large initiative.
> How much of that revenue is attributable to the manager's superior skill
The output of a manager is the output of their team. We can't A/B test managers (or at least, we probably shouldn't), but we can look at a cohort of managers and see how well their teams are generating business impact. Combined with other data (e.g., upward feedback surveys, level-skip 1on1s, retention metrics, etc.), we can measure the impact of a manager.
It helps that we are often saving ~400 hours/yr spending ~80 hours or less.
We can also see it in overtime hours being eliminated, and the (real) engineers are making more complex designs but the size of the team hasn't changed.(30% more complex in 2.5 years, same team size) They also havent replaced engineers that left the team.
Basically: We save enough hours that it is obvious. There are added benefits like not having humans do manual checks, so less prone to errors. (Although I argue that our bugs are equally as dangerous)
Making sure those $ saved didn’t screw things up somehow is the hard part to measure.
And - this is going to blow your mind - but sometimes you can actually make things more efficient by spending more money. Cutting costs can be the exact opposite of the right thing to do.
So while I am jealous that your productivity can be measured so simply, I’m dubious about how many programmers’ contributions can be so easily reduced to a simple metric.
