What does it even mean to be 5x more effective? Because that's the thing. Testimonials is where we're at because how do you measure "more effective"? I'm not making any claim on a number, 5x 10x? .2x because you have to go back and fix the shitty code it wrote? We don't have a good way to measure programmer productivity. We all know lines of code is ridiculous. And how do you account for how much brain space it takes up. If the LLM lets someone bang out useful code in between meetings all day, vs needing some truly quiet time for some deep deep work to properly get into the right zone... how do you metricize that and show, what, lines of code and interruptions per hour on a graph to have a line that goes up and to the right?
It turns out that it's actually a really hard problem to measure, so, even for a trillion dollar industry, it is hard.
What did you think of that video?
