undefined | Better HN

0 pointsthrowaway2849278mo ago0 comments

That is not true, usage of AI was decided randomly. From the paper:

"To directly measure the impact of AI tools on developer productivity, we conduct a randomized controlled trial by having 16 developers complete 246 tasks (2.0 hours on average) on well-known open-source repositories (23,000 stars on average) they regularly contribute to. Each task is randomly assigned to allow or disallow AI usage, and we measure how long it takes developers to complete tasks in each condition."

0 comments

sumeno8mo ago

Directly from the paper:

> If AI is allowed, developers can use any AI tools or models they choose, including no AI tooling if they expect it to not be helpful. If AI is not allowed, no generative AI tooling can be used.

AI is allowed not required

throwaway284927OP8mo ago

True, my bad, I didn't read you correctly. What you said was true.

I do believe however that it's important to emphasize the fact that they didn't got to choose in general, though, which I think your wording (even though it is correct) does not make evident.

j / k navigate · click thread line to collapse