- Lines of code, code commits and pull requests are all up 20-30%
- Comments per pull request are up 49% as human review becomes more crucial
- With reviewing up from 7 to 10 days on average, the new bottleneck is code review
A new Harvard University research paper has confirmed that AI is indeed excellent at boosting coding productivity, but humans just end up slowing the process back down anyway.
Authors Fiona Chen and James Stratton found a 30% increase in the amount of code developers can produce when using AI, but they found no statistically significant improvement in completed software.
With increases across lines of code produced (+30%), code commits (+20%) and pull requests (+23%), the Harvard-backed researchers went on to investigate why we're not seeing a 20-30% increase in end product.
AI has definitely sped up coding, but human delays negate much of those benefits
At the same time, the researchers also found increases across the time taken to review and merge pull requests (+49%), the number of reviewer comments per pull request (+35%) and even the proportion of employees doing code reviews (+14%), all of which implying that vibe coding is ultimately shifting the bottleneck further down the road instead of removing it altogether.
The paper also questions whether, with AI shifting the workload, companies are responding by allocating more workers to code review. Layoffs have become somewhat synonymous with AI – founded or not – and concerns about jobs continue, however the paper proves that human workers are still vital, just in other parts of the coding process.
Some thought it also given to the next stages of AI adoption, and whether it can help downstream in the code review stages. While AI is already being employed to some degrees in this part, "humans are still heavily involved with review."
There's also the consideration of added workload, with AI now capable of producing code much more quickly than humans ever could. The average review process has gone from taking 7.03 days to 10.48 days as a result of this shifted bottleneck.
The researchers also argue that AI-generated code needs substantially more corrections, begging the question whether it's actually helping at all.
Still, with AI adoption not resulting in any significant job losses, it seems, it's more a question about how companies can best use AI to support employees and reduce unnecessary workload, rather than how they can replace workers outright.
"The productivity gains are absorbed by downstream constraints in the production process," the paper concludes, adding that "code review becomes the bottleneck."
