Anthropic AI just did their own study of software engineers and admitted that relying on AI stunts skills [View all]

[highplainsdem]

There was a long thread about this on X, but fortunately there's also a post on their website that anyone can read

https://www.anthropic.com/research/AI-assistance-coding-skills

and the full study is here

https://arxiv.org/abs/2601.20245

available in PDF or HTML form.

Those can be read in their entirety. But I noticed that the thread on X, posted on January 29, offered the best summary of their findings in the first 4 tweets.This is the text from those:

AI can make work faster, but a fear is that relying on it may make it harder to learn new skills on the job.

We ran an experiment with software engineers to learn more. Coding with AI led to a decrease in mastery—but this depended on how people used it.

In a randomized-controlled trial, we assigned one group of junior engineers to an AI-assistance group and another to a no-AI group.

Both groups completed a coding task using a Python library they’d never seen before. Then they took a quiz covering concepts they’d just used.

Participants in the AI group finished faster by about two minutes (although this wasn’t statistically significant).

But on average, the AI group also scored significantly worse on the quiz—17% lower, or roughly two letter grades.

However, some in the AI group still scored highly while using AI assistance.

When we looked at the ways they completed the task, we saw they asked conceptual and clarifying questions to understand the code they were working with—rather than delegating or relying on AI.

The first reply I saw under that thread, with about a thousand likes, was this:

“we improved coding speed by 2 minutes and melted 17% of the skills” is the most honest summary of 2025–2026 AI dev culture i’ve seen so far

Btw, that actually understates the dumbing-down via AI, because they found that the engineers who relied most on AI scored 27% lower on the quiz, getting only 40% right.

The intro to the study at https://arxiv.org/abs/2601.20245 :

AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.

This is what use of AI does to stunt learning in adult software engineers.

Think of what it's doing to kids in school. And the AI peddlers keep pushing for it to be used more and more.

Kudos to Anthropic for publishing the study results.