Feeling Faster

Last week a research group called METR published a study that I haven’t been able to stop thinking about.

They took sixteen experienced open-source developers - people working on large, mature projects they knew well - and gave them real tasks from their own repositories. Some tasks they did with AI tools, some without. Before starting, the developers predicted the AI would make them about 24% faster. After finishing, they estimated it had made them about 20% faster.

It had actually made them 19% slower.

The gap between feeling and fact

The slowdown itself is interesting, and I’ll come back to it. But what stays with me is the gap. These weren’t beginners. They were exactly the kind of people I’ve been writing about for two years - senior, skilled, deeply familiar with their codebases. And they were confidently wrong about their own productivity, in the opposite direction from reality.

I’ve written posts with titles like “The 10x Developer Is Real Now.” I believed it when I wrote it. I mostly still do, for certain kinds of work. But this study made me ask an uncomfortable question: how do I know? How much of my sense of speed is real, and how much is the pleasant feeling of activity - the code appearing on the screen, the sense of motion, the absence of the blank page?

I don’t have a clean answer. That’s what makes it worth sitting with.

Why it might feel faster

I can imagine several reasons, and I suspect they’re all partly true.

Working with AI feels less effortful. Effort is what we usually use to judge how long something took, so less effort reads as more speed.

The waiting is different. Waiting for a model to respond feels like rest, not work, so we don’t count it.

Reviewing and correcting AI output is slow, careful work - but it’s diffuse, spread across many small moments, and we underestimate it.

And in a large, familiar codebase, an expert’s own knowledge is extraordinarily efficient. The AI doesn’t know the codebase’s history and unwritten rules. Explaining them, or fixing the code that ignores them, costs time that the expert would never have spent alone.

What I take from it

I don’t read this study as “AI doesn’t help.” The authors themselves were careful not to claim that. It was a particular setting - experts, in their own mature projects - and the tools are changing every few months. In other settings, especially unfamiliar code, new projects, and boilerplate, I’m still confident the gains are large.

What I take from it is humbler and, I think, more durable:

Feelings are not measurements. Not about productivity, and not about most things. The sense of progress is not the same as progress.

Expertise is easy to undervalue. The developers’ deep knowledge of their own systems was doing a lot of quiet work - so much that a powerful tool couldn’t easily improve on it. That knowledge is precious, and it’s built slowly.

Measure what matters, honestly. Not lines of code, not tasks touched, not how busy you feel. Did the thing ship? Did it work? Did it last? Those are harder to fake, even to yourself.

On self-deception

There’s an old observation, found in every tradition I know, that the hardest person to see clearly is yourself. We are very good at telling ourselves a flattering story about our own effort, our own wisdom, our own speed.

I don’t think the developers in this study were foolish. I think they were human. So am I. The study is a gentle reminder that the most important discipline isn’t any particular tool or technique. It’s the willingness to check your own story against reality, and to change your mind when the two disagree.

The feeling of going fast is not the same as arriving.