← The Signal

AI Made Duolingo Faster. Did It Make Them Ship More?

The most-cited AI productivity story has a footnote everyone skips — and it's the part that actually matters for your business.

Every few weeks a new number makes the rounds about how much faster AI makes teams. The one I keep seeing cited is Duolingo's — so let's start there, because the real version is more useful than the headline.

Per GitHub's published customer story, Duolingo rolled out Copilot across roughly 300 engineers and reported a 25% speed increase for developers working in unfamiliar code, 10% for those who already knew the codebase, a 67% drop in code-review turnaround, and 70% more changes pushed. On paper, a clear win. And it's real.

Here's the footnote nobody quotes. Researchers studying these tools at the organizational level have started describing an "AI productivity paradox": individuals feel dramatically faster and report being happier, but when leaders measure what the whole team actually ships, the company-wide numbers often barely move. One Microsoft study that tracked developers for several weeks found no statistically significant change in their measurable output after introducing an AI coding assistant — even though those same developers were sure they'd sped up.

Both things are true at once. That's the part worth sitting with.

Why doesn't the felt speed show up in the output? Because writing code was rarely the bottleneck. The slow parts of most work — in software and far beyond it — are deciding what to build, agreeing on it, waiting on someone else, and untangling what broke. AI makes the typing faster. It doesn't, on its own, make the deciding or the waiting faster. Point a speed boost at a step that was never your constraint and you get exactly what the studies found: people who feel productive and a business that ships about the same.

But look closer at Duolingo's own numbers. The biggest gain went to people working in code they didn't know yet. The smallest went to the experts. And the headline 67% faster reviews is partly credited not to the AI but to a workflow change they made alongside it. The lesson isn't "AI is overhyped." It's that the value landed in a specific place — lowering the cost of unfamiliarity, paired with a process fix. That's not luck. It's a choice about where to aim it.

So, the consultative version — the one I'd give a client:

For a business: before you buy seats for everyone, find your actual constraint. If your bottleneck is getting people up to speed in unfamiliar systems, AI is a strong bet — that's literally where Duolingo's biggest gain came from. If your bottleneck is decision-making, alignment, or handoffs, a faster keyboard won't fix it, and you'll be the company with a big bill and a flat throughput chart. Measure the bottleneck first. Buy second.

For an individual: the people getting the most from these tools aren't typing faster — they're spending the reclaimed time on the hard parts the tool can't touch. The judgment. The framing. The "should we even do this?" If AI just lets you produce more mediocre work faster, that's not a win. If it clears the busywork so you can think, that's the whole game.

The takeaway is one question. Not "where could we use AI?" — that has infinite cheap answers. The better one: where does our work actually slow down, and is that a place AI can reach?

If the honest answer is no, that's not a failure. It's the most valuable thing you can know before spending a dollar.

That's the signal for this Monday. See you next week. — Adam