Models Don't Have Scar Tissue

It’s review round eleven on a PR that the ticket described as a “small change”.

Claude Code wrote the fix. Copilot reviewed it on GitHub and found a serious problem: a race condition in a part of the code nobody had asked to touch. Claude fixed it. The fix introduced another problem, subtler, in a test that now passed for the wrong reasons. Round twelve. Round thirteen. At some point I stopped counting and started reading the whole diff, line by line, the way I used to.

None of this is unusual. It’s what happens now, almost every day.

I’ve been building software for more than twenty years, and something is happening that never happened before: at times, I feel like this job is in danger.

A new feeling

This isn’t the first wave I’ve seen. The cloud came, low-code was going to replace programmers, frameworks promised you’d never need to understand what was underneath. Every time, someone announced the end of the craft, and every time the craft changed shape and kept going. This time the feeling is different, and it’s hard to say why.

Part of it is speed. On September 22, two frontier models from two different companies shipped within hours of each other. The best agents now solve, on their own, more than half of a terminal benchmark designed to be hard, tasks that a year ago looked like territory for someone with experience.

Part of it is that I’m not watching from the outside. I’ve worked with AI every day for four years. A good share of the code that comes out of my hands is no longer written by my hands.

And yet, by the end of the day, the feeling flips.

What the agent doesn’t see

There’s a kind of knowledge that doesn’t show up in the code. It’s knowing that this queue will blow up the day the payment provider takes three extra seconds to respond. That this cache is fine until there are two regions. That an innocent schema change will break a consumer nobody remembers exists. You don’t read it in the docs. You learn it by taking the hit, at three in the morning, with production down and a rollback that doesn’t roll back what it was supposed to. It’s scar tissue.

Over time, that scar tissue turns into a nose for trouble. Architecture, scalability, distributed systems: all of that fits in a book, but the nose is the part the book can’t reach. It’s anticipating where the problem will come from before there’s any evidence it exists.

Today’s agents are very good at the level of the code. Where I still see them stumble is the level above: the context that isn’t written down anywhere, the second-order consequences, the “this works, but it’s not what the system needs”.

Is that a limitation of today, or something deeper, something in their nature? I have a hunch that it will be very hard for them to get there. But I don’t know. And I’m wary of anyone who claims to know for sure, on either side.

The rework moved

Five years ago, delivery was slower. A feature went through design, through discussion, through a PR a teammate read calmly. We found problems early, often before writing a line, in a conversation. There was little rework.

Today the code shows up in minutes. The problems show up later, in review loops that feed on themselves: every fix brings something new, because the agent fixes what you point out with the same confidence it had when it wrote the thing that was wrong. Ten, fifteen rounds to get a PR right that looked simple in the ticket.

I’m not saying it’s worse. I’m saying it’s different. AI didn’t speed us up so much as change where we find the problems. We went from thinking before to correcting after.

There’s one data point I keep coming back to. In 2025, METR measured experienced developers working in their own repositories, with and without AI. With AI, they took 19% longer. When they finished, they were convinced they had been 20% faster. A year later, when METR tried to run the study again, it hit a different problem: many developers no longer wanted to participate if there was a chance they’d have to work without AI.

I recognize myself in both. I don’t know if I’m faster. I know I don’t want to go back.

I notice it from the other side too, as a user. The software I use every day breaks more than it did a few years ago: screens that don’t load, state that doesn’t update, things that break in one release and come back in the next. I haven’t measured it, and I can’t prove where each bug comes from. But many of them have a shape I know well: code that passed the tests and that nobody fully understood.

The rules change every quarter

A good part of my job today is putting guardrails on AI. Instructions, context, guidelines, automated reviewers. Everything an agent needs to reach a goal without slipping in a regression or a severe bug along the way.

The problem is that what you build has an expiration date. Every new model reasons differently, follows instructions differently, fails differently. What worked three months ago sometimes gets in the way today. It’s like learning to drive a car that moves the pedals around every season.

Models get versioned. Every so often a new one replaces the last, and with it, part of what you knew about how to use it. A nose for trouble doesn’t. It accumulates.

Letting go of thinking

What worries me most isn’t in the code.

I see more and more important decisions made on whatever a chat returns. I don’t mean asking for a draft or organizing ideas, which is a reasonable use. I mean strategy: which market to go after, which product to pause, where to cut. People with a lot of responsibility asking a model and deciding on whatever the model hands them, already chewed. A survey by Board of 300 executives at US companies found this year that 61% count LLMs among the sources that most influence their strategic decisions. Among CFOs, almost half said they follow the AI’s recommendation even when it contradicts their own judgment.

But I think there’s something deeper than asking a model what to do. We no longer delegate only the answer. We delegate the question: which data to look at. And we rarely stop to ask whether it’s looking at the right data.

It goes something like this. An agent builds a dashboard and defines the metrics: what counts as an active user, what a conversion is, when retention starts being measured. It documents everything, neat and convincing. Weeks later, another agent, in another conversation, picks up that documentation to answer a different question. But the documentation no longer matches what the metric actually measures: an event was renamed, a filter was left half done, a definition was rewritten along the way. Nobody notices, because nobody was looking. The second agent reasons flawlessly on a crooked foundation, and the conclusion comes out plausible, well written, and without any smell. And wrong.

That’s the conclusion the decision gets made on.

In a large company, a bad decision gets absorbed. In a startup, it doesn’t. Startups pivot all the time based on data like this: they change customers, pricing, product, because a metric said something wasn’t working. In a bootstrapped company, with a few months of runway and a small team that bet everything, one wrong decision can be the last one.

None of this is the model’s fault. It did what it was asked. What’s missing is someone asking whether we’re looking at what we should be looking at, and that question doesn’t show up on any dashboard. The smell is exactly what matters. Sensitivity to context, the discomfort of something not adding up even when every number adds up, the memory of having seen this somewhere else and knowing how it ended. If you stop exercising that, it doesn’t vanish all at once. It atrophies slowly, without anyone noticing, until the day you need it.

A study Anthropic published this year points in that direction, at a small scale: developers who learned a new library with AI ended up understanding noticeably less than those who did it by hand. The biggest gap was in debugging, which is exactly the skill of noticing that something is wrong.

The bar went up

There’s a story going around: any junior with AI becomes a senior. My experience is the opposite. I think the barrier to entry into software engineering has never been higher.

To work well with agents today, you need to understand architecture, understand why something scales or doesn’t, spot when a test isn’t testing what it claims to, read a five-hundred-line diff and find the one line that matters. You need to know what a senior knows and, on top of that, notice when the tool is wrong with complete conviction. AI didn’t lower the bar. It automated the bottom rungs of the ladder.

The market numbers tell the same story from the other side. According to SignalFire’s latest talent report, entry-level hiring has dropped around 65% at big tech companies and 76% at early-stage startups compared to 2019. A Stanford study found that employment of developers aged 22 to 25 fell almost 20% from its peak in late 2022. Meanwhile, demand keeps moving up: more and more, what companies want is people with years of experience.

If a junior with AI were a senior, the market would be hiring juniors. It’s doing the opposite.

A study from Seoul National University interviewed senior and junior developers about this. One of the seniors put it better than I could:

“The reason you become a senior is that experience teaches you what not to do. Now juniors can no longer learn from bad examples.”

How that gets solved is a different essay.

Supervisors

I think we’re heading somewhere we already partly live: the engineer as a supervisor of agents. Less writing, more defining, reviewing, deciding.

But “supervisor” sounds more comfortable than it is. The job isn’t watching the AI work and approving. The job is making sure of the outcome with AI and, often, in spite of AI. In spite of its biases, its confidence, its tendency to solve the problem you described instead of the one you had. And for that, the only tool that doesn’t change with every release is what each of us brings: the hits we took, the systems we watched go down, and the nose for trouble they left behind.

So, is the craft in danger? I come back to the question from the beginning and I don’t have a clean answer. I think the craft of writing code, as we knew it, is changing for good. I think the craft of understanding systems, anticipating problems, and deciding well with incomplete information is worth more than ever. And I think the industry, right now, is confusing one with the other.

Maybe the fear is unfounded. Maybe not.

In the meantime, I open round fourteen and read the whole diff.