Measure your own coding habits before you believe anyone else's numbers

Part of "AI, engineering and what survives production", a series on the parts of building with AI that hold up once real traffic hits them.

There is a claim going round that you have probably absorbed by now: AI-assisted development is making codebases worse. Refactoring is down, duplication is up, we are all writing more and revising less. The numbers behind it are real, the samples are enormous, and I found I had started repeating the conclusion in conversation without ever having checked it.

Then it occurred to me that those figures are averages taken across hundreds of millions of changes from thousands of organisations, not one of which is mine. So what is the rate in your repository? Nobody has told you, and on current evidence nobody is going to. I set out to find mine, assumed it would take an afternoon, and spent three days discovering that the answer is far harder to get at than the confident version suggests.

So this is not a piece about what AI does to code. It is about how to ask that question of your own repository without arriving at a wrong answer, which turned out to be the genuinely difficult part. The tool I built to do it is git-habits: free, local, and it reads no source code whatsoever.

What git can actually tell you

Git history is a surprisingly rich behavioural record. Not of quality, about which it knows nothing at all, but of habits: how often you commit, how large those commits are, whether you go back and change what you wrote last month, and whether anybody still touches the old code. That is a narrower thing than quality and it is the thing the industry claims has changed, so it is the thing worth measuring.

Four signals are computable from commit metadata alone, without opening a single source file:

Moved lines. The share of changed lines sitting in files git detected as renamed or copied. It is the closest thing history offers to "somebody went back and reorganised this."

Legacy touch. The share of changes landing on files nobody has touched in a year or more. Whether old code is still maintained or merely accumulating.

Rework. The share of changes landing on files touched again within a fortnight. Short-cycle churn.

Commit shape. How many commits per working day, and how large each one is.

None of these is a quality measure, and I want to be clear that I am not pretending otherwise. They are habits. Habits are what the claim is actually about, so habits are what you can test, and a narrow measurement you trust beats a broad one you do not.

What the output looks like

Real output, from one of my own repositories, split at an arbitrary date:

git-habits compare  split at 2026-05-01

  ruleset v1.0  excluded 28,297 of 101,578 changed lines (27.9%)
    lockfiles            24,513
    data_dumps            3,778
    binary_media              5

  first month
    window            2026-04-03 to 2026-04-24  (21d span, 7 active)
    commits           136   (19.43/active day)
    changed lines     41,788
    lines/commit      mean 307.3  p50 90.0  p90 430.0
    moved (reuse)     0.16%  (1.6 per 1k changed lines)
    legacy touch      0.0%  (0.0 per 1k)
    rework <=14d      17.24%  (172.4 per 1k)
    AI co-authored    45 commits (33.1%)

  after
    window            2026-05-04 to 2026-07-18  (76d span, 11 active)
    commits           44   (4.0/active day)
    changed lines     31,493
    lines/commit      mean 715.8  p50 66.5  p90 2997.0
    ...

Two things in there are worth pointing at before we go further, because both are the kind of detail a dashboard would smooth over.

The exclusion report comes first, not last. Nearly 28% of the changed lines in that window were lockfiles and data dumps. You are told that before you read a single metric.

Look at the mean and the median diverge in the second window. Mean 715.8, median 66.5, p90 2,997. A handful of enormous commits have eaten the mean while the typical commit actually got smaller. Read only the mean and you would conclude the opposite of what happened.

Normalise per line, not per commit

This decision matters more than it looks. Measure anything "per commit" and, the moment your commit granularity changes, every metric moves while nothing underneath it has changed at all. A team that switches to squash merges halves its commit count overnight and could be forgiven for thinking it had halved its output. So I made per changed line the primary normalisation, which has the useful side effect of keeping the numbers comparable with the published research.

I kept the per-commit view as well, though, because it answers a different question and the gap between the two is where the artefacts hide. A metric that shifted only because commits got bigger is not a finding, and you want that visible rather than smoothed away.

One caveat on averages. In one repository I measured, the mean commit was 860 changed lines and the median was 16. A handful of bulk imports had eaten the mean entirely. Report the median. If a tool shows you only a mean, it is showing you its largest commits.

The five ways I nearly fooled myself

Every one of these produced a confident, wrong number during the build. They are in the tool's output and its README for that reason.

1. Generated files are most of your history

Lockfiles, build output, vendored dependencies, minified bundles, committed logs. Across the repositories I measured, generated artefacts accounted for between 48% and 62% of all changed lines. One npm install writes tens of thousands of lines to a lockfile, and it will drown every real signal you have.

So exclusions have to be on by default. But the corollary matters just as much: a tool that silently discards two thirds of your data has an invisible thumb on the scale. How would you know? You would not, which is why it must report what it removed and how much. If over half your churn vanishes into the exclusion list, you deserve to be told that before you read a single metric.

2. git log can return nothing and call it success

This one cost me a day. I ran a log with --format="COMMIT" as a record separator, parsed the output, and got clean zeroes across six repositories. No error. Exit code 0.

Git treats a format string containing no % placeholder as a named pretty-alias lookup. It finds no alias called COMMIT, emits nothing, and exits successfully. The pipeline downstream dutifully reported that nothing had happened.

I only caught it because a later start date returned more commits than an earlier one, which is impossible. Any tool built on parsing `git log` needs an assertion that it got something back. Silent zeroes are the most dangerous output a measurement tool can produce, because they look like findings.

3. Unknown is not zero

My exports captured commit subjects but not trailers, so the tool reported "0% AI co-authored" for a repository where 58% of commits carried a Co-Authored-By trailer. The data did not say zero. The data said nothing, and zero was the default.

Then a second version of the same bug: the exclusion step rebuilt each commit object and quietly dropped the trailers field, turning a known value back into an unknown that rendered as zero again.

A missing field and a measured zero must be different values all the way through, and they must render differently. Mine now prints "not captured by this source" and the JSON emits null.

4. Repository age fakes a trend

A repository younger than a year cannot contain year-old code, so its legacy-touch rate is structurally zero and rises as the calendar advances. Compare two windows of a maturing repository and you will see legacy touch climb impressively while nobody's behaviour has changed at all.

Worse, I hit a repository that looked eighteen months old and wasn't: it had been carved out of a larger one, so every file's history reset at the split. Real two-year-old code read as new.

If you take one thing from this piece: check where your repository actually came from before you measure its age.

5. --all can double your commit count

Including all refs picks up unmerged branch work. In one repository it took the commit count from 42 to 94. Neither number is wrong, they answer different questions: work you did, versus work that shipped.

Pick one, write it down, and apply it everywhere. The flags used should be printed with the results, because a comparison between two periods computed under different flags is not a comparison.

What it cannot do

It cannot detect AI. It detects when things changed, and any before-and-after split is only as honest as you are about when you changed method. Do you actually remember the week you started letting a model write your first drafts?

Which is worth dwelling on, because I got my own date wrong by sixteen days when asked to recall it three months later. That is why the tool has a detect command that looks for the step change itself and reports it as a candidate rather than a fact. If the person running the study cannot remember, no user will.

And moved-lines is a proxy. Git detects file renames, not the movement of a block of code between files, which is what the published research actually measures. That is a similarity-matching problem, and an approximation of it would give you numbers that look plausible and cannot be compared with the thing you would inevitably compare them against.

Go and check

Python 3.10 or newer, no dependencies.

git clone https://github.com/uxdw/git-habits && cd git-habits
python3 -m venv .venv && .venv/bin/pip install -e .

# no idea when your habits changed? most people do not
git-habits detect  --repo /your/repo --author "you@example.com"

# then compare either side of the date it suggests
git-habits compare --repo /your/repo --author "you@example.com" --split 2026-04-13

If the repository lives on a machine you cannot clone from, export its history there and analyse it here. The README has the command.

Its companion, [diff-habits](https://github.com/uxdw/diff-habits), goes one layer deeper. It reads diff contents and counts error-masking constructs, the empty catch blocks and suppressed type checks that remove the evidence of a failure rather than handle it. That is a separate tool rather than a flag on this one, deliberately: git-habits never opens a source file, which is what lets you run it against an employer's repository without a conversation. Reading source is a different decision and should be a different install. I have written that one up in the code that hides the evidence.

I am deliberately not telling you what my own numbers showed, because a before-and-after on one developer means very little without a control group. I have one, two colleagues in the same repositories who do not use AI at all, and that analysis is not finished. What I will say now is that the answer I was confident about after an afternoon was wrong, and every version since has been less exciting and more true.

Measure your own crossover. Do not inherit someone else's headline.

Written by Richard Atkins.