The code that hides the evidence

Part of "AI, engineering and what survives production", a series on the parts of building with AI that hold up once real traffic hits them.

There is a category of code that nobody defends in principle and everybody writes in practice.

try {
  await syncCustomerRecord(id)
} catch (e) {}

You do not think that is good. Neither do I, and I have written it. It gets written anyway, usually at the end of a long day, usually because the failure was intermittent and the deadline was not. And it is worse than the bug it silences, because the bug at least had the decency to announce itself.

The distinction that matters is between handling a failure and hiding one. A catch block that logs, retries, or rethrows with context is engineering. A catch block that swallows is a deleted alarm. From ten feet away the two look identical: both are three lines and a pair of braces, both make the red text stop.

Here is what interested me: this category is countable. Not perfectly, but well enough to see a trend in your own codebase. So I built a tool to count it, pointed it at seven repositories I had worked in, and got an answer I was not expecting.

What counts as masking

I split it into two tiers, and keeping them separate is the decision that makes the whole number defensible. My first version had a single score, and I abandoned it within an hour of pointing it at a real TypeScript codebase, for reasons I will come to.

Hard masking removes a signal outright and is difficult to justify:

Pattern               Example
--------------------  -------------------------------------------
Empty catch           catch (e) {}
Silenced exception    except ValueError: pass  /  rescue nil
Suppressed checking   @ts-ignore  /  # noqa  /  eslint-disable
Weakened typing       as any
Stubbed return        return None  # TODO
Discarded error       Go's _ assignment over an err

Soft masking is legitimate often enough that putting it in a headline would mislead: optional chaining, nullish defaults, TODO markers.

Consider user?.profile?.name. On a genuinely optional field that is correct code. The same expression, written because something in the chain was unexpectedly undefined and the ?. made the error go away, is a completely different act with identical syntax. No static tool can tell those apart, and I stopped trying. So soft constructs are counted separately, reported separately, and the judgement stays with you rather than being quietly made on your behalf.

That split is not fussiness, it is what I learned from the version I threw away. Fold optional chaining into one score and a modern TypeScript codebase produces an alarming number that means nothing at all. The first person to check it will find their own perfectly reasonable ?. in the count, conclude the tool is crying wolf, and never open it again. One number you can defend beats two you cannot.

Count only what was added

I measure additions and ignore deletions. A deletion cannot introduce a swallowed error, and counting both sides double-counts a line that has merely moved from one file to another.

Then normalise per thousand added lines, for the same reason as ever: a raw count tracks how much you wrote that month, and what you want to know is density. Otherwise a productive quarter looks like a decline in quality, and you will spend a week investigating the wrong thing.

What the output looks like

Real output, from the data pipeline behind this publication:

diff-habits scan

  a data pipeline
    commits / files      360 / 584
    added lines          121,560  (83,840 meaningful)
    error masking (hard) 229  (1.88 per 1k added)
    error masking (soft) 417  (3.43 per 1k added)
    top patterns         nullish_default=232, stub_return=187,
                         optional_chain=175, any_cast_ts=22, lint_suppress=20

The pattern breakdown is the part I actually act on. "229 hard hits" is a number I can do nothing with. "187 stubbed returns" is a Monday morning: I go and look at what those functions hand back when the thing they called did not answer, and I usually find two or three that should have been raising instead of shrugging.

What I found, and why it is not what I expected

Pointing this at seven repositories, hard-masking density ranged from 0.16 to 4.0 per thousand added lines. That is a twenty-five-fold spread across codebases written by the same three people.

The highest was a data pipeline that scrapes web content and calls language models. The lowest was a small marketing site.

Sit with that for a moment, because it is the whole lesson. The pipeline is not badly written. It is defensively written, correctly. Networks time out. Scraped pages change shape. Model output is unreliable by construction. Code that talks to unreliable things needs guards, and those guards look exactly like masking to a regex.

Which gives the finding I did not want and now think is the most useful thing here: error-masking density is dominated by problem domain, not by who or what wrote the code. I had a tidy hypothesis about authorship. The data said the strongest predictor was what the code talks to.

So if you compare masking density between two different codebases, what you learn is which one has the flakier dependencies. You learn nothing whatsoever about their authors. Compare a codebase with itself, over time. That is the only comparison this metric will support, and I would rather tell you that than let you draw a conclusion it cannot carry.

What a rise actually means

A rise is not automatically bad, and I was careful to build a tool that does not imply otherwise, because one that scolds you gets uninstalled.

Hardening a service against a genuinely flaky dependency raises the count and is usually the right call. So rising density says look here, not you have failed. What I want to know when I see it is whether the guards I added are logging, alerting, or falling back deliberately, or whether they are quietly returning empty and leaving the caller to draw its own conclusions. Those two look the same in a diff and could not be less alike in production.

The question the number should prompt is the one I now ask myself in review: if this construct fires at three in the morning, does anybody find out?

The honest limitations

These are regexes, not a parser. They over-count a legitimate optional chain and under-count a swallow spread across several lines. except ValueError: on one line followed by an indented pass on the next slips straight through, and catching that properly needs an AST rather than a pattern. I decided that was a worthwhile trade for something you can run on any language in a second, but it means the trend is the meaningful part and a single absolute number is not an audit.

Language scoping matters more than I expected. An early version cheerfully matched TypeScript optional chaining inside Python files, which is obvious in hindsight and was not obvious to me until the numbers came out strange. Patterns are scoped by file extension now. If you extend the pattern list, scope yours too.

Small counts are noise, and I nearly fooled myself with one. A repository of mine produced a single hit across six thousand lines, and for about ten minutes I read that as an impressively clean codebase. It is not a low density. It is not enough data to have a density at all.

Try it on yours

Python 3.10 or newer. It depends on git-habits, so exclusion rules stay identical between the two tools and their numbers remain comparable.

git clone https://github.com/uxdw/diff-habits && cd diff-habits
python3 -m venv .venv && .venv/bin/pip install -e .

diff-habits scan    --repo /your/repo --author "you@example.com"
diff-habits compare --repo /your/repo --author "you@example.com" --split 2026-01-01

Unlike git-habits, this one needs a real working repository. It reads diff bodies, and a metadata export does not contain them.

It reads locally and emits counts. Nothing is uploaded, and there is no network access to upload it with.

That is deliberate, and it is why this is a separate tool from its companion git-habits rather than a flag on it. git-habits works from commit metadata and never opens a source file, so you can run it on an employer's repository without a conversation. This one reads your code. That is a different decision, and it should be a different install rather than a flag you might not notice.

What is deliberately missing

The tool also implements block duplication, and it is off by default behind --experimental-duplication. The current approach counts any five-line sequence recurring anywhere in the history it walks, which conflates real copy and paste with code re-added after a refactor and with ordinary boilerplate. Measured against real repositories it reads about three orders of magnitude above published figures. It is useful as a trend within one repository and useless as an absolute number, so it says so and stays out of the default output.

And line-level move detection, the signal behind the widely repeated claim that refactoring is collapsing, is not implemented at all. It is a similarity-matching problem rather than a hashing one, and a naive version would produce numbers that look plausible and cannot survive comparison with the research they would be quoted against.

Both of those are in the README rather than an issue tracker, because a tool that quietly ships a broken metric is worse than one that admits to a gap. The whole point of measuring is to stop guessing. A measurement you cannot trust is just a guess wearing a number.

Companion piece: [measure your own coding habits before you believe anyone else's numbers](#), on what git history alone can tell you, and the five ways I nearly fooled myself getting there.

Written by Richard Atkins.