\"Zero matches\" and \"couldn't read the file\" look identical

I wrote the checks before fixing anything, then spent the project finding defects in my own checks. Every one of them looked correct when written.

I wrote the checks before touching any of the 671 items I had to fix. So I wouldn't start fixing by feel.

Then I spent the rest of the project finding defects in my own checks. They all looked correct when I wrote them.

Write the check before the fix

Reverse the order and it fails.

Fix first and the check quietly loosens to match what you produced. Your own approach becomes the standard.

I froze the standard in code without touching a single item first.

Don't let the allowlist become an amnesty

Existing matches go into an allowlist. But both directions must be errors.

The second one is the load-bearing half. Without it the allowlist never shrinks. With it, the list decreases monotonically. Mine went 802 → 0.

There is no command to add entries. If you can add, you have a route to escape instead of fix.

When it empties, keep the empty file. Delete it and the next match starts with "create somewhere to put it." An empty file makes fix, not add the default.

The gate and the working view must be different tools

The check only reports matches outside the allowlist. It's a gate — pass or fail.

Which means the person doing the fixing cannot see what's in the allowlist. That needs a separate command that ignores the list and dumps contents.

You cannot see the defects in your own tooling

I handed it to five people and two defects in my check surfaced immediately.

The number it displayed and the number it judged on were different. Display was raw character count; the judgement excluded identifiers. Fix by reading the display and you miss.

It stripped punctuation before comparing. Strings that only joined up across a sentence boundary were flagged as copied, which pushed people to mangle perfectly natural prose. Replacing punctuation with a separator token means it only matches when the other side has a break in the same place.

Right after that fix, matches went 0 → 16. The separator was being counted toward match length. A separator is not content.

Carve exceptions at "there is no other way to say it"

Identifiers (NEXT_PUBLIC_, package.json) and raw data (log lines) cannot be paraphrased.

Exclude them from both the length calculation and the text comparison. Otherwise things that need no fixing get flagged, and the attempt to fix them makes the writing worse.

"Zero matches" and "couldn't read it" are indistinguishable

This is the dangerous one.

Zero items scanned also prints "zero matches."

I once misread a YAML key name, scanned nothing, and got a green check. Fail loudly when you cannot read the input.

Same reasoning: distrust the counting tool itself. My first aggregation used a regex, missed YAML block scalars, and every value was wrong. I nearly reported those numbers.

Watch it fail before you believe it watches

I wrote a guard and mutation testing passed twice in a row.

• Searching by the shape of the call missed a second code path. Anything recorded more than one way slips past a shape match
• Matching on name alone then matched the definition file itself. "Is defined" got read as "is used." Exclude the definition side

Both looked right when written. Until you break it and watch it fail, it is not guarding anything.

Reconcile what's written against what runs

Policies, terms, and help text drift the day you change the implementation. Nothing breaks. The page renders. Tests pass. You find out when someone tells you.

Just before adding analytics, my privacy policy said cookies were used only for keeping you signed in. It became a lie the moment I added it.

The guard has to fail if either side changes alone. Check "is the tracking code present" against "does the policy say so" in both directions. One direction misses either the stale text or the missing text.

Don't grep HTML and conclude it isn't rendering

Verifying production, I grepped for ¥[0-9,]+ and concluded the price wasn't showing.

React emits ¥ and the number as separate nodes, so the HTML reads ¥<!-- -->14,800 and never matches as one string. Zero-width spaces inserted by Japanese typesetting do the same thing.

Read the rendered string (innerText) instead.

Some defects only appear when it runs

A missing CSP allowance blocks exactly one request while the page works normally. Reading the code will never show you that.

That an analytics tag fires an image beacon at a different domain was not knowable until I put a real ID in and opened the page.

Takeaways

• Write the check first. Reverse the order and the standard bends to the result
• Both allowlist directions are errors. A list that never shrinks is an amnesty
• Provide no way to add. Given an escape route, people escape
• You cannot see the defects in your own tooling. Hand it to someone
• Never trust "zero matches." Throw when the input can't be read
• Watch it fail. Passing is not evidence of guarding
• Reconcile docs and code in both directions. One direction misses half