Three independent proposals won't find the error. Adding a refuter killed one of them.

Five agents: three perspectives, one refuter, one synthesizer. The refuter killed a proposal, found the assumption all three shared, and caught two errors in the data I handed them.

I used five AI agents to decide how to rewrite 671 quiz questions. Three perspectives, one refuter, one synthesizer.

The shape came out of a five-agent cap, but nothing in it turned out to be waste. Generating independent proposals is not enough. You need something whose job is to kill them.

The refuter killed one of the three

One proposal went down for this:

> It never ran the verification it most strongly demanded, on its own headline number — it had confused the denominator with a subset.

Three proposals side by side did not surface that. I didn't catch it either, reading them myself.

The refuter's instructions were explicit:

• Do not read this to agree with it
• Find the place where you can say "this will fail in execution"
• Suspect any number quoted without being run

The real win: the assumption all three shared

All three silently assumed that the metric they measured was the same thing as the problem that had been reported. Only the refuter went at that.

When I actually asked the operator, the answer was two different things entirely. Fixing only the headline metric would have missed the target.

Splitting the perspectives buys you independence of perspective. It does not buy you independence of assumption.

Don't let the synthesizer compromise

The synthesizer was told plainly: discard anything the refuter killed.

Allow compromise and a dead proposal comes back diluted. "Take the best of both" does not un-refute the reasoning that killed one of them.

Hand everyone the same measured data, up front

Let each agent do its own aggregation and they recount the same thing repeatedly and disagree on the results. I dumped the analysis to a file first and had all of them read it.

On top of that: write only numbers you produced by running something. No estimates.

One agent still got its own recount wrong. Because the report was fully reproducible, the refuter could determine that the agent was the thing that was off. Shared measured data decides *who* is wrong, not just *that* something is.

It also found errors in what I supplied

The refuter flagged two errors in the measured data I had handed them. Both were real.

Instruct it to suspect the premises it was given, and it catches your mistakes too.

Make "what to ask" a required output field

I required the synthesizer to emit open_question — things the operator should decide.

What came back: ask the operator which items they actually found confusing; ten minutes settles the direction. That pays for itself before you gamble 4 to 70 hours.

All three proposals started from one remark by the operator, and not one of them proposed going back and asking. It doesn't appear unless you require the field.

I hand-copied the work split and got a quarter of it wrong

This one is mine.

Distributing 116 sites across five agents, I hand-copied the script's assignment output into JSON. I got 28 of them — a quarter — wrong. Names that didn't exist went out; 28 real ones never did.

I caught it by diffing against the real data right after launch, before a single edit landed.

Three fixes:

• Diff what you handed out against what generated it, as the first step after launch
• Verify names down to file existence before handing them over (added an assert)
• Remove the transcription step entirely. The script now emits a string you paste as-is

The cause wasn't carelessness. It was that a copying step existed at all.

Fix the standard in a document before distributing

Before handing out 390 rewrites, I did 76 myself and wrote the pattern into a document — good examples, bad examples, and what must not be touched.

The thing that breaks first when you split work is consistency of standard. One dialect per agent is worse than no standard at all.

Machine checks are the gate; the writing standard travels as prose. I also stated: do not loosen the check to pass. If it feels too strict, report it — don't edit it. Never leave the gate adjustable from the inside.

If everyone falls in the same hole, it's your handoff

Six agents, one video cut each: every one of them hit "the font file isn't here," and solved it four different ways — bare font names, embedding a font from another project, fetching from the network.

I only found out because each report was required to include "where the instructions and reality diverged."

When all of them raise the same point, it isn't the agents. It's a gap in what you handed them.

The next round, I folded the previous four failures into the instructions. None of the four recurred. But sealing holes reveals the next ones — three agents independently fell into a different one.

Requiring those reports is how you learn what to hand over next time.

Takeaways

• Proposals alone don't find errors. Stand up an independent refuter
• Separate perspectives still share assumptions. Only the refuter attacks those
• Don't let the synthesizer compromise. Killed proposals return diluted
• Distribute the same measured data first. It decides who is wrong
• Tell it to suspect your premises. It will catch your errors
• Require an "open questions" field. It doesn't appear otherwise
• Never build a step that requires transcription. I got a quarter wrong
• Everyone in the same hole means the handoff is missing something

Separately from this, I build a desktop AI agent called Wisp. It stands on your desktop, answers when you talk to it, and runs commands when you ask — always showing you what it's about to do first.