How to Decide Where a Person Reviews an Agent's Work
Every agent needs a stop in it. Name the review gate in the scope, before anyone writes code, or you will negotiate it after the first bad output.
· 4 min read
Patrick Laughlin, Founder, Chief AI Officer
Naval Academy, class of 2011. Decorated naval aviator, 10+ years. President, 2023 Inc. 5000 company.

I flew for ten years before I ran anything. The habit that stayed with me is not discipline and it is not teamwork. It is that every procedure I ever used was written for a bad day. Nobody writes a checklist for the flight where everything works.
That is the test I put on an AI agent now. Not what it does when the input is clean. What it does at four on a Friday when the input is wrong and the person reading the output is in a hurry.
So the first decision on a build is not the model, the stack, or the trigger. It is where the human signs.
The gate goes in the scope, not the retrospective
An agent is an eighty percent solution. It is a junior employee, and that is how you have to treat it. (Managing one gets a post of its own.) A junior employee is useful and fast and will hand you something confident and wrong roughly one time in five, which is fine, because you read it before it leaves the building.
The mistake is not trusting the model too much. The mistake is leaving the reading step undefined until someone is embarrassed. Then it gets designed in a hurry, by whoever is angriest.
Write it at the start instead. For every workflow, one sentence: this agent may do X unattended, it must hand Y to a named person, and nothing in category Z goes out without approval.
Two builds where the line landed in different places
A commercial landscaping company with many branches wanted its pricing to be consistent. Each branch had its own habits, so proposals for similar work could go out at different prices. We built a checker. It reads a draft proposal, compares it against the terms database and comparable past jobs, and flags what sits outside the band with the reason.
It does not price the job. It does not send the proposal. It cannot. Pricing authority belongs to the person who carries the margin, and moving that to a model would have been the fastest way to lose the account and deserve it.
A multi-office law firm was the other shape. In discovery there, ten of nineteen interviews named the same drain on the week: indexing and labeling case documents, often tens of thousands of pages at a time. How that count picked the first build is its own post.
That work gets a machine. Legal judgment does not. The report we delivered opened with the list of what not to build, and the practice of law was on it. A firm that gets told everything is automatable stops believing the parts that are.
Three questions that write the gate for you
What is the worst output this can produce, and who sees it first? If the honest answer is a customer or a regulator, the gate is not optional and it is not a setting.
Can the reviewer tell a good output from a bad one in under a minute? If checking the work costs as much as doing it, the agent is scoped too big. Cut it until the check is cheap. A summary that cites the three source records can be checked in seconds. A summary that cites nothing has to be redone to be verified.
What does approving actually consist of, and does the system record it? A click that nobody logs is not a control. If someone asks in nine months who approved a bad number, "the process" is not an answer.
The first release always has a reviewer
I will not put an agent in front of a customer with no review on the first release. Not because the model cannot do it. Because you cannot learn the failure modes of your own data from a demo, and the first month of real inputs teaches you things no test set will.
The version of this I say on calls: every agent needs a human in the loop, and nothing goes live without your team's approval. It sounds like a caution. It is the opposite. Once the gate is written down and the reviewer trusts it, you can move a great deal faster, because nobody has to argue about risk on every ticket.
Build the checklist for the bad day. Then you get to fly on the good ones.



