All pages
Close the loop · Lesson 2.2 · 10 min
Judges and second opinions
A panel of judges that scores a result before you spend time on it.
After this lesson you get a review from something other than the agent that wrote the work: a panel that scores several attempts at a hard decision, a critic that tries to refute a multi-file edit, and a second model or CLI agent that reads the same change cold.
Time: 10 minutes · Works on: macOS app.
Don't let the author grade the work
Ask an agent whether its own change is correct and it will usually say yes. It isn't lying. It has the same context, the same assumptions and the same blind spots that produced the change. Reviewing your own work in the same conversation mostly confirms it.
A useful second opinion differs from the author in at least one way:
- Fresh context. A reviewer that never saw the conversation reads the change the way a colleague would, without the author's reasons in mind.
- A different brief. "Try to refute this" finds problems that "is this good?" doesn't. Tell the reviewer its job is to find faults, and allow it to pass the change when there are none.
- A different model. Different models make different mistakes. A second model is most useful on exactly the cases where the first was confidently wrong.
- More than one attempt. For a design decision, the first idea is often not the best. Several independent attempts, scored against the same criteria, show you the alternatives before you commit to one.
What a judge is for
Judges and reviewers are a filter before your attention, not a replacement for proof. Use them to decide which result deserves your time, and which concerns to check first. Then verify with the tools from earlier in this course: tests, the browser, the diff.
Match the reviewer to the stakes:
| Situation | Second opinion |
|---|---|
| Routine one-file edit | None. Run the tests and read the diff. |
| Edit across several files | An adversarial critic on the combined diff |
| A design choice with real trade-offs | Several attempts, scored by a panel |
| A bug with several plausible causes | One investigator per theory |
| A change you're about to merge | A cold reviewer, ideally on a different model |
A panel costs several model calls per question. Spend it on decisions, not on renaming a variable.
A worked example
You need to choose how a CLI caches API responses: an in-memory map, a file per request, or SQLite. The agent's first suggestion is SQLite. Instead of arguing with it, you ask for a panel.
Three attempts come back, each written with a different priority: the simplest working solution, the most robust one, and the one with the best result for the user. Two judges score each from 0 to 10 with a reason. The file-per-request design wins on simplicity, and the robust attempt points out that concurrent runs of the CLI need a lock. The final answer starts from the winner and adds the lock.
You still write a test for two concurrent runs before accepting it. The panel chose the design. The test proves it.
Do it in TRMNLY
Call a judge panel. In the Agent tab, ask: "Put this to
judge_panel: should the response cache be in memory, one file per request, or SQLite? Make three attempts and have them judged." Thejudge_paneltool asks for approval, then:- runs 2 to 4 independent attempts (3 by default), each with read-only tools and its own lens: simplicity, robustness, the user-visible outcome, or performance;
- has 1 to 3 judges (2 by default) score every attempt from 0 to 10 for correctness, completeness and fit, with a rationale;
- returns the winner, a scoreboard of mean scores and rationales, and a final answer built from the winner plus the best ideas of the others.
The attempts and judges use this chat's tool model if one is set, and otherwise its default model. The diversity comes from the separate attempts and lenses, not from different models.
Use the ready-made prompt. In the Prompts tab, open the gear menu and choose Browse packs…. Subscribe to the TRMNLY pack. Its Get a judge panel's answer prompt asks the agent which approach won, the best idea from each runner-up, and the trade-off to flag before you commit.
Keep the multi-file critic on. In the agent's gear menu, Self-critique multi-file edits is on by default. When one turn wants to write two or more files, a model on the chat's own provider, the lighter one where the provider has one, reads the combined change with one instruction: try to refute it. Any objections are added to the pending approval cards, so you read the counter-argument where you click Approve. It's advisory and never blocks the approval.
Test competing theories for a bug. Ask the agent to use
debug_hypotheseswith two to six different root causes. It starts one read-only investigator per theory, each trying to disprove its own, and returns the theory that survives with its evidence. The TRMNLY pack's Test competing theories for a bug prompt sets this up.Get a read-only reviewer. Start a new chat with New chat in the gear menu (
⇧⌘N), and under Persona choose Reviewer. It reads, searches and runsgit_diff, but can't edit anything. Ask it to review the uncommitted changes and citefile:linefor each finding. It has no memory of the chat that wrote the code, which is the point.Ask a different model. Right-click your original request, or open the ⋯ on it, and choose Replay with model…. The conversation branches into a new chat named after the model you pick, and that model answers from the same starting point. Compare the two results side by side. You can also pick Claude Code (CLI), Codex (CLI) or Gemini CLI from the model menu under Your CLI subscription (no API credits) to have a different agent do the review.
Have a CLI agent leave review notes. In a split, run
claudeorcodexwith TRMNLY's tools connected over MCP, from Settings → AI Agent → External MCP clients. Ask it to readgit diffand record each finding withdiff_comment_add. The notes appear in More ▾ → Review next to your own. Delete the ones you disagree with, then Send to agent, so the author gets one combined review that you approved.Compare whole implementations. When the question is "which version of the code", not "which design", race it.
race_startgives one prompt to several agents, each in its own worktree. When they finish, compare them, withjudge_panelif it helps, and keep one withrace_pick. Race and pick covers this in full.Gate a fleet with a reviewer. In the Fleet strip on the Tasks tab, turn on Review first. A reviewer agent reads each branch's diff before it lands, can't edit anything, and rejects the branch when its verdict is missing or unreadable.
Check yourself
- You ran
judge_panelon a real decision and can name the winner and one idea taken from a runner-up. - You can say what model the panel's judges ran on.
- Self-critique multi-file edits is on, and you've seen what an objection looks like on an approval card.
- A reviewer with no memory of the original chat found something in the
diff, or confirmed it with specific
file:linereferences. - You verified the panel's choice with a test rather than taking its score as proof.
Go further
- Agent — personas, routing and the orchestration tools
- Fleet — races, the fleet reviewer and land policies
- Claude Code, Codex and Gemini — use a CLI agent as a model or in a pane
- Pick the right agent — which agent fits which job