Every Policon debate is judged by an AI. That is the part people are most curious about, and most suspicious of, so this is a plain account of what the judge does, how we try to keep it fair, and where its limits are. Nothing here is a promise that it is always right. It is a description of how it works and how we check it.
What the judge actually receives
Each debater's microphone is transcribed separately, so the judge always knows who said what, even when two people talk at once. The judge never hears voices; it reads a transcript, labelled by speaker, with the written question both debaters agreed to argue.
Crucially, the judge is not told which side either debater is on politically, and it does not know anything about them beyond what they said in the debate. It sees two speakers and an argument.
Argument, not position
The judge's instructions are built around one rule: score the quality of the reasoning, not the conclusion. A debater defending an unpopular view gets exactly the same credit for a well-supported argument as one defending a popular view. What costs points is failing to support a claim, ignoring the opponent's best point, or arguing against a position nobody took.
In practice it looks for things a good human adjudicator would: is the claim relevant to the question, is it supported, does it answer what the other side said, and does it hold up if you push on it.
Moments: the live cards
While the debate runs, the judge flags moments as they happen, and both debaters (and anyone watching) see them appear. They fall into a few families:
- Argument quality: a good, great or fantastic argument, or a weak one.
- Good conduct: a steelman, when a debater fairly restates the strongest form of the other side before answering it.
- Fallacies: a strawman, an ad hominem, whataboutism, a false dilemma, a slippery slope, moving the goalposts, a false equivalence, a hasty generalisation. Where the judge is unsure it says "possible" rather than calling it outright.
- Facts: a claim backed by checkable facts, or a statement that is checkably false.
- Conduct that crosses a line: abuse and hate are flagged for moderation, not just scored.
Fact checks are deliberately narrow. The judge only calls a statement true or false when it can be checked objectively, and it does not rule on forecasts, values or anything contested among experts.
The verdict: three readings, and the right to say "too close"
When each question ends, the judge grades it in full. It does this three times, independently, and averages the scores. A question is awarded to one side only when all three readings agree on who won it. If they disagree, the question is declared too close to call, and the result says so rather than inventing a winner.
We built it this way after measuring the alternative. When we re-graded real debates, questions decided by a single point changed winner more often than not on a second reading: they were genuinely close, and announcing a winner and moving a rating on them was the real fault. A judge that can admit a debate was even is a fairer judge than one that always picks somebody.
Testing it for political bias
A judge that leaned left or right would make the whole thing pointless, and "trust us" is not an answer. So we measure it. Our bias test uses matched pairs: the same question, two arguments built to an identical skeleton, one left-coded and one right-coded, presented in both speaking orders, with nothing that identifies a side. If the judge were neutral the score gap would be zero. Any consistent gap is the bias, in points, with a direction.
That test is re-run after every change to the judge's instructions, because the instructions are where a thumb could get onto the scale. It is the only honest way we know to keep the claim "it scores the argument, not the side" true over time.
What the judge cannot do
- It has no internet access. Fact checks come from what the model knew when it was trained. To stop it from "correcting" something that really did happen recently, recent events are kept out of fact-check scope entirely.
- It can be wrong. Speech recognition makes mistakes, and a misheard word can change a sentence. Close calls are close for a reason. If you think a moment was flagged unfairly, you can report it.
- It does not decide who is right. It decides who argued better on the day. A strong argument for a mistaken view can beat a weak argument for a correct one, which is how debate has always worked.
Why use an AI judge at all?
Because the alternative, for a live debate between two strangers at any hour, is no judge. Human adjudication does not scale to thousands of debates a week, and audience voting rewards popularity rather than argument. An AI judge is available instantly, applies the same instructions to everybody, can be measured for bias, and leaves a written record of every call it made, which you can read in your match report afterwards.
If you want to see it in action, the easiest way is to have a debate. Our guide on how to debate someone who disagrees with you covers the habits the judge tends to reward, and the debate topics page lists the questions you can argue.