What grounding an AI judge actually buys you.

Picture the moment just before you spend money on an ad. You've got a draft, maybe a few variations, and some tool is offering to tell you whether it's any good. It sounds confident. It gives you a score, maybe a letter grade, maybe a percentage. The question nobody asks out loud is: a score based on what, exactly?

That question turns out to matter more than it sounds like it should. A group of researchers trying to figure out how to price ads that show up inside chatbot answers ran straight into it. They needed to measure something called click-through intent, meaning whether a person reading an AI-generated response would actually click the ad embedded in it, which is the thing that drives your business whether you understand the term or not. There was no historical data to learn from, because this kind of ad barely exists yet. So they tried the obvious fallback: have humans rate it.

It didn't work. A prior calibration effort the researchers cite, led by Zhang and colleagues, found that human raters agreed with each other so rarely that their judgments performed no better than a model that makes no prediction at all. Human raters agreed with each other so rarely that the researchers measured their agreement at about 0.291 on a scale where 1.0 is perfect agreement. For context, that's closer to a coin flip than a shared judgment. The paper calls click intent "a highly subjective variable that resists standard statistical calibration," which is a careful way of saying: ask five people if an ad is clickable, and you'll get five different answers, and none of them will be more right than the others.

This is the part that should stop a small business owner mid-scroll. If trained human annotators, doing this as their actual job, can't agree on whether an ad is good, what exactly is a generic AI tool doing when it hands you a confidence score in two seconds?

The fix researchers landed on wasn't more opinion. It was defined, testable rules.

The researchers' answer wasn't to collect more human opinions, or to trust a bigger AI model's gut reaction. It was to build something with defined rules baked in. They simulated synthetic readers using fixed personality profiles based on the Big Five traits, a system researchers have used for decades to model how different kinds of people respond to things. Those simulated readers generated consistent labels, because they were built on a defined structure instead of a vibe.

Then they tested that structured approach against a generic, ungrounded one: a leading AI model simply asked to judge an ad's click-worthiness on its own, with no framework underneath, given no rules and asked to guess on instinct. The gap was not subtle. The structured evaluator hit 79% accuracy on tests designed to check whether it actually noticed when an ad's relevance changed. The generic zero-shot judges, meaning models given no defined framework and just asked to guess, scored 60 to 67%. The structured version also agreed with human preference in 86% of side-by-side comparisons across five different raters.

A confident answer and a correct answer are not the same thing, and an AI model has no built-in way to tell you which one it just gave you.

That 79-versus-67 gap is the whole argument in one number. Both systems are AI. Both produce a score. Only one of them was forced to check its judgment against something fixed and specific before it spoke. That one discipline is what separates a number worth trusting from a number that only looks like one, which is the judgment you can't make about your own ad, and the one Knowing makes for you.

Even the tests used to grade AI often fail the same way.

It would be convenient if this were an isolated problem with one narrow kind of ad-judging model. It isn't. Researchers at Stanford, Sanmi Koyejo and Sang Truong, presented work at COLM 2026 arguing that the benchmarks used to grade AI models generally have the same flaw: they often don't measure what they claim to measure. They borrowed a set of tools from psychometrics, the century-old science behind tests like the SAT, to check this directly, and ran the analysis across 56 widely used benchmarks. Benchmarks that claimed to test the same thing frequently disagreed with each other.

Their concrete example is worth sitting with. BBQ is one of the tests companies point to when they claim their AI is unbiased. A bias-detection test called BBQ is supposed to catch whether a model holds biased assumptions. But the researchers found it conflates two completely different skills: actually being unbiased, and simply being good at spotting a trick question. "A model that really is biased, if it's also good at spotting a trick question, will answer 'we don't know' and score as unbiased," the researchers note. The test can't tell the difference between a model that isn't biased and a model that's just clever enough to dodge the question. The score looks the same either way.

The same structure problem shows up in safety testing too. The Stanford researchers' second finding makes the same point from another angle. Safety tests for AI models are written almost entirely in English, and when researchers translate them to test other languages, two different things get tangled together: the model's actual guardrails might weaken in another language, or the translated test itself might just be a worse, noisier version of the original question. "The model's guardrails might get weaker, so attacks that fail in English succeed in Spanish or Swahili," the researchers write, and a single pass-fail score can't tell you which cause produced the result. A score without a defined structure behind it can't separate a real finding from a measurement artifact, meaning a result that comes from the test itself being flawed rather than from anything true about what it's testing.

What this means for the next tool that scores your ad.

Put these two pieces of research side by side and a pattern holds across both: ask an AI system to judge something complicated with no fixed framework underneath it, and you get a number that sounds authoritative and measures something closer to noise. Give that same system a defined structure, meaning specific checks, specific thresholds, specific facts about the thing being judged, and the accuracy gap is large enough to matter: 79% versus 67% in one study, a documented conflation of two different qualities in the other.

A separate academic paper on evaluating general-purpose AI systems, by Xiting Wang, Liming Jiang, and David Stillwell, makes the broader case for why this is hard in the first place: the authors report that tools like ChatGPT and Gemini are especially difficult to evaluate rigorously because they can do almost anything, which means a generic evaluation has almost nothing fixed to check against. That's a feature when you're asking a model to write a poem. It's a liability when you're asking one to tell you whether your ad is going to work.

So the practical question for any owner handed an AI-generated verdict on their ad isn't "how confident does this sound." It's "what did this check against." Was there a defined claim-density threshold, a documented rule about how human attention works, a specific fact about your brand's tone that you confirmed yourself? Or was it one model's unstructured impression, dressed up as a score? The research says those two things produce very different answers, even when they're delivered with the exact same confidence.