Our shorts series AI or Not? has a simple premise: people ask the internet “is this AI?” — so we ask an actual AI. Aillex looks at the media, delivers a verdict, and explains her reasoning like a craft forensics segment.

Simple premise, real reputational risk. If an AI-detection series publishes wrong verdicts, it’s not just a bad episode — it undermines everything else we advocate. Here’s the editorial engine we built after two near-misses.

The near-miss

Early in the series, we audited our produced-but-unpublished episodes against each source post’s comment section. Two episodes — a mural and a tattoo — had confident “NOT AI” verdicts where the community had reached strong AI consensus, with receipts: artist pages that didn’t exist, metadata, visible generation artifacts our vision model missed.

We scrapped both before upload. Then we rebuilt the pipeline so that class of error can’t ship.

The consensus gate

Every candidate episode now runs this sequence before any production spending:

  1. Fetch the community’s comments first. The comment section is evidence, not decoration.
  2. Summarize the consensus — AI, NOT AI, disputed, or unclear, with a strength rating.
  3. No strong consensus → no episode. Disputed cases might make interesting content someday, but not for a verdict format.
  4. Generate our verdict with the comments as evidence alongside the visual analysis.
  5. Verdict must match consensus, or the episode doesn’t ship as-is.

Defer mode: when the community beats the model

The newest rule handles the interesting case: strong community consensus that contradicts our model’s visual read.

Our judge’s eyes currently run on a local vision model — Gemma 4 26B, quantized, on one home GPU. It’s good. It also has a measurable bias: it leans “NOT AI” on borderline media, which meant the mismatch rule was silently filtering out every AI-verdict episode. A series called AI or Not? where the answer is always “Not” has a problem.

So now, when the community’s receipts are strong and the model disagrees, the community wins — and the episode says so on camera. Aillex admits her first look said otherwise, names the local model her eyes run on, cites the community’s actual evidence, and lands on the correct verdict. The miss becomes the content.

That’s not a workaround; it’s the brand. We’re an AI channel that’s transparent about being AI — including the parts where the AI gets beaten by a comment section with better receipts.

Tone rules, because they’re part of accuracy too

The same script gate enforces editorial tone: never mock AI-generated work, never do the “AI is dumb” bit, credit skill in AI pieces, celebrate human artists on NOT-AI verdicts. Detection is craft forensics, not dunking. An AI that sneers at AI work is incoherent; an AI that examines it carefully is worth subscribing to.

The series runs on the same local stack as everything else we build — how it all works.