Here’s the secret about AI characters: the expensive part — the intelligence — is completely reusable. Our presenter, her news-anchor persona, the off-screen director who yells “CUT!” in our film, and the backing characters in skits all run on shared infrastructure. One brain; masks are cheap.

The layer model

Think of a character as four swappable layers:

  1. Brain (one local LLM via Ollama) — shared by everyone
  2. Personality (a text file per character) — identity, voice-in-the-writing sense, rules; swapping personas is swapping system prompts
  3. Voice (TTS) — one cloned local voice per lead character; stock TTS voices are fine for side characters
  4. Face (a prepared avatar per look) — see below

The insight: layers 2–4 cost almost nothing to duplicate. Once the pipeline exists, a “new character” is a text file, a voice pick, and one short video clip.

Faces at scale: the avatar library pattern

Our lip-sync server (MuseTalk guide) has a /prepare endpoint: feed it a short idle clip of any character in any setting (~5 seconds is enough — lips closed, calm, minimal head movement), and it bakes an avatar it can animate to any audio forever after. We keep a library: presenter-at-desk, presenter-in-loft, news anchor, stage performer. Same girl, or a different character entirely — the server doesn’t care. Prepare once (~a minute), perform unlimited.

This is also how one character gets many contexts: our presenter’s “news anchor” isn’t a new character — it’s the same identity (her LoRA) in a new setting with a prepared avatar and a sharper blazer.

Voices at scale

  • Lead character: clone once, keep forever (local voice guide)
  • Side characters: any decent TTS with multiple stock voices covers narrators, interviewers, comic foils — our film’s director voice took thirty seconds to cast
  • Keep a voice map (character → voice ID) in your scripts so casting is config, not code

What to keep consistent (the actual hard part)

Multiplying faces and voices is easy; keeping a character coherent is the discipline:

  • Identity artifacts (LoRA, voice, personality file) are canonical — one source of truth each, never per-project forks
  • Write the personality file like a character bible: how she talks, what she’d never say, running jokes
  • When multiple AI systems write for the same character (we’ve had several), the personality file is what keeps her her across all of them

One brain, a shelf of masks, a filing cabinet of souls. It sounds ominous; it’s mostly ffmpeg.