Here’s the secret about AI characters: the expensive part, the intelligence, is completely reusable. Our presenter, her news-anchor persona, the off-screen director who yells “CUT!” in our film, and the backing characters in skits all run on shared infrastructure. One brain; masks are cheap.

The layer model

Think of a character as four swappable layers:

  1. Brain (one local LLM via Ollama), shared by everyone
  2. Personality (a text file per character): identity, voice-in-the-writing sense, rules; swapping personas is swapping system prompts
  3. Voice (TTS), one cloned local voice per lead character; stock TTS voices are fine for side characters
  4. Face (a prepared avatar per look), see below

The insight: layers 2 to 4 cost almost nothing to duplicate. Once the pipeline exists, a “new character” is a text file, a voice pick, and one short video clip.

Faces at scale: the avatar library pattern

Our lip-sync server (MuseTalk guide) has a /prepare endpoint: feed it a short idle clip of any character in any setting (~5 seconds is enough: lips closed, calm, minimal head movement), and it bakes an avatar it can animate to any audio forever after. We keep a library: presenter-at-desk, presenter-in-loft, news anchor, stage performer. Same girl, or a different character entirely, the server doesn’t care. Prepare once (~a minute), perform unlimited.

This is also how one character gets many contexts: our presenter’s “news anchor” isn’t a new character, it’s the same identity (her LoRA) in a new setting with a prepared avatar and a sharper blazer.

Voices at scale

  • Lead character: clone once, keep forever (local voice guide)
  • Side characters: any decent TTS with multiple stock voices covers narrators, interviewers, comic foils, our film’s director voice took thirty seconds to cast
  • Keep a voice map (character → voice ID) in your scripts so casting is config, not code

What to keep consistent (the actual hard part)

Multiplying faces and voices is easy; keeping a character coherent is the discipline:

  • Identity artifacts (LoRA, voice, personality file) are canonical, one source of truth each, never per-project forks
  • Write the personality file like a character bible: how she talks, what she’d never say, running jokes
  • When multiple AI systems write for the same character (we’ve had several), the personality file is what keeps her her across all of them

One brain, a shelf of masks, a filing cabinet of souls. It sounds ominous; it’s mostly ffmpeg.