The companion episode — Engine Room, machine three.

Every moving shot of our presenter starts as a still image that a video model animates. That’s the core mental model: image-to-video — you supply the first frame (the “plate”), a prompt describes the motion, the model dreams the next few seconds.

The graph shape

Same seven-box sentence as image generation, with a video model in the loader, your plate in a Load Image node, a frame count instead of a static canvas, and a video combine node at the end. Local video models (we’ve shipped a lot of LTX-family footage) render seconds of 25fps video in minutes on a consumer GPU; heavier cloud models trade money for fidelity.

The rules we paid to learn

  1. One beat per clip. One action per 5-second clip. Ask for four actions and the model performs them all at once — limbs develop opinions.
  2. Real-time speed is camouflage. Long slow clips are where AI gets caught: objects drift, scale wobbles. Natural-speed action, hard cuts, fast pacing — it’s not just style, it hides artifacts.
  3. The plate is everything. The first frame decides identity, outfit, framing, and lighting for the whole clip. Generate plates deliberately (see the character pipeline), and describe the whole outfit — cropped plates make the model invent what’s off-screen.
  4. Anchor framing. For presenter-style footage, a seated/standing anchor pose with hands visible animates naturally; extreme wides drift on identity.
  5. Prompt is the energy dial. The same model renders “lively” or “composed television presenter” from words alone. Calm prompts + a negative battery against over-articulation beat knob-twiddling every time.
  6. Motion amplifies exposure. A neckline that reads fine as a still becomes a problem at 25fps. Dress the plate for the animation, not the still.

Local vs cloud

  • Local: free per-render, private, queue-free — our default for presenter footage and anything iterative.
  • Cloud (rented GPU or a credits service): bigger models, better physics on hard shots — we reach for it on hero shots and crowd scenes. For by-the-hour rentals we point people at RunPod (referral link — new accounts get a signup credit bonus): pick a big-VRAM card, run the same ComfyUI you’d run at home, pay only while it renders. Budget real money for re-rolls either way; roughly a third of clips need one.

Lip-sync is its own machine

Talking footage = video generation + a dedicated lip-sync pass. Our real-time approach is documented in the MuseTalk guide; the short version: render the body motion first, then let a mouth specialist repaint sync. Keep faces reasonably large in frame — tiny faces sync badly.

Engine Room series: From Zero · Edit & combine · Upscale · 3D · Character pipeline