Train a Character LoRA: Make Your AI Character Look the Same in Every Image

The companion episode. Aillex narrates this exact recipe, stunt-double disaster included.

Every image of our presenter is the same character. Five episodes, a music video, hundreds of shorts. Same face, same hair gradient, same person across photoreal, anime, comic, and 3D-game styles. That consistency comes from one thing: a character LoRA, and this guide is the exact recipe.

What a LoRA actually is

Your base image model (~12 GB) knows how to draw people. Millions of them. It doesn’t know you (or your character). A LoRA is a small add-on file (~150 MB) trained to teach it exactly one new thing: how to draw this specific subject. You invoke it with a trigger word in your prompt, and the model reaches for the LoRA instead of inventing a stranger.

WHO vs HOW: the two kinds you’ll use

• A character LoRA teaches WHO, the face, hair, build. Identity stays constant while outfits and scenes change.
• A style LoRA teaches HOW, an art style (anime, comic, painterly). It changes rendering, not identity.
• They stack. Our character underneath + a borrowed style on top = our girl, in any art style. This is the single most useful trick in AI character work.

The training set (where quality is decided)

Everything downstream inherits from the dataset. Our working rules:

1. 20-40 images. More isn’t better past ~40; curation beats volume.
2. Vary everything except identity: angles (front, ¾, profile), lighting, distance, expression, outfit. If every training image is the same pose, the LoRA memorizes the pose along with the face.
3. Caption each image in plain language, describing everything except the identity (“a woman in a blazer at a desk, soft light”). What you don’t caption is what the LoRA absorbs as the subject.
4. Pick a weird, unique trigger word. Real words leak the base model’s associations into your character. Ours is invented and appears nowhere in the training captions except as the subject tag.
5. Vary the pose on purpose, and vary the angle on top of it. Our first pass for the Krea 2 build was twenty outfits shot four ways each, and almost all of it was standing. A model trained only on standing gives you a mannequin that resists sitting, kneeling, or lying down later. We added four more sets that crossed both axes on purpose, pose varied and each pose shot from several angles, so it learned what she looks like kneeling seen from behind. That set ran to roughly 480 images in the end, well past the 20 to 40 above, because we were also using it to bench a new base model. For a straightforward character LoRA the smaller number still holds.
6. Name the footwear on every full-body shot. Leave it out and you get bare feet. This is the same failure as leaving the bottom half of an outfit unspecified, and it trains in.
7. Consistent character across the set, if you’re bootstrapping a new character from generations, cull ruthlessly: any image where the face drifts trains the drift in.

Train it: local (free) vs cloud (hands-off)

Both produce the same artifact. We advocate local (the whole point of this site) but cloud training is a legitimate first LoRA, the browser flow on civitai.com (referral link, supports the channel at no extra cost) walks you through dataset upload, captioning, and the dials below.

There’s also a middle path: rent a big-VRAM card by the hour on RunPod (referral link, new accounts get a signup credit bonus) and run AI Toolkit on it. You keep the local workflow, your recipe, your dials, the file lands where you put it, without owning the hardware; a character LoRA’s evening-length run costs a few dollars.

Training locally? The two upgrades that actually matter are unglamorous: 64GB of DDR5 (~$150) so caching and bucketing never touch swap, and a 2TB NVMe because datasets, checkpoints, and epoch samples eat disk faster than anything else in this hobby.

Training on Krea 2 instead of a Z-Image-class base? The rules shift. Different prompt appetites, an always-on LoRA effect, and one file at one strength in place of a stack you have to balance. We benched all of it in Krea 2 prompting and LoRAs, including the traps that make a working setup look broken.

Caption words become control levers

This is the strongest thing we learned in a year of doing this, and it cost us five rounds of dataset work to find. Words you apply consistently during captioning turn into prompt controls after training.

We had been fighting a specific failure for months: ask for a buttoned blouse with the collar closed, get a deep scoop neckline anyway. Negatives did nothing. Describing the garment three different ways did nothing. Dropping the strength on the model we suspected moved it slightly and it snapped back. When something survives five honest attempts the problem is usually not the prompt, it is the foundation.

So on the next dataset we captioned necklines in three deliberate tiers, high, mid and low, and applied each one consistently to the right images. After training, the same prompt at the same seed with only that one word changed puts the collar exactly where we asked. The fight ended, and not because we found a better negative prompt.

Decide during captioning which words you want as switches later. Anything you describe consistently is a candidate: neckline, sleeve length, camera height, time of day. Bolting control on afterward with negatives does not work anywhere near as well.

Keep every epoch, and test them properly

Training hands you a snapshot at intervals, and the last one is the default. The default is often not the best, and most people never check because checking looks tedious. It is not: same prompt, same seed, one variable, five prompts each, four minutes.

The five that separate them for us are a tight face shot (the only framing where late epochs differ much), a full body, a conservative outfit, a real room, and one control prompt with no character in it at all. That last one matters most. Prompt “a middle-aged man in a red jacket” and if an epoch is overcooked it will drag your character’s face onto a stranger. When all your epochs pass the control, the decision comes down to something smaller, and in our case it was which epoch tracked our own caption vocabulary most faithfully. That is a real reason to pick an epoch and it beats squinting at faces.

The dials that matter (both tools expose them):

• Epochs, how many times training studies your set. Too few = it never learns you; too many = it can only draw the training images.
• Rank (network dim), how much detail capacity it has. Bigger isn’t better: a face doesn’t need a huge rank, and oversized ranks memorize noise.
• Our trick: train two passes and blend, one at half strength, then test both and the blend against a fixed seed grid before committing.

The four rules we paid to learn

These cost us real re-renders. They’re yours free:

1. Never drop the stack. We once skipped the LoRA on a wide shot, reasoning she’d be small in frame. The model sent a stunt double, wrong skin, wrong everything. Identity LoRAs earn their keep at every distance; if the framing fights the LoRA, fix it with wording (“close enough to read her face”), never by dropping the stack.
2. Describe the whole outfit. Crop the prompt at the waist and the model invents pants. Different pants. Every shot. If any downstream step (video, animation) will reveal more of the body than your still shows, the prompt must describe all of it.
3. She has a front. A face LoRA refuses a back-of-head shot. It will helpfully spin a new face around to the camera. Plan shots accordingly or train back/side references into the set.
4. Balance style strength. Stacking a style LoRA too strong eats the identity (our comic style at 0.9 erased her; at 0.55 it’s her in comic form). Find the highest style strength that keeps the face, and audit every LoRA in your stack, including ones that only misbehave in combination.

Bonus rule for the modest-of-heart: hairstyle is identity. Changing her flowing gradient to a ponytail weakened recognition as much as changing the face. If you want alternate hairstyles, train them in.

Bonus: instant character sheets

Once the LoRA holds, a character sheet is just a fixed-seed grid: same seed family, one variable per row (outfits, angles, expressions, styles). We generate labeled contact sheets for every wardrobe and style decision, that’s how every look on this channel gets picked. One LoRA in, model sheet out.

The payoff

One trained file holds her identity. Everything else, outfit, scene, style, medium, is a prompt away, and she’s her in all of it. Our image-to-3D pipeline, lip-sync, and every video on the channel sit on top of this single foundation.

Watch the companion episode, How I Stay Me, she narrates her own identity recipe, stunt-double disaster included. Questions or want to show off your own character LoRA? Join us at r/aillex.

Related

• 86 failures in one published episode — every class of mistake we found making local AI video, and the checks that now catch them.