Type-a-prompt generation makes new pictures. Instruction editing changes the picture you already have: “change her jacket to leather, keep everything else the same.” Locally, free, no masks. This is the machine we reach for more than any other.
The tool: an instruction-edit model
Modern edit models (we run Qwen Image Edit locally in ComfyUI) take an input image + a plain-English instruction, and re-render the image with the change applied. No brush, no inpaint mask — the model figures out what “the jacket” is.
The graph reads like the basic seven boxes, with two differences: the canvas comes from your loaded image (encoded, not empty), and the prompt node also sees the image.
Two speeds, two personalities
- Lightning mode (a 4-step accelerator LoRA): seconds-fast, and conservative — great for small, surgical edits.
- Full mode (20+ steps, higher CFG): slower, braver — bigger transformations, but more drift risk.
Our working doctrine:
- Small pose/prop nudges → Lightning, with an instruction that ends: “keep her face, hair, outfit, and the background exactly the same.” That trailing clause is load-bearing.
- Big changes → full mode, in small steps. One mega-instruction (“new outfit + new pose + new lighting”) tangles; three focused edits in a row win.
- Re-pick between steps. Generate a small batch per edit, pick the best, feed it forward.
What instruction editing won’t do
Fine asymmetric expression control (a wink, a raised single brow) resists editing — the model repaints the state it believes faces have. For those, go back to generation with weighted prompt craft (singular feature + repetition — see From Zero), or composite manually.
Combining two images
Two-image combine workflows take a subject reference and a target (a pose, an outfit, a scene) and merge them. In ComfyUI these are just graphs with two Load Image nodes feeding a combine-aware prompt encoder. Uses we ship weekly:
- putting a locked character into a new scene,
- borrowing a pose from a reference photo,
- outfit transfer between renders.
Expect drift on identity-critical combines — run your character LoRA in the same graph to hold the face (the character-LoRA guide explains why that works).
The honest failure list
- Edits inherit your input’s flaws: garbage in, confidently-polished garbage out.
- “Keep everything the same” is a request, not a contract — audit hands, logos, and background text after every edit.
- Waist-crop inputs make the model invent lower bodies. Edit full-framed sources when the output needs the full frame.
Engine Room series: From Zero · Video · Upscale · 3D · Character pipeline