Two-Character Dialogue in LTX 2.3: LoRA Control, Relay Storyboarding, and an Audio VAE Trap

2026-11-17来源 dev.toEnglish

Two people talking in generated video is harder than one person talking for reasons that are structural, not stylistic: two identities, two voices, and an interaction that has to stay coherent across both. This release addresses each of those separately.

The graphs

Workflow Nodes What it does
Relay 剧情 + 双人 LoRA 格式提示词导演 6 Prompt authoring helper
『最优实现』双人对话增强版 56 Base two-character dialogue
『最优实现』音频驱动双人对话生产力方案 72 Audio-driven variant
视频去字幕(参考)0.5版本 循环 156 Text-removal reference graph

The audio VAE trap — check this first

In 双人对话增强版:

  • CheckpointLoaderSimple → ltx-2.3-22b-distilled-1.1.safetensors
  • LTXVAudioVAELoader → ltx-2.3-22b-dev.safetensors

The checkpoint is the distilled build. The audio VAE loader points at the dev (non-distilled) build. Those are two different sets of weights, loaded into the same graph.

In the audio-driven graph both point at the distilled build. So the two sibling workflows disagree about which audio VAE is correct.

This does not crash. LTX's audio VAE and video checkpoint are loaded independently and nothing validates that they match. The result is audio that is generated, muxed, and subtly off — you notice it as "the voice does not quite sit in the scene", not as an error.

If you only check one thing in these workflows, check that the audio VAE loader and the checkpoint loader reference the same build.

The two LoRA strategies

双人对话增强版 uses Ltx2.3-Licon-VBVR-I2V-240K-R32.safetensors at weight 1.0 — the 240K likeness build. This carries identity across the frame, which for two characters means both of them.

音频驱动双人对话生产力方案 uses LTX2.3-IC-LORA-Dual-Character.safetensors at weight 0.75.

That 0.75 is the more interesting number. A dual-character LoRA at full weight makes the two identities compete for the same facial structure — the model tries to satisfy both references and produces an average face. Backing off to 0.75 lets each reference keep more of its own character.

This is a genuinely counterintuitive setting: for two characters you use less LoRA, not more.

Relay: prompt authoring as a graph

The 6-node Relay workflow is the smallest file in the set and does the least obvious job: it constructs the dual-character prompt format. That format has structure — who speaks when, with what description — and writing it by hand is error-prone enough that the author built a helper for it.

That is a small admission worth respecting: the prompt format is complicated enough to need software.

The 156-node subtitle removal graph

The largest file in the set is a text-removal reference graph, at nearly three times the node count of the dialogue graphs.

Worth noting the asymmetry: repairing text pollution is more graph than generating two characters talking. Regeneration inside a restricted region carries more internal plumbing than generation from a prompt.

Where four-graph dialogue still breaks

  1. Audio VAE mismatch (above). The highest-value thing to fix.
  2. Which character is which. Two references and no explicit slot naming means the model can swap them between segments.
  3. Lip sync under stylised faces. A face drawn with a rendering convention has fewer landmarks for the audio to drive. Sync errors read as stiffness.
  4. 0.75 is a starting point, not a default. Different character pairs need different weights; there is no single right value.

Practical order

Fix the audio VAE to match the checkpoint, run the 56-node graph with one short exchange, and confirm the two identities stay distinct. Only then move to the audio-driven 72-node graph, where you also have to get the audio right.

Workflows

Four graphs, published on RunningHub with invite code rh-v1111:

  • Relay story + dual-character LoRA prompt director
  • Optimal-implementation LTX 2.3 two-character dialogue, enhanced
  • Optimal-implementation LTX 2.3 audio-driven two-character dialogue
  • Video subtitle removal (reference), 0.5, looped

Files

The ComfyUI workflow exports are in the companion repository:

https://huggingface.co/AI-KSK/ksk-video-230-comfyui-workflows

Video walkthrough (Chinese): see episode description
Bilibili: see episode description

Notes

  • Workflow files only. Model weights are not included; each model keeps its own licence.
  • These are tutorial exports, not a packaged product.
  • Node counts and loader values quoted above are read from the published JSON files.
  • Independent AI-KSK release. Not affiliated with or endorsed by Lightricks or RunningHub.

原文:https://www.a8q9.cn/en/article/two-character-dialogue-in-ltx-2-3-lora-control-relay-storyboarding-and-an-audio--93d2bb.html

相关文章