Just landed here? This is the second pass. If you want to know how it started — the first 24 shots rendered overnight, the three video models tried and the first draft cut — read the first pass. The whole series is listed at the end of this post.

This is the second pass at a three-minute animated pilot, made end to end on our own GPUs. No frame, voice or note in it came from a hosted service: the stills are FLUX.2 [klein] 9B, the singing is ACE-Step v1 3.5B, the speaking shots are Wan 2.2 S2V 14B, and everything ran on two RTX 5090s in Dubai and an RTX PRO 6000 in Bangalore. Below is the whole of it in the order it happened — each step, and what was wrong with it — then the finished film in four versions, the numbers each one cost, and what measuring them turned up.

What was decided, if you read nothing else: Wan 2.2 S2V renders the speaking shots; LTX-2.5 and LatentSync were dropped; a 17-billion-parameter model was tested and rejected for re-staging the shot it was given; and the single largest fault in the film turned out to be the edit, not any model — 64% of every speaking clip was silence.

What is on this page: Step 2: the grandmother's battoulah · Step 3b: the other twelve shots, redrawn · Step 4: the chant · Step 5: clips from the new stills · Step 6: the same clips from two voice-driven video models · Step 7: InfiniteTalk or Wan 2.2 S2V? Five close-ups at 720p · Step 8: does a bigger model do better? HuMo 17B · Step 9: the lip-sync test · The whole pilot, per model · How we built it: engineering notes · What is still open

Last updated 18 Sep 2026, 14:15 (Dubai). This post follows pass 2 step by step and is updated as each step finishes.

Last night we made a complete first draft of a 24-shot animated pilot, in Arabic and in English, on our own GPUs (the overnight post). Looking at it shot by shot, some faults came from the image model and some from missing pieces. Pass 2 fixes them, and this post records every step: what we asked each model for, every take it gave, which one we picked and why, how long it took, and what went wrong.

Everything still runs on self-hosted models: FLUX.2 [klein] 9B for stills, LTX-2.5 (distilled) for clips and ACE-Step v1 3.5B for singing, on an NVIDIA GeForce RTX 5090 (32 GB VRAM) and an NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB VRAM). LatentSync 1.6 was tested for lip sync and then dropped: it distorts the face.

What pass 2 fixes

• People with the wrong details. Drawn from text alone, the image model put details on the wrong person in group shots: the two children’s bags swapped in every take of shot 18, an extra person in shots 1 and 17, a second mask on the girl in shot 1. Pass 2 draws these shots from a reference picture of each character.
• The grandmother’s mask. Her battoulah was drawn like a masquerade eye mask, or with its wooden strip hanging over her mouth, which also hides her lips when she speaks. Pass 2 rewrites how it is described and draws her a new reference.
• No chant. Shots 12 and 18 are the children singing the Hag Al Laila chant, and the draft had no singing at all. Pass 2 generates it and makes the singers’ mouths follow it.
• Lip sync. The video model gets the timing of speech right but not the mouth shapes. Pass 2 tests a dedicated lip-sync model on three existing shots.

Where pass 2 stands

Step	State
Character references (Salem, Noora, Yaddi)	Done: cropped from last week's character sheets
The grandmother's new mask and reference	Expression sheet right (take 3); all 6 full-length views wrong; her reference is a portrait cropped from take 3
Shot 18: do references fix who wears what?	Yes on 2 of 6 takes; take 1 picked
Shot 17	Take 3 picked; clips rendered and reviewed
Shots 23 and 10	All 6 takes wrong in each; reworded
Shot 1 (all four family members)	Drawing from four references
Shots 23 and 10, second wording	Shot 10: take 4 picked. Shot 23: all 6 wrong again; take 3 of a third wording picked
The chant: 16 takes (8 per shot)	Shot 18 take 1 and shot 12 take 6, confirmed by ear
Clips from the new stills	Shots 17, 1, 12, 10, 18 and 23 rendered
The other twelve shots, 3 takes each	Drawn and reviewed: 3, 6, 11 and 14 picked (6, 11 and 14 on a later wording); 19 still wrong after three wordings; 20 still wrong after two wordings; 4, 7, 15, 22 and 24 picked provisionally until the mask is decided; 16 needs new wording and waits for the mask decision
The same clips from InfiniteTalk and Wan 2.2 S2V	14 of 14 rendered
Lip-sync test (LatentSync 1.6)	Dropped on 17 Sep: it distorts the face while she speaks

Step 1: a reference picture for each character

Last week each character got a turnaround sheet from FLUX.2 [klein] 9B: the same figure from the front, three quarters, the side and the back. The image model can take pictures as well as words, so pass 2 cuts the front view out of each sheet and passes it in, with the prompt saying who is who (“the boy is the person in image 1 and the girl is the person in image 2”). Each character is cut two ways: full length for the shots in the lane, and just above the feet everywhere else, so the model does not copy sandals onto the carpet indoors.

• Salem, take 2. the long thin tassel hangs from the neck as written, the mole on his left cheek; take 1 has a flat placket instead of a tassel
• Noora, take 2. the emerald covering hides all her hair; take 1 shows black hair at the forehead, which the keyframe screen forbids
• Yaddi, take 1. the long thin tassel and the silver watch on his left wrist, a warm face; take 2's face is stiffer

One thing went wrong on the way. Cutting at a fixed share of the figure’s height still left the boys’ sandals in, because the floor shadow counts as part of the figure, and the backdrop gets lighter towards the floor. The cut now finds the feet by their colour (skin and brown leather, darker than the backdrop) and stops just above them; it has to look for the lowest such band, because the girl’s hands hang low enough to be found first.

Step 2: the grandmother’s battoulah

The first description already said the mask’s lower edge ends just above her upper lip and that her mouth is uncovered, and the model still hung the wooden strip over her mouth. So the new text describes the mask’s parts and ends all of them at the tip of her nose, and never names anything we do not want drawn:

a stiff metallic gold-green battoulah face mask with a faint purple sheen: a narrow band across her eyebrows, two small eye openings that leave her eyes clear, and a short panel over her nose with a thin wooden strip along its centre; the mask and the wooden strip both end at the tip of her nose; red cotton cords tie it at the back of her head; her cheeks, upper lip, lips, mouth and chin are bare and clearly visible

A real battoulah often covers the upper lip. Ending it at the nose keeps her lips visible for lip sync; a native Emirati reviewer still has to confirm it.

What changed. Compared with photographs of the Emirati battoulah, the mask we drew was wrong. The real mask has a band across the brow, a thin strip down the centre and panels over the nose and cheeks, and many reach down over the mouth; ours came out as a gold shield over the nose only. So the grandmother’s reference is now the full-length view that looks like the photographs (take 5 below), and each of her shots is drawn again with the mask’s parts named. With her mouth covered, her lines need no lip movement.

How we got there. Should her mouth be covered? A face covering that hides the mouth, and is part of the cloth rather than a separate mask, is what many viewers picture. What our culture notes say: the battoulah (burqa’a) is an Emirati custom that some elderly women still wear every day and younger women wear for ceremony, and it is a separate stiffened mask tied with red cords, not part of the abaya. Khaleej Times describes it as “a decorative, stiffened cloth mask that covers part of the face, typically the nose and eyebrows, and sometimes the mouth”. So a mask over her mouth is authentic too. We ended it at the nose only so her lips could be lip-synced, and with LatentSync dropped that reason is weaker: behind a mask, her lines need no mouth at all. The choice is still open, and a native Emirati reviewer should confirm it.

Expression sheet: three takes

Three portraits per take: calm, smiling, and speaking, to see whether the mouth stays clear while she talks.

Full-length view: six takes

Group shots need her whole figure, so we also asked for a full-length front view. None of the six came out right: the model drew goggles, a visor, or a mask over her mouth, and two takes show hair at the forehead. Under our own rule (three takes, three more only if all are wrong, never a seventh) shot 1 now waits. The expression sheet’s take 3 has the right mask, and cropping one portrait from it is the option on the table.

The mask as an open frame: six takes

Takes 1 to 3 edit full-length take 5 and change only the mask; takes 4 to 6 draw her from the words. Take 3 is her reference now: a band across the brow, a wooden strip ending at the upper lip and panels at the cheekbones, with her mouth, jaw and chin bare.

What we did. The calm portrait from the expression sheet’s take 3 became her reference for shot 1. It shows the mask we want, and her clothes are simple enough for the model to continue from the portrait down.

Step 3: shot 18, the test that references fix who wears what

Shot 18 is the brother and sister singing side by side, each with their own bag: hers crimson, his indigo. Drawn from text, every take gave them the wrong bags. It is drawn first because it has only two references; if it had failed, the three- and four-person shots would not have been drawn this way.

What it showed. References help but do not settle it: 2 of 6 takes are right, and the other four swap clothes or draw two of the same child. Both right takes show a thin dark line of hair at the girl’s forehead, which is faintly in her reference picture too, so the next fix belongs in the reference.

Shots 17, 23, 10 and 1

Drawn from references once shot 18 passed. Shot 17 came out right on take 3. Shots 23 and 10 went wrong in all three takes, each in a new way: the boy now stands in shot 23 as asked, but the grandfather sits against an upholstered armchair under a modern lamp, and in shot 10 the boy’s head keeps appearing at the frame’s edge where only his hand should be. Takes 4 to 6 went the same way: in shot 23 every one of the six takes reads “an armrest cushion” as an upholstered armchair (take 5 at least swaps the modern lamp for a lantern), and in shot 10 the boy’s head is at the edge in all six, with handles on most of the cups, where the shot asks for small handle-less coffee cups. Our rule is three takes, three more only if all are wrong, and never a seventh, so both shots stopped here for a decision: new wording for the set and the framing, or accept the closest take.

Shot 17

Shot 23

Shot 10

Shot 1

All three wrong in the same two ways: the grandmother in green with no battoulah, and the grandfather on a sofa. So straight to a second wording rather than takes 4 to 6.

Shots 23, 10 and 1, second wording

Both shots went wrong the same way in all six takes, so we changed the words rather than drawing more. The model reads “an armrest cushion” as an armchair, so the grandfather now sits “on a thick flat floor cushion laid on the carpet against the plain wall, a firm round bolster cushion on the floor at his side”, and shot 23’s “warm lamplight” is now “a small brass lantern standing on the carpet”. In shot 10 the sentence about the boy’s hand no longer says “boy”, which kept drawing his head: “a small right hand in the cuff of a white sleeve reaches in, low, at the height of the grandfather’s knee”. And the cups, drawn with handles although the text said handle-less, are now described by their shape: “shaped like a little bowl with smooth round sides”. In shot 1 the grandmother is now described beside her reference as “dressed all in black, a black abaya and a black headscarf, with her stiff gold-green battoulah on her face”, and the grandfather sits on the same floor cushion with a bolster. Naming the thing you do not want tends to draw it.

Shot 23, second wording

All wrong again: all 6 wrong the same way (the lantern stands where the cups are and catches the pour).

Shot 23, third wording

The lantern now stands “on the carpet in the far corner behind the grandfather”, and the still shows the moment before the pour: “the pot’s long curved spout tipped right over the top cup, which he holds up close under it.” The clip still animates the pour.

Picked: take 3 of the third wording.

Shot 10, second wording

All wrong again: all 3 wrong (a handle on the grandfather's cup): takes 4-6.

Shot 1, second wording

First redraw (the solid mask).

Second redraw (the open frame).

Picked: take 1.

Step 3b: the other twelve shots, redrawn

The pilot has 24 shots. After shots 18, 17, 23, 10 and 1, the other twelve that pass 1 got wrong were drawn again the same way: FLUX.2 [klein] 9B, from the character reference pictures, three takes each, with the pass 2 wording (the grandmother dressed all in black, her mask named only as “her stiff gold-green battoulah” and drawn from her reference portrait, floor cushions instead of armrest cushions, a lantern instead of lamplight). Each take has a written verdict. Shot 12 kept its pass 1 still; shots 2, 5, 8, 9, 13 and 21 were right in pass 1.

The grandmother’s six shots: 4, 7, 15, 16, 22 and 24

How her mask came out. In all 18 takes the battoulah is drawn as a gold nose shield from under her eyes to just above her upper lip, with no band across her eyebrows, and her mouth is bare in every one. That is not what the battoulah looks like (step 2), so each shot is drawn again from the new reference, shown under its first takes.

Shot 4: The grandmother, close, has just finished sewing the girl's crimson bag; the girl's hand reaches in

Picked: take 1.

First redraw (the solid mask).

Second redraw (the open frame).

Picked: take 1.

Shot 7: The grandmother under the window holds up the boy's indigo bag, smiling

Picked: take 1.

First redraw (the solid mask).

Second redraw (the open frame).

Picked: take 1.

Shot 15: The grandmother in the doorway with the indigo bag; the boy on the doorstep, his back to us

Picked: take 2.

First redraw (the solid mask).

Second redraw (the open frame).

Picked: take 2.

Shot 16: The courtyard: the boy on the doorstep with the coffee tray, the grandmother holding out his bag

All three wrong: she stands out in the courtyard by the gate instead of in the doorway in all 3, and the cups have handles.

First redraw (the solid mask).

All wrong: she stands by the gate instead of in the doorway in all 3 takes again, with the new mask right.

Second redraw (the open frame).

All wrong: all 6 takes wrong: in the doorway the bag floats, holding the bag she stands in the courtyard.

Shot 22: Evening: the grandmother, close, happy tears, her hand on the girl's head covering

Picked: take 1.

First redraw (the solid mask).

Second redraw (the open frame).

Picked: take 1.

Shot 24: Evening: the whole family in the majlis, the grandmother filling the children's bags

Picked: take 1.

First redraw (the solid mask).

Second redraw (the open frame).

Picked: take 1.

The other six: 3, 6, 11, 14, 19 and 20

Shot 3: The girl kneels and points toward the courtyard door, mid-word

Picked: take 2.

Shot 6: The boy stands and serves coffee to the seated grandfather

All three wrong: cups with handles; hands and props split differently in every take.

Second wording. The cups are named as tiny coffee bowls ("handle-less porcelain cups" drew handles), and the boy's hands are given by distance from the grandfather instead of left and right.

Picked: take 2 of the second wording.

Shot 11: The girl walks ahead down the lane with her empty bag, the grandfather behind

All three wrong: the grandfather walks level with the girl and the bag is not flat and empty in all 3.

Second wording. The grandfather walks "far behind her, many steps back and smaller in the frame", the wording that worked in shot 20, and she swings her one bag from one hand; its strap is no longer called a shoulder strap.

Picked: take 3 of the second wording.

Shot 14: The boy alone on the doorstep, the coffee tray beside him

All three wrong: the cups on the tray have handles (takes 1 and 2 checked at full size).

Second wording. The cups are named as tiny coffee bowls.

Picked: take 2 of the second wording.

Shot 19: From behind the two children at a neighbour's gate, adult hands offering a bowl of nuts

All three wrong: duplicated bags or bowls in all 3 and the bags never held open in front.

Second wording. Each child holds up "one bag" with its mouth pulled open between both hands, and the straps are no longer called shoulder straps (the bags came out twice, one worn and one held, though no reference picture has a bag).

All wrong again: one bag each now in all 3, but in every take the children reach for the bowl with their bags hanging from their wrists instead of holding them open.

Third wording. The adult hands now tip the bowl so the sweets pour into the boy's open bag, and the children's hands stay on their bags (in the second wording both children reached for the bowl).

All wrong again: all 6 takes wrong: in every take the children hold the bowl, or small bowls of their own, and the bags hang from their wrists; the tipped bowl was drawn once.

Shot 20: Dusk: the two children walk home hand in hand with full bags, the grandfather far behind

All three wrong: the two children do not hold hands in any take.

Second wording. Which hand holds which is spelled out ("her right hand holds his left hand"), each child carries "one bag", and the straps are no longer called shoulder straps.

All wrong again: all 6 takes wrong: the hand-holding came out once, with the grandfather's face on the boy, and the extra takes added children or doubled the bag.

Step 4: the chant

On Hag Al Laila, children go from door to door singing for sweets. The pilot uses one verified couplet, word for word:

عطونا الله يعطيكم، بيت مكة يوديكم

Give us, and God will give to you; may He take you to His House in Makkah.

A text-to-speech model cannot sing, so the chant comes from ACE-Step v1 3.5B, a self-hosted music model. We learned one thing about it on the way: it reads a language tag at the start of every lyric line and treats an untagged line as English, so every Arabic line starts with [ar]. Each take is 120 seconds, of which shot 18 uses about 11 and shot 12 about 4. Settings: 60 steps, cfg 3.5, shift 5.0, lyrics strength 0.99.

We checked each take with Whisper (faster-whisper small, on the CPU, forced to Arabic). It reads sung Gulf dialect loosely, so it spells the right words wrongly: take 1 of shot 18 comes back as عبتونا الله يا تيكوم بيتد مقا يوديكوم, which is the couplet’s shape. The words below are what it heard that are not the chant’s exact spelling; the pick is by ear. None of these takes has been checked by a native speaker yet.

A provisional pick, so the clips can move. Take 1 of shot 18 is regular: with word timings Whisper hears the couplet’s shape in every pass, and it sings it twice from 20.7 s to a clear break at 36.0 s. That is 15.3 seconds, and the shot has room for about 11.6, so the stretch is sped up 1.32 times with the pitch kept. Shot 12’s four remaining takes re-sing this same take solo, so both chant shots share one tune. If another take sounds better by ear, the clips are rendered again from it.

How the provisional picks were made, exactly. Nobody listened first: the picks were made from the transcripts alone. The screen scored each take three ways: how many of the couplet’s six words Whisper heard spelled exactly, how many words it heard that are not in the chant, and the shortest stretch holding the couplet twice (shot 18, at most 11 s) or its first half-line once (shot 12, at most 4 s). That automatic search found no stretch in any take (all 16 screened, shot 12’s takes 5 to 8 later the same evening), because Whisper’s spellings of sung words never matched the lyric exactly. So the word timings were read by hand. Take 1 repeats the same line, the couplet’s shape (seven words as Whisper splits them), pass after pass. A correction, found in our own audit the same evening: this paragraph first said the highest word score, shot 18’s take 5 at 4 of 6, came from a take stuck repeating one word. That was wrong. Take 5 is the clearest transcript of all: Whisper hears عطونا الله يعطيكم بيت مكيوديكم, the couplet with its last two words run together, ten times at about 7.5 s a pass, and its ten words “not in the chant” are that one run-together word. The provisional pick went to take 1 from its word timings, and take 5 was not compared with it by ear. For shot 12, as planned, takes 5 to 8 re-sing shot 18’s take 1 as a solo (audio-to-audio, seeds 1 and 2, denoise 0.35 and 0.5), so both chant shots share one tune. Take 6 has the half-line after a pause, from 6.4 to 9.9 s, with its first word spelled right, and its 3.5 seconds fit the 4-second window without speeding up. Melody, voices and claps were left for the ear.

By ear. Both picks were then listened to and kept. Shot 18’s take 5 was compared with take 1 by ear too; it is also good, and take 1 stays.

Shot 18: brother and sister together

Two children in unison, a cappella, hand claps only.

Shot 12: the girl alone

A tiny, shy voice that falters. Four more takes, re-singing the picked shot 18 tune solo, come after the pick.

Step 5: clips from the new stills

LTX-2.5 (distilled) at 1920×1088 and 24 fps, one render per clip from the picked still. Where someone speaks or sings, the voice or chant is held in the model’s audio input so it drives the mouth, the fix we found last night. Each shot in Arabic and English is its own render.

A publishing bug, fixed 17 Sep. The first copies of these clips on this page had no sound. The clips themselves always carried the voice or the chant, but the script that shrinks them for the web dropped the sound track unless it was handed a separate one. The web copies now keep the clip’s own sound. Shot 1 is silent by design.

Shot 18

Shot 17

Both languages: exactly three people from first frame to last (pass 1's extra old man is gone), bags and clothes right, Salem's mouth works through his line and the two languages move differently. Faults: the camera slowly pushes in although the prompt says the framing stays fixed; Noora's mouth opens in surprise where she should smile with it closed; Yaddi looks startled by the end rather than smiling and nodding.

Shot 23

First wording, both languages: the pour, the sip and the cup handed back are right, and the lantern stays in its corner. Fault: from about 10 s a second boy appears and the camera ends on a close-up of two boys, despite "exactly two people" in the prompt — pass 1's fault again. The prompt told the already standing boy to kneel beside the grandfather, and LTX acted it out with a new boy; the grandfather's stick also turns into a key-like shape. Re-rendered with a second motion wording: the boy bends forward instead of kneeling, and the camera holds the wide two-shot.

Second motion wording ("bends forward toward him"), both languages, checked on 8 frames each: the wide two-shot holds for about 10 s with the grandfather seated on his floor cushion and one boy, the pot, cups and lantern in place. Then LTX cuts inside the clip. Arabic: a dissolve at about 10 s into a close-up where the grandfather's hand rests on the boy's head, and a second boy is back beside him to the end. English: one boy throughout, but a cut to a close-up of him smiling at about 11.5 s, the grandfather out of frame; a walking stick also stands upright on its own on the carpet, and the boy drinks from a cup at about 3 s. Not picked: neither language holds the wide shot to the end.

Shot 10

Both languages: only the boy's hand reaches in at the right edge (no adult arm, no head), the grandfather hands over the cup and speaks. Fault: "leans forward on the stick to begin to rise" made LTX stand him up fully by 8 s, and the camera reframed to follow him although the framing should stay fixed, as pass 1's clip also did. Re-rendered with a second motion wording in which he leans forward a little and stays seated to the end. Second wording (wlf_p2_clip_s10_m2_ar/en): he stays seated to the end in both languages, the cup handover and the speech are right; used for the cut. Still wrong: the camera pushes in over the last 3 s although the framing should stay fixed (LTX does this on several pilot shots), and in the Arabic version his stick ends up in front of his mouth.

Shot 1

Silent, once for both languages. Four people from first frame to last, the camera pushes in toward the head of the room as asked, and the grandmother's battoulah stays rigid and in place. Fault: the grandfather turns an object over in his hands instead of resting them on his knees.

Shot 12

With the provisional solo chant held in the audio input: the girl's mouth opens with the singing around 2.5-3.5 s, closes after it, and she lowers her eyes as she falters, as the script asks. Fault: the framing drifts a little although it should stay fixed.

Step 6: the same clips from two voice-driven video models

With LatentSync dropped, the question is whether another video model matches mouths to the voice better than LTX-2.5, especially in Arabic. The other two video models in our first test, Wan 2.2 I2V and HunyuanVideo 1.5, take no audio, so their mouths cannot follow a voice. These two do: they take the voice as an input and animate the picture to it.

• InfiniteTalk (MeiGen, Apache 2.0), an audio adapter on Wan 2.1 I2V 14B (fp8), with the lightx2v speed LoRA: 6 steps, cfg 1, 832×480 at 25 fps. Clips longer than 81 frames are made in windows, each continuing from the last 9 frames of the one before.
• Wan 2.2 S2V 14B (Alibaba, Apache 2.0, fp8), with the Wan 2.2 lightx2v 4-step LoRA: 4 steps, cfg 1, shift 8, 832×480 at 16 fps, in 77-frame windows chained by its extend node.
• The same inputs as LTX: the same picked still, the same voice or chant file that LTX held in its audio input, and LTX’s prompt word for word; seed 1. Steps, cfg, LoRAs, samplers and window lengths are the ones in ComfyUI’s own template for each model. Both run at 832×480 so the clips compare (S2V’s template uses 640×640), and InfiniteTalk also gets a CLIP vision encoding of the still, as MeiGen’s own pipeline does. LTX’s clips are 1920×1088, so compare mouths and movement, not sharpness.
• Their audio encoders are wav2vec2 models: a Chinese one for InfiniteTalk, and for S2V the multilingual XLSR-53 model fine-tuned on English. How well that carries to Arabic is what these clips show.

A measured score: SyncNet

SyncNet v2, the standard lip-sync measure (from LatentSync’s evaluation code), tracks each face, crops it and scores how well the mouth movement matches the audio at the best offset. Higher confidence and lower distance are better. It was trained on real faces, so its absolute numbers on stylised 3D characters mean little; the comparison is the point. All clips were scaled to 832×480 first and given the same audio file.

• Only the spoken stretch. Each line is short: in shot 17 the voice runs from about 1.6 s to 3.3 s (Arabic) or 3.8 s (English) of 7 seconds, in shot 23 from 7.0 to 9.6 s of 13. Our first run scored whole clips, which is mostly silence in both voice files, and one row came out identical for both voices from a face that was only on screen during the silent end. So each clip is scored on the stretch where its voice is heard, plus half a second either side.
• The control. The same face, over the same stretch, is scored again with the other language’s voice. A model whose mouth follows the voice scores clearly higher on its own voice.
• Which face. The speaker is taken as the face with the highest own-voice score among faces on screen for most of the stretch.

Clip	Model	Spoken stretch	Confidence, own voice	Distance, own voice	Confidence, same face, other language’s voice (control)	Offset (frames)
Shot 17, Arabic	LTX-2.5 (distilled)	1.1-3.8 s	0.82	13.56	1.10	+0
Shot 17, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	1.1-3.8 s	4.90	8.82	1.49	-2
Shot 17, Arabic	Wan 2.2 S2V 14B	1.1-3.8 s	5.08	8.94	1.83	+0
Shot 17, English	LTX-2.5 (distilled)	1.1-4.3 s	0.72	13.38	0.79	+0
Shot 17, English	InfiniteTalk (on Wan 2.1 I2V 14B)	1.1-4.3 s	4.11	10.87	1.68	-2
Shot 17, English	Wan 2.2 S2V 14B	1.1-4.3 s	3.33	10.99	1.38	+0
Shot 10, Arabic	LTX-2.5 (distilled)	0.6-4.9 s	2.19	12.77	2.70	-2
Shot 10, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	0.6-4.9 s	4.15	10.82	1.87	-3
Shot 10, Arabic	Wan 2.2 S2V 14B	0.6-4.9 s	2.28	11.74	2.07	+1
Shot 10, English	LTX-2.5 (distilled)	0.7-4.5 s	4.89	10.54	1.19	-3
Shot 10, English	InfiniteTalk (on Wan 2.1 I2V 14B)	0.7-4.5 s	6.34	8.76	1.11	-1
Shot 10, English	Wan 2.2 S2V 14B	0.7-4.5 s	4.25	10.78	1.04	+1
Shot 23, Arabic	LTX-2.5 (distilled)	6.6-10.2 s	1.41	12.57	1.11	-15
Shot 23, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	6.6-10.2 s	3.14	10.74	1.69	-2
Shot 23, Arabic	Wan 2.2 S2V 14B	6.6-10.2 s	1.02	11.66	0.60	+1
Shot 23, English	LTX-2.5 (distilled)	6.5-10.1 s	3.61	10.54	1.89	-4
Shot 23, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.5-10.1 s	5.04	9.64	2.47	-2
Shot 23, English	Wan 2.2 S2V 14B	6.5-10.1 s	1.85	12.28	1.19	-12
Shot 12, chant	LTX-2.5 (distilled)	0.5-5.0 s	5.17	7.59	–	-4
Shot 12, chant	InfiniteTalk (on Wan 2.1 I2V 14B)	0.5-5.0 s	5.68	7.65	–	-3
Shot 12, chant	Wan 2.2 S2V 14B	0.5-5.0 s	4.26	8.33	–	+2

In short:

• LTX-2.5 (distilled): average confidence 2.68 with its own voice over 7 clips; on the 6 spoken clips, 2.27 with its own voice against 1.46 with the other language’s, and its own voice scored higher in 3 of 6
• InfiniteTalk (on Wan 2.1 I2V 14B): average confidence 4.77 with its own voice over 7 clips; on the 6 spoken clips, 4.61 with its own voice against 1.72 with the other language’s, and its own voice scored higher in 6 of 6
• Wan 2.2 S2V 14B: average confidence 3.15 with its own voice over 7 clips; on the 6 spoken clips, 2.97 with its own voice against 1.35 with the other language’s, and its own voice scored higher in 6 of 6

Shot 17, Arabic

Checked on six frames per model across the 7 seconds. LTX-2.5 slowly pushes the camera in, although the framing should stay fixed. InfiniteTalk and Wan 2.2 S2V both hold the framing, with the same three people, bags and gate throughout, but both freeze the running boy mid-stride: his legs stay apart and only his head, face and arm move. InfiniteTalk's faces are the more expressive (the girl smiles and blinks); S2V's change least.

Shot 17, English

Checked on six frames per model across the 7 seconds. LTX-2.5 slowly pushes the camera in, although the framing should stay fixed. InfiniteTalk and Wan 2.2 S2V both hold the framing, with the same three people, bags and gate throughout, but both freeze the running boy mid-stride: his legs stay apart and only his head, face and arm move. InfiniteTalk's faces are the more expressive (the girl smiles and blinks); S2V's change least.

Shot 10, Arabic, second motion wording

Checked on six frames per model across the 10 seconds. All three keep the grandfather seated and only a hand at the right edge. LTX-2.5 pushes in to a close-up, and the walking stick appears in his hand in front of his face. InfiniteTalk holds the framing, but he lifts the stick diagonally across himself and a second cup appears on the floor. Wan 2.2 S2V is the calmest: he holds his cup and talks, the stick stays on the floor, with a blur on his hand for a moment.

Shot 10, English, second motion wording

Checked on six frames per model across the 10 seconds. All three keep the grandfather seated and only a hand at the right edge. LTX-2.5 has him pick up the stick, uncross his legs for a moment and pushes in toward his face. InfiniteTalk holds the framing, but he lifts the stick diagonally across himself and a second cup appears on the floor, as in the Arabic clip. Wan 2.2 S2V is the calmest: he talks with his cup, sets it on the floor and picks it up again, and the stick stays down.

Shot 23, Arabic, second motion wording

Checked on six frames per model across the 13 seconds. LTX-2.5 cuts to a close-up for the last seconds (in the Arabic clip with a second boy). InfiniteTalk and Wan 2.2 S2V both hold the wide two-shot to the last frame with exactly two people. In InfiniteTalk the boy bends down to the tray and straightens again; in S2V he pours from the pot toward the grandfather, at one point with the spout close to the grandfather's face.

Shot 23, English, second motion wording

Checked on six frames per model across the 13 seconds. LTX-2.5 cuts to a close-up for the last seconds (in the Arabic clip with a second boy). InfiniteTalk and Wan 2.2 S2V both hold the wide two-shot to the last frame with exactly two people. In InfiniteTalk the boy bends down to the tray and straightens again; in S2V he pours from the pot toward the grandfather, at one point with the spout close to the grandfather's face.

Shot 12, the chant

Checked on six frames per model across the 7 seconds. LTX-2.5's girl moves most: her mouth opens with the singing, the bag swings, and the framing drifts a little. InfiniteTalk keeps the framing, opens her mouth with the singing around 3-4 s and lowers her eyes at the end, as the script asks. Wan 2.2 S2V keeps the framing, with the smallest mouth and face movement of the three.

Step 7: InfiniteTalk or Wan 2.2 S2V? Five close-ups at 720p

The answer, decided by eye on the clips below and then on the whole film: Wan 2.2 S2V. InfiniteTalk scores higher on the lip-sync measure (6.07 against 4.57 on the same ten clips) and its faces are livelier, but S2V holds the composition of the still it is given, never moves a mouth where nothing is said, and renders the same film in 176 minutes of GPU time against InfiniteTalk’s 433. It is now the model for the series, not just for this pilot.

LTX-2.5 is out for speaking shots. It scored worst in the table above, and worst of all in Arabic.

The pass 2 clips were medium and wide shots at 480p, where mouths are small. The deciding round is made the way delivery would be: 1280×720, on the pilot’s five single-speaker close and medium-close shots (3, 5, 8, 9 and 13), each in Arabic and English, from the picked still and the shot’s own voice, with the same template settings as before (InfiniteTalk on its 720p base). Rendered on the Dubai RTX 5090s, which from today render clips only.

Shot 3

Shot 5

Shot 8

Shot 9

Shot 13

Step 8: does a bigger model do better? HuMo 17B

Both models above are 14 billion parameters, and a 32 GB card can hold more than that. The largest voice-driven video model with open weights we can run is HuMo 17B, so it was worth the test: the same five close-ups, the same stills, the same voices, at 1280×720. Settings are ComfyUI’s own template for it — the 17B model at fp8, whisper large v3 as the audio encoder instead of wav2vec, the lightx2v speed LoRA, six steps at cfg 1 on uni_pc, shift 8, 25 fps. It renders 97 frames, which is 3.9 seconds, in 5.5 min on one RTX 5090.

Three things came out of it, and none of them favour the bigger model.

1. It makes one window and cannot continue. The node has no extend, so 3.9 seconds is the whole clip. Most shots in this pilot run 5 to 13 seconds, so it cannot render them at all, which is why there is no HuMo version of the full film.
2. It re-frames the shot instead of continuing the still. Given the same keyframe as the others, it treats it as a likeness reference and composes its own picture: shot 3 pushes from a wide kneeling shot into a close-up against a carved door, shot 13 loses the lane, shot 9 loses the room. For a pilot where the still IS the approved composition, that is disqualifying on its own — the shot that comes back is not the shot that was signed off.
3. It invents people. In shot 8 an enormous human face fills the right of the frame, unprompted and out of scale with the boy.

Clip	Model	Own voice	Same face, other language
Shot 3, Arabic	HuMo 17B	4.41	2.11
Shot 3, Arabic	Wan 2.2 S2V 14B	4.25	2.41
Shot 3, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.10	3.10
Shot 3, English	HuMo 17B	3.97	2.32
Shot 3, English	Wan 2.2 S2V 14B	6.54	1.98
Shot 3, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.01	1.64
Shot 5, Arabic	HuMo 17B	5.27	1.78
Shot 5, Arabic	Wan 2.2 S2V 14B	3.74	3.02
Shot 5, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	6.16	3.27
Shot 5, English	HuMo 17B	3.23	1.36
Shot 5, English	Wan 2.2 S2V 14B	5.04	2.05
Shot 5, English	InfiniteTalk (on Wan 2.1 I2V 14B)	4.29	1.67
Shot 8, Arabic	HuMo 17B	3.04	3.74
Shot 8, Arabic	Wan 2.2 S2V 14B	3.12	2.55
Shot 8, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.73	3.30
Shot 8, English	HuMo 17B	5.13	1.78
Shot 8, English	Wan 2.2 S2V 14B	4.41	1.28
Shot 8, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.28	2.04
Shot 9, Arabic	HuMo 17B	5.08	3.17
Shot 9, Arabic	Wan 2.2 S2V 14B	4.73	1.84
Shot 9, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	7.28	3.44
Shot 9, English	HuMo 17B	6.50	3.15
Shot 9, English	Wan 2.2 S2V 14B	6.05	2.15
Shot 9, English	InfiniteTalk (on Wan 2.1 I2V 14B)	7.97	2.17
Shot 13, Arabic	HuMo 17B	4.75	2.25
Shot 13, Arabic	Wan 2.2 S2V 14B	4.08	2.16
Shot 13, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.56	2.94
Shot 13, English	HuMo 17B	5.00	2.07
Shot 13, English	Wan 2.2 S2V 14B	3.73	1.28
Shot 13, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.35	1.32

Shot 3, HuMo 17B

Shot 5, HuMo 17B

Shot 8, HuMo 17B

Shot 9, HuMo 17B

Shot 13, HuMo 17B

Is four seconds the limit?

Not in the code: the node takes any length in multiples of four, up to 16,384 frames. It is the limit the model was trained to. HuMo’s own model card says the model is trained on 97-frame videos at 25 frames a second and that generating longer than 97 frames may degrade the performance, and the repository has no sliding window or extend mode at all, with a checkpoint for longer generation promised but not released. Wan 2.2 S2V chains 77-frame windows through an extend node and InfiniteTalk carries nine motion frames into the next window, so both can run as long as the shot needs; HuMo has to do it in one pass.

So we asked it for double. Shot 13 was rendered again at 193 frames — 7.7 seconds, twice the trained length, in one pass at 1280×720. It fits: 27 GB of the card’s 32 GB, and 16.6 min a clip against 5.5 min for 97 frames, so double the length costs three times the time. The picture holds all the way through: no melting, no collapse, no repeated stretch, and the mouth keeps moving to the end. What moves is the sync, and not by much: the score against its own voice goes Arabic from 4.75 to 3.08 and English from 5.00 to 5.12, while the control on the wrong voice stays far below it either way. Four seconds is a recommendation, not a wall.

Why the bigger model was not chosen

Its pictures are not the problem. Frame for frame HuMo draws as well as anything else here, and its lip sync measures level with the model we shipped: 4.64 against its own voice and 2.37 against the wrong one on these ten clips, where Wan 2.2 S2V scores 4.57 against 2.07. On some shots its version looks better, because it moves in closer than the still did. Four things still rule it out, in the order that decides it.

1. It will not keep the shot it is given. A pilot is approved still by still: the composition, the distance, what is in frame behind the character. HuMo treats that still as a likeness reference and re-stages the picture, so what comes back is a different shot of the same character. A model that redirects the film cannot be the one that renders it, however good the frame looks on its own.
2. It cannot continue a clip. Four seconds is one pass and there is no way to carry the last frames into the next window, which the other two models both do. Most shots in this pilot are longer than four seconds.
3. It adds people. An enormous unprompted face fills a third of shot 8. Extra characters are the fault we have spent the most redraws on, and a model that introduces them at render time would put that back after the stills were finally right.
4. It costs more per second of screen. 5.5 min for 3.9 seconds, against 4.2 min for a 5-second shot from the model we chose — and one film is 294 seconds of rendered footage, 188 seconds of screen with every speaking shot rendered twice, once per language.

What it settles: the parameter count was not the limit. A 14B model that keeps the composition it is given and can be extended window after window is worth more here than a 17B one that re-stages the shot and stops at four seconds. The useful thing to take from this is which axis mattered: control, not capability.

Step 9: the lip-sync test

LTX-2.5 moves the mouth at the right times but not in the right shapes. LatentSync 1.6 repaints the lower face of an existing clip to match a voice. We test it on three pilot shots, with unmasked faces first: 9 (the girl, close up), 13 (the grandfather, medium, with a beard) and 22 (the grandmother in her mask, close up), each in Arabic and English. It runs on an NVIDIA GeForce RTX 5090 (32 GB VRAM) at 20 steps, guidance 1.5.

Installing it on Windows took two changes. Its requirements pin a PyTorch build too old for this card, so it runs on PyTorch 2.8 with CUDA 12.8. And its face detector, insightface, is published only as source code with one C++ module that needs a compiler; that module is only used for 3D mesh rendering, so we installed insightface without it and put a stand-in in its place that raises an error if anything ever calls it. Face detection never does.

What we saw first. It found every face, including the stylised 3D ones and the grandmother behind her mask, and it redraws only the lower face. On the girl (shot 9) the mouth shapes are clearly redrawn; on the grandmother (shot 22) the changes are smaller, and it does not fix the mask’s strip reaching her lip, which is in the original clip.

The shaking, and the fix

Watching the first outputs, we saw the face shake. LatentSync finds the face again in every frame, rounds its landmarks to whole pixels, cuts the face out and pastes it back, so small differences from frame to frame move the pasted face. The option it calls smoothing only averages a sub-pixel offset of the nose. On top of that the crop is calculated in half precision, and the soft edge of the pasted area is recomputed every frame. Several people report the same shaking on the project’s issue tracker, with no fix released.

We measured it: for each frame, how far the pasted face sits from the original, on a band across the nose and on a second band across the brows, then how much that changes from one frame to the next. The first idea, shifting each frame back by the measured amount, fixed the band it was measured on and made the other worse, because the face also turns and changes size a little. What worked was fixing it inside LatentSync: find the face in every frame first, smooth that track over 15 frames (separately either side of any hard cut), then cut out and paste back with the smoothed positions, at full precision, through one fixed soft mask around the redrawn lower face. The model itself is unchanged.

Clip	Nose band, typical shift (px)	Brow band, typical shift (px)	Whole face, largest shift (px)	Same mouth movement as before the fix (correlation)
Shot 9, Arabic	0.37 → 0.02	0.11 → 0.03	0.82 → 0.26	0.99
Shot 9, English	0.34 → 0.03	0.14 → 0.03	0.92 → 0.32	0.99
Shot 13, Arabic	0.43 → 0.04	0.22 → 0.04	1.40 → 0.21	0.99
Shot 13, English	0.43 → 0.03	0.26 → 0.04	1.59 → 0.37	0.99
Shot 22, Arabic	0.41 → 0.02	0.40 → 0.04	1.14 → 0.13	0.99
Shot 22, English	0.35 → 0.02	0.29 → 0.03	1.06 → 0.13	0.99

Shot 9, the girl, Arabic

Shot 9, the girl, English

Shot 13, the grandfather, Arabic

Shot 13, the grandfather, English

Shot 22, the grandmother, Arabic

Shot 22, the grandmother, English

How long each job took

Execution time on the GPU from the ComfyUI history, excluding queueing. The first still on a card includes loading the model.

Job	Count	Fastest	Median	GPU
Grandmother's reference sheets	15	3 s	4 s	NVIDIA GeForce RTX 5090 (32 GB VRAM) and NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB VRAM)
Shot 18 stills	6	6 s	29 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
Shot 17 stills	3	16 s	29 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
Shot 23 stills	15	6 s	8 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
Shot 10 stills	12	5 s	8 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
Shot 1 stills	12	6 s	10 s	NVIDIA GeForce RTX 5090 (32 GB VRAM) and NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB VRAM)
The other twelve shots' stills	111	3 s	6 s	NVIDIA GeForce RTX 5090 (32 GB VRAM) and NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB VRAM)
Chant takes, shot 18	8	9 s	10 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
Chant takes, shot 12	8	9 s	35 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
LTX-2.5 clips	9	8.1 min	12.0 min	NVIDIA GeForce RTX 5090 (32 GB VRAM) and NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB VRAM)
InfiniteTalk clips, 480p	7	3.6 min	4.8 min	NVIDIA GeForce RTX 5090 (32 GB VRAM)
Wan 2.2 S2V clips, 480p	7	69 s	101 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
InfiniteTalk clips, 720p	2	11.6 min	11.7 min	NVIDIA GeForce RTX 5090 (32 GB VRAM)
Wan 2.2 S2V clips, 720p	19	4.1 min	4.3 min	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16_s09_ar	1	3.2 min	3.2 min	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16_s09_en	1	96 s	96 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16_s13_ar	1	96 s	96 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16_s13_en	1	93 s	93 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16_s22_ar	1	2.9 min	2.9 min	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16_s22_en	1	2.8 min	2.8 min	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16sm_s09_ar	1	81 s	81 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16sm_s09_en	1	81 s	81 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16sm_s13_ar	1	81 s	81 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16sm_s13_en	1	79 s	79 s	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16sm_s22_ar	1	2.4 min	2.4 min	NVIDIA GeForce RTX 5090 (32 GB VRAM)
wlf_p2_lips_ls16sm_s22_en	1	2.4 min	2.4 min	NVIDIA GeForce RTX 5090 (32 GB VRAM)

The whole pilot, per model

All 24 shots in Arabic and English at 1280×720, cut into the complete 3-minute pilot on the finished soundtrack, one film per video model. Speaking shots get their language’s voice, the chant shots the chant, the rest silence. Shots 16, 19 and 20 use their closest take, since no take of them is right yet; shots 5 and 21 still use pass 1 stills with the old mask. A black stretch is a shot not rendered yet.

The last finding: the film was cut to the wrong lengths

Watching the finished films, the pauses between lines are the first thing you notice. The obvious suspicion is the model — a voice-driven renderer handed a stretch of silence inventing mouth movement to fill it. Measuring it says otherwise, and the answer is duller and more useful: 64% of every speaking clip is silence, 83 seconds of voice against 147 seconds of nothing, and it is in the input before any model sees it. The shots were written two to five times longer than the lines written for them, and each voice track is padded to its shot. Shot 23 is the extreme: 13 seconds of clip for 2.6 seconds of speech, with one 7.1-second gap.

No re-rendering was needed to fix it. A clip’s mouth is locked to the voice at the offset the model was given, so cutting the window that holds the line keeps the sync frame for frame — and the window is not always the head of the clip: shot 23’s line sits at 7.0 to 9.6 seconds, so its cut starts at 6.6. Silent shots keep their length; the film still has to breathe. The result is 129 seconds against 188, with 59 seconds of dead air gone, from exactly the same renders.

Re-cutting turned up two things that were simply missing from every film above. The chant was not in them at all — shots 12 and 18 measured -91 dB, digital silence, although twelve takes had been sung and one picked; the soundtrack was the pass 1 dialogue master, which was made before the chant existed. Nor was the music bed. Both are in the cut below, the music side-chain ducked under the dialogue and the chant.

What the films cost to render

Every clip of every film, the card it rendered on and how long it took. Clip renders go to the Dubai RTX 5090s first and spill to the Bangalore RTX PRO 6000 when they are full; stills never run on the Dubai cards, which render clips around the clock.

Wan 2.2 S2V 14B: 38 clips, 176.2 min of GPU time on RTX 5090 (Dubai). The slowest was shot 18 (one version for both languages) at 7.0 min; the median clip took 4.2 min.

Where in the film	Shot	Version	GPU	Render time
0:00–0:07	Shot 1	both	RTX 5090 (Dubai)	4.1 min
0:07–0:13	Shot 2	both	RTX 5090 (Dubai)	4.1 min
0:13–0:18	Shot 3	Arabic	RTX 5090 (Dubai)	4.2 min
0:13–0:18	Shot 3	English	RTX 5090 (Dubai)	4.3 min
0:18–0:27	Shot 4	Arabic	RTX 5090 (Dubai)	4.3 min
0:18–0:27	Shot 4	English	RTX 5090 (Dubai)	4.3 min
0:27–0:32	Shot 5	Arabic	RTX 5090 (Dubai)	4.2 min
0:27–0:32	Shot 5	English	RTX 5090 (Dubai)	4.2 min
0:32–0:41	Shot 6	both	RTX 5090 (Dubai)	4.2 min
0:41–0:46	Shot 7	Arabic	RTX 5090 (Dubai)	4.1 min
0:41–0:46	Shot 7	English	RTX 5090 (Dubai)	4.1 min
0:46–0:54	Shot 8	Arabic	RTX 5090 (Dubai)	4.3 min
0:46–0:54	Shot 8	English	RTX 5090 (Dubai)	4.2 min
0:54–0:59	Shot 9	Arabic	RTX 5090 (Dubai)	4.2 min
0:54–0:59	Shot 9	English	RTX 5090 (Dubai)	4.2 min
0:59–1:09	Shot 10	Arabic	RTX 5090 (Dubai)	6.4 min
0:59–1:09	Shot 10	English	RTX 5090 (Dubai)	6.4 min
1:09–1:17	Shot 11	both	RTX 5090 (Dubai)	4.1 min
1:17–1:24	Shot 12	both	RTX 5090 (Dubai)	4.4 min
1:24–1:29	Shot 13	Arabic	RTX 5090 (Dubai)	4.1 min
1:24–1:29	Shot 13	English	RTX 5090 (Dubai)	4.1 min
1:29–1:37	Shot 14	both	RTX 5090 (Dubai)	4.1 min
1:37–1:45	Shot 15	Arabic	RTX 5090 (Dubai)	4.1 min
1:37–1:45	Shot 15	English	RTX 5090 (Dubai)	4.1 min
1:45–1:53	Shot 16	both	RTX 5090 (Dubai)	4.1 min
1:53–2:00	Shot 17	Arabic	RTX 5090 (Dubai)	4.2 min
1:53–2:00	Shot 17	English	RTX 5090 (Dubai)	4.4 min
2:00–2:12	Shot 18	both	RTX 5090 (Dubai)	7.0 min
2:12–2:20	Shot 19	both	RTX 5090 (Dubai)	4.1 min
2:20–2:28	Shot 20	Arabic	RTX 5090 (Dubai)	4.1 min
2:20–2:28	Shot 20	English	RTX 5090 (Dubai)	4.1 min
2:28–2:35	Shot 21	Arabic	RTX 5090 (Dubai)	4.6 min
2:28–2:35	Shot 21	English	RTX 5090 (Dubai)	4.4 min
2:35–2:45	Shot 22	Arabic	RTX 5090 (Dubai)	6.7 min
2:35–2:45	Shot 22	English	RTX 5090 (Dubai)	6.3 min
2:45–2:58	Shot 23	Arabic	RTX 5090 (Dubai)	6.6 min
2:45–2:58	Shot 23	English	RTX 5090 (Dubai)	6.5 min
2:58–3:07	Shot 24	both	RTX 5090 (Dubai)	4.2 min

InfiniteTalk (on Wan 2.1 I2V 14B): 38 clips, 432.7 min of GPU time on RTX 5090 (Dubai) and RTX PRO 6000. The slowest was shot 10 (English) at 15.6 min; the median clip took 11.4 min.

Where in the film	Shot	Version	GPU	Render time
0:00–0:07	Shot 1	both	RTX 5090 (Dubai)	11.4 min
0:07–0:13	Shot 2	both	RTX 5090 (Dubai)	7.6 min
0:13–0:18	Shot 3	Arabic	RTX 5090 (Dubai)	7.8 min
0:13–0:18	Shot 3	English	RTX 5090 (Dubai)	7.8 min
0:18–0:27	Shot 4	Arabic	RTX 5090 (Dubai)	15.5 min
0:18–0:27	Shot 4	English	RTX PRO 6000	12.2 min
0:27–0:32	Shot 5	Arabic	RTX 5090 (Dubai)	8.0 min
0:27–0:32	Shot 5	English	RTX 5090 (Dubai)	7.9 min
0:32–0:41	Shot 6	both	RTX 5090 (Dubai)	15.6 min
0:41–0:46	Shot 7	Arabic	RTX 5090 (Dubai)	7.9 min
0:41–0:46	Shot 7	English	RTX 5090 (Dubai)	8.2 min
0:46–0:54	Shot 8	Arabic	RTX 5090 (Dubai)	11.7 min
0:46–0:54	Shot 8	English	RTX 5090 (Dubai)	11.6 min
0:54–0:59	Shot 9	Arabic	RTX 5090 (Dubai)	7.6 min
0:54–0:59	Shot 9	English	RTX 5090 (Dubai)	7.9 min
0:59–1:09	Shot 10	Arabic	RTX 5090 (Dubai)	15.5 min
0:59–1:09	Shot 10	English	RTX 5090 (Dubai)	15.6 min
1:09–1:17	Shot 11	both	RTX 5090 (Dubai)	11.6 min
1:17–1:24	Shot 12	both	RTX 5090 (Dubai)	11.4 min
1:24–1:29	Shot 13	Arabic	RTX 5090 (Dubai)	8.0 min
1:24–1:29	Shot 13	English	RTX 5090 (Dubai)	7.7 min
1:29–1:37	Shot 14	both	RTX 5090 (Dubai)	11.5 min
1:37–1:45	Shot 15	Arabic	RTX 5090 (Dubai)	11.4 min
1:37–1:45	Shot 15	English	RTX 5090 (Dubai)	11.4 min
1:45–1:53	Shot 16	both	RTX 5090 (Dubai)	11.4 min
1:53–2:00	Shot 17	Arabic	RTX 5090 (Dubai)	11.4 min
1:53–2:00	Shot 17	English	RTX 5090 (Dubai)	11.4 min
2:00–2:12	Shot 18	both	RTX PRO 6000	15.3 min
2:12–2:20	Shot 19	both	RTX 5090 (Dubai)	11.4 min
2:20–2:28	Shot 20	Arabic	RTX 5090 (Dubai)	11.4 min
2:20–2:28	Shot 20	English	RTX 5090 (Dubai)	11.4 min
2:28–2:35	Shot 21	Arabic	RTX 5090 (Dubai)	11.4 min
2:28–2:35	Shot 21	English	RTX 5090 (Dubai)	11.4 min
2:35–2:45	Shot 22	Arabic	RTX PRO 6000	12.2 min
2:35–2:45	Shot 22	English	RTX 5090 (Dubai)	15.3 min
2:45–2:58	Shot 23	Arabic	RTX PRO 6000	15.4 min
2:45–2:58	Shot 23	English	RTX PRO 6000	15.4 min
2:58–3:07	Shot 24	both	RTX 5090 (Dubai)	15.2 min

The lip sync, measured on the film

SyncNet v2 on the spoken stretch of every speaking shot in both languages, scored twice: with the voice the clip was made from, and with the other language’s voice on the same face track. A model whose mouths follow the voice scores clearly higher on its own; one that just moves mouths scores about the same on both.

Shot	Model	Own voice	Same face, other language	Offset (frames)
Shot 3, Arabic	Wan 2.2 S2V 14B	4.25	2.41	+0
Shot 3, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.10	3.10	-2
Shot 3, English	Wan 2.2 S2V 14B	6.54	1.98	+1
Shot 3, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.01	1.64	-2
Shot 4, Arabic	Wan 2.2 S2V 14B	3.12	1.58	+1
Shot 4, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.26	1.76	-2
Shot 4, English	Wan 2.2 S2V 14B	4.77	2.04	+0
Shot 4, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.49	1.61	-2
Shot 5, Arabic	Wan 2.2 S2V 14B	3.74	3.02	+1
Shot 5, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	6.16	3.27	-3
Shot 5, English	Wan 2.2 S2V 14B	5.04	2.05	+0
Shot 5, English	InfiniteTalk (on Wan 2.1 I2V 14B)	4.29	1.67	-1
Shot 7, Arabic	Wan 2.2 S2V 14B	5.02	1.70	+1
Shot 7, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.57	1.22	-3
Shot 7, English	Wan 2.2 S2V 14B	3.81	1.93	+0
Shot 7, English	InfiniteTalk (on Wan 2.1 I2V 14B)	4.39	1.50	-2
Shot 8, Arabic	Wan 2.2 S2V 14B	3.12	2.55	+1
Shot 8, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.73	3.30	-2
Shot 8, English	Wan 2.2 S2V 14B	4.41	1.28	+0
Shot 8, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.28	2.04	-3
Shot 9, Arabic	Wan 2.2 S2V 14B	4.73	1.84	+0
Shot 9, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	7.28	3.44	-3
Shot 9, English	Wan 2.2 S2V 14B	6.05	2.15	+0
Shot 9, English	InfiniteTalk (on Wan 2.1 I2V 14B)	7.97	2.17	-1
Shot 10, Arabic	Wan 2.2 S2V 14B	1.45	2.60	+0
Shot 10, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	4.64	0.89	-3
Shot 10, English	Wan 2.2 S2V 14B	4.35	2.01	+1
Shot 10, English	InfiniteTalk (on Wan 2.1 I2V 14B)	5.86	2.13	-1
Shot 13, Arabic	Wan 2.2 S2V 14B	4.08	2.16	-1
Shot 13, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.56	2.94	-4
Shot 13, English	Wan 2.2 S2V 14B	3.73	1.28	-1
Shot 13, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.35	1.32	-2
Shot 15, Arabic	Wan 2.2 S2V 14B	3.79	2.30	+0
Shot 15, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	5.76	1.78	-3
Shot 15, English	Wan 2.2 S2V 14B	4.14	1.37	+1
Shot 15, English	InfiniteTalk (on Wan 2.1 I2V 14B)	4.04	1.19	-3
Shot 17, Arabic	Wan 2.2 S2V 14B	6.31	2.60	+0
Shot 17, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	7.06	1.76	-3
Shot 17, English	Wan 2.2 S2V 14B	5.20	2.62	+1
Shot 17, English	InfiniteTalk (on Wan 2.1 I2V 14B)	4.80	1.89	-1
Shot 20, Arabic	Wan 2.2 S2V 14B	4.74	2.93	+1
Shot 20, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	6.37	2.36	-1
Shot 20, English	Wan 2.2 S2V 14B	3.31	2.77	+0
Shot 20, English	InfiniteTalk (on Wan 2.1 I2V 14B)	5.22	2.66	-2
Shot 21, Arabic	Wan 2.2 S2V 14B	5.95	1.80	+1
Shot 21, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	6.20	2.69	-2
Shot 21, English	Wan 2.2 S2V 14B	5.12	1.98	+1
Shot 21, English	InfiniteTalk (on Wan 2.1 I2V 14B)	4.63	1.49	-3
Shot 22, Arabic	Wan 2.2 S2V 14B	3.68	0.87	+1
Shot 22, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	7.74	1.57	-2
Shot 22, English	Wan 2.2 S2V 14B	4.72	1.44	+1
Shot 22, English	InfiniteTalk (on Wan 2.1 I2V 14B)	6.75	1.33	-2
Shot 23, Arabic	Wan 2.2 S2V 14B	3.48	2.48	+0
Shot 23, Arabic	InfiniteTalk (on Wan 2.1 I2V 14B)	3.87	1.50	-1
Shot 23, English	Wan 2.2 S2V 14B	5.64	2.44	+0
Shot 23, English	InfiniteTalk (on Wan 2.1 I2V 14B)	3.70	2.84	-2

Across the film:

• Wan 2.2 S2V 14B: 4.44 against 2.08, higher on its own voice in 27 of 28 shots; Arabic alone 4.10 against 2.20
• InfiniteTalk (on Wan 2.1 I2V 14B): 5.68 against 2.04, higher on its own voice in 28 of 28 shots; Arabic alone 5.88 against 2.26

Every frame’s mouth against the soundtrack

Watching the films, one thing stands out before any question of quality: the long pauses between lines. The obvious suspicion is that a voice-driven model, handed a stretch with nothing said, invents mouth movement to fill it. That is measurable, so we measured it rather than arguing about it.

First the audio alone. Across the pilot’s speaking shots there are 83 seconds of voice and 147 seconds of silence inside the clips: 64% of every speaking clip is silence, and it is there before any model sees it. The shots were planned longer than the lines written for them, and each voice track is padded to the length of its shot. Shot 23 is the extreme: 13 seconds of clip for 2.6 seconds of speech, with a single 7.1-second gap. So the pause is ours, not the model’s.

Then every frame. Four shots in four models — about 2,000 frames — were read one frame at a time by a vision model (Qwen3-VL-8B on our own RTX A5000), asked only whether a mouth is open as if speaking. Each answer is then set against whether a voice is actually audible in that frame, from the clip’s own soundtrack. Two failures fall out of it: a mouth moving where nothing is said, and a mouth shut where a line is being spoken.

Shot	Model	Voice in the clip	Mouth moving in silence	Mouth shut while speaking
Shot 17	Wan 2.2 S2V 14B	1.8 s of 7.1 s	0%	26%
Shot 17	InfiniteTalk (on Wan 2.1 I2V 14B)	1.8 s of 7.0 s	9%	67%
Shot 17	LTX-2.5 (distilled)	1.8 s of 7.0 s	100%	0%
Shot 10	Wan 2.2 S2V 14B	3.3 s of 10.1 s	0%	94%
Shot 10	InfiniteTalk (on Wan 2.1 I2V 14B)	3.3 s of 10.0 s	17%	52%
Shot 10	LTX-2.5 (distilled)	3.4 s of 10.0 s	2%	32%
Shot 23	Wan 2.2 S2V 14B	2.7 s of 13.1 s	0%	100%
Shot 23	InfiniteTalk (on Wan 2.1 I2V 14B)	2.6 s of 13.0 s	20%	51%
Shot 23	LTX-2.5 (distilled)	2.7 s of 13.0 s	2%	44%
Shot 3	Wan 2.2 S2V 14B	2.4 s of 5.1 s	23%	3%
Shot 3	InfiniteTalk (on Wan 2.1 I2V 14B)	2.4 s of 5.0 s	17%	53%
Shot 3	HuMo 17B	2.4 s of 3.9 s	14%	51%

What it shows, in order of how much it matters:

1. The chosen model does not invent speech in the silence. Wan 2.2 S2V moves no mouth at all through the silent stretches of shots 10, 17 and 23 — 0% — so the pauses in the film are the soundtrack’s, and the fix for them is writing and shot length, not a different renderer.
2. But it animates nothing in the shots that hold two faces. In shot 23 the grandfather’s mouth is closed in all 314 frames while he speaks, and in shot 10 in 94% of the spoken frames. Both are the shots with two people in frame; where one face is clearly the speaker, as in shot 17, the same model opens the mouth through three quarters of the line. That is a real limit of single-speaker audio conditioning, and it was invisible to a five-frame sample and to the lip-sync score on its own.
3. LTX-2.5’s mouth is not still — it is permanently open. In shot 17 it reads open in 100% of 169 frames, including every frame where nothing is said. A mouth that never closes between syllables reads to the eye as no lip movement at all, which is exactly how it looks.
4. InfiniteTalk moves the mouth more, and less selectively: open in 9 to 20% of silent frames and shut in about half the spoken ones across all four shots — more motion, less tied to the words.

A note on the instrument, because it deserves the same scepticism as anything else here: a vision model reading a small face in a wide shot can mistake a closed mouth for a hidden one. So the rows that decide anything were checked by eye against the frames themselves — shot 23 at 8.2 and 8.6 seconds, shot 17 at 2.0 seconds in both models — and they hold.

How we built it: engineering notes

The rest of this post is the working record: the tools behind every render, the problems we hit, how we measured them, what did not work and what did. The measured numbers come from the run logs, measurement files and written review verdicts; the GPU-use reading was taken live with nvidia-smi, and the two code estimates in section 2 are estimates. The code is quoted from the files that ran.

1. The loop behind every shot

Pass 2 runs from a handful of Python scripts on our own machines, driving ComfyUI on four GPUs: three NVIDIA GeForce RTX 5090 (32 GB VRAM) cards and an NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB VRAM).

• A plan with exact names. wulfa_pilot_pass2.py plan builds every job from the shot list: the still takes, the clips, the chant takes. Each job has a fixed name (for example wlf_p2_key_s23_r3_t3: shot 23, third wording, take 3).
• Prompt changes as exact replacements. Every change to a shot’s wording is written as “replace this sentence with that one”, and the plan refuses to build if a sentence no longer occurs exactly once. A test suite checks that each character is named by its reference image and never by its code name, and that the new mask wording replaced the old one everywhere.
• Graphs checked before any GPU time. graphs asks the two Dubai ComfyUI servers for their node definitions and checks every node and input of every graph kind against them. The Bangalore and Bhilai servers, which also rendered clips, are not checked.
• An approval ledger. The render farm refuses any job whose exact name is not recorded as approved, and a name is spent once it is sent, so nothing renders twice by accident.
• Review by eye, written down. Every take of every still gets a written verdict and the pick is recorded with its reason. The rule: three takes, three more only if all three are wrong, never a seventh. When one wording fails the same way in every take, the wording changes rather than drawing more seeds.
• This post is built from those records (wulfa_pass2_blog.py), so its verdicts, timings and picks are the same ones the pipeline used.

A scheduling lesson. Putting FLUX.2 [klein] 9B, its 8B text encoder and ACE-Step on the same 32 GB card made them swap to system memory: a chant take that normally takes about 15 seconds took 200, and a still that takes 5 to 10 seconds stalled at 7% GPU use. Asking ComfyUI to unload its models between the two kinds of job cleared it.

2. Removing the shake from LatentSync

The problem. The first lip-sync outputs shook: the pasted face moved against the background. The original LTX-2.5 clips were steady, so the shake came from LatentSync.

How we measured it. For every frame we compared the original clip with the lip-synced one on a band of the face above the mouth, using phase correlation (OpenCV phaseCorrelate), which returns how far one image is shifted against the other to a fraction of a pixel. In the original that band does not move against itself, so any shift is LatentSync’s. The number that matters is how much that shift changes from one frame to the next: we report the median, the 90th percentile and the largest, in pixels at 1920×1088.

First measurement, before any fix: Shot 9, Arabic 0.34 / 0.69 / 1.00 px; Shot 9, English 0.34 / 0.60 / 2.53 px; Shot 13, Arabic 0.42 / 0.85 / 1.43 px; Shot 22, Arabic 0.43 / 0.81 / 1.17 px.

How the shake arises. LatentSync finds the face again in every frame, warps out a 420×560 region aligned on three points (the two brow centres and the nose), scales it to 512×512, redraws the lower face and pastes the result back with the inverse warp. If those points land a pixel differently from frame to frame, the pasted face moves.

First attempt, rejected: undo the shift after the fact. Without touching LatentSync, we measured each output frame’s shift on a band from the eyes to the tip of the nose, moved the frame back by it, and blended it into the original through a soft mask around the face (an even earlier version found the mask from the difference between the two clips, but re-encoding changes every pixel slightly, so on shot 13 it covered the whole frame and would have moved the background too). On the band it measured, the shake dropped about four times. The check that matters is a band the correction did not use:

Clip	Nose band, the one corrected (median / p90 / max px)	Brow band, not used (median / p90 / max px)
Shot 9, Arabic	0.37 / 0.64 / 0.81 → 0.09 / 0.21 / 0.42	0.11 / 0.50 / 0.83 → 0.28 / 0.56 / 0.85
Shot 13, Arabic	0.43 / 0.82 / 1.47 → 0.08 / 0.18 / 0.39	0.22 / 0.63 / 0.94 → 0.22 / 0.47 / 1.33
Shot 22, Arabic	0.41 / 0.77 / 1.33 → 0.09 / 0.21 / 0.44	0.40 / 0.70 / 1.24 → 0.20 / 0.45 / 0.85

The pasted face does not only slide; it also turns and changes size a little each frame, and a shift cannot undo that. So the fix had to go where the alignment is made.

Reading the code (bytedance/LatentSync at commit a229c39, the latest):

• Each frame’s 106 landmarks come from insightface on that frame alone, and are rounded to whole pixels twice (face_detector.py line 47, image_processor.py line 65), so the three anchor points move in one-pixel steps as the head moves smoothly.
• The option called smooth (affine_transform.py lines 136-143) only averages a sub-pixel offset of the nose, 0.2 of the previous frame and 0.8 of the current: in practice it does next to nothing to the jitter.
• The crop matrix is cast to half precision (the default since commit 211419a). At 1920×1088 its entries are large (the offset column runs to hundreds or thousands of pixels), and a 16-bit float stores them only to the nearest quarter, half, one or two units, so the crop snaps by a few tenths of a pixel from frame to frame (an estimate, not measured on the clips).
• The paste-back blends the whole crop rectangle through an edge whose width, int(sqrt(area))//20, is recomputed every frame, and its erosion and blur jump by two pixels whenever a small change in face size carries it across a step (one step per 20 px of the square root of the pasted area).

Others have reported the same thing, with no fix released: issue #67 (detection box and landmark jitter blamed for shaking; a Kalman filter and a second extraction pass suggested); issue #286 (the mouth area shaking on the latest code, no fix); issue #317 (flicker beside the nose on RTX 50-series cards); issue #334 (face shaking, no replies).

The fix, a patch to our copy of LatentSync (wulfa_patch_smoothing.py, 65 lines added across four files). The model is untouched.

1. Two passes instead of one. First find the three anchor points in every frame, without rounding. Then smooth each point’s path over time with a Savitzky-Golay filter (15 frames, second order), which removes frame-to-frame noise but follows a real head movement without lagging behind it. Only then crop every frame, and paste back with exactly the same smoothed matrix. A jump larger than a quarter of the brow distance between two frames is treated as a hard cut, and each side is smoothed separately: LTX sometimes puts a cut inside one clip.
2. Full precision for the crop and paste matrices.
3. A fixed paste mask. Instead of the whole crop rectangle with a per-frame edge, only the region the model actually regenerates (the black part of LatentSync’s own mouth mask) is blended back, through one feathered mask made once. Everything else in the frame is the original picture, not redrawn (only re-encoded).

The smoothing, as it runs:

def smooth_landmark_track(lm3, window, cut_jump):
    """WULFA: Savitzky-Golay (order 2) along time on (F, 3, 2) anchors, separately on each stretch between hard
    cuts (LTX can put a cut inside one clip)."""
    from scipy.signal import savgol_filter
    F = len(lm3)
    if window < 5 or F < 5:
        return lm3
    gap = np.linalg.norm(lm3[:, 1] - lm3[:, 0], axis=1)
    jump = np.linalg.norm(np.diff(lm3.mean(1), axis=0), axis=1) > cut_jump * gap[1:]
    bounds = [0, *(np.nonzero(jump)[0] + 1), F]
    out = lm3.copy()
    for a, b in zip(bounds[:-1], bounds[1:]):
        n = b - a
        w = min(window, n if n % 2 else n - 1)
        if w >= 5:
            out[a:b] = savgol_filter(lm3[a:b], w, 2, axis=0, mode="interp")
    return out

The fixed paste mask, built once from LatentSync’s mouth mask:

        # WULFA: blend back only the region the model regenerates (mask.png black) through one fixed feathered mask,
        # not the whole crop rectangle with a per-frame erosion width (int(sqrt(area))//20 flips with ~1% scale noise)
        import os
        if os.environ.get("WULFA_LS_SOFT_PASTE", "1") == "1":
            m = cv2.imread("latentsync/utils/mask.png", cv2.IMREAD_GRAYSCALE)
            fw, fh = self.restorer.face_size
            gen = 1.0 - cv2.resize(m, (fw, fh), interpolation=cv2.INTER_AREA).astype(np.float32) / 255.0
            gen = cv2.erode((gen > 0.5).astype(np.float32), np.ones((9, 9), np.uint8))
            gen = cv2.GaussianBlur(gen, (0, 0), 3.0)
            self.restorer.paste_mask = torch.from_numpy(gen)[None, None].to(device=device, dtype=torch.float32)

Two of the three changes can be switched off (WULFA_LS_SMOOTH=0 for per-frame alignment, WULFA_LS_SOFT_PASTE=0 for the released paste-back). The matrices stay full precision either way, so this comes close to the released code but does not match it. The before runs below were made with the released code, before the patch.

The result, on all six test clips, before the patch and after it:

Clip	Nose band (median / p90 / max px)	Brow band	Whole face	Mouth change from the original (mean grey levels)	Same mouth movement (correlation)	Run time (whole LatentSync run, wall clock)
Shot 9, Arabic	0.37 / 0.64 / 0.81 → 0.02 / 0.06 / 0.12	0.11 / 0.50 / 0.83 → 0.03 / 0.08 / 0.16	0.33 / 0.64 / 0.82 → 0.02 / 0.07 / 0.26	10.8 → 10.5	0.99	192 s → 82 s
Shot 9, English	0.34 / 0.56 / 0.89 → 0.03 / 0.07 / 0.14	0.14 / 0.45 / 0.61 → 0.03 / 0.08 / 0.12	0.32 / 0.55 / 0.92 → 0.02 / 0.07 / 0.32	11.1 → 11.0	0.99	96 s → 81 s
Shot 13, Arabic	0.43 / 0.82 / 1.47 → 0.04 / 0.08 / 0.12	0.22 / 0.63 / 0.94 → 0.04 / 0.09 / 0.16	0.43 / 0.87 / 1.40 → 0.04 / 0.11 / 0.21	10.1 → 9.7	0.99	97 s → 81 s
Shot 13, English	0.43 / 0.86 / 1.58 → 0.03 / 0.19 / 0.60	0.26 / 0.71 / 1.15 → 0.04 / 0.12 / 0.23	0.46 / 0.91 / 1.59 → 0.04 / 0.17 / 0.37	10.1 → 9.7	0.99	94 s → 79 s
Shot 22, Arabic	0.41 / 0.77 / 1.33 → 0.02 / 0.05 / 0.09	0.40 / 0.70 / 1.24 → 0.04 / 0.08 / 0.15	0.41 / 0.76 / 1.14 → 0.02 / 0.05 / 0.13	5.6 → 5.3	0.99	176 s → 142 s
Shot 22, English	0.35 / 0.70 / 1.05 → 0.02 / 0.03 / 0.06	0.29 / 0.56 / 1.18 → 0.03 / 0.06 / 0.11	0.34 / 0.67 / 1.06 → 0.01 / 0.03 / 0.13	5.0 → 4.7	0.99	166 s → 146 s

The shake went from about 0.34-0.43 px median on the nose band to 0.02-0.04 px, and the largest jump on the whole-face band from up to 1.59 px to at most 0.37 px (on the nose band at most 0.60 px, shot 13 English), while the mouth kept its movement. The patched runs were also quicker.

What it does not fix. It cannot remove what is in the original clip: on shot 22 the mask’s strip still reaches the grandmother’s lip. A 15-frame window can soften a very fast head turn; none of these shots has one. And a steady picture does not tell us the mouth shapes are right for the Arabic: that is a judgement by ear and eye, and for a native speaker.

And it does not stop LatentSync distorting the face. On shot 9’s smoothed version the girl’s face is distorted while she speaks: the small rounded mouth of the character comes back as a wider, more adult mouth with a row of teeth. LatentSync was dropped on 17 Sep; these outputs stay only as the record.

3. Running LatentSync on Windows with an RTX 5090

• PyTorch. LatentSync pins PyTorch 2.5.1 built for CUDA 12.1, which has no kernels for the RTX 5090’s architecture. It runs on PyTorch 2.8 built for CUDA 12.8, in its own Python 3.12 environment; the one package that needs Python 3.11 or older (mediapipe) is not used at inference.
• insightface without a compiler. Its face detector, insightface 0.7.3, is published only as source with one C++ extension, and the machine has no C++ compiler. That extension is only used for 3D face mesh rendering, so we built the package without it and put in a small stand-in module that raises an error if anything ever calls it. Face detection never does. The download was checked against PyPI’s published SHA-256.
• Frame rate. LatentSync works at 25 fps and re-samples anything else, which would drop or repeat frames. Our clips are 24 fps, so each clip is relabelled 25 fps frame for frame, the voice is sped up by 25/24 to match, and the output is relabelled back to 24 fps, checked for the original’s frame count, and given the original mastered voice track.
• A licence to buy. The insightface face models it downloads (buffalo_l) are licensed for non-commercial research only, so they need a commercial licence, or a different detector, before any delivered use.

4. Making the voice drive the mouth

LTX-2.5 can take an audio track alongside the picture. In our first overnight run, the Arabic and English clips of five shots came out identical to the pixel although each had its own voice (two more pairs differed only by card-to-card numeric noise): the node that joins the audio and video latents (LTXVConcatAVLatent) fills a missing audio mask with ones, which tells the sampler to regenerate the audio from noise, so the voice we fed in was ignored and the mouths only followed the word “speaks” in the prompt. Adding a zero noise mask to the audio latent (SolidMask into SetLatentNoiseMask) holds the voice fixed so it conditions the picture. We confirmed it on a speaking close-up from our India series, rendered twice from the same picture and seed with two different lines. The clips now differ: the lines end at 3.3 s and 3.7 s and both mouths are closed from 3.6 s, while the version without the mask was still mouthing at 4.2 and 4.5 s, about a second after its line had finished. Every speaking and singing clip in pass 2 is rendered this way.

5. What the models taught us about wording

Image and video models draw what the words name, including words written to rule something out. The fixes that worked replaced a word with a description of the thing we wanted, rather than adding a warning.

Shot	What we wrote	What we got	What worked
Shots 23, 10, 1	"against an armrest cushion"	An upholstered armchair in all 6 takes of shot 23 and a sofa in all 3 of shot 1	"a thick flat floor cushion laid on the carpet against the plain wall, a firm round bolster cushion on the floor at his side": no armchair in any take
Shot 23	"by warm lamplight"	A modern shaded lamp	"a small brass lantern standing on the carpet"
Shot 23	the lantern, and "a thin stream of pale golden-brown coffee filling the top cup"	The lantern standing where the cups were, and the coffee falling onto, into or beside it instead of into a cup, in 6 of 6 takes	The lantern "in the far corner behind the grandfather", and the picked still shows the spout tipped toward the grandfather with no stream falling; the clip animates the pour
Shot 10	"only a nine-year-old boy's small right hand and the cuff of his white kandura sleeve reach in"	The boy's head at the edge of the frame, in 6 of 6 takes	"low, at the height of the grandfather's knee, a small right hand in the cuff of a white sleeve reaches in", without the word "boy": no head in any take
Shot 10	"handle-less porcelain cups"	Cups with handles	"shaped like a little bowl with smooth round sides": the picked take (4) has none, but 5 of the 6 takes still gave the grandfather's cup a handle
Shot 18	each child's bag, from text alone	The bags swapped in all 6 pass 1 takes	A reference picture of each child: right in 2 of 6 takes
Shot 1	the grandmother from a portrait reference alone	Drawn in the girl's green, without her mask, in 3 of 3 takes	The reference plus her clothes and mask in words: right in 2 of 3 takes
Shot 23 (clip)	"kneels beside him", said of a boy already standing	LTX added a second boy to do the kneeling, in both languages	"bends forward toward him": one boy for the first 10 s in both languages, but LTX then cuts to a close-up, and the Arabic close-up has two boys again
Shot 10 (clip)	"leans forward on the stick to begin to rise"	He stood up fully and the camera followed	"leans forward a little on the stick and stays seated": he stays seated; the camera still pushes in

6. Singing the chant

ACE-Step v1 in ComfyUI reads a language tag at the start of each lyric line and treats a line without one as English, so every Arabic line starts with [ar]; without it the take still sings, just not the right words. It sang the couplet more slowly than the shots allow, about 7.5 seconds a pass.

For shot 18 the stretch where take 1 sings the couplet twice runs from 20.7 s to 36.0 s, 15.3 seconds, and the shot has room for 11.6, so it is sped up 1.32 times with the pitch kept (ffmpeg atempo) and placed at 0.3 s.

For shot 12, so both chant shots share one tune, four more takes were made by re-singing the shot 18 take solo, starting from its own encoded audio at 35% or 50% noise. Take 6’s stretch from 6.4 s to 9.9 s fits the shot’s four seconds without any speed change.

Both picks were listened to and kept, and shot 18’s take 5 was compared with take 1 by ear (take 1 stays); none of the chant has been checked by a native speaker yet.

7. Running InfiniteTalk and Wan 2.2 S2V on our own cards

• No new software. The ComfyUI we already run (0.34) has built-in nodes for three voice-driven video models: InfiniteTalk, Wan 2.2 S2V and HuMo. Only the model files were needed: about 60 GB from Comfy-Org’s repackaged Wan 2.1 and 2.2 repositories and Kijai’s audio-encoder repository, each file’s size checked against Hugging Face after download. The 168 GB InfiniteTalk repository we had downloaded the day before is the raw research release, not the files ComfyUI loads.
• Settings from ComfyUI’s own templates for each model, so the comparison is not our tuning of one against the other: InfiniteTalk with a speed LoRA at 6 steps, S2V with a 4-step speed LoRA, both cfg 1; both at 832×480 so the clips compare (S2V’s template is 640×640), and InfiniteTalk also gets a CLIP vision encoding of the still, as MeiGen’s own pipeline does.
• Long clips are made in windows. InfiniteTalk renders 81 frames at a time and starts each next window from the last 9 frames of the one before. Its node puts those 9 frames back at the front of the new window, and the template joins windows without removing them, which would repeat 9 frames at every join and push the picture 0.36 s further behind the voice each time. We drop them.

Checked on every join. Across the 20 joins in the 7 InfiniteTalk clips, the frame after each join is closest to the frame just before it, never to an earlier one, so no frames repeat. The picture does move more across some joins than around them: the largest is 5.6 times the typical change nearby (clip s12, frame 153), which can show as a small jolt.

Model	Clip length	Render time on an NVIDIA GeForce RTX 5090 (32 GB VRAM)
InfiniteTalk	7 s	257 s
InfiniteTalk	10 s	290 s
InfiniteTalk	13 s	355 s
Wan 2.2 S2V	7 s	76 s
Wan 2.2 S2V	10 s	101 s
Wan 2.2 S2V	13 s	108 s

8. Measuring lip sync with SyncNet

• What it is. SyncNet v2 is the usual lip-sync measure in research papers (reported as LSE-C for confidence and LSE-D for distance). LatentSync’s evaluation code includes it, with a face detector (S3FD) that tracks each face, crops it and scores the mouth against the audio at the best offset within 15 frames either way.
• The same footing for every model. Every clip is scaled to 832×480 first and given the same voice file LTX was given. The detector’s minimum face size was lowered from 100 to 40 pixels and it runs at full scale, because its defaults assume 1080p video.
• The control. Each clip is scored with its own voice and again with the other language’s voice for the same shot, on the same face track. SyncNet was trained on real faces, so its numbers on stylised characters are low for every model; what shows whether a model listens to the voice is the gap between the two.
• Only while the line is spoken. The first run scored whole clips. The lines are short (about 1.7 to 2.2 seconds of shot 17’s 7), so most of each score was silence, and one LTX row came out identical for both voices from a face on screen only during the silent end. The second run finds where the voice is heard (ffmpeg silencedetect at -40 dB) and scores that stretch plus half a second either side.
• A trap on the way. To run it on the CPU we hid the GPUs, and PyTorch still reported CUDA as available while counting zero devices, so the detector tried to load onto a GPU that was not there. The device is now chosen by counting devices.

What is still open

• Shots 10 and 23 have no lip sync at all: both hold two faces, and the model animates neither. They need re-staging to one speaker in frame, not re-rendering.
• Three shots do not yet do what the script says: 19 (the bags are not held open), 21 (not upside down over the tray) and 24 (she scoops into a tray, not into the children’s bags).
• A native Emirati reviewer to confirm the battoulah: the shape was wrong twice before the photographs settled it, and it is not our culture to be sure about.
• The pilot’s shots were written longer than its lines. Re-timing the cut recovered a third of the running time; the lines themselves are the next thing to look at.