Talking-Head Reels (VO spine + b-roll)

Read this when the project mode is talking-head. The montage pipeline cuts picture to a beat grid; here the spoken take is the spine and picture is cut to fit it. Everything else (probe, contact sheets, normalize, grade, export) is shared with music-video mode.

Length and budget

These run 60–90s, not the 30s a montage targets — a spoken point needs roughly 150–200 words to land. Set --target-duration accordingly at init; the 30s default will truncate the script.

Keep b-roll under ~40% of the runtime. This format is opinion, and the face carries it. Screen recordings and product/logo cards are illustration, not the reel; a stretch of them longer than a few seconds reads as a slideshow with narration. Cap any single app-screen beat around 3s.

Scripting to a teleprompter is normal here, which makes the take clean but front-loads dead air at line breaks — see Tightening a take below.

The shape

VO take  ├───────── continuous speech, never re-timed ─────────┤
picture  │ face │ b-roll │ b-roll │ face │ b-roll │  face+CTA  │

• Face segments keep their true source in-points. A face segment placed at a different in-point than its audio came from breaks lip-sync instantly. Do not let a generator pick these.
• B-roll covers the middle, where the talk is explanatory and the face adds nothing. Opening and closing on the face is what makes it read as a person talking to you rather than a slideshow.
• B-roll here is often not footage. Logo cards, headline cards, and app screen recordings are the usual material. Stills go in raw/ like any clip and normalize into Ken Burns motion (kb field) — a static card held for three seconds reads as dead air, a drifting one does not. Screen recordings are ordinary video, but probe flags them as duration outliers; extract the few seconds you want into work/extracts/ first.
• Third-party logos and press imagery carry usage terms. Keep a short note of what came from where alongside the project; nominative/editorial use of a company's marks in a commentary reel is not a licence to restyle them.
• The plan has no music key unless you want a quiet bed. render_reel.py treats a music-less plan as VO-spine and uses the clips' own audio at 0dB; add --natural-audio -6 to duck it under a bed.

Cut plan shape

Segments are ordinary cut_plan.json segments — the difference is how you pick in values, not the format.

{
  "segments": [
    {"clip": "work/mezz/take_01.mp4", "in": 0.0,  "duration": 4.2,  "note": "face — hook line"},
    {"clip": "work/mezz/take_01.mp4", "in": 4.2,  "duration": 6.0,  "note": "face — setup"},
    {"clip": "work/mezz/broll_02.mp4","in": 12.5, "duration": 3.4,  "note": "b-roll over VO 10.2-13.6"},
    {"clip": "work/mezz/take_01.mp4", "in": 13.6, "duration": 5.1,  "note": "face — payoff + CTA"}
  ]
}

The b-roll problem: the renderer concatenates segments, so a b-roll segment replaces its slice of VO audio instead of overlaying it. Two ways out:

1. Cut b-roll from the take's own timeline — keep the VO contiguous by letting b-roll segments carry no audio, then lay the full VO take over the finished picture as a separate pass (see Overlay pass below). Correct, one extra step.
2. Accept the swap on short b-roll — under ~1s the audio break is inaudible under a bed. Fine for accent cuts, wrong for anything longer.

B-roll that must not interrupt the VO

Segments are concatenated, so a b-roll segment replaces its slice of voiceover. Under about a second that is inaudible; longer and it punches a hole in the narration. --vo-overlay lays the continuous take over the cut picture:

python3 scripts/render_reel.py plan/cut_plan.json --grade clean \
    --vo-overlay raw/take_01.mp4 --vo-start 0 --out work/master.mp4

Picture cuts freely; the voice runs unbroken. --vo-start is where in the VO file the reel's t=0 sits. With this flag the segments' own audio is ignored entirely, so face in-points still have to line up with the VO timeline — --vo-overlay fixes the holes, not a drifting edit.

Removing a word or a phrase

tighten_vo.py removes silence; trim_words.py removes speech — a filler word, a fluffed clause, a line that did not survive:

python3 scripts/trim_words.py words.json --video raw/take.mp4 --list
python3 scripts/trim_words.py words.json --video raw/take.mp4 \
    --out work/extracts/take_trimmed.mp4 --remove "to a plan" --breath 0.12

Phrases are matched case- and punctuation-insensitively against the transcript. A phrase appearing more than once is refused, not guessed — disambiguate with --occurrence N, or use --remove-range I:J with indices from --list. --breath leaves held silence at each join so the cut does not sound clipped; zero sounds clipped.

It writes the same keep-range cut list as tighten_vo.py, so --emit-cuts here replays at 4K through tighten_vo.py --apply-cuts.

Re-transcribe after trimming — every word timestamp past the first cut has moved, so captions built on the old transcript will drift.

Dropping abandoned takes

When a speaker fluffs a line they usually back up and start the sentence again with the same opening. --drop-restarts finds those and removes the EARLIER attempt, keeping the completed one:

python3 scripts/trim_words.py words.json --video raw/take.mp4 --drop-restarts --dry-run

Always --dry-run first. It prints what it would cut; a false positive removes a real sentence. --drop-fillers separately removes standalone "um"/"uh".

Groups are split on punctuation or a pause, because an abandoned line is often left unpunctuated by the transcriber — the pause after it is the reliable signal.

Proving the cut still says what you meant

Every audio-driven edit here can clip a word at a join: tighten_vo.py trims to a margin, trim_words.py cuts on word boundaries, and a few frames either way takes a consonant with them. The result plays fine and is subtly wrong, and check_cuts.py cannot see it — it looks at pictures.

python3 scripts/verify_cut.py work/master.mp4 \
    --words work/take.words.json --cuts work/cuts.json

It re-transcribes the finished file and diffs it against the words the plan expected to survive. Below 95% similarity it fails and names the missing words; the fix is a larger --margin or --breath, then re-cut.

With no transcription backend installed it reports SKIPPED, not PASS — an unverifiable check must never read as a passing one.

Cleaning the audio

Room tone — HVAC, traffic, laptop fans — is the usual failing of a take recorded outside a booth. Clean it before tightening, because tighten_vo.py finds pauses by audio level and a noise floor makes real silence look loud:

python3 scripts/clean_audio.py raw/take_01.mp4 --out work/extracts/take_01_clean.mp4

Defaults to ffmpeg's afftdn — no install, no model file. --engine arnndn uses RNNoise if you supply a .rnnn model; --engine deepfilternet is the best quality and falls back to afftdn when deepFilter is not on PATH. --nr tunes the reduction in dB; past about 20 speech starts sounding underwater.

Do not pass --loudnorm for takes staying in this pipeline. export_variants.py already normalizes every platform master to -14 LUFS, and normalizing twice compounds pumping. The flag exists for audio leaving the pipeline, and runs a genuine two-pass (measure, then correct).

Tightening a take (pause removal)

Spoken takes carry dead air, and teleprompter reads carry more of it. Cut it:

python3 scripts/tighten_vo.py raw/take_01.mp4 --out work/extracts/take_01_tight.mp4

Prefers auto-editor (pip install auto-editor) and falls back to an ffmpeg silencedetect pass when it is absent — the two agree to within about a tenth of a second, so the fallback is not a downgrade. Sections are cut, never sped up, so pitch is untouched. Tuning lives in the script's --threshold and --margin help; raise the threshold if breaths survive, lower it if words get clipped, and never set margin to zero — it sounds clipped.

Editing at 1080 and reproducing at 4K

auto-editor is not resolution-independent — the same audio in a 1080 and a 4K copy of one take can lose different amounts, which shifts every face in-point and forces a re-plan. Capture the decisions once, replay them:

python3 scripts/tighten_vo.py take_1080.mp4 --out t.mp4  --emit-cuts work/cuts.json
python3 scripts/tighten_vo.py take_4k.mp4  --out t4k.mp4 --apply-cuts work/cuts.json

--emit-cuts forces the ffmpeg analyser, which reads audio only and so gives the same ranges at any resolution; --apply-cuts does no detection at all and replays the recorded ranges exactly. Verified identical to the millisecond across 1080 and 4K. The list is plain JSON — reviewable, and editable by hand if a range needs nudging.

It prints a JSON summary with how much came out. Re-probe the tightened file before planning against it — its timeline no longer matches the original take, so every face in-point you had is now wrong.

If you are reading silences by hand instead, note that silencedetect and volumedetect log at info level: -v error hides their output entirely. Use -hide_banner and read stderr.

Making the cards

Logo, headline, and stat cards render as real motion clips at mezzanine spec:

python3 scripts/make_cards.py cards.json --outdir work/cards

Each entry needs a title; kicker, subtitle, duration, bg, accent, and logo are optional. Output is ordinary video from that point on — put it in raw/ and probe it like any clip. Requires Node 18+, and the first run installs Remotion into <outdir>/_project, which takes several minutes and a few hundred MB. Without Node, build the card as a still image instead and let normalize_clips.py apply Ken Burns motion — the kb field exists for this.

Motion is deliberately restrained: a card is illustration under a voice, not a title sequence. Keep any single card near 3s and the whole card budget under the ~40% ceiling above.

Captions are mandatory here

These reels are watched muted. Two options, and they are not equivalent.

Plain — render_reel.py --srt captions.srt burns libass text. Fast, no extra dependency, but libass BorderStyle=3 draws a hard rectangle: no rounded corners, no per-word emphasis. Fine for a rough cut.

Styled — gen_captions.py produces the rounded-card look modern IG/TikTok captions use. Needs ImageMagick and word-level timestamps.

mlx_whisper take.wav --word-timestamps True --output-format json
python3 scripts/gen_captions.py review words.json --out work/captions/phrases.json
# read and fix phrases.json, then:
python3 scripts/gen_captions.py burn work/captions/phrases.json \
    --video work/master.mp4 --out work/master_captioned.mp4

The review stage is not optional. Whisper mishears proper nouns and numbers — "70.3" becomes "17.3", "borrowed" becomes "Boren". Read the phrase list and fix it before burning; re-rendering after the fact wastes a full pass.

Emphasis is explicit, never guessed: set "emphasis": [2] on a phrase to italicise its third word in the accent colour.

Position defaults to 0.72 of frame height — lower third, deliberately off the face. Raise it toward 1.0 to sit lower.

Card geometry is authored at 1080 wide and scaled to the real frame, so a 4K master gets proportionally sized captions rather than half-size ones.

Match --crf to the master you are burning onto. The burn re-encodes, so leaving it at the default while the master was rendered at a lower CRF throws that quality away — measured at 80 Mbps down to 44 on one archival master.

Caption against the final cut, not the raw take — timings from the raw take will not match a tightened or b-rolled timeline.

What still gets checked

check_cuts.py works unchanged, but read its FROZEN_CUT flags loosely on face segments: a locked-off talking head genuinely has near-identical frames across a cut. Black frames and duration drift are still real defects.