Generated Stills as B-Roll

Read this when a reel needs a shot nobody filmed — an establishing frame, a concept cutaway, a beat the footage does not cover.

Adapted from jbaruch/speaker-toolkit's illustrations skill. Its rules are about how image models behave, not about decks, so they transfer whole. What changes for reels is aspect and destination.

python3 scripts/gen_images.py plan/images.json --outdir raw/ --dry-run
python3 scripts/gen_images.py plan/images.json --outdir raw/

--dry-run prints the composed prompts and calls nothing. Use it first — generation costs money per image.

The spec

{
  "model": "gpt-image-2",
  "style_anchor": "Grainy 35mm film photograph. Muted teal shadows, warm amber highlights, heavy grain, shallow depth of field. No text, no lettering.",
  "images": [
    {"name": "still_dawn",  "prompt": "A lone cyclist climbing an empty mountain road at first light, seen from behind."},
    {"name": "still_hands", "prompt": "Close on a rider's hands on drop handlebars, road blurred beneath."}
  ]
}

Dispatch is by model prefix — gpt-image-* and chatgpt-image-* to OpenAI, imagen-* to Imagen, everything else to Gemini — so a model released after this was written works with no code change. Keys: OPENAI_API_KEY for OpenAI. For the Google families the same key travels under several names — GEMINI_API_KEY, GOOGLE_AI_API_KEY, GOOGLE_AI_API, GOOGLE_API_KEY — and the first one set wins. If yours comes from a secret manager, load it into the shell first (e.g. opload); a key that lives only in 1Password is not in the environment.

Finding the beats worth a picture

plan_broll.py reads the take's word-level transcript and works out where b-roll CAN go. It does not decide what deserves it — judging which claim wants a picture depends on knowing what the reel argues, and a keyword heuristic dressed as that judgment is confidently wrong.

mlx_whisper take.wav --word-timestamps True --output-format json
python3 scripts/plan_broll.py take.words.json --out plan/broll.json

What it settles, which is the part that gets done wrong by eye:

• sentence boundaries and their real timings
• the b-roll budget — 40% of runtime by default
• the opening and closing are excluded outright, because a hook has to land on a face and so does a CTA
• sentences too short to cover are dropped; every beat is capped so no still outstays its welcome

It offers every eligible slot and reports the budget separately, rather than spending it front-to-back. Greedy spending would silently hide the back half of the take from whoever is choosing.

Then read what each slot says and write a prompt for the few that earn one. Leave the rest empty — most sentences do not want a picture, and gen_images.py skips unfilled slots rather than rendering something blank. plan_broll.budget_check() tells you whether the ones you filled still clear the ceiling.

The file it writes is already a gen_images.py spec — fill style_anchor and the prompts, then render it directly. Each slot keeps its at/until, so the still lands on the sentence that earned it.

Style-anchor discipline

The anchor is style only: medium, palette, rendering technique, light, lettering treatment. It renders on every image in the set, and that is what makes six cutaways read as one piece instead of a stock-photo collage.

Per-image prompts carry only the scene. Scene content that creeps into the anchor appears in every frame — the usual reason a set drifts. The generator appends a COMPOSE ONLY THE SCENE guard as a backstop; a style-only anchor is the actual fix.

When iterating, extend the anchor rather than rewriting it. Specificity in the style is what holds the set together; trimming it for tidiness reverts the output to a generic look.

Edit or regenerate — the asymmetry

This is the rule worth internalising:

If the model must draw, regenerate. If it only erases, edit.

Edit prompts need three parts

--edit auto-appends the first two. The third is yours and is not optional:

1. DO NOT add any new elements. — suppresses aggressive decoration
2. Let the background continue naturally — no flat patch. — prevents flat-fill artifacts
3. An explicit keep list — --keep "Keep the rider. Keep the horizon."

Without (3) the model routinely removes neighbours of the thing you targeted.

python3 scripts/gen_images.py plan/images.json --outdir raw/ \
    --edit 2 "Erase the small caption in the lower left" \
    --keep "Keep the rider. Keep the horizon line."

Phrase an edit as a removal, never as the resulting state. "The sign is gone" contradicts guard (1) and the model obeys the guard — the sign survives.

Editing is wired for gpt-image-*. Imagen has no edit endpoint at all; regenerate there.

Aspect: why not 9:16

Stills are generated at the vendor's nearest portrait size — 2:3 for OpenAI and Gemini, 3:4 for Imagen — not at 9:16.

Gemini needs to be told. Given no aspect it returns landscape (measured 1408x768), which a 9:16 crop would gut; the generator always sends imageConfig.aspectRatio.

normalize_clips.py crops to the target aspect. A frame that is already exactly 9:16 arrives with no spare pixels, so render_reel.py --pan has nothing to reframe with and a crop that needs to shift has nowhere to go. Generating wider than the target is deliberate headroom.

They are stills, so give them motion

Output lands in raw/ like any footage. A static still held for three seconds reads as dead air — normalize_clips.py takes a kb (Ken Burns) field that drifts the frame. Set it. This is the same path photos already take through the pipeline.

After that they are ordinary clips: probe, plan, grade, export.

Iteration hygiene

Nothing is ever overwritten. A second render of still_dawn writes still_dawn_v2.png, then _v3. Stills take many attempts to converge, and stomping a near-good frame to try one more variation loses something that may have been the keeper. Delete the ones you do not want at the end; disk is cheaper than a render you cannot reproduce.

Cost and honesty

Every image is a paid API call. --dry-run is free and shows exactly what would be sent. --only 2,5 re-renders just those, rather than the whole set, when one frame missed.

Generated stills are optional. With no key set, gen_images.py says so and exits; nothing else in the pipeline depends on it.