Translate a music video 'shooting schedule' into frame-by-frame color-coded pixel grids for XMVP's cell-based painting pipeline. Given a song's BPM, section map, and scene descriptions, this skill generates per-frame JSON instruction files where each frame is a small pixel grid (e.g. 64x64) with per-cell color and annotation data, synced to musical beats and sections. Use whenever the user wants to plan a music video as pixel grids, create a frame-by-frame color map for animation, convert a storyboard or shooting schedule into cell-painting instructions, generate beat-synced frame data for XMVP's frame_canvas.py, or bridge narrative scene descriptions to per-pixel color grids. Also trigger for 'algo-rhythmic', 'rhythmic narrative', 'shooting schedule to frames', 'beat-synced pixel grids', 'music video frame map', 'cell painting instructions', or 'narrative-to-grid'. Works with cartoon_producer.py's music-video and music-agency modes.
81%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Passed
No findings from the security scan
Translate a music video shooting schedule into frame-by-frame, cell-by-cell color-coded pixel grids — one grid per frame of a long narrative sequence, synced to a song's BPM and sections.
Load these before generating any frame data:
references/pipeline-integration.md — How outputs connect to XMVP's frame_canvas.py, cartoon_producer.py, and unicode_visualizer.py. Data formats, file conventions, and the handoff protocol.references/grid-language.md — The "Grid Language" specification: how to encode narrative meaning, mood, motion, and emphasis into color values and cell annotations at the pixel level.These files contain the essential data contracts. Do not proceed without reading them.
A music video is a temporal narrative synchronized to musical structure. This skill is the translation layer between:
The grids are NOT the final visual output. They are instruction maps — structured blueprints that tell frame_canvas.py, an LLM image generator, or any cell-based renderer what to paint where and when. Think of them as the musical score to the orchestra's performance.
Every frame in an algo-rhythmic narrative is governed by three simultaneous temporal systems:
The skill's job is to synchronize all three clocks into a single grid-per-frame output stream.
The user provides:
analyze_audio() and analyze_audio_profile() for automatic BPM/section detection..xml): If provided, extract Bible/Story/Portions/Manifest data directly.INPUT PHASE 1 PHASE 2
┌──────────────┐ Temporal Scene
│ Song Metadata │──┐ Scaffold Decomposition
│ BPM/sections │ │ ┌──────────────────┐ ┌──────────────────────┐
└──────────────┘ ├───▶│ Beat grid │───▶│ Zones per scene │
┌──────────────┐ │ │ Section boundaries│ │ Color palettes │
│ Shooting │──┤ │ Scene boundaries │ │ Motion vectors │
│ Schedule │ │ │ Frame count │ │ Evolution curves │
└──────────────┘ │ └──────────────────┘ └──────────┬───────────┘
┌──────────────┐ │ │
│ Audio File │──┘ ▼
│ (optional) │ PHASE 3: Grid Synthesis
└──────────────┘ ┌──────────────────────┐
│ For each frame: │
│ Musical position │
│ Active scene(s) │
│ Per-cell RGB │
│ Beat modulation │
│ Transition blend │
└──────────┬───────────┘
│
▼
PHASE 4: Export
┌──────────────────────┐
│ manifest.json │
│ frames/*.json │
│ preview.html │
│ XMVP .xml (optional) │
└──────────────────────┘The temporal scaffold is the rhythmic skeleton that everything else hangs on.
# Core calculation
fps = (bpm / 60.0) * frames_per_beat # e.g. 120 BPM * 4 fpb = 8 fps
total_frames = int(duration * fps)
# Beat grid
frames_per_beat_exact = fps * (60.0 / bpm)
beat_boundaries = [int(i * frames_per_beat_exact) for i in range(total_beats)]
# Bar grid (assuming 4/4 time unless specified)
beats_per_bar = time_signature[0] # default 4
bar_boundaries = beat_boundaries[::beats_per_bar]{
"song": {
"title": "Track Name",
"bpm": 120,
"duration_sec": 210.5,
"time_signature": [4, 4],
"fps": 8.0,
"frames_per_beat": 4,
"total_frames": 1684,
"total_beats": 421,
"total_bars": 105
},
"sections": [
{
"name": "intro",
"start_sec": 0.0,
"end_sec": 16.0,
"start_frame": 0,
"end_frame": 128,
"start_beat": 0,
"end_beat": 32,
"energy": "low",
"mood": "mysterious"
}
],
"scenes": [
{
"id": 1,
"description": "Wide shot: empty highway at dawn, heat shimmer",
"section": "intro",
"start_frame": 0,
"end_frame": 64,
"palette": "dawn_warm",
"camera": "slow_push",
"zones": {
"sky": { "y_range": [0.0, 0.4], "base_color": "#2B1B4E" },
"horizon": { "y_range": [0.4, 0.5], "base_color": "#FF6B35" },
"road": { "y_range": [0.5, 1.0], "base_color": "#1A1A2E" }
}
}
]
}Each scene from the shooting schedule gets decomposed into spatial zones and temporal curves.
A zone is a rectangular region of the grid that carries consistent semantic meaning:
Zones are defined as fractional y-ranges (and optionally x-ranges) of the grid. They can overlap (later zones composite over earlier ones).
Each zone has evolution curves that define how its properties change frame-to-frame:
The Grid Language (see reference) defines how beats affect the grid:
This is the core generation loop. For each frame:
Each frame outputs a JSON file:
{
"frame_index": 42,
"time_sec": 5.25,
"beat": 10,
"beat_phase": 0.5,
"bar": 2,
"bar_beat": 2,
"section": "intro",
"scene_id": 1,
"transition": null,
"grid": {
"width": 64,
"height": 64,
"cells": [
[{"r": 43, "g": 27, "b": 78}, {"r": 44, "g": 28, "b": 79}],
"..."
]
},
"annotations": {
"zones_active": ["sky", "horizon", "road"],
"beat_emphasis": 0.0,
"narrative_weight": 0.2,
"camera_offset": [0.0, -0.02]
}
}For videos with thousands of frames, use the compact format — flat hex strings instead of nested objects:
{
"f": 42,
"t": 5.25,
"b": 10,
"bp": 0.5,
"s": "intro",
"sc": 1,
"px": "2B1B4E2C1C4F2D1D50..."
}Where px is a hex string of concatenated RGB values, row-major order. This reduces file size by ~80% vs. verbose format.
algo-rhythmic-output/
├── narrative_grid_manifest.json # Master index
├── temporal_scaffold.json # Beat/section map
├── scene_plans.json # Decomposed scenes
├── frames/
│ ├── frame_00000.json # Per-frame grids (compact)
│ ├── frame_00001.json
│ └── ...
├── palettes.json # Named color palettes used
└── preview.html # Animated grid viewer{
"version": "1.0",
"generator": "algo-rhythmic-narrative",
"song": { "...from scaffold..." },
"grid_dimensions": [64, 64],
"total_frames": 1684,
"format": "compact",
"frame_pattern": "frames/frame_{:05d}.json",
"xmvp_compat": {
"bible_style": "cell-painting instruction grids",
"portions_count": 8,
"manifest_segments": 1684
}
}To feed this into XMVP directly, generate an XMVP XML manifest where each segment references the corresponding frame grid:
<XMVP version="3.03">
<Bible>{"constraints": {"width": 64, "height": 64, "fps": 8, ...}, ...}</Bible>
<Story>{"title": "...", "synopsis": "...", ...}</Story>
<Manifest>{"segs": [
{"id": 0, "start_frame": 0, "end_frame": 1,
"prompt": "GRID_INSTRUCTION: Paint per frames/frame_00000.json RGB values.",
"grid_ref": "frames/frame_00000.json"},
...
]}</Manifest>
</XMVP>The grid_ref field is a non-standard extension. XMVP's cartoon_producer.py can be patched to recognize it, or the prompt text itself can encode the instruction for LLM-based painters.
Generate all frame JSONs directly in Claude. Write a Python script that:
/mnt/user-data/outputs/Generate a standalone Python generator script that the user runs locally:
temporal_scaffold.json and scene_plans.json in fullgrid_synthesizer.py — a standalone script that reads the scaffold + plans and outputs all frames--start-frame and --end-frame for partial generation--preview mode (renders every Nth frame)Always generate a preview HTML artifact that:
Before outputting, verify:
duration * fps (within +/- 1 frame)width * height * 6 charactersAlso appears in
since Nov 3, 2026
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.