or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

audio.mdcodecs.mdcontainers.mdfilters.mdindex.mdstreams.mdvideo.md
tile.json

tessl/pypi-av

Pythonic bindings for FFmpeg's libraries enabling multimedia processing with audio/video encoding, decoding, format conversion, and stream manipulation.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/av@15.1.x

To install, run

npx @tessl/cli install tessl/pypi-av@15.1.0

index.mddocs/

PyAV

PyAV is a comprehensive Python library that provides Pythonic bindings for the FFmpeg multimedia framework, enabling developers to work directly and precisely with media containers, streams, packets, codecs, and frames. It offers powerful capabilities for multimedia processing including audio and video encoding/decoding, format conversion, stream manipulation, and codec access, while managing the complex underlying FFmpeg details.

Package Information

  • Package Name: av
  • Language: Python
  • Installation: pip install av

Core Imports

import av

Open media files:

container = av.open('path/to/file.mp4')

Access specific components:

from av import AudioFrame, VideoFrame, Packet
from av.audio import AudioResampler
from av.video import VideoReformatter

Basic Usage

import av
import numpy as np

# Open an input container
container = av.open('input.mp4')

# Get the first video stream
video_stream = container.streams.video[0]

# Decode frames
for frame in container.decode(video_stream):
    # Convert to numpy array for processing
    array = frame.to_ndarray(format='rgb24')
    print(f"Frame shape: {array.shape}")
    
    # Process first few frames only
    if frame.index > 10:
        break

container.close()

# Create an output container
output = av.open('output.mp4', 'w')

# Add a video stream
video_stream = output.add_stream('h264', rate=30)
video_stream.width = 1920
video_stream.height = 1080

# Create and encode frames
for i in range(90):  # 3 seconds at 30fps
    # Create a frame from numpy array
    frame = VideoFrame.from_ndarray(
        np.random.randint(0, 255, (1080, 1920, 3), dtype=np.uint8),
        format='rgb24'
    )
    frame.pts = i
    frame.time_base = video_stream.time_base
    
    # Encode and write
    for packet in video_stream.encode(frame):
        output.mux(packet)

# Flush and close
for packet in video_stream.encode():
    output.mux(packet)
output.close()

Architecture

PyAV's design follows FFmpeg's architecture with Python-friendly abstractions:

  • Containers: Media file wrappers managing format, streams, and metadata
  • Streams: Individual audio/video/subtitle tracks with codec information
  • Packets: Compressed data units containing encoded media
  • Frames: Uncompressed media data (audio samples or video pixels)
  • Codecs: Encoder/decoder contexts for format conversion
  • Filters: Processing graphs for audio/video transformation

This architecture enables direct access to FFmpeg's capabilities while providing Pythonic interfaces, NumPy integration, and automatic memory management.

Capabilities

Container Operations

Core functionality for opening, reading, and writing media files. Supports input containers for reading media and output containers for writing, with comprehensive format support and metadata handling.

def open(file, mode='r', format=None, options=None, **kwargs):
    """
    Open a media container.
    
    Parameters:
    - file: File path, file object, or URL
    - mode: 'r' for reading, 'w' for writing
    - format: Container format (auto-detected if None)
    - options: Dict of container options
    
    Returns:
    InputContainer or OutputContainer
    """

class InputContainer:
    streams: StreamContainer
    metadata: dict[str, str]
    duration: int
    start_time: int
    
    def demux(*streams): ...
    def decode(*streams): ...
    def seek(offset, **kwargs): ...

class OutputContainer:
    def add_stream(codec, rate=None, **kwargs): ...
    def mux(packet): ...
    def close(): ...

Container Operations

Audio Processing

Comprehensive audio handling including frames, streams, format conversion, resampling, and FIFO buffering. Supports all major audio formats with NumPy integration.

class AudioFrame:
    samples: int
    sample_rate: int
    format: AudioFormat
    layout: AudioLayout
    planes: tuple[AudioPlane, ...]
    
    @staticmethod
    def from_ndarray(array, format='s16', layout='stereo', sample_rate=48000): ...
    def to_ndarray(format=None): ...

class AudioResampler:
    def __init__(self, format=None, layout=None, rate=None): ...
    def resample(frame): ...

class AudioStream:
    def encode(frame=None): ...
    def decode(packet): ...

Audio Processing

Video Processing

Complete video handling with frames, streams, format conversion, reformatting, and image operations. Includes support for all major video formats and pixel formats with NumPy/PIL integration.

class VideoFrame:
    width: int
    height: int
    format: VideoFormat
    planes: tuple[VideoPlane, ...]
    pts: int
    
    @staticmethod
    def from_ndarray(array, format='rgb24'): ...
    def to_ndarray(format=None): ...
    def to_image(): ...
    def reformat(width=None, height=None, format=None): ...

class VideoStream:
    def encode(frame=None): ...
    def decode(packet): ...

class VideoReformatter:
    def reformat(frame, width=None, height=None, format=None): ...

Video Processing

Codec Management

Codec contexts for encoding and decoding with hardware acceleration support. Provides access to all FFmpeg codecs with comprehensive parameter control.

class Codec:
    name: str
    type: str
    is_encoder: bool
    is_decoder: bool
    
    def create(kind=None): ...

class CodecContext:
    def open(codec=None): ...
    def encode(frame=None): ...
    def decode(packet): ...
    def flush_buffers(): ...

class HWAccel:
    @staticmethod
    def create(device_type, device=None): ...

Codec Management

Filter System

Audio and video filtering capabilities using FFmpeg's filter system. Supports filter graphs, custom filters, and processing pipelines.

class Graph:
    def add(filter, *args, **kwargs): ...
    def add_buffer(template): ...
    def configure(): ...
    def push(frame): ...
    def pull(): ...

class Filter:
    name: str
    description: str
    
class FilterContext:
    def link_to(target, output_idx=0, input_idx=0): ...
    def push(frame): ...
    def pull(): ...

Filter System

Packet and Stream Management

Low-level packet handling and stream operations for precise control over media data flow and timing.

class Packet:
    stream: Stream
    pts: int
    dts: int  
    size: int
    duration: int
    is_keyframe: bool
    
    def decode(): ...

class StreamContainer:
    video: tuple[VideoStream, ...]
    audio: tuple[AudioStream, ...]
    
    def best(kind): ...

class Stream:
    index: int
    type: str
    codec_context: CodecContext
    metadata: dict[str, str]

Packet and Stream Management

Global Functions and Constants

# Version information
__version__: str
library_versions: dict[str, tuple[int, int, int]]
ffmpeg_version_info: str

# Core functions
def open(file, mode='r', **kwargs): ...
def get_include() -> str: ...

# Available formats and codecs
formats_available: set[str]
codecs_available: set[str]
bitstream_filters_available: set[str]

Utility Modules

Logging

PyAV provides comprehensive logging capabilities for debugging and monitoring FFmpeg operations.

import av.logging

# Log levels
av.logging.PANIC: int
av.logging.FATAL: int  
av.logging.ERROR: int
av.logging.WARNING: int
av.logging.INFO: int
av.logging.VERBOSE: int
av.logging.DEBUG: int
av.logging.TRACE: int

# Logging functions
def get_level() -> int: ...
def set_level(level: int) -> None: ...
def log(level: int, name: str, message: str) -> None: ...

class Capture:
    """Context manager for capturing log messages."""
    logs: list[tuple[int, str, str]]
    
    def __enter__(self): ...
    def __exit__(self, *args): ...

Test Data

Access to sample media files for testing and development.

import av.datasets

def cached_download(url: str, name: str) -> str:
    """Download and cache test data."""

def fate(name: str) -> str:
    """Get FFmpeg test suite sample."""

def curated(name: str) -> str:
    """Get PyAV curated sample."""

Error Handling

class FFmpegError(Exception):
    errno: int
    strerror: str
    filename: str

class InvalidDataError(FFmpegError, ValueError): ...
class HTTPError(FFmpegError): ...
class LookupError(FFmpegError): ...

# Specific lookup errors
class DecoderNotFoundError(LookupError): ...
class EncoderNotFoundError(LookupError): ...
class FilterNotFoundError(LookupError): ...