tessl/pypi-youtube-dl

Command-line program to download videos from YouTube.com and other video sites

Overview

Eval results

Files

File Downloaders

Name: tessl/pypi-youtube-dl
Author: tessl

File downloaders are protocol-specific handlers that manage the actual file transfer process. They support various streaming protocols and handle network conditions, resume capabilities, and progress reporting.

Capabilities

Downloader Selection

Functions for automatically selecting the appropriate downloader based on content and protocol.

def get_suitable_downloader(info_dict, params={}):
    """
    Get the downloader class that can handle the info dict.
    
    Parameters:
    - info_dict (dict): Video information dictionary containing 'url' and other metadata
    - params (dict): Additional parameters for downloader selection
    
    Returns:
    class: Appropriate downloader class
    """

Base FileDownloader Class

Base class that all protocol-specific downloaders inherit from, providing common download functionality.

class FileDownloader:
    def __init__(self, ydl, params):
        """
        Base file downloader class.
        
        Parameters:
        - ydl: YoutubeDL instance
        - params (dict): Download parameters
        """
    
    def download(self, filename, info_dict):
        """
        Download the file to specified filename.
        
        Parameters:
        - filename (str): Target filename
        - info_dict (dict): Video information dictionary
        
        Returns:
        bool: True if successful, False otherwise
        """
    
    def real_download(self, filename, info_dict):
        """
        Real download implementation (overridden by subclasses).
        
        Parameters:
        - filename (str): Target filename  
        - info_dict (dict): Video information dictionary
        
        Returns:
        bool: Download success status
        """
    
    @staticmethod
    def parse_bytes(bytestr):
        """
        Parse byte string into integer value.
        
        Parameters:
        - bytestr (str): Byte string (e.g., '1024', '1K', '1M')
        
        Returns:
        int: Byte value
        """
    
    def slow_down(self, start_time, now, byte_counter):
        """
        Implement rate limiting if configured.
        
        Parameters:
        - start_time (float): Download start time
        - now (float): Current time
        - byte_counter (int): Bytes downloaded so far
        """
    
    def report_progress(self, s):
        """
        Report download progress.
        
        Parameters:
        - s (dict): Progress status dictionary
        """
    
    def report_resuming_byte(self, resume_len):
        """
        Report that download is resuming from specific byte.
        
        Parameters:
        - resume_len (int): Byte position to resume from
        """

HTTP Downloader

Standard HTTP/HTTPS download handler with resume support and range requests.

class HttpFD(FileDownloader):
    def real_download(self, filename, info_dict):
        """
        Download file via HTTP/HTTPS protocol.
        
        Parameters:
        - filename (str): Target filename
        - info_dict (dict): Video information with 'url' key
        
        Returns:
        bool: Download success status
        """

HLS Downloader

HTTP Live Streaming (HLS) downloader for .m3u8 playlists and streaming content.

class HlsFD(FileDownloader):
    def real_download(self, filename, info_dict):
        """
        Download HLS stream from m3u8 playlist.
        
        Parameters:
        - filename (str): Target filename
        - info_dict (dict): Video information with HLS URL
        
        Returns:
        bool: Download success status
        """
    
    @staticmethod
    def can_download(manifest_url):
        """
        Check if URL can be downloaded as HLS.
        
        Parameters:
        - manifest_url (str): URL to check
        
        Returns:
        bool: True if HLS download is possible
        """

DASH Downloader

Dynamic Adaptive Streaming over HTTP (DASH) downloader for segmented video content.

class DashSegmentsFD(FileDownloader):
    def real_download(self, filename, info_dict):
        """
        Download DASH segments and combine them.
        
        Parameters:
        - filename (str): Target filename
        - info_dict (dict): Video information with DASH manifest
        
        Returns:
        bool: Download success status
        """

RTMP Downloader

Real-Time Messaging Protocol (RTMP) downloader for Flash video streams.

class RtmpFD(FileDownloader):
    def real_download(self, filename, info_dict):
        """
        Download via RTMP protocol.
        
        Parameters:
        - filename (str): Target filename
        - info_dict (dict): Video information with RTMP URL
        
        Returns:
        bool: Download success status
        """
    
    @staticmethod
    def parse_rtmp_url(url):
        """
        Parse RTMP URL into components.
        
        Parameters:
        - url (str): RTMP URL
        
        Returns:
        dict: Parsed URL components (app, playpath, etc.)
        """

F4M Downloader

Flash Fragment Manifest (F4M) downloader for Adobe Flash streaming.

class F4mFD(FileDownloader):
    def real_download(self, filename, info_dict):
        """
        Download F4M fragmented content.
        
        Parameters:
        - filename (str): Target filename
        - info_dict (dict): Video information with F4M manifest
        
        Returns:
        bool: Download success status
        """

External Downloaders

Support for external download tools like aria2c, axel, curl, wget, and ffmpeg.

def get_external_downloader(external_downloader):
    """
    Get external downloader class by name.
    
    Parameters:
    - external_downloader (str): External downloader name
    
    Returns:
    class: External downloader class
    """

class FFmpegFD(ExternalFD):
    def _make_cmd(self, tmpfilename, info_dict):
        """
        Generate ffmpeg command for download.
        
        Parameters:
        - tmpfilename (str): Temporary filename
        - info_dict (dict): Video information
        
        Returns:
        list: Command line arguments
        """
    
    @classmethod
    def can_download(cls, info_dict):
        """
        Check if ffmpeg can handle this download.
        
        Parameters:
        - info_dict (dict): Video information
        
        Returns:
        bool: True if ffmpeg can download
        """

Protocol Support

youtube-dl supports multiple download protocols through its downloader system:

Supported Protocols

HTTP/HTTPS: Standard web downloads with resume support
HLS (m3u8): HTTP Live Streaming for adaptive bitrate content
DASH: Dynamic Adaptive Streaming over HTTP
RTMP/RTMPE/RTMPS: Real-Time Messaging Protocol variants
F4M: Flash Fragment Manifest for Adobe streaming
ISM: Smooth Streaming Manifest format
RTSP: Real Time Streaming Protocol

Protocol Mapping

PROTOCOL_MAP = {
    'rtmp': RtmpFD,
    'm3u8_native': HlsFD,
    'm3u8': FFmpegFD,
    'mms': RtspFD,
    'rtsp': RtspFD,
    'f4m': F4mFD,
    'http_dash_segments': DashSegmentsFD,
    'ism': IsmFD,
}

Download Parameters

Common parameters that control downloader behavior:

Network Options

socket_timeout (float): Socket timeout in seconds
http_chunk_size (int): HTTP download chunk size
buffersize (int): Download buffer size
ratelimit (int): Download rate limit in bytes/sec
retries (int): Number of retry attempts
fragment_retries (int): Fragment retry attempts

Resume and Recovery

continuedl (bool): Continue partial downloads
noresizebuffer (bool): Don't resize download buffer
skip_unavailable_fragments (bool): Skip missing fragments
keep_fragments (bool): Keep fragment files after download

Progress Reporting

noprogress (bool): Disable progress display
progress_with_newline (bool): Print progress on new lines
progress_hooks (list): Progress callback functions

Usage Examples

Basic Download with Custom Downloader

from youtube_dl import YoutubeDL

ydl_opts = {
    'external_downloader': 'aria2c',
    'external_downloader_args': ['-x', '16', '-s', '16']
}
with YoutubeDL(ydl_opts) as ydl:
    ydl.download(['https://www.youtube.com/watch?v=VIDEO_ID'])

Progress Hook Example

def progress_hook(d):
    if d['status'] == 'finished':
        print(f"Downloaded: {d['filename']}")
    elif d['status'] == 'downloading':
        percent = d.get('_percent_str', 'N/A')
        speed = d.get('_speed_str', 'N/A')
        print(f"Progress: {percent} at {speed}", end='\r')

ydl_opts = {
    'progress_hooks': [progress_hook],
}
with YoutubeDL(ydl_opts) as ydl:
    ydl.download(['https://www.youtube.com/watch?v=VIDEO_ID'])

Rate Limited Download

ydl_opts = {
    'ratelimit': 1024 * 1024,  # 1 MB/s
    'socket_timeout': 30,
    'retries': 5
}
with YoutubeDL(ydl_opts) as ydl:
    ydl.download(['https://www.youtube.com/watch?v=VIDEO_ID'])

Install with Tessl CLI