0
# File Downloaders
1

2
File downloaders are protocol-specific handlers that manage the actual file transfer process. They support various streaming protocols and handle network conditions, resume capabilities, and progress reporting.
3

4
## Capabilities
5

6
### Downloader Selection
7

8
Functions for automatically selecting the appropriate downloader based on content and protocol.
9

10
```python { .api }
11
def get_suitable_downloader(info_dict, params={}):
12
    """
13
    Get the downloader class that can handle the info dict.
14
    
15
    Parameters:
16
    - info_dict (dict): Video information dictionary containing 'url' and other metadata
17
    - params (dict): Additional parameters for downloader selection
18
    
19
    Returns:
20
    class: Appropriate downloader class
21
    """
22
```
23

24
### Base FileDownloader Class
25

26
Base class that all protocol-specific downloaders inherit from, providing common download functionality.
27

28
```python { .api }
29
class FileDownloader:
30
    def __init__(self, ydl, params):
31
        """
32
        Base file downloader class.
33
        
34
        Parameters:
35
        - ydl: YoutubeDL instance
36
        - params (dict): Download parameters
37
        """
38
    
39
    def download(self, filename, info_dict):
40
        """
41
        Download the file to specified filename.
42
        
43
        Parameters:
44
        - filename (str): Target filename
45
        - info_dict (dict): Video information dictionary
46
        
47
        Returns:
48
        bool: True if successful, False otherwise
49
        """
50
    
51
    def real_download(self, filename, info_dict):
52
        """
53
        Real download implementation (overridden by subclasses).
54
        
55
        Parameters:
56
        - filename (str): Target filename  
57
        - info_dict (dict): Video information dictionary
58
        
59
        Returns:
60
        bool: Download success status
61
        """
62
    
63
    @staticmethod
64
    def parse_bytes(bytestr):
65
        """
66
        Parse byte string into integer value.
67
        
68
        Parameters:
69
        - bytestr (str): Byte string (e.g., '1024', '1K', '1M')
70
        
71
        Returns:
72
        int: Byte value
73
        """
74
    
75
    def slow_down(self, start_time, now, byte_counter):
76
        """
77
        Implement rate limiting if configured.
78
        
79
        Parameters:
80
        - start_time (float): Download start time
81
        - now (float): Current time
82
        - byte_counter (int): Bytes downloaded so far
83
        """
84
    
85
    def report_progress(self, s):
86
        """
87
        Report download progress.
88
        
89
        Parameters:
90
        - s (dict): Progress status dictionary
91
        """
92
    
93
    def report_resuming_byte(self, resume_len):
94
        """
95
        Report that download is resuming from specific byte.
96
        
97
        Parameters:
98
        - resume_len (int): Byte position to resume from
99
        """
100
```
101

102
### HTTP Downloader
103

104
Standard HTTP/HTTPS download handler with resume support and range requests.
105

106
```python { .api }
107
class HttpFD(FileDownloader):
108
    def real_download(self, filename, info_dict):
109
        """
110
        Download file via HTTP/HTTPS protocol.
111
        
112
        Parameters:
113
        - filename (str): Target filename
114
        - info_dict (dict): Video information with 'url' key
115
        
116
        Returns:
117
        bool: Download success status
118
        """
119
```
120

121
### HLS Downloader
122

123
HTTP Live Streaming (HLS) downloader for .m3u8 playlists and streaming content.
124

125
```python { .api }
126
class HlsFD(FileDownloader):
127
    def real_download(self, filename, info_dict):
128
        """
129
        Download HLS stream from m3u8 playlist.
130
        
131
        Parameters:
132
        - filename (str): Target filename
133
        - info_dict (dict): Video information with HLS URL
134
        
135
        Returns:
136
        bool: Download success status
137
        """
138
    
139
    @staticmethod
140
    def can_download(manifest_url):
141
        """
142
        Check if URL can be downloaded as HLS.
143
        
144
        Parameters:
145
        - manifest_url (str): URL to check
146
        
147
        Returns:
148
        bool: True if HLS download is possible
149
        """
150
```
151

152
### DASH Downloader
153

154
Dynamic Adaptive Streaming over HTTP (DASH) downloader for segmented video content.
155

156
```python { .api }
157
class DashSegmentsFD(FileDownloader):
158
    def real_download(self, filename, info_dict):
159
        """
160
        Download DASH segments and combine them.
161
        
162
        Parameters:
163
        - filename (str): Target filename
164
        - info_dict (dict): Video information with DASH manifest
165
        
166
        Returns:
167
        bool: Download success status
168
        """
169
```
170

171
### RTMP Downloader
172

173
Real-Time Messaging Protocol (RTMP) downloader for Flash video streams.
174

175
```python { .api }
176
class RtmpFD(FileDownloader):
177
    def real_download(self, filename, info_dict):
178
        """
179
        Download via RTMP protocol.
180
        
181
        Parameters:
182
        - filename (str): Target filename
183
        - info_dict (dict): Video information with RTMP URL
184
        
185
        Returns:
186
        bool: Download success status
187
        """
188
    
189
    @staticmethod
190
    def parse_rtmp_url(url):
191
        """
192
        Parse RTMP URL into components.
193
        
194
        Parameters:
195
        - url (str): RTMP URL
196
        
197
        Returns:
198
        dict: Parsed URL components (app, playpath, etc.)
199
        """
200
```
201

202
### F4M Downloader
203

204
Flash Fragment Manifest (F4M) downloader for Adobe Flash streaming.
205

206
```python { .api }
207
class F4mFD(FileDownloader):
208
    def real_download(self, filename, info_dict):
209
        """
210
        Download F4M fragmented content.
211
        
212
        Parameters:
213
        - filename (str): Target filename
214
        - info_dict (dict): Video information with F4M manifest
215
        
216
        Returns:
217
        bool: Download success status
218
        """
219
```
220

221
### External Downloaders
222

223
Support for external download tools like aria2c, axel, curl, wget, and ffmpeg.
224

225
```python { .api }
226
def get_external_downloader(external_downloader):
227
    """
228
    Get external downloader class by name.
229
    
230
    Parameters:
231
    - external_downloader (str): External downloader name
232
    
233
    Returns:
234
    class: External downloader class
235
    """
236

237
class FFmpegFD(ExternalFD):
238
    def _make_cmd(self, tmpfilename, info_dict):
239
        """
240
        Generate ffmpeg command for download.
241
        
242
        Parameters:
243
        - tmpfilename (str): Temporary filename
244
        - info_dict (dict): Video information
245
        
246
        Returns:
247
        list: Command line arguments
248
        """
249
    
250
    @classmethod
251
    def can_download(cls, info_dict):
252
        """
253
        Check if ffmpeg can handle this download.
254
        
255
        Parameters:
256
        - info_dict (dict): Video information
257
        
258
        Returns:
259
        bool: True if ffmpeg can download
260
        """
261
```
262

263
## Protocol Support
264

265
youtube-dl supports multiple download protocols through its downloader system:
266

267
### Supported Protocols
268
- **HTTP/HTTPS**: Standard web downloads with resume support
269
- **HLS (m3u8)**: HTTP Live Streaming for adaptive bitrate content
270
- **DASH**: Dynamic Adaptive Streaming over HTTP
271
- **RTMP/RTMPE/RTMPS**: Real-Time Messaging Protocol variants
272
- **F4M**: Flash Fragment Manifest for Adobe streaming
273
- **ISM**: Smooth Streaming Manifest format
274
- **RTSP**: Real Time Streaming Protocol
275

276
### Protocol Mapping
277
```python
278
PROTOCOL_MAP = {
279
    'rtmp': RtmpFD,
280
    'm3u8_native': HlsFD,
281
    'm3u8': FFmpegFD,
282
    'mms': RtspFD,
283
    'rtsp': RtspFD,
284
    'f4m': F4mFD,
285
    'http_dash_segments': DashSegmentsFD,
286
    'ism': IsmFD,
287
}
288
```
289

290
## Download Parameters
291

292
Common parameters that control downloader behavior:
293

294
### Network Options
295
- `socket_timeout` (float): Socket timeout in seconds
296
- `http_chunk_size` (int): HTTP download chunk size
297
- `buffersize` (int): Download buffer size
298
- `ratelimit` (int): Download rate limit in bytes/sec
299
- `retries` (int): Number of retry attempts
300
- `fragment_retries` (int): Fragment retry attempts
301

302
### Resume and Recovery
303
- `continuedl` (bool): Continue partial downloads
304
- `noresizebuffer` (bool): Don't resize download buffer
305
- `skip_unavailable_fragments` (bool): Skip missing fragments
306
- `keep_fragments` (bool): Keep fragment files after download
307

308
### Progress Reporting
309
- `noprogress` (bool): Disable progress display
310
- `progress_with_newline` (bool): Print progress on new lines
311
- `progress_hooks` (list): Progress callback functions
312

313
## Usage Examples
314

315
### Basic Download with Custom Downloader
316
```python
317
from youtube_dl import YoutubeDL
318

319
ydl_opts = {
320
    'external_downloader': 'aria2c',
321
    'external_downloader_args': ['-x', '16', '-s', '16']
322
}
323
with YoutubeDL(ydl_opts) as ydl:
324
    ydl.download(['https://www.youtube.com/watch?v=VIDEO_ID'])
325
```
326

327
### Progress Hook Example
328
```python
329
def progress_hook(d):
330
    if d['status'] == 'finished':
331
        print(f"Downloaded: {d['filename']}")
332
    elif d['status'] == 'downloading':
333
        percent = d.get('_percent_str', 'N/A')
334
        speed = d.get('_speed_str', 'N/A')
335
        print(f"Progress: {percent} at {speed}", end='\r')
336

337
ydl_opts = {
338
    'progress_hooks': [progress_hook],
339
}
340
with YoutubeDL(ydl_opts) as ydl:
341
    ydl.download(['https://www.youtube.com/watch?v=VIDEO_ID'])
342
```
343

344
### Rate Limited Download
345
```python
346
ydl_opts = {
347
    'ratelimit': 1024 * 1024,  # 1 MB/s
348
    'socket_timeout': 30,
349
    'retries': 5
350
}
351
with YoutubeDL(ydl_opts) as ydl:
352
    ydl.download(['https://www.youtube.com/watch?v=VIDEO_ID'])
353
```