requests-toolbelt

A utility belt for advanced users of python-requests. This package provides a comprehensive collection of specialized tools and enhancements that extend the core functionality of HTTP requests, including streaming multipart encoding, SSL adapters, authentication utilities, cookie management, download tools, and parallel request execution.

Package Information

• Package Name: requests-toolbelt
• Language: Python
• Installation: pip install requests-toolbelt
• Requirements: Python 2.7, 3.4+; requests>=2.0.1,<3.0.0

Core Imports

import requests_toolbelt

Common usage patterns:

# Main exports - available directly from requests_toolbelt
from requests_toolbelt import (
    MultipartEncoder, MultipartEncoderMonitor, MultipartDecoder,
    GuessAuth, SSLAdapter, SourceAddressAdapter, StreamingIterator, user_agent,
    ImproperBodyPartContentException, NonMultipartContentTypeException
)

# Specific sub-module imports
from requests_toolbelt.auth.guess import GuessProxyAuth
from requests_toolbelt.adapters.x509 import X509Adapter
from requests_toolbelt.utils.dump import dump_response
from requests_toolbelt.sessions import BaseUrlSession
from requests_toolbelt import threaded

Basic Usage

import requests
from requests_toolbelt import MultipartEncoder

# Streaming multipart form uploads
encoder = MultipartEncoder({
    'field': 'value',
    'file': ('filename.txt', open('file.txt', 'rb'), 'text/plain')
})

response = requests.post(
    'https://httpbin.org/post',
    data=encoder,
    headers={'Content-Type': encoder.content_type}
)

# Automatic authentication detection
from requests_toolbelt import GuessAuth

response = requests.get(
    'https://httpbin.org/basic-auth/user/pass',
    auth=GuessAuth('user', 'pass')
)

# Debug request/response details
from requests_toolbelt.utils import dump_response
print(dump_response(response).decode('utf-8'))

Architecture

The requests-toolbelt library extends the requests ecosystem through several key architectural components:

• Adapters: HTTPAdapter subclasses that modify connection behavior (SSL protocols, source addressing, certificates)
• Authentication: Pluggable auth handlers that integrate with requests' auth system
• Multipart Processing: Streaming encoders/decoders for efficient handling of large multipart content
• Utilities: Helper functions for debugging, user-agent construction, and URL encoding
• Threading: Thread pool implementation for concurrent request execution
• Sessions: Enhanced session classes with additional capabilities

This modular design allows developers to selectively use only the components they need while maintaining full compatibility with the standard requests library.

Capabilities

Multipart Encoding and Decoding

Streaming multipart/form-data encoder for efficient file uploads and form submissions, plus decoder for parsing multipart responses. Includes progress monitoring and supports large files without loading everything into memory.

class MultipartEncoder:
    def __init__(self, fields, boundary=None, encoding='utf-8'): ...
    @property
    def content_type(self) -> str: ...

class MultipartEncoderMonitor:
    def __init__(self, encoder, callback): ...

class MultipartDecoder:
    def __init__(self, content, content_type): ...

Multipart Processing

Authentication

Automatic authentication detection and specialized authentication handlers including GuessAuth for detecting Basic/Digest auth and proxy authentication support.

class GuessAuth:
    def __init__(self, username, password): ...

Note: GuessProxyAuth is available via from requests_toolbelt.auth.guess import GuessProxyAuth

Authentication

HTTP Adapters

Specialized HTTPAdapter implementations for SSL protocol selection, source address binding, certificate verification, and socket options configuration.

class SSLAdapter: ...
class SourceAddressAdapter: ...

Note: X509Adapter, FingerprintAdapter, and others available via specific imports from requests_toolbelt.adapters.*

HTTP Adapters

Download Utilities

Tools for streaming downloads to files, monitoring download progress, and handling multiple download destinations simultaneously.

def stream_response_to_file(response, path=None, chunksize=512): ...
def tee(response, fileobject, chunksize=512, decode_content=True): ...
def get_download_file_path(response, path): ...

Download Utilities

Parallel Requests

Thread pool implementation for executing multiple HTTP requests concurrently with session reuse and customizable initialization.

def map(requests, **kwargs): ...

class Pool:
    def __init__(self, num_processes=None, initializer=None, initargs=None, job_queue=None): ...

Parallel Requests

Debugging and Utilities

Request/response dumping for debugging, enhanced URL encoding, user-agent construction, and other utility functions.

def dump_response(response, request_prefix=b'< ', response_prefix=b'> ', data_array=None): ...
def dump_all(response, request_prefix=b'< ', response_prefix=b'> '): ...
def user_agent(name, version, extras=None): ...

Utilities

Sessions and Streaming

Enhanced session classes with base URL support and streaming iterators for efficient handling of large uploads with known sizes.

class BaseUrlSession:
    def __init__(self, base_url=None): ...

class StreamingIterator:
    def __init__(self, size, iterator): ...

Sessions and Streaming

Cookies and Exceptions

Specialized cookie jar implementations and custom exception classes for better error handling.

class ForgetfulCookieJar: ...

class StreamingError(Exception): ...
class VersionMismatchError(Exception): ...

Cookies and Exceptions