tessl/pypi-requests-futures

Asynchronous Python HTTP requests using concurrent.futures for non-blocking operations

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

Requests-Futures

Name: tessl/pypi-requests-futures
Author: tessl

Asynchronous Python HTTP requests using concurrent.futures for non-blocking operations. This library provides a small add-on for the popular requests HTTP library, enabling asynchronous HTTP requests by returning Future objects instead of immediate Response objects.

Package Information

Package Name: requests-futures
Package Type: pypi
Language: Python
Installation: pip install requests-futures

Core Imports

from requests_futures.sessions import FuturesSession

Alternative import patterns:

import requests_futures.sessions
# Access as: requests_futures.sessions.FuturesSession

from requests_futures import sessions
# Access as: sessions.FuturesSession

Package-level imports:

import requests_futures
# Version info: requests_futures.__version__
# Package metadata: requests_futures.__title__, requests_futures.__author__

Basic Usage

from requests_futures.sessions import FuturesSession

# Create a session (default uses ThreadPoolExecutor with 8 workers)
session = FuturesSession()

# Start requests asynchronously
future_one = session.get('http://httpbin.org/get')
future_two = session.get('http://httpbin.org/get?foo=bar')

# Wait for results when needed
response_one = future_one.result()
response_two = future_two.result()

print(f'Response 1 status: {response_one.status_code}')
print(f'Response 2 status: {response_two.status_code}')

Architecture

requests-futures extends the requests library's Session class to provide asynchronous functionality:

FuturesSession: Core class that wraps requests.Session and executes HTTP requests asynchronously using concurrent.futures
Executor Support: Works with both ThreadPoolExecutor (default) and ProcessPoolExecutor for different concurrency models
Future Objects: All HTTP methods return concurrent.futures.Future objects instead of immediate Response objects
Context Management: Supports context manager protocol for automatic resource cleanup and request cancellation
Background Processing: Integrates with requests' hooks system for processing responses in background threads

Capabilities

FuturesSession Class

The main interface for asynchronous HTTP requests, extending requests.Session with concurrent.futures support.

class FuturesSession(Session):
    def __init__(
        self,
        executor=None,
        max_workers=8,
        session=None,
        adapter_kwargs=None,
        *args,
        **kwargs
    ): ...

Parameters:

executor (optional): Custom concurrent.futures executor instance (ThreadPoolExecutor or ProcessPoolExecutor)
max_workers (int, default=8): Number of worker threads when no executor is provided
session (optional): Existing requests.Session instance to wrap for asynchronous execution
adapter_kwargs (dict, optional): Additional keyword arguments for HTTPAdapter configuration
*args, **kwargs: Additional arguments passed to parent requests.Session class

Usage Examples:

Basic session creation:

session = FuturesSession()

Custom thread pool size:

session = FuturesSession(max_workers=16)

Custom executor:

from concurrent.futures import ThreadPoolExecutor
executor = ThreadPoolExecutor(max_workers=10)
session = FuturesSession(executor=executor)

Using existing requests session:

import requests
my_session = requests.Session()
my_session.headers.update({'User-Agent': 'MyApp/1.0'})
async_session = FuturesSession(session=my_session)

HTTP Request Methods

All HTTP methods return concurrent.futures.Future objects that resolve to requests.Response objects.

def request(self, *args, background_callback=None, **kwargs) -> Future[Response]: ...
def get(self, url, **kwargs) -> Future[Response]: ...
def post(self, url, data=None, json=None, **kwargs) -> Future[Response]: ...
def put(self, url, data=None, **kwargs) -> Future[Response]: ...
def patch(self, url, data=None, **kwargs) -> Future[Response]: ...
def delete(self, url, **kwargs) -> Future[Response]: ...
def head(self, url, **kwargs) -> Future[Response]: ...
def options(self, url, **kwargs) -> Future[Response]: ...

Parameters:

url (str): Target URL for the HTTP request
data (optional): Request body data for POST/PUT/PATCH requests
json (optional): JSON data to send in request body (POST only)
background_callback (callable, optional, DEPRECATED): Function called with (session, response) in background thread. Deprecated in favor of hooks system - will be removed in version 1.0
**kwargs: All standard requests library parameters (headers, params, timeout, etc.)

Returns: concurrent.futures.Future[requests.Response] - Future object that resolves to a Response

Usage Examples:

# GET request
future = session.get('https://api.example.com/users')
response = future.result()

# POST with JSON data
future = session.post('https://api.example.com/users', json={'name': 'John'})
response = future.result()

# Multiple concurrent requests
futures = [
    session.get(f'https://api.example.com/users/{i}')
    for i in range(5)
]

# Wait for all to complete
responses = [future.result() for future in futures]

Session Management

def close(self) -> None: ...

Closes the session and shuts down the owned executor. Called automatically when using the session as a context manager.

Usage Examples:

Manual cleanup:

session = FuturesSession()
# ... use session
session.close()

Context manager (recommended):

with FuturesSession() as session:
    future = session.get('https://api.example.com/data')
    response = future.result()
# Session automatically closed

Background Processing with Hooks

requests-futures integrates with the requests library's hooks system for background response processing:

def response_hook(resp, *args, **kwargs):
    # Parse JSON in background thread
    resp.data = resp.json()

session = FuturesSession()
future = session.get('https://api.example.com/data', 
                    hooks={'response': response_hook})
response = future.result()
# response.data is already parsed

Session-level hooks:

session = FuturesSession()
session.hooks['response'] = response_hook

future = session.get('https://api.example.com/data')
response = future.result()
# Hook applied to all requests

ProcessPoolExecutor Support

For CPU-intensive processing or memory isolation, use ProcessPoolExecutor:

from concurrent.futures import ProcessPoolExecutor

# Python 3.5+ (full support)
session = FuturesSession(executor=ProcessPoolExecutor(max_workers=4))

# Python 3.4 (requires existing session)
import requests
base_session = requests.Session()
session = FuturesSession(
    executor=ProcessPoolExecutor(max_workers=4),
    session=base_session
)

Requirements:

Python >= 3.4 required
Python < 3.5 requires providing a session parameter
All functions must be picklable (module-level functions only)

Working with Multiple Requests

Using concurrent.futures.as_completed() for processing responses as they arrive:

from concurrent.futures import as_completed

session = FuturesSession()
futures = [
    session.get(f'https://api.example.com/item/{i}')
    for i in range(10)
]

for future in as_completed(futures):
    response = future.result()
    print(f'Status: {response.status_code}, URL: {response.url}')

Attaching metadata to futures:

session = FuturesSession()
futures = []

for i in range(3):
    future = session.get('https://api.example.com/data')
    future.request_id = i  # Attach custom metadata
    futures.append(future)

for future in as_completed(futures):
    response = future.result()
    print(f'Request {future.request_id}: {response.status_code}')

Error Handling

Exceptions are deferred to the Future.result() call:

session = FuturesSession()
future = session.get('https://invalid-url-example.com')

try:
    response = future.result()
except requests.exceptions.ConnectionError as e:
    print(f'Connection failed: {e}')
except requests.exceptions.Timeout as e:
    print(f'Request timed out: {e}')

Compatibility Notes

Threading Support:

Python 2.7+ supported for ThreadPoolExecutor
Default executor uses 8 worker threads
Connection pool automatically sized to match worker count when max_workers > 10

Process Pool Support:

Python >= 3.4 required
Python 3.4: Must provide existing session parameter
Python >= 3.5: Full support without restrictions
Functions must be picklable (no lambda functions or instance methods)

RuntimeError Exceptions:

Raised when using ProcessPoolExecutor with non-picklable functions
Error message includes link to documentation for troubleshooting

Package Metadata

The package exposes version and metadata information:

# Module-level constants (from requests_futures.__init__)
__title__ = 'requests-futures'
__version__ = '1.0.2'
__author__ = 'Ross McFarland'
__license__ = 'Apache 2.0'
__copyright__ = 'Copyright 2013 Ross McFarland'

Usage:

import requests_futures
print(f"Version: {requests_futures.__version__}")
print(f"Author: {requests_futures.__author__}")

Types

# From concurrent.futures
class Future:
    def result(self, timeout=None) -> Response: ...
    def cancel(self) -> bool: ...
    def cancelled(self) -> bool: ...
    def done(self) -> bool: ...
    def add_done_callback(self, fn) -> None: ...

# From requests
class Response:
    status_code: int
    headers: dict
    text: str
    content: bytes
    url: str
    request: Request
    def json(self, **kwargs) -> dict: ...
    def raise_for_status(self) -> None: ...
    
class Session:
    headers: dict
    cookies: dict
    hooks: dict
    def request(self, method: str, url: str, **kwargs) -> Response: ...
    def get(self, url: str, **kwargs) -> Response: ...
    def post(self, url: str, **kwargs) -> Response: ...
    # ... other HTTP methods

docs

index.md

tile.json

tessl/pypi-requests-futures

index.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Requests-Futures

Package Information

Core Imports

Basic Usage

Architecture

Capabilities

FuturesSession Class

HTTP Request Methods

Session Management

Background Processing with Hooks

ProcessPoolExecutor Support

Working with Multiple Requests

Error Handling

Compatibility Notes

Package Metadata

Types

index.mddocs/