0
# Requests-Futures
1

2
Asynchronous Python HTTP requests using concurrent.futures for non-blocking operations. This library provides a small add-on for the popular requests HTTP library, enabling asynchronous HTTP requests by returning Future objects instead of immediate Response objects.
3

4
## Package Information
5

6
- **Package Name**: requests-futures
7
- **Package Type**: pypi
8
- **Language**: Python
9
- **Installation**: `pip install requests-futures`
10

11
## Core Imports
12

13
```python
14
from requests_futures.sessions import FuturesSession
15
```
16

17
Alternative import patterns:
18

19
```python
20
import requests_futures.sessions
21
# Access as: requests_futures.sessions.FuturesSession
22

23
from requests_futures import sessions
24
# Access as: sessions.FuturesSession
25
```
26

27
Package-level imports:
28

29
```python
30
import requests_futures
31
# Version info: requests_futures.__version__
32
# Package metadata: requests_futures.__title__, requests_futures.__author__
33
```
34

35
## Basic Usage
36

37
```python
38
from requests_futures.sessions import FuturesSession
39

40
# Create a session (default uses ThreadPoolExecutor with 8 workers)
41
session = FuturesSession()
42

43
# Start requests asynchronously
44
future_one = session.get('http://httpbin.org/get')
45
future_two = session.get('http://httpbin.org/get?foo=bar')
46

47
# Wait for results when needed
48
response_one = future_one.result()
49
response_two = future_two.result()
50

51
print(f'Response 1 status: {response_one.status_code}')
52
print(f'Response 2 status: {response_two.status_code}')
53
```
54

55
## Architecture
56

57
requests-futures extends the requests library's Session class to provide asynchronous functionality:
58

59
- **FuturesSession**: Core class that wraps requests.Session and executes HTTP requests asynchronously using concurrent.futures
60
- **Executor Support**: Works with both ThreadPoolExecutor (default) and ProcessPoolExecutor for different concurrency models
61
- **Future Objects**: All HTTP methods return concurrent.futures.Future objects instead of immediate Response objects
62
- **Context Management**: Supports context manager protocol for automatic resource cleanup and request cancellation
63
- **Background Processing**: Integrates with requests' hooks system for processing responses in background threads
64

65
## Capabilities
66

67
### FuturesSession Class
68

69
The main interface for asynchronous HTTP requests, extending requests.Session with concurrent.futures support.
70

71
```python { .api }
72
class FuturesSession(Session):
73
    def __init__(
74
        self,
75
        executor=None,
76
        max_workers=8,
77
        session=None,
78
        adapter_kwargs=None,
79
        *args,
80
        **kwargs
81
    ): ...
82
```
83

84
**Parameters:**
85
- `executor` (optional): Custom concurrent.futures executor instance (ThreadPoolExecutor or ProcessPoolExecutor)
86
- `max_workers` (int, default=8): Number of worker threads when no executor is provided
87
- `session` (optional): Existing requests.Session instance to wrap for asynchronous execution
88
- `adapter_kwargs` (dict, optional): Additional keyword arguments for HTTPAdapter configuration
89
- `*args, **kwargs`: Additional arguments passed to parent requests.Session class
90

91
**Usage Examples:**
92

93
Basic session creation:
94
```python
95
session = FuturesSession()
96
```
97

98
Custom thread pool size:
99
```python
100
session = FuturesSession(max_workers=16)
101
```
102

103
Custom executor:
104
```python
105
from concurrent.futures import ThreadPoolExecutor
106
executor = ThreadPoolExecutor(max_workers=10)
107
session = FuturesSession(executor=executor)
108
```
109

110
Using existing requests session:
111
```python
112
import requests
113
my_session = requests.Session()
114
my_session.headers.update({'User-Agent': 'MyApp/1.0'})
115
async_session = FuturesSession(session=my_session)
116
```
117

118
### HTTP Request Methods
119

120
All HTTP methods return concurrent.futures.Future objects that resolve to requests.Response objects.
121

122
```python { .api }
123
def request(self, *args, background_callback=None, **kwargs) -> Future[Response]: ...
124
def get(self, url, **kwargs) -> Future[Response]: ...
125
def post(self, url, data=None, json=None, **kwargs) -> Future[Response]: ...
126
def put(self, url, data=None, **kwargs) -> Future[Response]: ...
127
def patch(self, url, data=None, **kwargs) -> Future[Response]: ...
128
def delete(self, url, **kwargs) -> Future[Response]: ...
129
def head(self, url, **kwargs) -> Future[Response]: ...
130
def options(self, url, **kwargs) -> Future[Response]: ...
131
```
132

133
**Parameters:**
134
- `url` (str): Target URL for the HTTP request
135
- `data` (optional): Request body data for POST/PUT/PATCH requests
136
- `json` (optional): JSON data to send in request body (POST only)
137
- `background_callback` (callable, optional, **DEPRECATED**): Function called with (session, response) in background thread. **Deprecated in favor of hooks system - will be removed in version 1.0**
138
- `**kwargs`: All standard requests library parameters (headers, params, timeout, etc.)
139

140
**Returns:** `concurrent.futures.Future[requests.Response]` - Future object that resolves to a Response
141

142
**Usage Examples:**
143

144
```python
145
# GET request
146
future = session.get('https://api.example.com/users')
147
response = future.result()
148

149
# POST with JSON data
150
future = session.post('https://api.example.com/users', json={'name': 'John'})
151
response = future.result()
152

153
# Multiple concurrent requests
154
futures = [
155
    session.get(f'https://api.example.com/users/{i}')
156
    for i in range(5)
157
]
158

159
# Wait for all to complete
160
responses = [future.result() for future in futures]
161
```
162

163
### Session Management
164

165
```python { .api }
166
def close(self) -> None: ...
167
```
168

169
Closes the session and shuts down the owned executor. Called automatically when using the session as a context manager.
170

171
**Usage Examples:**
172

173
Manual cleanup:
174
```python
175
session = FuturesSession()
176
# ... use session
177
session.close()
178
```
179

180
Context manager (recommended):
181
```python
182
with FuturesSession() as session:
183
    future = session.get('https://api.example.com/data')
184
    response = future.result()
185
# Session automatically closed
186
```
187

188
### Background Processing with Hooks
189

190
requests-futures integrates with the requests library's hooks system for background response processing:
191

192
```python
193
def response_hook(resp, *args, **kwargs):
194
    # Parse JSON in background thread
195
    resp.data = resp.json()
196

197
session = FuturesSession()
198
future = session.get('https://api.example.com/data', 
199
                    hooks={'response': response_hook})
200
response = future.result()
201
# response.data is already parsed
202
```
203

204
Session-level hooks:
205
```python
206
session = FuturesSession()
207
session.hooks['response'] = response_hook
208

209
future = session.get('https://api.example.com/data')
210
response = future.result()
211
# Hook applied to all requests
212
```
213

214
### ProcessPoolExecutor Support
215

216
For CPU-intensive processing or memory isolation, use ProcessPoolExecutor:
217

218
```python
219
from concurrent.futures import ProcessPoolExecutor
220

221
# Python 3.5+ (full support)
222
session = FuturesSession(executor=ProcessPoolExecutor(max_workers=4))
223

224
# Python 3.4 (requires existing session)
225
import requests
226
base_session = requests.Session()
227
session = FuturesSession(
228
    executor=ProcessPoolExecutor(max_workers=4),
229
    session=base_session
230
)
231
```
232

233
**Requirements:**
234
- Python >= 3.4 required
235
- Python < 3.5 requires providing a `session` parameter
236
- All functions must be picklable (module-level functions only)
237

238
### Working with Multiple Requests
239

240
Using `concurrent.futures.as_completed()` for processing responses as they arrive:
241

242
```python
243
from concurrent.futures import as_completed
244

245
session = FuturesSession()
246
futures = [
247
    session.get(f'https://api.example.com/item/{i}')
248
    for i in range(10)
249
]
250

251
for future in as_completed(futures):
252
    response = future.result()
253
    print(f'Status: {response.status_code}, URL: {response.url}')
254
```
255

256
Attaching metadata to futures:
257
```python
258
session = FuturesSession()
259
futures = []
260

261
for i in range(3):
262
    future = session.get('https://api.example.com/data')
263
    future.request_id = i  # Attach custom metadata
264
    futures.append(future)
265

266
for future in as_completed(futures):
267
    response = future.result()
268
    print(f'Request {future.request_id}: {response.status_code}')
269
```
270

271
### Error Handling
272

273
Exceptions are deferred to the `Future.result()` call:
274

275
```python
276
session = FuturesSession()
277
future = session.get('https://invalid-url-example.com')
278

279
try:
280
    response = future.result()
281
except requests.exceptions.ConnectionError as e:
282
    print(f'Connection failed: {e}')
283
except requests.exceptions.Timeout as e:
284
    print(f'Request timed out: {e}')
285
```
286

287
### Compatibility Notes
288

289
**Threading Support:**
290
- Python 2.7+ supported for ThreadPoolExecutor
291
- Default executor uses 8 worker threads
292
- Connection pool automatically sized to match worker count when max_workers > 10
293

294
**Process Pool Support:**
295
- Python >= 3.4 required
296
- Python 3.4: Must provide existing `session` parameter
297
- Python >= 3.5: Full support without restrictions
298
- Functions must be picklable (no lambda functions or instance methods)
299

300
**RuntimeError Exceptions:**
301
- Raised when using ProcessPoolExecutor with non-picklable functions
302
- Error message includes link to documentation for troubleshooting
303

304
## Package Metadata
305

306
The package exposes version and metadata information:
307

308
```python { .api }
309
# Module-level constants (from requests_futures.__init__)
310
__title__ = 'requests-futures'
311
__version__ = '1.0.2'
312
__author__ = 'Ross McFarland'
313
__license__ = 'Apache 2.0'
314
__copyright__ = 'Copyright 2013 Ross McFarland'
315
```
316

317
**Usage:**
318
```python
319
import requests_futures
320
print(f"Version: {requests_futures.__version__}")
321
print(f"Author: {requests_futures.__author__}")
322
```
323

324
## Types
325

326
```python { .api }
327
# From concurrent.futures
328
class Future:
329
    def result(self, timeout=None) -> Response: ...
330
    def cancel(self) -> bool: ...
331
    def cancelled(self) -> bool: ...
332
    def done(self) -> bool: ...
333
    def add_done_callback(self, fn) -> None: ...
334

335
# From requests
336
class Response:
337
    status_code: int
338
    headers: dict
339
    text: str
340
    content: bytes
341
    url: str
342
    request: Request
343
    def json(self, **kwargs) -> dict: ...
344
    def raise_for_status(self) -> None: ...
345
    
346
class Session:
347
    headers: dict
348
    cookies: dict
349
    hooks: dict
350
    def request(self, method: str, url: str, **kwargs) -> Response: ...
351
    def get(self, url: str, **kwargs) -> Response: ...
352
    def post(self, url: str, **kwargs) -> Response: ...
353
    # ... other HTTP methods
354
```