0
# subprocess32
1

2
A backport of the Python 3 subprocess module for use on Python 2 with enhanced reliability and thread-safety. Provides the full Python 3.2 subprocess functionality plus timeout support from Python 3.3 and the run() API from Python 3.5. Includes a C extension module for reliable process handling in multi-threaded environments.
3

4
## Package Information
5

6
- **Package Name**: subprocess32
7
- **Language**: Python  
8
- **Installation**: `pip install subprocess32`
9
- **Python Version**: Python 2.6, 2.7 (backport of Python 3 features)
10
- **License**: Python Software Foundation License
11

12
## Core Imports
13

14
```python
15
import subprocess32 as subprocess
16
```
17

18
Alternative import pattern for cross-platform compatibility:
19

20
```python
21
import os
22
import sys
23

24
if os.name == 'posix' and sys.version_info[0] < 3:
25
    import subprocess32 as subprocess
26
else:
27
    import subprocess
28
```
29

30
## Basic Usage
31

32
```python
33
import subprocess32 as subprocess
34

35
# Simple command execution
36
result = subprocess.call(['ls', '-l'])
37

38
# Command with error checking
39
try:
40
    subprocess.check_call(['ls', '/nonexistent'])
41
except subprocess.CalledProcessError as e:
42
    print("Command failed with return code {}".format(e.returncode))
43

44
# Capture output
45
output = subprocess.check_output(['echo', 'Hello World'])
46
print(output.strip())
47

48
# Modern run() API (Python 3.5 style - available in subprocess32)
49
result = subprocess.run(['echo', 'Hello'], 
50
                       stdout=subprocess.PIPE, 
51
                       stderr=subprocess.PIPE)
52
print("Output: {}".format(result.stdout))
53
print("Return code: {}".format(result.returncode))
54

55
# Advanced process control with Popen
56
process = subprocess.Popen(['cat'], 
57
                          stdin=subprocess.PIPE, 
58
                          stdout=subprocess.PIPE, 
59
                          stderr=subprocess.PIPE)
60
stdout, stderr = process.communicate(input=b'Hello subprocess32')
61
print(stdout.decode())
62
```
63

64
## API Compatibility Notes
65

66
subprocess32 provides the complete Python 3.2 subprocess API plus several newer features:
67
- **Core API** (always available): `Popen`, `call`, `check_call`, `check_output`, `CalledProcessError`, `PIPE`, `STDOUT`  
68
- **Timeout support** (from Python 3.3): Available in all functions via `timeout` parameter, raises `TimeoutExpired`
69
- **run() API** (from Python 3.5): `run()` function and `CompletedProcess` class
70
- **Additional constants**: `DEVNULL` constant for discarding output
71
- **Windows constants**: Available only on Windows platforms
72

73
## Capabilities
74

75
### High-Level Functions
76

77
Simple functions for common process execution patterns, offering progressively more control and error handling.
78

79
```python { .api }
80
def call(*popenargs, **kwargs):
81
    """
82
    Run command with arguments. Wait for command to complete, then return the return code.
83
    
84
    Parameters:
85
    - *popenargs: Command and arguments as sequence or string
86
    - timeout: Maximum time to wait in seconds (raises TimeoutExpired if exceeded)
87
    - **kwargs: Additional arguments passed to Popen constructor
88
    
89
    Returns:
90
    int: Process return code (0 for success, non-zero for failure)
91
    """
92

93
def check_call(*popenargs, **kwargs):
94
    """
95
    Run command with arguments. Wait for command to complete. 
96
    If return code is zero, return normally. Otherwise raise CalledProcessError.
97
    
98
    Parameters:
99
    - *popenargs: Command and arguments as sequence or string
100
    - timeout: Maximum time to wait in seconds (raises TimeoutExpired if exceeded)
101
    - **kwargs: Additional arguments passed to Popen constructor
102
    
103
    Returns:
104
    None (raises CalledProcessError on non-zero exit)
105
    
106
    Raises:
107
    CalledProcessError: If command returns non-zero exit code
108
    TimeoutExpired: If timeout is exceeded
109
    """
110

111
def check_output(*popenargs, **kwargs):
112
    """
113
    Run command with arguments and return its output.
114
    If return code is zero, return stdout. Otherwise raise CalledProcessError.
115
    
116
    Parameters:
117
    - *popenargs: Command and arguments as sequence or string
118
    - timeout: Maximum time to wait in seconds (raises TimeoutExpired if exceeded)
119
    - **kwargs: Additional arguments passed to Popen constructor (stdout will be overridden)
120
    
121
    Note: The stdout argument is not allowed as it is used internally.
122
    
123
    Returns:
124
    bytes: Command's stdout output as byte string
125
    
126
    Raises:
127
    CalledProcessError: If command returns non-zero exit code
128
    TimeoutExpired: If timeout is exceeded
129
    """
130

131
def run(*popenargs, **kwargs):
132
    """
133
    Run command with arguments and return a CompletedProcess instance.
134
    Modern high-level interface for subprocess execution.
135
    
136
    Parameters:
137
    - *popenargs: Command and arguments as sequence or string
138
    - input: String or bytes to send to subprocess stdin (cannot be used with stdin argument)
139
    - timeout: Maximum time to wait in seconds (raises TimeoutExpired if exceeded)
140
    - check: If True, raise CalledProcessError on non-zero exit code
141
    - **kwargs: Additional arguments passed to Popen constructor
142
    
143
    Note: If input is provided, stdin argument cannot be used as it will be set to PIPE internally.
144
    
145
    Returns:
146
    CompletedProcess: Object containing args, returncode, stdout, stderr
147
    
148
    Raises:
149
    CalledProcessError: If check=True and command returns non-zero exit code
150
    TimeoutExpired: If timeout is exceeded
151
    """
152
```
153

154
### Process Control Class
155

156
Advanced process creation and management with full control over streams, environment, and execution.
157

158
```python { .api }
159
class Popen(object):
160
    """
161
    Execute a child program in a new process with flexible stream handling.
162
    
163
    Parameters:
164
    - args: Command and arguments as sequence or string
165
    - bufsize: Buffer size for subprocess streams (0=unbuffered, 1=line buffered, >1=buffer size)
166
    - executable: Replacement program to execute instead of args[0]
167
    - stdin: Standard input specification (None, PIPE, DEVNULL, file descriptor, or file object)
168
    - stdout: Standard output specification (None, PIPE, DEVNULL, STDOUT, file descriptor, or file object)
169
    - stderr: Standard error specification (None, PIPE, DEVNULL, STDOUT, file descriptor, or file object)
170
    - preexec_fn: Function to call in child process before exec (POSIX only)
171
    - close_fds: Whether to close file descriptors before exec (default True on POSIX)
172
    - shell: Whether to execute through shell (allows shell features but security risk)
173
    - cwd: Working directory for child process
174
    - env: Environment variables for child process (dict or None for inherit)
175
    - universal_newlines: Whether to open streams in text mode with universal newlines
176
    - startupinfo: Windows-specific startup information (Windows only)
177
    - creationflags: Windows-specific process creation flags (Windows only)
178
    - restore_signals: Whether to restore signals to default handlers before exec (POSIX only, default True)
179
    - start_new_session: Whether to start subprocess in new session (POSIX only, default False)
180
    - pass_fds: Sequence of file descriptors to keep open between parent and child (POSIX only)
181
    
182
    Attributes:
183
    - stdin: Standard input stream (if stdin=PIPE)
184
    - stdout: Standard output stream (if stdout=PIPE)  
185
    - stderr: Standard error stream (if stderr=PIPE)
186
    - pid: Process ID of child process
187
    - returncode: Return code of process (None if not terminated)
188
    """
189
    
190
    def __init__(self, args, bufsize=0, executable=None, stdin=None, stdout=None, 
191
                 stderr=None, preexec_fn=None, close_fds=True, shell=False, cwd=None, 
192
                 env=None, universal_newlines=False, startupinfo=None, creationflags=0,
193
                 restore_signals=True, start_new_session=False, pass_fds=()):
194
        pass
195
    
196
    def poll(self):
197
        """
198
        Check if child process has terminated.
199
        
200
        Returns:
201
        int or None: Return code if terminated, None if still running
202
        """
203
    
204
    def wait(self, timeout=None):
205
        """
206
        Wait for child process to terminate.
207
        
208
        Parameters:
209
        - timeout: Maximum time to wait in seconds (None for no timeout)
210
        
211
        Returns:
212
        int: Process return code
213
        
214
        Raises:
215
        TimeoutExpired: If timeout is exceeded
216
        """
217
    
218
    def communicate(self, input=None, timeout=None):
219
        """
220
        Interact with process: send data to stdin, read data from stdout/stderr.
221
        
222
        Parameters:
223
        - input: Data to send to stdin (bytes or string if universal_newlines=True)
224
        - timeout: Maximum time to wait in seconds (None for no timeout)
225
        
226
        Returns:
227
        tuple: (stdout_data, stderr_data) as bytes or strings
228
        
229
        Raises:
230
        TimeoutExpired: If timeout is exceeded
231
        """
232
    
233
    def send_signal(self, sig):
234
        """
235
        Send signal to the process.
236
        
237
        Parameters:
238
        - sig: Signal number to send
239
        """
240
    
241
    def terminate(self):
242
        """
243
        Terminate the process with SIGTERM (graceful shutdown).
244
        """
245
    
246
    def kill(self):
247
        """
248
        Kill the process with SIGKILL (immediate termination).
249
        """
250
    
251
    def __enter__(self):
252
        """Context manager entry."""
253
        return self
254
    
255
    def __exit__(self, type, value, traceback):
256
        """Context manager exit with automatic cleanup."""
257
        pass
258
```
259

260
### Result Data Class
261

262
Container for process execution results returned by the run() function.
263

264
```python { .api }
265
class CompletedProcess(object):
266
    """
267
    A process that has finished running, returned by run().
268
    
269
    Attributes:
270
    - args: The list or str args passed to run()
271
    - returncode: The exit code of the process (negative for signals)
272
    - stdout: The standard output (None if not captured)
273
    - stderr: The standard error (None if not captured)
274
    """
275
    
276
    def __init__(self, args, returncode, stdout=None, stderr=None):
277
        pass
278
    
279
    def check_returncode(self):
280
        """
281
        Raise CalledProcessError if the exit code is non-zero.
282
        
283
        Raises:
284
        CalledProcessError: If returncode is non-zero
285
        """
286
```
287

288
### Exception Classes
289

290
Comprehensive error handling for process execution failures and timeouts.
291

292
```python { .api }
293
class SubprocessError(Exception):
294
    """
295
    Base class for subprocess-related exceptions.
296
    """
297

298
class CalledProcessError(SubprocessError):
299
    """
300
    Raised when a process run with check_call() or check_output() returns non-zero exit status.
301
    
302
    Attributes:
303
    - cmd: Command that was executed
304
    - returncode: Non-zero exit status that caused the error
305
    - output: Output from stdout (if captured) - alias for stdout
306
    - stderr: Output from stderr (if captured)
307
    - stdout: Output from stdout (if captured) - alias for output
308
    """
309
    
310
    def __init__(self, returncode, cmd, output=None, stderr=None):
311
        pass
312

313
class TimeoutExpired(SubprocessError):
314
    """
315
    Raised when the timeout expires while waiting for a child process.
316
    
317
    Attributes:
318
    - cmd: Command that was executed
319
    - timeout: Timeout value in seconds that was exceeded
320
    - output: Output from stdout (if captured) - alias for stdout
321
    - stderr: Output from stderr (if captured)
322
    - stdout: Output from stdout (if captured) - alias for output
323
    """
324
    
325
    def __init__(self, cmd, timeout, output=None, stderr=None):
326
        pass
327
```
328

329
### Constants and Special Values
330

331
Special values for controlling subprocess stream handling and behavior.
332

333
```python { .api }
334
# Stream redirection constants (always available)
335
PIPE = -1      # Create a pipe to the subprocess
336
STDOUT = -2    # Redirect stderr to stdout  
337
DEVNULL = -3   # Redirect to os.devnull (discard output)
338
```
339

340
#### Windows-Specific Constants
341

342
The following constants are only available when running on Windows platforms:
343

344
```python { .api }
345
# Windows-specific constants (Windows only)
346
CREATE_NEW_CONSOLE = 16           # Create new console window
347
CREATE_NEW_PROCESS_GROUP = 512    # Create new process group
348
STD_INPUT_HANDLE = -10            # Standard input handle
349
STD_OUTPUT_HANDLE = -11           # Standard output handle
350
STD_ERROR_HANDLE = -12            # Standard error handle
351
SW_HIDE = 0                       # Hide window
352
STARTF_USESTDHANDLES = 256        # Use provided standard handles
353
STARTF_USESHOWWINDOW = 1          # Use provided window show state
354
```
355

356
### Utility Functions
357

358
Helper functions for command-line processing and compatibility.
359

360
```python { .api }
361
def list2cmdline(seq):
362
    """
363
    Translate a sequence of arguments into a command line string using 
364
    the same rules as the MS C runtime.
365
    
366
    Parameters:
367
    - seq: Sequence of command line arguments
368
    
369
    Returns:
370
    str: Command line string with proper quoting and escaping
371
    """
372
```
373

374
## Advanced Usage Examples
375

376
### Timeout Handling
377

378
```python
379
import subprocess32 as subprocess
380

381
try:
382
    # Command with timeout
383
    result = subprocess.run(['sleep', '10'], timeout=5)
384
except subprocess.TimeoutExpired as e:
385
    print("Command '{}' timed out after {} seconds".format(e.cmd, e.timeout))
386

387
# Timeout with Popen
388
process = subprocess.Popen(['long_running_command'])
389
try:
390
    stdout, stderr = process.communicate(timeout=30)
391
except subprocess.TimeoutExpired:
392
    process.kill()
393
    stdout, stderr = process.communicate()
394
    print("Process was killed due to timeout")
395
```
396

397
### Error Handling
398

399
```python
400
import subprocess32 as subprocess
401

402
try:
403
    result = subprocess.run(['false'], check=True)
404
except subprocess.CalledProcessError as e:
405
    print("Command failed with return code {}".format(e.returncode))
406
    if e.stdout:
407
        print("stdout: {}".format(e.stdout))
408
    if e.stderr:
409
        print("stderr: {}".format(e.stderr))
410
```
411

412
### Advanced Process Control
413

414
```python
415
import subprocess32 as subprocess
416

417
# Complex process with custom environment
418
env = os.environ.copy()
419
env['CUSTOM_VAR'] = 'value'
420

421
with subprocess.Popen(['command'], 
422
                     stdin=subprocess.PIPE,
423
                     stdout=subprocess.PIPE,
424
                     stderr=subprocess.PIPE,
425
                     env=env,
426
                     cwd='/tmp') as process:
427
    
428
    stdout, stderr = process.communicate(input=b'input data')
429
    if process.returncode != 0:
430
        print("Process failed: {}".format(stderr.decode()))
431
    else:
432
        print("Process succeeded: {}".format(stdout.decode()))
433
```
434

435
## Thread Safety
436

437
subprocess32 includes a C extension module that provides thread-safe process creation, addressing critical concurrency issues in Python 2's original subprocess module. This makes it safe to use in multi-threaded applications where the standard library subprocess module would be unreliable.
438

439
## Migration from Python 2 subprocess
440

441
subprocess32 is designed as a drop-in replacement:
442

443
```python
444
# Before
445
import subprocess
446

447
# After  
448
import subprocess32 as subprocess
449
```
450

451
All existing code using the subprocess module should work unchanged with subprocess32, while gaining the benefits of Python 3's improvements and thread-safety enhancements.