0
# PyRealSense2
1

2
Python bindings for Intel RealSense SDK 2.0, providing comprehensive access to Intel RealSense depth and color cameras. Enables computer vision, robotics, and augmented reality applications with depth sensing, 3D perception, and real-time camera streaming capabilities.
3

4
## Package Information
5

6
- **Package Name**: pyrealsense2
7
- **Language**: Python
8
- **Installation**: `pip install pyrealsense2`
9
- **Minimum Python**: 3.9+
10
- **License**: Apache 2.0
11

12
## Core Imports
13

14
```python
15
import pyrealsense2 as rs
16
```
17

18
## Basic Usage
19

20
```python
21
import pyrealsense2 as rs
22
import numpy as np
23

24
# Create pipeline and configure streams
25
pipeline = rs.pipeline()
26
config = rs.config()
27
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
28
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
29

30
# Start streaming
31
pipeline.start(config)
32

33
try:
34
    while True:
35
        # Wait for coherent frames
36
        frames = pipeline.wait_for_frames()
37
        depth_frame = frames.get_depth_frame()
38
        color_frame = frames.get_color_frame()
39
        
40
        if not depth_frame or not color_frame:
41
            continue
42
            
43
        # Convert to numpy arrays for processing
44
        depth_image = np.asanyarray(depth_frame.get_data())
45
        color_image = np.asanyarray(color_frame.get_data())
46
        
47
        # Get distance at center pixel
48
        width = depth_frame.get_width()
49
        height = depth_frame.get_height()
50
        center_distance = depth_frame.get_distance(width // 2, height // 2)
51
        print(f"Distance at center: {center_distance:.2f} meters")
52
        
53
finally:
54
    pipeline.stop()
55
```
56

57
## Architecture
58

59
PyRealSense2 provides both high-level and low-level APIs:
60

61
- **Pipeline API**: Simplified interface for common use cases with automatic configuration
62
- **Context & Device API**: Fine-grained device discovery and control
63
- **Sensor API**: Direct sensor access for advanced configuration
64
- **Frame Processing**: Rich ecosystem of filters and processing blocks
65
- **3D Processing**: Point cloud generation and coordinate transformations
66

67
The library integrates seamlessly with NumPy through Python's buffer protocol, enabling zero-copy data access for efficient processing with computer vision libraries like OpenCV and scientific computing frameworks.
68

69
## Capabilities
70

71
### Core Pipeline API
72

73
High-level streaming interface that simplifies camera configuration and frame capture. Automatically handles device selection, stream synchronization, and provides coherent framesets for easy processing.
74

75
```python { .api }
76
class pipeline:
77
    def __init__(context=None): ...
78
    def start() -> pipeline_profile: ...
79
    def start(config) -> pipeline_profile: ...
80
    def stop(): ...
81
    def wait_for_frames(timeout_ms=5000) -> frameset: ...
82
    
83
class config:
84
    def enable_stream(stream_type, width, height, format, framerate): ...
85
    def enable_device(serial_number): ...
86
    def disable_all_streams(): ...
87
```
88

89
[Pipeline API](./pipeline.md)
90

91
### Context & Device Management
92

93
Device discovery, enumeration, and management capabilities. Provides fine-grained control over connected RealSense devices with support for hotplug detection and device-specific features.
94

95
```python { .api }
96
class context:
97
    def query_devices() -> device_list: ...
98
    def set_devices_changed_callback(callback): ...
99
    
100
class device:
101
    def query_sensors() -> list[sensor]: ...
102
    def get_info(camera_info) -> str: ...
103
    def hardware_reset(): ...
104
```
105

106
[Context & Device Management](./context-device.md)
107

108
### Sensors & Streaming  
109

110
Low-level sensor control for advanced applications requiring precise stream configuration, custom frame processing, or multi-device synchronization.
111

112
```python { .api }
113
class sensor:
114
    def get_stream_profiles() -> list[stream_profile]: ...
115
    def open(profile): ...
116
    def start(callback): ...
117
    def stop(): ...
118
    
119
class depth_sensor(sensor):
120
    def get_depth_scale() -> float: ...
121
```
122

123
[Sensors & Streaming](./sensors.md)
124

125
### Frame Handling
126

127
Frame data access and manipulation with support for depth, color, infrared, and motion data. Provides buffer protocol integration for NumPy compatibility and metadata access.
128

129
```python { .api }
130
class frame:
131
    def get_data() -> BufData: ...
132
    def get_timestamp() -> float: ...
133
    def get_frame_number() -> int: ...
134
    
135
class depth_frame(video_frame):
136
    def get_distance(x, y) -> float: ...
137
    def get_units() -> float: ...
138
    
139
class frameset(frame):
140
    def get_depth_frame() -> depth_frame: ...
141
    def get_color_frame() -> video_frame: ...
142
```
143

144
[Frame Handling](./frames.md)
145

146
### Processing & Filters
147

148
Real-time frame processing with built-in filters for noise reduction, alignment, colorization, and format conversion. Supports custom processing blocks and filter chaining.
149

150
```python { .api }
151
class align(filter):
152
    def __init__(align_to_stream): ...
153
    
154
class colorizer(filter):
155
    def __init__(color_scheme=0): ...
156
    def colorize(depth_frame) -> video_frame: ...
157
    
158
class temporal_filter(filter):
159
    def __init__(smooth_alpha=0.4, smooth_delta=20.0, persistence_control=3): ...
160
```
161

162
[Processing & Filters](./processing.md)
163

164
### 3D Data & Point Clouds
165

166
3D processing capabilities including point cloud generation, coordinate transformations, and spatial analysis. Supports various output formats and texture mapping.
167

168
```python { .api }
169
class pointcloud(filter):
170
    def calculate(depth_frame) -> points: ...
171
    def map_to(texture_frame): ...
172
    
173
class points(frame):
174
    def get_vertices() -> BufData: ...
175
    def export_to_ply(filename, texture_frame): ...
176
    
177
# Utility functions
178
def rs2_project_point_to_pixel(intrinsics, point_3d) -> pixel_2d: ...
179
def rs2_deproject_pixel_to_point(intrinsics, pixel_2d, depth) -> point_3d: ...
180
```
181

182
[3D Data & Point Clouds](./pointclouds.md)
183

184
### Configuration & Options
185

186
Device option management with support for camera parameters, processing settings, and hardware features. Provides range validation and real-time adjustment capabilities.
187

188
```python { .api }
189
class options:
190
    def get_option(option) -> float: ...
191
    def set_option(option, value): ...
192
    def get_option_range(option) -> option_range: ...
193
    def supports(option) -> bool: ...
194
```
195

196
[Configuration & Options](./options.md)
197

198
### Recording & Playback
199

200
Session recording and playback functionality for development, testing, and data analysis. Supports compression and provides full control over playback timing.
201

202
```python { .api }
203
class recorder(device):
204
    def __init__(filename, device, enable_compression=False): ...
205
    def pause(): ...
206
    def resume(): ...
207
    
208
class playback(device):
209
    def seek(time_ns): ...
210
    def set_real_time(real_time): ...
211
    def get_duration() -> int: ...
212
```
213

214
[Recording & Playback](./recording.md)
215

216
### Advanced Mode API
217

218
Fine-grained control over RS400 series camera parameters including depth processing, color controls, and advanced calibration features. Enables deep customization for specialized applications.
219

220
```python { .api }
221
class rs400_advanced_mode:
222
    def toggle_advanced_mode(enable): ...
223
    def is_enabled() -> bool: ...
224
    def serialize_json() -> str: ...
225
    def load_json(json_content): ...
226
    
227
    def get_depth_control() -> STDepthControlGroup: ...
228
    def set_depth_control(group): ...
229
    def get_rsm() -> STRsm: ...
230
    def set_rsm(rsm): ...
231
```
232

233
[Advanced Mode API](./advanced-mode.md)
234

235
### Logging & Diagnostics
236

237
Global logging functions for debugging and diagnostics. Provides flexible logging output to console, files, or custom callbacks with comprehensive message processing capabilities.
238

239
```python { .api }
240
def log_to_console(min_severity): ...
241
def log_to_file(min_severity, file_path): ...
242
def log_to_callback(min_severity, callback_function): ...
243
def enable_rolling_log_file(severity, file_path, max_size): ...
244
def reset_logger(): ...
245
def log(severity, message): ...
246

247
class log_message:
248
    def get_severity() -> log_severity: ...
249
    def get_message() -> str: ...
250
    def get_filename() -> str: ...
251
    def get_line_number() -> int: ...
252

253
# Version information
254
rs.__version__        # API version string
255
rs.__full_version__   # Full version string
256
```
257

258
[Logging & Diagnostics](./logging.md)
259

260
## Error Handling
261

262
PyRealSense2 operations may raise exceptions of type `rs.error`. Common error scenarios include:
263

264
- Device disconnection during streaming
265
- Invalid stream configuration parameters
266
- Hardware failures or timeouts
267
- File I/O errors during recording/playback
268

269
```python
270
try:
271
    pipeline.start(config)
272
except rs.error as e:
273
    print(f"RealSense error: {e}")
274
except Exception as e:
275
    print(f"Unexpected error: {e}")
276
```
277

278
## Integration Notes
279

280
- **NumPy**: All frames support buffer protocol for zero-copy numpy array creation
281
- **OpenCV**: Direct compatibility with OpenCV image processing functions
282
- **Threading**: Frame callbacks execute on separate threads, requiring proper synchronization
283
- **Memory**: Frames are reference-counted; call `frame.keep()` to extend lifetime if needed