0
# Recording & Playback
1
2
Session recording and playback functionality for development, testing, and data analysis. Supports compression, provides full control over playback timing, and enables offline processing of RealSense data.
3
4
## Capabilities
5
6
### Recording
7
8
Real-time session recording to .bag files with configurable compression.
9
10
```python { .api }
11
class recorder(device):
12
def __init__(filename, device, enable_compression=False):
13
"""
14
Create recording device wrapper.
15
16
Args:
17
filename (str): Output .bag file path
18
device (device): Device to record from
19
enable_compression (bool): Enable data compression
20
"""
21
22
def pause():
23
"""Pause recording (frames still captured but not saved)."""
24
25
def resume():
26
"""Resume recording after pause."""
27
```
28
29
### Playback
30
31
Playback of recorded sessions with timing control and status monitoring.
32
33
```python { .api }
34
class playback(device):
35
def pause():
36
"""Pause playback."""
37
38
def resume():
39
"""Resume playback after pause."""
40
41
def file_name() -> str:
42
"""
43
Get the source file name.
44
45
Returns:
46
str: Path to .bag file being played
47
"""
48
49
def get_position() -> int:
50
"""
51
Get current playback position.
52
53
Returns:
54
int: Position in nanoseconds from start
55
"""
56
57
def get_duration() -> int:
58
"""
59
Get total recording duration.
60
61
Returns:
62
int: Duration in nanoseconds
63
"""
64
65
def seek(time_ns):
66
"""
67
Seek to specific time position.
68
69
Args:
70
time_ns (int): Target position in nanoseconds
71
"""
72
73
def is_real_time() -> bool:
74
"""
75
Check if playback is in real-time mode.
76
77
Returns:
78
bool: True if playing at original speed
79
"""
80
81
def set_real_time(real_time):
82
"""
83
Enable/disable real-time playback.
84
85
Args:
86
real_time (bool): True for original speed, False for as-fast-as-possible
87
"""
88
89
def set_status_changed_callback(callback):
90
"""
91
Set callback for playback status changes.
92
93
Args:
94
callback: Function called when playback status changes
95
"""
96
97
def current_status() -> playback_status:
98
"""
99
Get current playback status.
100
101
Returns:
102
playback_status: Current status (playing, paused, stopped)
103
"""
104
```
105
106
### Playback Status
107
108
Enumeration of playback states.
109
110
```python { .api }
111
rs.playback_status.unknown # Status unknown
112
rs.playback_status.playing # Currently playing
113
rs.playback_status.paused # Playback paused
114
rs.playback_status.stopped # Playback stopped/ended
115
```
116
117
## Usage Examples
118
119
### Basic Recording
120
121
```python
122
import pyrealsense2 as rs
123
import time
124
125
# Configure streams for recording
126
config = rs.config()
127
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
128
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
129
130
# Enable recording to file
131
config.enable_record_to_file("recording.bag")
132
133
# Start pipeline with recording
134
pipeline = rs.pipeline()
135
profile = pipeline.start(config)
136
137
print("Recording started...")
138
print("Press Ctrl+C to stop recording")
139
140
try:
141
frame_count = 0
142
start_time = time.time()
143
144
while True:
145
frames = pipeline.wait_for_frames()
146
frame_count += 1
147
148
# Print progress every second
149
if frame_count % 30 == 0:
150
elapsed = time.time() - start_time
151
print(f"Recorded {frame_count} framesets ({elapsed:.1f}s)")
152
153
except KeyboardInterrupt:
154
print("Recording stopped by user")
155
156
finally:
157
pipeline.stop()
158
elapsed = time.time() - start_time
159
print(f"Recording completed: {frame_count} framesets in {elapsed:.1f}s")
160
print("Saved to recording.bag")
161
```
162
163
### Advanced Recording with Device Wrapper
164
165
```python
166
import pyrealsense2 as rs
167
import time
168
169
# Get physical device
170
ctx = rs.context()
171
devices = ctx.query_devices()
172
173
if len(devices) == 0:
174
print("No RealSense devices found")
175
exit()
176
177
physical_device = devices[0]
178
print(f"Recording from: {physical_device.get_info(rs.camera_info.name)}")
179
180
# Create recorder wrapper with compression
181
recorder_device = rs.recorder("compressed_recording.bag", physical_device, True)
182
183
# Configure streams on recorder device
184
config = rs.config()
185
config.enable_device_from_file("compressed_recording.bag") # This tells config to use the recorder
186
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
187
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
188
config.enable_stream(rs.stream.infrared, 1, 640, 480, rs.format.y8, 30)
189
190
# Alternative: directly use the recorder device
191
pipeline = rs.pipeline()
192
193
# Configure without config.enable_record_to_file since we're using recorder device directly
194
config_direct = rs.config()
195
config_direct.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
196
config_direct.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
197
198
# We need to set the device directly for the pipeline to use our recorder
199
pipeline.start(config_direct)
200
201
print("Recording with compression enabled...")
202
203
try:
204
for i in range(300): # Record for 10 seconds at 30fps
205
frames = pipeline.wait_for_frames()
206
207
if i % 60 == 0: # Print every 2 seconds
208
print(f"Recording frame {i}/300")
209
210
# Demonstrate pause/resume functionality
211
if i == 150: # Pause halfway through
212
recorder_device.pause()
213
print("Recording paused")
214
time.sleep(1)
215
recorder_device.resume()
216
print("Recording resumed")
217
218
except Exception as e:
219
print(f"Recording error: {e}")
220
221
finally:
222
pipeline.stop()
223
print("Recording completed with compression")
224
```
225
226
### Basic Playback
227
228
```python
229
import pyrealsense2 as rs
230
231
# Configure playback from file
232
config = rs.config()
233
config.enable_device_from_file("recording.bag")
234
235
# Optionally configure repeat playback
236
config = rs.config()
237
config.enable_device_from_file("recording.bag", repeat_playback=True)
238
239
pipeline = rs.pipeline()
240
profile = pipeline.start(config)
241
242
# Get playback device for control
243
device = profile.get_device()
244
playback_device = device.as_playback()
245
246
print(f"Playing: {playback_device.file_name()}")
247
print(f"Duration: {playback_device.get_duration() / 1e9:.1f} seconds")
248
249
frame_count = 0
250
251
try:
252
while True:
253
frames = pipeline.wait_for_frames()
254
255
# Get timing information
256
timestamp = frames.get_timestamp()
257
position = playback_device.get_position()
258
259
frame_count += 1
260
261
if frame_count % 30 == 0:
262
print(f"Frame {frame_count}: timestamp={timestamp/1000:.3f}s, "
263
f"position={position/1e9:.3f}s")
264
265
# Process frames
266
depth_frame = frames.get_depth_frame()
267
color_frame = frames.get_color_frame()
268
269
if depth_frame and color_frame:
270
# Example processing
271
center_distance = depth_frame.get_distance(320, 240)
272
if center_distance > 0:
273
print(f" Center distance: {center_distance:.3f}m")
274
275
except rs.error as e:
276
print(f"Playback finished or error: {e}")
277
278
finally:
279
pipeline.stop()
280
print(f"Playback completed: {frame_count} frames processed")
281
```
282
283
### Playback Control and Seeking
284
285
```python
286
import pyrealsense2 as rs
287
import time
288
289
# Configure playback
290
config = rs.config()
291
config.enable_device_from_file("recording.bag", repeat_playback=False)
292
293
pipeline = rs.pipeline()
294
profile = pipeline.start(config)
295
296
device = profile.get_device()
297
playback = device.as_playback()
298
299
print(f"File: {playback.file_name()}")
300
duration_ns = playback.get_duration()
301
duration_s = duration_ns / 1e9
302
print(f"Duration: {duration_s:.1f} seconds")
303
304
# Set playback status callback
305
def status_callback(status):
306
print(f"Playback status changed to: {status}")
307
308
playback.set_status_changed_callback(status_callback)
309
310
try:
311
# Play at normal speed for a few seconds
312
print("Playing at normal speed...")
313
playback.set_real_time(True)
314
315
for i in range(90): # 3 seconds at 30fps
316
frames = pipeline.wait_for_frames()
317
position = playback.get_position()
318
print(f"Position: {position/1e9:.1f}s", end='\r')
319
320
# Pause playback
321
print("\nPausing playback...")
322
playback.pause()
323
time.sleep(2)
324
325
# Resume at fast speed
326
print("Resuming at maximum speed...")
327
playback.resume()
328
playback.set_real_time(False) # As fast as possible
329
330
# Play for another few seconds
331
for i in range(60):
332
frames = pipeline.wait_for_frames()
333
position = playback.get_position()
334
print(f"Fast position: {position/1e9:.1f}s", end='\r')
335
336
# Seek to middle of recording
337
middle_time = duration_ns // 2
338
print(f"\nSeeking to middle ({middle_time/1e9:.1f}s)...")
339
playback.seek(middle_time)
340
341
# Play from middle
342
playback.set_real_time(True)
343
for i in range(60):
344
frames = pipeline.wait_for_frames()
345
position = playback.get_position()
346
print(f"Middle position: {position/1e9:.1f}s", end='\r')
347
348
# Seek to specific timestamps
349
timestamps = [duration_ns * 0.1, duration_ns * 0.5, duration_ns * 0.9]
350
351
for timestamp in timestamps:
352
print(f"\nSeeking to {timestamp/1e9:.1f}s...")
353
playback.seek(timestamp)
354
355
# Get a few frames from this position
356
for j in range(10):
357
frames = pipeline.wait_for_frames()
358
frame_timestamp = frames.get_timestamp()
359
position = playback.get_position()
360
print(f" Frame timestamp: {frame_timestamp/1000:.3f}s, "
361
f"position: {position/1e9:.3f}s")
362
363
except rs.error as e:
364
print(f"\nPlayback error: {e}")
365
366
finally:
367
pipeline.stop()
368
print("Playback control demo completed")
369
```
370
371
### Multi-File Playback Analysis
372
373
```python
374
import pyrealsense2 as rs
375
import os
376
import glob
377
378
class PlaybackAnalyzer:
379
def __init__(self):
380
self.frame_counts = {}
381
self.durations = {}
382
self.stream_info = {}
383
384
def analyze_file(self, bag_file):
385
"""Analyze a single bag file."""
386
print(f"Analyzing {bag_file}...")
387
388
try:
389
config = rs.config()
390
config.enable_device_from_file(bag_file, repeat_playback=False)
391
392
pipeline = rs.pipeline()
393
profile = pipeline.start(config)
394
395
device = profile.get_device()
396
playback = device.as_playback()
397
398
# Get file info
399
duration = playback.get_duration()
400
self.durations[bag_file] = duration / 1e9
401
402
# Get stream information
403
streams = profile.get_streams()
404
stream_info = []
405
for stream_profile in streams:
406
if stream_profile.is_video_stream_profile():
407
vsp = stream_profile.as_video_stream_profile()
408
info = f"{vsp.stream_type().name} {vsp.width()}x{vsp.height()} {vsp.format().name}@{vsp.fps()}fps"
409
else:
410
info = f"{stream_profile.stream_type().name} {stream_profile.format().name}@{stream_profile.fps()}fps"
411
stream_info.append(info)
412
413
self.stream_info[bag_file] = stream_info
414
415
# Count frames by processing entire file
416
playback.set_real_time(False) # Fast playback
417
frame_count = 0
418
419
while True:
420
try:
421
frames = pipeline.wait_for_frames()
422
frame_count += 1
423
424
if frame_count % 100 == 0:
425
position = playback.get_position()
426
progress = (position / duration) * 100
427
print(f" Progress: {progress:.1f}%", end='\r')
428
429
except rs.error:
430
break # End of file
431
432
self.frame_counts[bag_file] = frame_count
433
pipeline.stop()
434
435
print(f" Completed: {frame_count} frames, {duration/1e9:.1f}s")
436
437
except Exception as e:
438
print(f" Error analyzing {bag_file}: {e}")
439
440
def analyze_directory(self, directory):
441
"""Analyze all .bag files in a directory."""
442
bag_files = glob.glob(os.path.join(directory, "*.bag"))
443
444
if not bag_files:
445
print(f"No .bag files found in {directory}")
446
return
447
448
print(f"Found {len(bag_files)} .bag files")
449
450
for bag_file in bag_files:
451
self.analyze_file(bag_file)
452
print() # Blank line between files
453
454
def print_summary(self):
455
"""Print analysis summary."""
456
if not self.frame_counts:
457
print("No files analyzed")
458
return
459
460
print("Analysis Summary:")
461
print("=" * 80)
462
463
total_frames = 0
464
total_duration = 0
465
466
for bag_file in self.frame_counts.keys():
467
filename = os.path.basename(bag_file)
468
frame_count = self.frame_counts[bag_file]
469
duration = self.durations[bag_file]
470
avg_fps = frame_count / duration if duration > 0 else 0
471
472
print(f"\nFile: {filename}")
473
print(f" Duration: {duration:.1f}s")
474
print(f" Frames: {frame_count}")
475
print(f" Average FPS: {avg_fps:.1f}")
476
print(f" Streams:")
477
for stream in self.stream_info[bag_file]:
478
print(f" - {stream}")
479
480
total_frames += frame_count
481
total_duration += duration
482
483
print(f"\nTotals:")
484
print(f" Files: {len(self.frame_counts)}")
485
print(f" Total duration: {total_duration:.1f}s ({total_duration/60:.1f} minutes)")
486
print(f" Total frames: {total_frames}")
487
print(f" Overall average FPS: {total_frames/total_duration:.1f}")
488
489
# Usage example
490
analyzer = PlaybackAnalyzer()
491
492
# Analyze single file
493
if os.path.exists("recording.bag"):
494
analyzer.analyze_file("recording.bag")
495
496
# Analyze directory of recordings
497
# analyzer.analyze_directory("./recordings")
498
499
analyzer.print_summary()
500
```
501
502
### Recording with Pipeline Control
503
504
```python
505
import pyrealsense2 as rs
506
import time
507
import signal
508
import sys
509
510
class PipelineRecorder:
511
def __init__(self, output_file):
512
self.output_file = output_file
513
self.pipeline = None
514
self.is_recording = False
515
self.frame_count = 0
516
self.start_time = None
517
518
def configure_streams(self):
519
"""Configure streams for recording."""
520
config = rs.config()
521
522
# Enable multiple streams
523
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
524
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
525
config.enable_stream(rs.stream.infrared, 1, 640, 480, rs.format.y8, 30)
526
config.enable_stream(rs.stream.infrared, 2, 640, 480, rs.format.y8, 30)
527
528
# Enable recording
529
config.enable_record_to_file(self.output_file)
530
531
return config
532
533
def start_recording(self):
534
"""Start the recording pipeline."""
535
if self.is_recording:
536
print("Already recording")
537
return
538
539
try:
540
config = self.configure_streams()
541
self.pipeline = rs.pipeline()
542
profile = self.pipeline.start(config)
543
544
self.is_recording = True
545
self.frame_count = 0
546
self.start_time = time.time()
547
548
# Get device info
549
device = profile.get_device()
550
print(f"Started recording from: {device.get_info(rs.camera_info.name)}")
551
print(f"Output file: {self.output_file}")
552
553
# Print stream configuration
554
streams = profile.get_streams()
555
print("Recording streams:")
556
for stream_profile in streams:
557
if stream_profile.is_video_stream_profile():
558
vsp = stream_profile.as_video_stream_profile()
559
print(f" {vsp.stream_type().name}: {vsp.width()}x{vsp.height()} "
560
f"{vsp.format().name} @ {vsp.fps()}fps")
561
else:
562
print(f" {stream_profile.stream_type().name}: "
563
f"{stream_profile.format().name} @ {stream_profile.fps()}fps")
564
565
except Exception as e:
566
print(f"Failed to start recording: {e}")
567
self.is_recording = False
568
569
def record_frames(self, max_frames=None, max_duration=None):
570
"""Record frames with optional limits."""
571
if not self.is_recording:
572
print("Not recording")
573
return
574
575
print("Recording frames... Press Ctrl+C to stop")
576
577
try:
578
while self.is_recording:
579
frames = self.pipeline.wait_for_frames()
580
self.frame_count += 1
581
582
# Print progress
583
if self.frame_count % 30 == 0:
584
elapsed = time.time() - self.start_time
585
fps = self.frame_count / elapsed
586
print(f"Recorded {self.frame_count} framesets "
587
f"({elapsed:.1f}s, {fps:.1f} fps)")
588
589
# Check limits
590
if max_frames and self.frame_count >= max_frames:
591
print(f"Reached frame limit: {max_frames}")
592
break
593
594
if max_duration and (time.time() - self.start_time) >= max_duration:
595
print(f"Reached time limit: {max_duration}s")
596
break
597
598
except KeyboardInterrupt:
599
print("\nRecording stopped by user")
600
except Exception as e:
601
print(f"Recording error: {e}")
602
603
def stop_recording(self):
604
"""Stop recording and cleanup."""
605
if not self.is_recording:
606
return
607
608
self.is_recording = False
609
610
if self.pipeline:
611
self.pipeline.stop()
612
self.pipeline = None
613
614
elapsed = time.time() - self.start_time if self.start_time else 0
615
avg_fps = self.frame_count / elapsed if elapsed > 0 else 0
616
617
print(f"Recording completed:")
618
print(f" Frames: {self.frame_count}")
619
print(f" Duration: {elapsed:.1f}s")
620
print(f" Average FPS: {avg_fps:.1f}")
621
print(f" File: {self.output_file}")
622
623
# Signal handler for clean shutdown
624
def signal_handler(sig, frame):
625
print("\nShutdown signal received")
626
if recorder:
627
recorder.stop_recording()
628
sys.exit(0)
629
630
# Setup recording
631
recorder = PipelineRecorder("controlled_recording.bag")
632
signal.signal(signal.SIGINT, signal_handler)
633
634
# Start recording
635
recorder.start_recording()
636
637
# Record for specific duration or frame count
638
recorder.record_frames(max_duration=30) # Record for 30 seconds
639
640
# Stop recording
641
recorder.stop_recording()
642
```
643
644
### Synchronized Recording and Playback
645
646
```python
647
import pyrealsense2 as rs
648
import time
649
import threading
650
651
class SynchronizedProcessor:
652
def __init__(self):
653
self.frame_queue = rs.frame_queue(capacity=100)
654
self.processed_count = 0
655
self.is_processing = False
656
657
def start_processing(self):
658
"""Start processing thread."""
659
self.is_processing = True
660
self.processing_thread = threading.Thread(target=self._processing_loop)
661
self.processing_thread.start()
662
663
def stop_processing(self):
664
"""Stop processing thread."""
665
self.is_processing = False
666
if hasattr(self, 'processing_thread'):
667
self.processing_thread.join()
668
669
def _processing_loop(self):
670
"""Background processing loop."""
671
while self.is_processing:
672
try:
673
frames = self.frame_queue.wait_for_frame(timeout_ms=100)
674
self._process_frameset(frames.as_frameset())
675
self.processed_count += 1
676
677
except rs.error:
678
continue # Timeout, continue loop
679
680
def _process_frameset(self, frameset):
681
"""Process a single frameset."""
682
# Example processing: just extract frame info
683
depth = frameset.get_depth_frame()
684
color = frameset.get_color_frame()
685
686
if depth and color:
687
timestamp = frameset.get_timestamp()
688
center_distance = depth.get_distance(320, 240)
689
690
if self.processed_count % 30 == 0:
691
print(f"Processed frame at {timestamp/1000:.3f}s, "
692
f"center distance: {center_distance:.3f}m")
693
694
def record_with_processing(duration_seconds=10):
695
"""Record while simultaneously processing frames."""
696
print(f"Recording with processing for {duration_seconds} seconds...")
697
698
# Setup processor
699
processor = SynchronizedProcessor()
700
processor.start_processing()
701
702
# Setup recording
703
config = rs.config()
704
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)
705
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)
706
config.enable_record_to_file("sync_recording.bag")
707
708
pipeline = rs.pipeline()
709
profile = pipeline.start(config)
710
711
try:
712
start_time = time.time()
713
frame_count = 0
714
715
while time.time() - start_time < duration_seconds:
716
frames = pipeline.wait_for_frames()
717
718
# Send to recorder (automatic via config)
719
# Send to processor
720
processor.frame_queue.enqueue(frames)
721
722
frame_count += 1
723
724
elapsed = time.time() - start_time
725
print(f"Recording completed: {frame_count} frames in {elapsed:.1f}s")
726
727
finally:
728
pipeline.stop()
729
processor.stop_processing()
730
731
print(f"Processed {processor.processed_count} framesets during recording")
732
733
def playback_with_processing(bag_file):
734
"""Playback with synchronized processing."""
735
print(f"Playing back {bag_file} with processing...")
736
737
# Setup processor
738
processor = SynchronizedProcessor()
739
processor.start_processing()
740
741
# Setup playback
742
config = rs.config()
743
config.enable_device_from_file(bag_file, repeat_playback=False)
744
745
pipeline = rs.pipeline()
746
profile = pipeline.start(config)
747
748
# Get playback device
749
device = profile.get_device()
750
playback = device.as_playback()
751
playback.set_real_time(True) # Real-time playback
752
753
try:
754
frame_count = 0
755
756
while True:
757
frames = pipeline.wait_for_frames()
758
759
# Send to processor
760
processor.frame_queue.enqueue(frames)
761
762
frame_count += 1
763
764
# Print progress
765
if frame_count % 60 == 0:
766
position = playback.get_position()
767
duration = playback.get_duration()
768
progress = (position / duration) * 100
769
print(f"Playback progress: {progress:.1f}%")
770
771
except rs.error:
772
print("Playback completed")
773
774
finally:
775
pipeline.stop()
776
processor.stop_processing()
777
778
print(f"Played back {frame_count} frames")
779
print(f"Processed {processor.processed_count} framesets during playback")
780
781
# Example usage
782
if __name__ == "__main__":
783
# Record with processing
784
record_with_processing(duration_seconds=5)
785
786
# Wait a moment
787
time.sleep(1)
788
789
# Playback with processing
790
playback_with_processing("sync_recording.bag")
791
```
792
793
## File Format and Compression
794
795
### .bag File Format
796
- ROS bag-compatible format
797
- Contains timestamped frame data
798
- Includes device metadata and stream configurations
799
- Supports multiple streams synchronized by timestamp
800
801
### Compression Options
802
- Lossless compression available for reduced file size
803
- Trade-off between file size and processing overhead
804
- Useful for long-duration recordings or storage constraints
805
806
### File Size Considerations
807
- Uncompressed: ~150MB per minute for depth+color at 640x480@30fps
808
- Compressed: ~50-80MB per minute depending on scene content
809
- Add ~25MB per minute for each additional infrared stream
810
811
## Use Cases
812
813
### Development and Testing
814
- Record test scenarios for reproducible debugging
815
- Validate algorithm performance across different datasets
816
- Compare processing results between different approaches
817
818
### Data Collection
819
- Gather training data for machine learning models
820
- Document camera performance in various conditions
821
- Create benchmark datasets for evaluation
822
823
### Offline Processing
824
- Process recorded data without live camera
825
- Apply different filter settings to same source data
826
- Generate processed results for analysis and comparison