0
# Audio System
1

2
Mopidy's audio system provides a GStreamer-based audio pipeline with playback control, volume management, and support for various audio formats and output methods. The system handles audio streaming, format conversion, and hardware integration through a flexible architecture.
3

4
## Capabilities
5

6
### Audio Actor
7

8
The main audio system actor that manages the GStreamer pipeline and audio playback operations.
9

10
```python { .api }
11
class Audio(pykka.ThreadingActor):
12
    """
13
    Main audio actor managing GStreamer-based audio pipeline.
14
    
15
    Parameters:
16
    - config: Audio configuration
17
    - mixer: Audio mixer instance
18
    """
19
    def __init__(self, config, mixer): ...
20
    
21
    def set_uri(self, uri):
22
        """
23
        Set URI for audio playback.
24
        
25
        Parameters:
26
        - uri (str): Audio URI to play
27
        
28
        Returns:
29
        - bool: True if URI was set successfully
30
        """
31
        ...
32
    
33
    def get_uri(self):
34
        """
35
        Get current audio URI.
36
        
37
        Returns:
38
        - str or None: Current audio URI
39
        """
40
        ...
41
    
42
    def start_playback(self):
43
        """
44
        Start audio playback.
45
        
46
        Returns:
47
        - bool: True if playback started successfully
48
        """
49
        ...
50
    
51
    def pause_playback(self):
52
        """
53
        Pause audio playback.
54
        
55
        Returns:
56
        - bool: True if playback was paused successfully
57
        """
58
        ...
59
    
60
    def prepare_change(self):
61
        """
62
        Prepare for URI change by pausing.
63
        
64
        Returns:
65
        - bool: True if prepared successfully
66
        """
67
        ...
68
    
69
    def stop_playback(self):
70
        """
71
        Stop audio playback and reset pipeline.
72
        
73
        Returns:
74
        - bool: True if playback was stopped successfully
75
        """
76
        ...
77
    
78
    def get_position(self):
79
        """
80
        Get current playback position.
81
        
82
        Returns:
83
        - int: Position in milliseconds
84
        """
85
        ...
86
    
87
    def set_position(self, position):
88
        """
89
        Seek to specific position.
90
        
91
        Parameters:
92
        - position (int): Position in milliseconds
93
        
94
        Returns:
95
        - bool: True if seek was successful
96
        """
97
        ...
98
    
99
    def get_volume(self):
100
        """
101
        Get current volume level.
102
        
103
        Returns:
104
        - int or None: Volume level (0-100)
105
        """
106
        ...
107
    
108
    def set_volume(self, volume):
109
        """
110
        Set volume level.
111
        
112
        Parameters:
113
        - volume (int): Volume level (0-100)
114
        
115
        Returns:
116
        - bool: True if volume was set successfully
117
        """
118
        ...
119
    
120
    def get_mute(self):
121
        """
122
        Get mute state.
123
        
124
        Returns:
125
        - bool or None: True if muted
126
        """
127
        ...
128
    
129
    def set_mute(self, mute):
130
        """
131
        Set mute state.
132
        
133
        Parameters:
134
        - mute (bool): True to mute, False to unmute
135
        
136
        Returns:
137
        - bool: True if mute state was changed successfully
138
        """
139
        ...
140
    
141
    def set_metadata(self, track):
142
        """
143
        Set track metadata for current playback.
144
        
145
        Parameters:
146
        - track (Track): Track metadata
147
        """
148
        ...
149
    
150
    def get_appsrc(self):
151
        """
152
        Get GStreamer appsrc element for custom audio sources.
153
        
154
        Returns:
155
        - GstElement or None: appsrc element
156
        """
157
        ...
158
```
159

160
Usage example:
161
```python
162
# Audio actor is typically managed by core system
163
audio_actor = Audio.start(config=audio_config, mixer=mixer_instance)
164
audio_ref = audio_actor.actor_ref
165

166
# Control playback
167
audio_ref.tell({"method": "set_uri", "uri": "file:///path/to/song.mp3"})
168
audio_ref.tell({"method": "start_playback"})
169
audio_ref.tell({"method": "set_volume", "volume": 75})
170
```
171

172
### Playback State Management
173

174
Constants and utilities for managing audio playback states.
175

176
```python { .api }
177
class PlaybackState:
178
    """Constants for audio playback states."""
179
    STOPPED = "stopped"
180
    PLAYING = "playing"
181
    PAUSED = "paused"
182
    
183
    @classmethod
184
    def all_states(cls):
185
        """
186
        Get all valid playback states.
187
        
188
        Returns:
189
        - list[str]: All playback state constants
190
        """
191
        ...
192
```
193

194
### Audio Utility Functions
195

196
Helper functions for audio processing, format detection, and stream management.
197

198
```python { .api }
199
def supported_uri_schemes():
200
    """
201
    Get list of supported URI schemes for audio playback.
202
    
203
    Returns:
204
    - list[str]: Supported URI schemes (file, http, https, etc.)
205
    """
206
    ...
207

208
def calculate_duration(num_samples, sample_rate):
209
    """
210
    Calculate duration from audio samples and sample rate.
211
    
212
    Parameters:
213
    - num_samples (int): Number of audio samples
214
    - sample_rate (int): Sample rate in Hz
215
    
216
    Returns:
217
    - int: Duration in milliseconds
218
    """
219
    ...
220

221
def millisecond_to_clocktime(milliseconds):
222
    """
223
    Convert milliseconds to GStreamer clocktime format.
224
    
225
    Parameters:
226
    - milliseconds (int): Time in milliseconds
227
    
228
    Returns:
229
    - int: Time in GStreamer clocktime format (nanoseconds)
230
    """
231
    ...
232

233
def create_buffer(data, timestamp=None, duration=None):
234
    """
235
    Create GStreamer buffer from audio data.
236
    
237
    Parameters:
238
    - data (bytes): Audio data
239
    - timestamp (int, optional): Buffer timestamp in nanoseconds
240
    - duration (int, optional): Buffer duration in nanoseconds
241
    
242
    Returns:
243
    - GstBuffer: GStreamer buffer object
244
    """
245
    ...
246
```
247

248
Usage example:
249
```python
250
from mopidy.audio import calculate_duration, supported_uri_schemes
251

252
# Check supported formats
253
schemes = supported_uri_schemes()
254
print(f"Supported schemes: {schemes}")
255

256
# Calculate duration for audio data
257
samples = 44100 * 180  # 3 minutes at 44.1kHz
258
duration_ms = calculate_duration(samples, 44100)
259
print(f"Duration: {duration_ms}ms")
260
```
261

262
### Audio Event Listeners
263

264
Event notification system for audio state changes and pipeline events.
265

266
```python { .api }
267
class AudioListener:
268
    """Base class for audio event listeners."""
269
    
270
    def reached_end_of_stream(self):
271
        """Called when audio stream reaches end."""
272
        ...
273
    
274
    def position_changed(self, position):
275
        """
276
        Called when playback position changes.
277
        
278
        Parameters:
279
        - position (int): New position in milliseconds
280
        """
281
        ...
282
    
283
    def state_changed(self, old_state, new_state):
284
        """
285
        Called when playback state changes.
286
        
287
        Parameters:
288
        - old_state (str): Previous playback state
289
        - new_state (str): New playback state
290
        """
291
        ...
292
    
293
    def stream_changed(self, uri):
294
        """
295
        Called when audio stream URI changes.
296
        
297
        Parameters:
298
        - uri (str): New stream URI
299
        """
300
        ...
301
    
302
    def tags_changed(self, tags):
303
        """
304
        Called when stream tags/metadata change.
305
        
306
        Parameters:
307
        - tags (dict): Stream tags and metadata
308
        """
309
        ...
310
```
311

312
Usage example:
313
```python
314
class MyAudioListener(AudioListener):
315
    def reached_end_of_stream(self):
316
        print("Track finished playing")
317
    
318
    def position_changed(self, position):
319
        print(f"Playback position: {position}ms")
320
    
321
    def state_changed(self, old_state, new_state):
322
        print(f"State changed: {old_state} -> {new_state}")
323

324
# Register listener with audio system
325
listener = MyAudioListener()
326
# Registration depends on specific audio system setup
327
```
328

329
### Audio Configuration
330

331
Configuration schema and options for the audio system.
332

333
```python { .api }
334
# Audio configuration schema
335
_audio_schema = ConfigSchema("audio")
336
_audio_schema["mixer"] = String()  # Mixer implementation to use
337
_audio_schema["mixer_volume"] = Integer(optional=True, minimum=0, maximum=100)
338
_audio_schema["output"] = String()  # Audio output configuration
339
_audio_schema["buffer_time"] = Integer(optional=True, minimum=1)  # Buffer time in milliseconds
340

341
# Example audio configuration
342
"""
343
[audio]
344
mixer = software
345
mixer_volume = 80
346
output = autoaudiosink
347
buffer_time = 200
348
"""
349
```
350

351
### GStreamer Pipeline Integration
352

353
Low-level GStreamer pipeline management and element handling.
354

355
```python { .api }
356
class GstElement:
357
    """Type hint for GStreamer elements."""
358
    pass
359

360
def get_audio_element():
361
    """
362
    Get main audio pipeline element.
363
    
364
    Returns:
365
    - GstElement: Main audio pipeline element
366
    """
367
    ...
368

369
def setup_output_element(config):
370
    """
371
    Set up audio output element based on configuration.
372
    
373
    Parameters:
374
    - config (dict): Audio configuration
375
    
376
    Returns:
377
    - GstElement: Configured output element
378
    """
379
    ...
380

381
def setup_mixer_element(config):
382
    """
383
    Set up audio mixer element based on configuration.
384
    
385
    Parameters:
386
    - config (dict): Audio configuration
387
    
388
    Returns:
389
    - GstElement: Configured mixer element
390
    """
391
    ...
392
```
393

394
### Audio Format Support
395

396
Information about supported audio formats, codecs, and containers.
397

398
```python { .api }
399
def get_supported_formats():
400
    """
401
    Get list of supported audio formats.
402
    
403
    Returns:
404
    - list[str]: Supported audio format extensions
405
    """
406
    ...
407

408
def get_supported_codecs():
409
    """
410
    Get list of supported audio codecs.
411
    
412
    Returns:
413
    - list[str]: Supported audio codec names
414
    """
415
    ...
416

417
def detect_format(uri):
418
    """
419
    Detect audio format from URI or file.
420
    
421
    Parameters:
422
    - uri (str): Audio URI or file path
423
    
424
    Returns:
425
    - str or None: Detected format or None if unknown
426
    """
427
    ...
428
```
429

430
### Stream Management
431

432
Utilities for handling audio streams, including network streams and local files.
433

434
```python { .api }
435
def prepare_stream(uri, headers=None, timeout=None):
436
    """
437
    Prepare audio stream for playback.
438
    
439
    Parameters:
440
    - uri (str): Stream URI
441
    - headers (dict, optional): HTTP headers for network streams
442
    - timeout (int, optional): Connection timeout in seconds
443
    
444
    Returns:
445
    - bool: True if stream was prepared successfully
446
    """
447
    ...
448

449
def get_stream_metadata(uri):
450
    """
451
    Get metadata from audio stream.
452
    
453
    Parameters:
454
    - uri (str): Stream URI
455
    
456
    Returns:
457
    - dict: Stream metadata (title, artist, album, etc.)
458
    """
459
    ...
460

461
def validate_stream_uri(uri):
462
    """
463
    Validate that URI can be played by audio system.
464
    
465
    Parameters:
466
    - uri (str): URI to validate
467
    
468
    Returns:
469
    - bool: True if URI is valid and playable
470
    """
471
    ...
472
```
473

474
Usage example:
475
```python
476
# Prepare and validate streams
477
uri = "http://stream.example.com/radio.mp3"
478
if validate_stream_uri(uri):
479
    headers = {"User-Agent": "Mopidy/3.4.2"}
480
    if prepare_stream(uri, headers=headers, timeout=30):
481
        metadata = get_stream_metadata(uri)
482
        print(f"Stream title: {metadata.get('title', 'Unknown')}")
483
```
484

485
### Audio Buffer Management
486

487
Low-level buffer management for custom audio sources and processing.
488

489
```python { .api }
490
def create_audio_buffer(data, sample_rate=44100, channels=2, format="S16LE"):
491
    """
492
    Create audio buffer from raw audio data.
493
    
494
    Parameters:
495
    - data (bytes): Raw audio data
496
    - sample_rate (int): Sample rate in Hz
497
    - channels (int): Number of audio channels  
498
    - format (str): Audio format (S16LE, F32LE, etc.)
499
    
500
    Returns:
501
    - GstBuffer: GStreamer audio buffer
502
    """
503
    ...
504

505
def buffer_to_samples(buffer):
506
    """
507
    Extract audio samples from GStreamer buffer.
508
    
509
    Parameters:
510
    - buffer (GstBuffer): GStreamer buffer
511
    
512
    Returns:
513
    - bytes: Raw audio sample data
514
    """
515
    ...
516

517
def get_buffer_info(buffer):
518
    """
519
    Get information about audio buffer.
520
    
521
    Parameters:
522
    - buffer (GstBuffer): GStreamer buffer
523
    
524
    Returns:
525
    - dict: Buffer info (duration, timestamp, size, etc.)
526
    """
527
    ...
528
```
529

530
### Audio Device Management
531

532
Interface for discovering and managing audio devices.
533

534
```python { .api }
535
def get_audio_devices():
536
    """
537
    Get list of available audio devices.
538
    
539
    Returns:
540
    - list[dict]: Available audio devices with properties
541
    """
542
    ...
543

544
def get_default_audio_device():
545
    """
546
    Get default audio output device.
547
    
548
    Returns:
549
    - dict: Default device information
550
    """
551
    ...
552

553
def set_audio_device(device_id):
554
    """
555
    Set audio output device.
556
    
557
    Parameters:
558
    - device_id (str): Device identifier
559
    
560
    Returns:
561
    - bool: True if device was set successfully
562
    """
563
    ...
564
```
565

566
## Audio System Architecture
567

568
### Pipeline Structure
569

570
The audio system uses GStreamer pipelines with the following general structure:
571

572
```
573
uridecodebin -> audioconvert -> audioresample -> volume -> output
574
```
575

576
Key components:
577
- **uridecodebin**: Automatically decodes audio from various URI schemes
578
- **audioconvert**: Converts between audio formats as needed
579
- **audioresample**: Resamples audio to match output requirements
580
- **volume**: Provides software volume control
581
- **output**: Routes audio to speakers, files, or other destinations
582

583
### Buffer Management
584

585
Audio data flows through the pipeline in discrete buffers, each containing:
586
- Raw audio samples
587
- Timestamp information
588
- Duration metadata
589
- Format specifications
590

591
### Event Handling
592

593
The audio system generates events for:
594
- Playback state changes (play, pause, stop)
595
- End of stream notifications
596
- Position updates
597
- Error conditions
598
- Metadata changes
599

600
### Thread Safety
601

602
The audio system uses Pykka actors to ensure thread-safe operation across multiple components while maintaining real-time audio performance requirements.
603

604
## Mixer System
605

606
Mopidy's mixer system provides audio volume and mute control across different audio backends and hardware configurations.
607

608
### Mixer Base Class
609

610
Base mixer implementation that backends can extend to provide volume control.
611

612
```python { .api }
613
class Mixer:
614
    """
615
    Audio mixer API for volume and mute control.
616
    
617
    Parameters:
618
    - config (dict): Mopidy configuration
619
    """
620
    
621
    name: str  # Name of the mixer (matches extension name)
622
    
623
    def __init__(self, config): ...
624
    
625
    def get_volume(self):
626
        """
627
        Get volume level on a linear scale from 0 to 100.
628
        
629
        Returns:
630
        - int or None: Volume level (0-100), None if unknown
631
        """
632
        ...
633
    
634
    def set_volume(self, volume):
635
        """
636
        Set volume level of the mixer.
637
        
638
        Parameters:
639
        - volume (int): Volume level (0-100)
640
        
641
        Returns:
642
        - bool: True if successful
643
        """
644
        ...
645
    
646
    def get_mute(self):
647
        """
648
        Get mute state of the mixer.
649
        
650
        Returns:
651
        - bool or None: True if muted, False if not, None if unknown
652
        """
653
        ...
654
    
655
    def set_mute(self, mute):
656
        """
657
        Mute or unmute the mixer.
658
        
659
        Parameters:
660
        - mute (bool): True to mute, False to unmute
661
        
662
        Returns:
663
        - bool: True if successful
664
        """
665
        ...
666
    
667
    def trigger_volume_changed(self, volume):
668
        """
669
        Send volume_changed event to all mixer listeners.
670
        
671
        Should be called by subclasses when volume changes.
672
        
673
        Parameters:
674
        - volume (int): New volume level
675
        """
676
        ...
677
    
678
    def trigger_mute_changed(self, mute):
679
        """
680
        Send mute_changed event to all mixer listeners.
681
        
682
        Should be called by subclasses when mute state changes.
683
        
684
        Parameters:
685
        - mute (bool): New mute state
686
        """
687
        ...
688
    
689
    def ping(self):
690
        """
691
        Check if the mixer actor is still alive.
692
        
693
        Returns:
694
        - bool: True if alive
695
        """
696
        ...
697
```
698

699
### Mixer Listener
700

701
Event listener interface for receiving mixer state change notifications.
702

703
```python { .api }
704
class MixerListener:
705
    """Listener interface for mixer events."""
706
    
707
    def volume_changed(self, volume):
708
        """
709
        Called after the volume has changed.
710
        
711
        Parameters:
712
        - volume (int): New volume level (0-100)
713
        """
714
        ...
715
    
716
    def mute_changed(self, mute):
717
        """
718
        Called after the mute state has changed.
719
        
720
        Parameters:
721
        - mute (bool): New mute state
722
        """
723
        ...
724

725
def send(event, **kwargs):
726
    """
727
    Helper to send mixer listener events.
728
    
729
    Parameters:
730
    - event (str): Event name ('volume_changed' or 'mute_changed')
731
    - **kwargs: Event parameters
732
    """
733
    ...
734
```
735

736
### Usage Examples
737

738
```python
739
# Custom mixer implementation
740
class MyMixer(Mixer):
741
    name = 'mymixer'
742
    
743
    def __init__(self, config):
744
        super().__init__(config)
745
        self._volume = 50
746
        self._muted = False
747
    
748
    def get_volume(self):
749
        return self._volume
750
    
751
    def set_volume(self, volume):
752
        if 0 <= volume <= 100:
753
            self._volume = volume
754
            self.trigger_volume_changed(volume)
755
            return True
756
        return False
757
    
758
    def get_mute(self):
759
        return self._muted
760
    
761
    def set_mute(self, mute):
762
        self._muted = bool(mute)
763
        self.trigger_mute_changed(self._muted)
764
        return True
765

766
# Using mixer events
767
class MyMixerListener(MixerListener):
768
    def volume_changed(self, volume):
769
        print(f"Volume changed to {volume}%")
770
    
771
    def mute_changed(self, mute):
772
        print(f"Mute {'enabled' if mute else 'disabled'}")
773
```