0
# Codec Processing
1

2
Codec processing functionality for parsing and handling H.264 video and ADTS/AAC audio streams with timing analysis, metadata extraction, and stream validation.
3

4
## Capabilities
5

6
### ADTS Stream Processing
7

8
Processes ADTS (Audio Data Transport Stream) frames containing AAC audio data with support for partial segments and timing analysis.
9

10
```javascript { .api }
11
/**
12
 * ADTS/AAC stream processor
13
 * @param handlePartialSegments - Whether to handle incomplete segments
14
 */
15
class AdtsStream extends Stream {
16
  constructor(handlePartialSegments?: boolean);
17
  
18
  /** Process audio packet data */
19
  push(packet: Uint8Array): void;
20
  
21
  /** Finalize processing and output remaining data */
22
  flush(): void;
23
  
24
  /** Reset internal parser state */
25
  reset(): void;
26
  
27
  /** End timeline processing */
28
  endTimeline(): void;
29
  
30
  /** Log skip warnings for discontinuities */
31
  skipWarn_(start: number, end: number): void;
32
}
33

34
/** Supported ADTS sampling frequencies in Hz */
35
const ADTS_SAMPLING_FREQUENCIES: number[];
36
```
37

38
**Usage Example:**
39

40
```javascript
41
const muxjs = require("mux.js");
42

43
const adtsStream = new muxjs.codecs.Adts(true);
44

45
adtsStream.on('data', function(audioData) {
46
  // audioData contains parsed AAC frames with timing
47
  console.log('Audio frame:', audioData);
48
});
49

50
// Process ADTS audio data
51
adtsStream.push(adtsBytes);
52
adtsStream.flush();
53
```
54

55
### H.264 Stream Processing
56

57
Comprehensive H.264 video stream processing including NAL unit parsing, timing analysis, and parameter set extraction.
58

59
```javascript { .api }
60
/**
61
 * H.264 video stream processor
62
 */
63
class H264Stream extends Stream {
64
  constructor();
65
  
66
  /** Process H.264 video packet */
67
  push(packet: Uint8Array): void;
68
  
69
  /** Flush buffered video data */
70
  flush(): void;
71
  
72
  /** Partial flush for segment boundaries */
73
  partialFlush(): void;
74
  
75
  /** Reset parser state */
76
  reset(): void;
77
  
78
  /** End timeline processing */
79
  endTimeline(): void;
80
}
81

82
/**
83
 * NAL (Network Abstraction Layer) unit byte stream parser
84
 */
85
class NalByteStream extends Stream {
86
  constructor();
87
  
88
  /** Process NAL byte data and extract units */
89
  push(data: Uint8Array): void;
90
  
91
  /** Reset NAL parser state */
92
  reset(): void;
93
  
94
  /** Flush remaining NAL data */
95
  flush(): void;
96
  
97
  /** End timeline processing */
98
  endTimeline(): void;
99
}
100

101
/** Profile IDCs that have optional SPS data */
102
const PROFILES_WITH_OPTIONAL_SPS_DATA: { [profileIdc: number]: boolean };
103
```
104

105
**Usage Example:**
106

107
```javascript
108
const muxjs = require("mux.js");
109

110
// Create H.264 processing pipeline
111
const nalByteStream = new muxjs.codecs.h264.NalByteStream();
112
const h264Stream = new muxjs.codecs.h264.H264Stream();
113

114
// Connect streams
115
nalByteStream.pipe(h264Stream);
116

117
h264Stream.on('data', function(videoData) {
118
  // videoData contains parsed H.264 frames with timing and metadata
119
  console.log('Video frame:', videoData);
120
});
121

122
// Process H.264 elementary stream
123
nalByteStream.push(h264Bytes);
124
nalByteStream.flush();
125
```
126

127
## Advanced Usage
128

129
### Codec Stream Pipeline
130

131
```javascript
132
const muxjs = require("mux.js");
133

134
// Create codec processing pipeline
135
const nalByteStream = new muxjs.codecs.h264.NalByteStream();
136
const h264Stream = new muxjs.codecs.h264.H264Stream();
137
const adtsStream = new muxjs.codecs.Adts();
138

139
// Set up video processing
140
nalByteStream.pipe(h264Stream);
141

142
h264Stream.on('data', function(videoData) {
143
  // Process parsed video frames
144
  console.log('Video PTS:', videoData.pts);
145
  console.log('Video DTS:', videoData.dts);
146
});
147

148
// Set up audio processing
149
adtsStream.on('data', function(audioData) {
150
  // Process parsed audio frames
151
  console.log('Audio PTS:', audioData.pts);
152
  console.log('Sample rate:', audioData.sampleRate);
153
});
154

155
// Handle errors
156
[nalByteStream, h264Stream, adtsStream].forEach(stream => {
157
  stream.on('error', function(error) {
158
    console.error('Codec processing error:', error);
159
  });
160
});
161
```
162

163
### Timing and Synchronization
164

165
```javascript
166
// Handle timing information from codec streams
167
h264Stream.on('data', function(videoData) {
168
  if (videoData.pts !== undefined) {
169
    console.log('Video presentation time:', videoData.pts);
170
  }
171
  if (videoData.dts !== undefined) {
172
    console.log('Video decode time:', videoData.dts);
173
  }
174
});
175

176
adtsStream.on('data', function(audioData) {
177
  if (audioData.pts !== undefined) {
178
    console.log('Audio presentation time:', audioData.pts);
179
    console.log('Sample rate:', audioData.config.sampleRate);
180
  }
181
});
182
```
183

184
## Constants and Configuration
185

186
### ADTS Sampling Frequencies
187

188
```javascript { .api }
189
/** 
190
 * Array of supported ADTS sampling frequencies in Hz
191
 * Index corresponds to sampling frequency index in ADTS header
192
 */
193
const ADTS_SAMPLING_FREQUENCIES: number[] = [
194
  96000, 88200, 64000, 48000, 44100, 32000,
195
  24000, 22050, 16000, 12000, 11025, 8000, 7350
196
];
197
```
198

199
### H.264 Profile Information
200

201
```javascript { .api }
202
/**
203
 * Profile IDCs that have optional SPS (Sequence Parameter Set) data
204
 * Used for determining parsing requirements
205
 */
206
const PROFILES_WITH_OPTIONAL_SPS_DATA: { [profileIdc: number]: boolean };
207
```
208

209
## Error Handling
210

211
Codec streams emit standard events for error handling:
212

213
```javascript
214
const adtsStream = new muxjs.codecs.Adts();
215

216
adtsStream.on('error', function(error) {
217
  console.error('ADTS parsing error:', error.message);
218
});
219

220
adtsStream.on('warning', function(warning) {
221
  console.warn('ADTS parsing warning:', warning.message);
222
  // Common warnings: discontinuities, missing frames, timing issues
223
});
224
```