0
# Partial Segment Processing
1

2
Partial segment processing functionality for handling incomplete media segments and progressive transmuxing scenarios, ideal for live streaming and adaptive bitrate applications where segments are processed incrementally.
3

4
## Capabilities
5

6
### Partial Transmuxer
7

8
Advanced transmuxer that automatically detects input format (MPEG-2 TS or ADTS/AAC) and processes segments progressively without requiring complete segments upfront.
9

10
```javascript { .api }
11
/**
12
 * Partial segment transmuxer with automatic format detection
13
 * @param options - Configuration options for partial processing
14
 */
15
class Transmuxer extends Stream {
16
  constructor(options?: PartialTransmuxerOptions);
17
  
18
  /** Process input bytes incrementally */
19
  push(bytes: Uint8Array): void;
20
  
21
  /** Complete processing and output final segments */
22
  flush(): void;
23
  
24
  /** Reset transmuxer state for new stream */
25
  reset(): void;
26
  
27
  /** Set base media decode time for timing alignment */
28
  setBaseMediaDecodeTime(time: number): void;
29
  
30
  /** Align GOP timing with another stream */
31
  alignGopsWith(gopsToAlignWith: any[]): void;
32
}
33

34
interface PartialTransmuxerOptions {
35
  /** Base media decode time in 90kHz units */
36
  baseMediaDecodeTime?: number;
37
  /** Keep original timestamps instead of rebasing */
38
  keepOriginalTimestamps?: boolean;
39
  /** Remove CEA-608 captions during processing */
40
  remux?: boolean;
41
  /** Enable real-time processing mode */
42
  realtime?: boolean;
43
}
44
```
45

46
**Usage Example:**
47

48
```javascript
49
const muxjs = require("mux.js");
50

51
const partialTransmuxer = new muxjs.partial.Transmuxer({
52
  baseMediaDecodeTime: 0,
53
  keepOriginalTimestamps: false,
54
  realtime: true
55
});
56

57
partialTransmuxer.on('data', function(segment) {
58
  if (segment.type === 'video') {
59
    console.log('Video segment:', segment.data.byteLength, 'bytes');
60
    // segment.data contains partial video segment
61
  } else if (segment.type === 'audio') {
62
    console.log('Audio segment:', segment.data.byteLength, 'bytes');
63
    // segment.data contains partial audio segment
64
  }
65
});
66

67
partialTransmuxer.on('partialdone', function() {
68
  console.log('Partial segment complete');
69
});
70

71
partialTransmuxer.on('done', function() {
72
  console.log('Complete segment processing finished');
73
});
74

75
// Process data incrementally as it arrives
76
partialTransmuxer.push(chunkData1);
77
partialTransmuxer.push(chunkData2);
78
partialTransmuxer.flush();
79
```
80

81
### Audio Segment Stream
82

83
Specialized stream for processing audio data into ISO BMFF media segments suitable for progressive loading and MSE consumption.
84

85
```javascript { .api }
86
/**
87
 * Constructs single-track audio segments from AAC data
88
 */
89
class AudioSegmentStream extends Stream {
90
  constructor(track: AudioTrack, options?: AudioSegmentOptions);
91
  
92
  /** Process audio frame data */
93
  push(data: AudioFrameData): void;
94
  
95
  /** Complete audio segment and emit output */
96
  flush(): void;
97
  
98
  /** Reset audio segment processor */
99
  reset(): void;
100
  
101
  /** Set earliest decode time for synchronization */
102
  setEarliestDts(dts: number): void;
103
}
104

105
interface AudioTrack {
106
  id: number;
107
  codec: string;
108
  sampleRate: number;
109
  channelCount: number;
110
  timescale: number;
111
  timelineStartInfo: TimelineInfo;
112
}
113

114
interface AudioFrameData {
115
  pts: number;
116
  dts?: number;
117
  data: Uint8Array;
118
  audioObjectType?: number;
119
  channelcount?: number;
120
  samplerate?: number;
121
}
122

123
interface AudioSegmentOptions {
124
  baseMediaDecodeTime?: number;
125
  keepOriginalTimestamps?: boolean;
126
}
127
```
128

129
**Usage Example:**
130

131
```javascript
132
const muxjs = require("mux.js");
133

134
// Define audio track configuration
135
const audioTrack = {
136
  id: 1,
137
  codec: 'mp4a.40.2', // AAC-LC
138
  sampleRate: 48000,
139
  channelCount: 2,
140
  timescale: 48000,
141
  timelineStartInfo: {
142
    baseMediaDecodeTime: 0
143
  }
144
};
145

146
const audioSegmentStream = new muxjs.partial.AudioSegmentStream(audioTrack, {
147
  baseMediaDecodeTime: 0
148
});
149

150
audioSegmentStream.on('data', function(segment) {
151
  console.log('Audio segment ready:', segment.byteLength, 'bytes');
152
  // segment contains ISO BMFF audio segment
153
});
154

155
// Process AAC frame data
156
audioSegmentStream.push({
157
  pts: 90000, // 1 second at 90kHz
158
  dts: 90000,
159
  data: aacFrameData,
160
  audioObjectType: 2, // AAC-LC
161
  channelcount: 2,
162
  samplerate: 48000
163
});
164
```
165

166
### Video Segment Stream
167

168
Specialized stream for processing video data into ISO BMFF media segments with support for progressive loading and synchronization.
169

170
```javascript { .api }
171
/**
172
 * Constructs single-track video segments from H.264 data
173
 */
174
class VideoSegmentStream extends Stream {
175
  constructor(track: VideoTrack, options?: VideoSegmentOptions);
176
  
177
  /** Process video frame data */
178
  push(data: VideoFrameData): void;
179
  
180
  /** Complete video segment and emit output */
181
  flush(): void;
182
  
183
  /** Reset video segment processor */
184
  reset(): void;
185
  
186
  /** Set earliest decode time for synchronization */
187
  setEarliestDts(dts: number): void;
188
}
189

190
interface VideoTrack {
191
  id: number;
192
  codec: string;
193
  width: number;
194
  height: number;
195
  timescale: number;
196
  timelineStartInfo: TimelineInfo;
197
}
198

199
interface VideoFrameData {
200
  pts: number;
201
  dts: number;
202
  data: Uint8Array;
203
  nalUnitType?: number;
204
  keyFrame?: boolean;
205
}
206

207
interface VideoSegmentOptions {
208
  baseMediaDecodeTime?: number;
209
  keepOriginalTimestamps?: boolean;
210
  alignGopsAtEnd?: boolean;
211
}
212

213
interface TimelineInfo {
214
  baseMediaDecodeTime: number;
215
  dts?: number;
216
}
217
```
218

219
**Usage Example:**
220

221
```javascript
222
const muxjs = require("mux.js");
223

224
// Define video track configuration
225
const videoTrack = {
226
  id: 2,
227
  codec: 'avc1.42E01E', // H.264 Baseline Profile
228
  width: 1920,
229
  height: 1080,
230
  timescale: 90000,
231
  timelineStartInfo: {
232
    baseMediaDecodeTime: 0
233
  }
234
};
235

236
const videoSegmentStream = new muxjs.partial.VideoSegmentStream(videoTrack, {
237
  baseMediaDecodeTime: 0,
238
  alignGopsAtEnd: true
239
});
240

241
videoSegmentStream.on('data', function(segment) {
242
  console.log('Video segment ready:', segment.byteLength, 'bytes');
243
  // segment contains ISO BMFF video segment
244
});
245

246
videoSegmentStream.on('timingInfo', function(timingInfo) {
247
  console.log('Video timing info:', timingInfo);
248
});
249

250
// Process H.264 frame data
251
videoSegmentStream.push({
252
  pts: 90000, // 1 second at 90kHz
253
  dts: 90000,
254
  data: h264FrameData,
255
  nalUnitType: 5, // IDR frame
256
  keyFrame: true
257
});
258
```
259

260
## Advanced Usage
261

262
### Progressive Streaming Pipeline
263

264
```javascript
265
const muxjs = require("mux.js");
266

267
// Create progressive streaming setup
268
const partialTransmuxer = new muxjs.partial.Transmuxer({
269
  baseMediaDecodeTime: 0,
270
  keepOriginalTimestamps: false,
271
  realtime: true
272
});
273

274
let audioBuffer = [];
275
let videoBuffer = [];
276

277
// Handle progressive segment data
278
partialTransmuxer.on('data', function(segment) {
279
  if (segment.type === 'audio') {
280
    audioBuffer.push(segment.data);
281
    
282
    // Emit audio segments when buffer reaches threshold
283
    if (audioBuffer.length >= 5) {
284
      const combinedAudio = concatenateBuffers(audioBuffer);
285
      sendToMediaSource(combinedAudio, 'audio');
286
      audioBuffer = [];
287
    }
288
  } else if (segment.type === 'video') {
289
    videoBuffer.push(segment.data);
290
    
291
    // Emit video segments when buffer reaches threshold
292
    if (videoBuffer.length >= 3) {
293
      const combinedVideo = concatenateBuffers(videoBuffer);
294
      sendToMediaSource(combinedVideo, 'video');
295
      videoBuffer = [];
296
    }
297
  }
298
});
299

300
// Handle timing events for synchronization
301
partialTransmuxer.on('videoTimingInfo', function(timingInfo) {
302
  console.log('Video timing:', timingInfo.start, '-', timingInfo.end);
303
});
304

305
partialTransmuxer.on('audioTimingInfo', function(timingInfo) {
306
  console.log('Audio timing:', timingInfo.start, '-', timingInfo.end);
307
});
308

309
// Process incoming stream chunks progressively
310
function processStreamChunk(chunkData) {
311
  partialTransmuxer.push(chunkData);
312
}
313

314
function finalizeStream() {
315
  partialTransmuxer.flush();
316
  
317
  // Send remaining buffered data
318
  if (audioBuffer.length > 0) {
319
    sendToMediaSource(concatenateBuffers(audioBuffer), 'audio');
320
  }
321
  if (videoBuffer.length > 0) {
322
    sendToMediaSource(concatenateBuffers(videoBuffer), 'video');
323
  }
324
}
325
```
326

327
### Live Stream Processing
328

329
```javascript
330
const muxjs = require("mux.js");
331

332
class LiveStreamProcessor {
333
  constructor() {
334
    this.partialTransmuxer = new muxjs.partial.Transmuxer({
335
      realtime: true,
336
      keepOriginalTimestamps: false
337
    });
338
    
339
    this.setupEventHandlers();
340
  }
341
  
342
  setupEventHandlers() {
343
    this.partialTransmuxer.on('data', (segment) => {
344
      this.handleSegment(segment);
345
    });
346
    
347
    this.partialTransmuxer.on('partialdone', () => {
348
      console.log('Partial segment processing complete');
349
    });
350
    
351
    this.partialTransmuxer.on('endedtimeline', () => {
352
      console.log('Timeline ended - preparing for discontinuity');
353
      this.prepareForDiscontinuity();
354
    });
355
  }
356
  
357
  processLiveChunk(data, timestamp) {
358
    // Set timing for live processing
359
    this.partialTransmuxer.setBaseMediaDecodeTime(timestamp);
360
    
361
    // Process the chunk
362
    this.partialTransmuxer.push(data);
363
  }
364
  
365
  handleSegment(segment) {
366
    // Emit segments immediately for low latency
367
    this.emit('segment', {
368
      type: segment.type,
369
      data: segment.data,
370
      timing: segment.timing
371
    });
372
  }
373
  
374
  prepareForDiscontinuity() {
375
    // Handle stream discontinuities (e.g., ad breaks, network issues)
376
    this.partialTransmuxer.reset();
377
  }
378
}
379

380
// Usage
381
const liveProcessor = new LiveStreamProcessor();
382

383
liveProcessor.on('segment', function(segment) {
384
  // Send segment to media source buffer immediately
385
  mediaSourceBuffer.appendBuffer(segment.data);
386
});
387
```
388

389
### Custom Segment Options
390

391
```javascript
392
// Advanced configuration for different streaming scenarios
393
const adaptiveStreamOptions = {
394
  baseMediaDecodeTime: 0,
395
  keepOriginalTimestamps: false,
396
  realtime: true,
397
  
398
  // Custom audio processing
399
  audioOptions: {
400
    segmentDuration: 2000, // 2 seconds
401
    maxBufferSize: 10 * 1024 * 1024 // 10MB
402
  },
403
  
404
  // Custom video processing  
405
  videoOptions: {
406
    alignGopsAtEnd: true,
407
    keyFrameInterval: 2000, // 2 seconds
408
    maxBufferSize: 50 * 1024 * 1024 // 50MB
409
  }
410
};
411

412
const adaptiveTransmuxer = new muxjs.partial.Transmuxer(adaptiveStreamOptions);
413

414
// Handle different quality levels
415
adaptiveTransmuxer.on('data', function(segment) {
416
  if (segment.type === 'video') {
417
    // Route video segments based on quality/bitrate
418
    routeVideoSegment(segment, detectQualityLevel(segment));
419
  } else if (segment.type === 'audio') {
420
    // Audio segments typically don't change quality as often
421
    routeAudioSegment(segment);
422
  }
423
});
424
```
425

426
## Error Handling
427

428
Partial processing streams provide detailed error information for troubleshooting progressive streaming issues:
429

430
```javascript
431
const partialTransmuxer = new muxjs.partial.Transmuxer();
432

433
partialTransmuxer.on('error', function(error) {
434
  console.error('Partial transmuxing error:', error.message);
435
  // Common errors: incomplete segments, timing misalignment, codec changes
436
});
437

438
partialTransmuxer.on('warning', function(warning) {
439
  console.warn('Partial transmuxing warning:', warning.message);
440
  // Common warnings: frame drops, timing adjustments, buffer overflows
441
});
442

443
// Handle timeline events for synchronization issues
444
partialTransmuxer.on('videoTimingInfo', function(timingInfo) {
445
  if (timingInfo.start > timingInfo.end) {
446
    console.warn('Invalid video timing detected');
447
  }
448
});
449

450
partialTransmuxer.on('audioTimingInfo', function(timingInfo) {
451
  if (Math.abs(timingInfo.start - timingInfo.end) > 10000) { // > ~100ms
452
    console.warn('Large audio timing gap detected');
453
  }
454
});
455
```