0
# Pipeline Configuration
1

2
Multiple processing pipeline implementations optimized for different hardware configurations and performance requirements. Pipelines handle the decoding of raw sensor data into usable RGB, depth, and infrared frames.
3

4
## Capabilities
5

6
### PacketPipeline Base Class
7

8
Abstract base class for all pipeline implementations providing common interface for RGB and depth packet processing.
9

10
```java { .api }
11
/**
12
 * Base class for other pipeline classes.
13
 * Methods in this class are reserved for internal use.
14
 */
15
abstract class PacketPipeline {
16
    /** Default constructor */
17
    PacketPipeline();
18
    
19
    /** Get RGB packet parser for color data processing */
20
    DataCallback getRgbPacketParser();
21
    
22
    /** Get IR packet parser for depth/IR data processing */
23
    DataCallback getIrPacketParser();
24
    
25
    /** Get RGB packet processor instance */
26
    RgbPacketProcessor getRgbPacketProcessor();
27
    
28
    /** Get depth packet processor instance */
29
    DepthPacketProcessor getDepthPacketProcessor();
30
}
31
```
32

33
### CPU Pipeline
34

35
CPU-based processing pipeline suitable for systems without GPU acceleration or for maximum compatibility.
36

37
```java { .api }
38
/**
39
 * Pipeline with CPU depth processing.
40
 * Most compatible option but may be slower than GPU-accelerated pipelines.
41
 */
42
class CpuPacketPipeline extends PacketPipeline {
43
    /** Default constructor for CPU-only processing */
44
    CpuPacketPipeline();
45
}
46
```
47

48
**Usage Examples:**
49

50
```java
51
// Create CPU pipeline for maximum compatibility
52
PacketPipeline pipeline = new CpuPacketPipeline();
53

54
// Open device with CPU pipeline
55
Freenect2Device device = freenect2.openDevice(serial, pipeline);
56

57
// CPU pipeline is automatically managed - no additional configuration needed
58
if (device != null) {
59
    System.out.println("Device opened with CPU pipeline");
60
    device.start();
61
}
62
```
63

64
### OpenGL Pipeline
65

66
OpenGL-accelerated processing pipeline for improved performance on systems with OpenGL support.
67

68
```java { .api }
69
/**
70
 * Pipeline with OpenGL depth processing.
71
 * Requires OpenGL support and may provide better performance than CPU pipeline.
72
 */
73
class OpenGLPacketPipeline extends PacketPipeline {
74
    /** Default constructor for OpenGL processing */
75
    OpenGLPacketPipeline();
76
    
77
    /** 
78
     * Constructor with OpenGL context and debug options
79
     * @param parent_opengl_context Existing OpenGL context to share (or null)
80
     * @param debug Enable debug output for troubleshooting
81
     */
82
    OpenGLPacketPipeline(Pointer parent_opengl_context, @Cast("bool") boolean debug);
83
}
84
```
85

86
**Usage Examples:**
87

88
```java
89
// Create OpenGL pipeline with default settings
90
PacketPipeline pipeline = new OpenGLPacketPipeline();
91

92
// Create OpenGL pipeline with debug enabled
93
PacketPipeline debugPipeline = new OpenGLPacketPipeline(null, true);
94

95
// Open device with OpenGL pipeline
96
try {
97
    Freenect2Device device = freenect2.openDevice(serial, pipeline);
98
    if (device != null) {
99
        System.out.println("Device opened with OpenGL pipeline");
100
        device.start();
101
    }
102
} catch (Exception e) {
103
    System.err.println("OpenGL pipeline failed, falling back to CPU");
104
    PacketPipeline cpuPipeline = new CpuPacketPipeline();
105
    Freenect2Device device = freenect2.openDevice(serial, cpuPipeline);
106
}
107
```
108

109
### Dump Pipeline
110

111
Debug pipeline that dumps raw packet data for analysis and troubleshooting.
112

113
```java { .api }
114
/**
115
 * Pipeline that dumps packets for debugging purposes.
116
 * Useful for analyzing raw sensor data and troubleshooting issues.
117
 * Provides access to internal depth processing tables.
118
 */
119
class DumpPacketPipeline extends PacketPipeline {
120
    /** Constructor for dump pipeline */
121
    DumpPacketPipeline();
122
    
123
    /** Get depth P0 tables required for depth decoding */
124
    @Cast("const unsigned char*") BytePointer getDepthP0Tables(@Cast("size_t*") SizeTPointer length);
125
    
126
    /** Get depth X lookup table */
127
    @Const FloatPointer getDepthXTable(@Cast("size_t*") SizeTPointer length);
128
    
129
    /** Get depth Z lookup table */
130
    @Const FloatPointer getDepthZTable(@Cast("size_t*") SizeTPointer length);
131
    
132
    /** Get depth value lookup table */
133
    @Const ShortPointer getDepthLookupTable(@Cast("size_t*") SizeTPointer length);
134
}
135
```
136

137
### Packet Processing Components
138

139
Individual components for processing different types of sensor data.
140

141
```java { .api }
142
/**
143
 * Callback interface for packet data processing
144
 */
145
abstract class DataCallback {
146
    // Implementation varies by packet type
147
}
148

149
/**
150
 * RGB packet processor for color frame data
151
 */
152
abstract class RgbPacketProcessor {
153
    // Processes color sensor packets into RGB frames
154
}
155

156
/**
157
 * Depth packet processor for depth/IR frame data  
158
 */
159
abstract class DepthPacketProcessor {
160
    // Processes ToF sensor packets into depth and IR frames
161
}
162

163
/**
164
 * Container for packet pipeline components
165
 */
166
class PacketPipelineComponents {
167
    // Aggregates processing components
168
}
169
```
170

171
## Pipeline Selection Guidelines
172

173
### Choosing the Right Pipeline
174

175
**CpuPacketPipeline**: 
176
- Best compatibility across all systems
177
- No GPU dependencies
178
- Slower processing but stable
179
- Recommended for: servers, headless systems, compatibility testing
180

181
**OpenGLPacketPipeline**:
182
- Hardware-accelerated processing
183
- Better performance on systems with OpenGL support
184
- May have driver dependencies
185
- Recommended for: desktop applications, real-time processing
186

187
**DumpPacketPipeline**:
188
- For debugging and analysis only
189
- Not suitable for production use
190
- Helpful for understanding raw sensor data
191

192
### Performance Considerations
193

194
```java
195
// Pipeline performance comparison example
196
long startTime, endTime;
197

198
// Test CPU pipeline
199
PacketPipeline cpuPipeline = new CpuPacketPipeline();
200
Freenect2Device device = freenect2.openDevice(serial, cpuPipeline);
201
device.start();
202

203
startTime = System.currentTimeMillis();
204
// Capture 10 frames for timing
205
for (int i = 0; i < 10; i++) {
206
    if (listener.waitForNewFrame(frames, 1000)) {
207
        listener.release(frames);
208
    }
209
}
210
endTime = System.currentTimeMillis();
211
long cpuTime = endTime - startTime;
212

213
device.stop();
214
device._close();
215

216
// Test OpenGL pipeline
217
PacketPipeline glPipeline = new OpenGLPacketPipeline();
218
device = freenect2.openDevice(serial, glPipeline);
219
device.start();
220

221
startTime = System.currentTimeMillis();
222
// Capture same 10 frames
223
for (int i = 0; i < 10; i++) {
224
    if (listener.waitForNewFrame(frames, 1000)) {
225
        listener.release(frames);
226
    }
227
}
228
endTime = System.currentTimeMillis();
229
long glTime = endTime - startTime;
230

231
System.out.println("CPU Pipeline: " + cpuTime + "ms");
232
System.out.println("OpenGL Pipeline: " + glTime + "ms");
233
System.out.println("Performance gain: " + (cpuTime / (float)glTime) + "x");
234
```
235

236
### Error Handling and Fallbacks
237

238
```java
239
/**
240
 * Robust pipeline selection with fallback support
241
 */
242
public static Freenect2Device openDeviceWithBestPipeline(
243
        Freenect2 freenect2, String serial) {
244
    
245
    // Try pipelines in order of preference
246
    PacketPipeline[] pipelines = {
247
        new OpenGLPacketPipeline(),  // Best performance
248
        new CpuPacketPipeline()      // Fallback compatibility
249
    };
250
    
251
    String[] pipelineNames = {"OpenGL", "CPU"};
252
    
253
    for (int i = 0; i < pipelines.length; i++) {
254
        try {
255
            Freenect2Device device = freenect2.openDevice(serial, pipelines[i]);
256
            if (device != null) {
257
                System.out.println("Successfully opened device with " + 
258
                                 pipelineNames[i] + " pipeline");
259
                return device;
260
            }
261
        } catch (Exception e) {
262
            System.err.println(pipelineNames[i] + " pipeline failed: " + 
263
                             e.getMessage());
264
            if (i == pipelines.length - 1) {
265
                System.err.println("All pipelines failed!");
266
                return null;
267
            }
268
        }
269
    }
270
    
271
    return null;
272
}
273
```
274

275
### Pipeline Lifecycle Management
276

277
```java
278
public class PipelineManager {
279
    private Freenect2 freenect2;
280
    private Freenect2Device device;
281
    private PacketPipeline currentPipeline;
282
    
283
    public boolean initializeWithPipeline(String pipelineType) {
284
        try {
285
            // Create pipeline based on type
286
            switch (pipelineType.toLowerCase()) {
287
                case "cpu":
288
                    currentPipeline = new CpuPacketPipeline();
289
                    break;
290
                case "opengl":
291
                    currentPipeline = new OpenGLPacketPipeline();
292
                    break;
293
                case "debug":
294
                    currentPipeline = new DumpPacketPipeline();
295
                    break;
296
                default:
297
                    throw new IllegalArgumentException("Unknown pipeline: " + pipelineType);
298
            }
299
            
300
            // Open device with selected pipeline
301
            String serial = freenect2.getDefaultDeviceSerialNumber().getString();
302
            device = freenect2.openDevice(serial, currentPipeline);
303
            
304
            if (device == null) {
305
                System.err.println("Failed to open device with " + pipelineType + " pipeline");
306
                return false;
307
            }
308
            
309
            System.out.println("Initialized with " + pipelineType + " pipeline");
310
            return true;
311
            
312
        } catch (Exception e) {
313
            System.err.println("Pipeline initialization failed: " + e.getMessage());
314
            return false;
315
        }
316
    }
317
    
318
    public void cleanup() {
319
        if (device != null) {
320
            device.stop();
321
            device._close();
322
            device = null;
323
        }
324
        currentPipeline = null;
325
    }
326
}