0
# libfreenect2
1

2
libfreenect2 is a JavaCPP Preset providing Java bindings for libfreenect2, enabling Java applications to interface with Microsoft Kinect for Windows v2 devices. It offers comprehensive access to Kinect v2 sensors including RGB images, depth information, and infrared data through a Java API that mirrors the native libfreenect2 functionality.
3

4
## Package Information
5

6
- **Package Name**: libfreenect2
7
- **Package Type**: maven
8
- **Language**: Java
9
- **Installation**: 
10
  ```xml
11
  <dependency>
12
      <groupId>org.bytedeco</groupId>
13
      <artifactId>libfreenect2-platform</artifactId>
14
      <version>0.2.0-1.5.12</version>
15
  </dependency>
16
  ```
17

18
## Core Imports
19

20
```java
21
import org.bytedeco.libfreenect2.*;
22
import static org.bytedeco.libfreenect2.global.freenect2.*;
23
import org.bytedeco.javacpp.*;
24
import org.bytedeco.javacpp.Loader;
25
```
26

27
## Basic Usage
28

29
```java
30
import org.bytedeco.libfreenect2.*;
31
import static org.bytedeco.libfreenect2.global.freenect2.*;
32

33
// Initialize library context
34
Loader.load(org.bytedeco.libfreenect2.global.freenect2.class);
35
Freenect2 freenect2 = new Freenect2();
36

37
// Find and open device
38
int deviceCount = freenect2.enumerateDevices();
39
if (deviceCount == 0) {
40
    System.out.println("No devices found");
41
    return;
42
}
43

44
String serial = freenect2.getDefaultDeviceSerialNumber().getString();
45
PacketPipeline pipeline = new CpuPacketPipeline();
46
Freenect2Device device = freenect2.openDevice(serial, pipeline);
47

48
// Configure frame listeners
49
int frameTypes = Frame.Color | Frame.Ir | Frame.Depth;
50
SyncMultiFrameListener listener = new SyncMultiFrameListener(frameTypes);
51
device.setColorFrameListener(listener);
52
device.setIrAndDepthFrameListener(listener);
53

54
// Start data capture
55
device.start();
56

57
// Capture frames
58
FrameMap frames = new FrameMap();
59
if (listener.waitForNewFrame(frames, 10000)) {
60
    Frame rgb = frames.get(Frame.Color);
61
    Frame depth = frames.get(Frame.Depth);
62
    Frame ir = frames.get(Frame.Ir);
63
    
64
    System.out.println("Color: " + rgb.width() + "x" + rgb.height());
65
    System.out.println("Depth: " + depth.width() + "x" + depth.height());
66
    
67
    listener.release(frames);
68
}
69

70
// Cleanup
71
device.stop();
72
device._close();
73
```
74

75
## Architecture
76

77
libfreenect2 is organized around several key components:
78

79
- **Device Context**: `Freenect2` class manages device discovery and initialization
80
- **Device Control**: `Freenect2Device` provides device configuration and stream control
81
- **Frame Processing**: `Frame`, `FrameListener`, and `FrameMap` handle image data flow
82
- **Processing Pipelines**: Multiple pipeline implementations (CPU, OpenGL, OpenCL, CUDA) for performance optimization
83
- **Registration System**: `Registration` class handles depth-to-color mapping and point cloud generation
84
- **Logging System**: Configurable logging with multiple severity levels
85

86
## Capabilities
87

88
### Device Management
89

90
Core device discovery, initialization, and control functionality for Kinect v2 devices.
91

92
```java { .api }
93
class Freenect2 {
94
    Freenect2();
95
    int enumerateDevices();
96
    BytePointer getDefaultDeviceSerialNumber();
97
    Freenect2Device openDevice(String serial, PacketPipeline factory);
98
}
99

100
class Freenect2Device {
101
    boolean start();
102
    boolean startStreams(boolean rgb, boolean depth);
103
    boolean stop();
104
    boolean _close();
105
}
106
```
107

108
[Device Management](./device-management.md)
109

110
### Frame Processing
111

112
Comprehensive frame handling system for RGB, depth, and infrared data streams with synchronous and asynchronous processing options.
113

114
```java { .api }
115
class Frame {
116
    static final int Color = 1;   // 1920x1080 BGRX/RGBX
117
    static final int Ir = 2;      // 512x424 float
118
    static final int Depth = 4;   // 512x424 float (mm)
119
    
120
    long width();
121
    long height();
122
    BytePointer data();
123
    int timestamp();
124
}
125

126
class SyncMultiFrameListener extends FrameListener {
127
    SyncMultiFrameListener(int frame_types);
128
    boolean waitForNewFrame(FrameMap frames, int milliseconds);
129
    void release(FrameMap frames);
130
}
131
```
132

133
[Frame Processing](./frame-processing.md)
134

135
### Pipeline Configuration
136

137
Multiple processing pipeline implementations optimized for different hardware configurations and performance requirements.
138

139
```java { .api }
140
abstract class PacketPipeline {
141
    RgbPacketProcessor getRgbPacketProcessor();
142
    DepthPacketProcessor getDepthPacketProcessor();
143
}
144

145
class CpuPacketPipeline extends PacketPipeline {
146
    CpuPacketPipeline();
147
}
148

149
class OpenGLPacketPipeline extends PacketPipeline {
150
    OpenGLPacketPipeline();
151
    OpenGLPacketPipeline(Pointer parent_opengl_context, boolean debug);
152
}
153
```
154

155
[Pipeline Configuration](./pipeline-configuration.md)
156

157
### Registration and Geometry
158

159
Advanced depth-to-color registration and 3D point cloud generation with camera calibration parameters.
160

161
```java { .api }
162
class Registration {
163
    Registration(Freenect2Device.IrCameraParams depth_p, 
164
                Freenect2Device.ColorCameraParams rgb_p);
165
    void apply(Frame rgb, Frame depth, Frame undistorted, Frame registered);
166
    void getPointXYZ(Frame undistorted, int r, int c, 
167
                    FloatPointer x, FloatPointer y, FloatPointer z);
168
}
169

170
class Freenect2Device.IrCameraParams {
171
    float fx(); float fy(); float cx(); float cy();
172
    float k1(); float k2(); float k3(); float p1(); float p2();
173
}
174
```
175

176
[Registration and Geometry](./registration-geometry.md)
177

178
### Logging
179

180
Configurable logging system with multiple severity levels and custom logger support.
181

182
```java { .api }
183
class Logger {
184
    static final int None = 0, Error = 1, Warning = 2, Info = 3, Debug = 4;
185
    static int getDefaultLevel();
186
    void log(int level, String message);
187
}
188
```
189

190
[Logging](./logging.md)
191

192
## Common Types
193

194
```java { .api }
195
// Frame container for multiple frame types
196
class FrameMap {
197
    Frame get(@Cast("libfreenect2::Frame::Type") int type);
198
    FrameMap put(@Cast("libfreenect2::Frame::Type") int type, Frame value);
199
    long size();
200
    boolean empty();
201
}
202

203
// Device configuration parameters
204
class Freenect2Device.Config {
205
    float MinDepth();           // Minimum distance clip (meter)  
206
    float MaxDepth();           // Maximum distance clip (meter)
207
    @Cast("bool") boolean EnableBilateralFilter();    // Remove flying pixels
208
    @Cast("bool") boolean EnableEdgeAwareFilter();    // Remove noisy edges
209
}
210

211
// Global constants  
212
class org.bytedeco.libfreenect2.global.freenect2 {
213
    static final String LIBFREENECT2_VERSION = "0.2.0";
214
    static final int LIBFREENECT2_API_VERSION = ((0 << 16) | 2);
215
    static final String LIBFREENECT2_TEGRAJPEG_LIBRARY = "";
216
}
217
```