or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

device-management.mdframe-processing.mdindex.mdlogging.mdpipeline-configuration.mdregistration-geometry.md
tile.json

tessl/maven-org-bytedeco--libfreenect2

JavaCPP Presets for libfreenect2 providing Java bindings for Kinect for Windows v2 device drivers

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.bytedeco/libfreenect2@0.2.x

To install, run

npx @tessl/cli install tessl/maven-org-bytedeco--libfreenect2@0.2.0

index.mddocs/

libfreenect2

libfreenect2 is a JavaCPP Preset providing Java bindings for libfreenect2, enabling Java applications to interface with Microsoft Kinect for Windows v2 devices. It offers comprehensive access to Kinect v2 sensors including RGB images, depth information, and infrared data through a Java API that mirrors the native libfreenect2 functionality.

Package Information

  • Package Name: libfreenect2
  • Package Type: maven
  • Language: Java
  • Installation: 
    <dependency>
    <groupId>org.bytedeco</groupId>
    <artifactId>libfreenect2-platform</artifactId>
    <version>0.2.0-1.5.12</version>
</dependency>

Core Imports

import org.bytedeco.libfreenect2.*;
import static org.bytedeco.libfreenect2.global.freenect2.*;
import org.bytedeco.javacpp.*;
import org.bytedeco.javacpp.Loader;

Basic Usage

import org.bytedeco.libfreenect2.*;
import static org.bytedeco.libfreenect2.global.freenect2.*;

// Initialize library context
Loader.load(org.bytedeco.libfreenect2.global.freenect2.class);
Freenect2 freenect2 = new Freenect2();

// Find and open device
int deviceCount = freenect2.enumerateDevices();
if (deviceCount == 0) {
    System.out.println("No devices found");
    return;
}

String serial = freenect2.getDefaultDeviceSerialNumber().getString();
PacketPipeline pipeline = new CpuPacketPipeline();
Freenect2Device device = freenect2.openDevice(serial, pipeline);

// Configure frame listeners
int frameTypes = Frame.Color | Frame.Ir | Frame.Depth;
SyncMultiFrameListener listener = new SyncMultiFrameListener(frameTypes);
device.setColorFrameListener(listener);
device.setIrAndDepthFrameListener(listener);

// Start data capture
device.start();

// Capture frames
FrameMap frames = new FrameMap();
if (listener.waitForNewFrame(frames, 10000)) {
    Frame rgb = frames.get(Frame.Color);
    Frame depth = frames.get(Frame.Depth);
    Frame ir = frames.get(Frame.Ir);
    
    System.out.println("Color: " + rgb.width() + "x" + rgb.height());
    System.out.println("Depth: " + depth.width() + "x" + depth.height());
    
    listener.release(frames);
}

// Cleanup
device.stop();
device._close();

Architecture

libfreenect2 is organized around several key components:

  • Device Context: Freenect2 class manages device discovery and initialization
  • Device Control: Freenect2Device provides device configuration and stream control
  • Frame Processing: Frame, FrameListener, and FrameMap handle image data flow
  • Processing Pipelines: Multiple pipeline implementations (CPU, OpenGL, OpenCL, CUDA) for performance optimization
  • Registration System: Registration class handles depth-to-color mapping and point cloud generation
  • Logging System: Configurable logging with multiple severity levels

Capabilities

Device Management

Core device discovery, initialization, and control functionality for Kinect v2 devices.

class Freenect2 {
    Freenect2();
    int enumerateDevices();
    BytePointer getDefaultDeviceSerialNumber();
    Freenect2Device openDevice(String serial, PacketPipeline factory);
}

class Freenect2Device {
    boolean start();
    boolean startStreams(boolean rgb, boolean depth);
    boolean stop();
    boolean _close();
}

Device Management

Frame Processing

Comprehensive frame handling system for RGB, depth, and infrared data streams with synchronous and asynchronous processing options.

class Frame {
    static final int Color = 1;   // 1920x1080 BGRX/RGBX
    static final int Ir = 2;      // 512x424 float
    static final int Depth = 4;   // 512x424 float (mm)
    
    long width();
    long height();
    BytePointer data();
    int timestamp();
}

class SyncMultiFrameListener extends FrameListener {
    SyncMultiFrameListener(int frame_types);
    boolean waitForNewFrame(FrameMap frames, int milliseconds);
    void release(FrameMap frames);
}

Frame Processing

Pipeline Configuration

Multiple processing pipeline implementations optimized for different hardware configurations and performance requirements.

abstract class PacketPipeline {
    RgbPacketProcessor getRgbPacketProcessor();
    DepthPacketProcessor getDepthPacketProcessor();
}

class CpuPacketPipeline extends PacketPipeline {
    CpuPacketPipeline();
}

class OpenGLPacketPipeline extends PacketPipeline {
    OpenGLPacketPipeline();
    OpenGLPacketPipeline(Pointer parent_opengl_context, boolean debug);
}

Pipeline Configuration

Registration and Geometry

Advanced depth-to-color registration and 3D point cloud generation with camera calibration parameters.

class Registration {
    Registration(Freenect2Device.IrCameraParams depth_p, 
                Freenect2Device.ColorCameraParams rgb_p);
    void apply(Frame rgb, Frame depth, Frame undistorted, Frame registered);
    void getPointXYZ(Frame undistorted, int r, int c, 
                    FloatPointer x, FloatPointer y, FloatPointer z);
}

class Freenect2Device.IrCameraParams {
    float fx(); float fy(); float cx(); float cy();
    float k1(); float k2(); float k3(); float p1(); float p2();
}

Registration and Geometry

Logging

Configurable logging system with multiple severity levels and custom logger support.

class Logger {
    static final int None = 0, Error = 1, Warning = 2, Info = 3, Debug = 4;
    static int getDefaultLevel();
    void log(int level, String message);
}

Logging

Common Types

// Frame container for multiple frame types
class FrameMap {
    Frame get(@Cast("libfreenect2::Frame::Type") int type);
    FrameMap put(@Cast("libfreenect2::Frame::Type") int type, Frame value);
    long size();
    boolean empty();
}

// Device configuration parameters
class Freenect2Device.Config {
    float MinDepth();           // Minimum distance clip (meter)  
    float MaxDepth();           // Maximum distance clip (meter)
    @Cast("bool") boolean EnableBilateralFilter();    // Remove flying pixels
    @Cast("bool") boolean EnableEdgeAwareFilter();    // Remove noisy edges
}

// Global constants  
class org.bytedeco.libfreenect2.global.freenect2 {
    static final String LIBFREENECT2_VERSION = "0.2.0";
    static final int LIBFREENECT2_API_VERSION = ((0 << 16) | 2);
    static final String LIBFREENECT2_TEGRAJPEG_LIBRARY = "";
}