tessl/npm-tensorflow-models--body-pix

Real-time person and body part segmentation using TensorFlow.js for web browsers with machine learning models.

—

Pending

Overview

Eval results

Files

Model Loading and Configuration

Name: tessl/npm-tensorflow-models--body-pix
Author: tessl

BodyPix model loading system with configurable architectures and performance characteristics. Choose between MobileNet for speed or ResNet for accuracy, with various configuration options for different use cases.

Capabilities

Load Function

Loads and initializes a BodyPix model with optional configuration.

/**
 * Loads a BodyPix model instance
 * @param config - Optional model configuration
 * @returns Promise resolving to loaded BodyPix model instance
 */
function load(config?: ModelConfig): Promise<BodyPix>;

interface ModelConfig {
  /** Neural network architecture - MobileNetV1 for speed, ResNet50 for accuracy */
  architecture: 'MobileNetV1' | 'ResNet50';
  /** Output stride - smaller values = higher accuracy, larger values = faster inference */
  outputStride: 8 | 16 | 32;
  /** Depth multiplier for MobileNet only - affects model size and accuracy */
  multiplier?: 0.50 | 0.75 | 1.0;
  /** Custom model URL for offline or alternative model hosting */
  modelUrl?: string;
  /** Weight quantization - higher values = larger model size, better accuracy */
  quantBytes?: 1 | 2 | 4;
}

Usage Examples:

import * as bodyPix from '@tensorflow-models/body-pix';

// Load with default configuration (MobileNetV1, balanced settings)
const net = await bodyPix.load();

// Load ResNet model for higher accuracy
const highAccuracyNet = await bodyPix.load({
  architecture: 'ResNet50',
  outputStride: 16,
  quantBytes: 4
});

// Load MobileNet model optimized for speed
const fastNet = await bodyPix.load({
  architecture: 'MobileNetV1',
  outputStride: 32,
  multiplier: 0.50,
  quantBytes: 2
});

// Load from custom URL
const customNet = await bodyPix.load({
  architecture: 'MobileNetV1',
  outputStride: 16,
  modelUrl: 'https://custom-server.com/bodypix-model/'
});

Model Architecture Options

MobileNetV1

Best for: Real-time applications, mobile devices, speed-critical use cases
Output Strides: 8, 16, 32
Multipliers: 0.50, 0.75, 1.0
Trade-offs: Faster inference, smaller model size, lower accuracy

ResNet50

Best for: High-accuracy applications, batch processing, desktop applications
Output Strides: 16, 32
Multipliers: 1.0 only
Trade-offs: Higher accuracy, larger model size, slower inference

Configuration Guidelines

For Real-time Video Applications:

const config = {
  architecture: 'MobileNetV1',
  outputStride: 16,
  multiplier: 0.75,
  quantBytes: 2
};

For High-Quality Image Processing:

const config = {
  architecture: 'ResNet50',
  outputStride: 16,
  quantBytes: 4
};

For Maximum Speed (30+ FPS):

const config = {
  architecture: 'MobileNetV1',
  outputStride: 32,
  multiplier: 0.50,
  quantBytes: 1
};

Low-Level Activation Methods

For advanced use cases requiring access to intermediate neural network outputs.

class BodyPix {
  /**
   * Returns low-level activation tensors for person segmentation
   * @param input - Image input
   * @param internalResolution - Internal processing resolution
   * @param segmentationThreshold - Segmentation confidence threshold
   * @returns Object containing segmentation tensor and intermediate outputs
   */
  segmentPersonActivation(
    input: BodyPixInput,
    internalResolution: BodyPixInternalResolution,
    segmentationThreshold?: number
  ): Promise<{
    segmentation: tf.Tensor2D;
    heatmapScores: tf.Tensor3D;
    offsets: tf.Tensor3D;
    displacementFwd: tf.Tensor3D;
    displacementBwd: tf.Tensor3D;
  }>;

  /**
   * Returns low-level activation tensors for body part segmentation  
   * @param input - Image input
   * @param internalResolution - Internal processing resolution
   * @param segmentationThreshold - Segmentation confidence threshold
   * @returns Object containing part segmentation tensor and intermediate outputs
   */
  segmentPersonPartsActivation(
    input: BodyPixInput,
    internalResolution: BodyPixInternalResolution,
    segmentationThreshold?: number
  ): Promise<{
    partSegmentation: tf.Tensor2D;
    heatmapScores: tf.Tensor3D;
    offsets: tf.Tensor3D;
    displacementFwd: tf.Tensor3D;
    displacementBwd: tf.Tensor3D;
  }>;
}

Model Disposal

class BodyPix {
  /**
   * Disposes the model and frees memory resources
   */
  dispose(): void;
}

Usage:

// Clean up model when done
net.dispose();

Performance Considerations

Output Stride: Lower values (8) provide higher accuracy but require more computation
Architecture: MobileNetV1 is 2-4x faster than ResNet50 but less accurate
Multiplier: Only affects MobileNetV1; lower values reduce model size and increase speed
Quantization: Lower quantBytes reduce model size and loading time but may affect accuracy
Memory: Call dispose() to free GPU memory when model is no longer needed

Install with Tessl CLI

npx tessl i tessl/npm-tensorflow-models--body-pix

docs

body-part-segmentation.md

index.md

model-loading.md

person-segmentation.md

rendering-effects.md

utilities.md

tile.json