tessl/npm-tensorflow-models--posenet

Pretrained PoseNet model in TensorFlow.js for real-time human pose estimation from images and video streams

—

Pending

Overview

Eval results

Files

Model Loading and Configuration

Name: tessl/npm-tensorflow-models--posenet
Author: tessl

PoseNet model loading and configuration system for selecting neural network architectures and performance parameters.

Capabilities

Load Function

Loads a PoseNet model with configurable architecture and performance settings.

/**
 * Load PoseNet model instance from checkpoint with configurable architecture
 * @param config - Model configuration options, defaults to MobileNetV1 setup
 * @returns Promise resolving to configured PoseNet instance
 */
function load(config?: ModelConfig): Promise<PoseNet>;

Usage Examples:

import * as posenet from '@tensorflow-models/posenet';

// Default MobileNetV1 model (fastest)
const net = await posenet.load();

// MobileNetV1 with custom configuration
const mobileNet = await posenet.load({
  architecture: 'MobileNetV1',
  outputStride: 16,
  inputResolution: { width: 640, height: 480 },
  multiplier: 0.75
});

// ResNet50 model (most accurate)
const resNet = await posenet.load({
  architecture: 'ResNet50',
  outputStride: 32,
  inputResolution: { width: 257, height: 200 },
  quantBytes: 2
});

// Custom model URL
const customNet = await posenet.load({
  architecture: 'MobileNetV1',
  outputStride: 16,
  inputResolution: 257,
  modelUrl: 'https://example.com/custom-posenet-model.json'
});

Model Configuration

Configuration interface for customizing model architecture and performance characteristics.

/**
 * Configuration options for PoseNet model loading
 */
interface ModelConfig {
  /** Neural network architecture: MobileNetV1 (fast) or ResNet50 (accurate) */
  architecture: PoseNetArchitecture;
  /** Output stride controlling resolution vs speed trade-off */
  outputStride: PoseNetOutputStride;
  /** Input image resolution for processing */
  inputResolution: InputResolution;
  /** MobileNetV1 depth multiplier (MobileNetV1 only) */
  multiplier?: MobileNetMultiplier;
  /** Custom model URL for local development or restricted access */
  modelUrl?: string;
  /** Weight quantization bytes for model size vs accuracy */
  quantBytes?: PoseNetQuantBytes;
}

type PoseNetArchitecture = 'ResNet50' | 'MobileNetV1';
type PoseNetOutputStride = 32 | 16 | 8;
type MobileNetMultiplier = 0.50 | 0.75 | 1.0;
type PoseNetQuantBytes = 1 | 2 | 4;
type InputResolution = number | {width: number, height: number};

Architecture Selection

Choose between two pre-trained neural network architectures:

MobileNetV1:

Faster inference, smaller model size
Output strides: 8, 16, 32
Multipliers: 0.50, 0.75, 1.0
Recommended for mobile and mid-range devices

ResNet50:

Higher accuracy, larger model size
Output strides: 16, 32
Fixed multiplier: 1.0
Recommended for high-end devices

Performance Parameters

Output Stride:

Controls resolution vs speed trade-off
Smaller values (8) = higher accuracy, slower inference
Larger values (32) = lower accuracy, faster inference

Input Resolution:

Image size fed to the model
Higher resolution = more accurate, slower processing
Can be number (square) or {width, height} object

Multiplier (MobileNetV1 only):

Controls model depth (number of channels)
1.0 = full depth, highest accuracy
0.50 = half depth, fastest inference
Required to be 1.0 when outputStride is 32

Quantization Bytes:

4 bytes = no quantization, highest accuracy (~90MB)
2 bytes = 2x smaller model, slightly lower accuracy (~45MB)
1 byte = 4x smaller model, lower accuracy (~22MB)

Default Configuration

The default configuration when no parameters are provided:

const MOBILENET_V1_CONFIG: ModelConfig = {
  architecture: 'MobileNetV1',
  outputStride: 16,
  multiplier: 0.75,
  inputResolution: 257,
};

PoseNet Class

The main class returned by the load function, providing pose estimation methods.

/**
 * Main PoseNet class for pose estimation
 */
class PoseNet {
  /** Underlying neural network model */
  readonly baseModel: BaseModel;
  /** Model input resolution as [height, width] */
  readonly inputResolution: [number, number];

  /** Estimate single person pose from input image */
  estimateSinglePose(input: PosenetInput, config?: SinglePersonInterfaceConfig): Promise<Pose>;
  
  /** Estimate multiple person poses from input image */
  estimateMultiplePoses(input: PosenetInput, config?: MultiPersonInferenceConfig): Promise<Pose[]>;
  
  /** Release GPU/CPU memory allocated by the model */
  dispose(): void;
}

type PosenetInput = ImageData | HTMLImageElement | HTMLCanvasElement | HTMLVideoElement | tf.Tensor3D;

Install with Tessl CLI

npx tessl i tessl/npm-tensorflow-models--posenet

docs