0
# MobileNet
1

2
MobileNet provides pretrained image classification models for TensorFlow.js that enable real-time image recognition in web browsers and Node.js applications. MobileNets are small, low-latency, low-power convolutional neural networks parameterized to meet resource constraints while maintaining competitive accuracy. The library offers a simple JavaScript API that can classify browser-based image elements (img, video, canvas) and returns predictions with confidence scores, supporting both MobileNetV1 and V2 architectures with configurable alpha parameters to trade accuracy for performance.
3

4
## Package Information
5

6
- **Package Name**: @tensorflow-models/mobilenet
7
- **Package Type**: npm
8
- **Language**: TypeScript
9
- **Installation**: `npm install @tensorflow-models/mobilenet`
10

11
**Peer Dependencies**: Requires `@tensorflow/tfjs-core` and `@tensorflow/tfjs-converter` (version ^4.9.0)
12

13
## Core Imports
14

15
```typescript
16
import * as mobilenet from '@tensorflow-models/mobilenet';
17
```
18

19
For ES6 modules:
20

21
```typescript
22
import { load, MobileNet, ModelConfig } from '@tensorflow-models/mobilenet';
23
```
24

25
For CommonJS:
26

27
```javascript
28
const mobilenet = require('@tensorflow-models/mobilenet');
29
```
30

31
## Basic Usage
32

33
```typescript
34
import * as mobilenet from '@tensorflow-models/mobilenet';
35

36
// Load the default model (MobileNetV1, alpha=1.0)
37
const model = await mobilenet.load();
38

39
// Get an image element from the DOM
40
const img = document.getElementById('myImage');
41

42
// Classify the image
43
const predictions = await model.classify(img);
44
console.log('Predictions:', predictions);
45
// Output: [{ className: "Egyptian cat", probability: 0.838 }, ...]
46

47
// Get embeddings for transfer learning
48
const embeddings = model.infer(img, true);
49
console.log('Embedding shape:', embeddings.shape); // [1, 1024]
50

51
// Get raw logits
52
const logits = model.infer(img, false);
53
console.log('Logits shape:', logits.shape); // [1, 1000]
54
```
55

56
## Architecture
57

58
MobileNet is built around several key components:
59

60
- **Model Loading**: Supports both predefined TensorFlow Hub models and custom model URLs
61
- **Image Processing**: Handles various input types (Tensor, DOM elements) with automatic preprocessing
62
- **Dual Architectures**: MobileNetV1 and V2 with different accuracy/performance tradeoffs
63
- **Alpha Scaling**: Width multipliers (0.25, 0.50, 0.75, 1.0) to control model size and performance
64
- **Multi-Purpose Output**: Supports both classification predictions and feature embeddings
65

66
## Capabilities
67

68
### Model Loading
69

70
Load a MobileNet model with specified configuration options.
71

72
```typescript { .api }
73
/**
74
 * Loads a MobileNet model with specified configuration
75
 * @param modelConfig - Configuration for model loading (defaults to version: 1, alpha: 1.0)
76
 * @returns Promise resolving to MobileNet instance
77
 */
78
function load(modelConfig?: ModelConfig): Promise<MobileNet>;
79

80
interface ModelConfig {
81
  /** The MobileNet version number (1 or 2). Defaults to 1 */
82
  version: MobileNetVersion;
83
  /** Width multiplier trading accuracy for performance. Defaults to 1.0 */
84
  alpha?: MobileNetAlpha;
85
  /** Custom model url or tf.io.IOHandler object */
86
  modelUrl?: string | tf.io.IOHandler;
87
  /** Input range expected by custom models, typically [0, 1] or [-1, 1] */
88
  inputRange?: [number, number];
89
}
90

91
type MobileNetVersion = 1 | 2;
92
type MobileNetAlpha = 0.25 | 0.50 | 0.75 | 1.0; // Note: 0.25 only available for version 1
93
```
94

95
**Usage Examples:**
96

97
```typescript
98
// Load default model (V1, alpha=1.0)
99
const model1 = await mobilenet.load();
100

101
// Load MobileNetV1 with smallest alpha for maximum speed
102
const fastModel = await mobilenet.load({
103
  version: 1,
104
  alpha: 0.25  // Only available for V1
105
});
106

107
// Load MobileNetV2 with alpha=0.5 for faster performance
108
const model2 = await mobilenet.load({
109
  version: 2,
110
  alpha: 0.5  // V2 supports 0.50, 0.75, 1.0 (no 0.25)
111
});
112

113
// Load custom model from URL
114
const model3 = await mobilenet.load({
115
  version: 1,
116
  modelUrl: 'https://my-custom-model-url',
117
  inputRange: [-1, 1]
118
});
119
```
120

121
### Image Classification
122

123
Classify images and return top predicted classes with probabilities.
124

125
```typescript { .api }
126
/**
127
 * Classifies an image returning top predicted classes with probabilities
128
 * @param img - Image to classify (Tensor, ImageData, or DOM element)
129
 * @param topk - Number of top predictions to return. Defaults to 3
130
 * @returns Promise resolving to array of predictions with class names and probabilities
131
 */
132
classify(
133
  img: tf.Tensor3D | ImageData | HTMLImageElement | HTMLCanvasElement | HTMLVideoElement,
134
  topk?: number
135
): Promise<Array<{className: string, probability: number}>>;
136
```
137

138
**Usage Examples:**
139

140
```typescript
141
// Classify image element
142
const img = document.getElementById('myImage');
143
const predictions = await model.classify(img);
144
console.log(predictions);
145
// [{ className: "Egyptian cat", probability: 0.838 }, 
146
//  { className: "tabby cat", probability: 0.046 }, ...]
147

148
// Get top 5 predictions
149
const top5 = await model.classify(img, 5);
150

151
// Classify tensor directly
152
const tensor = tf.zeros([224, 224, 3]);
153
const tensorPredictions = await model.classify(tensor);
154
```
155

156
### Feature Extraction
157

158
Extract feature embeddings or raw logits for transfer learning and custom applications.
159

160
```typescript { .api }
161
/**
162
 * Computes logits or embeddings for the provided image
163
 * @param img - Image to process (Tensor, ImageData, or DOM element)
164
 * @param embedding - If true, returns embedding features. If false, returns 1000-dim logits
165
 * @returns Tensor containing logits or embeddings
166
 */
167
infer(
168
  img: tf.Tensor | ImageData | HTMLImageElement | HTMLCanvasElement | HTMLVideoElement,
169
  embedding?: boolean
170
): tf.Tensor;
171
```
172

173
**Usage Examples:**
174

175
```typescript
176
// Get feature embeddings for transfer learning
177
const embeddings = model.infer(img, true);
178
console.log('Embedding shape:', embeddings.shape); // [1, 1024] for V1, varies by version/alpha
179

180
// Get raw logits for custom processing
181
const logits = model.infer(img, false); // default is false
182
console.log('Logits shape:', logits.shape); // [1, 1000]
183

184
// Process multiple images in batch (if input is batched tensor)
185
const batchTensor = tf.zeros([3, 224, 224, 3]);
186
const batchLogits = model.infer(batchTensor);
187
console.log('Batch logits shape:', batchLogits.shape); // [3, 1000]
188
```
189

190
### Model Interface
191

192
The loaded MobileNet model instance provides these methods:
193

194
```typescript { .api }
195
interface MobileNet {
196
  /** Initialize the model (called automatically by load()) */
197
  load(): Promise<void>;
198
  
199
  /** Extract logits or embeddings from an image */
200
  infer(
201
    img: tf.Tensor | ImageData | HTMLImageElement | HTMLCanvasElement | HTMLVideoElement,
202
    embedding?: boolean
203
  ): tf.Tensor;
204
  
205
  /** Classify an image and return top predictions */
206
  classify(
207
    img: tf.Tensor3D | ImageData | HTMLImageElement | HTMLCanvasElement | HTMLVideoElement,
208
    topk?: number
209
  ): Promise<Array<{className: string, probability: number}>>;
210
}
211
```
212

213
## Types
214

215
```typescript { .api }
216
/** MobileNet version numbers */
217
type MobileNetVersion = 1 | 2;
218

219
/** Alpha multipliers controlling model width (0.25 only available for version 1) */
220
type MobileNetAlpha = 0.25 | 0.50 | 0.75 | 1.0;
221

222
/** Configuration for model loading */
223
interface ModelConfig {
224
  /** The MobileNet version number (1 or 2). Defaults to 1 */
225
  version: MobileNetVersion;
226
  /** Width multiplier controlling model width. Defaults to 1.0 */
227
  alpha?: MobileNetAlpha;
228
  /** Custom model url or tf.io.IOHandler object */
229
  modelUrl?: string | tf.io.IOHandler;
230
  /** Input range expected by custom models, typically [0, 1] or [-1, 1] */
231
  inputRange?: [number, number];
232
}
233

234
/** Model information for predefined variants */
235
interface MobileNetInfo {
236
  url: string;
237
  inputRange: [number, number];
238
}
239

240
/** Classification result */
241
interface ClassificationResult {
242
  className: string;
243
  probability: number;
244
}
245
```
246

247
## Version Information
248

249
```typescript { .api }
250
/** Package version string */
251
export const version: string; // "2.1.1"
252
```
253

254
## Error Handling
255

256
The library throws errors in the following scenarios:
257

258
- **Missing TensorFlow.js**: Throws Error if `@tensorflow/tfjs-core` is not available
259
- **Invalid Version**: Throws Error for unsupported version numbers (only 1 and 2 are supported)
260
- **Invalid Alpha**: Throws Error for unsupported alpha values for the specified version
261
- **Model Loading**: Network errors when loading models from TensorFlow Hub or custom URLs
262

263
```typescript
264
try {
265
  const model = await mobilenet.load({ version: 3 }); // Invalid version
266
} catch (error) {
267
  console.error('Invalid version:', error.message);
268
}
269
```
270

271
## Supported Input Types
272

273
All image processing methods accept these input types:
274

275
- **`tf.Tensor`**: 3D tensor [height, width, channels] or 4D tensor [batch, height, width, channels]
276
- **`ImageData`**: Canvas ImageData object
277
- **`HTMLImageElement`**: HTML `<img>` elements
278
- **`HTMLCanvasElement`**: HTML `<canvas>` elements  
279
- **`HTMLVideoElement`**: HTML `<video>` elements
280

281
Images are automatically preprocessed to 224x224 pixels and normalized according to the model's expected input range.
282

283
## Performance Considerations
284

285
- **Version**: MobileNetV1 is faster, V2 is more accurate
286
- **Alpha**: Lower alpha values (0.25, 0.5) are faster but less accurate
287
- **Input Size**: All inputs are resized to 224x224, larger inputs require more preprocessing
288
- **Memory**: Call `tensor.dispose()` on returned tensors from `infer()` to prevent memory leaks