0
# Model Zoo
1

2
Collection of pre-trained ONNX models for specific face analysis tasks including detection, recognition, landmark detection, and attribute prediction. Models can be used individually or automatically managed through the FaceAnalysis pipeline.
3

4
## Capabilities
5

6
### Model Factory
7

8
Central function for loading and instantiating models from the model zoo.
9

10
```python { .api }
11
def get_model(name, **kwargs):
12
    """
13
    Load a model from the model zoo.
14
    
15
    Parameters:
16
    - name: str, model name or path to ONNX file
17
    - root: str, model storage directory (default: '~/.insightface')
18
    - download: bool, auto-download if model not found (default: True)
19
    - download_zip: bool, download as zip archive (default: False)
20
    - providers: list, ONNX execution providers (default: ['CPUExecutionProvider'])
21
    - provider_options: list, provider-specific options for execution providers
22
    - session: onnxruntime.InferenceSession, pre-existing ONNX session (optional)
23
    
24
    Returns:
25
    Model instance (ArcFaceONNX, RetinaFace, SCRFD, Landmark, Attribute, or INSwapper)
26
    based on automatic detection from model input shape and filename patterns
27
    """
28
```
29

30
### Face Recognition Models
31

32
Models for generating face embeddings and computing face similarity.
33

34
```python { .api }
35
class ArcFaceONNX:
36
    def __init__(self, model_file=None, session=None):
37
        """Initialize ArcFace recognition model."""
38
    
39
    def prepare(self, ctx_id, **kwargs):
40
        """
41
        Prepare model for inference.
42
        
43
        Parameters:
44
        - ctx_id: int, device context ID
45
        """
46
    
47
    def get(self, img, face) -> np.ndarray:
48
        """
49
        Extract face embedding from cropped face region.
50
        
51
        Parameters:
52
        - img: np.ndarray, input image
53
        - face: Face object with bbox and kps
54
        
55
        Returns:
56
        np.ndarray: face embedding vector (typically 512-dimensional)
57
        """
58
    
59
    def compute_sim(self, feat1, feat2) -> float:
60
        """
61
        Compute similarity between two face embeddings.
62
        
63
        Parameters:
64
        - feat1, feat2: np.ndarray, face embedding vectors
65
        
66
        Returns:
67
        float: cosine similarity score (-1 to 1)
68
        """
69
    
70
    def get_feat(self, imgs) -> np.ndarray:
71
        """
72
        Extract features from multiple face images.
73
        
74
        Parameters:
75
        - imgs: np.ndarray, batch of aligned face images
76
        
77
        Returns:
78
        np.ndarray: batch of face embeddings
79
        """
80
    
81
    def forward(self, batch_data) -> np.ndarray:
82
        """Forward pass through the model."""
83
    
84
    # Attributes
85
    taskname = 'recognition'
86
    input_size: tuple        # Model input size (height, width)
87
    input_shape: tuple       # Model input shape (batch, channels, height, width)
88
    input_mean: float        # Input normalization mean
89
    input_std: float         # Input normalization standard deviation
90
```
91

92
### Face Detection Models
93

94
Models for detecting faces and facial landmarks in images.
95

96
```python { .api }
97
class RetinaFace:
98
    def __init__(self, model_file=None, session=None):
99
        """Initialize RetinaFace detection model."""
100
    
101
    def prepare(self, ctx_id, **kwargs):
102
        """
103
        Prepare model for inference.
104
        
105
        Parameters:
106
        - ctx_id: int, device context ID
107
        - input_size: tuple, detection input size
108
        - det_thresh: float, detection threshold
109
        - nms_thresh: float, NMS threshold
110
        """
111
    
112
    def detect(self, img, input_size=None, max_num=0, metric='default') -> Tuple[np.ndarray, np.ndarray]:
113
        """
114
        Detect faces in image.
115
        
116
        Parameters:
117
        - img: np.ndarray, input image
118
        - input_size: tuple, resize image to this size for detection
119
        - max_num: int, maximum number of faces to return
120
        - metric: str, distance metric for NMS
121
        
122
        Returns:
123
        tuple: (bboxes, keypoints)
124
        - bboxes: np.ndarray, shape (N, 5) [x1, y1, x2, y2, score]
125
        - keypoints: np.ndarray, shape (N, 5, 2) 5-point facial landmarks
126
        """
127
    
128
    def forward(self, img, threshold):
129
        """Forward pass through detection model."""
130
    
131
    def nms(self, dets):
132
        """Apply non-maximum suppression to detections."""
133
    
134
    # Attributes
135
    taskname = 'detection'
136
    det_thresh: float        # Detection confidence threshold
137
    nms_thresh: float        # Non-maximum suppression threshold
138
    input_size: tuple        # Model input size
139
    use_kps: bool           # Whether model outputs keypoints
140
```
141

142
```python { .api }
143
class SCRFD:
144
    def __init__(self, model_file=None, session=None):
145
        """Initialize SCRFD (Sample and Computation Redistributed Face Detection) model."""
146
    
147
    def prepare(self, ctx_id, **kwargs):
148
        """Prepare model for inference."""
149
    
150
    def detect(self, img, input_size=None, max_num=0, metric='default') -> Tuple[np.ndarray, np.ndarray]:
151
        """
152
        Detect faces using SCRFD algorithm.
153
        
154
        Parameters and returns same as RetinaFace.detect()
155
        """
156
    
157
    def forward(self, img, threshold):
158
        """Forward pass through SCRFD model."""
159
    
160
    def nms(self, dets):
161
        """Apply non-maximum suppression."""
162
    
163
    # Attributes  
164
    taskname = 'detection'
165
    batched: bool           # Whether model supports batch processing
166
    det_thresh: float       # Detection threshold
167
    nms_thresh: float       # NMS threshold
168

169
def get_scrfd(name, download=False, root='~/.insightface/models', **kwargs):
170
    """
171
    Load SCRFD model with automatic download support.
172
    
173
    Parameters:
174
    - name: str, model name or path to ONNX file
175
    - download: bool, whether to auto-download if model not found
176
    - root: str, model storage directory
177
    - **kwargs: additional arguments passed to SCRFD constructor
178
    
179
    Returns:
180
    SCRFD: initialized SCRFD model instance
181
    """
182

183
def scrfd_2p5gkps(**kwargs):
184
    """Load SCRFD 2.5G model with keypoints support."""
185

186
def softmax(z) -> np.ndarray:
187
    """
188
    Apply softmax function to 2D array.
189
    
190
    Parameters:
191
    - z: np.ndarray, input array of shape (N, C)
192
    
193
    Returns:
194
    np.ndarray: softmax probabilities
195
    """
196

197
def distance2bbox(points, distance, max_shape=None) -> np.ndarray:
198
    """
199
    Decode distance prediction to bounding box coordinates.
200
    
201
    Parameters:
202
    - points: np.ndarray, center points, shape (n, 2) [x, y]
203
    - distance: np.ndarray, distances to 4 boundaries [left, top, right, bottom]
204
    - max_shape: tuple, optional image shape for clamping
205
    
206
    Returns:
207
    np.ndarray: decoded bounding boxes [x1, y1, x2, y2]
208
    """
209

210
def distance2kps(points, distance, max_shape=None) -> np.ndarray:
211
    """
212
    Decode distance prediction to keypoint coordinates.
213
    
214
    Parameters:
215
    - points: np.ndarray, reference points, shape (n, 2)
216
    - distance: np.ndarray, distance predictions to keypoints
217
    - max_shape: tuple, optional image shape for clamping
218
    
219
    Returns:
220
    np.ndarray: decoded keypoint coordinates
221
    """
222
```
223

224
### Landmark Detection Models
225

226
Models for detecting detailed facial landmarks.
227

228
```python { .api }
229
class Landmark:
230
    def __init__(self, model_file=None, session=None):
231
        """Initialize landmark detection model."""
232
    
233
    def prepare(self, ctx_id, **kwargs):
234
        """Prepare model for inference."""
235
    
236
    def get(self, img, face) -> np.ndarray:
237
        """
238
        Detect facial landmarks.
239
        
240
        Parameters:
241
        - img: np.ndarray, input image
242
        - face: Face object with bbox
243
        
244
        Returns:
245
        np.ndarray: landmark coordinates, shape depends on model
246
        - 2D landmarks: (N, 2) 
247
        - 3D landmarks: (N, 3)
248
        """
249
    
250
    # Attributes
251
    taskname: str           # e.g., 'landmark_2d_68', 'landmark_3d_68', 'landmark_2d_106'
252
    lmk_dim: int           # Landmark dimension: 2 or 3
253
    lmk_num: int           # Number of landmarks: 68, 106, etc.
254
    require_pose: bool     # Whether model requires pose estimation
255
```
256

257
### Attribute Prediction Models
258

259
Models for predicting face attributes like age and gender.
260

261
```python { .api }
262
class Attribute:
263
    def __init__(self, model_file=None, session=None):
264
        """Initialize attribute prediction model."""
265
    
266
    def prepare(self, ctx_id, **kwargs):
267
        """Prepare model for inference."""
268
    
269
    def get(self, img, face) -> Union[Tuple[int, int], np.ndarray]:
270
        """
271
        Predict face attributes.
272
        
273
        Parameters:
274
        - img: np.ndarray, input image
275
        - face: Face object with bbox
276
        
277
        Returns:
278
        For genderage models: tuple (gender, age) where gender is 0/1, age is int
279
        For other attribute models: np.ndarray of predictions
280
        """
281
    
282
    # Attributes
283
    taskname: str          # e.g., 'genderage', 'attribute_N'
284
```
285

286
### Face Swapping Models
287

288
Models for face swapping and identity transfer.
289

290
```python { .api }
291
class INSwapper:
292
    def __init__(self, model_file=None, session=None):
293
        """Initialize INSwapper face swapping model."""
294
    
295
    def forward(self, img, latent) -> np.ndarray:
296
        """
297
        Forward pass with latent representation.
298
        
299
        Parameters:
300
        - img: np.ndarray, target face image
301
        - latent: np.ndarray, source face latent representation
302
        
303
        Returns:
304
        np.ndarray: swapped face image
305
        """
306
    
307
    def get(self, img, target_face, source_face, paste_back=True) -> np.ndarray:
308
        """
309
        Perform face swapping.
310
        
311
        Parameters:
312
        - img: np.ndarray, input image containing target face
313
        - target_face: Face object, face to be replaced
314
        - source_face: Face object, source face for swapping
315
        - paste_back: bool, whether to paste result back to original image
316
        
317
        Returns:
318
        np.ndarray: result image with swapped face
319
        """
320
```
321

322
## Usage Examples
323

324
### Individual Model Usage
325

326
```python
327
from insightface.model_zoo import get_model
328
import cv2
329

330
# Load specific detection model
331
detector = get_model('retinaface_r50_v1')
332
detector.prepare(ctx_id=0, det_thresh=0.5)
333

334
# Detect faces
335
img = cv2.imread('group_photo.jpg')
336
bboxes, kpss = detector.detect(img)
337

338
print(f"Detected {len(bboxes)} faces")
339
for i, (bbox, kps) in enumerate(zip(bboxes, kpss)):
340
    x1, y1, x2, y2, score = bbox
341
    print(f"Face {i+1}: confidence={score:.3f}, box=({x1}, {y1}, {x2}, {y2})")
342
```
343

344
### Recognition Model Usage
345

346
```python
347
# Load recognition model
348
recognizer = get_model('arcface_r100_v1')
349
recognizer.prepare(ctx_id=0)
350

351
# Extract embeddings from detected faces
352
embeddings = []
353
for bbox, kps in zip(bboxes, kpss):
354
    # Create Face object
355
    face = Face(bbox=bbox, kps=kps, det_score=bbox[4])
356
    
357
    # Extract embedding
358
    embedding = recognizer.get(img, face)
359
    embeddings.append(embedding)
360

361
# Compare faces
362
if len(embeddings) >= 2:
363
    similarity = recognizer.compute_sim(embeddings[0], embeddings[1])
364
    print(f"Similarity between face 1 and 2: {similarity:.3f}")
365
```
366

367
### Model Download Management
368

369
```python
370
# Download specific model pack
371
model_path = get_model('buffalo_s', download=True, root='./models')
372

373
# Use with custom providers (GPU acceleration)
374
detector = get_model('scrfd_10g_bnkps', 
375
                    providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
376
detector.prepare(ctx_id=0)
377
```
378

379
### Attribute Prediction
380

381
```python
382
# Load gender/age prediction model
383
attr_model = get_model('genderage_v1')
384
attr_model.prepare(ctx_id=0)
385

386
# Predict attributes for detected faces
387
for bbox, kps in zip(bboxes, kpss):
388
    face = Face(bbox=bbox, kps=kps)
389
    gender, age = attr_model.get(img, face)
390
    
391
    gender_str = 'M' if gender == 1 else 'F'
392
    print(f"Face: {gender_str}, Age: {age}")
393
```
394

395
### Landmark Detection
396

397
```python
398
# Load 68-point landmark model
399
landmark_model = get_model('2d106det')
400
landmark_model.prepare(ctx_id=0)
401

402
# Detect detailed landmarks
403
for bbox, kps in zip(bboxes, kpss):
404
    face = Face(bbox=bbox, kps=kps)
405
    landmarks = landmark_model.get(img, face)
406
    
407
    print(f"Detected {len(landmarks)} landmarks")
408
    # landmarks shape: (106, 2) for 2D coordinates
409
```
410

411
### Available Model Packs
412

413
Common pre-trained model packs available for download:
414

415
- **buffalo_l**: Large model pack with detection, recognition, landmarks, and attributes
416
- **buffalo_m**: Medium-sized model pack for balanced performance
417
- **buffalo_s**: Small model pack for fast inference
418
- **antelopev2**: High-accuracy model pack with advanced features
419

420
Individual models:
421
- **retinaface_r50_v1**: RetinaFace detection with ResNet50 backbone
422
- **scrfd_10g_bnkps**: SCRFD detection model
423
- **arcface_r100_v1**: ArcFace recognition with ResNet100
424
- **genderage_v1**: Gender and age prediction model
425
- **2d106det**: 106-point landmark detection