0
# InsightFace
1

2
A comprehensive 2D and 3D face analysis toolkit that provides state-of-the-art algorithms for face recognition, face detection, and face alignment. Built primarily on ONNX Runtime for efficient inference, InsightFace offers pre-trained models, easy-to-use Python APIs, and capabilities for face swapping, pose estimation, and attribute analysis.
3

4
## Package Information
5

6
- **Package Name**: insightface
7
- **Language**: Python
8
- **Installation**: `pip install insightface`
9
- **Dependencies**: `onnxruntime`, `opencv-python`, `numpy`, `scikit-image`
10

11
## Core Imports
12

13
```python
14
import insightface
15
from insightface.app import FaceAnalysis
16
```
17

18
For specific model usage:
19

20
```python
21
from insightface.model_zoo import get_model
22
```
23

24
For utilities:
25

26
```python
27
from insightface.utils import download, ensure_available
28
```
29

30
## Basic Usage
31

32
```python
33
import insightface
34
from insightface.app import FaceAnalysis
35
import cv2
36

37
# Initialize face analysis with default model pack
38
app = FaceAnalysis(name='buffalo_l')
39
app.prepare(ctx_id=0, det_size=(640, 640))
40

41
# Load an image
42
img = cv2.imread('path/to/image.jpg')
43

44
# Analyze faces in the image
45
faces = app.get(img)
46

47
# Process each detected face
48
for face in faces:
49
    print(f"Age: {face.age}, Gender: {face.sex}")
50
    print(f"Detection confidence: {face.det_score}")
51
    print(f"Embedding shape: {face.embedding.shape}")
52
    
53
    # Access face coordinates and landmarks
54
    bbox = face.bbox.astype(int)
55
    print(f"Face location: {bbox}")
56
    
57
    if face.kps is not None:
58
        print(f"Facial landmarks: {face.kps.shape}")
59

60
# Draw detection results on image
61
result_img = app.draw_on(img, faces)
62
cv2.imwrite('result.jpg', result_img)
63
```
64

65
## Architecture
66

67
InsightFace is organized around a modular architecture:
68

69
- **FaceAnalysis**: High-level interface that orchestrates multiple models for complete face analysis
70
- **Model Zoo**: Collection of pre-trained ONNX models for specific tasks (detection, recognition, landmarks, attributes)
71
- **Face Object**: Dynamic result container that accumulates analysis results from different models
72
- **Utils**: Download, storage, and preprocessing utilities
73
- **Data**: Sample data and object loading utilities
74

75
The design allows flexible model selection and task-specific analysis while maintaining a simple high-level API.
76

77
## Capabilities
78

79
### Face Analysis Pipeline
80

81
High-level interface for complete face analysis including detection, recognition, landmark detection, and attribute prediction. Automatically manages multiple models and provides unified results.
82

83
```python { .api }
84
class FaceAnalysis:
85
    def __init__(self, name='buffalo_l', root='~/.insightface', allowed_modules=None, **kwargs): ...
86
    def prepare(self, ctx_id, det_thresh=0.5, det_size=(640, 640)): ...
87
    def get(self, img, max_num=0) -> List[Face]: ...
88
    def draw_on(self, img, faces) -> np.ndarray: ...
89
```
90

91
[Face Analysis](./face-analysis.md)
92

93
### Model Zoo
94

95
Collection of pre-trained models for specific face analysis tasks including detection (RetinaFace, SCRFD), recognition (ArcFace), landmark detection, and attribute prediction.
96

97
```python { .api }
98
def get_model(name, **kwargs): ...
99

100
class ArcFaceONNX:
101
    def prepare(self, ctx_id, **kwargs): ...
102
    def get(self, img, face) -> np.ndarray: ...
103
    def compute_sim(self, feat1, feat2) -> float: ...
104

105
class RetinaFace:
106
    def prepare(self, ctx_id, **kwargs): ...
107
    def detect(self, img, input_size=None, max_num=0, metric='default') -> Tuple[np.ndarray, np.ndarray]: ...
108
```
109

110
[Model Zoo](./model-zoo.md)
111

112
### Face Processing Utilities
113

114
Utilities for face alignment, transformation, and preprocessing including normalization, cropping, and coordinate transformations.
115

116
```python { .api }
117
def norm_crop(img, landmark, image_size=112, mode='arcface') -> np.ndarray: ...
118
def estimate_norm(lmk, image_size=112, mode='arcface') -> np.ndarray: ...
119
def transform(data, center, output_size, scale, rotation) -> Tuple[np.ndarray, np.ndarray]: ...
120
```
121

122
[Face Processing](./face-processing.md)
123

124
### 3D Mask Rendering
125

126
Advanced 3D face mask rendering using morphable models for face swapping, virtual try-on, and augmented reality applications.
127

128
```python { .api }
129
class MaskRenderer:
130
    def __init__(self, name='buffalo_l', root='~/.insightface', insfa=None): ...
131
    def prepare(self, ctx_id=0, det_thresh=0.5, det_size=(128, 128)): ...
132
    def render_mask(self, face_image, mask_image, params, input_is_rgb=False, auto_blend=True, positions=[0.1, 0.33, 0.9, 0.7]): ...
133
```
134

135
[3D Mask Rendering](./mask-rendering.md)
136

137
### Model Management
138

139
Download, storage, and management utilities for pre-trained models and model packs.
140

141
```python { .api }
142
def download(sub_dir, name, force=False, root='~/.insightface') -> str: ...
143
def ensure_available(sub_dir, name, root='~/.insightface') -> str: ...
144
def get_model_dir(name, root='~/.insightface') -> str: ...
145
```
146

147
[Model Management](./model-management.md)
148

149
### Sample Data and Objects
150

151
Utilities for loading sample images and objects for testing and development.
152

153
```python { .api }
154
def get_image(name, to_rgb=False) -> np.ndarray: ...
155
def get_object(name) -> Any: ...
156
```
157

158
[Sample Data](./sample-data.md)
159

160
### Command Line Interface
161

162
CLI commands for model management and dataset processing operations.
163

164
```python { .api }
165
class ModelDownloadCommand:
166
    def __init__(self, model: str, root: str, force: bool): ...
167
    def run(self) -> None: ...
168

169
def main() -> None: ...  # CLI entry point
170
```
171

172
[Command Line Interface](./cli.md)
173

174
### 3D Morphable Face Models
175

176
Advanced 3D face modeling capabilities using morphable models for face reconstruction, pose estimation, and 3D face analysis.
177

178
```python { .api }
179
class MorphabelModel:
180
    def __init__(self, model_path, model_type='BFM'): ...
181
    def generate_vertices(self, shape_para, exp_para) -> np.ndarray: ...
182
    def generate_colors(self, tex_para) -> np.ndarray: ...
183
    def transform(self, vertices, s, angles, t3d) -> np.ndarray: ...
184
    def fit(self, x, X_ind, max_iter=4, isShow=False) -> Tuple[np.ndarray, ...]: ...
185
```
186

187
[3D Morphable Models](./3d-models.md)
188

189
## Types
190

191
```python { .api }
192
class Face(dict):
193
    """Face detection and analysis result object with dynamic attributes."""
194
    def __init__(self, d=None, **kwargs): ...
195
    
196
    # Properties
197
    @property
198
    def embedding_norm -> float: ...
199
    @property  
200
    def normed_embedding -> np.ndarray: ...
201
    @property
202
    def sex -> str: ...  # 'M' or 'F'
203
    
204
    # Dynamic attributes set by models:
205
    # bbox: np.ndarray - Bounding box [x1, y1, x2, y2]
206
    # kps: np.ndarray - Facial keypoints/landmarks
207
    # det_score: float - Detection confidence score
208
    # embedding: np.ndarray - Face embedding vector
209
    # gender: int - Gender prediction (0=female, 1=male)
210
    # age: int - Age prediction
211
    # landmark_3d_68: np.ndarray - 68 3D facial landmarks
212
    # pose: np.ndarray - Head pose [pitch, yaw, roll]
213

214
# Common type aliases
215
ImageArray = np.ndarray
216
BoundingBox = np.ndarray  # Shape: (4,) [x1, y1, x2, y2]
217
Landmarks = np.ndarray    # Shape: (n_points, 2) or (n_points, 3)
218
Embedding = np.ndarray    # Shape: (512,) for most models
219
```