0
# Face Recognition
1

2
A comprehensive Python library for face detection, recognition, and manipulation using state-of-the-art deep learning models. Built on dlib's deep learning models with 99.38% accuracy on the Labeled Faces in the Wild benchmark, this library provides simple APIs for detecting faces, extracting facial landmarks, generating face encodings, and comparing faces for identification.
3

4
## Package Information
5

6
- **Package Name**: face_recognition
7
- **Language**: Python  
8
- **Installation**: `pip install face_recognition`
9
- **Dependencies**: dlib (>=19.3.0), numpy, Pillow, scipy (>=0.17.0), face_recognition_models (>=0.3.0), Click (>=6.0)
10
- **Platform Support**: macOS, Linux (Python 2.7, 3.3+)
11

12
## Core Imports
13

14
```python
15
import face_recognition
16
```
17

18
All functions are available directly from the main module:
19

20
```python
21
# Import specific functions
22
from face_recognition import load_image_file, face_locations, face_encodings, compare_faces
23
```
24

25
## Basic Usage
26

27
```python
28
import face_recognition
29

30
# Load images
31
known_image = face_recognition.load_image_file("known_person.jpg")
32
unknown_image = face_recognition.load_image_file("unknown_person.jpg")
33

34
# Find face encodings
35
known_encoding = face_recognition.face_encodings(known_image)[0]
36
unknown_encoding = face_recognition.face_encodings(unknown_image)[0]
37

38
# Compare faces
39
results = face_recognition.compare_faces([known_encoding], unknown_encoding)
40
print(results[0])  # True if match, False otherwise
41

42
# Get similarity distance (lower = more similar)
43
distance = face_recognition.face_distance([known_encoding], unknown_encoding)
44
print(distance[0])  # e.g., 0.45
45
```
46

47
## Architecture
48

49
The library is built on a modular architecture using dlib's machine learning models:
50

51
- **Detection Models**: HOG (CPU-optimized) and CNN (GPU-accelerated) for face detection
52
- **Landmark Models**: 68-point (detailed) and 5-point (fast) facial feature detection  
53
- **Recognition Model**: Deep neural network for generating 128-dimensional face encodings
54
- **Distance Calculation**: Euclidean distance for face comparison with configurable tolerance
55

56
This design enables a complete face recognition pipeline from raw images to identification results.
57

58
## Capabilities
59

60
### Image Loading
61

62
Load image files into numpy arrays suitable for face processing.
63

64
```python { .api }
65
def load_image_file(file, mode='RGB'):
66
    """
67
    Loads an image file (.jpg, .png, etc) into a numpy array.
68
    
69
    Args:
70
        file: Image file name or file object to load
71
        mode: Format to convert the image to. 'RGB' (8-bit RGB, 3 channels) or 'L' (black and white)
72
    
73
    Returns:
74
        numpy.ndarray: Image contents as numpy array
75
    """
76
```
77

78
### Face Detection
79

80
Detect and locate faces in images, returning bounding box coordinates.
81

82
```python { .api }
83
def face_locations(img, number_of_times_to_upsample=1, model="hog"):
84
    """
85
    Returns an array of bounding boxes of human faces in an image.
86
    
87
    Args:
88
        img: An image (as a numpy array)
89
        number_of_times_to_upsample: How many times to upsample the image looking for faces. Higher numbers find smaller faces
90
        model: Which face detection model to use. "hog" is less accurate but faster on CPUs. "cnn" is more accurate deep-learning model which is GPU/CUDA accelerated (if available)
91
    
92
    Returns:
93
        list: List of tuples of found face locations in CSS (top, right, bottom, left) order
94
    """
95
```
96

97
```python { .api }
98
def batch_face_locations(images, number_of_times_to_upsample=1, batch_size=128):
99
    """
100
    Returns a 2D array of bounding boxes of human faces in images using the CNN face detector.
101
    Optimized for GPU processing of multiple images at once.
102
    
103
    Args:
104
        images: A list of images (each as a numpy array)
105
        number_of_times_to_upsample: How many times to upsample the image looking for faces
106
        batch_size: How many images to include in each GPU processing batch
107
    
108
    Returns:
109
        list: 2D list of tuples of found face locations in CSS (top, right, bottom, left) order
110
    """
111
```
112

113
Usage example:
114

115
```python
116
import face_recognition
117

118
image = face_recognition.load_image_file("group_photo.jpg")
119

120
# Basic face detection
121
face_locations = face_recognition.face_locations(image)
122
print(f"Found {len(face_locations)} faces")
123

124
# For higher accuracy (GPU recommended)
125
face_locations_cnn = face_recognition.face_locations(image, model="cnn")
126

127
# Batch processing for multiple images
128
images = [face_recognition.load_image_file(f"photo_{i}.jpg") for i in range(5)]
129
batch_locations = face_recognition.batch_face_locations(images)
130
```
131

132
### Facial Landmark Detection
133

134
Extract detailed facial feature locations (eyes, nose, mouth, chin) from detected faces.
135

136
```python { .api }
137
def face_landmarks(face_image, face_locations=None, model="large"):
138
    """
139
    Given an image, returns a dict of face feature locations (eyes, nose, etc) for each face in the image.
140
    
141
    Args:
142
        face_image: Image to search
143
        face_locations: Optionally provide a list of face locations to check
144
        model: Which model to use. "large" (default) returns 68 points, "small" returns 5 points but is faster
145
    
146
    Returns:
147
        list: List of dicts of face feature locations (eyes, nose, etc)
148
    """
149
```
150

151
Landmark structure for "large" model (68 points):
152
- `chin`: 17 points along jawline
153
- `left_eyebrow`: 5 points  
154
- `right_eyebrow`: 5 points
155
- `nose_bridge`: 4 points
156
- `nose_tip`: 5 points
157
- `left_eye`: 6 points
158
- `right_eye`: 6 points  
159
- `top_lip`: 12 points
160
- `bottom_lip`: 12 points
161

162
Landmark structure for "small" model (5 points):
163
- `nose_tip`: 1 point
164
- `left_eye`: 2 points
165
- `right_eye`: 2 points
166

167
Usage example:
168

169
```python
170
import face_recognition
171

172
image = face_recognition.load_image_file("face.jpg")
173

174
# Get detailed facial landmarks (68 points)
175
landmarks = face_recognition.face_landmarks(image)
176
for face_landmarks in landmarks:
177
    # Access specific features
178
    left_eye = face_landmarks['left_eye']
179
    nose_tip = face_landmarks['nose_tip']
180
    
181
# Get faster landmarks (5 points)
182
landmarks_small = face_recognition.face_landmarks(image, model="small")
183
```
184

185
### Face Encoding Generation
186

187
Generate 128-dimensional numerical representations of faces for recognition and comparison.
188

189
```python { .api }
190
def face_encodings(face_image, known_face_locations=None, num_jitters=1, model="small"):
191
    """
192
    Given an image, return the 128-dimension face encoding for each face in the image.
193
    
194
    Args:
195
        face_image: The image that contains one or more faces
196
        known_face_locations: Optional - the bounding boxes of each face if you already know them
197
        num_jitters: How many times to re-sample the face when calculating encoding. Higher is more accurate, but slower (i.e. 100 is 100x slower)
198
        model: Which model to use. "small" (default) which only returns 5 points but is faster, or "large" which returns 68 points but is more detailed
199
    
200
    Returns:
201
        list: List of 128-dimensional face encodings (one for each face in the image)
202
    """
203
```
204

205
Usage example:
206

207
```python
208
import face_recognition
209

210
image = face_recognition.load_image_file("person.jpg")
211

212
# Generate face encodings
213
encodings = face_recognition.face_encodings(image)
214

215
if len(encodings) > 0:
216
    encoding = encodings[0]  # Get first face encoding
217
    print(f"Encoding shape: {encoding.shape}")  # (128,)
218
    
219
    # For higher accuracy (slower)
220
    accurate_encoding = face_recognition.face_encodings(image, num_jitters=100)[0]
221
```
222

223
### Face Comparison
224

225
Compare face encodings to determine if faces match or calculate similarity distances.
226

227
```python { .api }
228
def compare_faces(known_face_encodings, face_encoding_to_check, tolerance=0.6):
229
    """
230
    Compare a list of face encodings against a candidate encoding to see if they match.
231
    
232
    Args:
233
        known_face_encodings: A list of known face encodings
234
        face_encoding_to_check: A single face encoding to compare against the list
235
        tolerance: How much distance between faces to consider it a match. Lower is more strict. 0.6 is typical best performance
236
    
237
    Returns:
238
        list: List of True/False values indicating which known_face_encodings match the face encoding to check
239
    """
240
```
241

242
```python { .api }
243
def face_distance(face_encodings, face_to_compare):
244
    """
245
    Given a list of face encodings, compare them to a known face encoding and get a euclidean distance
246
    for each comparison face. The distance tells you how similar the faces are.
247
    
248
    Args:
249
        face_encodings: List of face encodings to compare
250
        face_to_compare: A face encoding to compare against
251
    
252
    Returns:
253
        numpy.ndarray: Array with the distance for each face in the same order as the 'face_encodings' array
254
    """
255
```
256

257
Usage example:
258

259
```python
260
import face_recognition
261

262
# Load known faces
263
known_image = face_recognition.load_image_file("known_person.jpg")
264
known_encoding = face_recognition.face_encodings(known_image)[0]
265

266
# Load unknown face
267
unknown_image = face_recognition.load_image_file("unknown_person.jpg")
268
unknown_encoding = face_recognition.face_encodings(unknown_image)[0]
269

270
# Compare faces
271
matches = face_recognition.compare_faces([known_encoding], unknown_encoding)
272
print(f"Match: {matches[0]}")
273

274
# Get distance (0.0 = identical, >1.0 = very different)
275
distances = face_recognition.face_distance([known_encoding], unknown_encoding)
276
print(f"Distance: {distances[0]:.2f}")
277

278
# Custom tolerance
279
strict_matches = face_recognition.compare_faces([known_encoding], unknown_encoding, tolerance=0.5)
280
```
281

282
## Command Line Tools
283

284
The package includes two command-line utilities for batch processing:
285

286
### Face Recognition CLI
287

288
Compare faces in images against a directory of known people:
289

290
```bash
291
face_recognition ./pictures_of_people_i_know/ ./unknown_pictures/
292
```
293

294
Additional options:
295
```bash
296
# Use CPU cores for parallel processing (default: 1, use -1 for all cores)
297
face_recognition --cpus 4 ./known_people/ ./unknown_pictures/
298

299
# Adjust tolerance for matching (default: 0.6, lower = stricter)
300
face_recognition --tolerance 0.5 ./known_people/ ./unknown_pictures/
301

302
# Show distance values in output
303
face_recognition --show-distance ./known_people/ ./unknown_pictures/
304
```
305

306
Output format: `filename,person_name` or `filename,person_name,distance` (with --show-distance)
307

308
### Face Detection CLI  
309

310
Detect and locate faces in images:
311

312
```bash
313
face_detection ./unknown_pictures/
314
```
315

316
Additional options:
317
```bash
318
# Use CNN model for better accuracy (default: hog)
319
face_detection --model cnn ./unknown_pictures/
320

321
# Use multiple CPU cores for parallel processing
322
face_detection --cpus 4 ./unknown_pictures/
323
```
324

325
Output format: `filename,top,right,bottom,left` (CSS-style coordinates)
326

327
## Data Formats
328

329
### Face Locations Format
330
Tuples in CSS order: `(top, right, bottom, left)` as pixel coordinates.
331

332
### Face Encodings Format
333
128-dimensional numpy arrays (float64) representing facial features.
334

335
### Image Format
336
- **Input**: Image files (.jpg, .png, etc.) or file objects
337
- **Internal**: numpy arrays in RGB or L (grayscale) format
338

339
## Error Handling
340

341
Common error patterns and handling:
342

343
```python
344
import face_recognition
345

346
# Check if faces were found
347
image = face_recognition.load_image_file("photo.jpg")
348
encodings = face_recognition.face_encodings(image)
349

350
if len(encodings) == 0:
351
    print("No faces found in image")
352
else:
353
    encoding = encodings[0]  # Safe to access first encoding
354

355
# Handle invalid model parameters
356
try:
357
    landmarks = face_recognition.face_landmarks(image, model="invalid")
358
except ValueError as e:
359
    print(f"Invalid model: {e}")
360

361
# Verify face_recognition_models is installed
362
try:
363
    import face_recognition_models
364
except ImportError:
365
    print("Please install face_recognition_models package")
366
```
367

368
## Performance Considerations
369

370
- **HOG model**: Faster on CPU, good for basic detection
371
- **CNN model**: More accurate, requires GPU for optimal performance  
372
- **Large landmark model**: 68 points, more detailed but slower
373
- **Small landmark model**: 5 points, faster processing
374
- **Batch processing**: Use `batch_face_locations()` for multiple images on GPU
375
- **Encoding jitters**: Higher `num_jitters` increases accuracy but reduces speed exponentially