0
# Camera Calibration and 3D Reconstruction
1
2
OpenCV's calib3d module provides comprehensive tools for camera calibration, 3D reconstruction, pose estimation, and geometric computer vision. This module enables you to calibrate cameras, correct lens distortion, perform stereo vision, estimate object poses, and reconstruct 3D scenes from 2D images.
3
4
## Capabilities
5
6
### Camera Calibration
7
8
Camera calibration determines the intrinsic parameters (focal length, optical center, distortion coefficients) and extrinsic parameters (rotation, translation) of a camera. This is essential for accurate 3D reconstruction and measurement tasks.
9
10
```python { .api }
11
retval, cameraMatrix, distCoeffs, rvecs, tvecs = cv2.calibrateCamera(
12
objectPoints, # List of 3D points in world coordinate space
13
imagePoints, # List of 2D points in image plane
14
imageSize, # Image size as (width, height)
15
cameraMatrix=None, # Initial camera matrix (3x3)
16
distCoeffs=None, # Initial distortion coefficients
17
flags=0, # Calibration flags
18
criteria=None # Termination criteria for iterative optimization
19
)
20
# Returns: reprojection error, camera matrix, distortion coefficients,
21
# rotation vectors, translation vectors for each pattern view
22
```
23
24
```python { .api }
25
retval, corners = cv2.findChessboardCorners(
26
image, # Source chessboard view (8-bit grayscale or color)
27
patternSize, # Number of inner corners per chessboard row and column
28
corners=None, # Output array of detected corners
29
flags=None # Various operation flags (CALIB_CB_*)
30
)
31
# Returns: boolean success indicator, array of detected corner coordinates
32
# Common flags:
33
# cv2.CALIB_CB_ADAPTIVE_THRESH - Use adaptive thresholding
34
# cv2.CALIB_CB_NORMALIZE_IMAGE - Normalize image gamma
35
# cv2.CALIB_CB_FAST_CHECK - Fast check for chessboard presence
36
```
37
38
```python { .api }
39
corners = cv2.cornerSubPix(
40
image, # Input grayscale image
41
corners, # Initial coordinates of corners
42
winSize, # Half of side length of search window
43
zeroZone, # Half of dead region size (-1, -1 to disable)
44
criteria # Termination criteria (type, maxCount, epsilon)
45
)
46
# Refines corner locations to subpixel accuracy
47
# Returns: refined corner coordinates
48
```
49
50
```python { .api }
51
image = cv2.drawChessboardCorners(
52
image, # Destination image (must be color or 8-bit grayscale)
53
patternSize, # Number of inner corners per row and column
54
corners, # Array of detected corners
55
patternWasFound # Indicates whether pattern was found
56
)
57
# Draws individual chessboard corners or entire pattern
58
# Returns: image with rendered corners
59
```
60
61
```python { .api }
62
retval, centers = cv2.findCirclesGrid(
63
image, # Grid view of input circles (8-bit grayscale or color)
64
patternSize, # Number of circles per row and column (points_per_row, points_per_column)
65
centers=None, # Output array of detected centers
66
flags=cv2.CALIB_CB_SYMMETRIC_GRID, # Operation flags
67
blobDetector=None # Feature detector that finds blobs (default: SimpleBlobDetector)
68
)
69
# Finds centers in the grid of circles pattern
70
# Returns: boolean success indicator (True if all centers found and ordered), array of detected centers
71
# Common flags:
72
# cv2.CALIB_CB_SYMMETRIC_GRID - Uses symmetric pattern of circles
73
# cv2.CALIB_CB_ASYMMETRIC_GRID - Uses asymmetric pattern of circles
74
# cv2.CALIB_CB_CLUSTERING - Uses special algorithm for grid detection (more robust to perspective distortions)
75
# Requires white space around the board for robust detection
76
```
77
78
**Calibration Flags:**
79
80
```python { .api }
81
# Camera calibration flags for cv2.calibrateCamera():
82
cv2.CALIB_USE_INTRINSIC_GUESS # Use provided cameraMatrix as initial guess
83
cv2.CALIB_FIX_PRINCIPAL_POINT # Fix the principal point at the center
84
cv2.CALIB_FIX_ASPECT_RATIO # Fix fx/fy ratio
85
cv2.CALIB_ZERO_TANGENT_DIST # Tangential distortion coefficients set to zero
86
cv2.CALIB_FIX_K1 # Fix k1 distortion coefficient
87
cv2.CALIB_FIX_K2 # Fix k2 distortion coefficient
88
cv2.CALIB_FIX_K3 # Fix k3 distortion coefficient
89
cv2.CALIB_FIX_K4 # Fix k4 distortion coefficient
90
cv2.CALIB_FIX_K5 # Fix k5 distortion coefficient
91
cv2.CALIB_FIX_K6 # Fix k6 distortion coefficient
92
cv2.CALIB_RATIONAL_MODEL # Enable k4, k5, k6 coefficients
93
cv2.CALIB_THIN_PRISM_MODEL # Enable s1, s2, s3, s4 coefficients
94
cv2.CALIB_FIX_S1_S2_S3_S4 # Fix thin prism distortion coefficients
95
cv2.CALIB_TILTED_MODEL # Enable tauX, tauY coefficients
96
cv2.CALIB_FIX_TAUX_TAUY # Fix tilted sensor coefficients
97
```
98
99
### Pose Estimation
100
101
Pose estimation determines the position and orientation of an object or camera relative to a coordinate system using correspondences between 3D points and their 2D image projections.
102
103
```python { .api }
104
retval, rvec, tvec = cv2.solvePnP(
105
objectPoints, # Array of 3D object points
106
imagePoints, # Array of corresponding 2D image points
107
cameraMatrix, # Camera intrinsic matrix (3x3)
108
distCoeffs, # Distortion coefficients
109
rvec=None, # Output rotation vector
110
tvec=None, # Output translation vector
111
useExtrinsicGuess=False, # Use provided rvec/tvec as initial guess
112
flags=cv2.SOLVEPNP_ITERATIVE # Method to use for pose estimation
113
)
114
# Returns: success indicator, rotation vector (3x1), translation vector (3x1)
115
```
116
117
```python { .api }
118
retval, rvec, tvec, inliers = cv2.solvePnPRansac(
119
objectPoints, # Array of 3D object points
120
imagePoints, # Array of corresponding 2D image points
121
cameraMatrix, # Camera intrinsic matrix (3x3)
122
distCoeffs, # Distortion coefficients
123
rvec=None, # Output rotation vector
124
tvec=None, # Output translation vector
125
useExtrinsicGuess=False, # Use provided rvec/tvec as initial guess
126
iterationsCount=100, # Number of iterations
127
reprojectionError=8.0, # Inlier threshold value
128
confidence=0.99, # Required confidence level
129
inliers=None, # Output vector of inlier indices
130
flags=cv2.SOLVEPNP_ITERATIVE # Method to use
131
)
132
# Robust pose estimation using RANSAC outlier rejection
133
# Returns: success indicator, rotation vector, translation vector, inlier indices
134
```
135
136
```python { .api }
137
imagePoints, jacobian = cv2.projectPoints(
138
objectPoints, # Array of 3D object points
139
rvec, # Rotation vector (3x1 or 3x3 matrix)
140
tvec, # Translation vector (3x1)
141
cameraMatrix, # Camera intrinsic matrix (3x3)
142
distCoeffs, # Distortion coefficients
143
imagePoints=None, # Output 2D image points
144
jacobian=None, # Optional output Jacobian matrix
145
aspectRatio=0 # Optional aspect ratio parameter
146
)
147
# Projects 3D points to image plane using camera parameters
148
# Returns: 2D image points, Jacobian matrix (optional)
149
```
150
151
```python { .api }
152
dst = cv2.Rodrigues(
153
src, # Input rotation vector (3x1) or rotation matrix (3x3)
154
dst=None, # Output rotation matrix (3x3) or rotation vector (3x1)
155
jacobian=None # Optional output Jacobian matrix
156
)
157
# Converts rotation vector to rotation matrix or vice versa
158
# Returns: rotation matrix (if input is vector) or rotation vector (if input is matrix),
159
# optional Jacobian
160
```
161
162
**PnP Solution Methods:**
163
164
```python { .api }
165
# Pose estimation methods for cv2.solvePnP() and cv2.solvePnPRansac():
166
cv2.SOLVEPNP_ITERATIVE # Iterative method based on Levenberg-Marquardt
167
cv2.SOLVEPNP_EPNP # EPnP: Efficient Perspective-n-Point (4+ points)
168
cv2.SOLVEPNP_P3P # P3P: Perspective-3-Point (exactly 3 points)
169
cv2.SOLVEPNP_DLS # DLS: Direct Least-Squares (4+ points)
170
cv2.SOLVEPNP_UPNP # UPnP: Unified Perspective-n-Point
171
cv2.SOLVEPNP_AP3P # AP3P: Alternative P3P (exactly 3 points)
172
cv2.SOLVEPNP_IPPE # IPPE: Infinitesimal Plane-Based Pose Estimation
173
cv2.SOLVEPNP_IPPE_SQUARE # IPPE for square planar objects
174
cv2.SOLVEPNP_SQPNP # SQPnP: Sequential Quadratic Programming PnP
175
```
176
177
### Undistortion
178
179
Lens distortion causes straight lines to appear curved in images. Undistortion removes radial and tangential distortion effects to produce geometrically correct images.
180
181
```python { .api }
182
dst = cv2.undistort(
183
src, # Input (distorted) image
184
cameraMatrix, # Camera intrinsic matrix (3x3)
185
distCoeffs, # Distortion coefficients (4, 5, 8, 12 or 14 elements)
186
dst=None, # Output (undistorted) image
187
newCameraMatrix=None # New camera matrix (3x3), default uses original
188
)
189
# Transforms image to compensate for lens distortion
190
# Returns: undistorted image
191
```
192
193
```python { .api }
194
newCameraMatrix, roi = cv2.getOptimalNewCameraMatrix(
195
cameraMatrix, # Input camera matrix
196
distCoeffs, # Distortion coefficients
197
imageSize, # Original image size (width, height)
198
alpha, # Free scaling parameter (0=no invalid pixels, 1=all pixels)
199
newImgSize=None, # New image size (width, height), default uses imageSize
200
centerPrincipalPoint=False # Whether to center principal point
201
)
202
# Computes optimal new camera matrix for undistortion
203
# Returns: new camera matrix, valid pixel ROI (x, y, width, height)
204
```
205
206
```python { .api }
207
map1, map2 = cv2.initUndistortRectifyMap(
208
cameraMatrix, # Input camera matrix
209
distCoeffs, # Distortion coefficients
210
R, # Optional rectification transformation (3x3)
211
newCameraMatrix, # New camera matrix (3x3)
212
size, # Undistorted image size (width, height)
213
m1type, # Type of first output map (CV_32FC1 or CV_16SC2)
214
map1=None, # First output map
215
map2=None # Second output map
216
)
217
# Computes undistortion and rectification transformation maps
218
# Use with cv2.remap() for efficient repeated undistortion
219
# Returns: map1 (x-coordinates or x-coordinates + y-coordinates),
220
# map2 (y-coordinates or interpolation weights)
221
```
222
223
```python { .api }
224
dst = cv2.undistortPoints(
225
src, # Observed point coordinates (2xN/Nx2 1-channel or 1xN/Nx1 2-channel)
226
cameraMatrix, # Camera matrix [fx 0 cx; 0 fy cy; 0 0 1]
227
distCoeffs, # Input vector of distortion coefficients (k1,k2,p1,p2[,k3[,k4,k5,k6[,s1,s2,s3,s4[,τx,τy]]]])
228
dst=None, # Output ideal point coordinates after undistortion and reverse perspective transformation
229
R=None, # Rectification transformation in object space (3x3 matrix), R1 or R2 from stereoRectify
230
P=None # New camera matrix (3x3) or new projection matrix (3x4), P1 or P2 from stereoRectify
231
)
232
# Computes the ideal point coordinates from the observed point coordinates
233
# Performs reverse transformation to projectPoints, working on sparse set of points instead of raster image
234
# If matrix P is identity or omitted, dst will contain normalized point coordinates
235
# Returns: undistorted point coordinates
236
```
237
238
### Stereo Vision
239
240
Stereo vision uses two cameras to capture the same scene from different viewpoints, enabling depth perception and 3D reconstruction.
241
242
```python { .api }
243
retval, cameraMatrix1, distCoeffs1, cameraMatrix2, distCoeffs2, R, T, E, F = cv2.stereoCalibrate(
244
objectPoints, # Vector of vectors of 3D calibration pattern points
245
imagePoints1, # Vector of vectors of 2D points in first camera
246
imagePoints2, # Vector of vectors of 2D points in second camera
247
cameraMatrix1, # Input/output first camera matrix
248
distCoeffs1, # Input/output first camera distortion coefficients
249
cameraMatrix2, # Input/output second camera matrix
250
distCoeffs2, # Input/output second camera distortion coefficients
251
imageSize, # Size of image (width, height)
252
R=None, # Output rotation matrix between cameras
253
T=None, # Output translation vector between cameras
254
E=None, # Output essential matrix
255
F=None, # Output fundamental matrix
256
flags=cv2.CALIB_FIX_INTRINSIC, # Calibration flags
257
criteria=None # Termination criteria for optimization
258
)
259
# Calibrates stereo camera system
260
# Returns: reprojection error, camera matrices, distortion coefficients,
261
# rotation matrix, translation vector, essential matrix, fundamental matrix
262
```
263
264
```python { .api }
265
R1, R2, P1, P2, Q, roi1, roi2 = cv2.stereoRectify(
266
cameraMatrix1, # First camera matrix
267
distCoeffs1, # First camera distortion coefficients
268
cameraMatrix2, # Second camera matrix
269
distCoeffs2, # Second camera distortion coefficients
270
imageSize, # Size of image (width, height)
271
R, # Rotation matrix between cameras
272
T, # Translation vector between cameras
273
R1=None, # Output rectification transform for first camera
274
R2=None, # Output rectification transform for second camera
275
P1=None, # Output projection matrix for first camera (3x4)
276
P2=None, # Output projection matrix for second camera (3x4)
277
Q=None, # Output disparity-to-depth mapping matrix (4x4)
278
flags=cv2.CALIB_ZERO_DISPARITY, # Operation flags
279
alpha=-1, # Free scaling parameter
280
newImageSize=None # New image size after rectification
281
)
282
# Computes rectification transforms for stereo camera pair
283
# Returns: rectification transforms (R1, R2), projection matrices (P1, P2),
284
# disparity-to-depth matrix (Q), valid pixel ROIs (roi1, roi2)
285
```
286
287
**Stereo Calibration Flags:**
288
289
```python { .api }
290
# Stereo calibration flags for cv2.stereoCalibrate():
291
cv2.CALIB_FIX_INTRINSIC # Fix intrinsic parameters (optimize only extrinsic)
292
cv2.CALIB_USE_INTRINSIC_GUESS # Optimize intrinsic parameters starting from provided values
293
cv2.CALIB_FIX_PRINCIPAL_POINT # Fix the principal points during optimization
294
cv2.CALIB_FIX_FOCAL_LENGTH # Fix fx and fy
295
cv2.CALIB_FIX_ASPECT_RATIO # Fix fx/fy ratio
296
cv2.CALIB_SAME_FOCAL_LENGTH # Enforce fx1=fx2 and fy1=fy2
297
cv2.CALIB_ZERO_TANGENT_DIST # Tangential distortion coefficients set to zero
298
cv2.CALIB_RATIONAL_MODEL # Enable k4, k5, k6 distortion coefficients
299
```
300
301
#### Stereo Correspondence
302
303
Stereo correspondence algorithms compute disparity maps (pixel displacement between stereo images), which can be converted to depth maps for 3D reconstruction.
304
305
```python { .api }
306
class StereoBM:
307
"""
308
Block Matching stereo correspondence algorithm.
309
Fast but less accurate than SGBM, works best with textured scenes.
310
"""
311
@staticmethod
312
def create(numDisparities=0, blockSize=21): ...
313
314
def compute(self, left, right, disparity=None) -> disparity:
315
"""
316
Computes disparity map for a rectified stereo pair.
317
318
Args:
319
left: Left 8-bit single-channel or 3-channel image
320
right: Right image of the same size and type
321
disparity: Output disparity map (16-bit signed, fixed-point)
322
323
Returns: Disparity map (divide by 16 to get pixel disparity)
324
"""
325
...
326
327
def setPreFilterCap(self, preFilterCap): ...
328
def setPreFilterSize(self, preFilterSize): ...
329
def setPreFilterType(self, preFilterType): ...
330
def setBlockSize(self, blockSize): ... # Block size (odd, typically 15-21)
331
def setNumDisparities(self, numDisparities): ... # Max disparity (multiple of 16)
332
def setTextureThreshold(self, textureThreshold): ...
333
def setUniquenessRatio(self, uniquenessRatio): ...
334
def setSpeckleWindowSize(self, speckleWindowSize): ...
335
def setSpeckleRange(self, speckleRange): ...
336
def setDisp12MaxDiff(self, disp12MaxDiff): ...
337
def setMinDisparity(self, minDisparity): ...
338
def setROI1(self, roi1): ...
339
def setROI2(self, roi2): ...
340
341
class StereoSGBM:
342
"""
343
Semi-Global Block Matching stereo correspondence algorithm.
344
More accurate than StereoBM, produces smoother disparity maps, handles textureless regions better.
345
"""
346
@staticmethod
347
def create(minDisparity=0, numDisparities=16, blockSize=3, P1=0, P2=0, disp12MaxDiff=0,
348
preFilterCap=0, uniquenessRatio=0, speckleWindowSize=0, speckleRange=0, mode=None): ...
349
350
def compute(self, left, right, disparity=None) -> disparity:
351
"""
352
Computes disparity map for a rectified stereo pair.
353
354
Args:
355
left: Left 8-bit single-channel or 3-channel image
356
right: Right image of the same size and type
357
disparity: Output disparity map (16-bit signed, fixed-point)
358
359
Returns: Disparity map (divide by 16 to get pixel disparity)
360
"""
361
...
362
363
def setMinDisparity(self, minDisparity): ...
364
def setNumDisparities(self, numDisparities): ... # Max disparity minus min (multiple of 16)
365
def setBlockSize(self, blockSize): ... # Block size (odd, typically 3-11)
366
def setP1(self, P1): ... # Penalty for disparity change of 1 pixel
367
def setP2(self, P2): ... # Penalty for disparity change > 1 pixel (should be > P1)
368
def setDisp12MaxDiff(self, disp12MaxDiff): ... # Max allowed difference in left-right check
369
def setPreFilterCap(self, preFilterCap): ...
370
def setUniquenessRatio(self, uniquenessRatio): ...
371
def setSpeckleWindowSize(self, speckleWindowSize): ...
372
def setSpeckleRange(self, speckleRange): ...
373
def setMode(self, mode): ... # MODE_SGBM (default) or MODE_HH (full-scale two-pass)
374
375
# Create stereo matchers
376
stereo_bm = cv2.StereoBM.create(numDisparities=16*5, blockSize=15)
377
stereo_sgbm = cv2.StereoSGBM.create(minDisparity=0, numDisparities=16*6, blockSize=5)
378
```
379
380
**SGBM Mode Constants:**
381
382
```python { .api }
383
cv2.StereoSGBM_MODE_SGBM: int # Standard SGBM mode
384
cv2.StereoSGBM_MODE_HH: int # Full-scale two-pass dynamic programming algorithm
385
cv2.StereoSGBM_MODE_SGBM_3WAY: int # 3-way mode for improved results
386
```
387
388
**Usage Example:**
389
390
```python
391
import cv2
392
import numpy as np
393
394
# Load rectified stereo pair
395
left_img = cv2.imread('left_rect.jpg', cv2.IMREAD_GRAYSCALE)
396
right_img = cv2.imread('right_rect.jpg', cv2.IMREAD_GRAYSCALE)
397
398
# Create SGBM object (more accurate than BM)
399
stereo = cv2.StereoSGBM.create(
400
minDisparity=0,
401
numDisparities=16*6, # Must be divisible by 16
402
blockSize=5, # Odd number, typically 3-11
403
P1=8 * 3 * 5**2, # Smoothness penalty for disparity changes
404
P2=32 * 3 * 5**2, # Larger penalty for larger disparity changes
405
disp12MaxDiff=1,
406
uniquenessRatio=10,
407
speckleWindowSize=100,
408
speckleRange=32,
409
mode=cv2.StereoSGBM_MODE_SGBM_3WAY
410
)
411
412
# Compute disparity
413
disparity = stereo.compute(left_img, right_img)
414
415
# Convert to float and normalize for visualization
416
disparity_normalized = cv2.normalize(disparity, None, 0, 255, cv2.NORM_MINMAX, cv2.CV_8U)
417
418
# Convert disparity to depth using Q matrix from stereoRectify
419
# depth_map = cv2.reprojectImageTo3D(disparity, Q)
420
```
421
422
### Fisheye Camera
423
424
Fisheye lenses have extreme wide-angle views with significant barrel distortion. OpenCV provides specialized calibration and undistortion functions for fisheye cameras.
425
426
```python { .api }
427
retval, K, D, rvecs, tvecs = cv2.fisheye.calibrate(
428
objectPoints, # Vector of vectors of 3D calibration pattern points
429
imagePoints, # Vector of vectors of 2D image points
430
image_size, # Image size (width, height)
431
K, # Input/output camera intrinsic matrix (3x3)
432
D, # Input/output distortion coefficients (4x1)
433
rvecs=None, # Output rotation vectors
434
tvecs=None, # Output translation vectors
435
flags=0, # Calibration flags
436
criteria=None # Termination criteria
437
)
438
# Calibrates fisheye camera model
439
# Returns: reprojection error, camera matrix, distortion coefficients,
440
# rotation vectors, translation vectors
441
```
442
443
```python { .api }
444
undistorted = cv2.fisheye.undistortImage(
445
distorted, # Input distorted image
446
K, # Camera intrinsic matrix (3x3)
447
D, # Distortion coefficients (4x1)
448
Knew=None, # New camera matrix (default uses K)
449
new_size=None # Output image size (default uses input size)
450
)
451
# Undistorts fisheye image
452
# Returns: undistorted image
453
```
454
455
**Fisheye Calibration Flags:**
456
457
```python { .api }
458
# Fisheye calibration flags for cv2.fisheye.calibrate():
459
cv2.fisheye.CALIB_USE_INTRINSIC_GUESS # Use provided K as initial estimate
460
cv2.fisheye.CALIB_RECOMPUTE_EXTRINSIC # Recompute extrinsic parameters
461
cv2.fisheye.CALIB_CHECK_COND # Check condition number for stability
462
cv2.fisheye.CALIB_FIX_SKEW # Fix skew coefficient at zero
463
cv2.fisheye.CALIB_FIX_K1 # Fix k1 distortion coefficient
464
cv2.fisheye.CALIB_FIX_K2 # Fix k2 distortion coefficient
465
cv2.fisheye.CALIB_FIX_K3 # Fix k3 distortion coefficient
466
cv2.fisheye.CALIB_FIX_K4 # Fix k4 distortion coefficient
467
cv2.fisheye.CALIB_FIX_INTRINSIC # Fix intrinsic parameters
468
cv2.fisheye.CALIB_FIX_PRINCIPAL_POINT # Fix principal point
469
```
470
471
### Geometric Relationships
472
473
Fundamental matrix, essential matrix, and homography describe geometric relationships between two views of a scene, essential for structure from motion and image alignment.
474
475
```python { .api }
476
H, mask = cv2.findHomography(
477
srcPoints, # Source points (Nx2 or Nx1x2 array)
478
dstPoints, # Destination points (Nx2 or Nx1x2 array)
479
method=0, # Method to compute homography (0, RANSAC, LMEDS, RHO)
480
ransacReprojThreshold=3.0, # Maximum reprojection error (RANSAC/RHO only)
481
mask=None, # Output mask of inliers
482
maxIters=2000, # Maximum number of RANSAC iterations
483
confidence=0.995 # Confidence level for RANSAC
484
)
485
# Finds perspective transformation (homography) between two planes
486
# Returns: 3x3 homography matrix, mask of inlier points
487
```
488
489
```python { .api }
490
F, mask = cv2.findFundamentalMat(
491
points1, # Points from first image (Nx2 or Nx1x2 array)
492
points2, # Points from second image (Nx2 or Nx1x2 array)
493
method=cv2.FM_RANSAC, # Method (FM_7POINT, FM_8POINT, FM_RANSAC, FM_LMEDS)
494
ransacReprojThreshold=3.0, # Distance threshold for RANSAC
495
confidence=0.99, # Confidence level (0 to 1)
496
mask=None # Output mask of inlier points
497
)
498
# Calculates fundamental matrix from corresponding points in two images
499
# Returns: 3x3 fundamental matrix (or multiple matrices for FM_7POINT), inlier mask
500
```
501
502
```python { .api }
503
E, mask = cv2.findEssentialMat(
504
points1, # Points from first image (Nx2 or Nx1x2 array)
505
points2, # Points from second image (Nx2 or Nx1x2 array)
506
cameraMatrix=None, # Camera intrinsic matrix (or focal length if scalar)
507
method=cv2.RANSAC, # Method (RANSAC, LMEDS)
508
prob=0.999, # Confidence level
509
threshold=1.0, # Distance threshold for RANSAC
510
mask=None # Output mask of inlier points
511
)
512
# Calculates essential matrix from corresponding points in two images
513
# Essential matrix relates corresponding points in calibrated cameras
514
# Returns: 3x3 essential matrix, inlier mask
515
```
516
517
**Geometric Estimation Methods:**
518
519
```python { .api }
520
# Methods for homography and fundamental matrix estimation:
521
cv2.FM_7POINT # 7-point algorithm (exactly 7 points)
522
cv2.FM_8POINT # 8-point algorithm (8+ points)
523
cv2.FM_RANSAC # RANSAC-based robust method
524
cv2.FM_LMEDS # Least-Median robust method
525
526
cv2.RANSAC # RANSAC (RANdom SAmple Consensus)
527
cv2.LMEDS # LMedS (Least-Median of Squares)
528
cv2.RHO # RHO algorithm (faster RANSAC variant)
529
```
530
531
### 3D Reconstruction
532
533
3D reconstruction recovers 3D structure from 2D images by triangulating corresponding points and estimating camera poses.
534
535
```python { .api }
536
points4D = cv2.triangulatePoints(
537
projMatr1, # Projection matrix for first camera (3x4)
538
projMatr2, # Projection matrix for second camera (3x4)
539
projPoints1, # 2D points in first image (2xN array)
540
projPoints2 # 2D points in second image (2xN array)
541
)
542
# Reconstructs 3D points from two views via triangulation
543
# Returns: 4xN array of homogeneous 3D points (divide by 4th coordinate)
544
```
545
546
```python { .api }
547
retval, R, t, mask = cv2.recoverPose(
548
E, # Essential matrix
549
points1, # Points from first image (Nx2 array)
550
points2, # Points from second image (Nx2 array)
551
cameraMatrix=None, # Camera intrinsic matrix (or focal length if scalar)
552
R=None, # Output rotation matrix
553
t=None, # Output translation vector
554
mask=None, # Input/output mask of inlier points
555
distanceThresh=None # Distance threshold for point validity
556
)
557
# Recovers relative camera rotation and translation from essential matrix
558
# Returns: number of inliers, rotation matrix, translation vector, inlier mask
559
```
560
561
```python { .api }
562
image3D = cv2.reprojectImageTo3D(
563
disparity, # Input disparity map (single-channel floating-point)
564
Q, # 4x4 disparity-to-depth mapping matrix (from stereoRectify)
565
image3D=None, # Output 3-channel floating-point image (X, Y, Z)
566
handleMissingValues=False, # Whether to handle invalid disparity values
567
ddepth=-1 # Depth of output image (default uses CV_32F)
568
)
569
# Reprojects disparity image to 3D space
570
# Returns: 3-channel image where each pixel contains (X, Y, Z) coordinates
571
```
572
573
### Transformation Estimation
574
575
These functions estimate various geometric transformations between sets of points, useful for image registration, alignment, and tracking.
576
577
```python { .api }
578
retval, inliers = cv2.estimateAffine2D(
579
from_, # Source 2D points (Nx2 or Nx1x2 array)
580
to, # Destination 2D points (Nx2 or Nx1x2 array)
581
inliers=None, # Output vector of inlier indices
582
method=cv2.RANSAC, # Robust estimation method (RANSAC or LMEDS)
583
ransacReprojThreshold=3.0, # Maximum reprojection error
584
maxIters=2000, # Maximum number of iterations
585
confidence=0.99, # Confidence level
586
refineIters=10 # Number of refinement iterations
587
)
588
# Estimates 2D affine transformation (6 DOF: rotation, translation, scale, shear)
589
# Returns: 2x3 affine transformation matrix, inlier mask
590
```
591
592
```python { .api }
593
retval, inliers, scale = cv2.estimateAffine3D(
594
src, # Source 3D points (Nx3 or Nx1x3 array)
595
dst, # Destination 3D points (Nx3 or Nx1x3 array)
596
out=None, # Output 3x4 affine transformation matrix
597
inliers=None, # Output vector of inlier indices
598
ransacThreshold=3.0, # Maximum reprojection error for RANSAC
599
confidence=0.99 # Confidence level (0 to 1)
600
)
601
# Estimates 3D affine transformation between two 3D point sets
602
# Returns: 3x4 affine transformation matrix, inlier mask, scale factor
603
```
604
605
```python { .api }
606
retval, inliers = cv2.estimateAffinePartial2D(
607
from_, # Source 2D points (Nx2 or Nx1x2 array)
608
to, # Destination 2D points (Nx2 or Nx1x2 array)
609
inliers=None, # Output vector of inlier indices
610
method=cv2.RANSAC, # Robust estimation method (RANSAC or LMEDS)
611
ransacReprojThreshold=3.0, # Maximum reprojection error
612
maxIters=2000, # Maximum number of iterations
613
confidence=0.99, # Confidence level
614
refineIters=10 # Number of refinement iterations
615
)
616
# Estimates partial 2D affine transformation (4 DOF: rotation, translation, uniform scale)
617
# No shear or non-uniform scaling allowed
618
# Returns: 2x3 affine transformation matrix, inlier mask
619
```
620
621
**Robust Estimation Methods:**
622
623
```python { .api }
624
# Methods for robust transformation estimation:
625
cv2.RANSAC # RANSAC (RANdom SAmple Consensus) - handles outliers
626
cv2.LMEDS # LMedS (Least-Median of Squares) - robust to 50% outliers
627
cv2.RHO # RHO algorithm - faster RANSAC variant
628
```
629