0
# Reduction Algorithms
1
2
In-processing mitigation techniques that cast fairness constraints as Lagrange multipliers in constrained optimization problems. These algorithms retrain models to satisfy fairness constraints while maintaining predictive performance by reducing the problem back to standard machine learning training.
3
4
## Capabilities
5
6
### ExponentiatedGradient
7
8
Implements the exponentiated gradient algorithm for fair classification and regression. Uses iterative reweighting to satisfy fairness constraints while optimizing for accuracy.
9
10
```python { .api }
11
class ExponentiatedGradient:
12
def __init__(self, estimator, constraints, *, objective=None, eps=0.01,
13
max_iter=50, nu=None, eta0=2.0, run_linprog_step=True,
14
sample_weight_name="sample_weight"):
15
"""
16
Exponentiated gradient algorithm for fair machine learning.
17
18
Parameters:
19
- estimator: sklearn estimator, base learning algorithm
20
- constraints: Moment object, fairness constraint to satisfy
21
- objective: Moment object, optimization objective (default: ErrorRate)
22
- eps: float, allowed fairness constraint violation
23
- max_iter: int, maximum number of iterations
24
- nu: float, convergence threshold (auto-set if None)
25
- eta0: float, initial learning rate
26
- run_linprog_step: bool, whether to run linear programming step
27
- sample_weight_name: str, parameter name for sample weights in estimator
28
"""
29
30
def fit(self, X, y, *, sensitive_features, sample_weight=None):
31
"""
32
Fit the model with fairness constraints.
33
34
Parameters:
35
- X: array-like, feature matrix
36
- y: array-like, target values
37
- sensitive_features: array-like, sensitive feature values
38
- sample_weight: array-like, optional sample weights
39
40
Returns:
41
self
42
"""
43
44
def predict(self, X):
45
"""
46
Make predictions using the fitted fair model.
47
48
Parameters:
49
- X: array-like, feature matrix
50
51
Returns:
52
array-like: Predicted values
53
"""
54
55
def predict_proba(self, X):
56
"""Predict class probabilities (classification only)."""
57
58
@property
59
def predictors_(self):
60
"""List of predictors from the optimization process."""
61
62
@property
63
def weights_(self):
64
"""Weights for combining predictors."""
65
66
@property
67
def last_iter_(self):
68
"""Number of iterations run."""
69
70
@property
71
def best_gap_(self):
72
"""Best constraint violation achieved."""
73
74
@property
75
def best_iter_(self):
76
"""Iteration with best constraint violation."""
77
```
78
79
#### Usage Example
80
81
```python
82
from fairlearn.reductions import ExponentiatedGradient, DemographicParity
83
from sklearn.linear_model import LogisticRegression
84
85
# Create constraint and estimator
86
constraint = DemographicParity()
87
estimator = LogisticRegression()
88
89
# Create and fit fair model
90
fair_model = ExponentiatedGradient(
91
estimator=estimator,
92
constraints=constraint,
93
eps=0.01,
94
max_iter=50
95
)
96
97
fair_model.fit(X_train, y_train, sensitive_features=A_train)
98
y_pred = fair_model.predict(X_test)
99
```
100
101
### GridSearch
102
103
Implements grid search over Lagrange multipliers for fair machine learning. Systematically explores the trade-off between fairness and accuracy.
104
105
```python { .api }
106
class GridSearch:
107
def __init__(self, estimator, constraints, *, selection_rule="tradeoff",
108
constraint_weight=0.5, grid_size=10, grid_limit=2.0,
109
grid_offset=None, sample_weight_name="sample_weight"):
110
"""
111
Grid search approach to fairness constraints.
112
113
Parameters:
114
- estimator: sklearn estimator, base learning algorithm
115
- constraints: Moment object, fairness constraint to satisfy
116
- selection_rule: str, how to select solution ("tradeoff" or "best_tradeoff")
117
- constraint_weight: float, weight for constraint vs. loss in tradeoff
118
- grid_size: int, number of Lagrange multipliers to try
119
- grid_limit: float, largest Lagrange multiplier to try
120
- grid_offset: dict, base values for Lagrange multipliers
121
- sample_weight_name: str, parameter name for sample weights
122
"""
123
124
def fit(self, X, y, *, sensitive_features, sample_weight=None):
125
"""
126
Fit the model with fairness constraints using grid search.
127
128
Parameters:
129
- X: array-like, feature matrix
130
- y: array-like, target values
131
- sensitive_features: array-like, sensitive feature values
132
- sample_weight: array-like, optional sample weights
133
134
Returns:
135
self
136
"""
137
138
def predict(self, X):
139
"""Make predictions using the selected fair model."""
140
141
def predict_proba(self, X):
142
"""Predict class probabilities (classification only)."""
143
144
@property
145
def predictors_(self):
146
"""All predictors from the grid search."""
147
148
@property
149
def lambda_vecs_(self):
150
"""Lagrange multiplier vectors from grid search."""
151
152
@property
153
def objectives_(self):
154
"""Objective values for each grid point."""
155
156
@property
157
def gammas_(self):
158
"""Constraint violations for each grid point."""
159
160
@property
161
def best_idx_(self):
162
"""Index of the selected predictor."""
163
```
164
165
## Fairness Constraints
166
167
### Base Constraint Classes
168
169
Abstract base classes that define the interface for fairness constraints.
170
171
```python { .api }
172
class Moment:
173
"""Base class for moment constraints."""
174
175
def load_data(self, X, y, *, sensitive_features, sample_weight=None):
176
"""Load and validate input data."""
177
178
def gamma(self, predictor):
179
"""Calculate constraint violation for a predictor."""
180
181
def project_lambda(self, lambda_vec):
182
"""Project Lagrange multiplier to feasible region."""
183
184
def signed_weights(self, lambda_vec):
185
"""Calculate signed weights for constraint."""
186
187
class ClassificationMoment(Moment):
188
"""Base class for classification fairness constraints."""
189
190
class LossMoment(Moment):
191
"""Base class for loss-based fairness constraints."""
192
```
193
194
### Demographic Parity Constraints
195
196
Constraints that require equal positive prediction rates across groups.
197
198
```python { .api }
199
class DemographicParity(ClassificationMoment):
200
"""
201
Demographic parity constraint requiring equal selection rates.
202
203
This constraint is satisfied when P(Y_hat=1 | A=a) is equal for all values of A,
204
where A represents the sensitive feature and Y_hat represents predictions.
205
"""
206
207
def __init__(self):
208
"""Initialize demographic parity constraint."""
209
```
210
211
### Equalized Odds Constraints
212
213
Constraints that require equal true positive and false positive rates across groups.
214
215
```python { .api }
216
class EqualizedOdds(ClassificationMoment):
217
"""
218
Equalized odds constraint requiring equal TPR and FPR across groups.
219
220
This constraint is satisfied when both:
221
- P(Y_hat=1 | Y=1, A=a) is equal for all values of A (equal TPR)
222
- P(Y_hat=1 | Y=0, A=a) is equal for all values of A (equal FPR)
223
"""
224
225
def __init__(self):
226
"""Initialize equalized odds constraint."""
227
```
228
229
### Equal Opportunity Constraints
230
231
Constraints that require equal true positive rates across groups.
232
233
```python { .api }
234
class TruePositiveRateParity(ClassificationMoment):
235
"""
236
Equal opportunity constraint requiring equal true positive rates.
237
238
This constraint is satisfied when P(Y_hat=1 | Y=1, A=a) is equal
239
for all values of A.
240
"""
241
242
def __init__(self):
243
"""Initialize true positive rate parity constraint."""
244
245
class FalsePositiveRateParity(ClassificationMoment):
246
"""False positive rate parity constraint."""
247
248
class ErrorRateParity(ClassificationMoment):
249
"""Error rate parity constraint."""
250
```
251
252
### Utility-Based Constraints
253
254
General utility parity constraints that can be customized for different fairness criteria.
255
256
```python { .api }
257
class UtilityParity(ClassificationMoment):
258
"""
259
General utility parity constraint.
260
261
Allows specification of different utility functions and events
262
for flexible fairness definitions.
263
"""
264
265
def __init__(self, *, difference_bound=None, ratio_bound=None):
266
"""
267
Initialize utility parity constraint.
268
269
Parameters:
270
- difference_bound: float, maximum allowed difference in utilities
271
- ratio_bound: tuple, (lower_bound, upper_bound) for utility ratios
272
"""
273
```
274
275
### Error Rate Constraints
276
277
Constraints based on error rates and loss functions.
278
279
```python { .api }
280
class ErrorRate(ClassificationMoment):
281
"""
282
Error rate constraint for equal error rates across groups.
283
"""
284
285
def __init__(self):
286
"""Initialize error rate constraint."""
287
```
288
289
### Bounded Group Loss Constraints
290
291
Constraints that bound the loss within each sensitive group.
292
293
```python { .api }
294
class BoundedGroupLoss(LossMoment):
295
"""
296
Constraint that bounds loss within each group.
297
298
Ensures that no group has loss significantly higher than others.
299
"""
300
301
def __init__(self, loss, *, upper_bound):
302
"""
303
Initialize bounded group loss constraint.
304
305
Parameters:
306
- loss: loss function object
307
- upper_bound: float, maximum allowed loss for any group
308
"""
309
310
def load_data(self, X, y, *, sensitive_features, sample_weight=None):
311
"""Load data and prepare for constraint computation."""
312
313
def gamma(self, predictor):
314
"""Calculate constraint violation."""
315
```
316
317
## Loss Functions
318
319
Loss functions used with bounded group loss constraints.
320
321
```python { .api }
322
class SquareLoss:
323
"""
324
Square loss function: (y_true - y_pred)^2
325
"""
326
327
def eval(self, y_true, y_pred):
328
"""Evaluate square loss."""
329
330
def eval_derivative(self, y_true, y_pred):
331
"""Evaluate derivative of square loss."""
332
333
class AbsoluteLoss:
334
"""
335
Absolute loss function: |y_true - y_pred|
336
"""
337
338
def eval(self, y_true, y_pred):
339
"""Evaluate absolute loss."""
340
341
def eval_derivative(self, y_true, y_pred):
342
"""Evaluate derivative of absolute loss."""
343
344
class ZeroOneLoss(AbsoluteLoss):
345
"""
346
Zero-one loss function: I(y_true != y_pred)
347
348
Inherits from AbsoluteLoss but evaluates to 0 or 1.
349
"""
350
```
351
352
## Advanced Usage
353
354
### Custom Constraints
355
356
You can create custom constraints by inheriting from the base constraint classes:
357
358
```python
359
from fairlearn.reductions import ClassificationMoment
360
361
class CustomFairnessConstraint(ClassificationMoment):
362
def __init__(self):
363
super().__init__()
364
365
def load_data(self, X, y, *, sensitive_features, sample_weight=None):
366
# Implement data loading logic
367
pass
368
369
def gamma(self, predictor):
370
# Implement constraint violation calculation
371
pass
372
```
373
374
### Combining with Scikit-learn
375
376
Reduction algorithms are designed to work seamlessly with scikit-learn estimators:
377
378
```python
379
from sklearn.ensemble import RandomForestClassifier
380
from sklearn.model_selection import cross_val_score
381
382
# Use any sklearn estimator
383
rf = RandomForestClassifier(n_estimators=100)
384
fair_rf = ExponentiatedGradient(rf, DemographicParity())
385
386
# Works with sklearn utilities
387
scores = cross_val_score(fair_rf, X, y, cv=5)
388
```