or run

npx @tessl/cli init

Version

Tile

Overview

Evals

Files

docs

adversarial.md assessment.md datasets.md index.md postprocessing.md preprocessing.md reductions.md

tile.json

tessl/pypi-fairlearn

A Python package to assess and improve fairness of machine learning models

Workspace: tessl
Visibility: Public
Created: 3 months ago
Last updated: 3 months ago
Describes: pkg:pypi/fairlearn@0.12.x

To install, run

npx @tessl/cli install tessl/pypi-fairlearn@0.12.0

Fairlearn

A comprehensive Python library for assessing and improving fairness of machine learning models. Fairlearn provides algorithms for mitigating unfairness in AI systems, focusing on group fairness through allocation harms and quality-of-service harms, with tools for both assessment and mitigation of bias.

Package Information

Package Name: fairlearn
Language: Python
Installation: pip install fairlearn

Core Imports

import fairlearn

Common imports for specific functionality:

# For fairness assessment
from fairlearn.metrics import MetricFrame, demographic_parity_difference, equalized_odds_difference

# For fairness mitigation algorithms
from fairlearn.reductions import ExponentiatedGradient, GridSearch, DemographicParity, EqualizedOdds

# For preprocessing and postprocessing
from fairlearn.preprocessing import CorrelationRemover
from fairlearn.postprocessing import ThresholdOptimizer

# For datasets
from fairlearn.datasets import fetch_adult, fetch_acs_income

Basic Usage

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from fairlearn.metrics import MetricFrame, demographic_parity_difference, equalized_odds_difference
from fairlearn.reductions import ExponentiatedGradient, DemographicParity
from fairlearn.datasets import fetch_adult

# Load sample dataset
X, y, sensitive_features = fetch_adult(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test, A_train, A_test = train_test_split(
    X, y, sensitive_features, test_size=0.3, random_state=42, stratify=y
)

# Train baseline model
baseline_model = LogisticRegression(random_state=42)
baseline_model.fit(X_train, y_train)
y_pred_baseline = baseline_model.predict(X_test)

# Assess fairness using MetricFrame
mf = MetricFrame(
    metrics={"accuracy": lambda y_true, y_pred: (y_true == y_pred).mean(),
             "selection_rate": lambda y_true, y_pred: y_pred.mean()},
    y_true=y_test,
    y_pred=y_pred_baseline,
    sensitive_features=A_test
)

print("Baseline model fairness:")
print(mf.by_group)
print(f"Demographic parity difference: {demographic_parity_difference(y_test, y_pred_baseline, sensitive_features=A_test)}")

# Mitigate unfairness using ExponentiatedGradient
constraint = DemographicParity()
mitigator = ExponentiatedGradient(baseline_model, constraint)
mitigator.fit(X_train, y_train, sensitive_features=A_train)
y_pred_mitigated = mitigator.predict(X_test)

print(f"Mitigated demographic parity difference: {demographic_parity_difference(y_test, y_pred_mitigated, sensitive_features=A_test)}")

Architecture

Fairlearn is structured around four main components:

Assessment (fairlearn.metrics): Tools to measure fairness through disaggregated metrics and fairness-specific functions
Mitigation: Multiple approaches to improve fairness:
- Reduction methods (fairlearn.reductions): Cast fairness as constrained optimization problems
- Preprocessing (fairlearn.preprocessing): Transform features to reduce correlation with sensitive attributes
- Postprocessing (fairlearn.postprocessing): Adjust model outputs to satisfy fairness constraints
- Adversarial (fairlearn.adversarial): Neural network approaches using adversarial training
Datasets (fairlearn.datasets): Standard datasets for benchmarking fairness algorithms
Utilities (fairlearn.utils): Supporting functions for data processing and validation

This design enables comprehensive fairness workflows from assessment through mitigation, supporting various fairness definitions and constraints to address both allocation harms (differences in quality or allocation of opportunities/resources) and quality-of-service harms (differences in quality of service across groups).

Capabilities

Fairness Assessment

Comprehensive tools for measuring fairness through disaggregated metrics across sensitive groups. The MetricFrame class provides the core functionality for computing and analyzing fairness metrics, while specialized fairness functions measure specific fairness criteria.

class MetricFrame:
    def __init__(self, metrics, y_true, y_pred, *, sensitive_features=None, 
                 control_features=None, sample_params=None): ...
    
def demographic_parity_difference(y_true, y_pred, *, sensitive_features,
                                  sample_weight=None): ...
def equalized_odds_difference(y_true, y_pred, *, sensitive_features,
                              sample_weight=None): ...
def equal_opportunity_difference(y_true, y_pred, *, sensitive_features,
                                 sample_weight=None): ...

Fairness Assessment

Reduction Algorithms

In-processing mitigation techniques that cast fairness constraints as Lagrange multipliers in constrained optimization problems. These algorithms retrain models to satisfy fairness constraints while maintaining predictive performance.

class ExponentiatedGradient:
    def __init__(self, estimator, constraints, *, eps=0.01, T=50, nu=None,
                 eta_mul=2.0, run_linprog_step=True, sample_weight_name="sample_weight"): ...
    def fit(self, X, y, *, sensitive_features, sample_weight=None): ...
    def predict(self, X): ...

class GridSearch:
    def __init__(self, estimator, constraints, *, selection_rule="tradeoff",
                 constraint_weight=0.5, grid_size=10, grid_limit=2.0,
                 grid_offset=None, sample_weight_name="sample_weight"): ...

Reduction Algorithms

Preprocessing

Preprocessing techniques that transform features to reduce correlation with sensitive attributes, addressing fairness at the data preparation stage.

class CorrelationRemover:
    def __init__(self, *, sensitive_feature_ids, alpha=1.0): ...
    def fit(self, X, y=None): ...
    def transform(self, X): ...
    def fit_transform(self, X, y=None): ...

Preprocessing

Postprocessing

Postprocessing techniques that adjust trained model outputs to satisfy fairness constraints without retraining, optimizing decision thresholds across groups.

class ThresholdOptimizer:
    def __init__(self, *, estimator=None, constraints="demographic_parity",
                 objective="accuracy_score", grid_size=1000,
                 flip=False, plot=False, prefit=False): ...
    def fit(self, X, y, *, sensitive_features, sample_weight=None, **kwargs): ...
    def predict(self, X, *, sensitive_features, random_state=None): ...

Postprocessing

Adversarial Training

Neural network-based approaches using adversarial training to learn fair representations while maintaining predictive utility.

class AdversarialFairnessClassifier:
    def __init__(self, backend="torch", *, predictor_model=None, adversary_model=None,
                 alpha=1.0, epochs=1, batch_size=32, shuffle=True, progress_updates=None,
                 skip_validation=False, callbacks=None, random_state=None): ...
    def fit(self, X, y, *, sensitive_features, sample_weight=None): ...
    def predict(self, X): ...
    def predict_proba(self, X): ...

class AdversarialFairnessRegressor:
    def __init__(self, backend="torch", *, predictor_model=None, adversary_model=None,
                 alpha=1.0, epochs=1, batch_size=32, shuffle=True, progress_updates=None,
                 skip_validation=False, callbacks=None, random_state=None): ...

Adversarial Training

Datasets

Standard datasets commonly used for fairness research and benchmarking, with consistent interfaces and built-in sensitive feature identification.

def fetch_adult(*, cache=True, data_home=None, as_frame=True, return_X_y=False): ...
def fetch_acs_income(*, cache=True, data_home=None, as_frame=True, return_X_y=False,
                     state="CA", year=2018, with_nulls=False, 
                     optimization="mem", accept_download=False): ...
def fetch_bank_marketing(*, cache=True, data_home=None, as_frame=True, return_X_y=False): ...
def fetch_boston(*, cache=True, data_home=None, as_frame=True, return_X_y=False, warn=True): ...

Datasets

Common Types

# Fairness constraint types for reduction algorithms
class Moment: ...
class ClassificationMoment(Moment): ...
class LossMoment(Moment): ...

# Common constraint implementations  
class DemographicParity(ClassificationMoment): ...
class EqualizedOdds(ClassificationMoment): ...
class TruePositiveRateParity(ClassificationMoment): ...
class ErrorRateParity(ClassificationMoment): ...

# Loss functions for bounded group loss constraints
class SquareLoss: ...
class AbsoluteLoss: ...
class ZeroOneLoss(AbsoluteLoss): ...

# Custom warning for dataset fairness issues
class DataFairnessWarning(UserWarning): ...

Version

Tile

Files

tessl/pypi-fairlearn

To install, run

index.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Fairlearn

Package Information

Core Imports

Basic Usage

Architecture

Capabilities

Fairness Assessment

Reduction Algorithms

Preprocessing

Postprocessing

Adversarial Training

Datasets

Common Types

index.mddocs/