tessl/pypi-metric-learn

Python implementations of metric learning algorithms

—

Pending

Overview

Eval results

Files

Weakly-Supervised Algorithms

Name: tessl/pypi-metric-learn
Author: tessl

Weakly-supervised metric learning algorithms that learn from constraints (pairs, triplets, quadruplets) rather than explicit class labels. These algorithms are useful when you have similarity/dissimilarity information but not necessarily class labels.

Capabilities

Information Theoretic Metric Learning (ITML)

Learns a Mahalanobis distance metric by minimizing the LogDet divergence subject to linear constraints on pairs of points. Maintains the initial metric structure while satisfying similarity/dissimilarity constraints.

class ITML(MahalanobisMixin, TransformerMixin):
    def __init__(self, gamma=1.0, max_iter=1000, tol=1e-3, prior='identity', 
                 verbose=False, preprocessor=None, random_state=None):
        """
        Parameters:
        - gamma: float, regularization parameter controlling the trade-off
        - max_iter: int, maximum number of iterations  
        - tol: float, convergence tolerance for optimization
        - prior: str or array-like, prior metric ('identity', 'covariance', 'random', or matrix)
        - verbose: bool, whether to print progress messages
        - preprocessor: array-like or callable, preprocessor for input data
        - random_state: int, random state for reproducibility
        """
    
    def fit(self, pairs, y):
        """
        Fit the ITML metric learner.
        
        Parameters:
        - pairs: array-like, shape=(n_constraints, 2, n_features) or (n_constraints, 2),
                3D array of pairs or 2D array of indices
        - y: array-like, shape=(n_constraints,), constraint labels (+1 for similar, -1 for dissimilar)
        
        Returns:
        - self: returns the instance itself
        """

Usage example:

from metric_learn import ITML
import numpy as np

# Create sample pairs and constraints
pairs = np.random.randn(100, 2, 4)  # 100 pairs of 4-dimensional points
y = np.random.choice([-1, 1], 100)  # Random similarity/dissimilarity labels

itml = ITML(gamma=1.0, max_iter=100)
itml.fit(pairs, y)

Least Squares Metric Learning (LSML)

Learns a metric by minimizing the sum of squared hinge losses over constraints. Formulates metric learning as a least squares problem with similarity/dissimilarity constraints.

class LSML(MahalanobisMixin, TransformerMixin):
    def __init__(self, tol=1e-3, max_iter=1000, verbose=False, preprocessor=None, random_state=None):
        """
        Parameters:
        - tol: float, convergence tolerance
        - max_iter: int, maximum number of iterations
        - verbose: bool, whether to print progress messages  
        - preprocessor: array-like or callable, preprocessor for input data
        - random_state: int, random state for reproducibility
        """
    
    def fit(self, pairs, y):
        """
        Fit the LSML metric learner.
        
        Parameters:
        - pairs: array-like, shape=(n_constraints, 2, n_features) or (n_constraints, 2),
                3D array of pairs or 2D array of indices
        - y: array-like, shape=(n_constraints,), constraint labels (+1 for similar, -1 for dissimilar)
        
        Returns:
        - self: returns the instance itself
        """

Sparse Determinant Metric Learning (SDML)

Learns a sparse Mahalanobis distance metric by optimizing a trade-off between satisfying distance constraints and sparsity. Useful when you want to identify relevant features for the distance metric.

class SDML(MahalanobisMixin, TransformerMixin):
    def __init__(self, balance_param=0.5, sparsity_param=0.01, use_cov=True, 
                 preprocessor=None, random_state=None):
        """
        Parameters:
        - balance_param: float, balance parameter between similar and dissimilar constraints
        - sparsity_param: float, sparsity regularization parameter
        - use_cov: bool, whether to use covariance in the regularization
        - preprocessor: array-like or callable, preprocessor for input data
        - random_state: int, random state for reproducibility
        """
    
    def fit(self, pairs, y):
        """
        Fit the SDML metric learner.
        
        Parameters:
        - pairs: array-like, shape=(n_constraints, 2, n_features) or (n_constraints, 2),
                3D array of pairs or 2D array of indices
        - y: array-like, shape=(n_constraints,), constraint labels (+1 for similar, -1 for dissimilar)
        
        Returns:
        - self: returns the instance itself
        """

Usage example:

from metric_learn import SDML
from sklearn.datasets import make_blobs

# Generate sample data and create pairs
X, _ = make_blobs(n_samples=100, centers=3, n_features=5, random_state=42)

# Create pairs (indices) and labels
pairs_idx = [(i, j) for i in range(20) for j in range(i+1, 30)]
y = [1 if np.linalg.norm(X[i] - X[j]) < 2.0 else -1 for i, j in pairs_idx]

sdml = SDML(sparsity_param=0.1, balance_param=0.5)
sdml.fit(pairs_idx, y)

Relative Components Analysis (RCA)

Learns a full rank Mahalanobis distance metric based on a weighted sum of in-chunklets covariance matrices. It applies a global linear transformation to assign large weights to relevant dimensions. Those relevant dimensions are estimated using "chunklets" - subsets of points that are known to belong to the same class.

class RCA(MahalanobisMixin, TransformerMixin):
    def __init__(self, n_components=None, preprocessor=None):
        """
        Parameters:
        - n_components: int or None, dimensionality of reduced space (if None, defaults to dimension of X)
        - preprocessor: array-like or callable, preprocessor for input data
        """
    
    def fit(self, X, chunks):
        """
        Learn the RCA model.
        
        Parameters:
        - X: array-like, shape=(n_samples, n_features), data matrix where each row is a single instance
        - chunks: array-like, shape=(n_samples,), array of ints where chunks[i] == j means point i belongs to chunklet j, 
                 and chunks[i] == -1 means point i doesn't belong to any chunklet
        
        Returns:
        - self: returns the instance itself
        """

Usage example:

from metric_learn import RCA
import numpy as np

# Sample data
X = np.array([[-0.05, 3.0], [0.05, -3.0], [0.1, -3.55], [-0.1, 3.55],
              [-0.95, -0.05], [0.95, 0.05], [0.4, 0.05], [-0.4, -0.05]])

# Chunklet labels: points 0,1 are in chunk 0; points 2,3 in chunk 1, etc.
chunks = [0, 0, 1, 1, 2, 2, 3, 3]

rca = RCA(n_components=2)
rca.fit(X, chunks)

Sparse Compositional Metric Learning (SCML)

Learns a squared Mahalanobis distance from triplet constraints by optimizing sparse positive weights assigned to a set of rank-one PSD bases. Uses a stochastic composite optimization scheme to handle high-dimensional sparse metrics.

class SCML(MahalanobisMixin, TransformerMixin):
    def __init__(self, beta=1e-5, basis='triplet_diffs', n_basis=None, gamma=5e-3, 
                 max_iter=10000, output_iter=500, batch_size=10, verbose=False, 
                 preprocessor=None, random_state=None):
        """
        Parameters:
        - beta: float, L1 regularization parameter
        - basis: str or array-like, set of bases to construct the metric ('triplet_diffs' or custom array)
        - n_basis: int or None, number of bases to use (if None, determined automatically)
        - gamma: float, learning rate parameter
        - max_iter: int, maximum number of iterations
        - output_iter: int, number of iterations between progress output
        - batch_size: int, size of mini-batches for stochastic optimization
        - verbose: bool, whether to print progress messages
        - preprocessor: array-like or callable, preprocessor for input data
        - random_state: int, random state for reproducibility
        """
    
    def fit(self, triplets):
        """
        Fit the SCML metric learner.
        
        Parameters:
        - triplets: array-like, shape=(n_constraints, 3, n_features) or (n_constraints, 3),
                   3D array of triplets (anchor, positive, negative) or 2D array of indices
        
        Returns:
        - self: returns the instance itself
        """

Usage example:

from metric_learn import SCML
import numpy as np

# Create triplet constraints: [anchor, positive, negative]
triplets_idx = [(0, 1, 5), (2, 3, 7), (4, 6, 9)]  # Indices of triplets

# Assuming you have data X
X = np.random.randn(20, 5)

scml = SCML(beta=1e-4, max_iter=1000, preprocessor=X)
scml.fit(triplets_idx)

Constraint Formats

All weakly-supervised algorithms accept constraints in similar formats:

Pair Constraints

# 3D array format: pairs contain actual data points
pairs_3d = np.array([
    [[1.0, 2.0], [1.1, 2.1]],  # Similar pair
    [[1.0, 2.0], [5.0, 6.0]]   # Dissimilar pair  
])
y = [1, -1]  # 1 for similar, -1 for dissimilar

# 2D array format: pairs contain indices (requires preprocessor)
pairs_2d = np.array([[0, 1], [0, 5]])  # Indices into dataset
y = [1, -1]

Working with Preprocessors

When using index-based constraints, set up a preprocessor:

from metric_learn import ITML
import numpy as np

# Your dataset
X = np.random.randn(100, 5)

# Index-based constraints
pairs_idx = [(0, 1), (2, 10), (5, 20)]
y = [1, -1, 1]

# Fit with preprocessor
itml = ITML(preprocessor=X)
itml.fit(pairs_idx, y)

Common Usage Pattern

from metric_learn import ITML, LSML, SDML
from metric_learn import Constraints
from sklearn.datasets import load_digits
import numpy as np

# Load data
X, y_true = load_digits(return_X_y=True)

# Generate constraints from true labels (for demonstration)
constraints = Constraints(y_true)
pos_pairs, neg_pairs = constraints.positive_negative_pairs(n_constraints=500)
pairs = np.vstack([pos_pairs, neg_pairs])
y_constraints = np.hstack([np.ones(len(pos_pairs)), -np.ones(len(neg_pairs))])

# Train different weakly-supervised learners
algorithms = {
    'ITML': ITML(preprocessor=X),
    'LSML': LSML(preprocessor=X), 
    'SDML': SDML(preprocessor=X)
}

for name, algorithm in algorithms.items():
    algorithm.fit(pairs, y_constraints)
    print(f"{name} fitted successfully")
    
    # Get learned transformation matrix
    L = algorithm.components_
    print(f"{name} learned transformation shape: {L.shape}")

Install with Tessl CLI