0
# Regression Methods
1

2
Conformal prediction methods for regression that provide prediction intervals with finite-sample coverage guarantees. MAPIE implements several regression approaches including split conformal, cross conformal, jackknife-after-bootstrap, time series, and conformalized quantile regression methods.
3

4
## Capabilities
5

6
### Split Conformal Regression
7

8
Implements the split conformal prediction method for regression, dividing data into training and conformalization sets. Provides efficient prediction intervals with exact finite-sample coverage guarantees.
9

10
```python { .api }
11
class SplitConformalRegressor:
12
    """
13
    Split conformal regression predictor.
14

15
    Parameters:
16
    - estimator: RegressorMixin, base regression estimator (default: LinearRegression())
17
    - confidence_level: Union[float, List[float]], target coverage level (default: 0.9)
18
    - conformity_score: Union[str, BaseRegressionScore], conformity score method (default: "absolute")
19
    - prefit: bool, whether estimator is already fitted (default: True)
20
    - n_jobs: Optional[int], number of parallel jobs
21
    - verbose: int, verbosity level (default: 0)
22
    """
23
    def __init__(self, estimator=None, confidence_level=0.9, conformity_score="absolute", prefit=True, n_jobs=None, verbose=0): ...
24

25
    def fit(self, X_train, y_train, fit_params=None):
26
        """
27
        Fit the base estimator.
28

29
        Parameters:
30
        - X_train: ArrayLike, training features
31
        - y_train: ArrayLike, training targets
32
        - fit_params: Optional[Dict], parameters passed to estimator.fit()
33

34
        Returns:
35
        Self
36
        """
37

38
    def conformalize(self, X_conformalize, y_conformalize, predict_params=None):
39
        """
40
        Estimate prediction intervals using conformalization set.
41

42
        Parameters:
43
        - X_conformalize: ArrayLike, conformalization features
44
        - y_conformalize: ArrayLike, conformalization targets
45
        - predict_params: Optional[Dict], parameters passed to estimator.predict()
46

47
        Returns:
48
        Self
49
        """
50

51
    def predict_interval(self, X, minimize_interval_width=False, allow_infinite_bounds=False):
52
        """
53
        Predict intervals for new data.
54

55
        Parameters:
56
        - X: ArrayLike, test features
57
        - minimize_interval_width: bool, whether to minimize interval width (default: False)
58
        - allow_infinite_bounds: bool, whether to allow infinite bounds (default: False)
59

60
        Returns:
61
        Tuple[NDArray, NDArray]: (predictions, intervals)
62
        """
63

64
    def predict(self, X):
65
        """
66
        Predict point estimates for new data.
67

68
        Parameters:
69
        - X: ArrayLike, test features
70

71
        Returns:
72
        NDArray: predictions
73
        """
74
```
75

76
### Cross Conformal Regression
77

78
Implements cross conformal prediction using cross-validation for better data utilization. Provides multiple aggregation methods and robust prediction intervals.
79

80
```python { .api }
81
class CrossConformalRegressor:
82
    """
83
    Cross conformal regression predictor.
84

85
    Parameters:
86
    - estimator: RegressorMixin, base regression estimator (default: LinearRegression())
87
    - confidence_level: Union[float, List[float]], target coverage level (default: 0.9)
88
    - conformity_score: Union[str, BaseRegressionScore], conformity score method (default: "absolute")
89
    - method: str, aggregation method ("plus", "base", "minmax") (default: "plus")
90
    - cv: Union[int, BaseCrossValidator], cross-validation strategy (default: 5)
91
    - n_jobs: Optional[int], number of parallel jobs
92
    - verbose: int, verbosity level (default: 0)
93
    - random_state: Optional[int], random seed
94
    """
95
    def __init__(self, estimator=None, confidence_level=0.9, conformity_score="absolute", method="plus", cv=5, n_jobs=None, verbose=0, random_state=None): ...
96

97
    def fit_conformalize(self, X, y, groups=None, fit_params=None, predict_params=None):
98
        """
99
        Fit estimators and compute conformity scores using cross-validation.
100

101
        Parameters:
102
        - X: ArrayLike, input features
103
        - y: ArrayLike, target values
104
        - groups: Optional[ArrayLike], group labels for cross-validation
105
        - fit_params: Optional[Dict], parameters passed to estimator.fit()
106
        - predict_params: Optional[Dict], parameters passed to estimator.predict()
107

108
        Returns:
109
        Self
110
        """
111

112
    def predict_interval(self, X, aggregate_predictions="mean", minimize_interval_width=False, allow_infinite_bounds=False):
113
        """
114
        Predict intervals using cross conformal method.
115

116
        Parameters:
117
        - X: ArrayLike, test features
118
        - aggregate_predictions: str, prediction aggregation method ("mean", "median") (default: "mean")
119
        - minimize_interval_width: bool, whether to minimize interval width (default: False)
120
        - allow_infinite_bounds: bool, whether to allow infinite bounds (default: False)
121

122
        Returns:
123
        Tuple[NDArray, NDArray]: (predictions, intervals)
124
        """
125

126
    def predict(self, X, aggregate_predictions="mean"):
127
        """
128
        Predict point estimates using ensemble aggregation.
129

130
        Parameters:
131
        - X: ArrayLike, test features
132
        - aggregate_predictions: str, aggregation method ("mean", "median") (default: "mean")
133

134
        Returns:
135
        NDArray: aggregated predictions
136
        """
137
```
138

139
### Jackknife-After-Bootstrap Regression
140

141
Implements the jackknife-after-bootstrap method for prediction intervals, combining bootstrap aggregation with conformal prediction for enhanced uncertainty quantification.
142

143
```python { .api }
144
class JackknifeAfterBootstrapRegressor:
145
    """
146
    Jackknife-after-bootstrap regression predictor.
147

148
    Parameters:
149
    - estimator: RegressorMixin, base regression estimator (default: LinearRegression())
150
    - confidence_level: Union[float, List[float]], target coverage level (default: 0.9)
151
    - conformity_score: Union[str, BaseRegressionScore], conformity score method (default: "absolute")
152
    - method: str, interval construction method ("plus", "base", "minmax") (default: "plus")
153
    - resampling: Union[int, Subsample], number of bootstrap samples or Subsample object (default: 30)
154
    - aggregation_method: str, prediction aggregation ("mean", "median") (default: "mean")
155
    - n_jobs: Optional[int], number of parallel jobs
156
    - verbose: int, verbosity level (default: 0)
157
    - random_state: Optional[int], random seed
158
    """
159
    def __init__(self, estimator=None, confidence_level=0.9, conformity_score="absolute", method="plus", resampling=30, aggregation_method="mean", n_jobs=None, verbose=0, random_state=None): ...
160

161
    def fit_conformalize(self, X, y, fit_params=None, predict_params=None):
162
        """
163
        Fit bootstrap ensemble and compute conformity scores.
164

165
        Parameters:
166
        - X: ArrayLike, input features
167
        - y: ArrayLike, target values
168
        - fit_params: Optional[Dict], parameters passed to estimator.fit()
169
        - predict_params: Optional[Dict], parameters passed to estimator.predict()
170

171
        Returns:
172
        Self
173
        """
174

175
    def predict_interval(self, X, ensemble=True, minimize_interval_width=False, allow_infinite_bounds=False):
176
        """
177
        Predict intervals using jackknife-after-bootstrap method.
178

179
        Parameters:
180
        - X: ArrayLike, test features
181
        - ensemble: bool, whether to use ensemble predictions (default: True)
182
        - minimize_interval_width: bool, whether to minimize interval width (default: False)
183
        - allow_infinite_bounds: bool, whether to allow infinite bounds (default: False)
184

185
        Returns:
186
        Tuple[NDArray, NDArray]: (predictions, intervals)
187
        """
188

189
    def predict(self, X, ensemble=True):
190
        """
191
        Predict point estimates using bootstrap ensemble.
192

193
        Parameters:
194
        - X: ArrayLike, test features
195
        - ensemble: bool, whether to use ensemble predictions (default: True)
196

197
        Returns:
198
        NDArray: predictions
199
        """
200
```
201

202
### Time Series Regression
203

204
Specialized conformal prediction for time series data, implementing EnbPI (Ensemble batch Prediction Intervals) and ACI (Adaptive Conformal Inference) methods with support for temporal dependencies.
205

206
```python { .api }
207
class TimeSeriesRegressor:
208
    """
209
    Time series conformal regression predictor.
210

211
    Parameters:
212
    - estimator: Optional[RegressorMixin], base time series estimator
213
    - method: str, time series method ("enbpi", "aci") (default: "enbpi")
214
    - cv: Optional[Union[int, str, BaseCrossValidator]], cross-validation strategy
215
    - n_jobs: Optional[int], number of parallel jobs
216
    - agg_function: Optional[str], aggregation function ("mean") (default: "mean")
217
    - verbose: int, verbosity level (default: 0)
218
    - conformity_score: Optional[BaseRegressionScore], conformity score method
219
    - random_state: Optional[Union[int, np.random.RandomState]], random seed
220
    """
221
    def __init__(self, estimator=None, method="enbpi", cv=None, n_jobs=None, agg_function="mean", verbose=0, conformity_score=None, random_state=None): ...
222

223
    def fit(self, X, y, sample_weight=None, **fit_params):
224
        """
225
        Fit the time series model.
226

227
        Parameters:
228
        - X: ArrayLike, time series features
229
        - y: ArrayLike, time series targets
230
        - sample_weight: Optional[ArrayLike], sample weights
231
        - **fit_params: additional parameters passed to estimator.fit()
232

233
        Returns:
234
        Self
235
        """
236

237
    def partial_fit(self, X, y, sample_weight=None, **fit_params):
238
        """
239
        Update the time series model with new data (for EnbPI method).
240

241
        Parameters:
242
        - X: ArrayLike, new time series features
243
        - y: ArrayLike, new time series targets
244
        - sample_weight: Optional[ArrayLike], sample weights
245
        - **fit_params: additional parameters
246

247
        Returns:
248
        Self
249
        """
250

251
    def predict(self, X, ensemble=False, alpha=None):
252
        """
253
        Predict point estimates and intervals for time series.
254

255
        Parameters:
256
        - X: ArrayLike, time series features
257
        - ensemble: bool, whether to use ensemble predictions (default: False)
258
        - alpha: Optional[Union[float, Iterable[float]]], significance levels
259

260
        Returns:
261
        Union[NDArray, Tuple[NDArray, NDArray]]: predictions or (predictions, intervals)
262
        """
263
```
264

265
### Conformalized Quantile Regression
266

267
Implements conformalized quantile regression, combining quantile regression with conformal prediction for distribution-free prediction intervals.
268

269
```python { .api }
270
class ConformalizedQuantileRegressor:
271
    """
272
    Conformalized quantile regression predictor.
273

274
    Parameters:
275
    - estimator: Optional[Union[RegressorMixin, Pipeline, List[Union[RegressorMixin, Pipeline]]]], quantile regressor(s)
276
    - confidence_level: float, target coverage level (default: 0.9)
277
    - prefit: bool, whether estimator(s) are already fitted (default: False)
278
    """
279
    def __init__(self, estimator=None, confidence_level=0.9, prefit=False): ...
280

281
    def fit(self, X_train, y_train, fit_params=None):
282
        """
283
        Fit quantile regressors for target, lower, and upper quantiles.
284

285
        Parameters:
286
        - X_train: ArrayLike, training features
287
        - y_train: ArrayLike, training targets
288
        - fit_params: Optional[Dict], parameters passed to estimator.fit()
289

290
        Returns:
291
        Self
292
        """
293

294
    def conformalize(self, X_conformalize, y_conformalize, sample_weight=None):
295
        """
296
        Conformalize the quantile predictions using conformalization set.
297

298
        Parameters:
299
        - X_conformalize: ArrayLike, conformalization features
300
        - y_conformalize: ArrayLike, conformalization targets
301
        - sample_weight: Optional[ArrayLike], sample weights
302

303
        Returns:
304
        Self
305
        """
306

307
    def predict_interval(self, X):
308
        """
309
        Predict conformalized quantile intervals.
310

311
        Parameters:
312
        - X: ArrayLike, test features
313

314
        Returns:
315
        Tuple[NDArray, NDArray]: (predictions, intervals)
316
        """
317

318
    def predict(self, X):
319
        """
320
        Predict point estimates using median quantile regressor.
321

322
        Parameters:
323
        - X: ArrayLike, test features
324

325
        Returns:
326
        NDArray: predictions
327
        """
328
```
329

330
## Usage Examples
331

332
### Basic Split Conformal Regression
333

334
```python
335
from sklearn.ensemble import RandomForestRegressor
336
from mapie.regression import SplitConformalRegressor
337
import numpy as np
338

339
# Prepare data
340
X_train, X_calib, y_train, y_calib = train_test_split(X, y, test_size=0.3)
341

342
# Fit base model
343
rf = RandomForestRegressor(n_estimators=100, random_state=42)
344
rf.fit(X_train, y_train)
345

346
# Create conformal predictor
347
mapie_reg = SplitConformalRegressor(
348
    estimator=rf,
349
    prefit=True,
350
    confidence_level=[0.8, 0.9, 0.95]
351
)
352

353
# Conformalize
354
mapie_reg.conformalize(X_calib, y_calib)
355

356
# Predict with intervals
357
y_pred, y_intervals = mapie_reg.predict_interval(X_test)
358
```
359

360
### Cross Conformal with Custom Conformity Score
361

362
```python
363
from mapie.regression import CrossConformalRegressor
364
from mapie.conformity_scores import GammaConformityScore
365

366
# Use gamma conformity score for heteroscedastic data
367
gamma_score = GammaConformityScore()
368

369
mapie_reg = CrossConformalRegressor(
370
    estimator=RandomForestRegressor(n_estimators=50),
371
    conformity_score=gamma_score,
372
    method="plus",
373
    cv=10,
374
    random_state=42
375
)
376

377
# Fit and predict
378
mapie_reg.fit_conformalize(X, y)
379
y_pred, y_intervals = mapie_reg.predict_interval(X_test)
380
```
381

382
## Conformity Scores
383

384
Available conformity scores for regression:
385

386
- `"absolute"`: Absolute residual score |y - ŷ|
387
- `"gamma"`: Gamma score for heteroscedastic data
388
- `"residual_normalized"`: Normalized residual score
389
- Custom conformity scores implementing `BaseRegressionScore`