0
# Cost Functions
1
2
Statistical cost functions for maximum likelihood estimation and least-squares fitting. These functions automatically set the correct errordef and support gradients, Numba acceleration, and various data formats.
3
4
## Capabilities
5
6
### Constants
7
8
Predefined errordef values for different types of cost functions.
9
10
```python { .api }
11
CHISQUARE: float = 1.0 # Chi-square errordef constant
12
NEGATIVE_LOG_LIKELIHOOD: float = 0.5 # Negative log-likelihood errordef constant
13
```
14
15
### Least Squares Fitting
16
17
Least-squares cost function for fitting models to (x, y, yerror) data with support for outlier-robust loss functions.
18
19
```python { .api }
20
class LeastSquares:
21
"""
22
Least-squares cost function with optional loss functions for outlier robustness.
23
"""
24
25
def __init__(self, x, y, yerror, model, *, loss="linear", verbose=0, grad=None, name=None):
26
"""
27
Initialize least-squares cost function.
28
29
Args:
30
x: Independent variable data (array-like)
31
y: Dependent variable data (array-like)
32
yerror: Uncertainties on y values (array-like or scalar)
33
model: Model function callable(x, *params) -> y_predicted
34
loss: Loss function ("linear", "soft_l1", "huber", "cauchy", "arctan")
35
"linear" = standard least-squares
36
"soft_l1" = robust to outliers
37
Other options provide different outlier treatments
38
verbose: Verbosity level (int, default: 0)
39
grad: Model gradient function (optional)
40
name: Parameter names (optional)
41
"""
42
43
@property
44
def errordef(self) -> float:
45
"""Errordef value (automatically 1.0 for least-squares)."""
46
47
def __call__(self, *args) -> float:
48
"""Evaluate cost function at parameter values."""
49
```
50
51
### Binned Likelihood Fitting
52
53
Maximum likelihood estimation for normalized probability density functions using binned data.
54
55
```python { .api }
56
class BinnedNLL:
57
"""
58
Binned negative log-likelihood for fitting normalized PDFs to binned data.
59
"""
60
61
def __init__(self, n, xe, model, use_pdf=True):
62
"""
63
Initialize binned NLL cost function.
64
65
Args:
66
n: Bin counts (array-like)
67
xe: Bin edges (array-like, length = len(n) + 1)
68
model: Model function callable(x, *params) -> probability_density
69
use_pdf: If True, model is a PDF; if False, model returns bin content
70
"""
71
72
@property
73
def errordef(self) -> float:
74
"""Errordef value (automatically 0.5 for likelihood)."""
75
```
76
77
### Unbinned Likelihood Fitting
78
79
Maximum likelihood estimation for normalized probability density functions using unbinned data.
80
81
```python { .api }
82
class UnbinnedNLL:
83
"""
84
Unbinned negative log-likelihood for fitting normalized PDFs to unbinned data.
85
"""
86
87
def __init__(self, data, model, use_pdf=True):
88
"""
89
Initialize unbinned NLL cost function.
90
91
Args:
92
data: Data points (array-like)
93
model: Model function callable(x, *params) -> probability_density
94
use_pdf: If True, model is normalized PDF; if False, model is unnormalized
95
"""
96
97
@property
98
def errordef(self) -> float:
99
"""Errordef value (automatically 0.5 for likelihood)."""
100
```
101
102
### Extended Likelihood Fitting
103
104
Maximum likelihood estimation for unnormalized densities with Poisson fluctuations in total yield.
105
106
```python { .api }
107
class ExtendedBinnedNLL:
108
"""
109
Extended binned NLL for fitting unnormalized densities to binned data.
110
"""
111
112
def __init__(self, n, xe, model):
113
"""
114
Initialize extended binned NLL cost function.
115
116
Args:
117
n: Bin counts (array-like)
118
xe: Bin edges (array-like, length = len(n) + 1)
119
model: Model function callable(x, *params) -> density_value
120
(not normalized, total integral becomes a fit parameter)
121
"""
122
123
class ExtendedUnbinnedNLL:
124
"""
125
Extended unbinned NLL for fitting unnormalized densities to unbinned data.
126
"""
127
128
def __init__(self, data, model):
129
"""
130
Initialize extended unbinned NLL cost function.
131
132
Args:
133
data: Data points (array-like)
134
model: Model function callable(x, *params) -> density_value
135
(not normalized, total integral becomes a fit parameter)
136
"""
137
```
138
139
### Template Fitting
140
141
Template fitting with bin-wise uncertainties on template histograms.
142
143
```python { .api }
144
class Template:
145
"""
146
Template fitting with uncertainties on template histograms.
147
"""
148
149
def __init__(self, n, xe, templates):
150
"""
151
Initialize template cost function.
152
153
Args:
154
n: Data histogram bin counts (array-like)
155
xe: Bin edges (array-like, length = len(n) + 1)
156
templates: List of template histograms, each can be:
157
- Simple array of bin contents
158
- (bin_contents, bin_errors) tuple
159
- Histogram object with values and errors
160
"""
161
162
@property
163
def errordef(self) -> float:
164
"""Errordef value (automatically 0.5 for likelihood-based template fit)."""
165
```
166
167
### Constraint Functions
168
169
Gaussian penalty terms for incorporating external constraints or regularization.
170
171
```python { .api }
172
class NormalConstraint:
173
"""
174
Gaussian penalty terms for parameter constraints.
175
"""
176
177
def __init__(self, params, values, errors):
178
"""
179
Initialize normal constraint.
180
181
Args:
182
params: Parameter names (list of strings)
183
values: Central values for constraints (array-like)
184
errors: Uncertainties on constraint values (array-like)
185
"""
186
187
@property
188
def errordef(self) -> float:
189
"""Errordef value (automatically 1.0 for chi-square-like constraint)."""
190
```
191
192
### Combined Cost Functions
193
194
Combine multiple cost functions with shared parameters.
195
196
```python { .api }
197
class CostSum:
198
"""
199
Combined cost function from multiple cost functions.
200
"""
201
202
def __init__(self, *costs):
203
"""
204
Initialize combined cost function.
205
206
Args:
207
*costs: Cost function objects to combine
208
"""
209
210
@property
211
def errordef(self) -> float:
212
"""Errordef value (computed from component cost functions)."""
213
214
def __call__(self, *args) -> float:
215
"""Evaluate combined cost function."""
216
```
217
218
### Base Classes and Interfaces
219
220
Protocol definitions for custom cost functions.
221
222
```python { .api }
223
class Cost:
224
"""
225
Base protocol for cost functions.
226
"""
227
228
def __call__(self, *args) -> float:
229
"""Evaluate cost function at parameter values."""
230
231
@property
232
def errordef(self) -> float:
233
"""Error definition for this cost function type."""
234
235
class Constant:
236
"""
237
Constant cost function (always returns same value).
238
"""
239
240
def __init__(self, value):
241
"""
242
Initialize constant cost function.
243
244
Args:
245
value: Constant value to return
246
"""
247
```
248
249
### Convenience Functions
250
251
Chi-square-like cost functions for common statistical scenarios.
252
253
```python { .api }
254
def chi2(y, ye, ym):
255
"""
256
Compute chi2-distributed cost for normally distributed data.
257
258
Args:
259
y: Observed values (array-like)
260
ye: Standard deviations of observed values (array-like)
261
ym: Expected values (array-like)
262
263
Returns:
264
float: Chi-square value
265
"""
266
267
def poisson_chi2(n, mu):
268
"""
269
Compute asymptotically chi2-distributed cost for Poisson-distributed data.
270
271
Args:
272
n: Observed counts (array-like)
273
mu: Expected counts (array-like)
274
275
Returns:
276
float: Asymptotic chi-square value
277
"""
278
279
def multinomial_chi2(n, mu):
280
"""
281
Compute asymptotically chi2-distributed cost for multinomially-distributed data.
282
283
Args:
284
n: Observed counts (array-like)
285
mu: Expected counts (array-like)
286
287
Returns:
288
float: Asymptotic chi-square value
289
"""
290
```
291
292
## Usage Examples
293
294
### Least Squares Fitting
295
296
```python
297
from iminuit import Minuit
298
from iminuit.cost import LeastSquares
299
import numpy as np
300
301
# Generate sample data
302
x = np.linspace(0, 10, 50)
303
y_true = 2 * x + 1
304
y_data = y_true + np.random.normal(0, 0.5, len(x))
305
y_errors = np.full(len(x), 0.5)
306
307
# Define linear model
308
def linear_model(x, slope, intercept):
309
return slope * x + intercept
310
311
# Create cost function
312
cost = LeastSquares(x, y_data, y_errors, linear_model)
313
314
# Minimize
315
m = Minuit(cost, slope=1, intercept=0)
316
m.migrad()
317
m.hesse()
318
319
print(f"Best fit: slope={m.values['slope']:.3f}, intercept={m.values['intercept']:.3f}")
320
```
321
322
### Outlier-Robust Fitting
323
324
```python
325
# Use soft_l1 loss for outlier robustness
326
cost_robust = LeastSquares(x, y_data, y_errors, linear_model, loss="soft_l1")
327
328
m_robust = Minuit(cost_robust, slope=1, intercept=0)
329
m_robust.migrad()
330
```
331
332
### Binned Likelihood Fitting
333
334
```python
335
from iminuit.cost import BinnedNLL
336
337
# Histogram data (bin counts and edges)
338
n = np.array([5, 12, 9, 15, 8, 6, 3])
339
xe = np.linspace(0, 7, 8) # 8 edges for 7 bins
340
341
# Gaussian PDF model
342
def gaussian_pdf(x, mu, sigma):
343
return np.exp(-0.5 * ((x - mu) / sigma)**2) / (sigma * np.sqrt(2 * np.pi))
344
345
# Create binned NLL
346
cost = BinnedNLL(n, xe, gaussian_pdf)
347
348
# Fit
349
m = Minuit(cost, mu=3.5, sigma=1.0)
350
m.migrad()
351
```
352
353
### Unbinned Likelihood Fitting
354
355
```python
356
from iminuit.cost import UnbinnedNLL
357
358
# Unbinned data points
359
data = np.random.normal(2.0, 1.5, 1000)
360
361
# Gaussian PDF model
362
def gaussian_pdf(x, mu, sigma):
363
return np.exp(-0.5 * ((x - mu) / sigma)**2) / (sigma * np.sqrt(2 * np.pi))
364
365
# Create unbinned NLL
366
cost = UnbinnedNLL(data, gaussian_pdf)
367
368
# Fit
369
m = Minuit(cost, mu=0, sigma=1)
370
m.migrad()
371
```
372
373
### Template Fitting
374
375
```python
376
from iminuit.cost import Template
377
378
# Data histogram
379
data_counts = np.array([10, 25, 35, 28, 15, 8])
380
bin_edges = np.linspace(0, 6, 7)
381
382
# Template histograms (signal and background)
383
signal_template = np.array([0, 5, 15, 12, 3, 0])
384
background_template = np.array([8, 12, 10, 8, 6, 4])
385
386
# Create template cost function
387
cost = Template(data_counts, bin_edges, [signal_template, background_template])
388
389
# Fit template amplitudes
390
m = Minuit(cost, x0=1.0, x1=1.0) # x0=signal scale, x1=background scale
391
m.migrad()
392
```
393
394
### Adding Constraints
395
396
```python
397
from iminuit.cost import NormalConstraint, CostSum
398
399
# Main fitting cost function
400
main_cost = LeastSquares(x, y_data, y_errors, linear_model)
401
402
# External constraint on slope parameter
403
constraint = NormalConstraint(['slope'], [2.1], [0.1])
404
405
# Combine cost functions
406
total_cost = CostSum(main_cost, constraint)
407
408
# Fit with constraint
409
m = Minuit(total_cost, slope=2.0, intercept=1.0)
410
m.migrad()
411
```
412
413
### Custom Cost Function
414
415
```python
416
# Define custom cost function
417
class MyCustomCost:
418
errordef = 1.0 # For chi-square-like cost
419
420
def __init__(self, data):
421
self.data = data
422
423
def __call__(self, param):
424
# Custom cost calculation
425
return np.sum((self.data - param)**2)
426
427
# Use with Minuit
428
cost = MyCustomCost(data)
429
m = Minuit(cost, param=1.0)
430
m.migrad()
431
```