0
# Model Diagnostics
1

2
Time series cross-validation and performance evaluation tools for Prophet model validation. Includes multiple performance metrics, rolling window analysis, and model comparison utilities.
3

4
## Capabilities
5

6
### Cross-Validation
7

8
Time series cross-validation functions for model evaluation.
9

10
```python { .api }
11
def cross_validation(
12
    model, 
13
    horizon, 
14
    period=None, 
15
    initial=None, 
16
    parallel=None, 
17
    cutoffs=None, 
18
    disable_tqdm=False, 
19
    extra_output_columns=None
20
):
21
    """
22
    Perform time series cross-validation.
23

24
    Parameters:
25
    - model: Prophet, fitted Prophet model
26
    - horizon: str or pd.Timedelta, forecast horizon (e.g., '30 days', '1 year')
27
    - period: str or pd.Timedelta, period between cutoff dates (default: 0.5 * horizon)
28
    - initial: str or pd.Timedelta, initial training period (default: 3 * horizon)
29
    - parallel: str, parallelization method ('processes', 'threads', or None)
30
    - cutoffs: list, custom cutoff dates (optional)
31
    - disable_tqdm: bool, disable progress bar (default: False)
32
    - extra_output_columns: list, additional columns to include in output
33

34
    Returns:
35
    - DataFrame with cross-validation results including 'ds', 'y', 'yhat', 'cutoff'
36
    """
37

38
def generate_cutoffs(df, horizon, initial, period):
39
    """
40
    Generate cutoff dates for cross-validation.
41

42
    Parameters:
43
    - df: DataFrame, historical data with 'ds' column
44
    - horizon: pd.Timedelta, forecast horizon
45
    - initial: pd.Timedelta, initial training period
46
    - period: pd.Timedelta, period between cutoffs
47

48
    Returns:
49
    - list of cutoff dates
50
    """
51

52
def single_cutoff_forecast(df, model, cutoff, horizon, predict_columns):
53
    """
54
    Generate forecast for a single cutoff date.
55

56
    Parameters:
57
    - df: DataFrame, historical data
58
    - model: Prophet, model template
59
    - cutoff: datetime, cutoff date
60
    - horizon: pd.Timedelta, forecast horizon
61
    - predict_columns: list, columns to include in prediction
62

63
    Returns:
64
    - DataFrame with forecast for the cutoff period
65
    """
66
```
67

68
### Performance Metrics
69

70
Calculate various performance metrics from cross-validation results.
71

72
```python { .api }
73
def performance_metrics(df, metrics=None, rolling_window=0.1, monthly=False):
74
    """
75
    Compute performance metrics from cross-validation results.
76

77
    Parameters:
78
    - df: DataFrame, cross-validation results from cross_validation()
79
    - metrics: list, list of metrics to compute (default: all available)
80
    - rolling_window: float, rolling window proportion for aggregation (default: 0.1)
81
    - monthly: bool, whether to compute monthly metrics (default: False)
82

83
    Returns:
84
    - DataFrame with performance metrics by horizon
85
    """
86

87
def mse(df, w):
88
    """
89
    Mean squared error.
90

91
    Parameters:
92
    - df: DataFrame, forecast errors
93
    - w: float, rolling window weight
94

95
    Returns:
96
    - float, MSE value
97
    """
98

99
def rmse(df, w):
100
    """
101
    Root mean squared error.
102

103
    Parameters:
104
    - df: DataFrame, forecast errors
105
    - w: float, rolling window weight
106

107
    Returns:
108
    - float, RMSE value
109
    """
110

111
def mae(df, w):
112
    """
113
    Mean absolute error.
114

115
    Parameters:
116
    - df: DataFrame, forecast errors
117
    - w: float, rolling window weight
118

119
    Returns:
120
    - float, MAE value
121
    """
122

123
def mape(df, w):
124
    """
125
    Mean absolute percent error.
126

127
    Parameters:
128
    - df: DataFrame, forecast errors
129
    - w: float, rolling window weight
130

131
    Returns:
132
    - float, MAPE value
133
    """
134

135
def mdape(df, w):
136
    """
137
    Median absolute percent error.
138

139
    Parameters:
140
    - df: DataFrame, forecast errors
141
    - w: float, rolling window weight
142

143
    Returns:
144
    - float, MDAPE value
145
    """
146

147
def smape(df, w):
148
    """
149
    Symmetric mean absolute percentage error.
150

151
    Parameters:
152
    - df: DataFrame, forecast errors
153
    - w: float, rolling window weight
154

155
    Returns:
156
    - float, SMAPE value
157
    """
158

159
def coverage(df, w):
160
    """
161
    Coverage of uncertainty intervals.
162

163
    Parameters:
164
    - df: DataFrame, forecast with uncertainty intervals
165
    - w: float, rolling window weight
166

167
    Returns:
168
    - float, coverage proportion
169
    """
170
```
171

172
### Utility Functions
173

174
Helper functions for model diagnostics and metric computation.
175

176
```python { .api }
177
def prophet_copy(m, cutoff=None):
178
    """
179
    Copy a Prophet object for cross-validation.
180

181
    Parameters:
182
    - m: Prophet, model to copy
183
    - cutoff: datetime, cutoff date for training data (optional)
184

185
    Returns:
186
    - Prophet, copied model
187
    """
188

189
def register_performance_metric(func):
190
    """
191
    Register a custom performance metric function.
192

193
    Parameters:
194
    - func: function, custom metric function with signature func(df, w) -> float
195

196
    Returns:
197
    - None (registers metric globally)
198
    """
199

200
def rolling_mean_by_h(x, h, w, name):
201
    """
202
    Compute rolling mean by forecast horizon.
203

204
    Parameters:
205
    - x: array, values to average
206
    - h: array, forecast horizons
207
    - w: float, rolling window proportion
208
    - name: str, metric name
209

210
    Returns:
211
    - DataFrame with rolling mean by horizon
212
    """
213

214
def rolling_median_by_h(x, h, w, name):
215
    """
216
    Compute rolling median by forecast horizon.
217

218
    Parameters:
219
    - x: array, values to compute median for
220
    - h: array, forecast horizons
221
    - w: float, rolling window proportion
222
    - name: str, metric name
223

224
    Returns:
225
    - DataFrame with rolling median by horizon
226
    """
227
```
228

229
### Global Variables
230

231
```python { .api }
232
PERFORMANCE_METRICS = {
233
    'mse': mse,
234
    'rmse': rmse,
235
    'mae': mae,
236
    'mape': mape,
237
    'mdape': mdape,
238
    'smape': smape,
239
    'coverage': coverage
240
}
241
# Dictionary of registered performance metrics
242
```
243

244
## Usage Examples
245

246
### Basic Cross-Validation
247

248
```python
249
from prophet.diagnostics import cross_validation, performance_metrics
250

251
# Perform cross-validation
252
df_cv = cross_validation(
253
    model, 
254
    horizon='30 days', 
255
    period='10 days', 
256
    initial='90 days'
257
)
258

259
# Compute performance metrics
260
df_p = performance_metrics(df_cv)
261
print(df_p)
262
```
263

264
### Custom Metrics and Analysis
265

266
```python
267
import numpy as np
268

269
# Register custom metric
270
def custom_metric(df, w):
271
    return np.mean(np.abs(df['y'] - df['yhat']) / df['y'])
272

273
register_performance_metric(custom_metric)
274

275
# Compute metrics with custom metric
276
df_p = performance_metrics(df_cv, metrics=['rmse', 'custom_metric'])
277

278
# Plot cross-validation results
279
from prophet.plot import plot_cross_validation_metric
280
fig = plot_cross_validation_metric(df_cv, 'rmse')
281
fig.show()
282
```
283

284
### Parallel Cross-Validation
285

286
```python
287
# Use parallel processing for faster cross-validation
288
df_cv = cross_validation(
289
    model,
290
    horizon='60 days',
291
    period='30 days', 
292
    initial='180 days',
293
    parallel='processes'  # or 'threads'
294
)
295

296
# Monthly aggregated metrics
297
df_monthly = performance_metrics(df_cv, monthly=True)
298
```