0
# Observations and Conditioning
1
2
Structured observation handling for Gaussian process conditioning, including standard exact observations and sparse approximations using inducing points. This module provides various observation types for scalable GP inference with different approximation methods.
3
4
## Capabilities
5
6
### Standard Observations
7
8
Exact observations that represent direct evaluations of Gaussian processes at specific input points. These provide exact Bayesian conditioning without approximations.
9
10
```python { .api }
11
class Observations(AbstractObservations):
12
def __init__(self, fdd, y):
13
"""
14
Create observations from FDD and values.
15
16
Parameters:
17
- fdd: Finite-dimensional distribution
18
- y: Observed values corresponding to fdd
19
"""
20
21
def __init__(self, *pairs):
22
"""
23
Create observations from multiple (FDD, values) pairs.
24
25
Parameters:
26
- *pairs: Sequence of (fdd, y) tuples
27
"""
28
29
fdd: FDD # FDD of observations
30
y: Any # Values of observations
31
```
32
33
```python { .api }
34
# Convenient alias
35
Obs = Observations
36
```
37
38
### Pseudo-Observations (Sparse Approximations)
39
40
Approximate observations using inducing points for scalable GP inference. Supports multiple approximation methods including VFE, FITC, and DTC.
41
42
```python { .api }
43
class PseudoObservations(AbstractPseudoObservations):
44
def __init__(self, u, fdd, y):
45
"""
46
Create pseudo-observations with inducing points.
47
48
Parameters:
49
- u: Inducing points (FDD)
50
- fdd: Finite-dimensional distribution of observations
51
- y: Observed values
52
"""
53
54
def __init__(self, us, *pairs):
55
"""
56
Create pseudo-observations with multiple observation pairs.
57
58
Parameters:
59
- us: Inducing points (FDD)
60
- *pairs: Sequence of (fdd, y) tuples
61
"""
62
63
def elbo(self, measure):
64
"""
65
Compute Evidence Lower BOund (ELBO) for the approximation.
66
67
Parameters:
68
- measure: Measure containing the prior
69
70
Returns:
71
- scalar: ELBO value for optimization
72
"""
73
74
u: FDD # Inducing points
75
fdd: FDD # FDD of observations
76
y: Any # Values of observations
77
method: str = "vfe" # Approximation method
78
```
79
80
### FITC Approximation
81
82
Fully Independent Training Conditional approximation that assumes conditional independence between observations given inducing points.
83
84
```python { .api }
85
class PseudoObservationsFITC(PseudoObservations):
86
def __init__(self, u, fdd, y):
87
"""FITC pseudo-observations."""
88
89
def __init__(self, us, *pairs):
90
"""FITC pseudo-observations with multiple pairs."""
91
92
method: str = "fitc"
93
```
94
95
### DTC Approximation
96
97
Deterministic Training Conditional approximation that uses a low-rank approximation to the prior covariance.
98
99
```python { .api }
100
class PseudoObservationsDTC(PseudoObservations):
101
def __init__(self, u, fdd, y):
102
"""DTC pseudo-observations."""
103
104
def __init__(self, us, *pairs):
105
"""DTC pseudo-observations with multiple pairs."""
106
107
method: str = "dtc"
108
```
109
110
### Convenient Aliases
111
112
Shortened names for common observation types for more concise code.
113
114
```python { .api }
115
# Standard observations
116
Obs = Observations
117
118
# Pseudo-observations
119
PseudoObs = PseudoObservations
120
SparseObs = PseudoObservations # Backward compatibility
121
122
# FITC approximation
123
PseudoObsFITC = PseudoObservationsFITC
124
125
# DTC approximation
126
PseudoObsDTC = PseudoObservationsDTC
127
128
# Backward compatibility
129
SparseObservations = PseudoObservations
130
```
131
132
### Observation Methods
133
134
Common methods available on all observation types for posterior inference and model evaluation.
135
136
```python { .api }
137
class AbstractObservations:
138
def posterior_kernel(self, measure, p_i, p_j):
139
"""
140
Get posterior kernel between processes.
141
142
Parameters:
143
- measure: Measure containing the processes
144
- p_i: First process
145
- p_j: Second process
146
147
Returns:
148
- Posterior kernel function
149
"""
150
151
def posterior_mean(self, measure, p):
152
"""
153
Get posterior mean for process.
154
155
Parameters:
156
- measure: Measure containing the process
157
- p: Process to get posterior mean for
158
159
Returns:
160
- Posterior mean function
161
"""
162
```
163
164
### Pseudo-Observation Specific Methods
165
166
Additional methods available on pseudo-observations for sparse approximation evaluation and optimization.
167
168
```python { .api }
169
class AbstractPseudoObservations:
170
def K_z(self, measure):
171
"""
172
Get kernel matrix of inducing points.
173
174
Parameters:
175
- measure: Measure to evaluate kernel with
176
177
Returns:
178
- Kernel matrix of inducing points
179
"""
180
181
def elbo(self, measure):
182
"""
183
Compute evidence lower bound (ELBO) for variational inference.
184
185
Parameters:
186
- measure: Measure to compute ELBO with
187
188
Returns:
189
- Evidence lower bound value
190
"""
191
192
def mu(self, measure):
193
"""
194
Mean of optimal approximating distribution.
195
196
Parameters:
197
- measure: Measure for computation
198
199
Returns:
200
- Mean vector of optimal distribution
201
"""
202
203
def A(self, measure):
204
"""
205
Corrective variance parameter for approximation.
206
207
Parameters:
208
- measure: Measure for computation
209
210
Returns:
211
- Corrective variance matrix
212
"""
213
```
214
215
### Utility Functions
216
217
Helper functions for combining and manipulating observations.
218
219
```python { .api }
220
def combine(*observations):
221
"""
222
Combine multiple FDDs or observation pairs.
223
224
Parameters:
225
- *observations: FDD objects or (FDD, values) pairs to combine
226
227
Returns:
228
- Combined observations object
229
"""
230
```
231
232
## Usage Examples
233
234
### Standard Exact Observations
235
236
```python
237
import stheno
238
import numpy as np
239
240
# Create GP and generate synthetic data
241
gp = stheno.GP(kernel=stheno.EQ())
242
x = np.linspace(0, 2, 10)
243
y = np.sin(x) + 0.1 * np.random.randn(len(x))
244
245
# Create observations
246
fdd = gp(x, noise=0.1)
247
obs = stheno.Observations(fdd, y)
248
# or equivalently:
249
obs = stheno.Obs(fdd, y)
250
251
# Condition GP on observations
252
posterior = gp.condition(obs)
253
254
# Make predictions
255
x_pred = np.linspace(0, 2, 50)
256
pred = posterior(x_pred)
257
mean, lower, upper = pred.marginal_credible_bounds()
258
```
259
260
### Multiple Observation Sets
261
262
```python
263
# Create multiple observation sets
264
x1 = np.linspace(0, 1, 5)
265
x2 = np.linspace(1.5, 2.5, 5)
266
y1 = np.sin(x1) + 0.1 * np.random.randn(len(x1))
267
y2 = np.cos(x2) + 0.1 * np.random.randn(len(x2))
268
269
fdd1 = gp(x1, noise=0.1)
270
fdd2 = gp(x2, noise=0.1)
271
272
# Combine into single observation set
273
obs = stheno.Observations((fdd1, y1), (fdd2, y2))
274
275
# Condition on all observations simultaneously
276
posterior = gp.condition(obs)
277
```
278
279
### Sparse Approximations with Inducing Points
280
281
```python
282
# Large dataset requiring sparse approximation
283
n_obs = 1000
284
n_inducing = 50
285
286
x_obs = np.random.uniform(0, 10, n_obs)
287
y_obs = np.sin(x_obs) + 0.2 * np.random.randn(n_obs)
288
289
# Select inducing point locations
290
x_inducing = np.linspace(0, 10, n_inducing)
291
292
# Create FDDs
293
fdd_obs = gp(x_obs, noise=0.2)
294
fdd_inducing = gp(x_inducing)
295
296
# VFE approximation (default)
297
pseudo_obs_vfe = stheno.PseudoObservations(fdd_inducing, fdd_obs, y_obs)
298
299
# FITC approximation
300
pseudo_obs_fitc = stheno.PseudoObservationsFITC(fdd_inducing, fdd_obs, y_obs)
301
302
# DTC approximation
303
pseudo_obs_dtc = stheno.PseudoObservationsDTC(fdd_inducing, fdd_obs, y_obs)
304
305
# Condition using sparse approximation
306
posterior_vfe = gp.condition(pseudo_obs_vfe)
307
posterior_fitc = gp.condition(pseudo_obs_fitc)
308
posterior_dtc = gp.condition(pseudo_obs_dtc)
309
```
310
311
### Model Selection with ELBO
312
313
```python
314
# Compare different numbers of inducing points using ELBO
315
inducing_counts = [10, 25, 50, 100]
316
elbos = []
317
318
for m in inducing_counts:
319
x_inducing = np.linspace(0, 10, m)
320
fdd_inducing = gp(x_inducing)
321
pseudo_obs = stheno.PseudoObservations(fdd_inducing, fdd_obs, y_obs)
322
323
# Compute ELBO for this configuration
324
elbo = pseudo_obs.elbo(gp.measure)
325
elbos.append(elbo)
326
print(f"Inducing points: {m}, ELBO: {elbo:.3f}")
327
328
# Select best configuration
329
best_m = inducing_counts[np.argmax(elbos)]
330
print(f"Best number of inducing points: {best_m}")
331
```
332
333
### Advanced Sparse GP Usage
334
335
```python
336
# Multi-output sparse GP
337
gp1 = stheno.GP(kernel=stheno.EQ(), name="output1")
338
gp2 = stheno.GP(kernel=stheno.Matern52(), name="output2")
339
340
# Shared inducing points for both outputs
341
x_inducing = np.linspace(0, 5, 30)
342
u1 = gp1(x_inducing)
343
u2 = gp2(x_inducing)
344
345
# Observations for each output
346
x1_obs = np.random.uniform(0, 5, 200)
347
x2_obs = np.random.uniform(0, 5, 150)
348
y1_obs = np.sin(x1_obs) + 0.1 * np.random.randn(len(x1_obs))
349
y2_obs = np.cos(x2_obs) + 0.1 * np.random.randn(len(x2_obs))
350
351
# Create sparse observations for each output
352
sparse_obs1 = stheno.PseudoObservations(u1, gp1(x1_obs), y1_obs)
353
sparse_obs2 = stheno.PseudoObservations(u2, gp2(x2_obs), y2_obs)
354
355
# Condition both processes
356
posterior1 = gp1.condition(sparse_obs1)
357
posterior2 = gp2.condition(sparse_obs2)
358
```
359
360
### Combining Exact and Sparse Observations
361
362
```python
363
# High-quality observations (exact)
364
x_exact = np.linspace(0, 1, 10)
365
y_exact = np.sin(x_exact) + 0.05 * np.random.randn(len(x_exact))
366
obs_exact = stheno.Observations(gp(x_exact, noise=0.05), y_exact)
367
368
# Large-scale noisy observations (sparse)
369
x_sparse = np.random.uniform(1, 5, 500)
370
y_sparse = np.sin(x_sparse) + 0.2 * np.random.randn(len(x_sparse))
371
x_inducing = np.linspace(1, 5, 25)
372
obs_sparse = stheno.PseudoObservations(
373
gp(x_inducing),
374
gp(x_sparse, noise=0.2),
375
y_sparse
376
)
377
378
# Combine both types of observations
379
combined_obs = stheno.combine(obs_exact, obs_sparse)
380
posterior = gp.condition(combined_obs)
381
```
382
383
### Accessing Approximation Properties
384
385
```python
386
# Create sparse observations
387
pseudo_obs = stheno.PseudoObservations(fdd_inducing, fdd_obs, y_obs)
388
389
# Access approximation properties
390
print(f"Approximation method: {pseudo_obs.method}")
391
print(f"Number of inducing points: {len(pseudo_obs.u.x)}")
392
print(f"Number of observations: {len(pseudo_obs.y)}")
393
394
# Get approximation-specific quantities
395
K_z = pseudo_obs.K_z(gp.measure) # Inducing point covariances
396
mu = pseudo_obs.mu(gp.measure) # Optimal mean
397
A = pseudo_obs.A(gp.measure) # Corrective variance
398
elbo = pseudo_obs.elbo(gp.measure) # Evidence lower bound
399
400
print(f"ELBO: {elbo:.3f}")
401
print(f"Inducing covariance shape: {K_z.shape}")
402
```
403
404
## Complete Sparse GP Workflow
405
406
### Sparse GP Regression with ELBO Optimization
407
408
```python
409
import stheno
410
import numpy as np
411
412
# Create GP and generate data
413
gp = stheno.GP(kernel=stheno.EQ())
414
x_obs = np.linspace(0, 10, 1000) # Many observations
415
y_obs = np.sin(x_obs) + 0.1 * np.random.randn(len(x_obs))
416
417
# Choose fewer inducing points for efficiency
418
x_inducing = np.linspace(0, 10, 50) # Much fewer inducing points
419
fdd_inducing = gp(x_inducing)
420
fdd_obs = gp(x_obs, noise=0.1)
421
422
# Create sparse observations using VFE approximation
423
sparse_obs = stheno.PseudoObs(fdd_inducing, fdd_obs, y_obs)
424
425
# Evaluate approximation quality
426
elbo = sparse_obs.elbo(gp.measure)
427
print(f"ELBO: {elbo:.2f}")
428
429
# Create posterior using sparse approximation
430
posterior = gp.measure.condition(sparse_obs)
431
post_gp = posterior(gp)
432
433
# Make predictions
434
x_test = np.linspace(0, 10, 200)
435
pred = post_gp(x_test)
436
mean, lower, upper = pred.marginal_credible_bounds()
437
438
print(f"Predictions computed using {len(x_inducing)} inducing points")
439
print(f"Instead of {len(x_obs)} full observations")
440
```
441
442
### Comparing Approximation Methods
443
444
```python
445
# Compare VFE, FITC, and DTC approximations
446
methods = [
447
("VFE", stheno.PseudoObs),
448
("FITC", stheno.PseudoObsFITC),
449
("DTC", stheno.PseudoObsDTC)
450
]
451
452
for name, obs_class in methods:
453
sparse_obs = obs_class(fdd_inducing, fdd_obs, y_obs)
454
elbo = sparse_obs.elbo(gp.measure)
455
print(f"{name} ELBO: {elbo:.2f}")
456
```