Probabilistic Programming in Python: Bayesian Modeling and Probabilistic Machine Learning with Theano
Agent Success
Agent success rate when using this tile
68%
Improvement
Agent success rate improvement when using this tile compared to baseline
0.94x
Baseline
Agent success rate without this tile
72%
{
"context": "Evaluates how well the solution uses PyMC's Gaussian process and HSGP APIs to build a 1D approximate regression model, fit it with MCMC, and generate posterior/prior predictive summaries per the spec. Checks that all required priors, sampling calls, and predictive routines from the package are applied correctly to the HSGP workflow.",
"type": "weighted_checklist",
"checklist": [
{
"name": "HSGP model",
"description": "Constructs an HSGP using pm.gp.HSGP (or pm.gp.HSGPPeriodic) inside a pm.Model with m=basis_functions and boundary length derived from the input range and grid_scale.",
"max_score": 25
},
{
"name": "Positive priors",
"description": "Lengthscale and marginal standard deviation are given positive PyMC priors (e.g., pm.HalfNormal/pm.Exponential) and passed into the HSGP parameters (ls, sigma).",
"max_score": 15
},
{
"name": "Latent prior",
"description": "Calls hsgp.prior(...) on the training inputs with dims/coords aligned to the training data length to produce latent function values for downstream use.",
"max_score": 15
},
{
"name": "Noise likelihood",
"description": "Defines a positive noise prior (such as pm.HalfNormal) and connects observed targets through a likelihood (e.g., pm.Normal) that uses the HSGP latent output as its mean.",
"max_score": 10
},
{
"name": "Posterior sampling",
"description": "Runs pm.sample with the provided draws/tune (and optional random_seed), relying on the gradient-based default step (NUTS) and returning InferenceData containing latent and noise posteriors.",
"max_score": 15
},
{
"name": "Posterior predictive",
"description": "Uses pm.sample_posterior_predictive (or pm.draw with pm.sample results) on the fitted model/idata to obtain predictive means/HDIs for new_x, preserving input order and expected shapes.",
"max_score": 10
},
{
"name": "Prior predictive",
"description": "Generates prior predictive draws for supplied grids via pm.sample_prior_predictive or hsgp.prior with pm.draw, producing a size x n_points array of finite values.",
"max_score": 10
}
]
}docs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10