CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/pypi-pymc3

Probabilistic Programming in Python: Bayesian Modeling and Probabilistic Machine Learning with Theano

Agent Success

Agent success rate when using this tile

68%

Improvement

Agent success rate improvement when using this tile compared to baseline

0.94x

Baseline

Agent success rate without this tile

72%

Overview
Eval results
Files

rubric.jsonevals/scenario-5/

{
  "context": "Evaluates how well the solution uses PyMC's Gaussian process and HSGP APIs to build a 1D approximate regression model, fit it with MCMC, and generate posterior/prior predictive summaries per the spec. Checks that all required priors, sampling calls, and predictive routines from the package are applied correctly to the HSGP workflow.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "HSGP model",
      "description": "Constructs an HSGP using pm.gp.HSGP (or pm.gp.HSGPPeriodic) inside a pm.Model with m=basis_functions and boundary length derived from the input range and grid_scale.",
      "max_score": 25
    },
    {
      "name": "Positive priors",
      "description": "Lengthscale and marginal standard deviation are given positive PyMC priors (e.g., pm.HalfNormal/pm.Exponential) and passed into the HSGP parameters (ls, sigma).",
      "max_score": 15
    },
    {
      "name": "Latent prior",
      "description": "Calls hsgp.prior(...) on the training inputs with dims/coords aligned to the training data length to produce latent function values for downstream use.",
      "max_score": 15
    },
    {
      "name": "Noise likelihood",
      "description": "Defines a positive noise prior (such as pm.HalfNormal) and connects observed targets through a likelihood (e.g., pm.Normal) that uses the HSGP latent output as its mean.",
      "max_score": 10
    },
    {
      "name": "Posterior sampling",
      "description": "Runs pm.sample with the provided draws/tune (and optional random_seed), relying on the gradient-based default step (NUTS) and returning InferenceData containing latent and noise posteriors.",
      "max_score": 15
    },
    {
      "name": "Posterior predictive",
      "description": "Uses pm.sample_posterior_predictive (or pm.draw with pm.sample results) on the fitted model/idata to obtain predictive means/HDIs for new_x, preserving input order and expected shapes.",
      "max_score": 10
    },
    {
      "name": "Prior predictive",
      "description": "Generates prior predictive draws for supplied grids via pm.sample_prior_predictive or hsgp.prior with pm.draw, producing a size x n_points array of finite values.",
      "max_score": 10
    }
  ]
}
tile.json