Probabilistic Programming in Python: Bayesian Modeling and Probabilistic Machine Learning with Theano
Agent Success
Agent success rate when using this tile
68%
Improvement
Agent success rate improvement when using this tile compared to baseline
0.94x
Baseline
Agent success rate without this tile
72%
{
"context": "Evaluates how well the solution uses PyMC's labeled dimension tooling for store/week data, including shared data containers that can be updated and minibatching without losing coordinate information. Scoring checks for correct use of coords/dims, mutable data updates, and minibatch scaling so inference matches full-data intent.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Coords Model",
"description": "Defines the model with pm.Model(coords=...) and applies dims referencing store/week coordinates on all stochastic and observed variables so posterior groups keep those labels.",
"max_score": 25
},
{
"name": "Shared Data",
"description": "Binds sales, price, and promo arrays through pm.Data/pm.MutableData with dims tied to coords instead of bare tensors.",
"max_score": 20
},
{
"name": "Data Updates",
"description": "Updates observed inputs via pm.set_data or Data.set_value without rebuilding the model while preserving coordinate alignment.",
"max_score": 15
},
{
"name": "Minibatch Inputs",
"description": "Uses pm.Minibatch for observed/features with total_size set to full dataset and dims consistent with labeled coords.",
"max_score": 20
},
{
"name": "Loglik Scaling",
"description": "Ensures minibatch log-likelihood is scaled to represent the full dataset (e.g., via pm.Minibatch total_size or manual weighting).",
"max_score": 10
},
{
"name": "Posterior Labels",
"description": "Posterior and posterior_predictive outputs retain store/week coordinate names, showing dims propagated through sampling and prediction.",
"max_score": 10
}
]
}docs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10