CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

running-placebo-analysis

Performs placebo-in-time sensitivity analysis with hierarchical null model and optional Bayesian assurance. Use when checking model robustness, verifying lack of pre-intervention effects, or estimating study power.

Quality

82%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

71%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A concise, well-structured overview with a clear workflow and no token waste, but it falls short on executability (no code example despite being a code-API skill) and its only progressive-disclosure pointer is a dead link. Fixing the reference path and adding a minimal runnable PlaceboInTime example would lift it substantially.

Suggestions

Add a minimal executable code example (import, construct PlaceboInTime with concrete n_folds and assurance arguments, call .run(experiment)) so the workflow is copy-paste ready.

Fix the reference link: the body points to 'reference/placebo_in_time.md' but no such file or references/ directory exists in the bundle — create the file at the referenced path or correct the path.

Include interpretation guidance for the evaluation step, e.g., what range of p_effect_outside_null or assurance values indicates the model is robust, to serve as a validation checkpoint.

DimensionReasoningScore

Conciseness

The body is lean with no padding: it assumes familiarity with CausalPy, PyMC, ITS/SC, and pipelines, and every line either states a workflow step or a domain-specific concept the model would not already know. No verbosity to trim.

5 / 5

Actionability

Concrete API entities are named — 'PlaceboInTime with n_folds', '.run(experiment)', 'Pipeline + SensitivityAnalysis', 'theta_new', 'p_effect_outside_null' — but there is no executable code, no import path, and no example constructor or parameter values. This sits between minimal high-level hints (2) and mostly executable guidance (4): real, specific API names but missing key execution details.

3 / 5

Workflow Clarity

The four-step sequence (Fit → Configure → Run → Evaluate) is clear and ordered, with step 4 naming what to inspect. Not 5 because there are no validation checkpoints or interpretation criteria (e.g., what value of p_effect_outside_null indicates robustness), and no error-recovery guidance.

4 / 5

Progressive Disclosure

The single reference is clearly signaled with a description of its contents ('Core API reference, usage examples, and hierarchical status-quo modeling'), but the referenced file reference/placebo_in_time.md does not exist in the bundle — no references/, scripts/, or assets/ directory is present — so the pointer is broken. Structure and signaling are good, but navigation fails against the actual bundle.

3 / 5

Total

15

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: concrete actions, explicit use-when triggers in the user's natural vocabulary, and a clearly bounded niche. Only minor room to improve via common synonyms (placebo test, power analysis).

DimensionReasoningScore

Specificity

Names several concrete actions — 'placebo-in-time sensitivity analysis', 'hierarchical null model', 'optional Bayesian assurance' — each tied to a concrete capability, though coverage of the full analysis output is moderately narrow. Not 5 because it lists three actions without the breadth of the comprehensive example; not 3 because more than 1-2 specific actions are named.

4 / 5

Completeness

Explicitly answers both: 'Performs placebo-in-time sensitivity analysis with hierarchical null model and optional Bayesian assurance' (what) and 'Use when checking model robustness, verifying lack of pre-intervention effects, or estimating study power' (when, with three concrete trigger phrases).

5 / 5

Trigger Term Quality

'model robustness', 'verifying lack of pre-intervention effects', and 'estimating study power' are phrases a user would naturally say. Not 5 because common synonyms like 'placebo test', 'falsification test', or 'power analysis' are missing; not 3 because keyword coverage goes well beyond a single generic term.

4 / 5

Distinctiveness Conflict Risk

A clear niche — placebo-in-time sensitivity analysis with Bayesian assurance — with distinct trigger phrases that no general-purpose skill would claim. Conflict risk is minimal.

5 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

relative_links

Relative link issues: 1 missing

Warning

Total

15

/

16

Passed

Repository

pymc-labs/CausalPy

Path

causalpy/skills/running-placebo-analysis/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.