pyvene-interventions

Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework. Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.

Quality

72%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/pyvene/SKILL.md

Quality

Content

61%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A thorough, well-structured skill with executable workflow examples and properly signaled one-level-deep references. Its main weaknesses are mild verbosity from duplicated/supplementary sections and the absence of explicit validation checkpoints in the batch sweep workflows.

Suggestions

Add an explicit validation/verification checkpoint to the sweep workflows (e.g., assert the patched probability or logit diff is finite before accumulating results, and re-check on anomalous values) to lift workflow clarity above the batch-operation cap.

Remove or merge the "Key Classes Reference" table into "Core Concepts" to eliminate duplication, and consider moving the "Comparison with Other Tools" and "Supported Models" tables into references/ to tighten the body.

Replace Workflow 3's "simplified" training loop with a minimal but complete runnable example (define a tiny dataloader and criterion, or note the exact expected shapes) so all workflows are equally actionable.

Dimension	Reasoning	Score
Conciseness	Mostly efficient with lean code examples, but the intro paragraph explains what pyvene is and the "Key Classes Reference" table duplicates "Core Concepts", while the closing comparison table and supported-models list add supplementary tokens that could be trimmed.	3 / 5
Actionability	Workflows 1, 2, and 4 give copy-paste-ready executable code with concrete config objects, but Workflow 3's training loop is explicitly "simplified" with undefined `dataloader` and `criterion`, leaving a minor gap.	4 / 5
Workflow Clarity	Workflows have numbered steps and Workflow 1 includes a checklist, but the layer/position sweeps are batch operations with no explicit validation or feedback checkpoint, so workflow clarity is capped at 3 per the batch-operation rule.	3 / 5
Progressive Disclosure	Clear sectioned overview with well-signaled one-level-deep references (references/README.md, api.md, tutorials.md, all verified present), though some supplementary reference-like content (comparison table, supported models, external resources) is inlined rather than split out.	4 / 5
	Total	14 / 20 Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-targeted description that clearly states the library's purpose and gives explicit usage triggers with concrete intervention techniques. It is held back only by a slightly generic lead verb and minor overlap with neighboring interpretability tools.

Dimension	Reasoning	Score
Specificity	Lists several concrete intervention types ("causal tracing, activation patching, interchange intervention training, or testing causal hypotheses"), though the lead verb "Provides guidance for performing" is somewhat generic, leaving minor gaps versus the comprehensive anchor.	4 / 5
Completeness	Explicitly answers both what ("Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework") and when ("Use when conducting causal tracing, activation patching...") with concrete trigger phrases.	5 / 5
Trigger Term Quality	Strong natural terms ("causal tracing", "activation patching", "interchange intervention training", "causal hypotheses", "pyvene") that users would say, but a few common variants (e.g. "interpretability", "circuit analysis", "DAS") are absent.	4 / 5
Distinctiveness Conflict Risk	Names a clear niche (pyvene's declarative intervention framework) with distinct triggers, but the broader interpretability/probing space overlaps with related tools like TransformerLens and SAELens, giving minor conflict risk.	4 / 5
	Total	17 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/pyvene/SKILL.md
Commit: fcd2e11

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.