pyvene-interventions

Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework. Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.

Quality

74%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/pyvene/SKILL.md

Quality

Content

61%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-organized, code-rich skill body with clear workflows and a clean reference table pointing to real bundle files. The main gaps are missing validation checkpoints in batch workflows and some token cost from repeated boilerplate.

Suggestions

Add validation/verification checkpoints to the batch and training workflows (e.g. assert patched logit shape, sanity-check baseline logit_diff before sweeping, validate trainable parameters are non-empty) to lift workflow_clarity above the batch cap of 3.

De-duplicate the repeated import/model-loading boilerplate across workflows by defining a shared snippet once, or move per-workflow setup into references/tutorials.md to improve conciseness.

Tighten the introductory paragraph ('pyvene is Stanford NLP's library...') and the explanatory prose around each workflow to assume Claude's familiarity with the domain.

Dimension	Reasoning	Score
Conciseness	Mostly code-forward and efficient, but repeated import/boilerplate blocks across four workflows and an introductory paragraph explaining what pyvene is add tokens that could be tightened.	3 / 5
Actionability	Provides concrete, mostly copy-paste-ready code across all four workflows with specific configs and calls; the IIT training loop is explicitly 'simplified' with undefined dataloader/criterion, a minor gap.	4 / 5
Workflow Clarity	Workflows are numbered and sequenced with a causal-tracing checklist, but batch sweeps (layer/position loops) and training loops lack validation/verification checkpoints, capping clarity at 3 per the batch-operation rule.	3 / 5
Progressive Disclosure	Good structure with a clear references table linking to existing files (README.md, api.md, tutorials.md) one level deep, though four full inline code workflows and several inline reference tables keep it just short of ideal separation.	4 / 5
	Total	14 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, well-structured description that clearly states both capability and explicit 'Use when' triggers tied to a specific named library and niche. Minor gains are available from adding common synonyms/abbreviations.

Suggestions

Add common synonyms/abbreviations users may say (e.g. 'IIT', 'DAS', 'distributed alignment search', 'causal interventions') to broaden trigger-term coverage from 4 to 5.

Optionally enumerate one or two more concrete actions (e.g. 'saving and sharing interventions via HuggingFace') to push specificity toward comprehensive coverage.

Dimension	Reasoning	Score
Specificity	Names the domain ('causal interventions on PyTorch models using pyvene's declarative intervention framework') and lists several specific techniques ('causal tracing, activation patching, interchange intervention training, testing causal hypotheses'), but stops short of the comprehensive action enumeration of a 5.	4 / 5
Completeness	Explicitly answers both what ('Provides guidance for performing causal interventions... using pyvene's declarative intervention framework') and when ('Use when conducting causal tracing, activation patching...') with concrete trigger phrases.	5 / 5
Trigger Term Quality	Covers natural terms a mechanistic-interpretability user would say ('causal tracing', 'activation patching', 'interchange intervention training', 'causal hypotheses'), but omits common synonyms like 'DAS', 'IIT', or 'causal interventions'.	4 / 5
Distinctiveness Conflict Risk	Names a specific library (pyvene) and a narrow niche (causal interventions on PyTorch models) with distinct triggers, yielding minimal conflict risk with other skills.	5 / 5
	Total	18 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/pyvene/SKILL.md
Commit: 543b17f

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.