CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

pyvene-interventions

Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework. Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.

63

Quality

79%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/pyvene/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

67%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-organized, highly actionable skill body with four sequenced, code-driven workflows and a properly signaled references bundle. The main weakness is conciseness — vanity metrics, link catalogs, and reference-table material that belongs in the bundle files inflate the token budget without adding guidance Claude can act on.

Suggestions

Trim the 'External Resources', 'Supported Models', and 'Comparison with Other Tools' sections (or move them into references/README.md) and drop the '840+ stars' count and 'Standard import' snippet to reduce token overhead.

Move the 'Key Classes Reference' and 'Component Targets' enumerations into references/api.md, keeping only one inline example in SKILL.md.

Complete Workflow 3's training loop (define the dataloader and criterion, or show a minimal concrete example) and replace the trailing '# 5. Visualize' comment with a short plotting snippet so every workflow is end-to-end executable.

DimensionReasoningScore

Conciseness

Mostly code and tables that earn their place, but there is noticeable padding: '840+ stars' (time-sensitive vanity metric), the 'Supported Models' list, the 'Comparison with Other Tools' table, the 'Standard import' snippet, a long 'External Resources' links section, and a version-bound note ('pyvene v0.1.8+ supports LoRAs') that isn't quarantined in a deprecation/versions section. This sits between the 'noticeably verbose' (2) and 'efficient' (4) anchors.

3 / 5

Actionability

Four complete workflows with concrete, mostly copy-paste-ready code (IntervenableConfig construction, patching sweeps, save/load, steering) plus a targeted 'Common Issues & Solutions' section with WRONG/RIGHT examples. Not a 5 because Workflow 3's training loop relies on undefined 'dataloader' and 'criterion' placeholders and Workflow 1 ends in a bare '# 5. Visualize' comment rather than executable code.

4 / 5

Workflow Clarity

Each workflow is explicitly sequenced with numbered inline steps ('# 1. Define clean and corrupted inputs' ... '# 5. Visualize'), and Workflow 1 includes a checklist; the 'Common Issues' section serves as error-recovery guidance. Falls short of 5 because validation checkpoints are largely implicit — e.g., no step verifying an intervention actually fired or that logits changed as expected before sweeping all layers.

4 / 5

Progressive Disclosure

The body is an overview with four workflow walkthroughs, and detailed API/tutorial material is correctly split into references/ (README.md, api.md, tutorials.md — all real files, ~830 lines) linked via a clearly labeled table with per-file descriptions, one level deep. Not a 5 because reference-style content (the 'Key Classes Reference' table, 'Component Targets' enumeration, tool comparison) is inlined in SKILL.md where it belongs in references/api.md.

4 / 5

Total

15

/

20

Passed

Description

83%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description in third-person voice with an explicit 'Use when...' clause enumerating concrete trigger activities. It clearly communicates what the skill does and when to reach for it, with only minor gaps in capability coverage and synonym variations.

DimensionReasoningScore

Specificity

The description names the domain ('causal interventions on PyTorch models using pyvene's declarative intervention framework') and lists several concrete actions — 'causal tracing, activation patching, interchange intervention training, or testing causal hypotheses'. It falls just short of the 5 anchor because the opening 'Provides guidance for performing' is slightly padded and a few capabilities (e.g., steering, saving/sharing interventions) are not mentioned.

4 / 5

Completeness

Both questions are explicitly answered: 'what' via 'Provides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework' and 'when' via an explicit 'Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses' clause with concrete trigger phrases. Not scoreable below 5 — this matches the top anchor including the explicit 'Use when...' construction.

5 / 5

Trigger Term Quality

Natural terms a user would say are present: 'causal tracing', 'activation patching', 'interchange intervention training', 'causal hypotheses', 'PyTorch', 'pyvene'. Not a 5 because common variations users might say — 'circuit analysis', 'steering', 'interpretability', 'intervention' alone — are missing.

4 / 5

Distinctiveness Conflict Risk

pyvene plus its specific technique names give it a clear niche with minimal conflict risk against unrelated skills. Not a 5 because related interpretability skills (TransformerLens/SAELens-style tasks) share the 'activation patching' / 'causal' vocabulary, so a small overlap risk remains.

4 / 5

Total

17

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

metadata_version

'metadata.version' is missing

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

14

/

16

Passed

Repository

synthetic-sciences/openscience

Path

backend/cli/skills/ml-training/pyvene/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.