CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

transformer-lens-interpretability

Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Low

Low-risk findings worth noting

SKILL.md
Quality
Evals
Security

Quality

Content

67%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill: three concrete workflows with executable code, checklists, and a clean references/ bundle that is properly linked one level deep. The main costs are token efficiency — background fluff, time-sensitive version notes, and API tables that duplicate references/api.md.

Suggestions

Trim the intro (maintainer names, star count) and move version notes into a clearly labeled compatibility/deprecated section or drop them — time-sensitive numbers penalize conciseness.

Move the ActivationCache Keys table, Key Classes Reference, and Supported Models table into references/api.md, keeping only 2-3 representative keys/classes inline with a pointer to the full reference.

Replace the placeholder comments ('# 5. Visualize results', '# 5. Identify top contributing heads') with a few lines of actual plotting/selection code, and add a brief validation note (e.g., sanity-check the baseline logit diff before iterating) to strengthen feedback loops.

DimensionReasoningScore

Conciseness

The bulk is efficient code and tables, but there is real padding: 'de facto standard library... Created by Neel Nanda and maintained by Bryce Meyer... (2,900+ stars)' is background Claude doesn't need, the star count and 'Version Notes' section ('v2.0', 'v3.0 (alpha)') are time-sensitive details not in a deprecated section, and the ActivationCache Keys and Key Classes tables duplicate content that belongs in references/api.md. Not a 2 because the core workflows are tight; not a 4 because several sections could be trimmed or moved.

3 / 5

Actionability

Three complete, mostly copy-paste-ready workflows (activation patching loop, IOI head-contribution decomposition, induction-head detection) plus WRONG/RIGHT snippets for common issues and install commands. Minor gaps keep it below 5: two steps are placeholder comments rather than code ('# 5. Visualize results (layer x position heatmap)', '# 5. Identify top contributing heads...').

4 / 5

Workflow Clarity

Each workflow is a clearly numbered sequence with follow-along code, and Workflows 1 and 2 end with explicit checklists that act as checkpoints ('[ ] Define clean and corrupted inputs that differ minimally'... '[ ] Validate with ablation experiments'). Not a 5 because there are no explicit validation/feedback loops (e.g., what to do when patching results are noisy) and Workflow 3 lacks a checklist; well above a 3 since checkpoints are present, just not recovery loops.

4 / 5

Progressive Disclosure

Good structure: the body is an overview and the 'Reference Documentation' table clearly signals one-level-deep links ([references/README.md], [references/api.md], [references/tutorials.md]) — all verified to exist — with contents described per file. Not a 5 because the ActivationCache Keys table, Key Classes Reference, and Supported Models table inline material that duplicates references/api.md rather than splitting it out; not a 3 because references are well-signaled and the split is otherwise sensible.

4 / 5

Total

15

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person voice, concrete mechanism names, and an explicit 'Use when...' clause with natural trigger phrases. The only weakness is slightly incomplete action coverage (circuit analysis, direct logit attribution) and a few missing synonym triggers.

DimensionReasoningScore

Specificity

Lists several concrete actions — 'inspect and manipulate transformer internals via HookPoints and activation caching', 'reverse-engineering model algorithms', 'studying attention patterns', 'performing activation patching' — with only minor gaps (circuit analysis and direct logit attribution are not named). Not a 5 because coverage stops short of the full capability set shown in the body; not a 3 because far more than 1-2 actions are specified.

4 / 5

Completeness

Explicitly answers both: what — 'Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching' — and when — 'Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments'. Both are concrete and stated with explicit trigger phrases, matching the top anchor exactly.

5 / 5

Trigger Term Quality

Natural terms a researcher would say are present: 'mechanistic interpretability', 'TransformerLens', 'activation patching', 'attention patterns', 'HookPoints', 'activation caching'. A few common variations are missing ('circuit analysis', 'logit lens', 'induction heads'), which keeps it below the comprehensive-synonym anchor of 5 but comfortably above the partial-coverage anchor of 3.

4 / 5

Distinctiveness Conflict Risk

The TransformerLens-specific niche ('HookPoints', 'activation caching', 'activation patching') has distinct triggers and minimal overlap risk; adjacent tools like SAELens or nnsight would have clearly different trigger vocabularies. Not a 4 because no closely related skill is plausibly triggered by these phrases.

5 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

OpenLAIR/dr-claw

Path

skills/mechanistic-interpretability/transformer-lens/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.