Tessl • The management layer for AI agent skills

SkillPublic

transformer-lens-interpretability

Provides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.

Quality

83%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill body with executable code for three core interpretability workflows and clearly signaled, verified reference files. The main improvements are completing the stubbed final steps of Workflows 1 and 2, adding validation checkpoints, and removing marketing prose and duplicated API tables.

Suggestions

Replace the stub comments ending Workflow 1 ('# 5. Visualize results') and Workflow 2 ('# 5. Identify top contributing heads') with executable code, or point explicitly to the section of references/tutorials.md that covers them.

Add explicit validation checkpoints to the workflows (e.g., verify the baseline logit difference is positive and of expected magnitude before running the full patching sweep) so failures are caught early rather than after expensive loops.

Trim non-actionable prose (creator attribution, GitHub star count, Version Notes) and remove the dangling papers.md link in references/README.md, since that file does not exist in the bundle.

Dimension	Reasoning	Score
Conciseness	The body is mostly library-specific and token-efficient (cache-key tables, hook gotchas, executable snippets), but includes trimmable padding such as creator attribution, the '(2,900+ stars)' note, and a Version Notes section, plus API tables that duplicate references/api.md.	4 / 5
Actionability	Nearly all guidance is concrete, copy-paste-ready code, but two workflows terminate in stub comments ('# 5. Visualize results (layer x position heatmap)', '# 5. Identify top contributing heads...') instead of executable code, leaving minor gaps.	4 / 5
Workflow Clarity	The three workflows are clearly sequenced with numbered steps and end-of-workflow checklists, but there are no explicit validation checkpoints or feedback loops (e.g., sanity-checking the baseline logit difference before proceeding with patching).	4 / 5
Progressive Disclosure	The Reference Documentation table clearly signals the three one-level-deep reference files, all of which exist and contain substantial content; however, references/README.md links a papers.md that does not exist, and some API detail inlined in SKILL.md duplicates references/api.md.	4 / 5
	Total	16 / 20 Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description with an explicit and specific 'Use when' clause paired with a clear statement of capability. Third-person voice is used correctly. It would benefit from a broader set of natural trigger synonyms (circuit analysis, causal tracing, logit lens) and a more action-oriented opening phrase.

Dimension	Reasoning	Score
Specificity	The description lists several concrete actions ('inspect and manipulate transformer internals via HookPoints and activation caching') but opens with the generic phrase 'Provides guidance for mechanistic interpretability research', leaving minor gaps versus a comprehensive action inventory.	4 / 5
Completeness	It explicitly answers both 'what' (inspect and manipulate transformer internals via HookPoints and activation caching) and 'when' with an explicit 'Use when' clause containing three concrete trigger phrases, matching the top anchor exactly.	5 / 5
Trigger Term Quality	Natural user phrasings like 'reverse-engineering model algorithms', 'studying attention patterns', and 'activation patching' are present, but common synonyms such as 'circuit analysis', 'causal tracing', or 'logit lens' are missing.	4 / 5
Distinctiveness Conflict Risk	TransformerLens is named specifically with its distinctive mechanisms (HookPoints, activation patching), creating a clear niche with minimal conflict risk against generic ML-training skills.	5 / 5
	Total	18 / 20 Passed

Validation

87%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	14 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/transformer-lens/SKILL.md
Revision: a08e0ad

Reviewed: 1 month ago

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.

transformer-lens-interpretability

Table of Contents

Is this your skill?