saelens

Train sparse autoencoders to interpret model features.

Quality

64%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./optional-skills/mlops/saelens/SKILL.md

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, highly actionable skill body: complete executable examples for loading, training, analyzing, and steering SAEs, with version-specific migration notes and troubleshooting pairs. The main costs are token weight — background on superposition/polysemanticity and content duplicated into references/ — and the lack of explicit validation feedback loops inside the training workflow.

Dimension	Reasoning	Score
Conciseness	Most of the body is dense, earned content (executable code, hyperparameter and metrics tables), but it re-explains concepts Claude already knows ("Individual neurons... are polysemantic", the MSE+L1 loss explainer, GitHub star counts, Anthropic research background) and duplicates installation/quick-start/steering code already present in references/, placing it at 'mostly efficient but includes some unnecessary explanation'.	3 / 5
Actionability	Every workflow ships complete, copy-paste-ready code with imports, real arguments (release/sae_id, nested v6 config values), and expected outputs, and the Common Issues section gives WRONG/RIGHT config pairs for the most likely failures — fully executable coverage of the common cases.	5 / 5
Workflow Clarity	The three workflows are clearly sequenced with numbered step comments and per-workflow checklists, and both evaluation-metric targets (L0, CE loss, dead features) and a reconstruction-error check are present; it falls short of 5 because fix-and-retry guidance lives in a separate 'Common Issues' section rather than being an explicit validate→fix→retry loop inside the workflows.	4 / 5
Progressive Disclosure	The bundle structure checks out — references/README.md, references/api.md, and references/tutorials.md all exist, are one level deep, and are clearly signaled via a link table — and SKILL.md is well-sectioned; however substantial reference material (Key Classes Reference, SAE Architectures, hyperparameter tables, steering/attribution code) is inlined and duplicated in the reference files rather than split out, so it is not a 5.	4 / 5
	Total	16 / 20 Passed

Description

53%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific about the niche (sparse autoencoders for interpreting model features) but terse to a fault: it covers only two of the skill's actual capabilities and provides no 'Use when' trigger guidance, so it relies entirely on a user already knowing the exact term 'sparse autoencoders'. Adding trigger phrases and the common synonym 'SAE' would substantially improve routing.

Suggestions

Add an explicit trigger clause, e.g. "Use when training or analyzing sparse autoencoders (SAEs), interpreting model features, or performing feature-based steering on TransformerLens models."

Enumerate more of the skill's concrete capabilities — loading pre-trained SAEs, feature analysis/attribution, steering — instead of only "train" and "interpret".

Include the natural synonym "SAE" and common phrasings like "mechanistic interpretability" so users who don't say "sparse autoencoders" still match.

Dimension	Reasoning	Score
Specificity	The description names the domain and two concrete actions ("Train sparse autoencoders to interpret model features") but omits other capabilities the skill covers (loading pre-trained SAEs, feature analysis, steering), so it matches the '1-2 concrete actions, not comprehensive' anchor rather than the 'several specific actions' anchor.	3 / 5
Completeness	It clearly answers 'what' ("Train sparse autoencoders to interpret model features") but contains no 'Use when...' clause or equivalent explicit trigger guidance, which caps completeness at 3 per the rubric; it is not a 4 because 'when' is entirely absent rather than merely imprecise.	3 / 5
Trigger Term Quality	"sparse autoencoders", "interpret", and "model features" are relevant keywords, but the description omits the most common natural variations users would say — "SAE", "mechanistic interpretability", "feature steering" — so it sits at 'some relevant keywords but missing common variations', below the 'good coverage' anchor.	3 / 5
Distinctiveness Conflict Risk	"Train sparse autoencoders" carves a clear niche with minimal conflict risk beyond closely related interpretability libraries (e.g., TransformerLens), matching 'mostly distinct; minor overlap risk'; it is not a 5 because the description does not include distinct trigger phrases that would fully separate it from adjacent skills.	4 / 5
	Total	13 / 20 Passed

Validation

81%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 13 / 16 Passed

Validation for skill structure

Criteria	Description	Result
metadata_version	'metadata.version' is missing	Warning
metadata_field	'metadata' should map string keys to string values	Warning
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	13 / 16 Passed

Repository: NousResearch/hermes-agent
Path: optional-skills/mlops/saelens/SKILL.md
Commit: 68fa7e9

Reviewed: 3 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.