sparse-autoencoder-training

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.

Quality

78%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/saelens/SKILL.md

Quality

Content

72%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable with executable code and well-organized one-level-deep references. It is held back by some motivational verbosity and by training workflows that lack inline validation feedback loops for a batch operation.

Suggestions

Add explicit validation feedback loops to the training workflow (e.g., 'After training, check L0 and CE loss score; if outside target range, adjust l1_coefficient or d_sae and retrain') instead of deferring all troubleshooting to the Common Issues section.

Trim the motivational/contextual sections ('The Problem: Polysemanticity & Superposition', 'Key Validation (Anthropic Research)' feature list, GitHub star count) since Claude already understands superposition and polysemanticity.

Move the inline reference material (Key Classes Reference, SAE Architectures table) into references/ to keep SKILL.md a lean overview.

Dimension	Reasoning	Score
Conciseness	The body is mostly efficient with concrete SAE-specific tables and code, but includes motivational padding ('The Problem: Polysemanticity & Superposition', GitHub star counts, a feature-list 'Key Validation' section) that could be tightened.	2 / 3
Actionability	Provides fully executable code for loading, training, analysis, steering, and attribution, plus concrete hyperparameter/metric tables and WRONG/RIGHT examples that are copy-paste ready.	3 / 3
Workflow Clarity	Three workflows are sequenced with checklists, but the batch training workflow lacks an explicit validate->fix->retry feedback loop; common issues are documented separately rather than as inline checkpoints, capping the score at 2.	2 / 3
Progressive Disclosure	The overview clearly signals one-level-deep references (README.md, api.md, tutorials.md) in a navigation table, all of which are real files, with content appropriately split between the body and references.	3 / 3
	Total	10 / 12 Passed

Description

85%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is strong: it states concrete capabilities, an explicit use-trigger, and occupies a distinct niche. Its main weakness is trigger terminology skewed toward research jargon rather than phrasing a typical user would naturally say.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions ('training and analyzing Sparse Autoencoders', 'decompose neural network activations into interpretable features') tied to a specific tool (SAELens).	3 / 3
Completeness	Explicitly states both what it does (train/analyze SAEs to decompose activations) and when to use it via a clear 'Use when discovering...' clause.	3 / 3
Trigger Term Quality	Triggers like 'discovering interpretable features', 'analyzing superposition', and 'monosemantic representations' are accurate but technical jargon unlikely to be said by a general user, missing common variations.	2 / 3
Distinctiveness Conflict Risk	The mechanistic-interpretability SAE niche is narrow and the triggers are specialized, making conflict with other skills unlikely.	3 / 3
	Total	11 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/saelens/SKILL.md
Commit: 52845c3

Reviewed: about 13 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.