CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

sparse-autoencoder-training

Provides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

81%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A highly actionable, well-sequenced skill body with complete executable code, validation checklists, metrics targets, and troubleshooting feedback loops, plus a properly structured references bundle. The main weakness is token efficiency: several sections re-explain superposition and SAE mechanics that Claude already knows, and the long body could push more material into the existing reference files.

Suggestions

Trim or relocate the conceptual background sections ("The Problem: Polysemanticity & Superposition", "What SAEs Learn", "Key Validation") — Claude already knows these concepts; keep only the SAELens-specific facts such as the loss formula and metrics targets.

Move the External Resources lists and the SAE Architectures comparison into references/ (e.g., tutorials.md or README.md), keeping the body to quick-start workflows plus the reference table.

Drop trivia like the GitHub star count and the commented-out Neuronpedia example block, replacing it with a one-line URL pattern.

DimensionReasoningScore

Conciseness

The bulk is efficient (executable code, hyperparameter and metrics tables, tight troubleshooting with WRONG/RIGHT pairs), but several sections explain concepts Claude already knows: "Individual neurons... are polysemantic", "SAEs are trained to reconstruct model activations through a sparse bottleneck", the encoder/decoder diagram, and trivia like "(1,100+ stars)". This matches anchor 3 (mostly efficient with some unnecessary explanation) rather than anchor 4, since the conceptual background spans multiple sections, though it falls short of anchor 2's pervasive padding.

3 / 5

Actionability

Three fully executable, copy-paste-ready workflows (loading/encoding, full training config with LanguageModelSAERunnerConfig, steering with hooks, logit attribution) plus concrete troubleshooting configs and a Neuronpedia lookup pattern cover the common cases. Matches anchor 5's fully executable, example-covered standard; anchor 4 would imply missing key details, and none of the core paths are pseudocode.

5 / 5

Workflow Clarity

Each workflow is a numbered step-by-step sequence with a closing checklist ("Validate L0 and CE loss recovery", "Check dead feature ratio"), explicit metrics target tables for validation, and a Common Issues section providing fix-retry feedback loops (e.g., dead features → enable warm-up and ghost grads). This matches anchor 5's explicit validation, feedback loops, and checklists; validation is present for the batch training operation, so no cap applies.

5 / 5

Progressive Disclosure

Good structure with a clearly signaled reference table pointing to real one-level-deep bundle files (references/README.md, api.md, tutorials.md — all verified to exist), and workflows/checklists appropriately inlined. Below anchor 5 because the ~375-line body still carries content that could live in the references (conceptual background, external resource lists, architecture comparison), and README.md overlaps the SKILL.md overview role.

4 / 5

Total

17

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: third-person voice, clear capability statement, and an explicit 'Use when' clause with concrete niche-appropriate trigger phrases. The only improvement space is broader synonym coverage (e.g., 'mechanistic interpretability', 'SAE training') and mentioning analysis capabilities like steering.

DimensionReasoningScore

Specificity

Names the domain and three concrete actions — "training and analyzing Sparse Autoencoders (SAEs)" and "decompose neural network activations into interpretable features" — comparable to the anchor-4 example of several specific actions with minor gaps (e.g., feature analysis/steering are not mentioned). It exceeds anchor 3 (only 1-2 actions, not comprehensive) but does not reach anchor 5's comprehensive multi-action coverage.

4 / 5

Completeness

Explicitly answers both questions: the "what" is training/analyzing SAEs to decompose activations into interpretable features, and an explicit "Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations" clause gives concrete trigger phrases — matching the anchor-5 example structure exactly. Anchor 4 would require the 'when' to be less specific than it is here.

5 / 5

Trigger Term Quality

Good natural keyword coverage: "Sparse Autoencoders (SAEs)", "SAELens", "interpretable features", "superposition", "monosemantic representations" are terms users in this niche would actually say. A few natural variations are missing (e.g., "SAE training", "feature discovery", "mechanistic interpretability"), which keeps it below anchor 5's comprehensive synonym coverage but above anchor 3's incomplete keyword set.

4 / 5

Distinctiveness Conflict Risk

The niche is highly specific — SAELens-based sparse-autoencoder work with triggers like "superposition" and "monosemantic representations" — giving minimal overlap risk with general interpretability or transformer-analysis skills. Anchor 4 would imply noticeable overlap with closely related skills, which these distinctive triggers avoid.

5 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

15

/

16

Passed

Repository

OpenLAIR/dr-claw

Path

skills/mechanistic-interpretability/saelens/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.