nnsight-remote-interpretability

Provides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote execution. Use when needing to run interpretability experiments on massive models (70B+) without local GPU resources, or when working with any PyTorch architecture.

Quality

78%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Fix and improve this skill with Tessl

tessl review fix ./04-mechanistic-interpretability/nnsight/SKILL.md

Quality

Content

72%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The body is highly actionable with copy-paste-ready code and good progressive disclosure via clearly linked reference files. It is weakened by repetition-driven verbosity and by the absence of validation/feedback loops in its batch patching and remote workflows.

Suggestions

Consolidate duplicated content — tracing basics, installation, and NDIF setup each appear multiple times across Core Concepts, workflows, and references/README — into a single source to cut token cost.

Add explicit validation checkpoints to Workflow 2's systematic patching sweep (e.g., verify shapes/ranges before assignment, assert the metric is finite) and a validate-fix-retry loop for remote NDIF runs that can time out or fail mid-graph.

Complete the patching sweep example by defining the undefined helpers (seq_len, corrupted, clean_cache, compute_metric) or clearly marking them as placeholders, so the runnable example is self-contained.

Dimension	Reasoning	Score
Conciseness	The ~430-line body is mostly efficient and assumes Claude's competence, but it repeats itself (tracing basics in Core Concepts, Workflow 1, and references/README; installation and NDIF setup duplicated) and could be tightened.	2 / 3
Actionability	Numerous full, copy-paste-ready code blocks (LanguageModel loading, trace/.save(), activation patching via [:]=, gradient retain_grad) plus specific install commands and an API reference table make the guidance concrete and executable.	3 / 3
Workflow Clarity	Workflows are numbered step-by-step with a checklist, but the batch patching sweep and remote execution (fragile/batch operations) lack explicit validation checkpoints or validate-fix-retry feedback loops, which caps this dimension at 2 per the rubric scoring notes.	2 / 3
Progressive Disclosure	A well-signaled 'Reference Documentation' table links the real bundle files (README.md, api.md, tutorials.md) one level deep with descriptions, with quick-start/workflows inline and API detail offloaded to references, giving easy navigation.	3 / 3
	Total	10 / 12 Passed

Description

85%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is specific, complete, and distinctive, explicitly covering both what the skill does and when to use it with an explicit trigger clause. Its only weakness is somewhat technical phrasing that omits natural user-facing terms a person would actually say.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions ('interpreting and manipulating neural network internals', 'run interpretability experiments on massive models (70B+)') plus the distinctive NDIF remote execution capability, matching the score-3 anchor for plural concrete actions.	3 / 3
Completeness	Clearly answers both what ('Provides guidance for interpreting and manipulating...') and when via an explicit 'Use when...' clause covering both massive-model and general PyTorch cases.	3 / 3
Trigger Term Quality	Contains relevant domain terms ('interpretability experiments', 'PyTorch architecture', 'neural network internals') but leans technical and misses common natural phrasings a user would actually say, such as 'interpret/probe a model' or 'mechanistic interpretability'.	2 / 3
Distinctiveness Conflict Risk	Anchored to the niche-specific tools nnsight and NDIF with the 70B+ remote-execution framing, making it clearly distinguishable and unlikely to trigger for the wrong skill despite a broad 'any PyTorch architecture' clause.	3 / 3
	Total	11 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: Orchestra-Research/AI-Research-SKILLs
Path: 04-mechanistic-interpretability/nnsight/SKILL.md
Commit: 773a529

Reviewed: about 12 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.