nnsight-remote-interpretability

Provides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote execution. Use when needing to run interpretability experiments on massive models (70B+) without local GPU resources, or when working with any PyTorch architecture.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Quality

Content

72%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-structured, highly actionable skill with excellent progressive disclosure and executable examples throughout. The main gaps are repetitive boilerplate that hurts conciseness and missing validation checkpoints in the patching and remote-execution workflows.

Suggestions

Tighten conciseness by extracting the repeated `LanguageModel('gpt2', device_map='auto')` load into a single shared snippet rather than restating it at the top of every workflow.

Add explicit validation checkpoints to the patching and remote workflows (e.g., assert saved tensor shapes before patching, and confirm NDIF results returned before analysis) so failures are caught early.

Consider moving the comparison-with-other-tools table and the external-resources link list into references/ to keep SKILL.md focused on the core workflows.

Dimension	Reasoning	Score
Conciseness	The body is efficient and free of Claude-knows-this padding, but ~430 lines repeat the same model-loading boilerplate across five workflows and include a comparison table plus external-link sections that could be tightened, matching the 'mostly efficient but could be tightened' anchor rather than fully lean.	2 / 3
Actionability	Code throughout is fully executable and copy-paste ready — specific model IDs, exact module paths (e.g. 'model.transformer.h[5].output[0]'), concrete install commands, and an API reference table — matching the 'fully executable code/commands; specific examples' anchor.	3 / 3
Workflow Clarity	Each of the five workflows has a 'Step-by-Step' sequence and Workflow 1 includes a checklist, but validation/verification checkpoints are absent for fragile operations like activation patching and remote execution; per the rubric, missing feedback loops for such operations caps this at 2.	2 / 3
Progressive Disclosure	SKILL.md is a clear overview with well-signaled, one-level-deep references to a real references/ folder (README.md, api.md, tutorials.md, all confirmed present) listed in a Reference Documentation table, with detailed API and tutorials appropriately split out — matching the 'clear overview with well-signaled one-level-deep references' anchor.	3 / 3
	Total	10 / 12 Passed

Description

100%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description with explicit capability statements and concrete 'Use when' triggers covering both the local and remote-execution use cases. Minor risk comes from the broad 'any PyTorch architecture' clause, but the niche remains clearly distinguishable.

Dimension	Reasoning	Score
Specificity	Lists multiple concrete actions — 'interpreting and manipulating neural network internals', 'optional NDIF remote execution', and 'run interpretability experiments on massive models (70B+)' — matching the 'lists multiple specific concrete actions' anchor.	3 / 3
Completeness	Clearly answers both 'what' ('Provides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote execution') and 'when' (the explicit 'Use when needing to...' clause), satisfying the anchor for explicit triggers on both.	3 / 3
Trigger Term Quality	Explicit 'Use when...' clauses give natural trigger phrases ('interpretability experiments on massive models (70B+)', 'without local GPU resources', 'any PyTorch architecture') a researcher would actually say, though 'NDIF remote execution' is somewhat jargon-heavy.	3 / 3
Distinctiveness Conflict Risk	The nnsight/NDIF remote-interpretability-on-massive-models niche is distinct and unlikely to trigger the wrong skill; the broad 'any PyTorch architecture' clause slightly raises overlap risk but the core triggers remain clearly differentiated.	3 / 3
	Total	12 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/nnsight/SKILL.md
Commit: 52845c3

Reviewed: about 13 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.