nnsight-remote-interpretability

Provides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote execution. Use when needing to run interpretability experiments on massive models (70B+) without local GPU resources, or when working with any PyTorch architecture.

Quality

65%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

High

Do not use without reviewing

Fix and improve this skill with Tessl

tessl review fix ./backend/cli/skills/ml-training/nnsight/SKILL.md

Quality

Content

65%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The skill body is a well-organized, actionable overview with strong progressive disclosure pointing to real reference files. Its weaknesses are mild verbosity/redundancy across workflows and missing validation checkpoints for batch operations like the patching sweep.

Suggestions

Add explicit validation checkpoints to batch workflows (e.g., assert patched shapes match, sanity-check metric ranges before/after the sweep) and a feedback loop so workflow_clarity can exceed the batch-operation cap of 3.

Remove repeated import/model-load boilerplate from each workflow block and reference a shared setup snippet, to tighten conciseness toward level 4-5.

Resolve undefined identifiers in the sweep example (define seq_len and compute_metric, or mark them as placeholders) so the actionability reaches fully copy-paste-ready level 5.

Dimension	Reasoning	Score
Conciseness	The body is mostly efficient with concrete code per workflow, but repeats boilerplate (import + model load in every block) and overlaps Common Issues with earlier examples, so it could be tightened; not level 4 due to several padded/redundant sections and not level 2 because it is not severely verbose.	3 / 5
Actionability	Provides mostly executable, copy-paste-ready code across activation analysis, patching, remote execution, cross-prompt sharing, and gradients, with only minor gaps (undefined seq_len and compute_metric in the sweep), matching the level-4 anchor rather than the fully-complete level-5.	4 / 5
Workflow Clarity	Workflows are numbered and sequenced with a checklist, but batch operations (the systematic patching sweep, remote runs) lack explicit validation checkpoints or feedback loops, and the rubric caps batch-operation workflows without validation at 3; not level 4 because the validation gaps are material.	3 / 5
Progressive Disclosure	SKILL.md serves as a clear overview with a well-signaled Reference Documentation table linking to real one-level-deep files (references/README.md, api.md, tutorials.md), splitting bulk detail out of the main file and matching the level-5 anchor.	5 / 5
	Total	15 / 20 Passed

Description

66%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly communicates a distinct niche (remote interpretability of massive models via nnsight/NDIF) and provides explicit "Use when..." trigger guidance. Its main weakness is moderate specificity and trigger-term breadth, which keep it from the top anchors.

Suggestions

Enumerate more concrete actions in the description (e.g., access module outputs, patch activations, compute gradients, share activations across prompts) to lift specificity toward level 4-5.

Add natural trigger synonyms users actually say ("mechanistic interpretability", "hook into model activations", "activation patching") to broaden trigger-term coverage.

Tighten the "what" clause from "Provides guidance for" to a more direct verb phrase ("Interpret and manipulate neural network internals...") for a clearer level-5 what-and-when pair.

Dimension	Reasoning	Score
Specificity	Names the nnsight/NDIF domain and 1-2 concrete actions ("interpreting and manipulating neural network internals", "remote execution") but does not enumerate a comprehensive set of specific actions, matching the level-3 anchor; not level 4 because it lacks several distinct specific actions.	3 / 5
Completeness	Explicitly answers both what ("Provides guidance for interpreting and manipulating...") and when ("Use when needing to run interpretability experiments on massive models..."), but the what is slightly generic and the when could be more specific, placing it just below the exemplary level-5 anchor.	4 / 5
Trigger Term Quality	Includes relevant natural phrases ("interpretability experiments", "massive models (70B+)", "any PyTorch architecture") but misses common variations/synonyms users say ("mech interp", "activations", "hooks"), fitting the level-3 anchor rather than the broader level-4 coverage.	3 / 5
Distinctiveness Conflict Risk	Targets a clear niche (nnsight + NDIF remote interpretability on 70B+ models) with distinct triggers and minimal overlap risk with other skills, matching the level-5 anchor.	5 / 5
	Total	15 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: synthetic-sciences/openscience
Path: backend/cli/skills/ml-training/nnsight/SKILL.md
Commit: 7dabbf0

Reviewed: 2 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.